Social Brand Engagement: Measuring What Matters for 100+ Brands on Twitter
Gauging success on any social platform has it’s pitfalls. First, you need to make sure your goals are in order. Next, you need to make sure your metrics are aligned to your goals. But the often forgotten step is to make sure you’re pulling your metrics in the right way - with process and consistency to actually measure what you’re trying to measure.
Today we’ll talk about success measurement, using a ton of data about big brands and media companies on Twitter. We’ll look at the results to understand what levels of success they are finding, and how that success is distributed.
First, we’ll need to start with a data set. I used the Twitter public API to grab data from over one hundred of the biggest brands on Twitter. I started with the Interbrand 100—a list of the most valuable brands in the world—and then added a few brands doing interesting things on social. We ended up with a list of 106 brands. The Twitter public API will give us the last 3,200 Tweets from each account, so I threw all those into a database to see what I could find. I ended up with over 250,000 brand Tweets to analyze.
— Harley-Davidson (@harleydavidson) August 24, 2014
We’ll look at engagement for today’s success metric. There are lots of different ways to look at social success for a brand - everything from reach to followers to direct revenue from social. But for a dataset this wide, we’ll need to look at a measurement that 1) matters for all social brands and 2) we can measure across 100+ brands. Engagement—interacting with prospects and customers via social media to keep a brand top of mind—does exactly that.
Measuring Engagement the Wrong Way
So how do we measure success with engagement across so many brands? If we want some summary statistics from a group of data this large, averages are usually the first place we’d run to. But averages, while easily digestible by the masses, can misrepresent what’s actually happening with the data. Instead of using the average approach, I’d rather look at distributions of success across our brands and brand Twitter posts—or said another way—what percentage of posts are achieving a certain level of success?
I’ll pick a few levels of engagement to gauge success for this exercise. We’ll look at brand posts that are getting more than 50 actions (retweets + favorites), and brand posts that are getting more than 100 actions. What percentage of overall posts are reaching these levels?
If we mine our full dataset of 250,000+ brand Tweets, we get the following results:
Brand Tweets with +50 actions per post: 10%
Brand Tweets with +100 actions per post: 6%
Hmm, that seems pretty low. And in fact, it is. Our numbers, based on this methodology, are actually flawed in a few ways. One flaw is that we’re just looking at the raw number of actions, and not actions based on the potential audience of each Tweet. If an account has 50,000 followers and gets 100 actions, that’s pretty solid. If a different account gets 100 actions from 50,000,000 followers, well, that’s a different story of success. We won’t dive into that side of things today, but I’ll cover that topic in a future blog post.
The second mistake we’re making is that we’re measuring success from all Tweets from the brand. We shouldn’t do that.
Tweets are actually made up of three different types of communication: original content, retweets, and replies.
First, let’s talk about retweets. Brands are in the habit of retweeting other content just like you and I do, and if we’re pulling those metrics into our database we’re also going to import the success metrics from the original Tweets. Because a lot of retweeted content is very popular, using this content as a measure of a brand’s success can inflate our numbers (and just doesn’t make sense.) We shouldn’t count that success for our brands, so we’ll need to filter those out.
— Torchy's Tacos (@TorchyTaco) August 21, 2014
Tweet @replies (sending a message directed to one or more specific Twitter users) have the opposite effect—they are mostly meant for one-on-one communication, and will therefore not have much engagement at all (and that’s ok.) But if we include them in our study, especially because replies make up about 43% of our brand Tweet database, we’re going to wildly skew our results. So let’s get rid of those and try running our numbers again.
Measuring Engagement the Right Way
After removing replies and retweets, we end up with 115,000+ Tweets of original brand content. And when we run our engagement analysis across this dataset, we see a lot more success happening across our one hundred brands:
Original Brand Tweets with +50 actions per post: 22%
Original Brand Tweets with +100 actions per post: 14%
That seems better. In addition to that, we’ll want to review the distribution of success across all of our brands to make sure that a handful of brands aren’t skewing our numbers. Let’s look at how many brands saw at least one 50+ action or 100+ action level of engagement from our 100+ brand list:
Brands That Have Seen 50+ Action Levels of Engagement: 93%
Brands That Have Seen 100+ Action Levels of Engagement: 90%
So success seems pretty well distributed, and there’s a lot of activity out there between brands and their audience.
What Did We Learn Today?
First of all, brands are seeing some pretty impressive levels of engagement on Twitter. After looking across a huge number of brands—the most valuable brands in the world that span from CPG to auto to consumer electronics to oil & gas—there appears to be a good amount of interaction between these brands and their audience.
Our second learning is about pulling data the right way. It’s easy for us to grab every Tweet, run a quick analysis, and call it a day. But getting under the hood with your data, and understanding the makeup of what the data means, can help you get closer to the real truth. Get to know your data, and the insights will follow.