When most marketers hear the term “engagement metrics” they think of open rate, click rate and conversions – the analytics we use to track a how well a message performed. Analyzing our messages in this way is useful because it allows us to compare this week’s to last week’s, isolate why changes in responses occur, and try to improve our subsequent messages based on what we learn.

However, there are also some shortcomings with this approach to engagement metrics. While measuring response at the message level ostensibly tells us how well our messages are performing, it is prone to error. For example, if you sent a message between Christmas and New Year’s and saw that your open and click rates were depressed, you would likely conclude that many of your subscribers were on vacation and not paying attention to emails. But some of your subscribers are on vacation every day, which means that every message you send suffers from the same audience absenteeism you see during the holidays. Look also at open rates. If you write excellent subject lines that telegraph the content of your emails very well, the people interested in that content are likely to open them. By the same token, a well-written subject line tells subscribers who are not interested in that content to ignore the message, which many do. You have a record of the openers being engaged, but can’t you also make the argument that those who read the subject line and decided that this particular message does not apply to them (maybe promoting a white paper on a topic that is outside of their area of responsibility) are engaged also? They have engaged with your message by not reading it, though the brand interaction may well be as strong as those who did read it.

What we normally think of as engagement metrics then do not really measure subscriber engagement as much as message engage-ability. A better metric to evaluate the health of our email program would be one that measures how many of our subscribers are engaged, and how deeply. Put another way, when you launch a series of emails to promote an upcoming conference, you are less interested in how many opens and clicks each message will generate as you are in how many of your subscribers are likely to attend. Subscriber engagement metrics can give us visibility into the results your email program might yield.

Here are a few ways to begin looking at subscriber engagement metrics:

Add the element of time.
When you look at a message’s engagement metrics, you might see that 200 subscribers out of your 1,000 subscribers opened it, and 50 clicked. You can deduce then that the 800 who did not open the message are not engaged with it, but does that not mean they are not engaged with your brand? To find out, add the element of time. Instead of tracking the number of opens and clicks for a single message, track them instead over a month or a quarter or a year. If you mail a weekly newsletter it is bound to reach some people when they do not have the time or inclination to read, which depresses that message’s metrics. But how many of your 1,000 subscribers have opened or clicked at least one time in the past month, or 4 sends? Chances are very good that if you average 200 opens and 50 clicks for each one, your total number of engaged users will be higher than that as it is not the same 200 people opening and 50 people clicking each time. Tracking over time lets you better size your engaged audience. You may have the attention of fully half of your list, instead of the 20% suggested by each message’s results.

Weight for recency.
A subscriber who clicked on yesterday’s newsletter may be no more engaged with your brand than one who clicked last week but not this week, but at some point the duration since a subscriber’s last interaction does matter. What that duration is depends on your content strategy and frequency. For example, many retailers see a huge percentage of their subscribers go dark for 11 months, only to return predictably during the holiday season. Trade associations that focus on a large Annual Conference see similar patterns. Other brands are relying on email to keep their subscribers engaged and connected throughout the year, so a lapse of even a few weeks can signal waning engagement. When you track subscriber engagement over time, consider weighting recent interactions more heavily. For example, someone who opened or clicked in the past two weeks might be considered engaged, while someone who last interacted between two and four weeks ago might be 50% engaged (or rather, you will need two of these people to equal a single fully engaged subscriber).

Identify tiers of engagement.
With engagement tiers you are not looking just at how recently someone interacted, but how likely someone is to interact with your next message. Recency alone does not get you there. For example, Subscriber A may have opened and clicked on a message yesterday, but had not interacted with your brand at all for the past three months. Subscriber B did not open or click yesterday’s message, but did open and click six of the past eight messages. Which of the two is more likely to interact with the next message? Build your engagement model to account for both recency and frequency in order to group your subscribers into tiers that more accurately reflect their level of engagement, and predict future interaction. Then compare your results on subsequent sends to see how what percentage of subscribers in each tier actually did interact. Does “Fully Engaged” mean that 100% interact with at least one message in a two week period, or 75%? Does “Moderately Engaged” mean that 25% interact, or is it as high as 50%? After a few sends, you will know how many subscribers are in each of your tiers and how likely each is to interact over a period of time. That allows you to calculate – and track – how engaged your list is overall.