Goodbye ROI, hello Dynamic Efficiency!

I think the acronym ROI is one of marketing’s Sacred Cows, perhaps the holiest of them all.

This is because most marketeers fear measurements like Count Dracula fears the wooden spike: it may kill them, exposing too openly the assumptions and approximations which make beancounters cringe

In an attempt to exorcise this fear, not unlike teenagers gorging on Nightmare movies, marketeers LOVE to talk about measurement, love to read books on measurement and attend sessions talking about measurement.

My skepticism is the skepticism of the engineer: for us, stuff you don’t measure does not exist: unlike those pesky physicists claiming the very act of measurement is what precipitates a waveform into the particle-like quantum state we can observe, we carry sliderules and calibers and like to slap a label with a number on things.

In marketing, Investments are usually clear and defined, Returns much less so, making the calculation of their ratio an exercise ranging from futile to outright false: you change a small assumption into your calculation of returns, and swing a project from Hero to Zero.

I have therefore usually refrained from entering this conversation for fear my skepticism would seep through the thin veil of diplomacy.

Recently, however, a couple of projects have helped me come to better terms with the whole measurement thing and can perhaps point to  a method robust enough to be applied in a more general way.

Rules for measuring

The basic rule for measurement is simple: it should be repeatable.

For example, the kilo of apples I buy at the fruit store should still weigh a kilo when I get home (unless I ate some, of course). Moreover, my kilo should be the same as the store kilo, otherwise there would be an argument about which one I am buying: in other words a repeatable measurement requires standardized units.

You may take this for granted, but it took humanity 99% of its existence to realize how important this was and to agree in 1875 on the metric system establishing the BIPM. As a consequence, should somebody wonder how much a kilo weighs, all they have to do is go to Sèvres and look at the internationally recognized platinum-iridium block which defines how much stuff there is in a kilo.

No such metric exists in marketing, with a twofold complication: the first layer is to decide what we MUST measure but the second is what we CAN measure.

In our case, the MUST part is easy: I sell widgets at a certain rate; I then deploy a marketing action and measure again the rate: if it changes, the difference is due to the marketing action and I can then calculate a meaningful ROI.

Unfortunately, in real life many things happen at the same time, and any marketing action is a complex entity made up of many subparts: for example an advertising campaign is a function of the visual, but also of the media plan or its frequency; establishing a clear cause-and-effect relationship is not easy.

Therefore we must build a model representing a detail fine enough to give me confidence such a cause-and-effect relationship can be established, making measurement effective.

A model for Dynamic Efficiency

The model I will use is the one I described in a previous post, and which we have used many times over with good results, also in projects which are not in B2B, but where the sell cycle is long enough to allow us to distinguish between the stages of the purchasing decision maturation.

Experience shows that the sell cycle moves through three different maturation stages:

The overall efficiency of the process depends on the individual conversion efficiencies of each stage: the faster a user moves from A to B to C, the faster s/he approaches the purchase; unfortunately, while measuring populations at each stage is not difficult, measuring directly flows requires advanced analytics involving in most cases individual tracking (cookies).

This problem, however, is not dissimilar to the problem of those who analyze electoral results: I know exactly how many people voted for party X, but don’t know how many of these did vote for the same party last time around.

This problem has been analyzed and the solution proposed by L.A.Goodman in the 50’s (Ecological Inference) has now been in use for over fifty years (probably more than any marketing metric), a track record long enough to give us some confidence about its accuracy.

Essentially, the problem consists in the evaluation of a Multivariate Linear Regression (sounds a mouthful, but it’s a single Excel function) on a given month’ stage population, giving us a table of coefficients who tell us where the people we had in stage A in January went to in February.

If we take the coefficients in the diagonal of each matrix, these represent exactly how well we are moving people from one stage to the next, and can therefore relate the behaviours of these granular efficiencies to specific actions we performed.

There are many other things that we can infer from the analysis of these tables (for example, how many people go straight to the last stage? How many people are moving backwards ?) allowing us an even better understanding of the dynamics in the population for our Social Media system, but the attractiveness of Dynamic Efficiency for me lies in how a simple measurement conveys a great deal of information we can act upon.

My slide rule is happy.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s