I wanted to share a quick takeaway from a panel discussion I attended at SXSW on ‘Designing for a Design Culture’. Several prominent folks from the gaming industry, Greg Street of Riot, John Erskine of Cloud Imperium Games and Ken Troop of Wizards of the Coast, shared wide-ranging experiences from their time in the industry, But there was one theme that came up a few times in a few different contexts. No one on the panel gave it a name so we’ll call it ‘Metrics Bleed.’
The best illustration of this came from Ken’s story on ‘common wordcount’. For those of you not familiar, Wizards of the Coast make a popular card game called Magic the Gathering, (which I played a lot of in my younger days). Common wordcount was a metric proxy for ‘simplicity’, essentially the fewer words used on a commonly used card to explain what it did, the simpler the gameplay. However, what they found over time, as the game increased in complexity, was that the fixation on keeping common wordcount low was simply migrating necessary detail to things like the rule book and compendium manuals. The game itself wasn’t staying simple, in-face the opposite. The metric looked good, but the intended effect was unsuccessful.
Of course, it’s not hard to see how similar things can occur in many aspects of our organisations and personal lives. Are you trying to re-purpose some of your time by successfully giving up one compulsive habit, only to have it eaten up again when you start scratching whatever itch the first habit served by picking up another?
It’s worth considering that, if the thing you’re trying to manage keeps popping up in different guises, and you’re doing the best possible job to mitigate the thing you’re measuring in each context, could it be an inexorable part of the system? The answer may be a resounding no, but it’s worth asking the question because if it is, you might want to think about embrace and optimisation instead of mitigation. In the example above, accepting that the card game is necessarily complex in order for players to feel a sense of mastery, and promoting it as such, could be an alternate strategy.
Whatever the context, it always pays to measure the end metric you’re optimising for as well as paying attention to the second order effects of whatever functional or tactical change you make in service of that end metric.