Digital transformation success has always been tricky to measure – partly because the goal lines are always changing. “The primary measure of success last year was survival,” David Cushman, research director in the emerging technology practice at IT consultancy and research firm HFS Research, recently told us. Accordingly, speed was often the key performance indicator.

But as organizations emerge from the frenzy of pandemic-fueled changes, those measures of success (and the way they are applied) may need to change. “IT leaders need to rethink [these metrics] now, otherwise they will equate their panic reaction and survival using digital tech a win,” says Yugal Joshi, vice president of digital, cloud, and application services research for Everest Group. “This will be problematic going forward.”

Where should you start with updating digital transformation metrics? “The starting point must take into account how the IT organization is supporting the business outcomes,” explains Greg Bentham, vice president, cloud infrastructure services at Capgemini Americas.

[ Get answers to key digital transformation questions and lessons from top CIOs: Download our digital transformation cheat sheet. ]

8 digital transformation metrics lessons learned

IT leaders can use some hard-won pieces of wisdom to determine how best to assess the effectiveness of their digital transformation initiatives going forward. Some have already become gospel — like the fact that basic uptime or availability reports have given way to more complicated measures of value like usability, processing speed, and accuracy. Others are a bit less obvious.

Here, we’ll dig into some of the more counterintuitive lessons learned about digital transformation metrics.

1. Paying down technical debt is not enough

Confusing transformation with retiring technical debt is a common mistake, says Jay Upchurch, CIO at SAS. “It’s logical that these two concepts are related as companies not born in the digital age carry years of technical debt that limits their ability to keep pace with digitally native industry peers,” Upchurch says. “Too often, those companies use the term ‘digital transformation’ as a funding tool to replace legacy technical debt.”

It’s an important first step, but it is only digital enablement. “To transform, you must have business engagement and sponsorship,” Upchurch says. “Technology can fuel it, but the business must be open to changing key business processes or reimagining business transactions in the markets they serve.”

Thus, organizations that measure digital transformation by the number of systems retired or migrated to the cloud may find their efforts failing to deliver business outcomes. “They didn’t actually ‘fail,’ though. They digitally enabled and increased technology awareness about the way they do business,” Upchurch says. “By adding business leadership engagement and strategy, they can move to digital optimization and eventually earn the right to digitally transform.”

[ Get exercises and approaches that make disparate teams stronger. Read the digital transformation ebook: Transformation Takes Practice. ]

2. Cybersecurity must be integrated

Cybersecurity has long been considered by many executives to be a cost to be managed or even a drag on overall performance. Today, however, “the realization that cybersecurity has to be part of every discussion is more pervasive now than ever,” says Bentham. “Regulations, now employed in many countries, are driving the accountability to companies, making them liable for damages to citizens, customers and the like.” Thus, technology leaders must incorporate cybersecurity investments into their digital plans and ROI calculations.

“The digital transformation strategist forges an early partnership with the cybersecurity organization and integrates them at all levels of the business and technology,” Bentham explains. “This integration allows the cyber professionals, who write or interpret cyber policies, to do so through a business lens.”

As more organizations evolve to a cloud-first model, their security metrics may need to evolve as well. “Because the cloud is more dynamic, new metrics like mean time to adapt (MTTA) or mean time to secure (MTTS) will apply,” says Vishal Jain, co-founder and CTO at Valtix. “Many organizations today often take days or weeks to adapt security to new applications or even changes in the cloud, introducing an unacceptable level of risk.”

3. The business value chain matters

IT leaders also must be mindful of the full impact of digital transformation initiatives. “Some companies jump into the middle of the value chain to drive transformation. But doing so can add pressure upstream and downstream in ways that cause needless confusion and tension,” says Upchurch. “If you drive transformation in one division, it may create pressure on other divisions that directly interact or depend on that division.” For example, transformation in the legal function may affect the product, sales, and finance organizations.

Thus, it’s important to understand the end-to-end value chain and be selective when driving positive business outcomes that can have a negative impact on other parts of the business. “Be selective when driving constructive disruption, and where possible, allow it to push downstream only in the value chain,” Upchurch advises. “If you don’t, the disruption becomes unmanageable, like the ripple effect of throwing a stone in a lake.”

4. Transparency and accountability are key

Joshi of Everest Group argues that most enterprises are already aware of the right kinds of digital transformation metrics, including operational, financial, risk, growth, customer engagement, employee experience, and speed to market measures. The problem is that they’re not applying them effectively. “What is needed is to track these metrics, hold people accountable, and run digital transformation initiatives [that] impact these metrics,” Joshi says. “The metrics need to be transparent, impactful, relevant, and measurable.”

5. Measure digital skills

“As companies tweak business strategy after COVID-19, those who valued their employees and customers throughout digital transformation will be the most successful,” says Kevin McCaffrey, CEO and founder of Tr3Dent. Consider metrics such as measuring available staff skills vs. skills needed to drive digital efforts, McCaffrey suggests.

Tech leaders must also consider how they access third-party skills, an important complement to internal capabilities in a tight talent market. “The digital transformation strategist who achieves success in this area balances where he/she sources the requisite skills required,” says Bentham of Capgemini Americas.

6. Measure ecosystem value

Speaking of partnerships, “digital transformation also unlocks endless opportunities to partner with new organizations and break into new markets,” McCaffrey says. “To track and respond to progress while forging these new alliances, IT leaders should consider keeping an eye on REP rates, which represents the revenue they’ve generated with partners in their ecosystem.”

7. Apply agile processes (and metrics) beyond IT

“The biggest lesson learned from the last year and a half is that contingency planning is a necessity,” says McCaffrey. “Before 2020, most companies had plans in place to mitigate natural disasters, but almost no one considered the possibility of a pandemic. This taught IT leaders and their C-suite colleagues that it’s essential to prepare for the unexpected, no matter how unlikely an event seems.”

Consider expanding agile processes, originally adopted and measured within IT organization, to the rest of the enterprise.

8. Elevate the people metrics

While it’s important to measure the technical progress of key initiatives like new software or IT environments, the impact on the people using them is the most important measure of success. “Digital transformation is about people, giving them the tools they need to do their job better and more efficiently,” says Khadim Batti, CEO of Whatfix. “Too often, the focus is on picking the latest and greatest software or computational technology to the exclusion of the specific needs of the people who will use that technology. And so, for us, the lesson is that to achieve a successful digital transformation, companies need to focus on their people just as much as the tools they use.” Metrics around employee satisfaction, user experience, and systems uptake must be elevated.

[ Culture change is the hardest part of digital transformation. Get the digital transformation eBook: Teaching an elephant to dance. ]