Benchmarking for Innovation Performance Amid Industry Convergence

A wearable device that monitors critically ill patients between doctor visits… a way to make international payments that is lightyears ahead of the current system… a robot that works collaboratively… These were some of the bold innovations discussed at the 2018 Mack Institute Spring Conference in San Francisco. All these inventions hold the potential to improve life around the world and to move companies into the future.

But of course, to get started, someone had to take a chance. Someone had to have faith that the idea could become a commercial product. And in the case of these innovations, it’s large, established firms that are really making things happen. They’re venturing outside of their core competencies to leverage resources, flex their muscles, and forge new identities for themselves. “There’s a lot of shifting going on across industries; there’s convergence happening,” said Mack Institute executive director Saikat Chaudhuri. “And everyone is thinking about how to benchmark innovation performance as they navigate these challenges.”

Mack Institute co-director Harbir Singh observed, “Companies are realizing that they need to act as venture capitalists.” He noted that corporate venture capital today has experienced a huge jump: it’s about 300% of what it was four years ago.

Singh identified some of the challenges for corporations as they foray into the venture capital space. They need to deploy their assessment of the science behind particular applications, and their assessment of market opportunity, but also need to try to “harness the creative energy of entrepreneurs.” Furthermore, established companies need to realize that while they have the advantage of access to markets and customers, they typically don’t have the diversification that venture capitalists possess. “Companies need to diversify their portfolio so they can address the risk that’s involved in corporate venture capital,” said Singh.

How do firms make wise bets on long-term innovation projects when prospects are distant and uncertain, Singh asked? What’s the right balance of risk and return in an innovation portfolio? And how can we measure the success of innovation? Singh and Chaudhuri led the group of top business and academic leaders to explore these issues and more.

How Incumbents Can Stay Ahead of the Game: Asheesh Birla, Ripple, and Alain Neyroud, IBM

Asheesh Birla
“I can watch streaming video from Mars, but I can’t send a payment to India instantly and cost-effectively,” said Asheesh Birla. The world’s payment infrastructures were never modernized for the Internet age, Birla explained, which is why moving money across borders is slow, unreliable, and expensive.

Birla is the senior vice president of product management at Ripple. Founded in 2012, the firm is the world’s only enterprise blockchain solution for global payments.

As a blockchain innovator, Ripple took an unusual path. The fledgling company decided to work with incumbents: banks and other established financial institutions. Initially, this decision did not go over well with many on the team and with venture capitalists, Birla noted. “When you work in Silicon Valley, you think incumbents are the yellow cabs: they’re going to go away, they’re dinosaurs.”

But the gamble paid off. Today Ripple serves banks of all sizes globally and has 100 signed customers, among them Bank of America, Santander, and American Express. Birla explained that Ripple makes international payments efficient by selling blockchain-based solutions. He noted that one of their beta products called xRapid is enabling instant payments while achieving a 40-70% cost savings in pilot programs in India, Brazil, Mexico and the Philippines. xRapid uses the digital asset XRP, which is the third most valuable digital asset and is faster and cheaper than the popular cryptocurrency Bitcoin.

Podcast: Listen to Birla discuss his work at Ripple here.

IBM’s Alain Neyroud said he helps incumbents innovate using cloud technology and blockchain. “Incumbents are back,” he asserted: we are seeing more and more disruption coming from established players.

Neyroud is Principal Architect and Blockchain Garage Lead for the IBM Cloud Garage, which was founded four years ago, He said the Cloud Garage’s approach gives companies the ability to innovate like a startup.

Design thinking is “central to the way we drive our projects,” said Neyroud. “We don’t start with technical conversations… we start at the business level where we review requirements, and we drive alignments around those requirements across all the participants.”

A key stage of the process involves helping clients create a minimum viable product. Neyroud offered the analogy of getting into the wedding cake business. Instead of throwing a huge amount of time and resources into creating elaborate cakes, you might want to start by making cupcakes, and gather valuable customer feedback. His message was, “Dream big, but act incrementally.”

Alain Neyroud

Podcast: Listen to Neyroud discuss IBM’s blockchain initiatives here.

Addressing the blockchain side of his work, Neyroud explained how the system can vastly improve communication among companies and their partners, banks, insurance companies, regulators, and auditors. Like Ripple’s Birla, he believes the way firms interact today is inefficient, expensive, and vulnerable. When companies rely on emails, calls, and paper exchanges, none of the parties ever sees the entire picture.

But with blockchain — a shared, replicated, permissioned ledger — “we give people the opportunity to create a shared view of the ecosystem. It’s as simple as that,” he said. Through blockchain, IBM facilitates process innovation across organizations.

What Competencies Are Needed for Innovation?: Katherine von Jan, Salesforce, and James Williams, Teradata

Innovation shouldn’t just be relegated to a special team or department: it should be the responsibility of everyone in the organization. That’s the view of Katherine von Jan, a vice president for strategic innovation at Salesforce.

She talked about building innovation competencies in the workplace, stating that the one quality that should be nurtured more is human vulnerability. Innovation is hard, but it’s not acceptable not to try. Employers must encourage employees to be vulnerable – to face the unknown, take risks – in order to solve some of the world’s greatest challenges.

“Throughout the innovation process, use your values to guide your behaviors,” von Jan said. “At Salesforce, trust is our #1 value. When we are innovating, we are always asking if our work will build or erode trust.” She told a story about how Salesforce experimented with a tool to better understand employee attrition. Through this effort, the team realized that the tool may inadvertently cause managers to reward employees who are exploring new roles just to receive a retention benefit.


Instead of deploying the tool, the project led the team to focus on addressing a more important question – why people might be thinking about other roles in the first place. Through research, the company found that employees often explore opportunities because they want to grow professionally. This finding is one of the many reasons Salesforce has doubled-down on workforce development and internal career resources, von Jan said, including an app that encourages employees to record candid, real-time feedback to help people learn in the “teachable moment.”

Von Jan also discussed human-centered design and new technologies such as artificial intelligence to augment business functions and invent new business models. “In the context of employee experience, imagine if, in addition to traditional career resources, we could use AI to provide every employee with their own personal AI talent agent,” she said. “We could democratize the opportunity to have a talent agent, to provide every individual with personalized, smart coaching and advice, much like way talent agents understand and advise actors and athletes in their careers. Many workers crave that kind of hyper-relevant insight and foresight.”

Also offering insights on building competencies for innovation was James Williams, a senior director for technology research and innovation at Teradata, the world’s leading provider of business analytics solutions.

He talked about Teradata’s ongoing journey to operate more collaboratively “in a way that we don’t feel a sense of ownership, and actually [work together] holistically,” he said. He noted that having a top-notch leadership team to promote collaboration and innovation is of tremendous benefit. “Innovation needs to be supported by the top at all times,” he said. “It cannot be something that you put down as a concept and then the senior leaders [just] say ‘go forth and innovate.’”

Every employee needs to think innovatively, he said, adding that it doesn’t necessarily mean inventing technology. For example, someone might figure out how to improve onboarding for new hires so the company can maximize its utilization. “Every single piece of our ecosystem in terms of how our corporation operates, has an effect on our overall product and efficiency,” he observed.

James Williams
An innovative culture promotes competencies, Williams said. The key competencies are creativity, including generating ideas and critical thinking; having an enterprising outlook; forecasting long-term consequences; and managing change, including risk-taking and challenging the status quo.

Another competency is being able to integrate different perspectives. Williams gave the example from when he was a CIO of dealing with computing scientists who refused to give up their much-loved system even when it aged out of true usefulness. (“Sometimes the baby is ugly,” Williams joked.) Employees need to be responsive to what others have to say.

Thinking Like a Venture Capitalist: Michael Dolbec, GE Ventures

Michael Dolbec, a senior managing director for GE Ventures, shared a window into how the venerable multinational conglomerate explores new opportunities. “GE Ventures is one of the clubs in the bag — to use a golf metaphor — for [GE to] engage with the external innovation ecosystem,” he said.

Dolbec noted that the natural tendency of large companies such as GE is to “do everything yourself… extract as much value as you can and get really good at very incremental innovation.” However, these companies are continually exposed to disruptors from their flanks. It’s a big problem that’s getting worse, he said, hence the need to engage with what’s going on in the larger environment.

He outlined GE’s broad range of investment areas including advanced manufacturing; energy and mobility; enterprise productivity applications; healthcare and life sciences; connected tech and services; and cyber, data, and cloud. He gave examples of innovative portfolio companies such as Rethink Robotics, which makes smart, collaborative robots, and Catalant, a global enterprise solution to access business expertise on demand.


In addition to venture investing, GE Ventures engages in business incubation and also technology licensing, which he noted was a major financial contributor.

As Dolbec explained the facts and figures around GE Ventures, he said that what’s behind it all are “tens of thousands of conversations that we’re having with entrepreneurs [and] venture people,” to detect trends in the industries. “I think what a lot of senior executives miss is that because we’re having all these conversations, we’re picking up on what’s working.”

Why Measure Innovation — And How to Do It?: Saikat Chaudhuri, Executive Director, and Harbir Singh, Co-Director, Mack Institute for Innovation Management

Chaudhuri and Singh led the participants in a workshop about measuring innovation. Why have metrics around innovation, they asked? Because it provides strategic direction by signaling shifts in priorities, and guides resource allocations. It helps hold managers accountable, and links incentives to reaching targets. Moreover, it helps firms assess the effectiveness of innovation spending, and diagnose and improve innovation performance.

But there are pitfalls, too. An atmosphere of closely measuring innovation could suppress the more adventurous ideas. Another problem is the business may be measuring what’s available to measure, but a different, unrecognized metric would actually be more useful. Above all, measurement can lead to an over-emphasis on the end result — such as ROI or ROE — at the expense of nurturing process and performance.

Chaudhuri presented over 20 possible metrics in the categories of Input, Process Effectiveness, and Output that could be used to measure innovation success: everything from R&D spending as a percentage of sales, to the number of new products launched, to customer satisfaction. He and Singh guided the audience to discuss the measurements they had found effective, as well as other areas where their companies might benefit.


One participant said her company was measuring “how fast you fail” in cases where there’s a risk of failure. This enabled the company to quickly move on to more promising projects. Another executive offered that her company was working on indexing ideas; for example, where they came from, their purpose, product versus service, internal versus external. She said this helped certain ideas “bubble up” that weren’t getting the attention they deserved.

An R&D manager shared that her department measures the number of patents and publications its researchers generate. While this helps track the labs’ output, she said, the connection to successful commercialization is not always apparent. Chaudhuri noted that in academia too, patents are a proxy for innovation, but getting from invention to innovation is not guaranteed.

Singh stated that although innovation, or input and output, are challenging to measure, that doesn’t mean that firms shouldn’t try. “Not measuring may be worse,” he observed, to murmurs of agreement. Furthermore, over time, the business can calibrate how it tracks projects. It can try to remove biases and subjective factors, and convert its measurement process into “something that is more customized” to the particular company.

In the end, there were no one-size-fits-all answers, but the participants walked away armed with new insights, frameworks, and best practices. They experienced a unique opportunity to share learnings and think deeply about new approaches for their own firms.

A Look at Smart Technology Portfolios: Tereza Nemessanyi, Microsoft, and Hanson Lenyoun, Samsung

Companies undergoing digital transformation often have to juggle a large volume of innovation initiatives sourced from many different places, said Tereza Nemessanyi, a Microsoft entrepreneur-in-residence. Enter Microsoft’s “Startups-as-a-Service.” “We can be your strategic partner to identify the right startups, stand up pilots, de-risk them, learn, and accelerate success,” she said.

Nemessanyi asserted that Microsoft is in a unique position to assist companies because it works with so many startups itself: about 2,700. “We’re touching startups at a whole range of levels — not just putting up money for venture investment,” she added, explaining, “If we’re going to test our tools… we have to be in the pit with the best startups, seeing how our stuff interacts with those technologies. It’s a critical requirement.”

Tereza Nemessanyi

Podcast: Listen to Nemessanyi discuss her work at Microsoft here.

She elaborated on some ways that Microsoft interacts with startup companies. One way is through what she termed “classic corporate development, M&A kind of stuff.” Another is M12, formerly Microsoft Ventures. And there is Microsoft ScaleUp, which involves accelerators across the world. Through this program, said Nemessanyi, series A-type startups that realize they’ve found product-market fit can be pulled into the Microsoft ecosystem so they can move to the next stage.

Like GE Ventures’ Dolbec, Nemessanyi talked about the importance of keeping an ear to the ground for what’s going on in various industries. She is “listening and creating filters” to understand what’s happening in the marketplace. “What’s interesting is we can sometimes intuit patterns before others do,” she stated. “Because we’re talking to a whole lot of people.”

People generally don’t think of healthcare when they think of Samsung, but the company has actually been in that space since about 1985, said Hanson Lenyoun, MD, Samsung’s director of business and clinical outcomes. He spoke about the electronics giant’s latest medical innovations.

Lenyoun said that as a reconstructive surgeon, he saw many patients with chronic conditions and non-healing wounds who underwent a successful surgery only to require another surgery a month later. Often it was because they weren’t taking care of themselves between doctor visits. “I got really frustrated around that, and started saying, ‘How can we build in ways to affect these patients… and keep them healthy, keep them off the OR table?’ The answer to me seemed obvious, and that was technology.”


He talked about Samsung’s healthcare technology portfolio. One offering is GearUp, an employee wellness and fatigue management solution. The wristwatch-like device is intended for safety-critical businesses such as aviation. Another product is Ask an Expert, a mobile health consultation app through which patients can “meet” with board-certified physicians 24/7.

Samsung has also gotten into virtual cardiac rehabilitation. Lenyoun pointed out that after a heart attack, rehabilitative exercise is key. He described a smartwatch and companion app for one’s phone that monitors the patient’s exercise routine. The care team can digitally track how the patient is progressing.

All in all, said Lenyoun, “We’re really changing the way that healthcare is delivered.”

It’s Not Just About Technology

In closing the conference, Chaudhuri identified four aspects of innovation. First, the fundamentals: the challenges of innovating at established organizations. Second, ways to build the necessary competencies. Third, ways to figure out “whether what we’re doing is on the right track.” And fourth, if innovation is “really all about experimentation anyway,” how do we set up viable portfolios?

Innovation can be internal or external, he said, but increasingly we’re looking to outside ecosystem solutions. This requires a mindset that’s very open and amenable to collaboration. He also emphasized the fact that, since no individual or company can truly know where the market is going, fast and agile adaptation is absolutely critical.


Singh expanded on Chaudhuri’s point about ecosystems. “Ecosystems are at the heart of innovation,” he commented. For example, “you have ecosystems for smartphones, healthcare, and many life science applications.” He noted that a key ongoing project at the Mack Institute involves studying the design of ecosystems. The work is really on interdependence, he said, and how different forms of interdependence between players result in different ways that value is generated.

Various types of ecosystems were reflected in the conference talks, said Singh. Among them were a learning and information processing ecosystem, which involved startups surrounding a large, strongly-positioned corporation. That could be seen in Nemessanyi’s Microsoft presentation, he said. And in Lenyoun’s Samsung talk, Singh observed “a very purposeful ecosystem with a particular application.”

Ultimately, Chaudhuri reminded the group, there’s so much more going on here than technological innovation. “The technology is almost a given,” he said. “It’s about the strategy and organization and processes that we use to approach innovation which allows us to be successful.”