We Need a New Way to Measure Patent Value

Listen to this interview as a podcast:

The value of a patent traditionally depends on the number of citations it receives from subsequent innovations. But new research calls into doubt whether citation numbers accurately reflect a patent’s true value. According to the working paper, “Cognitive Neighborhoods and the Valuation of Innovation: A Cross-National Analysis,” a patent’s assigned technology class actually has a significant impact on its citation numbers, making the true value of the patent more complicated to assess.

Knowledge@Wharton recently spoke with the paper’s authors, Wharton management professor Tyler Wry and doctoral student Adam Castor, to discuss their research and its implications. This study was funded by the Mack Institute.

An edited transcript of the conversation appears below.

This interview is part of a series produced by the Mack Institute and Knowledge@Wharton. Explore more interviews about Mack Institute research here.

Knowledge@Wharton: Tell us about your paper.

Adam Castor: This paper is trying to understand exactly how classification systems — which differ across countries — affect outcomes in innovation. In particular, it tries to understand how the classification system affects how patenting examiners search for prior art, and how they find what might be relevant or not relevant.

Knowledge@Wharton: What are some key takeaways of your paper?

Castor: The key takeaway is that it is really important to understand how the classification system affects the way that an innovation is going to be viewed. If it’s assigned to a certain class, a class that is much more central and much more related to other technology classes, it’s going to be much more likely to be picked up by others when they are doing future examination search, by not only patent examiners but also inventors. So it’s going to be much more likely to be cited, and probably much more likely to be more highly valued. So that’s an important key takeaway.

Tyler Wry: Something that is interesting to keep in mind when we are looking at patents and classification, and the valuation of intellectual property, is that there is an assumption that the number of citations that a patent gets in future innovation is a reflection of its value. And what Adam has done really nicely in this paper is show that it’s not just a reflection of the quality of the patent that is driving its valuation, but it really is where it gets slotted into the classification system.

So examiners like all of us use heuristics and shortcuts to think about what is related and what isn’t. And what we have done in the paper is actually modeled this out, and shown the structural characteristics of the different patent classes, in terms of how they relate to each other based on how examiners are searching.

“If it’s assigned to a certain class, a class that is much more central and much more related to other technology classes, it’s going to be … much more likely to be more highly valued.” –Adam Castor

The big takeaway is that the same piece of intellectual property, if it’s in a patent class that is in the neighborhood of many others, examiners are more likely to then cite it in future patent searches, and this is going to affect its valuation. So irrespective of quality, it’s examiners, it’s their cognition, and it’s the properties of the categorization system that really seem to be driving valuation in an important way, as well as perceptions of what is a breakthrough technology versus what isn’t.

Castor: Another important aspect that we find is that we looked across different patent systems — the U.S., Japan and Germany — and you can see that there are structural differences in the classifications across these different patent jurisdictions. This itself, even looking at the exact same innovation being applied for in these different systems, can have very different outcomes … in strong part due to the fact that the way the technology is classified is different.

And so this is really important for practitioners to be aware of, not just in terms of getting a better understanding exactly of the valuation itself, but as Tyler mentioned, what might be breakthrough and what might not be breakthrough [innovation]. What is the best way to ensure that something is going to be a breakthrough [innovation], and is going to be seen by the most people going forward?

Knowledge@Wharton: So what conclusions, if any, surprised you?

Castor: I expected that for technology classes that were very central, or very related to other technology classes, I thought there would be a downside in the sense that … it might attract attention away from that focal class. So if you think about the patent examiner/inventor who has a technology, and they’re looking back, if the technology class itself that they are originally searching is very broad, they might not have as much time to search directly in that primary class. So I was really expecting there to be a negative effect as well, at least in terms of seeing that.

What was surprising to us is that we didn’t find that at all. In fact, what we do find is that the difference in terms of what technologies get cited more often than not comes from citations outside the class. And there was actually no negative effect within the class itself, which I think for me was quite surprising. I thought there would be some negative associated with being more central.

Wry: It surprised me that the same piece of intellectual property applied for in different systems is valued so differently. You would expect that there is some sort of underlying quality that is reflected in the valuation of intellectual property, and the degree to which it’s picked up and built on. … This is true to a certain extent but depending on the properties of the classification system, this changes in really quite dramatic ways. And it really speaks to the cultural creation of innovation, and innovation outcomes.

“It surprised me that the same piece of intellectual property applied for in different systems is valued so differently.” –Tyler Wry

Knowledge@Wharton: What are some of the practical implications of your findings?

Castor: The first important thing to note especially for practitioners is to understand the differences across these systems. [However,] research might be one step away from really being able to explain how should patent applicants proceed in terms of trying to maximize value. So I think an area for future research is to understand exactly how much agency they have, in terms of what classes are assigned.

Certainly, there is at least indicators that the USPTO [United States Patent and Trademark Office], for instance, has a very strong hand in affecting what the eventual primary classification is going to be, which is what we are looking at. But also, when applicants file for a patent application they are required to include a class in their first application. So I think it’s clear that the classes are important; the next step is to understand exactly how much agency, and what can applicants do in order to ensure that their applications are going to be funneled into the classes that they want, and be seen by the proper examiners.

Knowledge@Wharton: So what sets your research apart? I think you kind of touched on that. Could you explain a little bit more?

Castor: Theoretically, there is a large difference in terms of the way that classification, understanding its use as a heuristic, has been used. This context is very different because we are trying to understand search, and particularly search for innovation. So generally, the research is focused on consumers looking for specific products, which is a slightly different context, one where they already have in their mind a certain type of product they are going to look for, and so the dynamics are very different.

When you are thinking about technology on the other hand, especially in this paper, we’re looking at nanotechnology — a very nascent area – [the goal] is to understand exactly how do people make sense of these new technologies and in particular, when they are searching, trying to understand how it’s related. Where might this new innovation or invention fit into the broader sense? And how might one frame it in order to best take advantage of the institution, in terms of its classification — aspects — becomes really important.

This is something that is missed, and more broadly, we have a much more fluid view of the way that these classification systems work. So theoretically, most of the research is focused on how these classifications are fixed in time, and all individuals really have the same premonition or the same cognitive schema of what these classification systems look like. Whereas we are taking a much more fluid look, and trying to look at how a new technology class — nanotechnology — is starting to become more mainstream, and how that affects innovations over time.

“Particularly for new innovations and new technologies, in order to garner widespread attention, you actually want to sometimes appear more broad.” –Adam Castor

Wry: The only thing I would add to that is that, I think one of the unique things about this work that we are doing is instead of looking at individual categories or individual classifications, which is the traditional approach in the academic literature, we are really highlighting the influence of the properties of the classification system.

And so we’re taking a level of analysis from here, and expanding it out to the broader topography of categories, and how they relate to each other. Now this leads to some different questions, and this is part of the basis for wanting to undertake this research in the first place. Because the traditional prediction is that if you want to be understood, and you want to be valued, you have to fit neatly within a particular category. This applies very generally, to technologies, to people, to organizations, and there is a large literature that has looked at this.

But what hasn’t really been looked at is how the embeddedness of the category that you are in, in a broader system of classification, affects outcomes for you, as a technology, a person, an organization, whatever it is, and that is really the piece that we are bringing to bear in this research. Showing that yes, it’s important what category you are in, but really what is driving the effects, especially when valuation is based on being on the radar of lots of other things, is where this class sits in the broader system.

Knowledge@Wharton: What conclusions from your research could be useful to venture capitalists, inventors and businesses?

Castor: Oftentimes we think of it as being necessary to fit into a single category, and have a very, very salient identity that fits into a single category. And that is important to a degree. But particularly for new innovations and new technologies, in order to garner widespread attention, you actually want to sometimes appear more broad.

We’re showing that it’s not a bad thing to actually fall into multiple different classes, because that’s going to garner much attention. So I think, fundamentally speaking, it’s important to have a salient identity, but at the same time our research is going in the direction of showing that there are a lot of benefits to spanning multiple different classes, product categories, etc., because that might garner more attention, which ultimately might benefit producers themselves and venture capitalists, in terms of thinking about who they should invest in.

Wry: And the other thing I would add to that is, to the extent that venture capitalists, or their audiences, are looking at patent citations as a way to value a startup’s IP, or even an established company’s intellectual property, they might be picking up some misleading stuff here. So doing close due diligence is still probably very important, especially when you’re thinking about something like nanotechnology that really is global in its scope, and revolutionary in its potential.

Unless you are actually getting under the hood, and understanding the nuts and bolts of what this technology is doing, what its potential is in the real world, you might be missing stuff if you are just looking at things like the traditional measures of forward citations; it is picked up by a lot of other innovations, so this must be important; or it’s picked up by lots of diverse innovations, so this must be a breakthrough. What we’re showing is that, there is a piece to that, that’s true. But if you are relying just on that, you are probably missing a big chunk of what is actually driving quality in this field.

This post also appears on the Knowledge@Wharton website.