[Disclaimers: I am an employee of Bristol Myers Squibb. The views expressed here are my own. Also, I am not a particularly good poker or chess player. It is one reason I am a popular invited guest to poker nights with friends.]
I posted on poll on Twitter to ask the question is drug discovery more like poker or chess. There were over 300 responses, with the results split nearly equally (54% poker, 46% chess).
My answer to the question, “Is drug discovery more like poker or chess?”, derives from the following truths:
Poker is a game of skill and chance, where critical information about how to win is hidden. In poker, one has to make probabilistic decisions with incomplete information.
Chess is a game of pure skill, where all information is available and – for the best players – decisions are deterministic. Unlike poker, chess contains no hidden information and very little luck.
Thus, my “answer” to the question is drug discovery is more like poker than chess – largely because of available information (poker = incomplete, chess = complete) and the importance on probabilistic (poker) vs deterministic (chess) decision-making. Here is more context.
Thinking in bets
I recently read the book “Thinking in Bets” by poker champion Annie Duke (@AnnieDuke). Her thesis is that most decisions we make in life are based on bets, whether we acknowledge this betting framework or not. And bets, by definition, take uncertainty into account when making decisions about future states. The book is excellent, and I highly recommend it.
The book makes a compelling case that by acknowledging uncertainty, we become more comfortable with the process of making good decisions rather than simply focusing on the outcome itself, which could have been the result of a good or bad decision. She refers to this as “resulting” or outcome bias. That is, a good outcome doesn’t mean you made good decisions to arrive at that outcome. Indeed, as in poker, you might win a hand – a successful outcome – simply because of luck. Just ask Pete Carroll.
There are many other features of “thinking in bets”, which I will not summarize here. If you are interested in learning more, here are some sources of information that I read/watched in addition to reading the book itself (here, here, here, here).
Why drug discovery is like poker
There are so many decisions that must be made on the way to getting a medicine approved (e.g., link to @DerekLoweblog post here). To name a few: choosing a target to pharmacologically perturb; inventing a molecule that faithfully recapitulates causal human biology; testing the molecule for favorable and unfavorable effects on in vivo physiology (first in non-human organisms and then in humans); designing a clinical trial to show benefit in patients with disease; articulating the benefit-risk of the medicine to regulators who ultimately approve a drug for commercialization.
At each step, there are many probabilistic decisions that can influence the outcome – some are known but many are not.
Those who wait for complete information (chess) will likely over-invest at each stage with the hope that they will gather all information to make a perfect decision. Such decision-makers will likely either fail to innovate (“We just don’t know!”) or be so far behind the competition that the medicines they produce will be incremental in their benefit to patients.
The flip side is that those who over-embrace chance or luck in decision-making will be too cavalier and sloppy in their decision-making framework.
The key is to get the right balance between gathering information and making decisions with incomplete information. I think it is critical to state clearly what is knowable at each decision-making inflection point, to understand the assumptions that go into the decision-making framework, to debate the investment required to gain more information vs the risk to advance with the knowledge available today, and to make the best possible information aggregating all of the above – i.e., to place a bet.
This approach requires a decision-making framework. Personally, I believe in a framework based on causal human biology (CHB) to clinical proof-of-concept (PoC) in small, translationally rich clinical trials (here). But this is just a framework based on assumptions, many of which are still debated (another @DerekLowe blog post link here).
Another benefit of the poker metaphor is the size of the bet. How much are you willing to invest to get to the next inflection point in a drug discovery program? If a probabilistic framework says that probability of success (PoS) is high and cost is low, then it is an easy decision to advance to the next inflection point. If the opposite is true – PoS is low and cost is high – then the decision is more difficult. If a true result leads to a transformation medicine, then perhaps it is worth the investment and risk; if a true result leads to incremental benefit, then perhaps it is not worth the investment.
Lastly, the poker metaphor acknowledges the other “players in the game” of drug discovery and development. There is competition from other companies, and there is uncertainty in how regulators will respond to a data package for approval.
Another way to think about this question is that if drug discovery and development were like chess, then companies that invent one successful drug should be able to do it again and again, right? They must have the formula for success, understand the rules, have complete information, etc. I just don’t think it is possible at this stage in human history.
Risks of the poker metaphor
But buyer beware. There is a trap to the poker metaphor. In poker, there are well-defined odds based on your cards and those of your competitors. In contrast, in drug discovery and development the odds at each step are not well-defined, as highlighted by @sciencescanner (Tweet here). As discussed above, I believe in a framework based on CHB to PoC…but this is a framework with incomplete information.
I would go one step further and say that the poker metaphor can be abused in drug discovery and development. Too often organization place a precise estimate on “probability of technical and regulatory success” (PTRS) together with a commercial model that predicts “net present value” (NPV) on a program. Organizations imply that they have precisely defined the odds of success and financial return of an investment…but too often there are too many assumptions that make these estimates wildly imprecise, especially for programs in discovery or early development.
Personal reflections on leadership traits
I will now take a brief detour and tell you about my results from a personality test used to determine my leadership potential. I scored very low on “confidence”, defined as the “belief that one controls the course of life’s events”, and “drives results”, defined as “consistently achieving results even under tough circumstances”. I consider myself a confident person who makes tough decisions to advance programs, so I found the results strange.
In discussing the results with my assessor, the answer that came back to me: confident leaders believe they control the outcome of events, and that these behaviors will drive results.
My response – based on probabilistic decision-making and thinking in bets – is that the outcome or “results” are less important than the decisions that lead to those outcomes. This is not to say that outcomes are not important – outcomes are ultimately what lead to a successful company. Rather, I argued that “drives results” is not the right way to frame success. We should be evaluated based on “drive decisions”.
As per “thinking in bets”, leaders who focus on decision-making in a complex field like drug discovery and development will, over-time, be more successful than leaders who simply “drive results” based on a framework focused solely on outcome. I indicated that the best way to influence favorable outcomes for the company is to focus on the steps that lead to the outcomes – probability of success, investment required to reach inflection points, assumptions that need to be true, transformational impact of a positive outcome, etc. That is, I believe that encouraging “drive results” is actually the WRONG behavior to promote in a complex organization that should instead encourage probabilistic decision-making. Confident leaders should drive decisions and results will follow.
Concluding remarks
On balance, I believe that drug discovery and development is more like poker than chess. I believe a framework for estimating probabilities (even if wrong!) and a decision-making matrix to advance programs to the next inflection point is a healthy approach. A benefit of the poker metaphor for is the emphasis on decision-making not the outcome. A risk is that we place too much emphasis on what is knowable today – at the extreme, precise estimates of PTRS and NPV to make decisions. One day, we may know all of the information required to lead to a successful drug (i.e., chess), but until then I will bet on probabilistic decision-making…even if I personally score low on certain leadership traits in a standardized test!