Everyone loves Freddie Mercury. The late frontman for Queen, he remains one of the greatest singers in the history of popular music. His vocal range was astounding, and his musicality was unsurpassed.
Just listen to the opening of “Bohemian Rhapsody.” The famous 6-minute suite begins with an amazing five-part a capella harmony (“Is this the real life?”). But the accompanying video shows the images of only four band members, and three of them are simply lip-syncing. All five parts of the harmony are sung by Freddie Mercury and are superimposed using multi-track recordings. The effect is breathtaking, and even more so when you learn that only one person is singing.
That is the essence of harmony. In music, it refers to simultaneously sounded (but different) musical notes combined to produce chords with a pleasing effect. Outside of music, it describes the result of forming a coherent and consistent whole (from disparate parts) that is pleasing to the mind and soul. Listening to each component part in isolation provides little meaning and joy; it is how the parts fit together that produces something of real value.
The principles of harmony are especially applicable to the interpretation of clinical trials in medicine. When several clinical trials have evaluated the same intervention, their results are rarely perfectly aligned. Often, certain findings in one trial are entirely in conflict with observations of the same effect in other trials. When the results of trials collide, which trial is correct?
There are two distinct approaches to reconciling the disparate results of two trials on the same research question.
One common approach is to dissect each of the trials to death, looking at each as if it were an isolated piece of evidence. It is a destructive process that pits the results of one trial against another, with the ultimate goal that only one trial is left standing at the end. This adversarial approach neglects the fact that both trials were carried out in their own unique way and were not designed to compete. The goal of the adversarial approach is to declare a winner. However, quite often, the process succeeds in destroying both trials, yielding a “hung jury.” The dissectors walk away from the bloody scene and declare that nothing has been learned, and that a third “tie-breaker” trial is needed.
The destructive analysis of both trials leading to a call for a tie-breaking trial is often both unscientific and unhelpful. In the end, the analysts often conclude that they cannot tell which trial is correct, because they assume that — when there are different results — only one study can be correct. Furthermore, if a reported benefit is a persuasive finding of a mortality reduction, it is often not clear that a tie-breaker trial is feasible, ethical, or can be financially supported.
A better process — although rarely carried out — is to take the position that both trials may have revealed the truth. They revealed different truths, not because one was right and one was wrong, but because the two studies asked different questions or studied different types of patients. By looking at them side-by-side, physicians engage in a process of reconciliation rather than mutual destruction. Pieces that have individual integrity are taken from each trial and combined into a harmonious whole. Conventionally, in regulatory circles, this process is commonly referred to as “looking at the totality of evidence.” It is not a destructive process; instead, it creates a whole that is greater than the sum of its parts.
I want to be clear on one important point. The totality of evidence approach is NOT a meta-analyses, and in fact, it is philosophically and intellectually the opposite of a meta-analysis. A meta-analysis simply seeks to combine data in a thoughtless process that is uninformed by differences in trial methodology or patient populations. It yields a forced summary estimate that obscures meaning and truthfulness. In contrast, the “totality of evidence” approach is not a process of forced summation, but instead, it is an intellectually challenging process of reconciliation.
Anyone (even those without any cognitive skills) can do a study-level meta-analysis. In contrast, executing a “totality of evidence” approach takes energy and work. It means getting into the details about how the trials were carried out, who was studied, and how the measurements were actually made. It often cannot be done quickly. Because of the effort required, many physicians are attracted to the adversarial process of mutual trial destruction, but that seeks only to identify a winner trial, which typically reveals only a half-truth.
If you like the “tie-breaker” approach to clinical trials, you must think that overtime competition in sports (especially when the overtime period ends suddenly after the first score) is a good way of identifying the better team. It is certainly fast, and it allows fans to move on with their lives. But it is rarely satisfying; it is not necessarily truthful; and it is certainly not harmonious.
Why am I talking about all of this?
In the past 6 months, the investigators of two trials evaluated the efficacy of transcatheter mitral valve repair in patients who had heart failure and severe functional mitral regurgitation. In the MITRA-FR trial, the investigators found no benefit of the procedure, whereas in the COAPT trial, the researchers reported dramatic benefits, including a reduction in mortality. Both trials were ultimately published in the same issue of the New England Journal of Medicine.
The results of the two trials could not have been more different. A meta-analysis of the two would have yielded nonsense. So many took the path of destructive analysis, attempting to blast both trials into nothingness. And to fill the resulting vacuum, there were calls for a “tie-breaker” trial. Of course, this assumed that someone knew how to design a tie-breaker trial and get it funded.
Paul Grayburn and I joined forces to begin a process of reconciliation, which placed the various pieces of data on the table and looked at the totality of evidence. Paul has long been a key intellectual leader in valvular heart disease, and our offices at Baylor University Medical Center in Dallas are only 50 feet away from each other. It did not take us long to discover that the trials had found different results because they had enrolled entirely different types of patients. Both trials yielded entirely valid results, and putting them together (with the help of Anna Sannino) created a harmonious whole.
What did the picture of harmony look like? If the disease process primarily affected the left ventricle, then fixing the mitral valve leaflets was not useful. That is what the MITRA-FR trial discovered. However, if the disease process primarily affected the supporting structures of the leaflets, then a leaflet-directed procedure could produce dramatic results. That is what the COAPT trial investigators reported.
Most amazingly, the solution to the puzzle was predicted by Richard Gorlin nearly 70 years ago when he first proposed his hydraulic formula for the mitral valve. (Disclosure: I was one of Gorlin’s cardiology fellows in the 1970s).
Once the pieces were correctly identified and played in the right order, the harmony became obvious. But viewed alone, each trial yielded a half-truth.
Is a tie-breaker trial still needed? If you saw the harmony, you would be amazed by the persuasiveness of the reconciliation. Clearly, more trials are needed, but they should be positioned to provide additional pieces to and refinements of the puzzle, not as “tie-breakers.”
Looking at and dissecting one trial out of the totality of evidence is akin to listening to only one part of the opening five-part harmony at the start of Bohemian Rhapsody. You might do that, but you wouldn’t be impressed, inspired, or feel whole.
Our brains are uniquely wired to integrate information. It would be a shame if we fell short of that potential.
Packer recently consulted for Actavis, Akcea, Amgen, AstraZeneca, Boehringer Ingelheim, Cardiorentis, Daiichi Sankyo, Gilead, J&J, Novo Nordisk, Pfizer, Sanofi, Synthetic Biologics, and Takeda. He chairs the EMPEROR Executive Committee for trials of empagliflozin for the treatment of heart failure. He was previously the co-PI of the PARADIGM-HF trial and serves on the Steering Committee of the PARAGON-HF trial, but has no financial relationship with Novartis.