Press "Enter" to skip to content

How to Massage Data to Feel Like a Winner

One of my good friends is a die-hard Oakland Raiders fan. Those of you who follow pro football know that Oakland has fallen on hard times recently. They went from being one of the best teams in the league two years ago to one of the worst teams in 2018 with a dismal 4-12 record. As a result, my friend has had to suffer the taunts of fans of better teams — like me. He has become despondent.

But it doesn’t have to be this way! The Raiders can quickly and easily turn their season around by using the tried-and-true techniques of medical research. If a pharmaceutical company did 16 clinical trials of their new potential blockbuster, Drug X, they would never let a 4-12 outcome get them down. When published, I guarantee those trial results would look a lot better than 4-12. The Oakland Raiders can use the same techniques to improve their season record.

Change the primary endpoint

Before a medical study begins, the researchers must identify exactly what it is that they are studying. This is called the “primary endpoint.” For example, the researchers studying Drug X could initially decide that their primary endpoint is whether Drug X reduces mortality over five years. What happens, though, if the study shows that Drug X did not, in fact, reduce mortality? What now? Well, often in that case, the researchers will scrutinize the study’s data to find out if Drug X showed some other benefit that they were not initially looking for. Let’s say that patients taking Drug X had fewer DVTs. This finding may have resulted purely by chance, but what the heck! They could publish a paper that says that Drug X reduces DVTs without, of course, mentioning that this was not the original primary endpoint of their study. It turns out that this practice is common in published research papers and is called “outcome switching.” How common? Well, according to this recent survey, outcome switching occurred in over 50% of the papers studied.

In a football game, the primary outcome is, of course, the final score. The Raiders lost 12 games in the 2018 season using the primary outcome of final score. But if we look closely at each of these 12 games, we might be able to find, by chance, another potential outcome we could switch to. Take, for example, when Oakland played the Indianapolis Colts on October 28th. The Raiders lost that game 42-28. But if we were to switch the outcome to the score at the end of three quarters, the Raiders win 28-21! We’ll publish that as a victory without saying that we changed the primary outcome. Similarly, in their second game of the season, the Raiders lost to the Denver Broncos 28-20. But if we change the outcome to the score at halftime, we can publish this as a win, 12-0! We can do the same thing for their first game against the Los Angeles Rams.

After changing these primary outcomes, Oakland’s record has improved to 7-9. We’re on our way!

Use composite outcomes

If a pharmaceutical researcher isn’t sure if Drug X will get positive results in any particular primary endpoint — like death, for example — they may instead add multiple other endpoints, hoping to get a hit on at least one. The additional endpoints could include heart attacks, strokes, or anything else they can think of, like DVTs or even inpatient hospital days. If any one of the many composite outcomes comes up positive, then the whole study can be published as positive. Of course, a DVT is much less important than, say, death, but since both are listed as equals in the composite endpoint, you would have to really read the fine print to find out if the “hit” was death or DVTs. Composite endpoints are also common in the medical literature. However, according to this article in the BMJ, the practice of using composite endpoints “will leave many readers confused, often with an exaggerated perception of how well interventions work.”

Composite endpoints turn out to be an immensely useful tool in re-evaluating the Oakland Raiders 2018 season! My composite endpoints for the Oakland Raiders games are these: final score, total yards, first downs, and time of possession. I applied this composite endpoint to each of the remaining Oakland losses. Take, for example, the third game of the season against the Miami Dolphins. Oakland lost that game 28-20, but Oakland had more total yards, more first downs, and a longer time of possession than Miami. Clearly, we can publish this as a victory for Oakland using our composite endpoints. Applying our composite endpoints, we can similarly change five other losses to victories.

Oakland’s record now is 13-3.

Simply don’t publish the negative results

This has long been the easiest and best way to bury a negative trial. Simply don’t publish it! Negative studies in the medical literature have long been much less likely to be published than positive studies. This “publication bias” has been such a big problem in pharmaceutical research that in 2004, many medical journals started requiring studies to be pre-registered in a clinical trial database. This ensured that negative studies could be tracked even if they were not published. So, is this requirement working? Not so much. According to this report, publication bias is still “alive and well.”

Publication bias can certainly help the Oakland Raiders. Their revised 2018 season still includes three losses. And all three were embarrassments where the Raiders got their butts kicked. Take the October 14th game against the Seattle Seahawks, for example. Not only was the final score a lopsided 27-3, but the Seahawks had far more total yardage, more first downs, more everything. Let’s forget that debacle by simply not publishing it! Let’s not publish the other two losses, either.

Oakland’s record now is 13-0. However, we’re still not done. The season is 16 games, not 13. We still have to find three other positive outcomes …

Publish a positive study more than once

If a medical researcher has a positive study, it can be tempting to publish the results in more than one medical journal. That way, the researcher gets two citations in their resume for the price of one! There are two ways to do this. The first is to submit the same data to multiple journals without telling them you have done so. “Duplicate publication” like this is a form of fraud, but, as this medical journal editor says, “Duplicate publication is more common than you think.”

Another way to get a study published multiple times is to publish only part of the study’s data and then later publish the rest of the data in a second article. For example, in our study of Drug X, we could publish the data showing the effect of the drug on mortality first and then later publish the data showing the effect of Drug X on DVTs. If the study is large enough and if the researchers slice the data thin enough, they can get many publications out of a single drug trial.

Let’s apply this principle to the Oakland Raiders. Their most impressive victory of the entire season was when they upset a very good Pittsburgh Steelers team — on the road, no less — on December 9th. We certainly want to publish that twice! Let’s also duplicate-publish the Raiders’ victories over the Denver Broncos and the Cleveland Browns.

Well, we’re finished. The Oakland Raiders’ final record after applying the principles of medical research is an undefeated 16-0. My friend can break out the champagne and let the celebration begin! And all of you other long-suffering Oakland Raiders fans out there — you’re welcome.

Jeffrey E. Keller, MD, FACEP, is a board-certified emergency physician with 25 years of experience before moving full time into his “true calling” of correctional medicine. He now works exclusively in jails and prisons, and blogs about correctional medicine at JailMedicine.com.

2019-03-07T13:30:00-0500

Source: MedicalNewsToday.com