Will people who live according to the “latest research” live healthier lives?

With the accessibility of information channels today, it has become so easy to obtain the latest scientific research progress through the media that many ordinary people will intercept the conclusions of medical research from mass media or professional journals and use them to guide their health behavior choices and even treatment plans. This may seem avant-garde, but it is actually a very irrational behavior.

Written by | Big Cat, Wang Chenguang

Every day, scientific journals around the world publish a large number of new advances and discoveries in life science and medical research, some of which involve health, medical treatment and drug development. These research results are usually the source of news reports in the health field; in the era of self-media, this information will be quickly delivered to the public through various channels, and many people who care about their own and their family's health often look for information that is beneficial to maintaining health or treating diseases from this information.

This shows two problems: first, the news media are unprofessional when disseminating topics such as medicine, health and lifestyle. Some immature medical advances are deliberately exaggerated, and even research results at the cellular level and in model animals such as mice are directly copied to the human body by half-knowledgeable media people; on the other hand, influenced by these "second-hand" media, the public who lack judgment directly uses these latest studies to "guide" their own health decisions, when in fact these research results are probably far from being concluded.

Media communication that puts the cart before the horse

A few years ago, JAMA magazine published an article by Dr. Ioannidis, which systematically analyzed those research papers on lifestyle that were included in Altmetic and received high attention from the media, and came to the conclusion that some health issues and lifestyles that deserve attention were intentionally or unintentionally ignored or neglected, while those insignificant details and speculative conclusions that lacked expert consensus were wantonly exaggerated.

Most scientific research, no matter how sophisticated or resource-intensive, rarely has an immediate impact on everyday life, but studies on lifestyles feel so relevant and so actionable that it’s understandable that issues around lifestyle choices have figured prominently in public discourse and have received such high attention.

Altmetric counts the attention that scientific papers receive in the news and social media, gives a comprehensive score, and lists the 100 highest-scoring articles each year. The included papers include a wide variety of research topics, and 49 of the 200 papers that attracted the most attention in 2017 and 2018 were related to people's lifestyles.

Dr. Ioannidis's analysis began by listing the lifestyle habits that seriously affect human health, with smoking being the top bad habit. Smoking will kill 1 billion people in the 21st century, and it is a behavior that can be changed. However, among the 49 lifestyle-related papers that received the most news and social media coverage, none specifically addressed tobacco abuse. Most of the high-scoring papers were related to nutrition, diet, or obesity (29 out of 49).

Obesity is also a major global health problem, but of the 29 papers, only three directly addressed obesity, while the other 26 discussed things like specific nutrients, foods, supplements, or fad diets. Only two of the 29 lifestyle-related papers reported that the targets studied had no effect on health. The papers that attracted the most attention tended to study factors that had little or no effect on health risks. For example, three widely watched papers studied whether coffee is good for longevity.

Even when papers reported on factors with high health risks, such as drinking (5 papers), the focus was usually not on the large and clear known risks of heavy drinking, but on the less certain risks of drinking small amounts of alcohol. In contrast, papers on what and how to eat accounted for a large proportion, such as eating certain nuts can make people live longer, and which dietary habits are related to which diseases.

The conclusion of this analysis is very clear: within the field of scientific research related to lifestyle, more serious health issues have not received the corresponding attention in media communication.

The authors argue that correctly communicating scientific information to the public and the media is both an opportunity and a challenge, and is closely related to public health issues and lifestyle choices. If the information being communicated is relevant to major health issues such as obesity or hypertension, and the information is true, the benefits will be considerable. On the contrary, if minor and inconclusive topics occupy too much of the public's attention, and less attention is paid to more critical, more definitive issues and their solutions, confusion will arise. Such confusion and even reversal of priorities may obscure key knowledge and scientific answers.

On the other hand, much of the information disseminated by the media is actually not scientifically agreed upon or even false. Sometimes two or more highly-watched papers on the same issue come to completely different or even opposite conclusions, and the effects of dietary fat and carbohydrates on health are a typical example. Among the 49 articles with the highest Altmetric scores, most are observational, non-randomized studies, or reviews based on such evidence.

Paradoxically, observational studies are more likely to make the news than rigorous randomized studies that don’t produce significant results (which may yield more plausible results but are less dramatic). Observational studies often report associations between one or more factors and health conditions, not causation, or that manipulating those factors will improve our health. This makes it hard to know how many widely touted studies actually produce useless or even wrong conclusions.

Due to subjective interpretation, the media often exaggerates some trivial risk or benefit factors, such as living near a traffic artery can cause dementia, eating green leafy vegetables can reduce the risk of dementia, and mobile phone radiation has an impact on health. Even if such conclusions can establish a correlation or even a certain causal relationship, the role of these factors is often too small to attract widespread public attention. This unreliable information can also be easily exploited by some businesses and other non-scientific groups.

Dr. Ioannidis made suggestions to the media: In order to maximize the benefits of public health actions, scientists and the media need to carefully select topics for dissemination, and these topics should have definitive conclusions; both scientific journals and the media should strategically select public health topics that need to be disseminated, give priority to interventions that can address major health risks, and report knowledge with clear expert consensus; biased observational studies should be published less frequently in popular magazines, and these reports should be placed in professional journals, with their limitations properly described in the abstract; news media should significantly reduce coverage of such studies, which is the social responsibility that the media and media professionals should have, and it is also the professional ethics that should be followed.

Dr. Ioannidis also suggested that the media should avoid publishing biased information from industries or other groups that have a significant negative impact on health. For example, businesses that do not acknowledge the health hazards of smoking and drinking (or even promote their health benefits) should be banned. When reporting on major health issues, not only scientific journals but also public media should declare whether the authors have conflicts of interest.

Understanding the Basics of Medical Papers

Based on the above understanding, it is easy for us to understand how outrageous it is for the public to directly use these latest studies that have been re-interpreted by the mass media to "guide" their own health decisions.

Even if you skip the "second-hand" and go directly to the original paper, it is unreliable, because it is very difficult for ordinary people who do not have professional knowledge to understand the research content described in a scientific paper and make correct judgments on the results. Based on this, the author does not recommend that ordinary people obtain the latest health and medical knowledge from newly published scientific papers. Even if a certain study involves personal interests (for example, a cancer patient happens to discover the progress of the treatment of his or her cancer), you should treat it with caution and consider consulting relatives and friends with professional backgrounds.

If a reader has a special interest in medicine and really wants to fully understand the latest medical progress of a certain disease, it is recommended to start by understanding scientific papers. Only by fully understanding the basic knowledge of a field and understanding those professional terms can we correctly understand the content presented in a scientific paper and make a judgment on whether these contents are helpful for health decisions.

According to the recommendations of the NIH (National Institutes of Health), understanding the following questions can help you understand the weight of a scientific research paper and the credibility of its conclusions.

What is the purpose of the research? What question does it want to answer?

Basic research in life sciences and medicine aims to understand basic life phenomena, physiology and disease mechanisms, while the purpose of translational research is to find methods for diagnosing and treating diseases in order to provide a basis for human clinical trials. Both basic and translational research can be conducted in the laboratory or through clinical trials.

Clinical trials are designed to test the safety and effectiveness of a diagnostic or therapeutic method, including exploratory and in-depth studies. Preliminary, exploratory or pilot studies provide evidence for the safety and effectiveness of a treatment and help determine whether a larger and more systematically designed clinical trial is warranted.

Systematically designed and carefully planned large clinical trials provide more reliable evidence of whether a diagnosis or treatment is effective and safe. This process is complex to design and implement, takes a long time and is expensive, and is usually only conducted when smaller preliminary studies show that the treatment or drug is likely to benefit patients.

If it involves clinical research or epidemiological survey, how large is the size of the participating population?

Studies of large numbers of people usually produce more reliable conclusions than studies with only a small number of participants. Larger studies can increase the accuracy of the results and reduce the chance of the results. But paradoxically, the larger the scale, the longer it takes and the more expensive it is. Statistics can help estimate the size of participants needed for clinical research.

How is the clinical trial protocol designed? What measures are taken to reduce bias in the trial?

In a controlled clinical trial, researchers compare the effects of different treatment groups, choosing participants who are as identical as possible in all other respects. For example, the results of one group of participants who receive a new treatment (the "experimental group") can be compared with the results of another group who receive a standard treatment (the "control group"), with the control group providing a standard for the experimental group to measure against.

The design of the control group is also very particular in professional clinical trials. It can be a placebo with no medicinal effect, or it can be given the current clinical standard treatment. In actual practice, the latter is more common. In a simple and ideal situation, participants are randomly assigned to one of the research groups. This helps to ensure that the two groups of people are as similar as possible except for the different treatment methods. In most cases, the treatment of the control group and the experimental group is carried out in a "double-blind" manner, that is, the researchers providing treatment and the volunteers participating in the clinical trial do not know whether they are receiving the test drug or the control drug, and the data is mastered and counted by a third party.

This is because it is often very difficult to avoid bias in clinical trials. If patients know what treatment they have received, or researchers know which treatment patients have received, it may affect the judgment of the treatment's effect.

Is there a potential conflict of interest?

When reviewing the results of any study, it is important to look for potential conflicts of interest and other sources of bias. Find out who funded the study, whether the researchers have a vested interest in the study, and whether there is similar evidence from other independent sources. Much of this information can be found in the article itself, as most medical journals require authors to provide a statement of conflicts of interest and information about any relevant financial relationships.

How do the findings of this study compare and contrast with previous studies?

The strongest evidence about whether a treatment is effective and safe comes from the results of multiple studies by different researchers. A single study rarely provides final, definitive conclusions, and repeated studies are needed, including those that use the same methods but different researchers and participants. In addition, independent assessments that analyze the results of multiple studies and rigorously review the quality of the data are valuable; these assessments are called systematic reviews and meta-analyses.

What does it mean when a study result is described as having a statistically significant difference vs a clinically meaningful difference?

"Statistically significant" means that the differences between the study groups were probably not due to chance. "Clinically significant" is a measure of the effect (treatment effect) observed in the study. For example, a study may find a statistically significant difference between two treatment groups, but the differences may not be large enough to significantly affect the clinical treatment of patients, so they are not considered to be clinically important for the treatment of patients.

There is an old Chinese saying that a long illness makes a doctor. It is completely understandable that patients and their families would turn to the latest research progress in treatment when they are suffering from illness, especially when there is no mature treatment plan for the disease at hand. However, in most cases, we still cannot encourage people who do not have professional knowledge to seek treatment from the latest research results, especially to change or abandon existing treatment plans.

References

[1] Neglecting Major Health Problems and Broadcasting Minor, Uncertain Issues in Lifestyle Science. October 18, 2019. doi:10.1001/jama.2019.17576"

[2] Ioannidis JPA. Neglecting Major Health Problems and Broadcasting Minor, Uncertain Issues in Lifestyle Science. JAMA. 2019 Dec 3;322(21):2069-2070.

[3] Ranganathan P, Pramesh CS, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res. 2015;6(3):169-170.

[4] https://www.nih.gov/about-nih/what-we-do/science-health-public-trust/checklist-communicating-science-health-research-public

This article is supported by the Science Popularization China Starry Sky Project

Produced by: China Association for Science and Technology Department of Science Popularization

Producer: China Science and Technology Press Co., Ltd., Beijing Zhongke Xinghe Culture Media Co., Ltd.

Special Tips

1. Go to the "Featured Column" at the bottom of the menu of the "Fanpu" WeChat public account to read a series of popular science articles on different topics.

2. Fanpu provides a function to search articles by month. Follow the official account and reply with the four-digit year + month, such as "1903", to get the article index for March 2019, and so on.

Copyright statement: Personal forwarding is welcome. Any form of media or organization is not allowed to reprint or excerpt without authorization. For reprint authorization, please contact the backstage of the "Fanpu" WeChat public account.