A statistician, an assistant professor of research in Health Policies and Administration at Pennsylvania State University (Penn State) in the USA, clarifies the concept of 'controlling' variables in scientific studies—a term frequently used in news reports to enhance the credibility of research.
Recently, The Washington Post published articles about studies involving GLP-1-based medications. In one case, it was noted that women using GLP-1 lost more weight than those in the control group. In another study, the same outlet reported that researchers controlled for factors such as age, sex, race, and smoking, discovering that using these medications increased the risk of skeletal disorders.
The expert emphasizes that understanding what constitutes control is fundamental to assessing the degree of evidence support for a study's conclusions and to judging the media coverage of new discoveries. The meaning of control varies depending on the study design, which generally divides into experimental or observational.
In an experimental study, researchers administer a treatment or medication to some participants, while others form the control group, which does not receive the intervention. In the example of the study on weight loss and hormonal disorders, researchers compared the results between those who received GLP-1 and those who did not, meaning they directly influenced the data generation.
In observational studies, researchers analyze pre-existing data, such as electronic health records, to identify patterns. They use statistical tools to verify how factors like age or sex may have impacted the results, regardless of who received treatment. This process aims to ensure that the link between treatment and outcome is not influenced by known variables that affect both.
In the study on bone injuries and GLP-1, controlling for age, sex, race, and smoking involved comparing injuries among individuals belonging to each category, checking differences between users and non-users of the medication. Without this control, it might seem that GLP-1 causes more injuries simply because many users also smoke, with smoking being an independent risk factor.
In an experimental trial of a new drug, scientists randomly assign the treatment. Participants without the medication usually receive a placebo. Furthermore, if there are known factors that can affect the outcome, such as disease severity or other medications in use, scientists can randomize the treatment within these groups, a procedure called stratification. This prevents researchers from disproportionately assigning participants with specific characteristics to the treatment group or the placebo.
For instance, an experimental study from 2021 investigated weight loss in type 2 diabetics who used GLP-1. Researchers ensured that the treatment and control groups were randomly selected among people with different initial glycemic levels and who were taking different diabetes medications, minimizing biases.
Due to logistical and financial challenges, researchers often resort to observational studies, also called real-world evidence studies, using existing records without the possibility of randomization. In these cases, statistical tools are used to account for factors that may have affected treatments and outcomes.
In an observational study on GLP-1, for example, researchers analyzed medical record databases to estimate the weight loss of GLP-1 users compared to those using other medications for type 2 diabetes.
In observational studies, neglecting crucial variables can undermine the credibility of findings. A study might claim benefits of a nutritional supplement without controlling for initial health differences between users and non-users. In this scenario, healthier individuals might have chosen to take the supplement, creating a false perception of benefit.
Many of these conclusions are not reproducible in experimental studies that use randomization to control these biases. However, trying to control all possible variables is also problematic, because if they are not related to the variables of interest, increasing controls makes mathematical estimates less accurate. Controlling confounding variables can even distort the effect the study seeks to measure.
The expert advises that when analyzing the results of observational studies, one should evaluate whether the study leans towards too few controls or too many variables. He made an online application available to help compare observational and experimental analyses with simulated data, reinforcing that one does not need to be a scientist to examine such research, but it is vital to recognize that simple terms like 'controlled' have very distinct meanings.