Research: Fact or Fiction?

Research: Fact or Fiction?

In Episode 200 we visited with our new co-host, Carrie Nagy, Toxic Epidemiologist, and discussed some key tips for evaluating research. Carrie is an an expert in toxins and environmental health. She advises the public on how to live,
work and play in a toxin-free environment.


As we all know, research is used to hawk products, justify therapy (both legitimate and illegitimate), market and promote drugs and to give credence to all manner of things. But how credible is the research that is being cited? And what do you look for when evaluating research?

Firstly, you need to understand what type of study is being cited, here is a brief overview of common types of research studies:

Cohort Study: This study follows a group of healthy people over a long period of time to find out who gets a certain disease.

Cross-sectional study: This study takes a snapshot of people's current lifestyle and disease status.

Case-control: Studies compare lifestyle factors between 2 groups of people, those with and without a certain disease.

Clinical trials: Studies which test the effects of drugs or supplements vs. placebo. These are often cited in marketing and promoting of drugs and supplements.

Meta-research studies: These studies collect data from lots of related studies to find out statistical information about treatments or herbs and drugs. These are very popular in the alternative medicine arena and can be inconclusive because the studies they look at turn out to be poorly designed.


Secondly, it is important to know who pays for the research, who is publishing the information and what that information is being used for. Was the study funded by a drug company, for example, or an independent group such as a university, government group or foundation. The influence of the pharmaceutical companies has extended deep into academia and many believe that this is influencing study outcomes there as well.

Why does this matter? Because money has influence. Scientists, like politicians, are not immune to its effect. Dr. Mildred K. Cho, an ethicist at the University of Pennsylvania, last year reported a study in The Annals of Internal Medicine found that 98 percent, or 39 of 40, drug-company-sponsored articles published in several journals had outcomes favoring the drug of interest, whereas 79 percent, or 89 of 112, without acknowledged drug company support favored the drug of interest.

In addition, you need to follow the chain of custody of this information and see what happens to it. A new study carried out by the Institute for Evidence-Based Medicine in Germany has found that 94% of the information contained in promotional literature sent to doctors by pharmaceutical companies has absolutely no basis in scientific fact. Virtually all of the information in the promotional brochures has been distorted or exaggerated, the study revealed.

For example, as quoted in the study, medical guidelines from scientific groups are misquoted, the side effects of drugs are minimized, groups of patients are wrongly defined, study results are suppressed, treatment effects are exaggerated, risks are manipulated, and many effects of drugs are actually drawn from animal studies, not from human studies, even though the drugs are intended for human consumption.


(An interesting aside here, the current class action lawsuit against Merck, the makers of Vioxx is a perfect example of the effects of supressed study results. Vioxx is a Cox-2 inhibitor which means it interrupts the production of that enzyme. Well, it turns out that Cox-2 is not only involved in inflammation, but also cardiac function. Hence, the side effects that have led to heart problems and an enormous number of lawsuits.)

Studies of herbal supplements and alternative therapies are often no better:

The truth is, supplement makers are not obliged to do any research. A 1994 law exempts them from having to prove, as drug manufacturers do, that their products are harmless and effective.

Often relying on as few as a dozen subjects, these studies are scaled-down versions of the double-blind, placebo-controlled clinical trials required before drugs can be approved. Some are published in abbreviated form at meetings of
scientific organizations,or in obscure journals, providing a basis for marketing claims like ''clinically proven.'' Again follow where the information goes.


Thirdly, look at study design:

The "gold standard" for evaluating new medicine/therapy/treatment are randomized, double-blind clinical trials. Randomized means that the subjects are randomly assigned to either a group which receives a placebo or another group which receives the drug or intervention. Double blinded means neither the participant in the study nor the person performing the study know who is receiving what.

greenhealth_03_doubleblind.gif

Some things can't be studied this way, for example it may be unethical in some cases. Also, some types of therapy have inherent difficulties.


Take acupuncture, it is very difficult to create a placebo needle or to blind the person applying the needles. There are also some inherent difficulties with placebo itself. To learn more about this check out my next blog entry, "Placebo: Control or Just More Bad Science?"

Fourthly, how many people are involved in the study. If it is only a dozen or 2 then the findings are not very significant. Studies of complementary and alternative medicine often have relatively few subjects because it is harder to get funding for this type of research. The NIH (National Institute of Health) is helping to change this, but most alternative practitioners and schools do not have the resources to do larger, more meaningful studies.

Statistically, the more people you have the stronger the results of the findings. This is why most clinical trials are done in 3 phases. Phase I has 12-24 subjects and is used to check for safety and dosage, Phase II has 100-300 subjects and is used to check for efficacy and side effects, and Phase III has 1,000-3,000 subjects and is used to monitor reactions to long term drug use.

Lastly, look to see how credible this information is and try to compare it with other studies. This is why meta-research can be quite helpful. These studies look at groups of studies and if the study design is decent you can learn what the general consensus on the product, drug or supplement being studied is.

Also, look at who publishes the findings, if it comes from some obscure journal published under the cover of darkness then you should question its findings. Use your common sense and don't believe everything you read or see on TV. When it comes to research there is a lot of information that should be questioned out there.

Your Intrepid Host,

Marc