Evaluating Credibility of Meditation Experiments

How to evaluate the credibility of meditation experiments? What are the harms of meditation and complementary therapies?

This post suggests ways to evaluate for yourself the credibility of meditation experiments. I also present my thesis that meditators who also believe in subtle life energy (prana, chi or qi) are more likely to seek out and harm themselves by using Complementary Alternative Medicine (CAM) including acupuncture, special diets, guided imagery, tai chi, qigong, and any sort of energetic, psychic, or spiritual healing used for the treatment of specific medical conditions or disease symptoms.

Post Contents (click link to jump to section in this post)

Meditation as Complementary Alternative Medicine (CAM)
How Common are Meditation and CAM Therapies?
CAM practiced in absence of evidence
For credible evidence CAM therapies must demonstrate
Biological mechanisms: subtle energy (prana, qi or chi)?
“Active Ingredient” in Meditation?
Placebo and Meditation or CAM
Six Points of Belief Affects Effectiveness
Magical-, Spiritual-Thinking: Gateway to Meditation and CAM
How to Evaluate the Research Yourself
Six Steps of Increasing Credibility of Experiments
Flaws with Meditation Experiments and RCTs
Harms of Meditation or CAM Treatments

Listen to this blog post: Evaluating Credibility of Meditation Experiments

Meditation as Complementary Alternative Medicine

Firstly, meditation used for medical or psychological treatment is, in the medical and scientific domain, considered CAM (complementary and alternative medicine). CAM are physical, mental, chemical, or psychic interventions such as acupuncture, chiropractic and osteopathic, deep breathing, special diets, homeopathy, herbs, guided imagery, meditation, megavitamin therapy, massage, hypnosis, yoga, tai chi, qigong, and any sort of energetic, psychic, or spiritual healing used for the treatment of specific medical conditions or disease symptoms.

How Common are Meditation and CAM Therapies?

The US National Institute of Health published a 2012 and the Centers for Disease Control published a 2007 report showing most common CAM therapies, which includes meditation. Meditation has significant increases for usage as a therapy. Note the two charts indicate there are commonalities between the other 9 most common therapies as they relate to underlying beliefs in subtle life energy, prana or chi/qi (which we discuss below).

CAM practiced in absence of evidence

CAM therapies are practiced in the absence of:

1) Scientific (credible) evidence proving their effectiveness, and;

2) A plausible biological explanation for why they should work.

Meditation interventions and CAM therapies have failed to meet their burden of proof as an effective treatment for medical and psychological intervention. Why?

For credible evidence CAM therapies must demonstrate:

1) A biological basis which is plausible and credible;

2) A provision the treatment could be proved to be ineffective. Also called falsifiability.

There is no plausible biological explanation that meditation techniques by themselves are more effective as a treatment than ordinary relaxation or placebo.

One explanation could be the patient’s belief is largely responsible for any benefits felt or experienced from meditation treatment. In other words, the placebo effect is what creates any significant felt results from the treatment. In other words, any effects from treatment result from the beliefs in the person’s mind or imagination. We will discuss placebo further below. First though, let’s return to whether there’s any plausible, credible biological basis for the effectiveness of meditation techniques.

Biological mechanisms: subtle energy (prana, qi or chi)?

Most meditation techniques are derived from Eastern Buddhist or Hindu traditions. These Eastern traditions posit there is some kind of subtle life energy (prana, qi, or chi) within and without the human body. Proponents of the subtle life energy (prana, qi, or chi) hypothesis say practice of meditation techniques can unblock or improve the flow of subtle life energy within the physical body. Thereby promoting health, healing both physically, psychologically, and spiritually. Supposedly there is a subtle (scientifically undetectable) network of energy centers (nadis or chakras) within the human body.

The biological basis for such a highly speculative invisible energy (prana, qi, or life force) and a subtle energy body is implausible, as yet undetectable, and has no credibility in modern medicine. Not only is there no biological mechanism or evidence of the energy but meditation techniques themselves don’t appear to be the “active ingredient”.

“Active Ingredient” in Meditation?

If the results of meditation or any CAM treatments were greater than a placebo the treatments would be accepted as medicine, that is evidence-based medicine. To-date, no one has come up with a credible placebo to demonstrate that meditation is the “active ingredient” which gives the results or benefits. Nor has anyone yet devised any credible, replicable experiments to demonstrate that meditation is more effective than ordinary relaxation, exercise, or cognitive psychotherapy.

Meditation studies presented in the mainstream media or news are often headlined as a viable or promising complementary alternative medical (CAM) or psychological treatment. Yet, the facts are meditation and CAM have not been demonstrated to be more effective than a sham treatments. Let’s now discuss the effects of placebo in meditation and CAM.

Placebo and Meditation or CAM

To reiterate, belief in and practices in meditation and CAM persists even after:

1) The scientific evidence shows no effectiveness (greater than a placebo or sham treatment) and

2) Their biological basis is not plausible and has been discredited. [1]

Our expectations that an intervention or treatment (such as meditation) can help sometimes gives us actual benefits. The placebo effect results when a fake treatment–an inactive substance like a sugar pill or meditation technique–can sometimes improve a patient’s condition simply because the person has the expectation the treatment will be helpful.[2]

In Snake Oil Science: The Truth About Complementary and Alternative Medicine, R. Barker Bausell, a former Research Director of the National Institute for Health-funded Complementary Medicine Program, points out that,

If a completely inactive pill, ointment, or procedure (in other words, a placebo), accompanied by the expectation of effectiveness, can result in pain relief, then surely any therapy–no matter how bizarre–that we consider credible enough to seek out (and pay for) can also result in pain relief [or give us feelings of control and well-being], compliments of the placebo effect.[3]

The benefits some people feel from meditation practice could largely depend on practitioner’s belief and may be temporary. What beliefs make it possible to feel psychological an physical benefits?

Six Points of Belief Affects Effectiveness

There was a time when I fully believed that meditation techniques, as well as many CAM treatments, helped me relieve pain or gain mental or spiritual control. Why? Because I:

1. Wanted to believe;
2. Needed to believe;
3. Was certain these beliefs fit my worldview and religious principles;
4. Had many acquaintances who also shared these beliefs;
5. Knew persons I respected who advocated these beliefs;
6. Interpreted personal experiences as evidence for the effectiveness of these beliefs.[4]

These six points fit neatly into a worldview I adopted about Eastern mysticism, yoga philosophy, and magical- aka “spiritual”-thinking.

Magical-, Spiritual-Thinking: Gateway to Meditation and CAM

What I discovered was my beliefs in yoga meditation–containing implausible, subtle energies such as prana, qi, and chakras–linked directly to why I sought out and paid for CAM: my acupuncture treatment helped me because the needles are stuck in supposed energy meridians in the body. In reflexology (massaging of feet or hands) the tender spots on my feet mapped to energy blockages in the body. Same with psychic healing, mind over matter, and positive affirmations. All these ‘treatments” were rooted in the same magical-, spiritual-thinking that lead me to believe in the effectiveness in yoga meditation practice.

How to Evaluate the Research Yourself

Why take someone else’s word on these research studies into meditation or CAM? You could evaluate the evidence yourself. Granted that you also are open to learning from experts, like Bausell and his excellent book Snake Oil Science: The Truth About Complementary and Alternative Medicine. When evaluating scientific papers Bausell recommends focusing on:

Methods and procedure section.

Where items such as blinding, randomization, sample size, and dropout/attrition are discussed. We discuss these items further below.

Results section.

Where the authors mention what was or was not statistically significant.
Researchers who conduct systematic reviews or meta-studies (studies of numerous studies) disregard investigator conclusions, writes Bausell in Snake Oil Science, which is one reason why systematic reviews are considered more effective and reliable than individual studies.

Bausell recommends ignoring the investigators’ discussions and conclusions sections because this is where authors may try to put a positive spin on their findings. An interpretation of one experiment is not enough. Researchers want to review the body of evidence (as in systematic review noted above) in high-quality studies to see if the findings have been independently validated and replicated.[5]

Six Steps of Increasing Credibility of Experiments

To evaluate the credibility of experiments it is necessary to understand the methods and procedures are more important than the conclusion[6]. In Snake Oil Science: The Truth About Complementary and Alternative Medicine, Bausell gives six steps for evaluating the credibility of CAM experiments (which includes meditation treatments).

In order of increasing credibility the six steps for evaluating experiments are:

1. Randomized[7] controlled trials (RCTs) are more credible than nonrandomized trials.
2. Large trials, with at least 50 patients per group (preferably more than 100) are more credible than small trials.
3. Large, double-blinded[8] RCTs using placebo groups are more credible than RCTs not using placebos.
4. Large double-blinded randomized, placebo-controlled trials with 20% or less drop-out rates are more credible than those with higher attrition rates (patients who drop-out of the trial before it is finished).
5. Large double-blinded randomized, placebo-controlled trials with 20% or less drop-out rates published in high-quality journals are more credible.
6. Large double-blinded randomized, placebo-controlled trials with 20% or less drop-out rates published in high-quality journals that have been independently validated and replicated by other investigators are more credible.

The gold standard for clinical studies or medical experiments are RCTs. To conduct RCTs you’d select a group of individuals from a wide population and randomly assign them to either a meditation or a control group.[10]

Flaws with Meditation Experiments and RCTs

Although RCTs are considered the gold standard for clinical studies, including meditation experiments, they may be flawed by:

Bias to believe – Often participants (patients) in the experiment are particularly interested in meditation. They join the study because of their interest in mediation. They join due to their belief that meditation can benefit them–which enhances the placebo effect.

Controls – The biggest problem with meditation studies is finding the right kind of activity for the control group. That is the placebo-controlled group that you will compare with meditation treatment group. Most researchers say you can’t and instead of placebo use active control groups. Participants in active control groups partake (instead of meditation treatment) in controlled relaxation, cognitive therapy, or exercise.[9]

Blinding – What can be done to “blind” participants as to which group, meditation or (placebo) control, they are in? The challenge is that researchers also are not blinded and know which group they are treating. When researchers and participants don’t know which group they are in the study is higher quality and is said to be double-blinded (when both study subjects and clinicians are ignorant which group they are working with). High quality studies “blind” both researchers and participants.

Harms of Meditation or CAM Treatments

The harm of using meditation or CAM to treat physical or psychological problems is the delay of credible diagnosis and treatment. Earlier diagnosis and treatment could have saved many people from further harm. Meditators often also believe in energetic, psychic, or spiritual healing, which are often a component of belief in CAM treatments (as we noted above with the 10 Most Common CAM therapies in the NIH and CDC charts above).

When belief or disbelief is the “active ingredient” in a treatment then we should not claim that a therapy itself is effective. It’s the belief that is effective, not the treatment. Unfortunately, people who meditate as treatment probably also believe in or use CAM therapies. Use of CAM or meditation therapies can delay diagnosis and treatment for serious illnesses, including cancer.

I’m not implying all meditators are the same; they all use meditation as prophylactic (treatment to prevent disease). Obviously, many meditators use meditation for other reasons. A thesis I put forward in this post is many meditators, especially those who believe in subtle life energy, prana, chi or qi, are more likely to seek out and be harmed by CAM treatments. There’s no credible scientific evidence that CAM or meditation therapies offer anything more than temporary relief of stress. There is no credible evidence that meditation therapies are more effective the relaxation, sleep, drugs or placebo (fake or sham treatment).

Question for readers: What’s your experience or observation? Do meditation beliefs in prana, chi or qi lead to increased use CAM therapies?

Special thanks to Scott D. Jacobsen, Editor at Conatus News, and Founder of In-Sight: Independent Interview-Based Journal and In-Sight Publishing for his editorial assistance and comments prior to publication of this post. Without Scott’s help and encouragement this post would not be published.

Notes

1 Snake Oil Science: The Truth About Complementary and Alternative Medicine,  R. Barker Bausell. (2007) Oxford University Press. Ch. Rise of Complementary and Alternative Therapies: Definitions of CAM p 21

2 Placebo: Latin “to please”, is any irrelevent procedure or inert substance that produces a genuine psychological or physiological response. The placebo effect, or placebo response, is a phenomenon in which a placebo–a fake treatment, an inactive substance like sugar, distilled water, or saline solution–may improve a patient’s condition simply because the person expects it will be helpful. Bausell p 30.

3 Bausell p 256. On p 292 we read “Neither a placebo nor a CAM therapy is going to cure anything that will not resolve itself or that the body does not have the capacity to deal with. Both a placebo and a CAM therapy that appeals to you, however, are equally capable of relieving pain if it isn’t too severe.” Using meditation techniques also can give someone a sense of pain relief or psychological control.

4 Adapted from Bausell, Snake Oil Science: The Truth About Complementary and Alternative Medicine.

5 Bausell. Snake Oil Science: The Truth About Complementary and Alternative Medicine, p182

6 “In science, methodology is the detailed process used to arrive at a scientific conclusion. As you can imagine, the more the methodology is flawed, the more likely researchers are to come to an inaccurate conclusion.”– Bo Bennett. For elaboration about why method and procedure is more important than conclusion readers are encouraged to listen to or read Dr. Bennett’s Methodology Over Conclusion https://www.thedrboshow.com/tools/bg/Bo/TheDrBoShow/B7RvXyZw/Methodology-Over-Conclusion.

7 “A randomized controlled trial (or randomized control trial; RCT) is a type of scientific (often medical) experiment, where the people being studied are randomly allocated one or other of the different treatments under study. The RCT is often considered the gold standard for a clinical trial”. Retrieved from Wikipedia on Mar. 20, 2016, from https://en.wikipedia.org/wiki/Randomized_controlled_trial

8 Double-blinded study: A medical study in which both the subjects participating and the researchers are unaware of whether the actual or a placebo (sham/fake) treatment or procedure has been given. Retrieved from MedicineNet.com on Mar. 20, 2016, from http://www.medicinenet.com/script/main/art.asp?articlekey=11177

9 The Buddha Pill: Can Meditation Change You? Miguel Farias and Catherine Wickholm. Watkins Publishing (2015). p 56. This easy to read and excellent book goes in-depth into the history and latest meditation research, it’s flaws, and promises.

10 Additional Resources for Evaluating the Credibility of Meditation or CAM Experiments

Quality Assessment of Controlled Intervention Studies checklist, National Institute of Health
http://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools/rct

Cochrane Handbook for Systematic Reviews of Interventions
http://handbook.cochrane.org/chapter_8/8_4_introduction_to_sources_of_bias_in_clinical_trials.htm

Medical Nonsense, Interview with Dr. Angie Feazel Mattke. Skepticality Podcast, Episode 278 (2016) https://www.skepticality.com/medical-nonsense/

Science-Based Medicine: Exploring issues and controversies in the relationship between science and medicine. Search results for “meditation”
https://www.sciencebasedmedicine.org/?s=meditation