Are you outcome or results driven?
Do you know what being “evidence-based” means as a fitness professional?
Do you know how to think scientifically? Can you balance information and action?
I ask myself those questions, and they are tough ones. Because it’s work. It’s work to stay on top of “evidence”.
Jamie Hale sent me his new book to read, In Evidence We Trust, and I’ll admit it took me a damn long time to read it and get around to writing this review. If I write a review of something I thoroughly read it, and this wasn’t an easy read, but it was worth it. If you want to find out more about Jamie and his qualifications click here.
*All quotes are from the book unless otherwise stated.
In short, he is a researcher, coach, consultant and author. Here is his “mission statement”:
“The key message with my work is don’t be afraid to question authority. The only true authority in science is science itself. Approach fitness and nutrition and everyday life with an open mind and realize the majority of information you have probably been exposed to is misleading and in some cases deceitful. Keep up to date with the current research. Don’t be afraid to be skeptical. Never judge an individuals exercise, nutrition and knowledge of science by their degrees, certificates, physique, or athletic ability, but by their passion, ability to explain and willingness to provide evidence for their beliefs. Not many of the so-called fitness experts are willing to debate their statements when challenged. If you are not willing to debate your statements with formidable opponents you shouldn’t be making those statements (within reason).”
The gist of In Evidence We Trust was, “How to think scientifically/critically, what is science, how to judge evidence and how to understand research methods, statistics and the need for rationality, especially in fitness!”
There is contributions from different coaches, a thorough breakdown of how to read research and even quizzes at the end to do.
If I sold you on that sentence alone, you don’t have to keep reading. Click here to purchase (I am not an affiliate).
Why this interests me:
Lately I’ve heard a lot of talk about the “evidence” crowd. Or those trainers and coaches who preach “science” and don’t actually know much about what they are preaching. Can anyone be evidence-based? Sure. Evidence is always there to see. What makes us not see it? What makes us dismiss some over others? How do we judge the quality of evidence and why does it matter?
I’m not the expert on this fyi. I am learning from those who make it their area of expertise, and want to help pass it on. Jamie says,
“We are constantly exposed to Newspapers, TV, so-called experts and other sources of information that tell use what is right and wrong (blogs, e-books, text books, professors, other coaches, “experts” etc). The lack of emphasis on critical thinking leads to various problems in the decision making process. These problems make it difficult to distinguish fact from fiction.”
Does that sound familiar?
Someone told me this once…the mark of a good scientist is someone who is always willing to look at the evidence. AKA they are never married to their “evidence”. Being evidenced-based is like the opposite of getting married to your beliefs. It’s the commitment to never be too committed. To always be willing to examine what’s going on, even if you are in a very cozy and committed relationship with your methods and ideals.
I think a lot of us forget to think about thinking as work. To think about our own thinking is work. As much work as updating your client’s programs or evaluating their progress. To think critically, and be evidence-based is a lot of work. And to stay that way, is work as well.
But before I delve in further, let me tell you a little story.
Something a lot of people don’t know about me was that I was raised in a strict religious cult. I grew up in a commune, quite isolated from regular society. My childhood was quite sheltered in certain contexts, and I was raised learning obedience, not questioning my elders, having respect for elders, believing the Bible unquestioningly and “believing without seeing” (God, heaven, the devil etc). We were taught that everything we did in this world was for our future heavenly life, not this life on earth. I was taught strongly what to believe and how to look at the world. Certain things were unequivocally wrong, and there was no debate about it.
Fear, respect and obedience to someone who did not exist was the rule. You could not question god. And there was attitudes, actions and trains of thought that were not allowed. To question was a sin. Unless the question was something like “How can I do more of god’s will?” or something faith-ish like that. Of course who god chose to speak through meant that instructions and beliefs changed. Only “we” (our religious leaders) had the exclusive on what god really meant (sound familiar?).
For instance, here are some rules and beliefs I grew up under. They will sound hilarious, but someone-someone-religious-leader decided that they were rules at the time.
1. Looking in the mirror was prideful. You should not want to spend time looking at yourself in the mirror. You should in fact avoid it.
2. Prayer was the best way to control any situation. Every other sentence was “Let’s pray” (ok, thats a bit exaggerated…..every other other sentence).
3. God was building a mansion in heaven for you, and how nice the mansion would be depended on how well you obeyed the rules…oops I mean God.
4. People “out there” would contaminate you. They would try to weaken your faith, and you had to guard against them by limiting contact with them, and with TV, books, concerts, music etc.
5. White sugar and white bread rot your teeth. Literally. Most movies are evil influences (well some truth to that), so we watched Pollyanna, The Sound of Music (we couldn’t watch the fun part where the Nazi’s chase them though) and the Ten Commandments a lot. Great movies actually, but I saw them dozens of time as a child. Aladdin was not allowed because it had “an evil spirit genie”. And we couldn’t watch the part in Pinocchio where he turns into a donkey….cause that was like evil spirits or something.
6. If anything bad happened to you, it was a punishment from God. If you were so angelic that God couldn’t possible be punishing YOU, then it was because it was a test of your faith. Like Job in the Bible. Or you had secret sins no one knew about except God (I was a good little child, so I heard about secret sins a lot).
7. Questions that aren’t about faith are of the devil. If something doesnt make sense and you question it, its because you don’t have enough faith, so stop questioning. Just have faith. Or pray against your doubt. You don’t have enough faith if you are still questioning.
8. God talked to you from heaven. And if something God “said” didn’t make sense or was contradictory with another guys god-message it was either because 1.) You weren’t “in tune” enough 2.) You fucked up the message with your lack of faith 3.) You are not enlightened enough to interpret it 4.) The other guy was the one who didnt get the right message 5.) Whoever had more money was righter
9. As a baby in heaven before you being born, you picked your parents out. This one NEVER made sense to me hahahah. Imagine a kid thinking this was right and growing up wondering “What the fuck was I thinking up there picking you?”
10. There are demons and angels everywhere fighting for souls.
I wanted very badly to see an angel when I was a kid. I read all the angel stories, all the miracle stories, and wanted something like that to happen to me. I sat up all night one night mentally “willing” my mind to see my guardian angel. I felt I saw an outline of something at the end of my bed. Hahahahah. Part of my mind said it was bullshit, but I wanted to believe it very badly, cause after all, my angel should be nicer and let me see him. My angel was a boy to me. Thankfully, I didn’t try to compel my angel to rescue me by standing in traffic…..but I thought about it.
I’m not listing this to bash religion, but to give you a bit of background for why I think Jamie Hale’s book is important and to compare the mindset that is often encouraged by certain fads, trends or coaches. These are examples of beliefs I grew up under. I grew up being taught not to question and to have lots and lots of faith. Faith, faith, faith.
You can see a lot of these “faith” trends in the diet and fitness world. While people might say the comparison is too extreme, to that I say, no it’s not. Belief is not exclusive to religion. Religious beliefs are often more visible, but no less insidious then a diet belief. It doesn’t make it wrong, per se, but it does dictate actions, and those actions could be very damaging to you or your clients. If you want to believe something you better know why you do, and examine the evidence.
A belief that sugar is bad is why you avoid any kind of sugar period, even at your birthday party or a wedding. A belief that gay love is wrong will lead you to verbally abusing homosexuality. Your actions come from your belief. It’s the same mechanism. Belief drives action.
When you are coaching someone, you are often up against their arsenal of “beliefs”. The biggest being beliefs about themselves, and why they are where they are, or who they are right now. Some people have developed better self-awareness and emotional maturity and aim to apply objectivity to themselves and their actions (which can never be completely objective unless you could literally “step out” of your own brain…..which you can’t). These people are more open to examining their beliefs about themselves. Others have never even thought of it in those terms before! I know I didn’t. If you were asked to list, in private, everything you believe about yourself, what would it say? Try it.
Here are beliefs I come across from clients:
“I just can’t lose fat.”
“I have always had a crappy back.”
“I just don’t move that way.”
“I just don’t have the time.”
“I look horrible.”
Beliefs about their training:
“I have a weak core.”
“I can’t get this exercise.”
“I never have been able to do that.”
“My body is disgusting. My legs are disgusting and huge.”
“I’m so slow. I don’t learn well. I am unathletic.”
Now their beliefs may or may not be true. What we need to find as coaches …….
You have this person, who is a collection of processes, thoughts, cells, years, input, genes and experiences. And you have to filter through that and find the evidence of “what is” to affect their “what will be” aka progress. When someone tells me about themselves, chances are half of it is totally irrelevant, cannot be measured, or is entirely subjective. We turn to our education and experience to do what makes sense to us if we want change. A coach does exactly the same thing as their client, except a “professional” implies that you know more about what you are doing, and spend much more time studying it. But the process is the same. But if we don’t address our beliefs, how we get evidence, and how we choose and apply information, it could be a complete waste of time. You can just spin your wheels or latch on to something that seems plausible to you with no way to measure if it works, how and why.
How do you know what matters? Being evidence-based means knowing how to judge evidence and information, so you can know how to judge what applies, what doesn’t and what might, and set up your plans and recommendations accordingly. “If I see this, what does it mean and how do I know?”
This is where this book comes in. It breaks down how to think like a scientist and what that actually means.
Back to my personal story briefly. I got a good education when I was younger for the most part. I was home-schooled till high school. I got exposed to a lot of good input. I had music appreciation classes, dance, language studies (I grew up in Thailand), and read a ton. My education was probably superior in a lot of ways. I learned about science, but never how to think scientifically. History, geography etc was all taught under the “filter” of religion. Evolution was a lie, and homosexuality was sinful. I learned plenty of historical events, but they were filtered through what the religious leaders thought was “god’s will” or not. The USA was always evil and against god’s will. Except MacArthur and his occupation of Japan. Mao was pictured as good! Hitler was still bad. Quite confusing. There was no hierarchy of knowledge, because God was the whole hierarchy! After god, it was your religious leaders….you know, speaking for god. It actually was hard to stop praying eventually as a young adult. I literally felt like god might strike me dead. It passed, but it was stressful for awhile. But the strength of the beliefs I was raised under took a long time to release their hold.
I was taught what to think, not how to think. I was strictly taught what to think, and reminded that there was serious consequences for not thinking like that. Does this sound like any fitness dogma you can name? How many fitness professionals do you know that can stay relatively unemotional and objective about certain topics and not resort to mental-gymnastics to avoid addressing clear questions and inquiries?
Diets give the best examples, but it applies to lots of things people believe. “But you are not allowed to judge my belief.” Well, pardon my french, but of course I fucking can. You are not allowed to judge my judging. I am allowed to follow a standard I set for myself, what I view as superior, and you do not have to live up to MY standard, but I am allowed to judge your standard, against mine for evaluation.
“But how can you judge standards?” Aha, well here is the crutch. For information, knowledge and the establishment of what is known as facts, there is a standard by which we can judge. It’s called science. Staying as far away from science as possible, will tell me a lot about you. It tells me you don’t like inquiry, and your evidence can’t live up to scrutiny. It tells me maybe you don’t have any evidence at all. And I can most certainly judge that. Beware of that which does not want to be examined.
Science is a standard. Science is a standard for information. It is not the information, it is the ruler by which to judge the information. Ignoring the evidence and the quality of evidence tells me that you are more married to the belief, than to the results. And that’s all that makes you evidence-based or not, irregardless of the arguments, and the questions and the wonderings. All that is fine. Being unable to recognize any kind of standard, or disregard reality IS the problem.
Telling stories isn’t knowing. “Anecdote is not science.”
On what is science?
So after all this, what is science and what is it not?
“Science is not the mysterious, distant, smoking-test-tube sort of a priesthood that many imagine it to be. Rather, it is simply an organized, formal method of “finding out.”
This is what gets me most when people throw out the famous sentence “Oh well there’s studies for everything.” Well, yes there is. But science is not about PROOF, its about an organized way to find out if there IS proof. Big difference.
This excerpt from the book explains what science is, and what it is NOT. Pay attention to the NOT, because it’s one of the first crys of “foul” from those who dislike having science brought into a discussion:
“Some people have the erroneous assumption (the NOT) that science claims certainty; when in fact, science makes no such claims. Scientific knowledge is tentative, and the tentative nature of science is one of its strong points. Science, unlike faith-based belief accepts the preponderance of evidence and changes it’s stance if the evidence warrants. Science takes us where the evidence leads.
Being a scientific thinker enables one to understand evidence and resist falling for nonsensical claims. The more one learns about scientific thinking the more one becomes aware of what is not known, and the more aware one becomes of science’s willingness to modify claims when warranted. Science is not about the need for closure, but the need for establishing principles that are tentative.”
Jamie uses the bead analogy to help us grasp this important concept.
“This model depicts a bead on a horizontal wire that can move left or right. A 0 appears on the far left end and a 1 appears on the far right. The 0 corresponds with total disbelief, and the 1 corresponds with total belief (absolute certainty).”
The idea is that the bead should never reach the far left or far right. It is impossible to be THAT certain about anything. And unhealthy. “The more the evidence suggest the belief is true, the closer the bead should be to 1.” And vice versa. This is how you can judge information that is “more” true or “less” true or more uncertain.
On a formula for setting up how you use the information you have as a coach (works for just a normal person too):
Here’s a formula Stuart McMillan came up with for his coaching and how to take information and apply it to an athletes or clients plan. He calls it; KTG = Know, Think, Guess.
He shared this with the coaching interns during my week at the World Athletics Center in Arizona.
I mentioned this in another post, but it applies here as a cornerstone principle to dealing with information and developing plans for people. Or even in thinking about how something applies.
75% of what he does he knows it works. There is plenty of research and anecdotal experience to back it up. If someone argues about stuff that fits in this category then you know they are full of shit. Something like failing to recognize the energy equation, or arguing that ketogenic diets are superior for performance. This is the stuff, that is closest to fact. It has the most backing. Of course, what we “know” can change, but anything that fits into here, has substantial evidence behind it.
20% he thinks it works. This is stuff that top coaches might have various philosophies on. This is more of a gray area. This is where best guesses come in. Not crazy enough to be complete experimentation, but it is much more client/athlete or context specific. The evidence might straddle the fence a bit more.
5% is the complete guessing. This is where you can fiddle and play and try out. Nothing in this category should be dangerous (well, hopefully not), but its where you can play around a bit. This is what drives continued passion and interest for your profession. This is true experimentation. It might not have warranted research yet, or is too obscure. This is the stuff some coaches might “swear by”, but it hasn’t been “proven” in research yet.
I define being “evidence-based” as someone who wants to actively stay close to as high a standard of information and recommendations as possible for their work and the majority of it is based on protocols that have been proven to work, and have solid evidence. Nowadays that’s not too hard. We have a lot of information on sport science and nutrition. Information you can be pretty damn sure of (75%), information that you can be almost sure of (20%), information that I get to fuck around with for experimenting (5%). If someone knows nothing of the basics, then run far away till they do.
On staying flexible and embracing uncertainty:
Matt Perryman writes about our need for stability or certainty and it definitely applies to how we view the information we have, “Stable is a synonym for inflexible. This is equally true whether the curve-ball is a radical idea or a life-altering black swan event.”
That idea used to scare me. That everything is more uncertain than certain. Dan Pfaff calls it respecting the chaos that is life and the human body. But once I gave up having to be “certain” of one idea or another, I became a much better decision-maker. And if clients pay me to make decisions….well then you can clearly see the benefits. Learning becomes less ego-driven, and challenging your ideas or thoughts is less scary. This is what “having an open mind” means to me. It’s being able to examine without the need to come to a definitive answer.
“Being a scientific thinker enables one to understand evidence and resist falling for nonsensical claims. The more one learns about scientific thinking the more one becomes aware of what is not known, and the more aware one becomes of science’s willingness to modify claims when warranted. Science is not about the need for closure, but the need for establishing principles that are tentative.”
As a child I was taught many nonsensical things, so this topic has a bit of bite in it from me.
Do you have any evidence to back it up? If so, what kind? If it’s low on the hierarchy of knowledge, then stay open about it. If it’s high, place more confidence in it.
Does it seem to go against the laws of nature, AND have no quality evidence at all? It’s quackery plan and simple.
Does it work but you don’t know why? Keep at it, but avoid making claims about it. You can hypothesize. Scientific thinkers never claim absolutism.
If something works, or you think you know why it works but don’t have any high-quality evidence, keep doing it. Coaches have been doing this for ages. If something is working, keep doing it. Part of the process of coming to knowledge, is the active examination of our world. This is also why correlational studies have value. Its the persistence in wanting to find out, that leads to finding out.
On common sense:
“Referring to common sense as reason for a particular claim is a mistake. Yesterday’s common sense is often today’s common nonsense. Once upon a time it was common sense that the world was flat. History is replete with examples of the failure of common sense.”
Common sense is not so common. Does this mean to avoid our intuitions completely and walk around scared of our own conclusions? No. It means you have to actively work at being vigilant in your own thinking. Jumping to “it makes sense” doesn’t usually make sense upon closer examination! And a lot of “common sense” can’t hold up under questions. Have that sentence mean something. If it makes sense to you….it’s simply because you actually have thought about it and have determined your reasons.
Here’s what I do. Instead of jumping to “that makes sense” before I have actually processed it a bit in my mind, I will say.
Try it. It implies thoughtfulness and puts you in a state to BE thoughtful. If you are talking to someone who is very opinionated and emotional about their “facts” saying “hmmmm” can help them feel that you are actually listening, which can then help put them in a state to be open to productive discussion, yet it does not tie you down to implied agreement. It’s a useful little sound.
On tips for reading research and misconceptions about science:
Jamie asked several professionals these two questions in the book:
1-Do you have tips for people tat are interested in enhancing their ability to read scientific research?
2-What is the biggest (or at least one of the biggest misconceptions) misconception about science?
This part of the book was one of my favorites. Here are a couple answers I particularly liked.
“Reading the studies is definitely a good skill tohave, but being able to look at all the studies holistically without losing your sanity is also very important. Studies are good, but they don’t always apply to you so you shouldn’t worry about everything you hear, just that which matters.” – Kurtis Frank
“That science is what only scientists can do. Instead, science is a powerful system to ask questions and get answers, and every person in the 21st century should be science-literate, same way we learn math, reading, foreign languages…”- Alvaro Fernandez
“The biggest misconception I run across from people I talk to and from students is that they think science is a belief system. They have a perception that science involves the beliefs that scientists have about how things work. They don’t understand the data collection and analysis aspects of the scientific method, and when faced with a scientific conclusion they feel that it’s OK to say, “Well, I don’t believe that.” – Jonathan Gore
On rationality vs intelligence and why “smart” people can get be wrong:
“So, it may come as a surprise to find out that intelligent people are not necessarily rational people.It may surprise some people to learn there is more to good thinking than intelligence. In fact, intelligence is a weak to moderate predictor of many rational thinking skills. In some instances, intelligence shows zero correlation with rational thinking skills. An example of the dissociation of intelligence and rationality is seen with myside bias. Myside bias is displayed when people evaluate and gather evidence in a manner biased towards their own beliefs and opinions. ”
“Intelligence reflects reasoning abilities across a wide variety of domains particularly novel ones. In addition, intelligence reflects general declarative knowledge acquired through acculturated learning. Rationality reflects appropriate goal setting, goal optimization, and holding evidence-based beliefs.“
“Epistemic rationality is about what is true and instrumental rationality is about what to do. For our beliefs to be rational they must correspond to the way the world is—they must be true. For our actions to be rational they must be the best means toward our goals—they must be the best things to do.”
“Humans naturally default to the processing mechanisms that require less effort, even if they are less accurate.”
We are lazy. We can be as lazy in thinking, as we can be in getting a workout in. Except, a lot of people don’t think of thinking as work. It doesn’t seem intuitive. You can furrow your brow and listen to respond, and then jump in and preach…..but do you actually know? How do you know it?
If you ask these questions, you’ll be much better at getting around to the answers you want. Better yet, you might come up with better questions.
But if you really want to be evidence-based, put your money where your mouth is and buy this book. It will help you be more “evidence-based” to start off with.
Jamie Hale can be found here.