Recently I was pondering this question: What was the most valuable thing I learned in college? Out of everything I learned, what has been the most helpful? If I could go back and only take one class, what class would that be? Or if I could tell my kids to take only one class, which class would I tell them to take?

After sorting it all out, I think that class would be: Critical Inquiry and Research Methods. 

 

Was it my favorite class? Nope. Not at all. In fact, it was rather boring and tedious compared to some of my other college classes. But I feel like it ended up being one of the most valuable skills I ever gained in my entire college experience.

 

 

Why should we take the time to educate ourselves about research methods?

Why is it important to understand the principles behind research, such as the importance of sample size, different types of samples, or to understand terms such as “confounding variable”, “selective bias”, “selective use of data,” “preliminary study,” “validity,” “independent variable,” etc.? Why does it matter?

 

 

Here’s a few reasons why knowing a few of the basic principles of research methods can make a world of difference:

It can help you become a better consumer of information. 

Because…

Not all scientific studies are created equal! 

 

But the media, or we, as uninformed consumers, often tend to treat them as if they are. (For more on this, please see the embedded videos towards the end of this post!)

 

 

Understanding research methods is important because:

It is not possible to differentiate between biased research outcomes from unbiased ones without a proper understanding of each aspect of research.”¹  

 

Different research methods have different purposes, and understanding these differences can help us measure how accurate the conclusions of any certain study likely are. The type of study and the methods used affect how the results should be interpreted. For example, would the thing that is being researched by means of a case study actually be better served through conducting a longitudinal study? Is the sample random, systematic, or stratified? Because it makes a difference! Does the research have a high measure of validity? [validity = whether or not a study measures what it is supposed to measure]. Because validity will be affected depending on how the study is conducted. The results of research with a high measure of validity will be able to provide stronger evidence.²  

 

 

Here’s just one example of a common pitfall you can easily avoid by understanding some basic principles of research:

 

Correlation does not equal causation!

 

Often people confuse correlation with causation. But just because a research study shows a strong correlation between ice cream sales and shark attacks, it does NOT mean that ice cream causes shark attacks. They are simply correlated– there is likely a third variable at play: high temperatures. (People are more likely to be outside swimming and buying ice cream during warm weather).

Now it doesn’t mean that correlation is useless– it can be a great starting point for further research. We just shouldn’t take it at face value. We need to look deeper to see if the other variables are being controlled, or if there are confounding variables that are likely influencing the results of the research.

 

Relying on “common sense” or “folk wisdom” can sometimes lead us astray.

Just because you’ve always believed or been told that something is true, doesn’t mean that it is.

 

Dieter F. Uchtdorf said:

Part of our problem in the quest for truth is that human wisdom has disappointed us so often. We have so many examples of things that mankind once ‘knew’ were true but have since been proven false.

For example, in spite of one-time over­whelming consensus, the earth isn’t flat. The stars don’t revolve around the earth. Eating a tomato will not cause instant death. And, of course, man actually can fly—even break the sound barrier.”

 

Often folk wisdom gets handed down from generation to generation based solely off someone’s anecdotal evidence. But there’s a problem there– isolated events that are tied to an individual’s personal experience may have been true for that person in that specific circumstance, but that doesn’t mean you can generalize it to be true in every circumstance and for every person. Because other factors weren’t controlled. In order to establish whether or not something holds true across the board, you’d have to be able to conduct valid experiments where all other variables are controlled to determine if the results are still able to be replicated.

 

 

Helps you be a more independent thinker.

Learning about research methods can help you develop your critical thinking skills and be a more self-reliant life-long learner. Not just blindly accepting what people or ”experts” tell you without questioning it or digging deeper for yourself.  It rescues you from basing your opinions off of skewed data or faulty research that is often presented as fact. Which prevents you from being led into great errors.

 

 

It can help you cut through all the confusion!

Trying to come to any certain conclusion about what is true and what is not can be confusing, especially when there are so many different research articles all coming to different conclusions. Sometimes it can make you want to throw your hands up in the air or even throw in the towel at the whole scientific community, and conclude that they must not know anything since many studies seem contradictory.

I’m grateful for scientists and researchers. They are not the enemy. The enemy lies within ourselves and our own ignorance.  Lack of knowledge can lead to prejudice [a preconceived opinion that is not based on reason or actual experience], based on fear of things we don’t fully understand. Educating ourselves can help us rise above prejudice and misinformation.

 

It is always good to keep in mind that just because something is printed on paper, appears on the Internet, is frequently repeated, or has a powerful group of followers doesn’t make it true.

Sometimes untrue claims or information are presented in such a way that they appear quite credible. . .What may seem contradictory now may be perfectly understandable as we search for and receive more trustworthy information.” (Dieter F. Uchtdorf)

—————————————–

Don’t believe every claim just because it’s backed up by “research”

Just because there are a hundred research studies that say that a certain something is true, doesn’t necessarily mean that it is. Because if all one hundred of those research studies are using deeply flawed methods, then it doesn’t matter how many times you replicate it– those numbers don’t really prove anything. But you wouldn’t know it unless you looked into it yourself, and unless you knew what specifically to look for. 

 

“If you read that ibuprofen relieves headaches faster than aspirin, or that children learn more in private than in public schools, you should not believe it just because the findings are based on ‘research.’ “

(Charles Stangor, from his book Research Methods for the Behavioral Sciences, page 13. Screenshot below)

 

Bottom line: Just because “research shows” this or that, doesn’t automatically make it true. 

 

Who’s behind it?

People have agendas. Who is funding it? Do they have a conflict of interest, or a reason why they’d want to have this research study done? “Follow the money” can be a useful bit of counsel. But, this is not the only thing to consider. Just because a certain someone is funding it doesn’t necessarily mean the results are automatically skewed and worthless. The research methods themselves are important to consider and hold more weight in the end.

 

Don’t fall victim to manipulation

 

If you don’t know how to go in and analyze a research study yourself, you are victim to being manipulated by whoever is trying to convince you of something. And that’s scary! Because there are people who are very good at presenting lies in ways that seem totally credible, even to people who are otherwise educated and discerning. 

 

It can be easy to manipulate research and conduct it in such a way that you manipulate the results to sway public opinion. Marketing experts know that if they can run a commercial that says “Research shows” something, or spits out an impressive statistic, that many people will simply take their word for it and be convinced enough to buy into something.

 

The problem is, that if the research methods used are not the ones best suited to the subject being studied- or if there are confounding variables that are being overlooked, the findings will not be reliable, no matter how plausible and convincing they sound.

 

Be Wary of Headlines

We often hear about the importance of making sure it is a “peer-reviewed study.” And rightly so. But even if it’s peer-reviewed doesn’t mean that the conclusions drawn by news media headlines are accurate. (News media is notorious for misrepresenting or misinterpreting research, often making the researchers themselves cringe. Why? Because no one wants to read an article with the headline: “New research study shows that there may be a link between such-and-such and such-and-such, BUT more research needs to be done, and this actually might not even be accurate because it’s a preliminary study and the sample size was ridiculously small.”🤣 No, their headlines need to be short, and because the way they word their headlines affects their viewership and click-through rate, even well-meaning journalists are going to try to put a more attention-grabbing twist on it). As a result, headlines can often be a little out-of-context and misleading.

 

Now I’m not saying that journalists are bad or that all news media is out to get us. I believe that there are many good journalists and reporters who seek to do the best they can and are trying to represent information accurately. But it’s just the nature of the industry. They need views. Their headlines have to be succinct. That’s a tall order and often results in the over-simplification of complex research. So we as consumers have to take some responsibility to look into the research ourselves rather than just accepting everything we read at face value.

 

We shouldn’t generalize! But we so often do. Results of a particular study are generally not applicable to the general population. Results of preliminary studies on mice don’t usually translate to humans. Results of studies done on women can’t necessarily be generalized to apply to men.  That’s why it’s important to take a close look at who we are sampling. Or to be aware of other things that can greatly skew the results, like the “placebo effect” or “observer bias.”

 

 

 

Here’s a video that does an excellent job explaining why we shouldn’t take every scientific study at face value:

And I also highly, highly recommend watching these 4 short video clips below! They do a good job of presenting some of these things in an engaging way and explain it a lot better than I can:

 

(While we’re on the subject, taking a statistics class
in college is also super valuable!)

 

 

 

 

 

Resisting our Natural Impulse to Jump to Conclusions Too Hastily

 

 

Simon Weil said it beautifully:

“All faulty connection of ideas..are due to the fact that thought has seized upon some idea too hastily, and being thus prematurely blocked, is not open to the truth…”  

 

It’s something we all do; it’s just human nature. But we ought to take care not to jump to conclusions or form opinions too hastily, based on the information we come across & current research findings. 

 

Dr. Kate Holbrook expounded on this idea (italics added):

 

“To make ourselves feel less vulnerable, we humans like to define things. We often grab onto a definition [or research finding!] quickly because having the definition makes us feel comfortable and safe. As Weil suggests, the problem with the speed is that we can reach a wrong conclusion. I believe this too quickly finding and holding tight to a potentially false definition– is what leads to some of our greatest sins against one another, such as bigotry, which rears its ugly head in every nation and every political party, as we misunderstand each other’s intentions and lash out in response. Excessive speed can also mean we teach a false principle in a lesson, or argue on social media, in a way that treats someone unfairly.” (Kate Holbrook)

 

 

Smart people can sometimes have different opinions, even on valid research.

 

Even if it is a well-done research study, that doesn’t necessarily mean that the results drawn are all-conclusive. There’s often still room for interpretation and further study, especially in the social sciences. There may be two very intelligent people reading the same research study who come to different conclusions or have different takeaways on the research. And that’s important to understand.

But what if I never took a research methods class?

So what if you never had the chance to take a class like this in college? Never fear! There’s hope! Luckily, because of Covid, many college professors have been putting up their class lectures on YouTube, for free! So I can almost guarantee that through a quick online search, you’ll be able to find lots of wonderful resources now, available for anyone who wants to become more adept at research methods and deciphering a faulty research conclusion from a sound one.

 

I’m not saying I’m not perfect at it. But knowledge is power! And when applied, it can really help us in sorting out errors and discerning truth when faulty studies are presented as reliable and credible, when they are in fact, not.

 

So next time you have stats thrown at you or you see a news headline citing ‘research’, don’t forget to take a step back and look a little deeper before jumping to conclusions, or accepting someone else’s.