Is That So?
The Art of Evaluating Information

Mark Isaak      

We are exposed to information in quantities unprecedented in history, and even that amount is minuscule compared to the amount available to us with just a little effort. But the quality of that information is highly variable. Much of it simply isn't true. People lie and exaggerate for personal gain or to avoid harm. Crusaders distort messages to make their side look better. Rumors get spread for a variety of reasons which have nothing to do with the truth of the rumor. And people simply make mistakes. Decisions we make based on this information will affect not only us personally, but also our family, friends, and society as a whole. Obviously, it is important to know whether the bases for our decisions are accurate or not. How can we separate the reliable claims from the false ones?

The surest way to evaluate information is to check out the evidence for oneself. Unfortunately, that is not always practical, or even possible. The other common means of deciding accuracy is to give higher credence to trustworthy sources. Unfortunately, this is not always possible either, since we don't always know the trustworthiness and reliability of the source, or even what the source is. Even when we have accurate information, we can still run into problems of misinterpretation, over-generalization, or other judgment errors. Clearly, other tools for evaluating information would be most useful. Fortunately, there are three straightforward principles which are very effective for accurately evaluating claims: go exploring, question motives, and expect to make mistakes.

1. Go Exploring

A basic requirement for evaluating information is a wide base of knowledge. This will help in several ways. First, we can't effectively evaluate a claim if we don't know what the claimant is saying. Much information concerns issues of some specialization or complexity, issues that a basic education might not cover. Many people, for example, do not know what the theory of evolution really says. Until they learn, any judgment they make on the subject, no matter how soundly based and argued, will be meaningless, because they are not judging the subject they think they are judging.

Second, a wide base of knowledge lets us see how well ideas fit together. It may seem futile to accumulate information before we can evaluate it, but a body of information can aid its own evaluation. Sound ideas tend to fit together consistently, but bogus claims, even if they get some immediate support from other claims, eventually lead to contradictions. Also, ideas which are widely built upon are likely to be sound, but unsound ideas rarely go anywhere. When calculus was first developed, many people were suspicious of the idea of calculations which break things into an infinite number of infinitely small parts, but the many successful engineering designs (among other things) relying upon calculus in the centuries since have proved its worth. Astrology, on the other hand, has been in use for thousands of years, yet there are still no other fields which are based upon astrology. Once we have a fair amount of background knowledge, we can test claims against each other.

Third, as we explore new avenues, we can encounter ideas which lead us to question beliefs we would not have questioned otherwise. I once believed, simply because it was what I always heard, that exposure to cold temperatures contributed to causing colds. I would probably still believe that, except I have since heard from several sources that temperature has nothing to do with it, and that staying cooped up indoors can actually make catching cold more likely.

To acquire a broad base of knowledge, we have to go after it; it will not come to us. How should we seek it? There are several possible ways. Read. Go to school. Cultivate friends with different interests. Try new hobbies. Travel. All of these are effective means of learning. Use whichever appeal to you.

In deciding what areas to explore, the important principle to keep in mind is to look at things you have not looked at before. In particular, look beyond stuff that feeds your existing beliefs and preconceptions. If you are a liberal, for example, read some conservative publications; they probably have ideas and information you have not looked at closely. Likewise, if you are conservative, read liberal works. And try to read them with an open mind (more on this later). Don't read everything you believe.

Some subjects are particularly useful to know for evaluating information. Philosophy can show how some ideas have been dealt with before. Psychology can teach what patterns of thinking people are prone to. A basic understanding of the scientific method is generally helpful, particularly in pointing out the usefulness of control groups. Some understanding of statistics is helpful, in part for showing the worthlessness of anecdotal information applied to general claims, but also for such concepts as confidence intervals and sampling space. However, knowledge of any and all other subjects has value, too.

Good background knowledge is valuable for evaluating claims on the spur of the moment. When we have time to devote to examining a claim, we can focus our research more narrowly. It is still imperative, though, that we learn and understand both (or all) sides of the issue. Most claims, whether true or not, look good if viewed from only one side. If one side is all you look at, you are not examining a claim; you are admiring it. This only makes it harder to let go of the belief should you, by chance, have chosen to admire a side that isn't true. Diverse input is needed to effectively evaluate even the most specific claims.

2. Question Motives

Much insight into the credibility of claims can come simply from questioning the motives behind the claims. People's emotions surrounding an issue affect how they gather evidence, evaluate it, and present it to others. Several common motives affect credibility.

Personal gain is an obvious motive behind advertising claims which encourages the claims to be slanted a particular way, often to the point of outright dishonesty. The motive of personal gain can occur in a variety of other situations, too, such as people trying to protect their reputation or advance their career.

Entertainment motivates many stories to get repeated solely because they are good stories. Urban legends fall in that category. Even true stories often get important but dull parts removed or exciting details added in the retelling.

Fears and desires motivate wishful thinking. Oftentimes people will convince themselves that what they want to be true really is true, and then they will communicate that belief to others.

Laziness can play a role. It is easier not to double-check all your facts and simply to rely on memory. Since memory is highly fallible, this allows many errors to enter people's assertions.

A desire for answers can lead to unfounded beliefs. Given a choice between uncertainty and a plausible but baseless explanation, many people will choose the explanation, despite the fact that it is essentially a work of fiction.

All of these motives and more can lead to false information. Detecting any of them should make you start to be suspicious--the stronger the motive, the more questionable the claim. Of course, you should also be careful in attributing motives; don't ascribe nefarious motives just because you don't like the information. Usually it is possible to get a good idea of someone's motives by putting yourself in their situation and/or looking at the motives other people have had in similar situations.

Examining motives can also give greater credibility to some information. If someone says something against their self-interest, there is a greater likelihood of its being true. Also, people do not like being proved wrong, so information is more reliable if it is easily checked. In particular, information is more trustworthy if it comes with a reference to the source it was derived from.

Most especially, we should examine our own motives when deciding how much credence to give to information. All of the motives above apply to us as well as to other people, and other people are not the only ones who can fool themselves. Our motives will influence how we evaluate claims; if we want an honest evaluation, we will have to be aware of how our motives influence us. Probably no other practice is more important for accurate evaluation.

3. Expect Mistakes

Certainty about anything in the outside world is impossible because there are opportunities for error everywhere. Our sources could be wrong; our memory or reasoning could be wrong; even our own senses could be wrong. In fact, all of the above will be wrong at various times. With perfect knowledge unattainable and mistakes so easy, it would be unwise to accept any conclusion as fixed. Always allow for the possibility that it might be wrong, and be ready to change it if new and better information comes along. Of course, some of the things we know are more reliable than others, but everything has at least a little bit of a possibility of being wrong.

Some people think beliefs based on faith must be held with certainty, but faith is not about certainty. Faith is belief which we choose because the belief is good for us and because there is nothing else upon which to base the belief. Because faith beliefs are something we choose, they are fallible, too. Certainty based on faith is a pretend certainty. The purpose of faith is to improve our lives; holding stubbornly to one faith belief can prevent us from finding a better one.

On occasion, we might find a faith belief conflicting with objective evidence. Different people have different approaches for resolving such conflicts. The uncertain nature of both faith and evidence makes it impossible to say which approach is right; we must each decide for ourselves. However, it is important to remember that, while objective evidence applies to everyone, faith is a personal choice, specific to the individual. Hence any decision based on faith applies only to the person making it and cannot determine what is right or wrong for anyone else.

The prospect of being wrong makes most people uncomfortable. Being wrong has connotations of failure, and it frequently implies a need to change away from a state of mind with which we are comfortable. But the mistakes and uncertainty are there whether we like it or not, and the rewards for admitting them are many. Admitting a mistake places us on a more productive path and shows us a lesson for avoiding the error in the future, and most people respect those who can admit mistakes. And practicing such admittance lets us worry less and take more risks. With practice and some determination, one can learn to delight in discovering that one was wrong.

But remember, too, that correcting a mistake still does not mean you are necessarily right. The correction itself could be wrong. Although our confidence can be improved, although we can become more likely to be right, the admonition to expect mistakes holds always. If we forget that, we will be stuck where we are with little hope of becoming more accurate.

Concluding Remarks

Exploring for new knowledge takes lots of work. Honest examination of one's own motives takes courage. Admitting the possibility of mistakes takes humility. None of that is easy. Nobody can be expected to follow these principles perfectly. However, even a little effort put into following them will make us a little better at evaluating information.

Is it worth it? In some cases, the answer will be no. Correcting useless trivia is not productive if the effort could better be spent elsewhere. And for some people, some beliefs just carry too many emotional attachments; simply questioning those beliefs may be too painful or emotionally unsatisfying for them.

In most cases, though, the benefits should outweigh the costs. There can be immediate practical benefits such as avoiding health quackery and investment rip-offs. More generally, having a better understanding of what is accurate and what is not allows us to live more harmoniously in the world. Honesty is a valuable ideal, and these principles are merely guidelines for becoming more honest in our world view. Finally, work, courage, and humility are also valuable ideals in themselves. Following the road to truth not only leads to a worthwhile destination, it makes us better people in the process.

Back to Mark Isaak's home page