Heuristics in political voting.

Heuristics in political voting.

On the way to vote in the general election John and Sarah were discussing which candidate and political party they were going to vote for. There were two main candidates up for consideration in the general election. The first candidate was seeking a second term in office whilst the second candidate was opposed to the first on policies but had higher approval ratings. Sarah mentioned to John that she was going to vote for the first candidate because the economy had done well under his administration. John said that he was going to vote for the second candidate because she had higher approval ratings, as such, was more popular than the first candidate. They both went on to cast their votes for their favoured candidates and continued on in the day as normal.

In the example of John and Sarah above we see a typical quick discussion with some reasons given for choosing one candidate over another in a general election. Like many voters, John and Sarah think about the state of the economy and approval ratings when trying to decide on who to endorse for public office. Voting in this way is not necessarily the most rational way to vote, however, this demonstrates some of the cognitive biases that influence our decisions during the voting process. When pressed to give a reason why we have voted in the way that we did many of us would say that we weighed up the policies of each candidate (and political party). We like to think of ourselves as rational decision makers however this is not so. Cognitive biases influence our decisions far more than would like to think so.

house of commons.png

General elections are some of the most important decisions that we can make. Although general elections in the United Kingdom are every four years the political party that goes on to win an election can influence our lives in almost every way. The incumbent Prime Minister takes charge of the direction of economic, health, judicial, educational and defence policy. If you have been saving up for the deposit for that nice first home and you are planning to take out the mortgage on the home in the next couple of years the housing policies of the Prime Minister are important, some ministers may help first time buyers whilst others may not.

When putting the ‘X’ next to the preferred candidate’s name and political party at the ballot box we all have a preference for some policies over another (e.g., conservative or liberal policies). John may like the idea of the nationalisation of railways, as such he decides to vote for the party that has historically been in favour of nationalising the railway network. By voting in this way John has made an affective (i.e. emotional) response towards a political party and voted by using the likability heuristic rather than thinking through all of the policies of each political party (Brady & Sniderman, 1985). The likeability heuristic has help John make a quick decision about who to vote without the need for agonising over the decision.

Another rule of thumb (or heuristic) is to evaluate the incumbent Prime Minister on the basis of the economy’s current performance – if the economy is doing well than the Prime Minister is a good candidate (Popkin et al., 1992; Schneider et al., 1985). One problem with this ‘economic performance heuristic’ is that it makes the assumption that the Prime Minister controls the economy, for the most part this is incorrect. The Prime Minister can decide on policies, but they do not control the economy. This ‘economic performance heuristic’ merely rewards good economic luck whilst punishing poor economic luck. For the Prime Minister’s cabinet, this rules of thumb provides incentives to sacrifice long-term economic growth for small boosts in activity, particularly, in election years (Fiorina et al., 1981).


A third heuristic that many of us use when deciding on who to vote for in a general election, or even local elections, is the ‘approval rating heuristic’. One cleverly designed study in 1993 by Jeffery Mondak at the University of Pittsburgh, Pennsylvania showed that participants used Ronald Reagan’s approval ratings as a cue to performance. When Reagan had high approval ratings, participants said that they would be more willing to vote for him, compared to when he had low approval ratings, Reagan’s policies did not differ with popularity. The study by Mondak demonstrates one important heuristic, that is that, we are more likely to vote for a candidate with high approval ratings than low approval ratings simply because they are more popular than the other candidates.

Heuristics aren’t only important when making decisions in elections they also play a role in the outcome of referendums. Under the direct democracy system of Switzerland, the Swiss have more referendums than any other country. In 2016 alone the Swiss had 13 referendums on a variety of subjects such as highway construction, wine production and economic policy (see Table 1 for an example). At first thought the idea of having a lot of referendums seems like a good idea because the public get their say on all government policies, however this system has its problems. When voting in a referendum we are expected to become informed about a diverse and highly complex serious of issues in our spare-time. One study by Gruner and Hertig (1983) collected data from the regular referendum voters in Switzerland after each vote from 1977 to 1983, they found that only 20% of the voters were actually well informed about the issues at stake, many of the voters only knew the name of the referendum.

(Table 1)


So, how do we make these decisions about the referendums that are crucial for the direction of government policy? Two of the heuristics that are important decision-making tools for referendum voting are the status quo bias and the likeability heuristic (Passy, 1993; Clarke et al., 2012). The status quo bias suggests that we favour the known over unknown, and reject the new and untested in favour of the familiar. As we can see in the table above many voters choose to stick with the familiar and reject the new without any clear reason. One example of this is the Swiss referendum on joining the European Economic Area (EEA) in December 1992. Following the voting academics asked the voters for their reasons and how they voted (Passy, 1993). On average 30% of the voters could not give any reason for how they voted (despite voting), 50% were able to give one reason and 20% gave two reasons, we can take this as an indication of the level of knowledge that each voter had. Many of the voters voted to reject joining the EEA by using the status quo heuristic.

The second of the heuristics that is involved in decision-making in referendums was demonstrated in Britain’s Alternative Vote referendum on the 5th of May 2011. This referendum is unique because there were some strong political personalities involved in campaigning for and against the alternative ballot. By the time of the referendum Nick Clegg, leader of the Liberal Democrat party had become decidedly unpopular, he was part of the coalition government and went back on many of his campaign policies. Nick Clegg campaigned for the Alternative Ballot. Following the voting many voters stated that they voted against the alternative vote because they no longer liked Nick Clegg (Clarke et al., 2012). Voters used the likeability heuristic rather than weighing up the consequences of voting for or against the referendum.

Like John and Sarah if you are agonising about who to vote for in a general election of which way to vote during a referendum bear-in-mind that there are many cognitive biases (heuristics) can help you make the decision. For better or for worse even when making highly important decisions cognitive biases such as the likeability heuristic, approval rating heuristic, the status quo bias and economic performance rule of thumb influence our decisions. To avoid these cognitive biases, we can start by thinking about the reasons why are considering voting the way we are.

Cognitive reflection and cognitive reflection-like items

Cognitive reflection and cognitive reflection-like items.

Below are a list of items from the academic literature.

Original CRT (3 – item Frederick, 2005)

  1. A bat and a ball cost £1.10 in total. The bat costs a dollar more than the ball. How much does the ball cost? (Intuitive answer 10 pence; correct answer 5 pence).
  1. If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? (Intuitive answer 100 minutes; correct answer 5 minutes).
  1. In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake? (Intuitive answer 24 days; correct answer 47 days).

4 –items from Toplak (2014).

  1. If John can drink one barrel of water in 6 days, and Mary can drink one barrel of water in 12 days, how long would it take them to drink one barrel of water together? (Intuitive answer 9; correct answer 4).
  1. A man buys a pig for £60, sells it for £70, buys it back for £80, and sells it finally for £90. How much has he made? (Intuitive answer £10; correct answer £20).
  1. Simon decided to invest £8,000 in the stock market one day early in 2008. Six months after he invested, on July 17, the stocks he had purchased were down 50%. Fortunately for Simon, from July 17 to October 17, the stocks he had purchased went up 75%. At this point, Simon has: a.  broken even in the stock market. b. is ahead of where he began. c. has lost money. (Intuitive answer b; correct answer c  value is £7000).
  1. Jerry received both the 15th highest and the 15th lowest mark in the class. How many students are in the class? (Intuitive answer 30; correct answer 29).

2016 CRT (Thomson & Oppenheimer, 2016)

  1. If you’re running a race and you pass the person in second place, what place are you in? (Intuitive answer 1st; correct answer 2nd).
  1. A farmer had 15 sheep and all but 8 died. How many are left? (Intuitive answer 7; correct answer 8).
  1. Emily’s father had three daughters. The first two are named April and May. What is the third daughter’s name? (Intuitive answer June; correct answer Emily).
  1. How many cubic feet of dirt are there in a hole that 3’ deep x 3’ wide x 3’ long? (Intuitive answer 27; correct answer none).

Decoy / control items.

  1. A cargo hold ship had 500 crates of oranges. At the ship’s first stop, 100 crates were unloaded. At the second stop, 200 more were unloaded. How many crates or oranges were left after the second stop? (answer 200 crates)
  1. Sara, Emma, and Sophia embark on a river trip. Each of them brings one supply item for the trip: a kayak, a cooler of sandwiches, and a bag of apples. Sara brought the apples and Emma didn’t bring anything edible. What did Sophia bring? (answer cooler of sandwiches)
  1. An expedition on a mountain climbing trip was traveling with eleven horse packs. Each horse can carry only three packs. How many horses does the expedition need?(answer 4 horses)
  1. A mechanic shop had five silver cars, two red cars, and one blue car in the garage. During the day, three silver cars and one red car were picked up, and one black car was dropped off. How many silver cars were in the garage at the end of the day?(answer two silver cars)  

2-items used for boosting in Shtulman & McCallum (2014).

  1. A house contains a living room and a den that are perfectly square. The living room has 4 times the square footage of the den. If the walls in the den are 10 feet long, how long are the walls in the living room? (Intuitive answer 40; correct answer 20)
  1. A store owner reduced the price of a $100 pair of shoes by another 10 percent. How much do the shoes cost now? (Intuitive answer 80; correct answer 81)

Baron et al., (2015)

Arithmetic items with lures.

  1. If it takes 2 nurses 2 minutes to measure the blood pressure of 2 patients, how long would it take 200 nurses to measure the blood pressure of 200 patients?
  1. Soup and salad cost $5.50 in total. The soup costs a dollar more than the salad. How much does the salad cost?
  1. Sally is making sun tea. Every hour, the concentration of the tea doubles. IF it takes 6 hours for the tea to be ready, how long would it take for the tea to reach half of the final concentration? (Finucane & Gullion, 2010)

Other items.

  1. Jack is looking at Anne but Anne is looking George. Jack is married but George is not. Is a married person looking at an unmarried person? a. Yes b. No   c. Cannot be determined (Toplak & Stanovich, 2002)
  1. Ann’s father has a total of five daughters: Lala, Lele, Lili, Lolo and__. What is the name of the fifth daughter?
  1. On the side of a boat hangs a ladder with six rungs. Each rung is one foot from the next one, and the bottom rung is resting on the surface of the water. The tide rises at a rate of one foot an hour. How long will it take the water to reach the top rung? a. 5 hours b. 6 hours c. never.

Primi et al., (2015) Cognitive reflection test – long scale.

  1. If three elves can wrap three toys in an hour, how many elves are needed to wrap six toys in 2 hours? (Intuitive answer 6 elves: correct answer 3 elves)
  1. In an athletics team, tall members are three times more likely to win a medal than short members. This year the team had won 60 medals so far. How many of these have been won by short athletes? (Intuitive answer 20 medals; correct answer 15 medals)
  1. If you flipped a fair coin three times, what is the probability that it would land ‘Heads’ at least once? _____ percent
  1. A car and a bus are on a collision course, driving toward each other. The car is going 70 miles an hour. The bus is going 80 miles an hour. How far apart are they one minute before they collide? ___ miles
  1. Ellen and Kim are running around a track. They run equally fast but Ellen started later. When Ellen has run 5 laps, Kim has run 15 laps. When Ellen had run 30 laps, how many has Kim run? ___ laps
  1. An ice cream vendor sells 2/3 of her stock of ice creams on sunny days and 1/3 of her stock on cloudy days. Yesterday, it was a sunny day, and she sold 300 ice creams. Today is a cloudy day. How many can she expect to sell?
  1. In a class, there are 42 children. There are 12 more girls than boys. How many girls are there in the class?

Ackerman (2014) items – translated from Hebrew

  1. A frog fell into a hole 30 meters deep. Every day it climbs up 3 m, but during the night it slides 2 m back down. How many days will it take the frog to climb out of the hole?(Intuitive answer 30 days; correct answer 28 days)
  1. Apple mash is comprised of 99% water and 1% apple solids. I left 100 kg mash in the sun and some of the water evaporated. Now the water is 98% of the mash. What is the mash weight? (Intuitive answer 99; correct answer 50)
  1. If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5% what is the chance that a person found to have a positive result actually has the disease, assuming that you know nothing about the person’s signs or symptoms?(Intuitive answer 95; correct answer 2)
  1. Every day, a bakery sells 400 cookies. When the manager is not there, 20% of the cookies made that day are eaten by the staff. How many additional cookies should be made on the manager’s day off to ensure that 400 cookies can be sold? (Intuitive answers 80, 500; correct answers 100)
  1. Steve was standing in a long line. To amuse himself he counted the people waiting, and saw that he stood 38th from the beginning and 56th from the end of the line. How many people are stood in the line? (Intuitive answers 94 or 92; correct answers 93)
  1. Ants are walking in a line. A bad-mannered ant cuts in front of the ant waling second. What is the rude ant’s place in the line? (Intuitive answer 1st; correct answers 2nd)

Tremoliere & De Neys (2014) modified congruent and incongruent versions of the bat-and-ball.


  1. A Ferrari and a Ford together cost $190,000. The Ferrari costs $100,000 more than the Ford. How much does the Ford cost? (Intuitive answer $90,000: correct answer $45,000)


  1. A Rolls-Royce and a Ferrari together cost $190,000. The Rolls-Royce costs $100,000 more than the Ferrari. How much does the Ferrari cost? (No intuitive answer; correct answer $45,000)

De Neys et al., (2013) control version of the bat-and-ball

  1. A magazine and a banana together cost $2.90. The magazine costs $2. How much does the banana cost? (No intuitive answer; correct answer 90 cents)

Oldrati (2016) booster items.

  1. A rope ladder hangs over the side of a boat with the bottom rung on the surface of the water. The rope ladder has 6 runs that are 30 cm apart from each other. The tide rises 70 cm. How many rungs will stick out of the water at high tide? (Intuitive answer 3 rungs; correct answer 6 rungs)
  1. There are 12 one-cent stamps in a dozen. How many two-cent stamps are there in a dozen? (Intuitive answer 6 stamps; correct answer 12 stamps)
  1. A farmer makes 4 piles of hay in one corner of a field and other 5 piles in another corner. If he merges them how many piles will he have? (Intuitive answer 9 piles; correct answer 1 pile)
  1. You are participating in a run. You overtake the second runner in the last meters before the finish line. In what position did you finish? (Intuitive answer first position; correct answer second position)
  1. 25 soldiers are standing in a row 3 m from each other. How long is the row? (Intuitive answer 75 m; correct answer 72 m)
  1. A snail starts climbing up a five-meter-high wall in the morning. During day it climbs 2 m and during the night it slips back 1 m. How many days will it take the snail to reach the top of the wall? (Intuitive answer 5 days; correct answer 4 days)
  1. A brick weighs 1 kg plus half a brick. How much does half a brick weigh? (Intuitive answer .5 kg; correct answer 1 kg)
  1. There are 5 white and 5 black socks in Franco’s drawer. Franco’s room is in the dark. How many socks should Franco take out of the drawer to be sure that he gets a matching pair? (Intuitive answer It cannot be determined; correct answer 3 socks)
  1. You go to bed at eight. You set your old analogue alarm clock to wake you up at nine. How many hours of sleep will you get? (Intuitive answer 13 h; correct answer 1 h)
  1. One month has 28 days. How man of the 11 months left have 30 days? (Intuitive answer 4 months; correct answer 11 months)


If you use any of these please reference the original source paper (cited in brackets above the items) and if possible this blog.

In APA format this blog is cited as…

Edgcumbe, D (2017). Cognitive reflection and cognitive reflection-like items [Blog post]. Retrieved from https://absolutedecisionsblog.wordpress.com/2017/01/08/cognitive-reflection-and-cognitive-reflection-like-items/


Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives. 19(4), 25-42.

Toplak, M., West, R. & Stanovich, K. (2014). Assessing miserly information processing: An expansion of the Cognitive Reflection Test. Thinking & Reasoning, 20(2), 147-168.

Thomson, K. & Oppenheimer D. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11(1), 99.

Shtulman, A. & McCallum, K. (2014). Cognitive reflection predicts science understanding. In Proceedings of the 35th Annual Conference of Cognitive Science Society (pp. 2937-2942).

Baron, J., Scott, S., Fincher, K. & Metz, S. (2015). Why does the Cognitive Reflection Test (sometimes) predict utilitarian moral judgment (and other things)? Journal of Applied Research in Memory and Cognition, 4(3), 265-284.

Toplak, M. & Stanovich, K. (2002). The domain specificity and generality of disjunctive reasoning: Searching for a generalizable critical thinking skills. Journal of Educational Psychology, 94(1), 197.

Primi, C., Morsanyi, K., Chiesi, F., Donati, M. & Hamilton, J. (2015). The development and testing of a new version of the cognitive reflection test applying item response theory (IRT). Journal of Behavioral Decision Making,

Finucane, M. & Gullion, C. (2010). Developing a tool for measuring the decision-making competence of older adults. Psychology and aging, 25(2), 271.

Ackerman, R. (2014). The diminishing criterion model for metacognitive regulation of time investment. Journal of Experimental Psychology: General, 143(3), 1349.

Tremoliere, B., De Neys, W. & Bonnefon, J. (2014). The grim reasoner: Analytical reasoning under mortality salience. Thinking & Reasoning, 20(3), 333-351.

De Neys, W., Rossi, S. & Houde, O. (2013). Bats, balls, and substitution sensitivity: Cognitive misers are no happy fools. Psychonomic Bulletin & Review, 20(2), 269-273.

Oldrati, V., Patricelli, J., Colombo, B. & Antonietti, A. (2016). The role of dorsolateral prefrontal cortex in inhibition mechanism: A study on cognitive reflection test and similar through neuromodulation. Neuropsychologia, 91, 499-508.

Cognitive reflection test