Hindsight Bias and Insufficient Surprise

[Flannel_Jesus]

Only Humean made this post (Just left in the relevant bit):

@Only_Humean

There’s an interesting experiment described in Daniel Kahneman’s book, Thinking Fast and Slow.

In it, subjects are told they will undergo three tests.

• In one test, they made to feel (strong, but non-damaging) discomfort which is suddenly alleviated; they are to hold their hands in icewater for one minute. During this time, they report their relative pain levels.
• The other test is exactly the same procedure, but instead of removing their hands after a minute, the water is then slowly warmed by around a degree for thirty seconds, and then they remove their hands. The second test causes more pain, aggregated over time; the same as the first test and then thirty seconds’ reduced discomfort on top of that.
• The test order is randomised, and after the second test and a rest of a few minutes, they are allowed to choose which of the tests they wish to repeat for the final test.
  The interesting result is that the first group have more negative memories and associations with the experiment than the second, longer test: 80% of subjects chose to repeat the second test, rather than the first.

Faust responded:

@Faust

yeaaahhh. But some of us didn’t need experiments by scientists to tell us that.

I find this response interesting, in a sort of ‘Wow, your brain is really fucking with you now’ sort of way.
You see, hindsight’s 20/20

I imagine, if OH had counter-factually posted about the exact same experiment with the opposite result, Faust would have thought the exact same thing.
OH:
“When given the option of having 1 minute of strong discomfort, versus the option of 1 minute of the same level of strong discomfort followed by 30 seconds of more mild discomfort, most people would choose to have the 1 minute of strong discomfort without the added 30 seconds of discomfort.”
Faust:
“Psh, I didn’t need scientists to tell me that. Why the hell would people want to have more pain?”

Right? Like…the exact opposite result of the experiment is as, if not more, intuitive than what happened. So…no matter what result OH posts, the result is going to seem obvious to someone who’s not adjusting for their hindsight bias.

Even if you reasoned though it prior to the experiment, and thought that that’s what the result would be, I can’t imagine anybody could justifiably have a high degree of certainty that that would be the result. 60, 70% max.

Hindsight bias is a real psychological phenomenon, as well. It’s not just something some clever philosophers made up. It’s been experimentally demonstrated that people systematically overestimate the predictability of the results that they’re told are true. They systematically, and very consistently, think things are more obvious than they really are, simply because they were told beforehand that they’re true.

This article talks about it some.

Daphna Baratz (1983) tested college students’ sense of the obvious. She gave them pairs of supposed social findings, one true (for example, “In prosperous times people spend a larger proportion of their income than during a recession” or “People who go to church regular tend to have more children than people who go to church infrequently”), the other its opposite. Her finding: Whether given the truth or its opposite, most students rated a supposed finding as something “I would have predicted.”

That underlined portion…that just makes me laugh, really really hard. If you’re told X, you think it’s obvious. If you’re told notX, you think it’s obvious.

Surely only one of those two can be obvious, right? Right? They can’t both be obvious! ‘Obviously X is true – and obviously it’s not true’.

I don’t really like it when people devalue science with comments like ‘I could have figured that out on my own.’
My ass you could have. If you were told the exact opposite, you would say the same thing.

[Innovice]

is it okay if I [intentionally] devalue science for completely different reasons?

[Moreno]

@Flannel Jesus

I don’t really like it when people devalue science with comments like ‘I could have figured that out on my own.’
My ass you could have. If you were told the exact opposite, you would say the same thing.

I am very sympathetic to the postion(s) you are taking in this thread, but I think you are making a rule here where it would be better to simply be skeptical. Some scientific research does merit a duh. Like did you know that MRI scans of bullies reveal that they feel good when others suffer - when shown images of people being picked on - compared to the brains of non-bullies whose brains did not ‘reward them’ upon seeing such images. I dunno, to me that is predictable. I would not have bet my home on the result, but I would have been surprised if this was not the case. IOW I am saying that some research does reveal things that are fairly obvious. However skepticism about people’s reactions is utterly warrented.

What would your prediction be about a study that looked to see if people who watch TV a lot and socialize a little are less happy than those who socialize a lot and watch TV a little?

I am predicting you will get the right answer.

Note: just to be clear. This does not mean you should take people’s ‘Oh, I knew that,’ at face value. Hardly. Just throwing some nuance into the generalization.

[Flannel_Jesus]

I’m not saying that all scientific results should be surprising. It’s certainly the case that many studies do only confirm completely obvious common sense. Definitely and without a doubt. If some study confirmed that people feel pain when they get hit in the nuts, for example, and Faust had said, “I didn’t need science to tell me that,” that’d be totally reasonable.

The study in question, though, was quite different. There are two possible results to the study.

1. The result that did happen: people prefer to experience extra pain to reduced degree.
2. People prefer to experience less overall pain.

To say that 2 is obviously not the case and we don’t need any experiment to confirm it, to me, is a great case of hindsight bias. If, counterfactually, 2 WAS the result, it’s not very hard to imagine Faust saying, “I don’t need science to tell me that.” In other words, no matter what the result, Faust is going to think that he knew that would be the result. But obviously one can’t know X and know notX, right? They can’t both be equally obvious. And I really don’t think that Faust could have had a high degree of certainty that the result would have been 1 prior to hearing that the result was 1. I’m not Faust of course, and I understand why it might seem a little bit presumptuous, but…well, just look at 2. If he did have a high degree of certainty that 1 would be the case, he either came to that conclusion completely irrationally, or he knows more about human psychology than the vast majority of psychologists themselves. (Yes, you can come to a true conclusion irrationally, in case you got tripped up on that bit)

[peterpan]

Could it be that Faust’s reaction :

Faust wrote:yeaaahhh. But some of us didn't need experiments by scientists to tell us that.

is just the usual reaction to ‘science’ that people have when met by results of experiments that fail to provide a ‘wow-factor’.
Kind of like “why do scientists think they are so smart to tell me what I already knew?”.

[Flannel_Jesus]

That’s the problem, peterpan. To pretend like you already knew it, to act like it was more predictable than it in fact was, is what hindsight bias is. Answers seem obvious in retrospect.