Wednesday, April 26, 2017

Not all publicity is good publicity, not even in science.

“Any publicity is good publicity” is a reaction I frequently get to my complaints about flaky science coverage. I find this attitude disturbing, especially when it comes from scientists.

[img src: gamedesigndojo.com]


To begin with, it’s an idiotic stance towards journalism in general – basically a permission for journalists to write nonsense. Just imagine having the same attitude towards articles on any other topic, say, immigration: Simply shrug off whether the news accurately reports survey results or even correctly uses the word “immigrant.” In that case I hope we agree that not all publicity is good publicity, neither in terms of information transfer nor in terms of public engagement.

Besides, as United Airlines and Pepsi recently served to illustrate, sometimes all you want is that they stop talking about you.

But, you may say, science is different. Scientists have little to lose and much to win from an increased interest in their research.

Well, if you think so, you either haven’t had much experience with science communication or you haven’t paid attention. Thanks to this blog, I have a lot first-hand experience with public engagement due to science writers’ diarrhea. And most of what I witness isn’t beneficial for science at all.

The most serious problem is the awakening after overhype. It’s when people start asking “Whatever happened to this?” Why are we still paying string theorists? Weren’t we supposed to have a theory of quantum gravity by 2015? Why do physicists still don’t know what dark matter is made of? Why can I still not have a meaningful conversation with my phone, where is my quantum computer, and whatever happened to negative mass particles?

That’s a predictable and wide-spread backlash from disappointed hope. Once excitement fades, the consequence is a strong headwind of public ridicule and reduced trust. And that’s for good reasons, because people were, in fact, fooled. In IT development, it goes under the (branded but catchy) name Hype Cycle

[Hype Cycle. Image: Wikipedia]

There isn’t much data on it, but academic research plausibly goes through the same “through of disillusionment” when it falls short of expectations. The more hype, the more hangover when promises don’t pan out, which is why, eg, string theory today takes most of the fire while loop quantum gravity – though in many regards even more of a disappointment – flies mostly under the radar. In the valley of disappointment, then, researchers are haunted both by dwindling financial support as well as by their colleagues’ snark. (If you think that’s not happening, wait for it.)

This overhype backlash, it’s important to emphasize, isn’t a problem journalists worry about. They’ll just drop the topic and move on to the next. We, in science, are the ones who pay for the myth that any publicity is good publicity.

In the long run the consequences are even worse. Too many never-heard-of-again breakthroughs leave even the interested layman with the impression that scientists can no longer be taken seriously. Add to this a lack of knowledge about where to find quality information, and inevitable some fraction of the public will conclude scientific results can’t be trusted, period.

If you have a hard time believing what I say, all you have to do is read comments people leave on such misleading science articles. They almost all fall into two categories. It’s either “this is a crappy piece of science writing” or “mainstream scientists are incompetent impostors.” In both cases the commenters doubt the research in question is as valuable as it was presented.

If you can stomach it, check the I-Fucking-Love-Science facebook comment section every once in a while. It's eye-opening. On recent reports from the latest LHC anomaly, for example, you find gems like “I wish I had a job that dealt with invisible particles, and then make up funny names for them! And then actually get a paycheck for something no one can see! Wow!” and “But have we created a Black Hole yet? That's what I want to know.” Black Holes at the LHC were the worst hype I can recall in my field, and it still haunts us.

Another big concern with science coverage is its impact on the scientific community. I have spoken about this many times with my colleagues, but nobody listens even though it’s not all that complicated: Our attention is influenced by what ideas we are repeatedly exposed to, and all-over-the-news topics therefore bring a high risk of streamlining our interests.

Almost everyone I ever talked to about this simply denied such influence exists because they are experts and know better and they aren’t affected by what they read. Unfortunately, many scientific studies have demonstrated that humans pay more attention to what they hear about repeatedly, and we perceive something as more important the more other people talk about it. That’s human nature.

Other studies that have shown such cognitive biases are neither correlated nor anti-correlated with intelligence. In other words, just because you’re smart doesn’t mean you’re not biased. Some techniques are known to alleviate cognitive biases but the scientific community does not presently used these techniques. (Ample references eg in “Blind Spot,” by Banaji, Greenwald, and Martin.)

I have seen this happening over and over again. My favorite example is the “OPERA anomaly” that seemed to show neutrinos could travel faster than the speed of light. The data had a high statistical significance, and yet it was pretty clear from the start that the result had to be wrong – it was in conflict with other measurements.

But the OPERA anomaly was all over the news. And of course physicists talked about it. They talked about it on the corridor, and at lunch, and in the coffee break. And they did what scientists do: They thought about it.

The more they talked about it, the more interesting it became. And they began to wonder whether not there might be something to it after all. And if maybe one could write a paper about it because, well, we’ve been thinking about it.

Everybody who I spoke to about the OPERA anomaly began their elaboration with a variant of “It’s almost certainly wrong, but...” In the end, it didn’t matter they thought it was wrong – what mattered was merely that it had become socially acceptable to work on it. And every time the media picked it up again, fuel was added to the fire. What was the result? A lot of wasted time.

For physicists, however, sociology isn’t science, and so they don’t want to believe social dynamics is something they should pay attention to. And as long as they don’t pay attention to how media coverage affects their objectivity, publicity skews judgement and promotes a rich-get-richer trend.

Ah, then, you might argue, at least exposure will help you get tenure because your university likes it if their employees make it into the news. Indeed, the “any publicity is good” line I get to hear mainly as justification from people whose research just got hyped.

But if your university measures academic success by popularity, you should be very worried about what this does to your and your colleagues’ scientific integrity. It’s a strong incentive for sexy-yet-shallow, headline-worthy research that won’t lead anywhere in the long run. If you hunt after that incentive, you’re putting your own benefit over the collective benefit society would get from a well-working academic system. In my view, that makes you a hurdle to progress.

What, then, is the result of hype? The public loses: Trust in research. Scientists lose: Objectivity. Who wins? The news sites that place an ad next to their big headlines.

But hey, you might finally admit, it’s just so awesome to see my name printed in the news. Fine by me, if that's your reasoning. Because the more bullshit appears in the press, the more traffic my cleaning service gets. Just don’t say I didn’t warn you.

34 comments:

Harbles said...

Missing is consideration of vested interests vs pure search for knowledge for its own sake.
Journalists want eyeballs with the associated clicks or subscriptions. Outside interests have their own motivations, climate science controversy and the energy cartels for example. Scientists want research grant money and universities want alumni and government capital contributions for facilities, etc. The public wants new and better ways of doing things regardless of how much new and easy stuff they already have.
Stir it all together and you have the situation you have described above, nobody is happy.
If only we could all just go back to our childhood fascination with the wonders of the universe without worrying about who wants to make a buck off it.

Matthew Rapaport said...

Good points as always Dr. H. But what to do about it? Should exotic research results be kept secret until confirmed - possibly years later? Given that even conservative papers that do not hype tentative findings are as open to the press as to other researchers in the field and researchers do not control the media, what can be done? Stories can be "good stories" whether they are later confirmed or not because the ideas themselves are exciting and their intermediate results potentially useful even if the main idea behind them turns out to be wrong. String theory a good example. No solution to quantum gravity, but lots of useful math discoveries.

Matt Grayson said...

United Airlines, Bee (at least I think so)

--Matt

Andy K said...

"Any publicity is good publicity" = "Fake news is acceptable" = "I'm a lazy thinker"

brainmoleculermarketing.com said...

The science of science communication is like - zippo. In addition, we have zero coherent theories, let alone evidence of the cause of human behavior, let alone how media stuff (might) get translated to behavior. What also are the dependent variables.

Anecdotally we could say that, esp in the USA, more science communications correlates with more hostile and aggressive behaviors against science, including at the government level. Happy-talk platitudes obstruct problem-solving - around science communications. i would guess....lol

Uncle Al said...

Social intent debases science. Science is hugely diluted by fraud ("social sciences," National Institutes of Health) and egregiously diluted by welcoming the loudly unqualified (who will be "elevated" into competence).

http://blogs.discovermagazine.com/science-sushi/2017/04/22/why-i-march-every-day/
...March for Science? The real folks were busy working.

The price of US commonplace insulin has doubled, and still more. Science cannot save diabetics from the reasons, only from the need - and that forbidden by ethicists.

Pentcho Valev said...

Fraudulent hype is not new - Einstein and Eddington started it. Even nowadays it is constant overhype that keeps Einstein's obviously false theory alive. I could give convincing examples but you are a faithful Einsteinian and won't allow this to happen on your blog.

Louis Tagliaferro said...

Sabine said,

"In other words, just because you’re smart doesn’t mean you’re not biased."

"...it didn’t matter they thought it was wrong – what mattered was merely that it had become socially acceptable to work on it."

"Ah, then, you might argue, at least exposure will help you get tenure because your university likes it if their employees make it into the news.."


As a regular reader of your blog I know I am preaching to the choir, yet the underlying human behavior the statements above represents is a reason science progresses more slowly than it needs too. Lest we forget all of us are prone to such behavior and least likely to see, or admit it when applicable to ourselves.

Good science starts with continual review of one’s own behavior, bias, and conflicts of interest.

James Glyer said...

Maybe the only thing wrong with this piece is that it has as one of its tags: Rant. I do not see it as a rant, and it is not really aimed at being a rant.

Thanks again for good science exposition.

Unknown said...

The problem is the lack of consequence for the journalists; their business model essentially resembles spamming :- you put out a lot of messages at very low cost to yourself, and for each response you reap a relatively large reward. Journalists get page hits and ad income, or page hits and promotion prospects; spammers get a smaller number of responses but a larger payoff when they get your account details.

A response to the problem is to track and name the journalists and publications that are producing and hosting the rubbish content, so that poor reporting is penalised by a poor score on some arbitrary scale.

John Anderson said...

Robert P. Crease of SUNY Stony Brook has suggested a new position “Science Critics” as a bridge between scientists and the general audience. See his book “The Great Equations”. Also something in IOPSCIENCE.

If you’re a mystic, then sloppy press reporting encourages balony creation and other garbage like the movie “What the Bleep Do We Know?”

I would imagine the interest in the Opera faster-than-light neutrino result benefited from the Lotto effect: It’s probably wrong, but, hey, if it’s right, game on!

Alvin Toffler is famous for the phrase “information overload”. (Actually, he wasn’t the first.) But this implies, a finite capacity, at least individually to store and juggle information. I couldn’t find a relevant phrase I remember from someone like Toffler: finite content architecture. E. g. newspapers have only so much space. What they choose to print results in other things being pushed out completely, and so what is considered current events changes.

This would be a good topic for a panel discussion at the Aspen Center for Physics. Check it out.

Sabine Hossenfelder said...

Matt,

Oops, thanks, I've fixed that.

Sabine Hossenfelder said...

John,

Well, aren't journalists supposed to be critics? In any case, while it's clear that sloppy science writing isn't helping, I think much of the blame for this goes to scientists themselves.

Sabine Hossenfelder said...

Unknown,

Yes, as I suggested here, journalism would benefit from better quality control, not unlike scientific publishing, eg by simply stating whether or not fact-checking has taken place and whether or not the writer belongs to one or the other association. Having said that, however, in the end it's a question of supply and demand. As long as people want overhype, someone will produce overhype, and I think there's very little scientists can do about this.

What scientists *can* do something about, on the other hand, is to prevent that this hype affects the community. Again, we're making our lives too easy by just blaming science writers or funding bodies - it's our responsibility to get this right. Best,

B.

temp20 said...

Dear Sabine, you're good at writing and you're a good physicist. Can we PLEASE for once have a positive post, an enthusiastic one? Do you ever smile? Please stop bitching, stop rants, take a deep breath and relax. Life is beautiful! Thanks for all your work!

hardasgnials said...

The central issue is that there are no bottom limits a weak ego won't stoop to try and prop itself up.

Mind you, I just had to say that 'cause it’s just so awesome to see my name printed in the comments!


keep up the great writing,
s.

Phillip Helbig said...

Best faster-than-light OPERA paper: https://arxiv.org/abs/1110.2832

Dig that abstract!

Rob van Son (Not a physicist, just an amateur) said...

B.
A Nature editorial points out the political, partisan, divisions in scientific interests that will also influence journalism:
https://www.nature.com/news/how-to-judge-a-book-by-its-network-1.21771

It is based on this study:
https://www.nature.com/articles/s41562-017-0079

"Passionate disagreements about climate change, stem cell research and evolution raise concerns that science has become a new battlefield in the culture wars. We used data derived from millions of online co-purchases as a behavioural indicator for whether shared interest in science bridges political differences or selective attention reinforces existing divisions. Findings reveal partisan preferences both within and across scientific disciplines. "

Phillip Helbig said...

"Passionate disagreements about climate change, stem cell research and evolution raise concerns that science has become a new battlefield in the culture wars."

Really three quite different examples. Climate change: no serious scientist doubts that human-caused global warming is a fact and is a problem. (Most who deny it are the types who have a "God said it; I believe it; that settles it" bumper sticker on their pickup truck.) Stem-cell research: no debate about the science itself, but rather about what type of applications society should accept and what not. Evolution: only religious nuts doubt it.

Rob van Son (Not a physicist, just an amateur) said...

@Philip
"Really three quite different examples."

Could be, but those who accept or deny one tend to do it for all three of them together.

DocG said...

Excellent essay, Bee. Thank you.

But if anything fills the bill as far as over-hype is concerned, it's all the drama over "climate change," no? That's how I see it, anyhow. And no, I'm NOT an ultra-conservative who has a problem with "big government." In fact I'm very close to being a Sanders socialist.

I've studied the c.c. literature and as far as I can see it just doesn't add up. Looks to me like there are a great many very serious problems with this particular "science," and yet each and every day the media is exploding with how this place or that place, this creature or that creature is "endangered" by "climate change" and unless we do something NOW (now being anywhere from 30 years ago to today) we are all LOST!

I've been very curious for some time to learn more about your take on this issue, Bee. If anyone could change my mind on this it would be you. Any thoughts?

M_Malenfant said...

A good article worth further considerations.
As you already point out, the probably critical point is, that scientists themselves do little to avoid hypes or even push hype stories themselves.
Sometimes when reading a strange or seemingly misleadingly hyped story I try to look up the original press release to find out, what's really behind the story. Not rarely the overhype can already be found there. Worse, the real science / sientific progress is often hardly discernable between catchy terms. Sometimes even the abstract of the publication hardly reveals the scientific content of the work - at least for a scientifically trained but non-expert in the field reader.
I would not blame journalists too much for not investing heavily in deeper analysis - though some lower tone and perhaps leaving out topics where it is clear, that on journalistic level there is no serious information transport would be helpful. But that affects areas like politics, too.
I see a concerning gap between the scientific research frontier and what can be communicated (or taught) to a broader public. I appreciate your writing against this widening gap. I think this will be an important point not only for the advancement of science, but even the whole society. The March for Science may be an indication, that scientists become more aware of this.

jim_h said...

I have to believe medical research has been hurt by the endless cycle of hype, disappointment, disillusionment. Cancer has been cured every year since the 60s; how excited can the public be about funding the latest 'promising' study?

Phillip Helbig said...

"Could be, but those who accept or deny one tend to do it for all three of them together."

Denying evolution and anthropogenic global warming often go hand-in-hand, both being religiously motivated. As for stem-cell research, the situation is more complex. In the USA, atheists tend to favour it no matter (almost) what the goal is, perhaps because religious nuts think that it is the work of the devil. In more secular societies, it is really independent of the other two, many questioning it not because "man shouldn't play God" but because not everything which is possible is necessarily desirable. Also, similar to the case with "green genetic engineering" (golden rice and so on), in more secular societies people tend to take a larger view of potential risks and might err more on the side of caution. (There is also general scepticism about anything which allows one to feed more people as long as population growth continues. Malthus is still valid and feeding more now means that even more will starve when the next limit is reached.)

Ambi Valent said...

M_Malenfant wrote: "[...] I see a concerning gap between the scientific research frontier and what can be communicated (or taught) to a broader public. [...]"

I think if you replace 'can be communicated' to 'is communicated', you can see an even larger problem. Take both theories of relativity: The math is very exact, and very well in agreement with observation (except for the galactic level, where the gravity of visible matter cannot fully explain the velocity of matter orbiting the galactic cemter), but most of what's published just lists up where relativity shows different behavior than newtonian physics, but really sucks at explaining the effects. So the readers are pulled out of a world they understand and forced into one they don't.

I just re-read the comic "The Talk" on quantum computing and think that maybe equivalent attempts on other fields would not just be possible, but also beneficial.

Ambi Valent said...

@DocG

It would help if you actually said what you dislike about climate science. Is it whether the data is correct (i.e. temps going up, ice going down)? Or whether humans are a cause? Or is it whether or not climate change would be dangerous?

Dennis Pennis said...

The reason people should not accept the narrative that effectively places science journalism in the hot seat, is that the people that peddle it nearly always ark out of the scientil, fic community. There's never much in the way of evidence.

On the face of things, the arrow of evidence is in the opposite direction. It is the scientists that hype their research. Journalists are particularly dependent upon the information source, in the science journalism space, relative to journalism more generally.

By all 0ccam-lover best reasoning, it is entirely superfluous to introduce science writers into the fray here. We already know there is a problem in science of hype. A problem that worsens.

Starts with a Bang hypes with the best of them chasing those google clicks. Perhaps the author will shed light on the situation there.

If science writers hype, it is because they are invited to. The scientist involved in this, don't just demur to the bar with an authority vacuum in the wake. They ENCOURAGE it. At best. At worst they are themselves the HYPE.

Furthermore, the problems underlying all this are not superficial, and cannot be written down to some sort of fly in the publicity ointment. It's fundamental.

Rob van Son (Not a physicist, just an amateur) said...

@Dennis Pennis
"If science writers hype, it is because they are invited to."

Journalists have their own standards to live up too. One is to never write a story without checking with independend sources.

A newspaper writes that my children are the smartest and most beautiful children that ever lived, because I said so.

Who will be to blame when people are disappointed when they meet them? I or the journalists who did not do their work?

(This is obviously a hypothetical example, my children really are perfect :-) )

hardasgnials said...

@DocG

Depending on the "literature" you've read, yes maybe numbers don't add up. If you've looked instead at peer reviewed articles by climate scientists, (which have been available since the '70s) there's really no question as to what is happening. There are three things to know.

First, the Earth is going through an unprecedented fast rate of changes which include overall growing global temperatures. There are people who will say that Earth has been hotter before, which is very true, but then fail to express is that there's never been as drastic a change in as short a period of time (aside from maybe the mass extinction approximately 65 million years b.p.).

Now if that still doesn't seem bad enough, remember the acidification of the worlds oceans. As carbon dioxide goes into the atmosphere, some of it "sinks" into the seas (among other places, such as permafrost--see below). Soda water is great for an upset stomach, but see how long your goldfish lasts in a glass of it (please, don't really do that).

Finally, there's the permafrost feedback loop to consider. As the globe gets hotter, the permafrost across the Arctic will begin to melt, releasing tons of stored methane and other greenhouse gasses directly to the atmosphere, compounding the warming effect.

Obviously, it's a very complex model to simulate (that seems to be the peg Trump, Pruitt and his gang of climate change deniers seem to currently be hanging their hats on). While it may be impossible to nail down predictions with certainty, the outcomes are basic: we do nothing and either a)end up on a dying world or b)we get lucky and narrowly escape a catastrophic extinction level event; alternatively we could reduce the Earth's man made carbon emissions and either a)narrowly avoid a catastrophic extinction level event which we accomplished as a species, or b)life goes on as normal, but now we are completely independent of carbon based fuels so we no longer need to debate our energy requirement impact on the environment and the general health of the lands and oceans is improved.

One of those options should really stand out as something to avoid.

Ambi Valent said...

Sabine,

the original topic was research that was hyped, then believed to be correct by the public, only to later be rejected.

But there's also research that is actually successful, but wrongly believed by the public to have been hyped and then rejected. How high do you estimate the danger of the latter, and do you think journalism and/or education cam fix this? (A lot of this thread went out on this tangent)

johnduffieldblog said...

Good post Sabine. I've had my head down recently doing some writing which has involved a lot of digging into physics papers and the chronological history of hypotheses. And I utterly empathize with what you're saying. In fact I'd venture to suggest the situation is much much worse than people appreciate.

Henning Dekant said...

When the notorious OPERA FTL neutrino story was reported it prompted me to take a break from my usual subject quantum computing, because it was so egregiously ridiculous.

Unfortunately, I don't see a cure for the terrible pop science writing other than what you are doing gloriously, and I try to do with much less success. I.e. one needs to write about it.

The hype cycle in IT is much healthier than in science, because there is more and better media coverage. IT depts need to understand the evolving technologies and they pay for quality reporting. Also they understand that there is a hype cycle, and that this is just the way human nature, and the industry operates (which is why Gartner came up with the concept to begin with).

The problem is hype without push-back. Scott Aaronson provided push-back to D-Wave, and I loved to report on the controversy, while rooting for D-Wave. But I'd argue that this dynamic was actually good publicity for both. It balanced the expectations towards a realistic appreciation of what D-Wave accomplished, i.e. an actual quantum annealer, that does something useful with qubits, but is unlikely to demonstrate the kind of quantum supremacy we expect from a universal quantum gate model.

You perform an amazing public service as "clearing house" for all sorts of physics BS news. But we need more writers like you, and that's the crux of the problem, people of your caliber don't tend to go into journalism.

Alvin said...

Sabine,

I am surprised by your statement that loop quantum gravity is an even bigger disappointment than string theory. Given the relatively small size of the LQG community compared to the string theorists', there's been a lot of progress from the original work of Ashtekar reformulating GR in his eponymous variables to work of Rovelli, Smolin and others on quantization of geometric observables to spin networks and viable spinfoam amplitudes to loop quantum cosmology and its resolution of the Big Bang singularity with the Big Bounce scenario... It seems like a very active field.

So can you expand on your statement that LQG is a disappointment? Especially as it seems you're affiliated with the field (or were affiliated in the past?).

Sabine Hossenfelder said...

Alvin,

I am surprised that you believe I've worked on LQG. How is this? This is a genuine question. It's happened several times to me before that people think I have something to do with LQG, even though it's trivial to find out I never worked in the field. Not that this is specific to LQG - people also think I'm a string theorist if they don't like string theory and I work on particle dark matter if they don't like particle dark matter and so on. Whatever they dislike, I'm supposed to embody it.

Having said that, the disappointment is that nothing's come out of it. I have no clue what progress you are even talking about, other than producing papers which might or might not have something to do with nature. Where's the prediction for tensor modes? Does or doesn't it reproduce the effective limit? Does or doesn't it recover local Lorentz-invariance?

Do not mix up LQG with LQC - we both know it's not the same thing and I'm tired of this conflation.