This is one reason science has adopted the notion of a "blind" analysis in which the techniques used to analyze the data are developed without looking at the data. This makes it harder to tweak an analysis to discover what you're hoping to discover.
Usually blind analyses are developed with a mixture of simulated data and known "null" data. For example, if you were developing a blind analysis to look for Vulcan in telescope data, you might create an artificial telescope that renders simulated planets and use that to refine and characterize whatever imaging techniques you're using, and also look apply those techniques to data from real telescopes pointed far away from the sun to make sure they're not finding false positives.
In particle physics, a half century ago, before blind analyses were common, there were a lot of particle "discoveries" resulting from seeing some anomaly in the data, then introducing more and more cuts to refine the data until it was a huge statistically significant peak. But then these would disappear as soon as more data appeared, because ultimately what they'd found was statistical noise.
At least some of the evidence for the Higgs boson comes from blind analyses on LHC data.
I was just skimming through a paper by Dyson (the physicist not the inventory) describing some conjectures in the number theory of partitions and he says: I hope these... do not suffer the fate of the planet Vulkan. I guess he meant that they are attractive at first glance but ultimately amount to nothing.
Cannot remember ever coming across Vulkan before this, ever.
Agreeing with the general point but specifically: proposing a new planet in an era when new planets were being discovered periodically is more conservative than upending all of physics. So people were following Occam’s razor there.
Dark matter would be a much better example of a possible modern-day Planet Vulcan. Although I don't think anyone has claimed to directly observe dark matter.
I don't understand what makes this a cautionary tale. Observation didn't quite match theory. An entirely reasonable hypothesis was proposed: an undiscovered planet. Attempts were made to prove it by discovering said planet. They failed. Eventually, a genius came up with a new theory that explained the observation, and a whole lot of other stuff besides.
This all seems like science and scientists getting the job done, a tale of scientific discovery and triumph, not cautionary.
"They failed." is incorrect. For several decades the consensus was that they'd succeeded, with a few dissenting voices. It wasn't until Einstein came up with a theory that perfectly explained the precession of Mercury that it became obvious that the observations were bogus. That's the part which should make you cautious, both when believing other people's observations and when making observations yourself.
IIRC, there was the Planet X search ~100 years ago. Much later it was pointed out either there was a miscalculation of Neptune's mass or a slight issue with Newton's theory. Since then it we know that Planet X does not exist.
Now we have the search for Planet 9, details for both:
This is one reason science has adopted the notion of a "blind" analysis in which the techniques used to analyze the data are developed without looking at the data. This makes it harder to tweak an analysis to discover what you're hoping to discover.
Usually blind analyses are developed with a mixture of simulated data and known "null" data. For example, if you were developing a blind analysis to look for Vulcan in telescope data, you might create an artificial telescope that renders simulated planets and use that to refine and characterize whatever imaging techniques you're using, and also look apply those techniques to data from real telescopes pointed far away from the sun to make sure they're not finding false positives.
In particle physics, a half century ago, before blind analyses were common, there were a lot of particle "discoveries" resulting from seeing some anomaly in the data, then introducing more and more cuts to refine the data until it was a huge statistically significant peak. But then these would disappear as soon as more data appeared, because ultimately what they'd found was statistical noise.
At least some of the evidence for the Higgs boson comes from blind analyses on LHC data.
Holy Bader Meinhof effect in action.
I was just skimming through a paper by Dyson (the physicist not the inventory) describing some conjectures in the number theory of partitions and he says: I hope these... do not suffer the fate of the planet Vulkan. I guess he meant that they are attractive at first glance but ultimately amount to nothing.
Cannot remember ever coming across Vulkan before this, ever.
Agreeing with the general point but specifically: proposing a new planet in an era when new planets were being discovered periodically is more conservative than upending all of physics. So people were following Occam’s razor there.
Dark matter would be a much better example of a possible modern-day Planet Vulcan. Although I don't think anyone has claimed to directly observe dark matter.
I don't understand what makes this a cautionary tale. Observation didn't quite match theory. An entirely reasonable hypothesis was proposed: an undiscovered planet. Attempts were made to prove it by discovering said planet. They failed. Eventually, a genius came up with a new theory that explained the observation, and a whole lot of other stuff besides.
This all seems like science and scientists getting the job done, a tale of scientific discovery and triumph, not cautionary.
"They failed." is incorrect. For several decades the consensus was that they'd succeeded, with a few dissenting voices. It wasn't until Einstein came up with a theory that perfectly explained the precession of Mercury that it became obvious that the observations were bogus. That's the part which should make you cautious, both when believing other people's observations and when making observations yourself.
Is that true? That's not how I understood it, and sources I'm finding online seem to agree e.g. https://aas.org/posts/news/2017/08/month-astronomical-histor... and https://research-repository.st-andrews.ac.uk/bitstream/handl....
IIRC, there was the Planet X search ~100 years ago. Much later it was pointed out either there was a miscalculation of Neptune's mass or a slight issue with Newton's theory. Since then it we know that Planet X does not exist.
Now we have the search for Planet 9, details for both:
https://en.wikipedia.org/wiki/Planets_beyond_Neptune
I am hoping its a grapefruit sized black hole