Do you remember Gilligan’s Island? For many people of a certain age, “The Professor” was our first impression of what a scientist was like. Even in those simpler times, though, you probably couldn’t find anyone like the professor; a jack of all trades, he sort of knew everything about everything (except, apparently, how to make a boat).
Real scientists tend to hyper-specialize. Getting grant money, publication pages, and just advancing the state of the art means that you get more and more focused on more obscure things. It is getting to the point that two scientists in the same field may not be able to really understand each other. You see the same thing in engineering to some degree. Not many digital designers can talk about the frequency dependence of Early effect in bipolar transistors, but not many device gurus can talk intelligently about reservation techniques for superscalar CPUs.
There’s now a website that lets you guess if a physics paper title is real or if it made up jibberish. The site, snarXiv, gets the real titles from arXiv, the site that contains many preprint papers. For example, we were asked to guess if “Brane Worlds with Bolts” was a real paper or if it was “Anthropic Approaches to the Flavor Problem.” (For the record, it was the one about branes.) Give it a whirl!
Remember also, there is no monolithic “scientist”…or for that matter, “science”. There are theoretical scientists, application scientists… process scientists, bench monkeys, etc.; all of which are classically a “scientist”, but outside of their realms are poorly equipped. The applications and process scientists are more of an engineer or technician for an established discipline more than breaking truly new ground in “the art” – and they are mostly found in the corporate world refining an industrial process.
Corollary to this: the term for a scientist outside of his or her specialty is “layperson.” As a software developer, it’s scary how often I stumble across embedded systems devs who–while sneering at career web devs who couldn’t write assembly to save their lives–proceed to utterly bungle writing modern, scalable multiuser applications.
Conversely, I’ve seen web dev types roll their eyes at bitcounting dinosaurs while failing to shoehorn their latest project into the micro controller of the week.
Humility and self-awareness are valuable, in some ways more than the actual technical skills.
Even inside their respective discipline.
Go to any specialty conference & at least one talk you’ll see a heated debate about some frontier of the discipline.
WHY people hold an opinion is less important than what the actual opinion is.
It doesn’t matter if you have 3 PhDs 2 Nobel prizes and a Fields medal, if you use shit evidence, your conclusion is worthless.
Currently I am working on my PhD in theoretical particle physics and all what Williams said is absolutely true. Although my colleague works in the same realm, I have absolutely no idea what exactly he is doing. The same may be true for him as well. With that said, even I have problems to distinguish between real publications on arXiv and fake publications on snarXiv. The latest publications remind me a bit of gibberish technobabble :)
In my younger days I could make up stores how things worked and people believed me.
I was so good at it and so good at doing things right and even applying what I had said that I think god got jealous.
And he has take his revenge. My Kids (now young adults) no longer believe anything that I say at all. And I mean nothing…
Like I said God took his revenge on me and gave me two gifted Boys that know better.
Nothing like having kids for forcing to improve your skills– or else upgrade your bullshit– one or the other! ;-)
“There’s now a website that lets you guess if a physics paper title is real or if it made up jibberish.”
Doesn’t that just tend to make the situation worse? ;-)
And I read somewhere recently that 2/3 of research cannot be reproduced, because the experimental procedure is poorly described. Tax dollars at work.
IMO this is exactly the wrong attitude. Assuming what you say is true* framing it as a waste of tax dollars intuitively leads to the conclusion of tying grant money to oversight from the public, or the public by way of elected officials. In other words, I believe it leads to putting research under the direction of those least qualified to assess its worth.
I believe the root of the problem is at least in part the “publish or perish” expectation. It produces perverse incentives to compromise quality of research in favor of speedy results, or chase what’s publishable rather than most valuable to society.
* this isn’t snark. I’ve heard it said about psychology, but not science in general and don’t have time to do real research
In a rush to get stuff published, unspeakable “crimes” are committed. They should be funded, but not on the basis of the number of papers they produce.
Agreed, I’ve read a ton of these documents and there isn’t anything reproducible unless you happened to have access to their lab or maybe enough money to fund their research
It depends greatly on the field of study. Physics for example has had it’s fair share of overturned results so at least in the particle physics world 95% confidence is not confident enough, they require 5 sigma or 99.99994% for papers on new particles or effects.
Feeding into the 60% figure is the fact that almost no funding goes to verifying results. You might get a couple meta-studies that just read a couple hundred papers and call out poor materials & methods sections but very few people are out there actually rerunning experiments. That’s not to say the results of those 60% papers are for sure wrong, only that there is no way to exactly replicate the poorly described experiment. Fortunately few papers have civilization shattering consequences if they’re right or wrong.
Equally luckily, if a researcher does run the same experiment & get different results, scientists correct the mistakes.
“Feeding into the 60% figure is the fact that almost no funding goes to verifying results.”
Richard Feymann wrote about the problem that causes. Funding is available for “building on” someone else’s research, not double checking it. So, “research” continues and continues, and Primary Investigators don’t know that their thesis is based on erroneous assumptions that have been handed down to them.
And it used to be that failed research was not published, making it difficult for peers to review and maybe correct the research, or outright duplicate it not knowing a previous attempt had failed, or why it failed.
But I do like the bumper sticker that says,
“Back off Man!
I’m a scientist!”
B^)
The professor knew how to build a boat, but he also knew the probability of a threesome with ginger Lynn and Mary Anne. If he just waited long enough he could have both of them individually and as a menage et trois. Best guess? He calculated the likelihood of being stranded on a exotic island with two beautiful women, along with apparent abundant food, fresh water, shelter and escape from working.
Yeah, I’d be focused on inventing a distillery and make shift condoms too! Lol!