The hypothesis fallacy; or please explain to me why EVERY scientific experiment (whether hard or social) needs a hypothesis

Scientist

Bear with me here, because I’m about to prove how simplistic and primitive my mind is.  I need you all to help enlighten me.

Some high school students I know got an assignment to set up and complete an experiment.  Some of the experiments they came up with include looking at plant growth under different circumstances, or rust development under different circumstances, or human responses to certain stimuli.  This strikes me as a very sensible project for budding young scientists.

Plant growth experiment

My confusion arises from the fact that the students are required, as part of setting up the experiment, to include a hypothesis — or, in other words, they have to begin the experiment with an assumption about its outcome.  For example, a student measuring the effect of different fertilizers on otherwise identically situated plants, in addition to establishing the controls and variable(s), must also announce before starting the experiment that she believes that the more expensive fertilizers will work better.  Then, she’s supposed to see whether the data she collects supports this hypothesis.  (I.e., she proves or disproves her hypothesis.)

Here’s my problem:  I don’t understand why there is a scientific virtue to going into an experiment with a pre-determined conclusion.  It seems to me that it’s much more intelligent, in most, if not all cases, to go in with a question, and then to create an experiment that has sufficient controls to answer that question and that question alone.  My hostility to the hypothesis as a prerequisite arises because I suspect that a pre-determined hypothesis risks affecting the outcome.  Sherlock Holmes thought this too:

Sherlock Holmes by Sidney Paget

It is a capital mistake to theorize before you have all the evidence. It biases the judgment.

Exactly.  The scientist who has decided in advance that spending more on fertilizer results in better plant outcomes may subconsciously lavish a little more care or do things a little differently with the plant getting the good fertilizer.  The experiment is less likely to be tainted by the scientist’s biases if the scientist begins by asking “Which fertilizer is better?” rather than announcing “I think the more expensive fertilizers are better.”

Birth Control Pills

This is not just an idle question about high school projects.  I’ve noted the disdain that I have for Bay Area breast cancer studies that assume the culprit for the unusually high cancer rate in the Bay Area arises from too much bacon (evil factory farming) or from power lines (evil global warming).  One could just as easily announce that the hypothesis is that Bay Area women have high breast cancer rates because they get too much radiation from too many mammograms, or they have too many abortions (at too young an age), or they delay childbearing for too long, or they overuse of the Pill, etc.  If my study focused as narrowly on my assumptions, as these heavily Leftist studies focus on their assumptions, both studies would show that women who had done one or more of those things had higher cancer rates.

Establishing these almost random correlations (given the ridiculously biased parameters underpinning the various hypotheses) wouldn’t prove causation; instead, they would just prove that the scientist’s own prejudices forced the data down a narrow pathway.  Doesn’t it make more sense to find out about everything from diet, to environment, to lifestyle/sexual choices, and then, a la Sherlock Holmes, to see where the facts lead?

Global warming

This same “hypothesis fallacy,” for want of a better phrase, strikes me as one of the major problems with the whole global warming hysteria.  Various Leftists advanced the hypothesis that fossil fuels (which we know can contribute to pollution, and that Leftists believe give an unfair economic advantage to the First World) are evil, and then they set about proving their evil-ness.  If climate change is a genuine concern, wouldn’t it have made more sense to start with the question — “what’s going on?” — than to start with the answer — “Fossil fuels are changing our climate.”  After all, if your set-in-stone hypothesis isn’t even in the ball park, it means that your experiments are not only worthless, but they’ve also managed to ignore other, more relevant, data.

I understand that the hypothesis is a standard requirement for scientific experiments and has been since the Enlightenment.  I’ve explained, with a little help from Sherlock Holmes, why I think the hypothesis requirement taints, rather than advances, science.  Now that I’ve acquainted you with the contents of my brain, can you please explain to me why the scientific community is correct, and why Sherlock Holmes and I are wrong.

 

Be Sociable, Share!

Comments

  1. Mike Devx says

    You’re right, Book, the scientific method generally requires the statement of a hypothesis prior to the carrying out of the experiment and the accumulation of its data.
     
    I myself see no purpose to the hypothesis.  It needn’t taint the experiment, though.  A hypothesis can be proven WRONG by an experiment; but a hypothesis can never be proven correct by any experiment.  In these sadly unscientific days in which we live, this distinction is often lost, and that is where the taint creeps in.
     
    I suppose you could think of a hypothesis, if it is well formed, to be a “pre-theory”.  The hypothesis must survive multiple experiments designed to attack it, and each experiment must be repeatable and, when repeated, has the same results and the same conclusion must logically follow.  If so, then a hypothesis can become accepted as a theory.  So the purpose of the hypothesis can be seen to be a “proposed theory”.
     
    Far more important to me is that the data in an experiment must also be published, not just the conclusions.  And, if you do not publish your raw data in conjunction with your results, you ought to be laughed out of town.  I would note that global warming advocates do not like having to publish their raw data and almost never do, and they deserve to be objects of scorn and ridicule as a result.
     

  2. Michael Adams says

    Mr. Devx,
     
    I would add that the mingling of the terms hypothesis and theory is part of our muddled approach to science.  Experiments do test hypotheses, but the real test of a theory is whether it was able to predict the result of the experiment. A theory is a conceptual framework on which we can hang data, those already known, and those we suspect we shall discover. The periodic table, Darwinian evolution, even Lanarkian evolution, Jane Jacobs’s Systems of Survival, are all examples of theories, which can frequently be expressed as a curve on a graph. As new data are acquired, we plot them on the curve, or we  alter the graph, and, sometimes, as with global warming, we have to scrap most of the theory in its entirety.
    The hypothesis is not a big hairy deal, it is just, very simply, the reason for the experiment.
    BTW, Martel has a copy of Systems.
     
     

  3. 11B40 says

    Greetings:
     
    I’ve always thought of the hypothesis (besides keeping our Greek forebears in the process) as a form of intellectual discipline and an indicator to the reader about whence thou art cometh.  Sometimes refining the hypothesis is a lot more trouble than it may seem to warrant, especially in the pseudo-social sciences, but necessary, probably not. The science is in the data or observations and the conclusion.

  4. pst314 says

    “I don’t understand why there is a scientific virtue to going into an experiment with a pre-determined conclusion.”
    Because Thomas Kuhn said so. :-)  If a scientist tries to avoid framing hypotheses first, well, then he or she’s just Doing It Wrong according the man who never did any science himself.
    Now, I realize that it’s hard (virtually impossible?) to start without any expectations of what results an experiment might yield, but that’s not the same as asserting that the only right way to do science is to first frame and then test hypotheses.

  5. Freddie Sykes says

    For me, the scientific method entails observations which are used to form a hypothesis followed by the design of experiments to test the hypothesis followed by adjusting the hypothesis based on the observations from the experiment. Experiments are expensive which tends to limit the “let us try it and see what happens” approach from even the teaching of science.

    The fact that a hypothesis is not support by an experiment can still advance science. For example, the Michelson–Morley experiment performed in 1887 failed to shed any light on the existence of the generally accepted Theory of the Ether. Einstein used this experiment to generate his hypothesis of the Theories of Relativity which he advanced thru thought experiments and suggestions for additional experiments that were beyond his capacity to perform. Later observations, like the bending of light in a gravitation field which required a solar eclipse, were predicted by his hypothesis.

    Ideas on causes of cancer are not subject to experimentation but rather lend themselves to easily abused tool of statistical analysis. Ideas on cures for cancer do lend themselves to experimentation. Theories like evolution and climate change are based on hypothesis formed from observation.

    We do not have enough data or computing power to rigorously explain climate. Being confined to our one planet precludes experimentation.  One suggestion I have on predictive climate computer programs is to demand that they be able to predict the immediate past by using data up to some point in time. Not be able to do that should preclude them from receiving any large sums of grant money.    

  6. pst314 says

    “This same ‘hypothesis fallacy,’ for want of a better phrase, strikes me as one of the major problems with the whole global warming hysteria.”
    No, I see it as ideology trumping honesty.
    It would be perfectly reasonable to start with a hypothesis that human activity is driving global warming via greenhouse gases. However, an honest program of research would test the hypothesis not only by looking for evidence for and against anthropogenic causes, but would also consider and test all other possible causes. The failure exhibited is not one of premature hypothesis, but of a controlling ideology which tempts its partisans to ignore, suppress, and falsify inconvenient data.

  7. says

    AHHHH. I’m pulling out my hair! 
    To begin (and I’ll leave off at the beginning for now), just two things. FIrst, the QUESTION is the reason for the experiment. Questions generate hypotheses. 
    Second, there’s an important distinction to be made between a hypothesis in the sense of “I hypothesize such and so will happen if I fail to water these beans” (based on previous observations of how things work in the world) and a *statistical* hypothesis. I suspect that what the high school teacher is trying to move toward is the statistical sense (although I’m giving far too much credit to the teacher). A statistical hypothesis in the hard sciences is a testable hypothesis. More often than not– and simplistically– one tests the *null* hypothesis: there is no difference between measures of Y (the dependent variable) based on different conditions of X (the independent.) (Null hypothesis testing is contested in the soft sciences for reasons I have tried to but have never understood.)  It is this sense of hypothesis testing the teacher should be trying to introduce. 
    Scientific hypotheses therefore should not be “biased” and do not presuppose the outcome.
    Coming along with the concept of null hypothesis testing is the concept of *significance*. Things happen by chance or because of factors we haven’t identified and therefore cannot control. When can we legitimately call to groups (beans raised in the dark vs. those in the light) different, when we know random events occur? 
    I will now prattle on for a minute to illustrate these two points. Let’s talk about commercially available sunflower seeds (for bird seed) and light. The question is, do commercially available… need light to grow? 
    Null hypothesis: (All other things– heat, water, etc. being controlled for, i.e., no different between the two treatment groups) plants grown in light will grow to the three two-leaf stage in the same amount of time as plants grown in the dark. (Sunflower seedlings have sets of two leaves along a major stem.) Notice how I’ve already built into my hypothesis one very desecrate measurable dependent variable– time. 
    (To be very simplistic) I’ll accept my null hypothesis if … . For a high school science project of this sort, “If” could be as simple as “if 75% (far better than flip a coin chance) of seedlings in both the light & dark treatment achieve three pairs of leaves by the 7th day after sprouting. (Real significance testing in real science is a bit more stringent! But it’s here I’d ask you to think about drug trials. How sure do you want to be that the affect of Drug A *really* differed from a sugar-pill?) 
    Let’s look at two possible outcomes.
    1. By day 7 80/100 of sunflower seedlings grown in light have three pairs of leave, and 40/100 of the dark seedlings do. It doesn’t take any sophisticated stats to reject our null hypothesis and conclude that light is necessary for sunflowers to grow to the three leaf pair stage. From that, we may be justified in concluding that light is necessary for sunflower seedling growth.
    2.By day 7… Light 76; Dark, 74. Hummm. 
    It’s this dilemna that teaching null hypothesis testing is supposed to get at. We said, 75%. And so, again, we reject our null hypothesis. And begin to ask other questions. 
    Sorry. But thanks for the exercise– flawed though it may be. It’s been a while.
    And by the way– there are “albino” sunflower seedlings. They’re crazy. White as snow. 

  8. says

     
    What a great crowd!!  They’ve pretty much covered it above….
     
    “Science” properly understood doesn’t “prove” things — that’s Geometry.  Hypotheses are either well-supported, weakly supported, or rejected.  Once they’ve been supported by a range of experiments (IF the hypothesis is correct, the experiment will produce “A”, and it does….then, IF the hypothesis is correct, then this experiment will produce “B”, and it does….etc.) it may graduate to being called a “theory”.  Testing ought to continue, because some pretty widely-accepted theories have been demolished by finding a significant area where it fails to make accurate predictions.  In practice, if people in the field like the theory, they don’t bother with more testing unless some major counter-examples come to light….and sometimes not even then.
     
    It’s true that the hypothesis is often set up specifically to be rejected; but the null hypothesis is often  the opposite of the “guess” about what is really expected.  Because you can only know for sure that the hypothesis is “wrong”.  There are lots of uncontrolled variables that might account for a positive result.  For instance, when they were testing anadramous fish, they captured fish from upstream in Branch A and Branch B, took them back downriver and released them with smelly stuff stuff in their “noses”.  The hypothesis was that there would be no difference in their decisions about which branch they’d go up — fish from Branch A would return to it, and those from Branch B would return to it.  Naturally, that didn’t happen, so the hypothesis that their sense of smell made no difference was rejected, and the idea that they “sniffed” their way back to their natal stream was supported.
     
    Humans are often overly attached to their own ideas, and scientists aren’t immune.  Ego isn’t the only reason…others are various, including social, financial, and ideological.  Which pretty well explains Lysenkoism, as well as Catastrophic Anthropogenic Global Warming.  Each of these has wreaked enormous damage, and only one of them has been rejected as of this writing.
     

  9. JKB says

    I can confirm the bias in global warming research.  Way back in the early 1990s, I had a boss who’d just transferred from the government office that awarded grants for global warming research.  As it happens just a few months earlier, I had read about the biases and the problems with temperature measurements, etc.  So I said to him the problem was that research hypothesizing no warming wasn’t funded.  Rather than getting educated on how wrong I was, I was exposed to the naked reality.  His response was “Because it isn’t true”.  After a bit of discussion about how could it be science if you predetermined the outcome, I let the matter drop as we had to live and work closely together for the next year.  
     
    I ran into this when I was in school.  How to develop a legitimate hypothesis.  Sadly, such high school endeavors devolve into your predetermination since high school kids don’t have the proper observations from which to develop a testable hypothesis.  Most high school science projects being data collecting observations rather than testing hypotheses.   To take your fertilizer example.  The correct process would be to grow using two different fertilizers with the fertilizer being the only variable, then a hypothesis would be developed as to why one did better than the other, such as quality of ingredients, unique micronutrients, etc.  The hypothesis would suppose it was say, a micronutrient, so then an experiment would be developed with that micronutrient as the only variable.  Repeat until the variation in some element caused better growth.  In reality, in science class experiment time, students really only have time for the observation phase and perhaps come up with a plausible initial hypothesis.
     
    Sad part is, apparently, many scientist never figured out the observation, hypothesize, test, refine, repeat until current hypothesis yields results.  Science advances as much on the failed hypotheses as it does on the brilliant “success”.  Unfortunately, of late, we only reward the “success” and punish those thousands who methodically eliminated the other possible hypotheses with not only obscurity but also no tenure.  

  10. JKB says

    BTW, I found this 1980s talk by Bell Lab’s researcher Richard Hemming.  I wish I had read it way back in college
    It’s not right on point to this post but it is an excellent peer into research getting done:
    You and Your Research 
     
    I especially liked this part:
     
    Now for the matter of drive. You observe that most great scientists have tremendous drive. I worked for ten years with John Tukey at Bell Labs. He had tremendous drive. One day about three or four years after I joined, I discovered that John Tukey was slightly younger than I was. John was a genius and I clearly was not. Well I went storming into Bode’s office and said, “How can anybody my age know as much as John Tukey does?” He leaned back in his chair, put his hands behind his head, grinned slightly, and said, “You would be surprised Hamming, how much you would know if you worked as hard as he did that many years.” I simply slunk out of the office!
    What Bode was saying was this: “Knowledge and productivity are like compound interest.” Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity – it is very much like compound interest. I don’t want to give you a rate, but it is a very high rate. Given two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime. I took Bode’s remark to heart; I spent a good deal more of my time for some years trying to work a bit harder and I found, in fact, I could get more work done. I don’t like to say it in front of my wife, but I did sort of neglect her sometimes; I needed to study. You have to neglect things if you intend to get what you want done. There’s no question about this.

  11. Spartacus says

    Back in the day, my geometry teacher explained to the class how it was that we first discovered that houseflies’ sense of hearing is embedded in their wings, and not in “ears” as we have.  An experiment was performed in which a bunch of houseflies were divided into two groups.  The wings of the flies in the experimental group were carefully pulled off, whereas the flies in the control group were allowed to keep theirs.  Then the conductor of the experiment clapped his hands loudly right over both groups of flies.  The control group flew away, but the experimental group, not having heard the clap because they had no wings, stayed put.  Q.E.D.
     
    (I am pleased to report that my geometry teacher told this story with an amused twinkle in his eye.)

  12. says

    Hypothesis as the core of scientific method comes from the philosopher Karl Popper. Popper claimed for scientific theory to be scientific it must be falsifiable. His aim was to discredit claims that Marxism is scientific. Controversially Darwin’s Theory of Evolution does not count as science either by this criterion. 
    Thomas Kuhn, mentioned above, came along and upset the apple cart. Once Popper’s ideas were accepted, claims were made that Popper’s method had antecedents all the way back to Ancient Greece. Kuhn investigated the way scientific theories had emerged and found that change came through what he called Paradigm Shift: There is a consensus, then a new theory emerges, the older generation reject it, younger scientists are more sympathetic, and eventually the older generation dies out and the new theory becomes the new consensus. 
    So Book, your instinctive suspicion of Method is not without merit. 

  13. says

    I think you answered your own question, Book, when you talked about breast cancer research:  “Doesn’t it make more sense to find out about everything from diet, to environment, to lifestyle/sexual choices, and then, a la Sherlock Holmes, to see where the facts lead?”  How do you define “everything”?  Can you imagine the size (and cost!) of the project that tried to measure every possible variable all at once, including all of their possible sub-variables and interrelationships? 

    As a practical matter, each study chooses one, or a small group, of factors to study at a time.  Each study is designed to either eliminate that factor or demonstrate that it is worthy of further study.  In your little list you name diet, environment, and lifestyle/sexual choices, but there are an infinite number of other possible variables and each of the ones you named has a number of sub-variables.  Using hypotheses simply allows the researcher to attack this large mass of variables systematically.

    Note that a hypothesis is a question.  It asks whether a certain variable produces a certain, or any, outcome.  But putting the question in the form of a hypothesis forces a focus and a rigor onto the research.  Rather than a broad, impossible to answer question like “What causes breast cancer?” or “What causes global warming?” a hypothesis asks a narrower, answerable, question like “Is there a correlation between a gluten-rich diet and breast cancer?” or “Does this predictive model in fact predict changes in the weather over time?”  Hope this helps.

  14. Owen Glendower says

    Book, you are absolutely correct in your thinking, because the hypothesis should never be the FIRST step.  This is in fact a classic and well-documented ERROR.
     
    Good brief outline of the scientific method here: http://teacher.nsrl.rochester.edu/phy_labs/appendixe/appendixe.html
     
    From the intro: “As a famous scientist once said, ‘Smart people (like smart lawyers) can come up with very good explanations for mistaken points of view.’”
     
    A brief excerpt.  Note what the FIRST step is:
     
    The scientific method has four steps
     
    1. Observation and description of a phenomenon or group of phenomena.
    2. Formulation of an hypothesis to explain the phenomena. In physics, the hypothesis often takes the form of a causal mechanism or a mathematical relation.
    3. Use of the hypothesis to predict the existence of other phenomena, or to predict quantitatively the results of new observations.
    4. Performance of experimental tests of the predictions by several independent experimenters and properly performed experiments.

  15. says

    There is a difference between questions that can be addressed via controlled experiments and those that can’t be.
    If your hypothesis is “adding X% nickel to steel makes it stronger,” then you cook up one batch of steel with the nickel, another without, stick ‘em both in the testing machine and see which one breaks first.
    An analogous test for the hypothesis “there is more breast cancer in the Bay Area is because of all the bacon-eating and electric blankets” would be to take all the women in the area, clone them, and then move them to a location which would be exactly like the one they were in before, except that they would be prohibited from eating bacon or sleeping under electric blankets. (You’d also have to clone their husbands and children, of course.) Then just wait 50 years and compare the results of the two groups.
    Obviously you can’t do this, practically, legally, or morally. And it’s too simplistic to just compare the rates for electric blanket users vs non-users, because maybe the blanket users are (for example) more affluent and have fewer children, and number of children is the REAL causative factor.
    This is why multivariate analysis was created…techniques such as multiple regression can be used to statistically analyze a whole set of interrelated variables and consider the correlations between them, such as the relationship between affluence and religion and the relationship between religion and child-bearing, etc etc. In effect, you are statistically holding everything else constant to determine what the REAL effect of a particular variable like electric blankets might be.
    But there are problems with this approach too…you can never be sure there isn’t some variable you haven’t thought of. Maybe people who suffered some horrible childhood abuse like to have electric blankets for comfort, and the childhood abuse also has a direct impact on eating some kind of “comfort food,” and it’s the comfort food in question that REALLY causes breast cancer. If you didn’t include the abuse and food variables in your study, you’ll falsely blame the blankets.
    But put in TOO many variables, and spurious correlations will start showing up purely by chance.
    I’m pretty sure that there are a lot of social science researchers, maybe some medical researchers as well, who don’t really understand the limitations of the statistical methods they’re using…and by the time the study results flow through the university PR departments and the journalists of cluelessness, things can get very confusing & misleading indeed.

  16. JKB says

    This post reveals what in my old age I’ve come to realize about high school, it’s play acting.  American high schools are set up to simulate college and even the undergraduate is set up to simulate the Ph.D “environment”.  It is not a bad set up but since it isn’t admitted to in the front end, many students come to see the requirements as bogus.  A high school student is assigned a foreshortened task of conducting an experiment or writing a research paper or literary criticism.  But as used to be admitted up front, the high schooler and undergraduate do not have the knowledge or life experience to do those tasks for real.  The simulation isn’t bad but I think you’d get more buy in by admitting the shortcomings up front rather than stressing students out with all the big talk when the grading is going to occur at a much lower level.   Unfortunately, most instructors aren’t aware of the limitations and present the work as actually being something other than a simulation.  
     
    Add to that that in science you proceed from a firm world built on solid foundations to more and more uncertainty as you learn more and research more. There is an old geeky geodesy joke:
    When I was in elementary school they told me the world was round.  When I was in high school they said, “No, the world is a sphere.”  When I was in college, I learned that the earth was really a geoid.  Finally I though I have this earth shape pinned down.  I rushed to the library where I looked up this new word, geoid.  The definition of which is “earth-shaped”.  
     
    That joke kills as the worldwide forum on geodetic datums and ellipsoids.  

  17. MacG says

    Spartacus, Love it!  
     
    JKB18, This reminds me why a Phd is considered to be the acronym for Piled higher and deeper :)  I have heard it said that they know a certain fossil is of a certain age because of the layer of rock that is it encased in. But when asked for evidence of how old the rock is they will point to the fossils within it that layer.  An associated difficulty is that the geologic column exists nowhere in it’s entirety so this is why the circular reasoning is employed when it comes to fossil/strata dating. Are the ‘they’s from different disciplines or is what I have heard been oversimplified and perhaps flat out wrong??
     
    Book, in Biblical interpretation land (I suppose interpretation in general) such approaches are called eisegesis and exegesis.  Eisegesis is reading into the text what you want it to say, what is called ‘proof-texting’, the ‘Name it and claim it’ ‘grab it and have it’ ‘see it and seize it’ TV preachers are largely guilty of this. Whereas exegesis is drawing out of the text what is really there, letting it inform you.  Now lest we become letterest in interpretation, exegesis works best on the foundation of ‘a text without a context is a pretext’.  
     
    For example when Jesus said that ‘unless one eats my body and drinks my blood he cannot be my disciple’  was he really promoting a cannibal faith?   Even though he was a good Jew in whom they could find no false teaching (save for the ‘I Am’ claim) some thought he taught so.  The context is the great body of his teaching.  The use of parable and metaphor was through out.  He not only referred to himself as the bread of life but the living water of which if you drink you shall never thirst again, the gate through which people enter the Kingdom of G-d, the good shepherd etc.  The blood and body verse sounds more like unless you walk in my moccasins you cannot be my disciple.  Unless you consume me as I have consumed the Word of the Lord ( Eze 3, John 1 ), nourish yourself on me, empowered by me, you cannot be my disciple. Still, a hard teaching for wannabee followers.  Matthew 16 has a bit on the disciples struggles with interpretation (vs 5-12) and a bit on the cost of discipleship (vs 24-26).

  18. says

    Thank you, everyone, for those interesting and fact-filled responses.  It seems that my problem isn’t so much with the use of the scientific method as with the common misuse of that same method. 

    Don Quixote’s point is also a good one, which is that a wide open question becomes useless because it makes it impossible to assemble meaningful data.  I’d still prefer a narrow question (“Does what we eat affect breast cancer rates?”) to a careless and biased pre-answer (“Bacon is the culprit.”).

    One thing DQ’s response made me realize is that I automatically assumed that, because we have computers, it’s much easier to assemble and, even more importantly, sort data.  In the old days, 1,000 women describing their every meal for a year would generate an insane amount of data.  Nowadays, though, you can have them fill out a daily diary on the computer, that data is automatically put into a spreadsheet and, with the press of a button, you can sort the information in as many ways as you can imagine — age, ethnicity, types of foods, etc.

    The old hypothesis method, in a way, is predicated upon the great difficulty people used to have in making sense of lots of data.  That’s simply not the case anymore.  Nevertheless, whether you have narrow, hand-sorted data, or massive amounts of machine-sorted data, the primary rule is always going to be “garbage in, garbage out.”  You still have to ask intelligent questions.

    I’m starting to ramble, so I’ll sign off here before I expose even more of my ignorance.

  19. says

    Book, you are right that a question as broad as “Does what we eat” can be tested today, what with computers to churn the data.  But, there are at least two limitations to such an approach.  First, you won’t really be testing “what we eat”; you’ll be testing that subset eaten by a significant portion of the study group.  I don’t imagine you’d have a statistically significant sample of grits eaters in Marin, for example, or chow mein eaters in the hills of West Virginia.  Second, massaging the data in hundreds of different ways will lead you to a lot of dead ends.  At the usual 95% significance rate, one of every 20 statistically significant relationships will turn out to be meaningless. 

    Owen G’s point is an excellent one.  A bit of observation up front can help narrow the research substantially.  If your observations, or prior studies, have suggested that bacon might cause breast cancer, there is nothing wrong with a study to test that specific hypothesis, although the hypothesis would be stated much more objectively than you have it and the study would be objective as well.  The “null” hypothesis would be along the lines of “there is no relationship between the amount of bacon consumed and the frequency of breast cancer.”  That’s a perfectly legitimate question.

    Book, I suspect that more than distrusting the method, you distrust the people who are using it.  But don’t blame the hammer for being a hammer.  Hammers are useful implements.  Blame the thug swinging it for breaking windows.  Or, in an analogy you will probably appreciate in light of your recent posts, don’t blame the gun, blame the shooter.

  20. says

     
    GREAT story, Spartacus – I could have used your Geometry teacher back in the day!!
     
    Caedmon: when it comes to the origin of things, it’s true that by Popper’s standards, neither Darwinism nor any other story is “scientific”, and we would all do well to remember this. 
     
    On the other hand, MANY things within the larger Darwinian paradigm are indeed testable using experiments – some have been tested experimentally and have been supported.  Others, the ability of DNA mutations to produce useful information for example, have not been supported experimentally.  In fact, quite the opposite.  I don’t notice that any modification of the Darwinian story has taken place as a result of this, though.  Which should tell us something.

  21. Caped Crusader says

    Spartacus, what a wonderful explanation of the scientific method and how it works. It is too true and most do not have a twinkle in their eye, or even realize that they should have one! This is why @80% of scientific “findings” are later proven false. Most “breakthrough” scientific findings are born of random observation when something unusual happens and a person with a tenacious and scientific mind determines to find out “why” it happened. That was a brilliant teacher!

  22. Caped Crusader says

    The most difficult commodity to obtain in this world are true scientific facts not influenced, either purposely or unintentionally, by the inquirers predudices.

  23. says

    The discussions I recall of the Scientific Method (capital letters carefully in place) included one step that seems to be missing here:  Notice something about the world around you and develop an idea about what may be going on.  So, for example, you may notice that people wearing green shirts seem to win more foot races than people wearing red shirts.
    You might then propose an experiment where people are handed red or green shirts at random to see if that makes a difference in their performance in foot races.  Your hypothesis is that green shirts enable people to run faster.  (Or more generally, that the color of a shirt has an effect on a runner’s performance.)  (And of course, the null hypothesis is that performance is independent of shirt color.)
    But unless you had made that initial observation, you would have no reason to form your hypothesis in the first place.
    FWIW, Wikipedia seems to concur:
     
    “Four essential elements[34][35][36] of the scientific method[37] are iterations,[38][39] recursions,[40] interleavings, or orderings of the following:

    Characterizations (observations,[41] definitions, and measurements of the subject of inquiry)
    Hypotheses[42][43] (theoretical, hypothetical explanations of observations and measurements of the subject)[44]
    Predictions (reasoning including logical deduction[45] from the hypothesis or theory)
    Experiments[46] (tests of all of the above)”

     
    “Characterization” is the act of noticing something that hints at a possible cause-effect relationship.  Growing plants under a variety of different conditions could very easily be part of that step.
     
    However, it seems to me, if a teacher insists on a formal statement of a hypothesis, a student is perfectly justified in offering the null hypothesis:  “I hypothesize there will be no difference between the test group and the control group.”

  24. says

    A hypothesis was merely designed to lay out the framework of the idea, and to create an experiment to test that hypothesis’ conclusions. If things look solid, the hypothesis upgrades to a theory. If the theory is thus proven in an acceptable fashion, then it graduates to a law. But even while laws still exist and are used, even while better theories are out there, it’s mainly because the experimental body is 100% in favor of laws, but not 100% in favor of theories. Even though theories might explain reality better in some ways. (Relativity vs Newtonian gravity)
     
    The Left seeks to jump their hypothesis directly to a law, without any need for the intermediate steps and work required. That’s basically about it. A means to an end: their end being the utopia where all of us belong to them as slaves. To clean up the environment, of course.
     

  25. Ron19 says

    “My neighbor just told me she is going in tomorrow to get treated for breast cancer.
    There sure is a lot of that going around!”
    Congratulations.  You’ve just created a hypothesis.
    If that’s the end of it, you don’t need the scientific method, research, government or non-government funding, or any of that stuff.  You already have your foregone conclusion.
    On the other hand, if you did some research, and decided that Marin County has a higher breast cancer rate due to pill A than due to pill B, you now have the start of a plan for proving it, and becoming the next millionaire Erin Brockovich.  
    Now you say that all else being equal in Marin County, your hypothesis is that pill A has a 50% higher cancer rate than pill B.  Then you design an experiment and a test to prove the hypothesis.  If your experiment and test do not prove your hypothesis, then you start over.
    However, if from the time your neighbor told you about her breast cancer, you decide that pill A is the culprit and that you’re going to design a rigged experiment to show everybody that the problem is pill A, what you’re doing is not science, no matter how many times you say that it is.
    http://www.scientificmethod.com/bpg06_basicsteps.html

    There are charlatans just about everywhere you look.  I understand that you are a lawyer, Book.  Have you ever heard of a shady lawyer?  I have.  John Edwards comes to mind.   I know there are shady scientists.  The pharmaceutical companies I have worked with have fraud detection policies in effect, and aggressively investigate any hint of fraud that rears its head.  Their corporate lifeblood depends on doing honest research only.
    From what little I know of legal practice, I think a scientific hypothesis is to a scientist what a legal brief is to a lawyer.

  26. Ron19 says

     Odds and Ends

    It is a capital mistake to theorize before you have all the evidence. It biases the judgment.

    Sherlock Holmes never had “all” the evidence; usually he had just barely enough to arrest a villain that confessed or did himself in.  Case Solved.

    If you really want a light introduction to statistics, try “Statistics for Dummies.”

    The “science projects” that grade school and high school teachers assign to students are usually as authentic as the Baking Soda Volcano project.

    Most teachers K-12-post doc are clueless about statistics or scientific methods.  What they teach about it is worse than ignorance, and twice as dangerous.

    If your kid does get interested in science on some subject, there are lots of organizations that will work with them in various ways, such as 4-H or a nearby college.

    “A man who acts as his own lawyer has a fool for a client.”

    Rush Limbaugh is not quite as good as he says he is on explaining science, finances, unemployment insurance, etc., but he is dyn-o-mite with lefty politics and agendas.  And he does let you know about good sources of information on the other subjects, such as Roy Spencer on global warning.

    Once your interest is piqued about a topic or issue, you can wander around Amazon, doing searches and following book recommendations.

    Wikipedia may not be is not reliable, but it can suggest things to check out elsewhere on the net.

Trackbacks

Leave a Reply