Course: BIS 105 Environmental Biology

Unit 1 Learning Module

Experimental Design

Experimental Design

Controlled Experiments

In an old joke, a person claims that they are snapping their fingers "to keep tigers away," and justifies their behavior by saying, "See, it works!" While this experiment does not falsify the hypothesis "snapping your fingers keeps tigers away," it does not support the hypothesis either, because not snapping your fingers will also keep tigers away. It also follows that not snapping your fingers will not cause tigers to suddenly appear.

Are tigers really scared of snapping fingers, or is it more likely they are just not found in your neighborhood? Considering which of the hypotheses is more likely to be true can help you arrive at a valid answer. This principle, called Occam’s razor states that the explanation for a phenomenon should make as few assumptions as possible. In this case, the hypothesis “there are no tigers in my neighborhood to begin with” is more likely, because it makes the least number of assumptions about the situation.

To demonstrate a cause and effect hypothesis, an experiment must often show that, for example, a phenomenon occurs after a certain treatment is given to a subject, and that the phenomenon does not occur in the absence of the treatment.

One way of finding this out is to perform a controlled experiment. A controlled experiment generally compares the results obtained from an experimental sample against a control sample. The control sample is almost identical to the experimental sample except for the one variable whose effect is being tested. A good example would be a drug trial. The sample or group receiving the drug would be the experimental group, and the group receiving the placebo would be the control. A placebo is a form of medicine that does not contain the drug that is being tested.

Controlled experiments can be conducted when it is difficult to exactly control all the conditions in an experiment. In this case, the experiment begins by creating two or more sample groups that are similar in as many ways as possible, which means that both groups should respond in the same way if given the same treatment.

Once the groups have been formed, the experimenter tries to treat them identically except for the one variable that he or she wants to study (the independent variable). Usually neither the patients nor the doctor know which group receives the real drug, which serves to isolate the effects of the drug and allow the researchers to be sure the drug does work, and that the effects seen in the patients are not due to the patients believing they are getting better. This type of experiment is called a double blind experiment.

Controlled experiments can be carried out on many things other than people; some are even carried out in space! The wheat plants shown below are being grown in the International Space Station to study the effects of microgravity on plant growth. Researchers hope that one day enough plants could be grown during spaceflight to feed hungry astronauts and cosmonauts. The investigation also measured the amount of oxygen the plants can produce in the hope that plants could become a cheap and effective way to provide oxygen during space travel.

Spaceflight participant Anousheh Ansari holds a miniature wheat plant grown in the Zvezda Service Module of the International Space Station.

Experiments Without Controls

The term experiment usually means a controlled experiment, but sometimes controlled experiments are difficult or impossible to do. In this case researchers carry out natural experiments. When scientists conduct a study in nature instead of the more controlled environment of a lab setting, they cannot control variables such as sunlight, temperature, or moisture. Natural experiments therefore depend on the scientist’s observations of the system under study rather than controlling just one or a few variables as happens in controlled experiments.

For a natural experiment, researchers attempt to collect data in such a way that the effects of all the variables can be determined, and where the effects of the variation remains fairly constant so that the effects of other factors can be determined. Natural experiments are a common research tool in areas of study where controlled experiments are difficult to carry out. Examples include: astronomy -the study of stars, planets, comets, galaxies and phenomena that originate outside Earth's atmosphere, paleontology - the study of prehistoric life forms through the examination of fossils, and meteorology - the study of Earth’s atmosphere.

In astronomy it is impossible, when testing the hypothesis "suns are collapsed clouds of hydrogen", to start out with a giant cloud of hydrogen, and then carry out the experiment of waiting a few billion years for it to form a sun. However, by observing various clouds of hydrogen in various states of collapse, and other phenomena related to the hypothesis, such as the nebula shown below, researchers can collect data they need to support (or maybe falsify) the hypothesis.

An early example of this type of experiment was the first verification in the 1600s that light does not travel from place to place instantaneously, but instead has a speed that can be measured. Observation of the appearance of the moons of Jupiter were slightly delayed when Jupiter was farther from Earth, as opposed to when Jupiter was closer to Earth. This phenomenon was used to demonstrate that the difference in the time of appearance of the moons was consistent with a measurable speed of light.

The Helix nebula, located about 700 light-years away in the constellation Aquarius, belongs to a class of objects called planetary nebulae. Planetary nebulae are the remains of stars that once looked a lot like our sun. When sun-like stars die, they puff out their outer gaseous layers. These layers are heated by the hot core of the dead star, called a white dwarf, and shine with infrared and visible colors. Scientists can study the birth and death of stars by analyzing the types of light that are emitted from nebulae.

Natural Experiments

There are situations where it would be wrong or harmful to carry out an experiment. In these cases, scientists carry out a natural experiment, or an investigation without an experiment. For example, alcohol can cause developmental defects in fetuses, leading to mental and physical problems, through a condition called fetal alcohol syndrome.

Certain researchers want to study the effects of alcohol on fetal development, but it would be considered wrong or unethical to ask a group of pregnant women to drink alcohol to study its effects on their children. Instead, researchers carry out a natural experiment in which they study data that is gathered from mothers of children with fetal alcohol syndrome, or pregnant women who continue to drink alcohol during pregnancy. The researchers will try to reduce the number of variables in the study (such as the amount or type of alcohol consumed), which might affect their data. It is important to note that the researchers do not influence or encourage the consumption of alcohol; they collect this information from volunteers.

Field Experiments

Field experiments are so named to distinguish them from lab experiments. Field experiments have the advantage that observations are made in a natural setting rather than in a human-made laboratory environment. However, like natural experiments, field experiments can get contaminated, and conditions like the weather are not easy to control. Experimental conditions can be controlled with more precision and certainty in the lab.

An introduction to the Prince William Sound Field Experiment can be seen at http://www.youtube.com/watch?v=OpQngP9HmKo (4:49).

Predictions

A prediction is a statement that tells what will happen under specific conditions. It can be expressed in the form: If A is true, then B will also be true. Predictions are based on confirmed hypotheses shown to be true or not proved to be false.

For researchers to be confident that their predictions will be useful and descriptive, their data must have as few errors as possible. Accuracy is the measure of how close a calculated or measured quantity is to its actual value. Accuracy is closely related to precision, also called reproducibility or repeatability. Reproducibility and repeatability of experiments are cornerstones of scientific methods. If no other researcher can reproduce or repeat the results of a certain study, then the results of the study will not be accepted as valid. Results are called valid only if they are both accurate and precise.

A useful tool to help explain the difference between accuracy and precision is a target, shown in below. In this analogy, repeated measurements are the arrows that are fired at a target. Accuracy describes the closeness of arrows to the bulls eye at the center. Arrows that hit closer to the bulls eye are more accurate. Arrows that are grouped together more tightly are more precise.

Accuracy and Precision

A visual analogy of accuracy and precision. Left target: High accuracy but low precision; Right target: low accuracy but high precision. The results of calculations or a measurement can be accurate but not precise; precise but not accurate; neither accurate nor precise; or accurate and precise. A collection of bulls eyes right around the center of the target would be both accurate and precise.

Experimental Error

An error is a boundary on the precision and accuracy of the result of a measurement. Some errors are caused by unpredictable changes in the measuring devices (such as balances, rulers, or calipers), but other errors can be caused by reading a measuring device incorrectly or by using broken or malfunctioning equipment. Such errors can have an impact on the reliability of the experiment’s results; they affect the accuracy of measurements. For example, you use a balance to obtain the mass of a 100 gram block. Three measurements that you get are: 93.1 g, 92.0 g, and 91.8 g. The measurements are precise, as they are close together, but they are not accurate.

If the cause of the error can be identified, then it can usually be eliminated or minimized. Reducing the number of possible errors by careful measurement and using a large enough sample size to reduce the effect of errors will improve the reliability of your results.

Peer Review

After the data from an experiment a scientist will analyze the results and write a scientific report. To convey the information learned to other scientists, a researcher will submit their findings to a scientific journal. The editor of the journal forwards the proposed article to several other scientists with areas of similar expertise to review the findings. This process is called peer review. The purpose of peer review is to examine for any flaws or overlooked factors before publication. Reviewers can approve the work, accept with revisions, or reject it. Only after approval will a manuscript be published for others to read. The process is not without drawbacks, but overall is seen as a method to ensure only high quality work is published.