-
Notifications
You must be signed in to change notification settings - Fork 3
/
s15-02-experiments.html
204 lines (187 loc) · 22.3 KB
/
s15-02-experiments.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<link href="shared/bookhub.css" rel="stylesheet" type="text/css">
<title>Experiments</title>
</head>
<body>
<div id=navbar-top class="navbar">
<div class="navbar-part left">
<a href="s15-01-focus-groups.html"><img src="shared/images/batch-left.png"></a> <a href="s15-01-focus-groups.html">Previous Section</a>
</div>
<div class="navbar-part middle">
<a href="index.html"><img src="shared/images/batch-up.png"></a> <a href="index.html">Table of Contents</a>
</div>
<div class="navbar-part right">
<a href="s15-03-ethnomethodology-and-conversat.html">Next Section</a> <a href="s15-03-ethnomethodology-and-conversat.html"><img src="shared/images/batch-right.png"></a>
</div>
</div>
<div id="book-content">
<div class="section" id="blackstone_1.0-ch12_s02" condition="start-of-chunk" version="5.0" lang="en">
<h2 class="title editable block">
<span class="title-prefix">12.2</span> Experiments</h2>
<div class="learning_objectives editable block" id="blackstone_1.0-ch12_s02_n01">
<h3 class="title">Learning Objectives</h3>
<ol class="orderedlist" id="blackstone_1.0-ch12_s02_l01">
<li>Define <em class="emphasis">experiment</em>.</li>
<li>Distinguish “true” experiments from preexperimental designs.</li>
<li>Identify the core features of true experimental designs.</li>
<li>Describe the difference between an experimental group and a control group.</li>
<li>Identify and describe the various types of true experimental designs.</li>
<li>Identify and describe the various types of preexperimental designs.</li>
<li>Name the key strengths and weaknesses of experiments.</li>
<li>Define <em class="emphasis">internal validity</em> and <em class="emphasis">external validity</em>.</li>
</ol>
</div>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p01">Experiments are an excellent data collection strategy for those wishing to observe the consequences of very specific actions or stimuli. Most commonly a quantitative research method, experiments are used more often by psychologists than sociologists, but understanding what experiments are and how they are conducted is useful for all social scientists, whether they actually plan to use this methodology or simply aim to understand findings based on experimental designs. An <span class="margin_term"><a class="glossterm">experiment</a><span class="glossdef">A method of data collection designed to test hypotheses under controlled conditions.</span></span> is a method of data collection designed to test hypotheses under controlled conditions. Students in my research methods classes often use the term <em class="emphasis">experiment</em> to describe all kinds of empirical research projects, but in social scientific research, the term has a unique meaning and should not be used to describe <em class="emphasis">all</em> research methodologies.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p02">Several kinds of experimental designs exist. In general, designs considered to be “true experiments” contain three key features: independent and dependent variables, pretesting and posttesting, and experimental and control groups. In the <span class="margin_term"><a class="glossterm">classic experiment</a><span class="glossdef">The effect of a stimulus is tested by comparing an experimental group to a control group.</span></span>, the effect of a stimulus is tested by comparing two groups: one that is exposed to the stimulus (the <span class="margin_term"><a class="glossterm">experimental group</a><span class="glossdef">The group of participants who receive the stimulus in an experiment.</span></span>) and another that does not receive the stimulus (the <span class="margin_term"><a class="glossterm">control group</a><span class="glossdef">The group of participants who do not receive the stimulus in an experiment.</span></span>). In other words, the effects of an independent variable upon a dependent variable are tested. Because the researcher’s interest lies in the effects of an independent variable, she must measure participants on the dependent variable before and after the independent variable (or stimulus) is administered. Thus pretesting and posttesting are both important steps in a classic experiment.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p03">One example of experimental research can be found in Shannon K. McCoy and Brenda Major’s (2003)<span class="footnote" id="blackstone_1.0-fn12_011">McCoy, S. K., & Major, B. (2003). Group identification moderates emotional response to perceived prejudice. <em class="emphasis">Personality and Social Psychology Bulletin, 29,</em> 1005–1017.</span> study of people’s perceptions of prejudice. In one portion of this multifaceted study, all participants were given a pretest to assess their levels of depression. No significant differences in depression were found between the experimental and control groups during the pretest. Participants in the experimental group were then asked to read an article suggesting that prejudice against their own racial group is severe and pervasive, while participants in the control group were asked to read an article suggesting that prejudice against a racial group <em class="emphasis">other than</em> their own is severe and pervasive. Upon measuring depression scores during the posttest period, the researchers discovered that those who had received the experimental stimulus (the article citing prejudice against their same racial group) reported greater depression than those in the control group. This is just one of many examples of social scientific experimental research.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p04">In addition to the classic experimental design, there are two other ways of designing experiments that are considered to fall within the purview of “true” experiments (Babbie, 2010; Campbell & Stanley, 1963).<span class="footnote" id="blackstone_1.0-fn12_012">Babbie, E. (2010). <em class="emphasis">The practice of social research</em> (12th ed.). Belmont, CA: Wadsworth; Campbell, D., & Stanley, J. (1963). <em class="emphasis">Experimental and quasi-experimental designs for research</em>. Chicago, IL: Rand McNally.</span> They are the Solomon four-group design and the posttest-only control group design. In the former, four groups exist. Two groups are treated as they would be in a classic experiment. Another group receives the stimulus and is then given the posttest. The remaining group does not receive the stimulus but is given the posttest. <a class="xref" href="#blackstone_1.0-ch12_s02_t01">Table 12.2 "Solomon Four-Group Design"</a> illustrates the features of each of the four groups in the Solomon four-group design.</p>
<div class="table block" id="blackstone_1.0-ch12_s02_t01" frame="all">
<p class="title"><span class="title-prefix">Table 12.2</span> Solomon Four-Group Design</p>
<table cellpadding="0" cellspacing="0">
<thead>
<tr>
<th></th>
<th>Pretest</th>
<th>Stimulus</th>
<th>Posttest</th>
<th>No stimulus</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong class="emphasis bold">Group 1</strong></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr>
<td><strong class="emphasis bold">Group 2</strong></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr>
<td><strong class="emphasis bold">Group 3</strong></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr>
<td><strong class="emphasis bold">Group 4</strong></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
</tbody>
</table>
</div>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p05">Finally, the posttest only control group is also considered a “true” experimental design though it lacks any pretest group. In this design, participants are assigned to either an experimental or a control group. Individuals are then measured on some dependent variable following the administration of an experimental stimulus to the experimental group. In theory, as long as the control and experimental groups have been determined randomly, no pretest is needed.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p06">Time, other resources such as funding, and even one’s topic may limit a researcher’s ability to conduct a true experiment. For researchers in the medical and health sciences, conducting a true experiment could require denying needed treatment to patients, which is a clear ethical violation. Even those whose research may not involve the administration of needed medications or treatments may be limited in their ability to conduct a classic experiment. In social scientific experiments, for example, it might not be equitable or ethical to provide a large financial or other reward only to members of the experimental group. When random assignment of participants into experimental and control groups is not feasible, researchers may turn to a <span class="margin_term"><a class="glossterm">preexperimental design</a><span class="glossdef">Experimental design used when random assignment of participants into experimental and control groups is not feasible.</span></span> (Campbell & Stanley, 1963).<span class="footnote" id="blackstone_1.0-fn12_013">Campbell, D., & Stanley, J. (1963). <em class="emphasis">Experimental and quasi-experimental designs for research</em>. Chicago, IL: Rand McNally.</span> However, this type of design comes with some unique disadvantages, which we’ll describe as we review the preexperimental designs available.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p07">If we wished to measure the impact of some natural disaster, for example, Hurricane Katrina, we might conduct a preexperiment by identifying an experimental group from a community that experienced the hurricane and a control group from a similar community that had not been hit by the hurricane. This study design, called a <span class="margin_term"><a class="glossterm">static group comparison</a><span class="glossdef">An experiment that includes a comparison control group that did not experience the stimulus; it involves experimental and control groups determined by a factor or factors other than random assignment.</span></span>, has the advantage of including a comparison control group that did not experience the stimulus (in this case, the hurricane) but the disadvantage of containing experimental and control groups that were determined by a factor or factors other than random assignment. As you might have guessed from our example, static group comparisons are useful in cases where a researcher cannot control or predict whether, when, or how the stimulus is administered, as in the case of natural disasters.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p08">In cases where the administration of the stimulus is quite costly or otherwise not possible, a <span class="margin_term"><a class="glossterm">one-shot case study</a><span class="glossdef">An experiment that contains no pretest and no control group.</span></span> design might be used. In this instance, no pretest is administered, nor is a control group present. In our example of the study of the impact of Hurricane Katrina, a researcher using this design would test the impact of Katrina only among a community that was hit by the hurricane and not seek out a comparison group from a community that did not experience the hurricane. Researchers using this design must be extremely cautious about making claims regarding the effect of the stimulus, though the design could be useful for exploratory studies aimed at testing one’s measures or the feasibility of further study.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p09">Finally, if a researcher is unlikely to be able to identify a sample large enough to split into multiple groups, or if he or she simply doesn’t have access to a control group, the researcher might use a <span class="margin_term"><a class="glossterm">one-group pre-/posttest</a><span class="glossdef">An experiment in which pre- and posttests are both taken but there is no control group.</span></span> design. In this instance, pre- and posttests are both taken but, as stated, there is no control group to which to compare the experimental group. We might be able to study of the impact of Hurricane Katrina using this design if we’d been collecting data on the impacted communities prior to the hurricane. We could then collect similar data after the hurricane. Applying this design involves a bit of serendipity and chance. Without having collected data from impacted communities prior to the hurricane, we would be unable to employ a one-group pre-/posttest design to study Hurricane Katrina’s impact.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p10"><a class="xref" href="#blackstone_1.0-ch12_s02_t02">Table 12.3 "Preexperimental Designs"</a> summarizes each of the preceding examples of preexperimental designs.</p>
<div class="table block" id="blackstone_1.0-ch12_s02_t02" frame="all">
<p class="title"><span class="title-prefix">Table 12.3</span> Preexperimental Designs</p>
<table cellpadding="0" cellspacing="0">
<thead>
<tr>
<th></th>
<th>Pretest</th>
<th>Posttest</th>
<th>Experimental group</th>
<th>Control group</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong class="emphasis bold">One-shot case study</strong></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr>
<td><strong class="emphasis bold">Static group comparison</strong></td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr>
<td><strong class="emphasis bold">One-group pre-/posttest</strong></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
</tbody>
</table>
</div>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p11">As implied by the preceding examples where we considered studying the impact of Hurricane Katrina, experiments do not necessarily need to take place in the controlled setting of a lab. In fact, many applied researchers rely on experiments to assess the impact and effectiveness of various programs and policies. You might recall our discussion of the police experiment described in <a class="xref" href="blackstone_1.0-ch02#blackstone_1.0-ch02">Chapter 2 "Linking Methods With Theory"</a>. It is an excellent example of an applied experiment. Researchers did not “subject” participants to conditions in a lab setting; instead, they applied their stimulus (in this case, arrest) to some subjects in the field and they also had a control group in the field that did not receive the stimulus (and therefore were not arrested).</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p12">Finally, a review of some of the strengths and weaknesses of experiments as a method of data collection is in order. A strength of this method, particularly in cases where experiments are conducted in lab settings, is that the researcher has substantial control over the conditions to which participants are subjected. Experiments are also generally easier to replicate than are other methods of data collection. Again, this is particularly true in cases where an experiment has been conducted in a lab setting.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p13">As sociologists, who are especially attentive to how social <em class="emphasis">context</em> shapes social life, are likely to point out, a disadvantage of experiments is that they are rather artificial. How often do real-world social interactions occur in the same way that they do in a lab? Experiments that are conducted in applied settings may not be as subject to artificiality, though then their conditions are less easily controlled. Experiments also present a few unique concerns regarding validity. Problems of <span class="margin_term"><a class="glossterm">external validity</a><span class="glossdef">The extent to which the conditions of an experiment adequately represent those of the world outside the boundaries of the experiment.</span></span> might arise when the conditions of an experiment don’t adequately represent those of the world outside the boundaries of the experiment. In the case of McCoy and Major’s (2003)<span class="footnote" id="blackstone_1.0-fn12_014">McCoy, S. K., & Major, B. (2003). Group identification moderates emotional response to perceived prejudice. <em class="emphasis">Personality and Social Psychology Bulletin, 29,</em> 1005–1017.</span> research on prejudice described earlier in this section, for example, the questions to ask with regard to external validity are these: Can we say with certainty that the stimulus applied to the experimental group resembles the stimuli that people are likely to encounter in their real lives outside of the lab? Will reading an article on prejudice against one’s race in a lab have the same impact that it would outside of the lab? This is not to suggest that experimental research is not or cannot be valid, but experimental researchers must always be aware that external validity problems can occur and be forthcoming in their reports of findings about this potential weakness. Concerns about <span class="margin_term"><a class="glossterm">internal validity</a><span class="glossdef">The extent to which we can be confident that an experiment’s stimulus actually produced the observed effect or whether something else caused the effect.</span></span> also arise in experimental designs. These have to do with our level of confidence about whether the stimulus actually produced the observed effect or whether some other factor, such as other conditions of the experiment or changes in participants over time, may have produced the effect.</p>
<p class="para editable block" id="blackstone_1.0-ch12_s02_p14">In sum, the potential strengths and weaknesses of experiments as a method of data collection in social scientific research include the following:</p>
<div class="table block" id="blackstone_1.0-ch12_s02_t03" frame="all">
<p class="title"><span class="title-prefix">Table 12.4</span> Strengths and Weaknesses of Experimental Research</p>
<table cellpadding="0" cellspacing="0">
<thead>
<tr>
<th>Strengths</th>
<th>Weaknesses</th>
</tr>
</thead>
<tbody>
<tr>
<td>Researcher control</td>
<td>Artificiality</td>
</tr>
<tr>
<td>Reliability</td>
<td>Unique concerns about internal and external validity</td>
</tr>
</tbody>
</table>
</div>
<div class="key_takeaways editable block" id="blackstone_1.0-ch12_s02_n02">
<h3 class="title">Key Takeaways</h3>
<ul class="itemizedlist" id="blackstone_1.0-ch12_s02_l02">
<li>Experiments are designed to test hypotheses under controlled conditions.</li>
<li>True experimental designs differ from preexperimental designs.</li>
<li>Preexperimental designs each lack one of the core features of true experimental designs.</li>
<li>Experiments enable researchers to have great control over the conditions to which participants are subjected and are typically easier to replicate than other methods of data collection.</li>
<li>Experiments come with some degree of artificiality and may run into problems of external or internal validity.</li>
</ul>
</div>
<div class="exercises editable block" id="blackstone_1.0-ch12_s02_n03">
<h3 class="title">Exercises</h3>
<ol class="orderedlist" id="blackstone_1.0-ch12_s02_l03">
<li>Taking into consideration your own research topic of interest, how might you conduct an experiment to learn more about your topic? Which experiment type would you use, and why?</li>
<li>Do you agree or disagree with the sociological critique that experiments are artificial? Why or why not? How important is this weakness? Do the strengths of experimental research outweigh this drawback?</li>
<li>Be a research participant! The Social Psychology Network offers many online opportunities to participate in social psychological experiments. Check them out at <a class="link" target="_blank" href="http://www.socialpsychology.org/expts.htm">http://www.socialpsychology.org/expts.htm</a>.</li>
</ol>
</div>
</div>
</div>
<div id=navbar-bottom class="navbar">
<div class="navbar-part left">
<a href="s15-01-focus-groups.html"><img src="shared/images/batch-left.png"></a> <a href="s15-01-focus-groups.html">Previous Section</a>
</div>
<div class="navbar-part middle">
<a href="index.html"><img src="shared/images/batch-up.png"></a> <a href="index.html">Table of Contents</a>
</div>
<div class="navbar-part right">
<a href="s15-03-ethnomethodology-and-conversat.html">Next Section</a> <a href="s15-03-ethnomethodology-and-conversat.html"><img src="shared/images/batch-right.png"></a>
</div>
</div>
</div>
<script type="text/javascript" src="shared/book.js"></script>
</body>
</html>