forked from saylordotorg/text_principles-of-sociological-inquiry-qualitative-and-quantitative-methods
-
Notifications
You must be signed in to change notification settings - Fork 0
/
s17-01-reading-reports-of-sociologica.html
188 lines (172 loc) · 19.7 KB
/
s17-01-reading-reports-of-sociologica.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<link href="shared/bookhub.css" rel="stylesheet" type="text/css">
<title>Reading Reports of Sociological Research</title>
</head>
<body>
<div id=navbar-top class="navbar">
<div class="navbar-part left">
<a href="s16-04-disseminating-findings.html"><img src="shared/images/batch-left.png"></a> <a href="s16-04-disseminating-findings.html">Previous Section</a>
</div>
<div class="navbar-part middle">
<a href="index.html"><img src="shared/images/batch-up.png"></a> <a href="index.html">Table of Contents</a>
</div>
<div class="navbar-part right">
<a href="s17-02-being-a-responsible-consumer-o.html">Next Section</a> <a href="s17-02-being-a-responsible-consumer-o.html"><img src="shared/images/batch-right.png"></a>
</div>
</div>
<div id="book-content">
<div class="section" id="blackstone_1.0-ch14_s01" condition="start-of-chunk" version="5.0" lang="en">
<h2 class="title editable block">
<span class="title-prefix">14.1</span> Reading Reports of Sociological Research</h2>
<div class="learning_objectives editable block" id="blackstone_1.0-ch14_s01_n01">
<h3 class="title">Learning Objectives</h3>
<ol class="orderedlist" id="blackstone_1.0-ch14_s01_l01">
<li>Identify what one can learn from an article simply by reading its abstract and its acknowledgments.</li>
<li>Describe how tables presenting causal relationships are typically presented.</li>
<li>Identify several key questions to ask when reading research reports.</li>
</ol>
</div>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p01">By now you should have a good idea about the basic components of sociological research projects. You know how sociological research is designed, and you are familiar with how to frame a review of sociological literature. In <a class="xref" href="blackstone_1.0-ch05#blackstone_1.0-ch05">Chapter 5 "Research Design"</a>, we discussed the various components of a research project and presented some tips on how to review literature as you design your own research project. But I hope that you’ll find the sociological literature to be of interest and relevance to you beyond figuring out how to summarize and critique it in relation to your research plans. We sociologists like to think the research we do matters, but it cannot matter if our research reports go unread or are not understandable. In this section we’ll review some material from <a class="xref" href="blackstone_1.0-ch05#blackstone_1.0-ch05">Chapter 5 "Research Design"</a> regarding sociological literature and we’ll consider some additional tips for how to read and understand reports of sociological research.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p02">As mentioned in <a class="xref" href="blackstone_1.0-ch05#blackstone_1.0-ch05">Chapter 5 "Research Design"</a>, reading the abstract that appears in most reports of scholarly research will provide you with an excellent, easily digestible review of a study’s major findings and of the framework the author is using to position her findings. Abstracts typically contain just a few hundred words, so reading them is a nice way to quickly familiarize yourself with a study. Another thing to look for as you set out to read and comprehend a research report is the author’s acknowledgments. Who supported the work by providing feedback or other assistance? If relevant, are you familiar with the research of those who provided feedback on the report you are about to read? Are any organizations mentioned as having supported the research in some way, either through funding or by providing other resources to the researcher? Familiarizing yourself with an author’s acknowledgments will give you additional contextual information within which to frame and understand what you are about to read.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p03">Once you have read the abstract and acknowledgments, you could next peruse the discussion section near the end of the report, as suggested in <a class="xref" href="blackstone_1.0-ch05#blackstone_1.0-ch05">Chapter 5 "Research Design"</a>. You might also take a look at any tables that are included in the article. A <span class="margin_term"><a class="glossterm">table</a><span class="glossdef">A tool used by researchers who wish to present large amounts of data in a succinct format. Tables are most commonly used in reports of quantitative research findings.</span></span> provides a quick, condensed summary of the report’s key findings. The use of tables is not limited to one form or type of data, though they are used most commonly in quantitative research. Tables are a concise way to report large amounts of data. Some tables present descriptive information about a researcher’s sample. These tables will likely contain frequencies (N) and percentages (%). For example, if gender happened to be an important variable for the researcher’s analysis, a descriptive table would show how many and what percent of all study participants are women and how many/what percent are men. Frequencies, or “how many,” will probably be listed as <em class="emphasis">N</em>, while the percent symbol (%) might be used to indicate percentages.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p04">In a table presenting a causal relationship, independent variable attributes are typically presented in the table’s columns, while dependent variable attributes are presented in rows. This allows the reader to scan across a table’s rows to see how values on the dependent variable attributes change as the independent variable attribute values change. Tables displaying results of quantitative analysis will also likely include some information about the strength and statistical significance of the relationships presented in the table. These details tell the reader how likely it is that the relationships presented will have occurred simply by chance.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p05">Let’s look at a specific example. <a class="xref" href="#blackstone_1.0-ch14_s01_t01">Table 14.1 "Percentage Reporting Harassing Behaviors at Work"</a>, based on data from my study of older workers, presents the causal relationship between gender and experiencing harassing behaviors at work. In this example, gender is the independent variable and the harassing behaviors listed are the dependent variables.<span class="footnote" id="blackstone_1.0-fn14_001">It wouldn’t make any sense to say that people’s workplace experiences <em class="emphasis">cause</em> their gender, so in this example, the question of which is the independent variable and which are the dependent variables has a pretty obvious answer.</span> I have therefore placed gender in the table’s columns and harassing behaviors in the table’s rows. Reading across the table’s top row, we see that 2.9% of women in the sample reported experiencing subtle or obvious threats to their safety at work, while 4.7% of men in the sample reported the same. We can read across each of the rows of the table in this way. Reading across the bottom row, we see that 9.4% of women in the sample reported experiencing staring or invasion of their personal space at work while just 2.3% of men in the sample reported having the same experience.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p06">Of course, we cannot assume that these patterns didn’t simply occur by chance. How confident can we be that the findings presented in the table did not occur by chance? This is where tests of statistical significance come in handy. <span class="margin_term"><a class="glossterm">Statistical significance</a><span class="glossdef">A report of the likelihood that relationships observed could be caused by something other than chance.</span></span> tells us the likelihood that the relationships we observe could be caused by something other than chance. While your statistics class will give you more specific details on tests of statistical significance and reading quantitative tables, the important thing to be aware of as a nonexpert reader of tables is that some of the relationships presented will be statistically significant and others may not be. Tables should provide information about the statistical significance of the relationships presented. When reading a researcher’s conclusions, be sure to pay attention to which relationships are statistically significant and which are not.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p07">In <a class="xref" href="#blackstone_1.0-ch14_s01_t01">Table 14.1 "Percentage Reporting Harassing Behaviors at Work"</a>, you’ll see that a <em class="emphasis">p</em> value is noted in the last very column of the table. A <span class="margin_term"><a class="glossterm"><em class="emphasis bolditalic">p</em> value</a><span class="glossdef">A statistical measure of the probability that there is no relationship between the variables under study.</span></span> is a statistical measure of the probability that there is no relationship between the variables under study. Another way of putting this is that the <em class="emphasis">p</em> value provides guidance on whether or not we should reject the null hypothesis. The <span class="margin_term"><a class="glossterm">null hypothesis</a><span class="glossdef">The assumption that no relationship exists between variables in question.</span></span> is simply the assumption that no relationship exists between the variables in question. In <a class="xref" href="#blackstone_1.0-ch14_s01_t01">Table 14.1 "Percentage Reporting Harassing Behaviors at Work"</a>, we see that for the first behavior listed, the <em class="emphasis">p</em> value is 0.623. This means that there is a 62.3% chance that the null hypothesis is correct in this case. In other words, it seems likely that any relationship between observed gender and experiencing threats to safety at work in this sample is simply due to chance.</p>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p08">In the final row of the table, however, we see that the <em class="emphasis">p</em> value is 0.039. In other words, there is a 3.9% chance that the null hypothesis is correct. Thus we can be somewhat more confident than in the preceding example that there may be some relationship between a person’s gender and his experiencing the behavior noted in this row. We might say that this finding is significant at the .05 level. This means that the probability that the relationship between gender and experiencing staring or invasion of personal space at work is due to sampling error alone is less than 5 in 100. Notice that I’m hedging my bets here by using words like <em class="emphasis">somewhat</em> and <em class="emphasis">may be</em>. When testing hypotheses, social scientists generally couch their findings in terms of rejecting the null hypothesis rather than making bold statements about the relationships observed in their tables. You can learn more about creating tables, reading tables, and tests of statistical significance in a class focused exclusively on statistical analysis. For now, I hope this brief introduction to reading tables will give you more confidence in your ability to read and understand the quantitative tables you encounter while reading reports of sociological research.</p>
<div class="table block" id="blackstone_1.0-ch14_s01_t01" frame="all">
<p class="title"><span class="title-prefix">Table 14.1</span> Percentage Reporting Harassing Behaviors at Work</p>
<table cellpadding="0" cellspacing="0">
<thead>
<tr>
<th>Behavior Experienced at work</th>
<th>Women</th>
<th>Men</th>
<th>
<em class="emphasis">p</em> value</th>
</tr>
</thead>
<tbody>
<tr>
<td>Subtle or obvious threats to your safety</td>
<td>2.9%</td>
<td>4.7%</td>
<td>0.623</td>
</tr>
<tr>
<td>Being hit, pushed, or grabbed</td>
<td>2.2%</td>
<td>4.7%</td>
<td>0.480</td>
</tr>
<tr>
<td>Comments or behaviors that demean your gender</td>
<td>6.5%</td>
<td>2.3%</td>
<td>0.184</td>
</tr>
<tr>
<td>Comments or behaviors that demean your age</td>
<td>13.8%</td>
<td>9.3%</td>
<td>0.407</td>
</tr>
<tr>
<td>Staring or invasion of your personal space</td>
<td>9.4%</td>
<td>2.3%</td>
<td>0.039</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="4">
<em class="emphasis">Note:</em> Sample size was 138 for women and 43 for men.</th>
</tr>
</tfoot>
</table>
</div>
<p class="para editable block" id="blackstone_1.0-ch14_s01_p09">Having read the tables in a research report, along with the abstract, acknowledgments, and discussion in the report, you are finally ready to read the report in its entirety. As you read a research report, there are several questions you can ask yourself about each section, from abstract to conclusion. Those questions are summarized in <a class="xref" href="#blackstone_1.0-ch14_s01_t02">Table 14.2 "Questions Worth Asking While Reading Research Reports"</a>. Keep in mind that the questions covered here are designed to help you, the reader, to think critically about the research you come across and to get a general understanding of the strengths, weaknesses, and key takeaways from a given study. I hope that by considering how you might respond to the following questions while reading research reports, you’ll feel confident that you could describe the report to others and discuss its meaning and impact with them.</p>
<div class="table block" id="blackstone_1.0-ch14_s01_t02" frame="all">
<p class="title"><span class="title-prefix">Table 14.2</span> Questions Worth Asking While Reading Research Reports</p>
<table cellpadding="0" cellspacing="0">
<thead>
<tr>
<th>Report section</th>
<th>Questions worth asking</th>
</tr>
</thead>
<tbody>
<tr>
<td>Abstract</td>
<td>What are the key findings? How were those findings reached? What framework does the researcher employ?</td>
</tr>
<tr>
<td>Acknowledgments</td>
<td>Who are this study’s major stakeholders? Who provided feedback? Who provided support in the form of funding or other resources?</td>
</tr>
<tr>
<td>Introduction</td>
<td>How does the author frame his or her research focus? What other possible ways of framing the problem exist? Why might the author have chosen this particular way of framing the problem?</td>
</tr>
<tr>
<td>Literature review</td>
<td>How selective does the researcher appear to have been in identifying relevant literature to discuss? Does the review of literature appear appropriately extensive? Does the researcher provide a critical review?</td>
</tr>
<tr>
<td>Sample</td>
<td>Was probability sampling or nonprobability sampling employed? What is the researcher’s sample? What is the researcher’s population? What claims will the researcher be able to make based on the sample? What are the sample’s major strengths and major weaknesses?</td>
</tr>
<tr>
<td>Data collection</td>
<td>How were the data collected? What do you know about the relative strengths and weaknesses of the method employed? What other methods of data collection might have been employed, and why was this particular method employed? What do you know about the data collection strategy and instruments (e.g., questions asked, locations observed)? What <em class="emphasis">don’t</em> you know about the data collection strategy and instruments?</td>
</tr>
<tr>
<td>Data analysis</td>
<td>How were the data analyzed? Is there enough information provided that you feel confident that the proper analytic procedures were employed accurately?</td>
</tr>
<tr>
<td>Results</td>
<td>What are the study’s major findings? Are findings linked back to previously described research questions, objectives, hypotheses, and literature? Are sufficient amounts of data (e.g., quotes and observations in qualitative work, statistics in quantitative work) provided in order to support conclusions drawn? Are tables readable?</td>
</tr>
<tr>
<td>Discussion/conclusion</td>
<td>Does the author generalize to some population beyond her or his sample? How are these claims presented? Are claims made supported by data provided in the results section (e.g., supporting quotes, statistical significance)? Have limitations of the study been fully disclosed and adequately addressed? Are implications sufficiently explored?</td>
</tr>
</tbody>
</table>
</div>
<div class="key_takeaways editable block" id="blackstone_1.0-ch14_s01_n02">
<h3 class="title">Key Takeaways</h3>
<ul class="itemizedlist" id="blackstone_1.0-ch14_s01_l02">
<li>In tables presenting causal relationships, the independent variable is typically presented in the table’s columns while the dependent variables are presented in the table’s rows.</li>
<li>When reading a research report, there are several key questions you should ask yourself for each section of the report.</li>
</ul>
</div>
<div class="exercises editable block" id="blackstone_1.0-ch14_s01_n03">
<h3 class="title">Exercises</h3>
<ol class="orderedlist" id="blackstone_1.0-ch14_s01_l03">
<li>Find a table in a research report of your choosing. Challenge yourself to summarize the relationships represented by the table. Check your work by reading the Findings section of the article.</li>
<li>Read a scholarly article from start to finish, answering the questions outlined in <a class="xref" href="#blackstone_1.0-ch14_s01_t02">Table 14.2 "Questions Worth Asking While Reading Research Reports"</a> as you read through each section.</li>
</ol>
</div>
</div>
</div>
<div id=navbar-bottom class="navbar">
<div class="navbar-part left">
<a href="s16-04-disseminating-findings.html"><img src="shared/images/batch-left.png"></a> <a href="s16-04-disseminating-findings.html">Previous Section</a>
</div>
<div class="navbar-part middle">
<a href="index.html"><img src="shared/images/batch-up.png"></a> <a href="index.html">Table of Contents</a>
</div>
<div class="navbar-part right">
<a href="s17-02-being-a-responsible-consumer-o.html">Next Section</a> <a href="s17-02-being-a-responsible-consumer-o.html"><img src="shared/images/batch-right.png"></a>
</div>
</div>
</div>
<script type="text/javascript" src="shared/book.js"></script>
</body>
</html>