18-Tools-SamplingVariation.Rmd

# Sampling variation {#SamplingVariation}


```{r, child = if (knitr::is_html_output()) {'./introductions/18-Tools-SamplingVariation-HTML.Rmd'} else {'./introductions/18-Tools-SamplingVariation-LaTeX.Rmd'}}
```


## Introduction {#SamplingVariationIntro}

The last two chapters introduced tools to apply the [decision-making process](#DecisionMaking) (Sect. \@ref(MakingDecisionsInResearch)) that is used in research:

1. Make an **assumption** about the population *parameter*.
1. Based on this assumption, describe what values the sample *statistic* might reasonably be **expected**  from all possible samples.
1. **Observe** the sample data from just one of those may possible samples.
1. **Decide** if the sample statistic seems *consistent* with the expectation, or if it *contradicts* the expectation.


Realising that the sample we study is only one of countless possible samples that could have been chosen is important.


::: {.importantBox .important data-latex="{iconmonstr-warning-8-240.png}"}
Remember: Studying a sample leads to the following observations:
\vspace{-2ex}

* Every sample is likely to be different.
* Our sample is one of countless possible samples from the population.
* Every sample is likely to produce a different value for the sample statistic.
* Hence we only observe one of the many possible values for the sample statistic.
\vspace{-2ex}

Since many values for the sample statistic are possible, the possible values of the sample statistic vary (called *sampling variation*) and have a *distribution* (called a *sampling distribution*).
:::


Furthermore, under certain conditions, the variation of the *sample statistic* (such as the sample mean, etc.) from all possible samples can be described *approximately by a normal distribution*.
As a result, the expected behaviour of these statistics can be *described*, so we know what to **expect** from the sample *statistic* when the assumption is true.

We saw this in Sect. \@ref(MakingDecisionsInResearch): the sample proportion of red cards in a sample of 15 varied from hand to hand, and was approximately distributed as a normal distribution.
This is no accident: *Many sample statistics vary from sample to sample with an approximate normal distribution* if certain conditions are met.
<!--This is called the *Central Limit Theorem*.-->

Any distribution that describes how a sample statistic varies for all possible samples is called a *sampling distribution*: how the value of the sample statistic varies from sample to sample for all possible samples.
The *sampling distribution* often has a normal distribution shape.


::: {.definition #SamplingDistribution name="Sampling distribution"}
A **sampling distribution** is the distribution of some sample statistic, showing how its value varies from sample to sample.
:::


## Sample proportions have a distribution {#SamplingDistributionProportions}

As with any sample statistic, sample proportions vary from sample to sample (Sect. \@ref(MakingDecisionsInResearch)); that is, *sampling variation* exists, so sample proportions have a *sampling distribution*.
Consider a European roulette wheel
`r if (knitr::is_latex_output()) {
   'shown below (the online version has an animation):'
} else {
   'shown below in the animation:'
}`
a ball is spun and can land on any number on the wheel from 0 to 36 (inclusive).


<!-- Show Roulette wheel, and some text: -->
```{r, child = if (knitr::is_html_output() ) {'./children/HTML/RouletteWheel-HTML.Rmd'} else { './children/LaTeX/RouletteWheel-LaTeX.Rmd'}}
```


```{r results='hide'}
# Reset seed
set.seed(8723064)
```


Computer simulation can be used to demonstrate what happens if the wheel was spun, over and over again, for $n = 15$ spins each time, and the proportion of odd-spins was recorded for each repetition.
Clearly, the proportion of odd spins $\hat{p}$ can vary from sample to sample (sampling variation) for one sample of $n = 15$ spins, as shown by the histogram (Fig. \@ref(fig:RouletteWheelHist), top left panel).
We can see that, for many repetitions, $\hat{p}$ is rarely smaller than 0.2, and rarely larger than 0.8.

If the wheel was spun (say) $n = 25$ times (rather than $15$ times), $\hat{p}$ again varies (Fig. \@ref(fig:RouletteWheelHist), top right panel): the values of $\hat{p}$ vary from sample to sample.
The same process can be repeated for many repetitions of (say) $n = 100$ and $n = 200$ spins (Fig. \@ref(fig:RouletteWheelHist), bottom panels).

Notice that as the sample size $n$ gets larger, the *distribution* of the values of $\hat{p}$ look more like a normal distribution, and the variation gets smaller.
With $200$ spins, for instance, observing a sample proportion smaller than $0.4$ or greater than $0.6$ seems highly unusual, but these are not uncommon at all for $15$ spins.

::: {.thinkBox .think data-latex="{iconmonstr-light-bulb-2-240.png}"}
Suppose we spun a roulette wheel 100 times, and observed 31 even numbers.
What would you conclude?\label{thinkBox:SpinProportion}
:::


```{r RouletteWheelHist, results='hide', fig.width=5, fig.cap="Sampling distributions for the proportion of roulette wheel spins that show an odd number", fig.align="center", out.width="100%", fig.width=9.25}
p <- 18/37
spins <- c(15, 25, 100, 200)

se <- sqrt( p * (1 - p) / spins )

num.sims <- 5000

par( mfrow = c(2, 2))

### Spin the wheel spins[1] * num.sims. times 
### and grab each sim set from there

xNorm <- seq(0, 1, 
             length = 100)

propOdd <- function(x){
  sum( (x%%2 != 0 ) ) / length(x)
}

set.seed(37945000)
spinNumbersAll <- sample( 0:36, 
                          spins[1] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[1], num.sims) )
sampleP <- apply( spinNumbers, 
                  MARGIN = 2, 
                  FUN = propOdd )

break.list <- seq(0, 1, 
                  by = 1/15) + 1/30 

out <- hist( sampleP,
             breaks = break.list,
             xlim = c(0, 1),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample proportions", 
             ylab = "",
             main = paste("From", spins[1], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = p, 
               sd = se[1]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)





spinNumbersAll <- sample( 0:36, 
                          spins[2] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll,   
                      dim = c(spins[2], num.sims) )
sampleP <- apply( spinNumbers, 2, propOdd )

break.list <- seq(0, 1, 
                  by = 0.04) + 0.02 

out <- hist( sampleP, 
             breaks = break.list, 
             xlim = c(0, 1),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample proportions", 
             ylab = "",
             main = paste("From", spins[2], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = p, 
               sd = se[2]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)





spinNumbersAll <- sample( 0:36,
                          spins[3] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[3], num.sims) )
sampleP <- apply( spinNumbers, 2, propOdd )
break.list <- seq(0, 1, 
                  by = 0.05) + 0.05

out <- hist( sampleP, 
             breaks = break.list, 
             xlim = c(0, 1),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample proportions", 
             ylab = "",
             main = paste("From", spins[3], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = p, 
               sd = se[3]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)


spinNumbersAll <- sample( 0:36, 
                          spins[4] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[4], num.sims) )
sampleP <- apply( spinNumbers, 2, propOdd )
break.list <- seq(0, 1, 
                  by = 0.025) + 0.05

out <- hist( sampleP, 
             breaks = break.list, 
             xlim = c(0, 1),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample proportions", 
             ylab = "",
             main = paste("From", spins[4], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = p, 
               sd = se[4])
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)

```


::: {.importantBox .important data-latex="{iconmonstr-warning-8-240.png}"}
The values of the sample proportion vary from sample to sample.
The distribution of the possible values of the sample statistic (in this case the *sample* proportion) from sample to sample is called a *sampling distribution*.

Under certain conditions, the sampling distribution of a sample proportion is described by an approximate a normal distribution.
In general, the approximation gets better as the sample size gets larger, and the possible values of $\hat{p}$ vary less as the sample size gets larger.

The mean of this sampling distribution is called the *sampling mean*; the standard deviation of this sampling distribution is called the *standard error* (Fig. \@ref(fig:StatisticVariesAcrossSamples)). 
:::


```{r StatisticVariesAcrossSamples, fig.align="center", fig.width=5.5, out.width="70%", fig.cap="Describing how the value of the sample statistic varies across all possible samples"}
#
par(mar = c(0.25, 0.25, 4, 0.25) )
out <- plotNormal(mu = 0,
                  sd = 1, 
                  ylim = c(-0.45, 0.7),
                  main = "The value of the sample statistic varies\nacross all possible samples",
                  showXaxis = FALSE )


lines( x = c(-3, -3),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(-2, -2),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(-1, -1),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(0, 0),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(1, 1),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(2, 2),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)
lines( x = c(3, 3),
       y = c(0, max(out$y)),
       col = "grey",
       lwd = 2)


text(x = 0,
     y = 1.25 * max(out$y),
     pos = 3,
     labels = "Sampling\nmean")
arrows(x0 = 0,
       y0 = 1.25 * max(out$y),
       x1 = 0,
       y1 = max(out$y),
       angle = 15,
       length = 0.1,
       lwd = 2)

arrows(x0 = 0,
       y0 = 0.16 * max(out$y),
       x1 = 1,
       y1 = 0.16 * max(out$y),
       angle = 15,
       length = 0.1,
       lwd = 2)

text(x = 0.5,
     y = 0.16 * max(out$y),
     pos = 3,
     labels = "Standard\nerror") 


arrows(x0 = 3,
       y0 = -0.10,
       x1 = -3,
       y1 = -0.10,
       angle = 15,
       code = 3,
       length = 0.1,
       lwd = 2)
text(x = 0,
     y = -0.12,
     pos = 1,
     labels = "Possible values of the sample statistic, from all possible samples")

arrows(x0 = -0.5,
       y0 = -0.42,
       x1 = -3,
       y1 = -0.42,
       angle = 15,
       length = 0.1,
       lwd = 2)
text(x = -1.75,
     y = -0.29,
     pos = 1,cex = 0.9,
     labels = "Smaller values than average")



arrows(x0 = 0.5,
       y0 = -0.42,
       x1 = 3,
       y1 = -0.42,
       angle = 15,
       length = 0.1,
       lwd = 2)
text(x = 1.75,
     y = -0.29,
     pos = 1,cex = 0.9,
     labels = "Larger values than average")

```

## Sample means have a distribution {#SamplingDistributionMeans}

Like all sample statistics, the sample mean varies from sample to sample (Sect. \@ref(MakingDecisionsInResearch)) just like sample proportions; that is, *sampling variation* exists, so the sample means have a *sampling distribution*.


<div style="float:right; width: 222x; border: 1px; padding:10px">
<img src="Illustrations/luck-839037_640.jpg" width="200px"/>
</div>



Consider a European roulette wheel again (Sect. \@ref(SamplingDistributionProportions)).
Rather than recording the sample proportion of odd-spins, suppose the *sample mean* of the numbers spun was recorded.
If the wheel is spun (say) $15$ times, the *sample* mean of the spins $\bar{x}$ will vary from one set of $15$ spins to another.
Of course, spinning the wheel $30$, $50$ or $100$ times also shows that the *sample* mean $\bar{x}$ can vary too.
How much does it vary?

Again, computer simulation can be used to demonstrate what could happen if the wheel was spun $15$ times, over and over and over again, and the mean of the spun numbers was recorded for each repetition.
Clearly, the sample mean spin $\bar{x}$ can vary from sample to sample (sampling variation) for $n = 15$ spins (Fig. \@ref(fig:RouletteWheelHistx), top left panel).

When $n = 15$, the sample mean $\bar{x}$ indeed varies from sample to sample, and the *distribution* of the values of $\bar{x}$ have an approximate normal distribution.
If the wheel was spun more than $15$ times (say, $n = 50$ times) something similar occurs (Fig. \@ref(fig:RouletteWheelHistx), top right panel): the values of $\bar{x}$ vary from sample to sample, and the values have an approximate normal distribution.
In fact, the values of $\bar{x}$ have a normal distribution for other numbers of spins also (Fig. \@ref(fig:RouletteWheelHistx), bottom panels).


```{r RouletteWheelHistx, results='hide', fig.width=9.25, fig.cap="Sampling distributions for the mean of the numbers after a roulette wheel spins a certain number of times", fig.align="center", out.width="100%"}
mu <- sum( 0:36 )/ 37
sigma <- sqrt( sum( ( (0:36) - mu )^2 )/37 )

spins <- c(15, 50, 100, 250)

se <- sigma / sqrt( spins )


num.sims <- 5000

par( mfrow = c(2, 2))

### Spin the wheel spins[1] * num.sims. times 
### and grab each sim set from there
set.seed(37389457)

xNorm <- seq(0, 37, 
             length = 100)


spinNumbersAll <- sample( 0:36, 
                          spins[1] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[1], num.sims) )
sampleMeans <- colMeans(spinNumbers)
break.list <- seq(0, 37, 
                  by = 1)

out <- hist( sampleMeans,
             breaks = break.list,
             xlim = c(5, 30),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample means", 
             ylab = "",
             main = paste("From", spins[1], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = mu, 
               sd = se[1]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)


spinNumbersAll <- sample( 0:36, 
                          spins[2] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[2], num.sims) )
sampleMeans <- colMeans(spinNumbers)
break.list <- seq(0, 37, 
                  by = 1)

out <- hist( sampleMeans, 
             breaks = break.list,
             xlim = c(5, 30),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample means", 
             ylab = "",
             main = paste("From", spins[2], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = mu, 
               sd = se[2])
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)


spinNumbersAll <- sample( 0:36, 
                          spins[3] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[3], num.sims) )
sampleMeans <- colMeans(spinNumbers)
break.list <- seq(0, 37, 
                  by = 0.5)

out <- hist( sampleMeans, 
             breaks = break.list,
             xlim = c(5, 30),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample means", 
             ylab = "",
             main = paste("From", spins[3], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = mu, 
               sd = se[3]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),
       col = "black",
       lwd = 2)
axis(side = 1)


spinNumbersAll <- sample( 0:36, 
                          spins[4] * num.sims, 
                          replace = TRUE)
spinNumbers <- array( spinNumbersAll, 
                      dim = c(spins[4], num.sims) )
sampleMeans <- colMeans(spinNumbers)
break.list <- seq(0, 37, 
                  by = 0.5)

out <- hist( sampleMeans, 
             breaks = break.list,
             xlim = c(5, 30),
             axes = FALSE,
             col = plot.colour,
             xlab = "Sample means", 
             ylab = "",
             main = paste("From", spins[4], "spins\nof the wheel") )
yNorm <- dnorm(xNorm, 
               mean = mu, 
               sd = se[4]) 
lines( xNorm,
       (yNorm) / max(yNorm) * max(out$count),  
       col = "black",
       lwd = 2)
axis(side = 1)
```


::: {.importantBox .important data-latex="{iconmonstr-warning-8-240.png}"}
The values of the sample mean vary from sample to sample.
The distribution of the possible values of a sample statistic, in this case the *sample* mean, is called a *sampling distribution*.

Under certain conditions, the sampling distribution of a sample mean is described by an approximate a normal distribution.
In general, the approximation gets better as the sample size gets larger, and the possible values of $\bar{x}$ vary less as the sample size gets larger.

The mean of this sampling distribution is called the *sampling mean*; the standard deviation of this sampling distribution is called the *standard error* (Fig. \@ref(fig:StatisticVariesAcrossSamples)).
:::


::: {.thinkBox .think data-latex="{iconmonstr-light-bulb-2-240.png}"}
Suppose we spun a roulette wheel $100$ times, and the mean of the observed numbers was $24.5$.
What would you conclude?\label{thinkBox:SpinMean}

`r if (!knitr::is_html_output()) '<!--'`
`r webexercises::hide()`
From Fig. \ref{fig:RouletteWheelHist} (bottom-left), highly unlikely from a fair wheel.
Or perhaps you were very lucky...
`r webexercises::unhide()`
`r if (!knitr::is_html_output()) '-->'`
:::


As we have seen, each sample is likely to be different, so *any* statistic is likely to be vary from sample to sample.
(Recall that the value of the *population* parameter does not change.)
This variation in the possible values of the observed sampling statistic is called *sampling variation*.



## Sampling means and standard errors {#StandardErrors}

The value of a sample statistic can vary from sample to sample, depending on which sample is selected.
However, only one sample is taken and hence one value of the sample statistic is observed.
The specific value of the sample statistic that is observed depends on which one of those countless samples is selected.

This means that all the possible values of sample statistics that we could potentially observe have a *distribution* (specifically, a *sampling distribution*).
Perhaps surprisingly, under certain conditions, the sampling distribution is often a *normal distribution*, as we have seen.

The *mean* of this sampling distribution is called the *sampling mean*.
The sampling mean tells us what the average value of the sample statistic will be, across all possible samples.


::: {.definition #SamplingMeanError name="Sampling mean"}
The *sampling mean* is the mean of the sampling distribution of a statistic.
:::



The *standard deviation* of this sampling distribution is called the *standard error*.
The standard error tells us how much the value of the sample statistic is likely to vary across, across all of the possible samples.

Figs. \@ref(fig:RouletteWheelHist) and \@ref(fig:RouletteWheelHistx) show that the variation in the values of the sample statistic get smaller for larger sample sizes.
In other words, the standard deviation appears to get *smaller* as the sample sizes get *larger*: the sample statistics show less variation for larger $n$.

This makes sense: *larger* samples generally produce more precise estimates.
(After all, that's the advantage of using larger samples: all else being equal, larger samples are preferred as they produce more precise estimates (Sect. \@ref(PrecisionAccuracy)).

The standard deviation of a sampling distribution is given a special name: a *standard error*.
The standard error is a measure of how precisely the *sample* [statistic](#StatisticsAndParameters) estimates the *population* [parameter](#StatisticsAndParameters).
If every possible sample (of a given size) was found, and the statistic computed from each sample, the standard deviation of all these estimates is the *standard error*.


::: {.definition #StandardError name="Standard error"}
A *standard error* is the standard deviation of the sampling distribution of a statistic.
:::


::: {.example #StandardErrors name="Standard errors"}
In Fig. \@ref(fig:RouletteWheelHistx), a sample of $250$ (i.e., $250$ spins) is unlikely to produce a sample mean larger than $20$, or smaller than $15$. 
However, in a sample of size $15$ (i.e., $15$ spins) sample means near $15$ and $20$ are quite commonplace.

In samples of size $100$, the variation is smaller than in samples of size $15$.
Hence, the *standard error* (the deviation of the sampling normal distributions) will be smaller for sample of size $250$ than for sample of size $15$.
:::


Recall from Sect. \@ref(SamplingVariationIntro) that, for many sample statistics, *the variation from sample to sample can be approximately described by a normal distribution* (the *sampling distribution*) if certain conditions are met.
Furthermore, the *standard deviation of this normal distribution is called the standard error*.
The standard error is a special name given to the *standard deviation* that describes the variation in a sample estimate that varies from *from sample to sample*.


::: {.tipBox .tip data-latex="{iconmonstr-info-6-240.png}"}
The standard error is an unfortunate term: It is not an *error* or *mistake*, or even *standard*.
(For example, there is no such thing as a '*non*-standard error'.)
:::


## Standard deviation vs standard error

Even
`r if (knitr::is_latex_output()) {
   'experienced researchers confuse the meaning and the usage of the terms'
} else {
   '[experienced researchers confuse the meaning and the usage of the terms](https://retractionwatch.com/2019/12/09/authors-retract-two-studies-on-high-blood-pressure-and-supplements-after-realizing-theyd-made-a-common-error/#more-118562)'
}`
*standard deviation* and *standard error* [@ko2014inappropriate].
Understanding the difference is important.

The [*standard deviation*](#VariationStdDev), in general, quantifies the amount of variation in any quantity that varies.
The *standard error* is a name specifically given to the standard deviation that describes a sampling distribution.

In research, *standard deviations* are used to describe the variation in the values of a variable for individual observations (for *quantitative* data): how observations vary from *individual to individual*.
The *standard error* is used to describe how *sample estimates* are likely to vary from sample to sample (i.e., to describe the precision of sample estimates).

Crucially, the standard error *is* a standard deviation, but is used specifically to describe the variation in sampling distributions.
*Any* numerical quantity estimated from a sample (a *statistic*) can vary from sample to sample, and so has sampling variation, a sampling distribution, and hence a standard error.
(Not all statistics have a *normal* distribution, however.)


::: {.importantBox .important data-latex="{iconmonstr-warning-8-240.png}"}
*Any* quantity estimated from a sample (i.e., a statistic) has a standard error. 
Equivalently: *all* statistics vary from sample to sample and have a standard error.
:::


::: {.tipBox .tip data-latex="{iconmonstr-info-6-240.png}"}
The *standard error* is often abbreviated to 'SE' or 's.e.'.
For example, the '[standard error of the sample mean](#SamplingDistributionMeans)' is often written $\text{s.e.}(\bar{x})$, and the '[standard error of the sample proportion](#SamplingDistributionProportions)' is often written $\text{s.e.}(\hat{p})$.
:::


<iframe src="https://learningapps.org/watch?v=pk8ucviua22" style="border:0px;width:100%;height:800px" allowfullscreen="true" webkitallowfullscreen="true" mozallowfullscreen="true"></iframe>


## Summary {#Chap18-Summary}

A **sampling distribution** describes how all possible values of a sample statistic is likely to vary from sample to sample.
Under certain circumstances, the sampling distribution often can be described by a **normal distribution**.
The standard deviation of this normal distribution is called a **standard error**.
The standard error is the name specifically given to the standard deviation that describes the variation in the sample statistic *across all possible samples*.


## Quick review questions {#Chap18-QuickReview}

::: {.webex-check .webex-box}
1. *Why* is the phrase 'the standard error of the population proportion' inappropriate?\tightlist  
`r if( knitr::is_html_output() ) {
	longmcq( c(
		answer = "Because only sample statistics have values that vary from sample to sample",
		"Because proportions do not have a standard error",
		"Because proportions only have a standard deviation",
		"Because proportions come from qualitative data"
		))}`  
`r if( !knitr::is_html_output() ) {
' 	 * Because only sample statistics have values that vary from sample to sample
 	 * Because proportions do not have a standard error
 	 * Because proportions only have a standard deviation
   * Because proportions come from qualitative data'
}`

1. Which *one* the following *does not* have a standard error?  
`r if( knitr::is_html_output() ) {
	longmcq( c(
		"The sample mean",
		"The difference between two sample means",
		"The sample proportion",
		"The sample odds ratio",
		answer = "The sample size"
		))}`  
`r if( !knitr::is_html_output() ) {
' 	 * The sample mean
 	 * The difference between two sample means
 	 * The sample proportion
   * The sample odds ratio
   * The sample size'
}`
1. True or false: Sampling variation refers to how sample sizes vary.
`r if( knitr::is_html_output() ) {
	torf( answer=FALSE )}`
1. True or false: Sampling distributions describe parameters.
`r if( knitr::is_html_output() ) {
	torf( answer=FALSE )}`
1. True or false: Statistics do not vary from sample to sample.
`r if( knitr::is_html_output() ) {
	torf( answer=FALSE )}`
1. True or false: Populations are numerically summarised using parameters
`r if( knitr::is_html_output() ) {
	torf( answer=TRUE )}`
1. True of false: The *standard deviation* is a *standard error* of something quite specific.
`r if( knitr::is_html_output() ) {
	torf( answer=FALSE )}`
1. True or false: 
Sampling distributions are always *normal* distributions.  
`r if( knitr::is_html_output() ) {
	torf( answer=FALSE )}`
1. True or false: 
Sampling variation measures the amount of variation in the individuals in the sample (for the variable of interest).  
`r if( knitr::is_html_output() ) {
	torf( answer = FALSE )}`
1. True or false: 
The standard error measures the size of the error when we use a sample to estimate a population.  
`r if( knitr::is_html_output() ) {
	torf( answer = FALSE )}`
1. True or false: 
In general terms, a standard deviation measures the amount of variation.   
`r if( knitr::is_html_output() ) {
	torf( answer = TRUE )}`
1. True or false: 
The standard error is a standard deviation, of something quite specific.  
`r if( knitr::is_html_output() ) {
	torf( answer = TRUE )}`
1. True or false: 
Sampling variation measures the size of the sample.   
`r if( knitr::is_html_output() ) {
	torf( answer = FALSE )}`
:::


## Exercises {#SamplingVariationExercises}

Selected answers are available in Sect. \@ref(SamplingVariationExercisesAnswer).


::: {.exercise #StdErrorOrStdDeviation}
In the following scenarios, would a *standard deviation* or a *standard error* be the appropriate way to measure the amount of variation?
Explain.

1. Researchers are studying the spending habits of customers. 
   They would like to measure the variation in the amount spent by shoppers per transaction at a supermarket.
1. Researchers are studying the time it takes for inner-city office workers to travel to work each morning. 
   They would like to determine the precision with which their estimate (a mean of 47 minutes) has been measured.
1. A study examined the effect of taking a pain-relieving drug on children.
   The researchers wish to describe the sample they used in the study, including a description of how the ages of the children in the study vary.
1. A study examined the effect of taking a pain-relieving drug in teenagers.
   The researchers wished to report the percentage of teenagers in the sample that experienced side-effects with some indication of the precision of that estimate.
:::


::: {.exercise #HasStandardError}
Which of the following have a *standard error*?

:::::: {.cols data-latex=""}

:::: {.col data-latex="{0.4\textwidth}"}

1. The population proportion.
1. The sample median.
1. The sample IQR.
::::

:::: {.col data-latex="{0.05\textwidth}"}
\ 
<!-- an empty Div (with a white space), serving as
a column separator -->
::::

:::: {.col data-latex="{0.5\textwidth}"}
1. The sample standard deviation.
1. The population odds.
::::
::::::

:::


::: {.exercise #QuoteStdError}
A research article made this statement:

> Although [...] samples should always be summarized by the mean and SD [standard deviation], authors often use the standard error of the mean (SEM) to describe the variability of their sample [...]
> Although the SD and the SEM are related [...], they give two very different types of information.
> 
> --- @nagele2003misuse

If the standard error of the mean is not used to 'describe the variability of the sample', then what *is* it used for?
How would you explain the difference between the *standard error* and the *standard deviation* to researchers who misuse the terms?
:::






<!-- QUICK REVIEW ANSWERS -->
`r if (knitr::is_html_output()) '<!--'`
::: {.EOCanswerBox .EOCanswer data-latex="{iconmonstr-check-mark-14-240.png}"}
**Answers to in-chapter questions:**


- Sect. \ref{thinkBox:SpinProportion}: From Fig. \ref{fig:RouletteWheelHist} (bottom-left), highly unlikely from a fair wheel. 
  Or perhaps you were very unlucky...
- Sect. \ref{thinkBox:SpinMean}: From Fig. \ref{fig:RouletteWheelHistx} (bottom-left), highly unlikely from a fair wheel.
  Or perhaps you were very lucky...

- \textbf{\textit{Quick Revision} questions:}
**1a.** Because only *sample statistics* vary from sample to sample. 
**2.** The sample size.
**3.** False.
**4.** False.
**5.** False.
**6.** True.
**7.** False.
**8.** False.
**9.** False.
**10.** False.
**11.** True.
**12.** True.
**13.** False.
:::
`r if (knitr::is_html_output()) '-->'`