jbstatistics
Confidence Intervals for One Mean: Determining the Required Sample Size
updated
(I'll get back to stats videos soon. This one's mainly for fun, but inspired by a decade of frustration. In case it's not obvious, I play both parts.)
D2L (Brightspace) quizzes do not allow an instructor to change the answer to a numeric response (arithmetic) question and regrade quiz attempts. There's a related problem on multiple-select questions. This has cost me personally at least a hundred hours over the last decade. Any fix is ad-hoc and time consuming, and leads to confusing feedback for students, inferior learning outcomes, and wasted time for instructors that could be used to do something productive.
There are only so many possible explanations as to why D2L has not implemented this “feature”, despite pleas from instructors over the years. This video represents one possible explanation.
Canadian demographic information is based on Statistics Canada numbers found here:
www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000501
0:18 Brief review of conditional probability and independence.
1:39 Multiplication rule for two events
2:49 Simple urn example
4:32 Cautionary example
6:43 Multiplication rule for 3 events
7:35 Example involving Canadian demographics
The temperature data is from here:
climate.weather.gc.ca/climate_data/daily_data_e.html?StationID=51459&timeframe=2&StartYear=1840&EndYear=2021&Day=22&Year=2020&Month=7#
(The original data was given in Celsius, but I converted it to Fahrenheit for use in this video.)
0:00 Introduction and example
1:36 Illustration of the effect of the additive constant on measures of central tendency and variability.
3:58 Illustration of the effect of the multiplicative constant on measures of central tendency and variability.
6:52 Formulas for the new mean, standard deviation, and variance after the linear transformation.
7:50 A numerical example of the effect of a linear transformation on the mean, median, standard deviation, and variance (converting temperatures from Fahrenheit to Celsius).
R code to calculate the mean and standard deviation for the examples:
mean(c(3,4,5))
[1] 4
sd(c(3,4,5))
[1] 1
Source of the jumping fish example:
Brunt et al. (2016). Amphibious fish jump better on land after acclimation to a terrestrial environment. Journal of Experimental Biology, 219:3204-3207.
(Thanks to the authors for supplying the raw data!)
Link to a National Geographic YouTube video of these fish jumping: https:// youtu.be/TZzW1pfwmfU
0:00 The basics
6:00 Real world example
8:10 What boxplots tend to look like under symmetry and skewness.
10:44 Brief discussion of outliers and large sample sizes.
(This is a bit of a different video for me, and if you're just looking for help with specific topics in a statistics course, you may not find it helpful. But there's some good stuff in here.)
Here I address what might seem at first like bit of a strange or uninformative question: In repeated sampling from a normally distributed population, on average what proportion of sample means would a randomly selected 95% CI for mu capture? I work through the calculations, then I discuss how this notion relates to bad confidence interval interpretations and reproducibility* studies.
This was inspired by a bad confidence interval interpretation that I heard a number of years ago (and have heard variants of ever since), where, when interpreting a 95% confidence interval for the population mean, the individual stated:
``If you repeat the same study a million times, then the mean of each one of those samples should fall in the interval 95% of the time.'' [Edited slightly to improve the readability.]
This is a poor interpretation of the interval, and simply untrue. It's just not the case. So, then, what is the probability a randomly selected 95% confidence interval for mu captures the mean of another sample of the same size from the same population?
This has applications in reproducibility* studies, and I briefly discuss that after working through the calculations. My discussion is not intended to be a complete discussion of issues in reproducibility*, just a brief discussion of how the question I answer relates.
*In this video I use "reproducible" and "replicable" interchangeably. I know there has been much discussion in some circles of differences between those terms. Apologies if you find my casual use of these terms problematic or misleading.
Reference for the paper I bring up:
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. \textit{Science}, 349(6251), 1--8.
Reference for a different discussion about how there is extra variability when comparing two statistics, and why the results found in the reference above might not be as bad as they appear at first blush:
Patil P., Peng R. D., Leek J. T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspect. Psychol. Sci. 11 539--544. 10.1177/1745691616646366
At the end, I briefly discuss the normality assumption, and how that leads to beta_1 hat being normally distributed. While I do discuss the real deal there, I go over it fairly quickly, as the main point of the video is deriving E(beta_1 hat) and Var(beta_1 hat).
Note that any time I use "errors" or "error terms" in this video, I am referring to the theoretical error terms (the epsilons) and not observed residuals from sample data.
Time stamps:
0:00 Brief discussion the simple linear regression model, assumptions, and some tools we will use.
2:58 Deriving E(beta_1 hat)
5:06 Deriving Var(beta_1 hat)
8:49 Discussion of normality of beta_1 hat.
I know from experience that some of my students have trouble seeing this (especially when it comes up in a formulaic approach to a problem), and I wanted to have a video that I could point them to.
My Law of Total Probability video is here: youtu.be/7t9jyikrG7w
If you are already familiar with the problem, and you are just looking for help with the mathematics of the derivation, the derivation starts at 3:26.
At the end of the video, I illustrate that sum(X_i-X bar)(Y_i - Y bar) = sum X_i(Y_i - Y bar) =sum Y_i(X_i - X bar) , and that sum(X_i-X bar)^2 = sum X_i(X_i - X bar).
There are, of course, a number of ways of expressing the formula for the slope estimator, and I make no attempt to list them all in this video.
Looking for a proof that if A and B are independent, so are their complements? That's here: youtu.be/bnDpZNlVZ3k
Here I prove that if events A and B are independent, so are Ac and Bc. I make use of De Morgan's Laws, without offering a formal proof of that part (but I do provide a brief Venn diagram justification of the needed bit).
More probability and statistics videos will follow.
(I'm on a bit of a probability run, but looking forward to getting back to statistics videos in the near future.)
This one turned out to be long, as I had a number of points I wanted to discuss. Here's the breakdown:
0:20. The definition of independence, showing what that means in terms of conditional probability, and some hand-waving discussion of what independence means.
3:48. Very simple examples (P(A) = x, P(B) = y, etc.).
6:20. Die rolling examples.
10:03. Discussion of the fact that if A and B are independent, so are (A and Bc), (Ac and B), and (Ac and Bc), including a hand-waving justification. The previous example involved a lead-in to this. (The hand-waving is legit happening behind the scenes.)
11:15. Visual illustration of independence.
14:39. Playing card examples.
18:19. Discussion of how we sometimes *assume* events are independent (e.g. heads on first toss of a fair coin, heads on second toss), and how this is an *assumption*, and not something that can be proven mathematically (despite what you might see elsewhere).
19:24. Discussion of how the term "independent" can have a different meaning in everyday English compared to its usage in probability, and how that is sometimes a cause for confusion.
The sudden death data is slightly modified from:
Naneix et al. (2015). Sudden adult death: An autopsy series of 534 cases with gender and control comparison. Journal of Forensic and Legal Medicine, 32:10-15.
The data was pulled from their Figure 3, and I pooled the Abdominal/pelvian and undetermined groups into "other", to make the example work better visually and have it be easier to follow. I took some slight liberties here, as "undetermined" is not the same as "other". Conscious choice, y'all.
Examples:
0:58. An example using the conditional probability formula, where we are given P(A), P(B), and P(A U B).
3:06: Die rolling. Everybody's fave. P(AUB|C).
4:51. Two-way table, involving real data from above. Limited on interpretation, and focussing on finding various conditional probabilities.
8:05. Conditional probability involving 3 events, visualized with a Venn diagram. P(A n C | B n C), P(B^c|A U C).
11:21. Example of determining whether P(A|B) = P(A), P(A|B) is less than P(A), or P(A|B) is greater than P(A), based on common knowledge and without being given probabilities.
13:00. Informal illustration that if P(A|B) is greater than P(A) then P(B|A) is greater than P(B), and if P(A|B) is less than P(A) then P(B|A) is less than P(B).
14:40. If A is a subset of B, and P(A) is greater than 0, what can be said of P(A|B) and P(B|A)?
I work through an example of a one-sample t test on a mean, and (intentionally) make many false statements. Some of them might sound pretty reasonable. The lesson is: Get your statistics help from a reputable source!
Wrong statements, with explanations:
0:25: “Is the mean weight of cucumbers in my garden equal to 200 grams?” Not wrong, but it’s a bad example on a number of fronts:
-It’s not a question anybody would ask.
-The mean weight of cucumbers is in the garden is not going to equal exactly 200 grams, and we know that going in.
-A t test might provide evidence on whether the true mean weight differs from 200 grams, but it’s not going to tell us that the true mean equals 200 grams.
-If I really wanted to know the true mean weight (of the cucumbers that currently exist in my garden), I could probably pick them all and find out the true value. Or perhaps measure them on the vine with minimal error.
0:55: “H_0: X bar = 200 grams.” Hypotheses never involve statistics or the value of statistics.
1:25: “H_a” X bar greater than 200 grams.” The choice of alternative must never be based on the current sample’s data. It’s cheating to pick the alternative based on the observed data. And again, hypotheses never involve statistics or the values of statistics.
1:40: “…we just pick alpha, our significance level, to be 0.05.” While this is fairly common practice, it usually doesn’t make any sense and definitely doesn’t make any sense here. Here, no decision needs to be made, and we would simply assess the strength of the evidence against H_0 using the p-value. (Some will disagree with me on this front.)
2:00: “I could have picked any cucumbers in my garden, so every cucumber had the same chance of being picked.” While it is true that I could have picked any set of 4 cucumbers in the garden, that doesn’t imply they all had the same chance of being selected.
2:04: “Yes, this was a simple random sample.” Even though there was likely some randomness involved, that doesn’t make it a simple random sample. A SRS has a very specific meaning, one that was almost surely not the case here.
2:23: “Knowing what the first cucumber weighed tells me nothing about what the second cucumber weighs. They were weighed independently, and the observations are therefore independent.” This is just silly.
2:37: “This is not an important question for a t test.” One of the assumptions of the one-sample t test is normality, and that assumption is very important for small sample sizes.
2:51: “The t statistic has a t distribution, and we won’t need to concern ourselves with normality.” The t statistic has a t distribution only if H_0 is true and the assumptions (including normality) are true.
3:00: “The t statistic is X bar - mu” The hypothesized mean (mu_0) is subtracted in the numerator of the test statistic, not the true mean mu (we don’t know mu, and if we did know it we wouldn’t be carrying out the test!).
3:08: “Mu is the true mean weight of cucumbers in my garden, and that’s simply given here in the null hypothesis.” While it is the case that mu represents the true mean weight of cucumbers in the garden, the value of 200 in the null hypothesis is the hypothesized mean (mu_0) and not the true mean mu.
3:20: “Minus the true value of mu”. Again, we subtract the hypothesized mean, not the true mean.
3:43: “For a t test, the degrees of freedom are always n-1.” For one-sample t tests on a single mean, like the one in this video, the degrees of freedom are n-1. But there are many different types of t tests, and the degrees of freedom vary between them.
4:25: “The p-value is the probability of getting a sample mean that is at least as large as the observed sample mean of 214 grams.” This is a common misconception. If H_0 is true, and the assumptions are true, then the p-value is the probability of getting a t statistic at least as large as the one observed (1.12). For a t test like this one, this is not the same as the probability of getting a sample mean that is at least as large as 214 (even under Ho).
4:33: “If I were to go out to my garden and get another 4 cucumbers, the probability of getting a mean weight of 214 grams or more would be 0.17.” This is possibly my favourite line in the video, as it sounds pretty good but is simply not true. We don’t know what this probability is, as it depends on unknown parameters (mu, sigma).
4:46: “So we have pretty strong evidence that the null hypothesis is true, and that the true mean weight of cucumbers in my garden is actually 200 grams.” We found no meaningful evidence against the null hypothesis, but that most definitely doesn’t imply that it’s true. Lack of evidence against H_0 does not imply strong evidence for it. In addition, we essentially know going in that the true mean weight of cucumbers in the garden is not going to be *exactly* 200 grams.
I will get back to statistics videos in the not-too-distant future. Right now, I'm hammering away on probability for a little while.
It's good to be back. I will post many more videos this year.
All plots were created in R. The appropriate diameters of the circles and distance between the centres of the circles were calculated in R using the package venneuler.
Not all plots in this video are officially called Venn diagrams.
This is an updated and refined version of an earlier video. Those looking for the original version can find it at http://youtu.be/OvTEhNL96v0.
This is an updated and revised version of an earlier video. Those looking for my original Intro to Discrete Random Variables video can find it at: http://youtu.be/0P5WRKihQ4E
This video is very similar in content and results to my video that investigates the effect of violations of the normality assumption on inference procedures for a single variance.
The example in this video involves tail lengths of male and female lizards. The summary statistics and distributions of the tail lengths are from Table 1 and Figure 2 in:
Qu et al. (2011). Sexual dimorphism and female reproduction in two
sympatric toad-headed lizards, Phrynocephalus frontalis
and P. versicolor (Agamidae). Animal Biology. 61:139-151.
I don't work through any calculations in this video, but I have another video in which I work through a complete example.
The summary statistics and distribution of the 2D:4D ratios were estimated from Figures 1, 2, and 3 in:
Ozener et al. (2014). Inbreeding Is Associated with Lower 2D:4D Digit Ratio. American Journal of Human Biology. 26:183-188.
I work through an example at http://youtu.be/tsLGbpu\NPk
The cereal data is from a sample of 15 bags of cereal that I collected and weighed.
I have a video introduction to these procedures available at: http://youtu.be/lyd4V8DFCjM
The fat content in deep fried chicken sandwiches is based on information from the USDA Nutrient Database: http://ndb.nal.usda.gov/ndb/foods/show/6781?fg=&man=&lfacet=&format=&count=&max=25&offset=&sort=&qlookup=mcdonald%27s+mcchicken
The example involves an experiment investigating a possible effect of magnetic pulse on the ability of homing pigeons to navigate. The data used in this video is taken from:
Holland et al. (2013). A magnetic pulse does not affect homing pigeon navigation: a GPS tracking experiment. The Journal of Experimental Biology, 216: 2192-2200.
See the article for further details on the experiment.
If you want more information about these procedures, I have a video introduction to inference procedures to two proportions available at: http://youtu.be/g0at6LpYvHc.
The male birth rate data is from:
Koshy et al. (2010). Parental smoking and increased likelihood of female births. Annals of Human Biology, 37(6): 789-800.
The data on the proportion of male births is from:
Koshy et al. (2010). Parental smoking and increased likelihood of female births. Annals of Human Biology, 37(6): 789--800.
For those using R, here is the R code for the example in this video:
NB R uses a different definition of the random variable than I do here. I define the random variable X to be the number of trials required to get the first success. R defines the random variable to be the number of *failures* before getting the first success (let's call this Y). Then Y = X - 1, and we'll have to make this adjustment when using dgeom, pgeom, or rgeom. Some might find this confusing, and if you do, don't use these functions.
Sampling from a large population where 30% have CPR training until we get the first person with CPR training.
Finding the probability that it happens on the sixth person sampled:
(.3)*(.7)^5
[1] 0.050421
or
dgeom(6-1,.3)
[1] 0.050421
Finding the probability that it happens on or before the third person sampled:
.3+.3*.7+.3*.7^2
[1] 0.657
or
1-.7^3
[1] 0.657
or
pgeom(3-1,.3)
[1] 0.657
In this proof I use the fact that the sampling distribution of the sample mean has a mean of mu and a variance of sigma^2/n. If you need that to be shown as well, I show that in this video: http://youtu.be/7mYDHbrLEQo.
The mean and standard deviation of the heights of American males is based on information from:
McDowell MA, Fryar CD, Ogden CL, Flegal
KM. Anthropometric reference data for children
and adults: United States, 2003-2006.
National health statistics reports; no 10.
Hyattsville, MD: National Center for Health
Statistics. 2008.
(The mean is taken directly from this document, and the standard deviation is estimated based on the given percentiles.)
The guinea pig survival time data can be found in:
Doksum, K. (1974). Empirical probability plots and statistical inference
for nonlinear models in the two-sample case. Annals of Statistics, 2:267--277.
In this video it is assumed that we are dealing with sample data, and not data representing the entire population. This will be the case the vast majority of the time in practice.
The birth weight data is from random sample of 1000 males drawn from Table 7-2 (Live births, by birth weight and geography -- Males) of the Statistics Canada publication 84F0210X, available at \url{http://www.statcan.gc.ca/pub/84f0210x/2009000/t011-eng.htm}.
This video contains some mathematical details regarding the t distribution (the pdf, mean, variance, how the distribution arises). Depending on your needs, this may not be important to you. In that event, I also have a video with a less technical introduction to the t distribution, available at http://youtu.be/Uv6nGIgZMVw.
Plutonium-239 example (an average of 2.3 radioactive decays per second).
Finding the probability of exactly 3 radioactive decays in a 2 second period:
dpois(3,2*2.3)
[1] 0.1630676
Finding the probability of no more than 3 radioactive decays in a 2 second period:
dpois(0,2*2.3)+dpois(1,2*2.3)+dpois(2,2*2.3)+dpois(3,2*2.3)
[1] 0.3257063
or
ppois(3,2*2.3)
[1] 0.3257063


