t-tests for a single sample (Lab 5)

Objectives

Today, we’ll combine some plotting and some t-tests. As we’ve already discussed in class, it turns out that however complicated the math is for doing a t-test by hand, it’s infinitely simpler in Jamovi or in another software program. We’ll also talk a bit more about doing z-tests for a sample.

Remember that a z-test for a sample is one where we compare a sample mean to a population where we know the population mean and variance, whereas a one-sample t-test is one where we compare a sample mean to a population with known population mean, but unknown population variance.

You’ll turn in an “answer sheet” on Brightspace. Please be sure to turn that in by the the time your lab starts next week.

The data

Today you’ll be looking at the friends dataset again. The friends dataset is the same as late week, available on Brightspace. It is unchanged from the data you downloaded in the last lab, if you’ve got that saved. (But it is different from the first time we used the friends data!)

Manipulating this dataset in Jamovi

Under the Data ribbon menu, use a Filter function to filter out people with only one sibling listed in the column called siblings (i.e., remove people who have only one sibling, but people with zero siblings should be included.) Said another way: you want everyone with 0 siblings or more than one, but not one sibling.

How do I do this? Learn more about filters (click to expand)

If you’re not sure how to say “not” in the filter: “not” is an exclamation point (!), as it is in many software or programming languages. Thus, writing != is like saying “not equivalent”. If you’re wanting to instead use “or”, Jamovi expects you to use the word or in there. Note that <> is not a way that works for Jamovi for you to write “not equal”.

To recap (using x here instead of a variable; you could imagine replacing it with siblings):

• “is x equal to 5?” – x == 5
• “is x NOT equal to 5?” – x != 5
• “is x more than 5?” – x > 5
• “is x less than or equal to 5?” – x <= 5
• “is x more than 5 or less than 2?” – x > 5 or x < 2

1. In your answer sheet, write the code you used for filtering only rows where siblings isn’t 1. This is #1 for your answer sheet.

2. Find the mean of the siblings variable when the filter is on (so folks who have one sibling are not included). (You should probably use Analyses: Exploration: Descriptives.) Then find the mean of siblings when nothing is filtered out. Which one is larger? These answers are #2 on your answer sheet.

3. Turn off the filter. Using Jamovi (or, if you prefer, Excel/Sheets), make a bar plot of gram.followers split by siblings. Your final plot should not have anyone filtered out, and should have 6 separate bars. See more than that? Try switching the gram.followers to make sure it’s marked as a continuous (rather than nominal) variable. Save (or screenshot) this plot and include it as #3 in your answer sheet. Remember that you can save a plot by right-clicking on it in Jamovi and Exporting it. If right-click and Copy doesn’t work, you should export as PNG and then share that image.

   You may also want to create a histogram for this instagram follower data when it is not split by siblings, just to see what the data look like. Are they normal?

4. Look at the plot below. For #4 on your answer sheet, explain what is different in this plot compared to the bar plot you created in Jamovi. I certainly don’t object to obvious details (e.g., this shows points instead of bars), but which plot do you think better allows you to make conclusions about this relationship?

z-tests for a single sample

Suppose we are interested in whether our sample has, on average, the same age at first job as people in general. We could say our research hypothesis is that our sample has a different mean age of a first job, but the null hypothesis is that it is the same. (Step 1!)

You might say then that:

\[H_0: \mu_{our~sample}=\mu_{people~in~general}\]

\[H_1: \mu_{our~sample}\neq{}\mu_{people~in~general}\]

Now, how do we define people in general? This is pretty tricky in most cases—which is part of the challenge and why we’re about to stop using z-tests, since we can’t truly know this information. For now, though, let’s imagine that “people in general” are Americans, and guess at when most Americans have their first job of any kind. However, we can choose a guess here as—perhaps 15, which is one year after most Americans are legally allowed to work. Is our sample different from that number?

Because we’re still doing a z-test here, let’s also imagine a population standard deviation of, say, 2. Remember that we need this information when we’re doing a one-sample z-test. (But that it is unavailable in the one-sample t-test! We’ll get there in a moment.)

Step 2: Determine the characteristics of the comparison distribution

Okay, let’s get the information about our sample and therefore define the comparison distribution. We’ll use this info for calculating z.

We asked people “At what age did you have your first job? (Write 0 if you have never had a job.)” This means that we might have some zeros in our sample. Let’s exclude them. Filter out participants who answered 0 for firstjobage. We’ll leave this filter on while answering this question.

5. This question has four parts which I would like you to label a, b, c, and d. We’ll start with three and get to the fourth later. For your answer sheet, (a) will be the \(M\) (the sample mean), (b) will be the mean of the comparison distribution where \(\mu_M=\mu\), (c) will be the standard deviation of the comparison distribution (\(\sigma_M\)), and (d) will—a bit further down—be the standard deviation for our sample, or \(S\), from which we’ll calculate \(S_M\) a bit later. But I’ll walk you through getting each of these.

1. Get the mean age for our sample’s firstjobage in Jamovi (from Descriptives) and write it down. This is #5a.

We don’t need to calculate the population standard deviation—we’ve got it (even if we’ve made it up). (That is, we’re not yet using the SD of the sample to define our comparison distribution, but the \(\sigma\) of the population.) We have the information about the population already. We know that \(\mu=15\) (because I told you) and that \(\sigma=2\) (again, because I told you). But now we need to define the comparison distribution.

2. Based on the central limit theorem, we know that \(\mu_M=\mu\). So the comparison distribution’s mean is the same as the population distribution’s mean. Write the comparison distribution’s mean value as #5b. This is as simple as it seems. Use the value of \(\mu\).

3. To get the standard deviation of the comparison distribution, we need to get the SD for the sampling distribution of the mean, which has a standard deviation equal to the standard error of the mean—that’s \(\sigma_M=SEM=\frac{\sigma}{\sqrt{n}}\). We have an n—it’s the number of participants with data for firstjobage, and Jamovi probably gave it to you when you found the mean. (We don’t include those with missing data here.) Use a calculator, your phone, a spreadsheet—whatever method—to calculate this. Write the comparison distribution’s SD down as #5c.

So, we can define our comparison distribution as follows: it is a z-distribution based on the sampling distribution of the mean, which has a mean and SD defined as you wrote above and is normally distributed.

We’ll come back to 5d a bit later.

Step 3: Determine the sample cutoff score to reject the null hypothesis

We’re still using the same cut-offs from all the z-tests here, which correspond to any z-distribution with a significance level of \(p < .05\). That means that our “extreme” scores are those less than -1.96 or more than +1.96—i.e., that \(z_{crit}=\pm1.96\).

Step 4: Determine your sample’s score

6. Okay, let’s find z. Use the equation below, where M is the sample mean, \(\mu_M\) is the mean for the sampling distribution of the means, and \(\sigma_M\) is the SEM and is the standard deviation of the sampling distribution of the means. (You’re essentially going to be plugging in the answers from #5.) Find z and write it as answer #6.

\[z=\frac{M_{sample}-\mu_{M}}{\sigma_M}\]

Step 5: Decide whether or not to reject the null hypothesis

Compare your z-score to the cutoff score. If it’s positive: is it larger than +1.96? If it’s negative: is it smaller than -1.96? This is not an answer for your answer sheet.

Based on the z-score and the cutoff, should we reject the null hypothesis? (click to expand)

Assuming you excluded the 0s for firstjobage by filtering them out: Yes, we should reject the null. The z-value you found was larger (more rare) than 1.96, meaning that you can conclude that this is different from the null distribution. We reject the null hypothesis.

If you did not exclude those 0s, you’ll find that you shouldn’t reject the null. This is because you have a lower mean for our sample, and a much larger standard deviation.

Switching from z to t

Now we’ll try to do this whole thing again, with a slight difference: looking at the most basic kind of t-score. A one-sample t-test has one primary difference from the basic z-test: rather than assuming that we know the population’s standard deviation, we instead accept that we do not. All we know in this instance is the population mean. So, here, let’s repeat this test. Only this time, we’re going to calculate the standard deviation ourselves from our sample. And this time, let’s compare to a population mean that assumes more people are working earlier—perhaps count the kid with the lemonade stand—and imagine a population mean of \(\mu=14\).

What do we need to do to find an estimate of the population standard deviation? Well, the SD of our sample is an unbiased estimate of the population standard deviation! (This is why we talked about reducing its bias by subtracting 1 from N a few weeks ago and again more recently!)

Let’s go through the steps of hypothesis testing.

Step 1

Step 1 hasn’t changed. \(H_0: \mu_{our~sample}=\mu_{people~in~general}\) and \(H_1: \mu_{our~sample}\neq{}\mu_{people~in~general}\)

Step 2

We’re describing the t-distribution for comparison.

Our steps for doing this (after noting that its shape is that of a t distribution) are:

1. Use the sample data to estimate population standard deviation
2. Estimate the standard deviation for the distribution of means—the standard error based on the SD for the sample and its sample size

For (i), we can look back at the Descriptives in Jamovi. What is the standard deviation for our sample? That is the estimate of the population standard deviation, \(S\).

5. 4. Write down the standard deviation for our sample—the dispersion for how much ages of first jobs vary. (Go back and edit: Add this as #5d.)

To do (ii), remember that \(S_M=\frac{S}{\sqrt{n}}\), where \(S_M\) is the standard deviation of the comparison distribution, \(S\) is the standard deviation estimated from the sample, and \(n\) is the sample size.

7. Calculate \(S_M\). This is answer #7.

We have now defined our comparison distribution!

It’s the t-distribution with \(df=n-1\), based on a mean of \(\mu_M=\mu=14\), and a standard deviation of \(S_M\) (the standard error of the mean) which you just found in #7. If you’re keeping track of the things we know about our sample and comparison distributions of first job ages for this question, you might add the \(\mu_M\) and the df.

Step 3

Skip the t-table…

Professor Matt Bognar of the University of Iowa created a free app for probability distributions called Probability Distributions. You can get it by clicking for iOS or Android.

I think that there is a conceptual point to using the t-table, and it will be what I expect you to use on an exam. However, I’ll walk you through using this app below if you’d like to.

First, though, pull out the t-table on Brightspace or the one in your book. Look up the critical t-value at the df we’re using for this test. (Remember, our n and therefore our df are only based on the participants who have data for the firstjobage variable. And of course, \(df=n-1\).) Look up the column for two-tailed tests (since we default to that) and to find if \(p<.05\) (since, again, that’s our default). What is the \(t_{crit}\) for that df? You may need to round down for your df when using the t-table.

If you ever need to round your df in a t-table, always round down.

8. You can write your resulting cutoff value like the following: \(t_{crit}(df)=\pm?.??\) – for example, if your df was 4, you’d write \(t_{crit}(4)=\pm2.78\). Add the critical t-value you find as #8.

Use the probability distributions app (click to expand)

Once you have found your df, open up the probability distributions app. You can find probabilities for z by clicking “Normal”. However, here, I’m asking you to use t, so click that letter. You will see four options you can choose: one dropdown and three text entry fields. At the top, it reads \(X~t(\nu)\). The app uses the Greek letter \(\nu\) (nu) to indicate the df. Yes, it looks like a v.

1. To the center, it should have selected the option P(X<x)=—read it as “the probability that a given value is less than your selected value”. You can change this to find the different tails of your distribution. Scroll down to make it read 2P(X>|x|)= – this is for a two-tailed test, which we are doing.
2. Enter the df in the white blank for \(\nu\).
3. For the moment, leave the blue blank for x= blank.
4. In the red box at the right, enter the probability you’re looking for—here, because we’re doing a two-tailed test, you should enter .05.
5. Click done.

The app should show a diagram with the cutoff values in blue (\(x=\) it) and a pale red fill coloring the critical regions. At this sample size, the t-curve looks relatively normal.

Play around with the options. See what the app can do.

Step 4: Determine your sample’s score

Okay, now we can calculate t. The formula to do this looks quite similar to the one for z, with the exception (to repeat) that the only thing we’re claiming to “know” at this point is the population mean—the \(S_M\) is coming from our estimate of the population standard deviation, \(S\), where \(S_M=\frac{S}{\sqrt{n}}\) and S is the sample’s standard deviation:

\[t=\frac{M-\mu_M}{S_M}\]

The M is our sample mean. The \(\mu_M\) is the mean of the comparison distribution, here equal to the population mean.

9. Calculate t. This is answer #9.

Step 5: Decide whether or not to reject the null hypothesis

10. Is the value you found in #9 more extreme than the critical value we identified? Compare it to the cutoff. Write your conclusion as #10. Be specific: do you reject the null hypothesis for this test? Then write your conclusions, including the results of the test. Use the information below on how to write up the results and your conclusion.

How to write up rejecting the null

If you were going to reject the null, you might write (but would not use these numbers):

We reject the null. People in our sample had a different first job age than those on average, \(t(4)=2.99, p<.05\).

We usually also report the means, so for example you might write:

We reject the null. People in our sample (\(M=XX\)) had a different first job age than those on average (\(\mu=14\)), \(t(4)=2.99, p<.05\).

Note that for p-values, which will always be between 0 and 1, no leading 0 is required. (You don’t have to write \(p<0.05\); writing \(p<.05\) is fine.)

If you have a “really rare” result where \(p<.001\), Jamovi might tell you that. However, you should always report the results based on the question you asked. If you asked “is \(p<.05\)?”, which we did here, you should answer that question by writing \(p<.05\) even if the actual p-value is a lot smaller.

How to write up failing to reject the null

If you failed to find evidence to reject the null hypothesis, you might write (but also would not use these numbers):

We fail to reject the null. People in our sample (\(M=XX\)) did not have a significantly different first job age than those on average (\(\mu=14\)), \(t(4)=0.30, p>.05\).

Note that here we use the “more than” sign, >, because our probability is more than 5% that we would find a result as rare as this. Struggling to remember this? A silly but maybe helpful mnemonic is “if p is low, the null must go!” When you have a small p-value, you can reject the null. When you do not—when \(p>.05\)—you cannot.

Remember that a p-value is the probability of obtaining results from the test that are at least as extreme as what we found if the null were true. So when \(p<.05\) we conclude that it is probably not the case that the null is true—we reject it, whereas when \(p>.05\) we conclude that it is entirely possible that the null is actually true—and therefore we retain it.

The “phrase” \(p<.05\) is something we write when we have rejected the null, because it means that we’re concluding that it was unlikely under the null. We can also find an exact p-value and report that, but I recommend only doing so when you have failed to reject the null.

Use the probability distributions app to find an exact p-value (click to expand)

If you’ve downloaded the app, you can enter your df and the t you found into the app, and it will give you an exact p-value in the red box. Make sure that the final option on the chooseable menu is selected (2P(X>|x|)=).

If you find the p-value and you have rejected the null hypothesis, the p-value you found should be less than .05. You should report that \(p<.05\).

If you find the p-value and you have failed to reject the null hypothesis, the p-value you found should be greater than .05. You could report the p-value. For example:

We fail to reject the null. People in our sample (\(M=XX\)) did not have a significantly different first job age than those on average (\(\mu=14\)), \(t(4)=0.30, p=.78\).

Here, \(p>.05\), and therefore you would still reject the null.

Run an actual t-test in Jamovi

Okay, now we get to do this the easy way. In Jamovi, under Analyses, select T-Tests, then One Sample T-Test. Put firstjobage into Dependent Variables.

Under Hypothesis, put our guess at the population mean (\(\mu=14\)) into the “Test Value” box.

If you do not change the “test value”, your t-value will be wrong

The default is for Jamovi to test for a change from 0. Of course our sample is different from 0! Change it to the population mean of 14.

You should see the same Statistic (that’s the t-value) and df as you found! The t might be rounded slightly differently than you found, though. What you’ll also see is an exact p-value. If you got something different than the t-value Jamovi gives you, discuss and ask for help if needed. Your p-value should also match the one the app gave you if you used that.

Lastly, under Additional Statistics (still in the t-test pane), click on “Mean difference” and “Confidence Interval”. You’ll see two new columns pop up to the right of your t-test results. What might those be? What’s the confidence interval showing?

What are the mean difference and confidence interval? (click to expand)

The mean difference is our sample mean MINUS the “population mean” that we gave it (14). The confidence interval is showing that there’s a huge variety there. It’s possible the real confidence interval is as high as the bigger number—which would be a relatively large difference!—but it’s also possible it’s negative. If the confidence interval overlaps with 0, that is a good sign that the test wouldn’t be statistically significant.

Great! You now know how to run a one-sample t-test in Jamovi.

Try on your own

Turn off any filters that you have enabled.

Can you tell me if our sample has a different number of covidtimes (i.e., how many times they reported having had covid) compared to a population mean of 1? (That seems low because it is, but bear with me on trying it.)

11. Decide what test you’re doing, then follow the five steps of null hypothesis significance testing (NHST) to find your answer. As answer #11, write up your results (like above) to tell me whether our sample has a different number of times they’ve had covid compared to that supposed population mean. You don’t need to share your steps; just give me the conclusion, written like I showed you above. (If you’re not sure, you can show your work.) Use Jamovi—or do it by hand—it’s up to you.

12. Now, do another test. Suppose we have a question about how many times our sample eats at Klein per week. We think that most people probably eat two meals per day at Klein, so we guess that the population mean should be around 14 (\(\mu=14\)). We measured “How many times do you eat at Klein per week?” with the variable eatatklein. Create a boxplot of this variable, and determine if there are any outliers. (Exclude them if so.) Then, run a t-test for one sample to compare our data with the population mean. If you used Jamovi in #11, do this one by hand, and vice versa. Does our sample eat at Klein more or less than that supposed population mean? Report the resuls for #12.