Does increasing sample variance increase likelihood of rejecting null hypothesis?

The probability of not committing a Type II error is called the power of a hypothesis test.

Effect Size

To compute the power of the test, one offers an alternative view about the "true" value of the population parameter, assuming that the null hypothesis is false. The effect size is the difference between the true value and the value specified in the null hypothesis.

Effect size = True value - Hypothesized value

For example, suppose the null hypothesis states that a population mean is equal to 100. A researcher might ask: What is the probability of rejecting the null hypothesis if the true population mean is equal to 90? In this example, the effect size would be 90 - 100, which equals -10.

Factors That Affect Power

The power of a hypothesis test is affected by three factors.

• Sample size (n). Other things being equal, the greater the sample size, the greater the power of the test.
• Significance level (α). The lower the significance level, the lower the power of the test. If you reduce the significance level (e.g., from 0.05 to 0.01), the region of acceptance gets bigger. As a result, you are less likely to reject the null hypothesis. This means you are less likely to reject the null hypothesis when it is false, so you are more likely to make a Type II error. In short, the power of the test is reduced when you reduce the significance level; and vice versa.
• The "true" value of the parameter being tested. The greater the difference between the "true" value of a parameter and the value specified in the null hypothesis, the greater the power of the test. That is, the greater the effect size, the greater the power of the test.

Test Your Understanding

Problem 1

Other things being equal, which of the following actions will reduce the power of a hypothesis test?

I. Increasing sample size.
II. Changing the significance level from 0.01 to 0.05.
III. Increasing beta, the probability of a Type II error.

(A) I only
(B) II only
(C) III only
(D) All of the above
(E) None of the above

Solution

The correct answer is (C). Increasing sample size makes the hypothesis test more sensitive - more likely to reject the null hypothesis when it is, in fact, false. Changing the significance level from 0.01 to 0.05 makes the region of acceptance smaller, which makes the hypothesis test more likely to reject the null hypothesis, thus increasing the power of the test. Since, by definition, power is equal to one minus beta, the power of a test will get smaller as beta gets bigger.

Problem 2

Suppose a researcher conducts an experiment to test a hypothesis. If she doubles her sample size, which of the following will increase?

I. The power of the hypothesis test.
II. The effect size of the hypothesis test.
III. The probability of making a Type II error.

(A) I only
(B) II only
(C) III only
(D) All of the above
(E) None of the above

Solution

The correct answer is (A). Increasing sample size makes the hypothesis test more sensitive - more likely to reject the null hypothesis when it is, in fact, false. Thus, it increases the power of the test. The effect size is not affected by sample size. And the probability of making a Type II error gets smaller, not bigger, as sample size increases.

2 Answers By Expert Tutors

Dattaprabhakar G. answered • 07/19/14

Expert Tutor for Stat and Math at all levels

Michael B. is looking only at two groups.

The question is ambiguous.  It mentions t-test, but, does not say what does the phrase "increasing the sample variance" mean?  Are we talking about the t-test for one sample?  Two-sample?  "Matched-pairs"?

Generally speaking, increasing the sample variance implies increasing its square-root the sample std dev, which in turn, increases the estimated std error of the sample mean.  Now, the estimated std error appears in the denominator of the t-test, so when the std error of the sample mean increases, the value of the t-statistic decreases.

Whether or not it increases the likelihood of "rejecting H_o" depends on what the alternative hypothesis H_1  is. The answer depends on the rejection region.

I need further information along the lines stated above, (what type of t-test?, what alternative hypothesis)  before the question can be answered at all. At his point there are too many holes in the statement of the problem.

Michael B. answered • 07/19/14

Doctor Loves to Teach Computer, Statistics

You can do this with no calculations or formulas if you understand the idea of the t-test. In fact, you don't even need to know you're using a t-test. Just think about the underlying problem.

You have two groups with different means, and you want to know how likely it is, if the null hypothesis were true, that the difference between those two means (mean1-mean2) would be as great as you have observed. What does "increasing the sample variance" imply about the estimated variability of the population? If the population variability is huge, then will it be more or less likely that the sample means will vary by at least a given amount? If it is more or less likely to find two groups varying by that amount given H0, then how does that affect the likelihood of rejecting H0?

Does increasing sample size increases the likelihood of rejecting the null hypothesis?

How does sample variance influence the likelihood of rejecting the null hypothesis?

How does variance affect the null hypothesis?

What is the effect of increasing the sample variance?