Significance tests for a proportion

The applet below allows one to visually investigate hypothesis tests for a proportion. Specify the sample size (n), the true proportion (True p), the null value for the proportion (Null p) and the alternative for the test (Alternative). When you click the Simulate button, 100 separate samples of size n will be selected from a population with a proportion of successes equal to True p. For each of the 100 samples, a hypothesis test based on the Z statistic is performed, and the results from each test are displayed in the plots to the right. The test statistic for each test is shown in the top plot, and the P-value is shown in the bottom plot. The green and blue lines represent the cut offs for rejecting the null with the 0.05 and 0.01 level tests, respectively. Additional simulations can be carried out by clicking the Simulate button multiple times. The cumulative number of times that each test rejects the null hypothesis is also tabled. Press the Clear button to clear existing results and start a new simulation. Things to try with the applet:

  • Simulate at least 1000 tests with n = 100, p = 0.5, Null p = 0.5 and the not equal alternative. What proportion of the 0.05 level tests reject the null? What proportion of the 0.01 level tests reject the null?
  • Simulate at least 1000 tests with n = 100, p = 0.5, Null p = 0.2 and the not equal alternative. What proportion of the 0.05 level tests reject the null? What proportion of the 0.01 level tests reject the null?
  • Simulate at least 1000 tests with n = 30, p = 0.1, Null p = 0.1 and the not equal alternative. What proportion of the 0.05 level tests reject the null? What proportion of the 0.01 level tests reject the null?
  • Simulate at least 1000 tests with n = 30, p = 0.1, Null p = 0.5 and the not equal alternative. What proportion of the 0.05 level tests reject the null? What proportion of the 0.01 level tests reject the null?
  • You can also use this applet to simulate a power curve. For a fixed value of null p, simulate 1000 tests (10 clicks of the simulate button) for each value of true p over a range of 0.1,0.2,...,0.9. Plot the simulated power curve which is the proportion of times the null is rejected versus the value of true p. You can repeat this process for different types of alternatives and for different values of null p and n in order to get a better understanding of how these quantities impact the power curve of the test.