What does it mean if a researcher sets his/her alpha at say .01, and rejects the null hypothesis? How does this differ from setting the alpha at .05 and rejecting the null? In which case is the researcher going to be most likely to reject the null hypothesis?

In statistical hypothesis testing, the alpha level (α) is the probability of rejecting the null hypothesis when it is true. It is also known as the significance level.

1. If a researcher sets the alpha at .01, it means that they are willing to accept a 1% chance of incorrectly rejecting the null hypothesis, i.e., making a Type I error. If they reject the null hypothesis under this condition, it means that the results of their study are statistically significant at the 1% level. In other words, there is only a 1% probability that the observed results occurred by chance alone if the null hypothesis is true.

2. If the alpha is set at .05, the researcher is willing to accept a 5% chance of incorrectly rejecting the null hypothesis. Rejecting the null hypothesis under this condition means that the results are statistically significant at the 5% level. That is, there is a 5% probability that the observed results occurred by chance alone if the null hypothesis is true.

3. The difference between setting the alpha at .01 and .05 lies in the degree of certainty required to reject the null hypothesis. An alpha of .01 requires a higher degree of certainty (or more compelling evidence in the data) to reject the null hypothesis than an alpha of .05.

4. The researcher is more likely to reject the null hypothesis when the alpha is set at .05 than at .01. This is because the alpha level determines the threshold for what is considered statistically significant. A higher alpha level (like .05) means a higher threshold for statistical significance, making it easier to reject the null hypothesis. Conversely, a lower alpha level (like .01) means a lower threshold for statistical significance, making it harder to reject the null hypothesis.