Understanding tests in statistics, everyone should know this
People often confuse statistical significance with a number of samples that is wrong and specify n=<some value> under statistical significance which does not make sense.
For checking the statistical significance of an insight we would state the null hypothesis and alternate hypothesis, then calculate the p-value. If the p-value i.e. probability of observed results of a test assumes that the null hypothesis is true.
Lastly, the significance level is decided (alpha=0.05 is most common) and then if :
p-value is greater than alpha: H0 (Null hypothesis) is true
p-value is less than alpha: HA (Alternate hypothesis) is true
Majorly Hypothesis tests are divided into the following categories:
Parametric Vs Non-Parametric Tests :
There are tests that have underlying assumptions that the outcome data is normal and those are called parametric tests. Non-parametric tests on the other hand have fewer underlying assumptions, sometimes they are called distribution-free tests.
Non-parametric tests are used when your data isn’t normal. Therefore the key is to figure out if you have normally distributed data. For example, you could look at the distribution of your data. If your data is approximately normal, then you can use parametric statistical tests.
Framework for implementing any statistical test :
Step 1: State H0 and HA i.e. Null hypothesis and Alternate Hypothesis respectively
Step 2: State confidence level or level of significance (alpha =0.05.0.01 etc) based on the sensitivity
Step 3: Select appropriate tests and find test statistics.
Step 4: Establish or draw the critical region.
Step 5: Decision Making based on calculations. This step can be done by any of the following approaches:
- Critical value approach (Reject H0 if |test_statistics| > critical value else fail to reject H0)
2. p-value approach (Reject H0 if p-value < alpha else fail to reject H0)
3. Confidence Interval Approach
In the next article , I will explain these steps with an example and will also go through errors in hypothesis testing.
Thanks!
