How to Analyze Kolmogorov-Smirnov Test - Non Parametric

The Kolmogorov–Smirnov test is a non-parametric analysis designed to determine whether two independent samples come from two different populations. This test was originally developed by Smirnov in 1939. However, due to similarities between the test proposed by Smirnov and the goodness-of-fit test previously developed by Kolmogorov in 1933, Daniel in 1980 subsequently referred to this test as the Kolmogorov-Smirnov Test. This test serves as an alternative to the T-Test for two independent samples, particularly when the assumption of normality cannot be met.

Case Example

Sourced from: Handbook of Parametric and Nonparametric Statistical Procedures, Fifth Edition By David J. Sheskin

Example 12.1

To assess the efficacy of a new antidepressant, ten patients suffering from clinical depression are randomly placed into one of two groups. Five patients are placed in Group 1, who are administered the antidepressant for a period of six months. The other five patients are placed in Group 2, who are administered a placebo for the same six-month period. Assume that before initiating the experimental treatment, the researcher ensures that the depression levels in both groups are identical. After six months have elapsed, all ten subjects are evaluated by a psychiatrist (who is blind to the experimental conditions of the subjects) regarding their level of depression. The psychiatrist's depression scores for the five subjects in each group are as follows (higher scores indicate greater depression): Group 1: 11, 1, 0, 2, 0; Group 2: 11, 11, 5, 8, 4. Do the data suggest that the antidepressant is effective?

Analysis Steps

The following are the steps for conducting the Kolmogorov–Smirnov test and the post hoc test using SmartstatXL, an Excel Add-in:

1. Activate the worksheet (Sheet) to be analyzed.
2. Place the cursor on the dataset (for creating the dataset, see Data Preparation method). The dataset can be organized in two layouts:
3. Grouped based on level/tier (comparison between tiers)
4. Grouped based on variable (comparison between variables)
   
5. If the active cell is not on the dataset, SmartstatXL will automatically try to determine the dataset.
6. Activate the SmartstatXL Tab
7. Click on the Non-Parametric Menu. SmartstatXL will display a dialog box to confirm whether the dataset is correct or not (usually the cell address of the dataset is automatically selected correctly).
   
8. If it is correct, click the Next Button
9. Next, the Non-Parametric Test dialog box will appear:
   
10. If the data layout used is based on comparison between variables, the following dialog box will appear:
    
11. Finally, press the "OK" button

Analysis Results

The following is the Output Analysis of the Kolmogorov–Smirnov Test:

Statistical Summary

From the analysis results, we aim to determine whether the new antidepressant has a significant effect in reducing depression levels compared to the placebo. To ascertain this, we use the Kolmogorov-Smirnov test.

Let us interpret these results:

1. Proposed Hypotheses:
   • H0 (Null Hypothesis): The cumulative distribution of both groups is the same. This means there is no difference between the group given the antidepressant and the group given the placebo.
   • H1 (Alternative Hypothesis): The cumulative distribution of both groups is different.
2. Statistical Results:
   • M value is 0.800.
   • Chi Square value is 6.400.
   • p-value (probability) is 0.041.
   • The critical value for this test is 5.991.
3. Interpretation:
   • Since the Chi Square value (6.400) is greater than the critical value (5.991), and the p-value (0.041) is smaller than 0.05, we reject the null hypothesis (H0). This means there is sufficient evidence to state that there is a significant difference between the group given the antidepressant and the group given the placebo in terms of depression levels.

Conclusion:

Based on the Kolmogorov-Smirnov Test analysis, there is sufficient evidence to claim that the new antidepressant is effective in reducing depression levels compared to the placebo at the 0.05 significance level.

Average Comparison

Table of mean values, Mean Rank, Sum Rank, and Post hoc Test

From the table, we can see the comparison between the two groups based on their average values:

1. Data Description:
   1. Group 1: With an average of 2.80.
   2. Group 2: With an average of 7.80.
2. Subset Interpretation:
   1. Group 1 has a subset label "a," while Group 2 has a subset label "b." This indicates that there is a significant difference between the two groups based on their average values.

Conclusion:

Based on the average comparison, Group 2 has a higher level of depression compared to Group 1.

Comparison with Previous Mann-Whitney Test: In the Mann-Whitney Test results, although there is a difference in the average values between the two groups, the statistical test results show that the difference is not significant at the 0.05 level. However, in the Kolmogorov-Smirnov Test results, we find that the difference between the two groups is significant.

This highlights the importance of choosing the right statistical test based on the nature of the data and the research objectives. Although both tests are non-parametric tests used for comparing two groups, they have different assumptions and calculation methods, which can lead to different conclusions.