Fisher's LSD - Post Hoc Test

Fisher's LSD (Least Significant Difference)

The LSD test is a procedure for testing the difference between the simplest and most used treatment means.  This method was introduced by Fisher (1935), so it is also known as the Fisher's LSD [Least Significant Difference] Method.  To use the LSD test, the attributes we need are the error mean square value (MSE), the significant level, the degree of freedom (df) of the error, and the t-student table to determine the critical value of the comparison test.

There are several things to note in using this test:

1. Use the LSD test when the F test in the analysis of variance is significant
2. The LSD procedure will maintain a significant level of ≤ 0.05 only if the comparison of all combinations of mean value pairs of treatment ≤ 3 treatments
3. Use the LSD test for planned comparison without regard to the number of treatments.  For example, if we want to compare all the mean treatments with the controls, the LSD test can be used even if it is more than 3 treatments.

The formula for calculating the value of LSD is as follows:

$$\begin{matrix}LSD=t_{\alpha/2,dfe}.s_{\bar{Y}};where\ s_{\bar{Y}}=\sqrt{\frac{2MSE}{r}}\ \\if\ number\ of\ replication\ different:\\LSD=t_{\alpha/2,dfe}\sqrt{MSE(\frac{1}{r_i}+\frac{1}{r_j})}\ \\\end{matrix}$$

where r is the sum of the multiplications, MSE = Mean Square of Error obtained from the analysis of the variance, α = significant level, dfe = error-degree of freedom.  The value of t is the value obtained from the table of t-students at a significant level α with the degree of freedom = dfe (In the table t-student it is usually predetermined for two-way testing, so α in the table the actual value of α/2). 

In the LSD test, to assess whether the two mean values of the treatment differ statistically, we compare the calculated LSD values with the absolute  difference between the two means.  If the difference is greater than the LSD value, it is said that the two means differ markedly at the level of α.  Mathematically, the statement can be summarized:

The LSD test states m_i and m_j differs to a significant degree if: |  am_i - m_j | > LSD

Fisher's Protected t Tests: Experiment wise Type I Error Rate (a_EW)

1. Protected conditions:
   • Ho is entirely correct :           m₁ = m₂ = ... = m_k
   • Or only one of H₀ is correct:   m₁ = m_{2 3}¹m
2. Unprotected conditions:
   • Partial H₀ is true:
   • $\begin{matrix}\mu_1=\mu_2=\mu_3\neq\mu_4\\\mu_1=\mu_2\neq\mu_3=\mu_4\\\mu_1\neq\mu_2=\mu_3=\mu_4\\\end{matrix}$
3. In this case, if testing more than one H0 then the Type I (a_EW) error will increase very quickly with the increasing number of treatments
4. The Type I error value is calculated according to the formula:
   • $\alpha_{EW}=1-(1-\alpha)^j\ with\ \ j=\frac{t(t-1)}{2}$

Calculation Procedure:

As an illustration of the use of the LSD test, for example, there are Variousness Analysis Calculation Results along with the Mean Table of treatments as presented in the following Table : 

Anova Table of Nitrogen Content

Table of Mean Nitrogen Content

Planned Comparison

Here there are 6 treatments that we will compare, so it is not appropriate if the LSD test is used to compare all pairs of treatment combinations (pair wise comparisons).  However, if we want to compare all treatment means with controls (planned comparison), the LSD test can be used even if it is more than 3 treatments.  In this case, the Combined is used as a control (comparison) so that all treatment means are only compared to the control.

1. Calculate the LSD value:
   • Determine the value of the MSE and its degree of freedom obtained from the Analysis of variance Table.
   • MSE = 11.7887
   • df = 24
   • Specify the t-student value. 
   • There are two parameters needed to determine the t-student value, namely the significant level (α) and the error-degree of freedom (df).  In this example, the values df= 24 (see df error in the Analysis of Their Variance table) and α = 0.05. Next, specify the value of t_{(0.05/2, 24)}.
   • To find the value of t_{(0.05/2, 24)} we can see it in the table The distribution of t-students at a significant level of 0.05 with a degree of freedom of 24.  Consider the following image to specify the t-table.
     
     
   • From the table we get the value of the value of t_{(0.05/2, 24)} = 2.064
   • Calculate the LSD value by using the following formula:
   • $\begin{matrix}LSD=t_{0.05/2;24}\sqrt{\frac{2MSE}{r}}\\=2.064\times\sqrt{\frac{2(11.79)}{5}}\\=4.482\ \ mg\\\end{matrix}$
2. Test criteria:
   • Compare the absolute value of the difference between the two means that we will see the difference with the LSD value with the following test criteria:
   • $ if\ \ \left|\mu_i-\mu_j\right|\ \ \left\langle\ \ \begin{matrix}>4.48\ reject\ H_0,\ two\ means\ are\ significantly\ different\\\le 4.48\ accept\ H_0,\ two\ means\ are\ not\ significantly\ different.\\\end{matrix}\right.$
   • In this case, we only compared the difference between the controls (Combined) and the mean of the other treatments.

Description:

*=significant different; ns = not significant
give the notation of the letter "a" if it is not significantly different from the combined and give the letter "b" if it is significantly different
The treatment of 3Dok13, 3Dok5, and 3Dok1 is different from combined.

Pairwise comparison

Although we do not appropriately use the LSD test to compare all combinations of treatment pairs if the number of treatments > 3, but as an illustration in the use of the LSD test, here will be described the procedure for comparing all pairs of treatment mean combinations.  As a comparison, here will be reused the Analysis of variance table and the same treatment mean table as in the planned LSD test use case above.

1. Calculate the LSD value:
   • From the previous discussion, we have obtained an LSD value of 4.482 mg.
2. Test criteria:
   • Compare the absolute value of the difference between the two means that we will see the difference with the LSD value with the following test criteria:
     $ if\ \ \left|\mu_i-\mu_j\right|\ \ \left\langle\ \ \begin{matrix}>4.48\ reject\ H_0,\ two\ means\ are\ significantly\ different\\\le 4.48\ accept\ H_0,\ two\ means\ are\ not\ significantly\ different.\\\end{matrix}\right.$

Description: *=different significant

Another way of presenting the comparison of all pairs of combinations of means of simpler and more informative treatment is to use letter notation as a differentiator.  There are several ways in which letter notation is given as a differentiator of the mean treatment. All of them have the same basic logic/principle, which is to compare the difference in the mean treatment with the comparison value.  Here only 2 ways are described in the provision of such letter notation.

First Method

Create a Matrix (Crosstabulation) the mean difference between all combinations of treatment pairs.  Since it is a handlebar, just make a lower triangle matrix table.

Stages of Creating mean difference matrix table:

Step 1: Sort the treatment mean value from small to large or vice versa. In this example, the mean treatment is sorted from small to large

Step 2: Create a Matrix Form of the bidirectional table (crosstabulation) as in the following example:

Step 3: Fill in the table body contained in the red box with the mean difference in treatment.

Step 4: The full result is as follows:

Step 5.  Compare the difference in the mean treatment contained in the table body (bordered in red) with the comparison value, in this case the LSD value.  Give the code "ns" when the mean difference value is smaller or equal to the LSD value and give the code "*" when it is greater than the LSD value.  Next give the same line for treatment that does not differ markedly and finally give the letter notation. The full stages can be seen in the table description.

Information: 

1. The number on the body of the Table is the value of the difference between the means of the treatment
2. Compare the mean difference with the LSD value (4.48).  Give the code "ns" when the mean difference value is less or equal to the LSD value, and give the symbol "*"when the mean difference value is greater than the LSD value
3. On each column (starting from the 4th column) drag/give the same line starting from the number 0. Continue drawing the line in the same column at the mean difference value given the symbol "ns" and end when below it there is already a symbol "*".  Continue comparison the treatment in the next column.  The detailed stages are as follows:
   • In each column (in this example, columns 4 to 9) compare the difference in mean with the value of LSD, if it is smaller than the value of LSD (ns), give/draw the same line to the right of it.  The granting of the same line is stopped when the mean difference value > the LSD value (finds the "*" sign).
   • 3Dok13 vs other treatments (comparison in the 4th column)
     • For example, if we compare 3Dok13 with 3Dok4, compare the difference (1.38) with the value of LSD = 4.50. Since 1.38 ≤ 4.48 which indicates no difference, then we give the same line to the two means.
     • 3Dok13 vs Combined: Compare the difference (5.44) with LSD = 4.48. Since 5.44 > 4.48 (*), stop! The same line is not given again. Continue comparison the treatment in the next column, in this case the 5th column is 3Dok4 vs other comparison.
   • 3Dok4 vs other treatments (comparison in the 5th column).
     • 3Dok4 vs Combined. Compare the difference (4.06) with LSD = 4.48. Since 4.06 ≤ 4.48 (ns) so give the same line to the column 3Dok4.
     • 3Dok4 vs 3Dok7: 5.28 > 4.48 (*); stop! the same line is not given again. Continue with the comparison in the next column, the 6th column which is the Combined vs other treatment. etc. comes to the comparison in the 9th column.
4. Ignore (discard) Lines that are red, because they are already represented by the lines in the previous comparison!
5. Finally, provide the letter code for the line.  The line that is notated is only the line that is black.  The first black line is given the letter "a", the second letter "b", the third letter "c", etc... ignore the lettering of the red line (the line already represented by the other line).

Second Method

Another way of giving letter notation as a differentiator on the presentation of the comparison of all pairs of combinations of the mean treatment is as in the following step.

1. Calculate the value of LSD (in the previous calculation, LSD = 4.482 mg)
2. Sort the mean value of the treatment from small to large or vice versa. In this example, the mean treatment is sorted from small to large
3. The next step is to calculate the difference between the mean treatments to determine which treatment means are the same and which are different.  The next step is to determine the letters on those mean values.  How to determine this letter is a bit complicated, but if we are used to it, it is not a problem anymore.  In principle, the work is carried out repeatedly (iteration) between the step forward (the provision of new letter notation) and the step back (to double-check whether there is an mean treatment in the previous sequence that has the same notation as the new notation).

The full stages are as follows: 

1. Step Forward #1:
   • Give the letter quality "a" to the mean value of the first smallest treatment, which is 13.26.
   • add the value 13.26 with the LSD value: 13.26 + 4.482 = 17.742. 
   • give the same letter for the mean value of the treatment that is less than or equal to the value of 17.742.

• It appears that 3Dok13 (13.26) is no different from 3Dok4 (14.64), because 14.64 ≤ 17.742, while 3Dok13 (13.26) is already different from Combined because 18.70 > 17.742!
• Since the Combined Treatment is different from 3Dok13, then give the letter "b" to the mean value of the treatment.

2. Step Back:

• We've given the letter b to the Combined treatment, but we don't yet know if the combined is different from the other treatments located between 3Dok13 and the Combined!  In this example the treatment between 3Dok13 and Combined is only one treatment, which is 3Dok4.  A more practical way is to check the step back, which is to compare the Combined with the previous treatment whose value is smaller than the combined but greater than 3Dok13.
• Subtract combined mean value by LSD value: 18.70-4.482 = 14.218
• Give the letter 'b" to the mean value of the treatment whose value ≥ 14.218.
• In this case, it turns out that the treatment of 3Dok4 (14.64) is ≥ 14.218, so that in addition to being given the letter "a" code, it is also given the letter "b" because it is no different from combined.

3. Step Forward #2:

• The starting point now starts from the smallest treatment that gets the letter "b", in this case on the 3Dok4 (14.64) treatment.
• add the mean value of the treatment 3Dok4 (14.64) with the LSD value: 14.64 + 4.482 = 19.912. 
• give the same letter (in this case "b") for the mean value of the treatment that is less than or equal to the value of 19.912.  Incidentally, in this case, the letter "b" is only up to the combined treatment which has previously received the letter "b". The result is still the same as before.

• It appears that 3Dok7 (19.92) is different from 3Dok4 because 19.92 > 19.912.
• Because it is different, then give the letter "c" to the mean value of the treatment of 3Dok7 (19.92).

4. Step Back:

• Subtract the mean value of 3Dok7 (19.92) by the LSD value: 19.92-4.482 = 15.438
• Give the letter 'c" to the mean value of the treatment whose value ≥ 15.438.
• In this case, it turns out that the Combined treatment (18.70) value ≥ 15.438, so that in addition to being given the letter code "b", it also gets the letter "c" because it is no different from 3Dok7.

5. Step Forward #3:

• The starting point now starts from the smallest treatment that gets the letter "c", in this case on the Combined treatment (18.70).
• add the mean value of the Combined treatment (18.70) with the LSD value: 18.70 + 4.482 = 23.182. 
• give the same letter (in this case "c") for the mean value of the treatment that is less than or equal to the value of 23.182.  It also happens that in this case, the letter "c" is only up to the treatment of 3Dok7 (19.92) which has previously got the letter "c". The result is still the same as before.

• It appears that 3Dok5 (23.98) is different from Combined because 23.98 > 23.182.

• Because it is different, then give the letter "d" to the mean value of the treatment 3Dok5 (23.98).

6. Step Back:

• Subtract the mean value of 3Dok5 (23.98) with the LSD value: 23.98-4.482 = 19.498
• Give the letter 'd" to the mean value of the treatment whose value ≥ 19.498.
• In this case, it turns out that the 3Dok7 (19.92) treatment value ≥ 19.498, so that in addition to being coded with the letter "c", it also gets the letter "d" because it is no different from 3Dok5. 

7. Step Forward to #4:

• The starting point now starts from the smallest treatment that gets the letter "d", in this case on the 3Dok7 (19.92) treatment.
• add the mean value of the treatment 3Dok7 (19.92) with the LSD value: 19.92 + 4.482 = 24.402. 
• give the same letter (in this case "d") for the mean value of the treatment that is less than or equal to the value of 24.402.  It also happens that in this case, the letter "d" is only up to the treatment of 3Dok5 (23.98) which has previously got the letter "d" so the result is still the same as before.
• It appears that 3Dok1 (28.82) is different from 3Dok7 (19.92) because 28.82 > 24.402.
• Because it is different, then give the letter "e" to the mean value of the treatment 3Dok1 (28.82).

8. Step Back:

• Subtract the mean value of 3Dok1 (28.82) by the LSD value: 28.82-4.482 = 24.338
• Give the letter "e" to the mean value of the treatment whose value ≥ 24.338.
• In this case, it turns out that there is no other treatment anymore whose value ≥ 24.338, so only 3Dok1 gets the letter "e".

Setting notation manually is a little tricky, isn't it? We must work on some stages that are done repeatedly between the steps forward and backward.   Many stages vary depending on the number of mean treatments we will compare and the distribution of variance of differences between the mean treatments.  In this case there are 4 steps forward so that the notation is given to the 5th letter, namely "e".  If it is only 1 stage, then the notation is given only up to the letter "b".  In general: 

Count of step forward = stage of notation – 1.

With the rapid advancement of computing technology, the above problems are no longer a problem.  Many statistical software already provide notation or grouping non-different mean values in the same group.  For example, SPSS and SAS have automatically provided letter notation if we do a follow-up test (post hoc) and STATISTICA is a non-different mean value in the same group (column) whose analogy is the same as giving letter notation or by way of giving lines.  Look again at the Table above, there are 5 groups/groups/columns.  The first column is analogous to the letter "a", the 2nd is analogous to the letter "b" etc.  If the mean value is marked on two columns (e.g. 3Dok4 treatment in the case above) it means that he is given two letters or 2 lines!   

If we are used to it, the procedure for compiling the letter notation above can actually be summarized in the form of groupings on the same column for mean values that do not differ markedly, as in the following table example:

Information:

↓ Comparison with the next mean value (step forward)

→ the granting of a new notation (move on to the next column)

↑ Comparison with the previous mean value (step back)

Calculation using SmartstatXL Excel Add-In