F-Test : A Complete Guide

Two sample t-tests can validate a hypothesis containing only two groups at a time. For samples involving three or more groups, the t-test becomes tedious, as you have to perform the tests for each combination of the groups. Also, the possibility of a type-1 error increases in this process. You use ANOVA in such cases.

Analysis of variance (ANOVA) can determine whether the means of three or more groups are different. ANOVA uses F-tests to statistically test the equality of means.

To understand how ANOVA is applied, let’s go over a simple case:  

A test was conducted at a workplace, and the feedback on the three e-commerce platforms was recorded in a data set, which is as follows:

To begin with, create a null hypothesis ($H_o$) for your ANOVA test. In this case, your null hypothesis will be: “All the platforms are equally popular”. The alternate hypothesis (), thus, becomes “At least one of the platforms has different popularity from the rest”. Represent this information as:

H0: 𝛍1 = 𝛍2 = ... = 𝛍k (where k is the number of different populations or groups or treatment levels, in your case, it’s 3). By writing this, you suggest that the ‘mean’ of the different populations will be the same, which is your null hypothesis. If the statement above is proved at the end of your test, it will imply that all the platforms are equally popular. If not, then you accept your alternative hypothesis (HA).

There are a couple of things you should keep a note of while using ANOVA:

• You must be thinking that it is a fairly simple problem for us. You calculate the means of the three groups and compare them to reject or not reject the null hypothesis. Unfortunately, it is not that simple after all because your hypothesis considers the mean of a particular ‘population’, while your data set only has a ‘sample’ of that ‘population’. So, the mean that you calculate will be of the sample and not of the population. For instance, in your case, the people who have given feedback for, say, Amazon, are not the only ones who have used Amazon. There are many others too. But they form your sample, which is why the mean that you will calculate here will be of this ‘sample’ and not of the ‘population’.

• Another question that might crop up in your mind is: Why is the process called analysis of ‘variance’ when you are comparing ‘means’? This is because the math that you will use later in the process will require the concept of variance to study the means of the groups. It will tell you how the means vary or differ.

Now, as variance is the central idea behind ANOVA, let’s briefly revisit the topic:

1. Variance is the average squared deviation of a datapoint from the distribution mean. The distance between the sample mean and each datapoint is measured and squared. Then, you add them together and take the average. The formula is:

Here, s2 represents the variance, x represents the sample datapoints, x- bar represents the sample mean, and n represents the number of sample points.

2. If you momentarily ignore the average part, what you are left with is the ‘sum of squares’. So, the sum of squares is the variance without finding the average of the sum of squared deviations.

The sum of squares is given by: 

$SS=\sum (x-x̅{)}^2$

Let’s have a look at the dataset again:

We have talked about two kinds of calculations that you have to make in accordance with the variance. ‘Sum of squares between’ accounts for the variation between the groups, and ‘sum of squares within’ accounts for the variation within a group. The total sum of squares is the sum of all the variations that there are, and it gives us the deviation of each observation from the grand mean of the dataset. To understand this more clearly, let’s look at your case:

• SSB represents the variation of the mean feedback of a company, say Flipkart, from the grand mean of all the feedback.

• SSW represents the variation of all the feedback in a company from the mean of its feedback.

• TSS represents the variation of all the feedback in your dataset from the grand mean.

Let’s look at the basic formula you will be using:

Total sum of squares = Sum of squares between + sum of squares within the group

            (TSS)                                       (SSB)                                            (SSW)

(Image source: https://www.easycalculation.com/formulas/eta-squared-formula.html)

Here, ‘i’ represents the observations in a group or a treatment level, and ‘j’ refers to a particular group or a treatment level. In your case, ‘i’ will represent all the feedback of, say, Amazon, and ‘j’ refers to a particular group and can be Amazon, Flipkart, or Snapdeal.

: It represents the number of observations in a group. In your case, it will be the number of times feedback is received for, say, Snapdeal.

: It represents all the observations that have been recorded in the dataset.

: It represents the mean of a particular group or treatment.

: It represents the grand mean of all the observations. In your case, this will be the mean of all the feedback that has been collected.

Let’s now calculate the aforementioned measures for your data:

After you have calculated this data, the next step is to analyse the ANOVA table:                     

You have already calculated the sum of squares. Now, ‘df’ here represents the degrees of freedom.

• Between groups, df = Number of groups - 1

• Within a group, df = Total number of observations - Number of groups

• Degrees of freedom for the complete dataset = Total number of observations - 1    

Let’s calculate the degrees of freedom for your observations:

Mean square = Sum of squares/df

Using this formula, you can find the mean square between the groups as well as within the group.

Let’s calculate the mean squares for your calculations:

Before moving any further, let’s first see what the F-test is.

F-tests are named after the test statistic F, which was named in honour of Sir Ronald Fisher. The F-statistic is simply a ratio of two variances.

To use the F-test to determine whether group means are equal, all you need to do is include the correct variances in the ratio. In one-way ANOVA, the F-statistic is given by this ratio:

F = Variation between the sample means / variation within the samples

  = (MSB/MSW)

Now, you have to calculate the critical F-value using the F-distribution table for a given significance level and compare it with your calculated F-value. In your case, p < 0.05. The table looks like this:

Degrees of freedom of the numerator will be that of the df between the groups.

Degrees of freedom of the denominator will be that of the df within the group.

The intersection will give us the critical F-value. Now, you compare your calculated F-value with the critical F-value.

• If the calculated F < the critical F, you will fail to reject the null hypothesis.

• If the calculated F > the critical F, you will reject the null hypothesis.

Let’s now do the final calculations in your case to see whether Amazon, Flipkart and Snapdeal are equally popular:

Your F critical value is 3.3690, and your calculated value comes out to be 4.78. Therefore, you will reject the null hypothesis and accept the alternative hypothesis. Therefore, Amazon, Flipkart and Snapdeal are not equally popular.