What is Multinomial Logistic Regression? Definition & Examples

Logistic regression is a concept used in many fields, including machine learning. It is an example of a supervised machine learning algorithm that predicts or calculates the probability of occurrence of a binary (yes/no) event. For example, if we use machine learning to determine if an email is spam or not, we apply the logistic regression model. Now, the multinomial logistic regression classification technique is an extension of the logistic regression algorithm and solves problems that have multiple outcomes. 

Enrol for the Machine Learning Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

This article will look at the multinomial regression algorithm and its use in machine learning.

Variables in a Logistic Regression Model

The response variable is categorical in any logistic regression model. The value of a response, outcome, or dependent variable responds to changes in the independent or explanatory variables. 

Source

Here’s an example to understand the difference between an explanatory variable and a response variable.

Suppose a scientist wants to compare the effect of two different fertilisers, X and Y, on plant growth. The scientist randomly selects 100 plants from a field and applies fertiliser X to half of them for a week and fertiliser B to the other half for the same period. After a week, the scientist measures the average plant growth in each.

In this example, the type of fertiliser (X or Y) is the independent or explanatory variable the scientist changes to observe an effect on plant growth. On the contrary, plant growth is the dependent or response variable changing due to fertiliser application.

The Three Types Of Logistic Regression

Before we go into the details of multinomial logistic regression, let’s look at the three kinds of logistic regression models.

Understanding the Multinomial Logistic Regression Algorithm

The multinomial logistic regression model is a classification algorithm that extends the concept of logistic regression to solve multiclass possible outcome problems with one or more independent variables. While binary logistic regression predicts binary outcomes (0 or 1, yes or no, spam or not spam, etc.), the multinomial regression model predicts one out of k possible outcomes (k can be any arbitrary positive integer). Multinomial logistic regression is also known as softmax regression, multiclass logistic regression, multinomial logit model, conditional maximum entropy model, and maximum entropy (MaxEnt) classifier. 

Dependent Variable in Multinomial Logistic Regression

The dependent variable in the multinomial logistic regression algorithm can have two or more possible classes/outcomes. The dependent variable is nominal, meaning there is no order in the target-dependent classes. In other words, we cannot meaningfully order these classes. Thus, multinomial logistic regression is practical to make classifications based on the values of a set of predictor variables. 

Here’s a simple example to understand the dependent and independent variables in multinomial logistic regression:

Suppose we have a machine learning model that uses multinomial logistic regression to predict the ice cream flavour a person is likely to choose. Here, factors such as the person’s age, gender, mood, occasion, income status, and price of ice cream are the independent variables that determine the ice cream flavour the person will possibly go for. In this example, the dependent variable is the ice cream flavour that can belong to many categories (chocolate, vanilla, butterscotch, coffee, etc.).

Assumptions for Multinomial Logistic Regression

When you choose multinomial logistic regression as the classification algorithm for your machine learning problem, you must ensure the data you analyse satisfies the following assumptions. 

Assumption #1

The dependent variable should be either nominal or ordinal. A nominal variable has two or more categories with no meaningful ordering, such as three types of cuisines: Continental, Chinese, and Italian. On the contrary, ordinal variables have two or more categories with an order. An example of an ordinal variable would be the grades in an exam, that is, Excellent (A), Good (B), and Average ( C ). 

Assumption #2

You have a set of one or more independent variables that can be continuous, nominal, or ordinal. Continuous variables are numeric variables and can have an infinite number of values within a specified range. However, you must treat ordinal independent variables as continuous or nominal when running a multinomial logistic regression in SPSS.

Assumption #3

The observations must be independent, and the dependent variables must be mutually exhaustive and exclusive. Mutually exhaustive implies every observation must fall into some category of the dependent variable. On the other hand, mutually exclusive means when there are two or more categories of the variable, no observation falls into more than one category.

Assumption #4

There must be no multicollinearity amidst independent variables. Multicollinearity happens when more than two independent variables have a high correlation, making it difficult to understand the contribution of each independent variable to the dependent variable category. 

Assumption #5

The data points must not have outliers, highly influential points, or high leverage values. 

Assumption #6

Lastly, any constant independent variable and the dependent variable’s logit transformation must have a linear relationship. The idea behind a logit is to restrict the probability values between 0 and 1 using a logarithmic function. Most multinomial logistic regression models are based on logit functions. 

Multinomial Regression Using SPSS

IBM® SPSS® Statistics is a robust statistical software platform used by researchers for various complex statistical data analyses. This section will give an example of data analysis using multinomial logistic regression on the SPSS platform.

For multinomial logistic regression, we consider a situation to understand if a person’s income and belief that taxes are too high determine the political party they vote for. The dependent variable is the political party with three categories (A, B, and C). The independent variables are the person’s income, and their belief that taxes are too high with four ordered categories (“Strongly Agree,” “Agree,” Disagree,” and “Strongly Disagree”).

Here are the steps of the analysis in SPSS Statistics:

1. Under the SPSS main menu, click Analyze > Regression > Multinomial Logistic…

Source

The following Multinomial Logistic Regression dialogue box opens.

Source

2. Enter “politics” (the dependent variable), “tax_too_high” (the ordinal variable ), and “income” (the covariate variable).in the Dependent, Factor(s), and Covariate(s) boxes, respectively.

Source

3. Now, click on the Statistics option, and you’ll get the following Multinomial Logistic Regression: Statistics dialogue box.

Source

4. Check the following boxes: Cell probabilities, Classification table, and Goodness-of-fit.

Source

5. Click Continue to return to the Multinomial Logistic Regression dialogue box.

6. Click OK to generate the results.

Summing Up

Multinomial logistic regression is used for classification problems when the dependent variable or output is categorical or dichotomous. While analysing data using multinomial regression, you must keep in mind certain assumptions around the different dependent and independent variables, observations, and training data available. Multinomial logistic regression is a widely used classification algorithm in machine learning. 

If you’re looking to enhance your machine learning and AI skills, here’s your chance with upGrad’s Master of Science in Machine Learning & AI in association with Liverpool John Moores University. 

Programme Highlights:

• Master’s degree from LJMU
• Executive PGP from IIIT Bangalore
• 750+ hours of learning
• 90 live sessions
• 15+ case studies and assignments
• 11 coding assignments
• Comprehensive coverage of 20 top languages, tools, and libraries

Sign up today to get exclusive upGrad benefits such as 360-degree career support services, hands-on projects and hackathons, personalised mentorship from industry experts, peer networking opportunities, and much more!