How the Random Forest Algorithm Works in Machine Learning

Detailed Working Mechanism of Random Forest Algorithm in Machine Learning

Building the Forest

"Building the forest" in a Random Forest algorithm pertains to the methodology of generating numerous individual decision trees, each developed using a random selection of the data and features, which are subsequently merged to create the "forest."

The steps below describe - How the Random Forest Algorithm Works in Machine Learning:

Step 1: Choose random samples from a specified dataset or training set.

Step 2: This algorithm will create a decision tree for each training dataset.

Step 3: The voting process will occur by averaging the decision tree.

Step 4: Ultimately, choose the prediction result that received the highest number of votes as the final outcome.

Making Predictions

In a Random Forest algorithm, outcomes are generated by aggregating the predictions from numerous decision trees, with each tree trained on a random portion of the dataset. The ultimate prediction is established by performing a majority vote (for classification) or averaging the results (for regression) among all trees in the forest.

Applications of Random Forest Algorithm

Healthcare

In healthcare, RFs provide numerous opportunities for early diagnosis that are not only more affordable than neural networks but also address the ethical issues related to NNs. Although neural networks demonstrate remarkable effectiveness in various clinical prediction tests, their application in the actual clinical setting is challenging because they function as black-box models, which results in a lack of interpretability.

Although decision-making in neural networks lacks traceability, it is entirely clear in the scenario of random forest. Healthcare providers can comprehend the reasons behind the decisions made by the random forest. For instance, if an individual suffers negative side effects or passes away from treatment, they can clarify the reasoning behind the algorithm's decision.

Finance

In the finance industry, random forest analysis can be used to forecast mortgage defaults and detect or thwart fraud. Consequently, the algorithm assesses whether the customer is prone to default or not. To identify fraud, it can examine a sequence of transactions and assess their likelihood of being fraudulent.

An additional illustration. RFs can be trained to predict the likelihood of a customer closing their account by analyzing transaction patterns and frequency. By utilizing this model on the total population of current users, we can predict churn for the upcoming months. This gives the company highly valuable business insights that assist in pinpointing bottlenecks and establishing a lasting collaboration with customers.

If you're in finance, discover the fintech development services offered by our skilled team.

E-Commerce

The algorithm is being utilized more frequently in e-commerce to predict sales.

Imagine you are attempting to forecast if an online customer will purchase a product after viewing an advertisement on Facebook. In this scenario, there might be just a few shoppers who made a purchase after viewing the advertisement (perhaps 5% made a purchase), whereas a significantly larger group of shoppers did not make a purchase.

Applying the random forest classifier to customer information like age, gender, personal preferences, and interests will enable you to forecast which customers are likely to purchase and which are not with fairly high precision. By directing an advertising campaign towards potential customers, you’ll enhance your marketing investment and boost sales.

Also Read: Decision Tree Interview Questions & Answers [For Beginners & Experienced]

Advantages & Limitations of Random Forest Algorithm

In the following section, we examine the main benefits that render Random Forest a powerful instrument for numerous data-oriented activities. 

Advantages

• Capability to understand non-linear decision boundaries 

Random Forest is an ensemble learning method that employs various decision trees to generate predictions. It can represent intricate, non-linear connections between characteristics and the target variable. 

• Elevated precision 

It minimizes the overfitting issue in decision trees and aids in enhancing the precision. It decreases prediction variance in comparison to an individual decision tree. 

• Adaptable and strong 

Random Forest is capable of managing a diverse range of data types, such as numeric and categorical data. It can manage outliers and missing values, and does not need feature scaling since it employs a rule-based method instead of calculating distances. 

• Significance of Features 

Random forest offers insights into the significance of each feature within the data, which can greatly assist in comprehending the underlying patterns. 

• Scalability 

Random forest is capable of managing extensive datasets with high dimensionality, which makes it a favored option in various industries. 

• Simultaneous processing 

Trees can be generated concurrently, as there is no dependency among iterations, thereby accelerating the training duration. 

Limitations

Like any approach, weighing the limitations against the advantages can assist you in developing a well-rounded perspective on the algorithm. When using random forests, some limitations to keep in mind are as follows: 

• Interpretation 

More difficult to interpret than an individual decision tree, as the prediction cannot be clarified by just one diagram. Nonetheless, the significance of the variables can still be obtained. 

• Algorithmic Complexity 

Random forest may require considerable computational resources, especially when dealing with extensive datasets. It demands significant memory, which may pose a limitation when operating with restricted resources. 

• Noise sensitivity 

While random forest generally avoids overfitting, it can still happen in some situations, especially when dealing with noisy data. 

Implementation of Random Forest in Python using Scikit-learn

We will now demonstrate the implementation of the Random Forest Classifier using a dataset from Kaggle. The dataset will be downloaded directly from Kaggle using the Kaggle API, preprocessed for missing values and categorical variables, and then used to train a Random Forest model for classification. We will evaluate the model's performance using accuracy and a classification report.

Titanic Dataset Overview

The Titanic dataset is widely used in machine learning to demonstrate classification models. It contains passenger details from the RMS Titanic disaster, and the objective is to predict whether a passenger survived based on their characteristics.

Dataset Source

The dataset is available on Kaggle under the title "Titanic - Machine Learning from Disaster".

STEP 1: Downloading the Titanic Dataset from Kaggle in Google Colab

To fetch the dataset directly from Kaggle into Google Colab, follow these steps:

1. Setting Up Kaggle API in Google Colab

Kaggle provides an API that allows us to download datasets programmatically. To use it, we need to upload our Kaggle API token (kaggle.json) to authenticate.

1.1: Upload kaggle.json

1. Go to your Kaggle account → Click on your profile picture → Select "Account".
2. Scroll down to the API section and click "Create New API Token".
3. A file named kaggle.json will be downloaded. This file contains your API credentials.
4. Upload this file to Google Colab.

1.2: Run the Following Code to Set Up Kaggle API

Before downloading the dataset, we need to place kaggle.json in the correct directory and set the appropriate permissions.

# Install Kaggle API if not installed
!pip install kaggle
# Make a directory for the Kaggle token and move it there
!mkdir -p ~/.kaggle
!cp kaggle.json ~/.kaggle/
# Set proper file permissions
!chmod 600 ~/.kaggle/kaggle.json
# Verify if Kaggle API is working
!kaggle datasets list

2. Download the Titanic Dataset from Kaggle

Kaggle datasets are stored in competitions or datasets sections. The Titanic dataset belongs to a Kaggle competition, so we use the following command to download it:

# Download the Titanic dataset from Kaggle competition
!kaggle competitions download -c titanic  
# Unzip the dataset
!unzip titanic.zip  

3. Extracted Files

After running this command, you should see the following extracted files:

• train.csv – This contains the training data (used for model training).
• test.csv – This contains the test data (used for predictions).
• gender_submission.csv – A sample
• submission file for the Kaggle competition.

STEP 2: Load the Dataset into Pandas 

We will now use Pandas to load and inspect the dataset.

import pandas as pd
# Load the training dataset
df = pd.read_csv("train.csv")
# Display the first few rows of the dataset
df.head()

Output for Step 2:

STEP 3: Check Dataset Information

Before proceeding with preprocessing, it's essential to understand the dataset's structure, missing values, and data types.

Run the following commands to get an overview

# Display dataset information
df.info()
# Check for missing values
df.isnull().sum()  #This will give a count of missing values in each column, guiding us on data cleaning strategies.

Output for Step 3:

STEP 4: Data Preprocessing

Before we train our Random Forest Classifier, we need to clean and preprocess the dataset.

Step 4.1: Handling Missing Values

We already checked for missing values using df.isnull().sum().

Now we will fill the Missing values with either Mean, Median or The most common value, from its neighbours.

# Drop the "Cabin" column
df.drop(columns=["Cabin"], inplace=True)
# Fill missing "Age" values with the median
df["Age"].fillna(df["Age"].median(), inplace=True)
# Fill missing "Embarked" values with the most common value
df["Embarked"].fillna(df["Embarked"].mode()[0], inplace=True)
# Verify that no missing values remain
df.isnull().sum()

Step 4.2: Encoding Categorical Variables

Machine learning models cannot work with categorical text data, so we must convert these into numbers.

The Titanic dataset has the following categorical variables:

Sex (male, female) → Convert to 0 and 1.

Embarked (C, Q, S) → Convert to numerical values using one-hot encoding.

# Convert "Sex" column to numerical (0 = female, 1 = male)
df["Sex"] = df["Sex"].map({"male": 1, "female": 0})
# One-hot encode "Embarked" column
df = pd.get_dummies(df, columns=["Embarked"], drop_first=True)
# Display dataset after encoding
df.head()

Step 4.3: Splitting the Data into Training and Testing Sets

Features (X) – Independent variables used for prediction.

Target (y) – The Survived column (1 = survived, 0 = did not survive).

We will also split the data into training and testing sets using an 80-20 split.

from sklearn.model_selection import train_test_split
# Define feature columns (excluding "Survived" and irrelevant columns)
X = df.drop(columns=["Survived", "Name", "Ticket", "PassengerId"])  # Drop non-useful columns
y = df["Survived"]  # Target variable
# Split into training (80%) and testing (20%) sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Display shape of training and testing sets
X_train.shape, X_test.shape

STEP 5: Training the Random Forest Classifier

Now that we have preprocessed the data, we can train a Random Forest Classifier.

This model is an ensemble learning method that builds multiple decision trees and combines their predictions to improve accuracy and reduce overfitting.

Step 5.1: Importing and Training the Model

We will use Scikit-Learn’s RandomForestClassifier and fit it to our training data.

from sklearn.ensemble import RandomForestClassifier
# Initialize the Random Forest model with default parameters
rf_model = RandomForestClassifier(random_state=42, n_estimators=100)
# Train the model
rf_model.fit(X_train, y_train)

Output of Step 5.1

Step 5.2: Making Predictions

Now that our model is trained, we will use it to predict survival on the test dataset.

# Predict on the test set

y_pred = rf_model.predict(X_test)

Step 5.3: Model Evaluation

To evaluate the model, we will calculate accuracy, precision, recall, and F1-score.

These metrics will help us understand how well our model performs.

Accuracy – How many total predictions were correct.

Precision & Recall – Performance for each class (Survived vs. Not Survived).

F1-score – A balance between precision and recall.

from sklearn.metrics import accuracy_score, classification_report
# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Model Accuracy: {accuracy:.2f}")
# Print classification report
print("Classification Report:\n", classification_report(y_test, y_pred))

Final Accuracy and Classification report (Output to step 5.3)

Interpreting the Model Performance

Our Random Forest Classifier achieved an accuracy of 81%, meaning that 81% of the test data was correctly classified. Let’s break down the classification report:

Class 0 (Did Not Survive)

• Precision: 0.83 → When the model predicts a passenger did not survive, it is 83% correct.
• Recall: 0.86 → The model correctly identifies 86% of all actual non-survivors.
• F1-score: 0.84 → A balanced measure of precision and recall.

Class 1 (Survived)

• Precision: 0.79 → When the model predicts a passenger survived, it is 79% correct.
• Recall: 0.74 → The model correctly identifies 74% of all actual survivors.
• F1-score: 0.76 → A balance between precision and recall.

Overall Model Performance

• The macro average (0.80) considers both classes equally and suggests a good balance in prediction quality.
• The weighted average (0.81) accounts for class imbalance, meaning the model is fairly reliable in predicting both survival and non-survival.
• The slightly lower recall for class 1 suggests that the model misses some actual survivors, which could be improved with hyperparameter tuning.

Also Read: A Day in the Life of a Machine Learning Engineer: What do they do?

How upGrad will help You

UpGrad offers thorough courses in machine learning that will assist you in understanding and mastering the Random Forest algorithm, encompassing the theoretical foundations, practical execution, and sophisticated techniques tied to this influential ensemble learning approach. This enables you to effectively utilize it for real-world data analysis and prediction across different fields such as healthcare, finance, and marketing, with support on data preparation, feature engineering, hyperparameter tuning, model assessment, and result interpretation. 

The following courses from upGrad can prove to be very beneficial:

• Executive Diploma in Machine Learning and AI with IIIT-B
• Post Graduate Certificate in Machine Learning & NLP (Executive)
• Post Graduate Certificate in Machine Learning & Deep Learning (Executive)
• Advanced Certificate Program in GenerativeAI

Take the next step in your Machine Learning journey with confidence—upGrad’s free counseling session can guide you toward the right career path.

Expand your expertise with the best resources available. Browse the programs below to find your ideal fit in Best Machine Learning and AI Courses Online.

Best Machine Learning and AI Courses Online

Discover in-demand Machine Learning skills to expand your expertise. Explore the programs below to find the perfect fit for your goals.

In-demand Machine Learning Skills

Discover popular AI and ML blogs and free courses to deepen your expertise. Explore the programs below to find your perfect fit.

Popular AI and ML Blogs & Free Courses

References
https://aiml.com/what-are-the-advantages-and-disadvantages-of-random-forest/
https://www.pickl.ai/blog/advantages-and-disadvantages-random-forest/
https://serokell.io/blog/random-forest-classification
https://data36.com/random-forest-in-python/