Explore Courses
Liverpool Business SchoolLiverpool Business SchoolMBA by Liverpool Business School
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA (Master of Business Administration)
  • 15 Months
Popular
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Business Administration (MBA)
  • 12 Months
New
Birla Institute of Management Technology Birla Institute of Management Technology Post Graduate Diploma in Management (BIMTECH)
  • 24 Months
Liverpool John Moores UniversityLiverpool John Moores UniversityMS in Data Science
  • 18 Months
Popular
IIIT BangaloreIIIT BangalorePost Graduate Programme in Data Science & AI (Executive)
  • 12 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
upGradupGradData Science Bootcamp with AI
  • 6 Months
New
University of MarylandIIIT BangalorePost Graduate Certificate in Data Science & AI (Executive)
  • 8-8.5 Months
upGradupGradData Science Bootcamp with AI
  • 6 months
Popular
upGrad KnowledgeHutupGrad KnowledgeHutData Engineer Bootcamp
  • Self-Paced
upGradupGradCertificate Course in Business Analytics & Consulting in association with PwC India
  • 06 Months
OP Jindal Global UniversityOP Jindal Global UniversityMaster of Design in User Experience Design
  • 12 Months
Popular
WoolfWoolfMaster of Science in Computer Science
  • 18 Months
New
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Rushford, GenevaRushford Business SchoolDBA Doctorate in Technology (Computer Science)
  • 36 Months
IIIT BangaloreIIIT BangaloreCloud Computing and DevOps Program (Executive)
  • 8 Months
New
upGrad KnowledgeHutupGrad KnowledgeHutAWS Solutions Architect Certification
  • 32 Hours
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Popular
upGradupGradUI/UX Bootcamp
  • 3 Months
upGradupGradCloud Computing Bootcamp
  • 7.5 Months
Golden Gate University Golden Gate University Doctor of Business Administration in Digital Leadership
  • 36 Months
New
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Golden Gate University Golden Gate University Doctor of Business Administration (DBA)
  • 36 Months
Bestseller
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDoctorate of Business Administration (DBA)
  • 36 Months
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (DBA)
  • 36 Months
KnowledgeHut upGradKnowledgeHut upGradSAFe® 6.0 Certified ScrumMaster (SSM) Training
  • Self-Paced
KnowledgeHut upGradKnowledgeHut upGradPMP® certification
  • Self-Paced
IIM KozhikodeIIM KozhikodeProfessional Certification in HR Management and Analytics
  • 6 Months
Bestseller
Duke CEDuke CEPost Graduate Certificate in Product Management
  • 4-8 Months
Bestseller
upGrad KnowledgeHutupGrad KnowledgeHutLeading SAFe® 6.0 Certification
  • 16 Hours
Popular
upGrad KnowledgeHutupGrad KnowledgeHutCertified ScrumMaster®(CSM) Training
  • 16 Hours
Bestseller
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 4 Months
upGrad KnowledgeHutupGrad KnowledgeHutSAFe® 6.0 POPM Certification
  • 16 Hours
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Science in Artificial Intelligence and Data Science
  • 12 Months
Bestseller
Liverpool John Moores University Liverpool John Moores University MS in Machine Learning & AI
  • 18 Months
Popular
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
IIIT BangaloreIIIT BangaloreExecutive Post Graduate Programme in Machine Learning & AI
  • 13 Months
Bestseller
IIITBIIITBExecutive Program in Generative AI for Leaders
  • 4 Months
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
IIIT BangaloreIIIT BangalorePost Graduate Certificate in Machine Learning & Deep Learning (Executive)
  • 8 Months
Bestseller
Jindal Global UniversityJindal Global UniversityMaster of Design in User Experience
  • 12 Months
New
Liverpool Business SchoolLiverpool Business SchoolMBA with Marketing Concentration
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA with Marketing Concentration
  • 15 Months
Popular
MICAMICAAdvanced Certificate in Digital Marketing and Communication
  • 6 Months
Bestseller
MICAMICAAdvanced Certificate in Brand Communication Management
  • 5 Months
Popular
upGradupGradDigital Marketing Accelerator Program
  • 05 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Corporate & Financial Law
  • 12 Months
Bestseller
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in AI and Emerging Technologies (Blended Learning Program)
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Intellectual Property & Technology Law
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Dispute Resolution
  • 12 Months
upGradupGradContract Law Certificate Program
  • Self paced
New
ESGCI, ParisESGCI, ParisDoctorate of Business Administration (DBA) from ESGCI, Paris
  • 36 Months
Golden Gate University Golden Gate University Doctor of Business Administration From Golden Gate University, San Francisco
  • 36 Months
Rushford Business SchoolRushford Business SchoolDoctor of Business Administration from Rushford Business School, Switzerland)
  • 36 Months
Edgewood CollegeEdgewood CollegeDoctorate of Business Administration from Edgewood College
  • 24 Months
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with Concentration in Generative AI
  • 36 Months
Golden Gate University Golden Gate University DBA in Digital Leadership from Golden Gate University, San Francisco
  • 36 Months
Liverpool Business SchoolLiverpool Business SchoolMBA by Liverpool Business School
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA (Master of Business Administration)
  • 15 Months
Popular
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Business Administration (MBA)
  • 12 Months
New
Deakin Business School and Institute of Management Technology, GhaziabadDeakin Business School and IMT, GhaziabadMBA (Master of Business Administration)
  • 12 Months
Liverpool John Moores UniversityLiverpool John Moores UniversityMS in Data Science
  • 18 Months
Bestseller
O.P.Jindal Global UniversityO.P.Jindal Global UniversityMaster of Science in Artificial Intelligence and Data Science
  • 12 Months
Bestseller
IIIT BangaloreIIIT BangalorePost Graduate Programme in Data Science (Executive)
  • 12 Months
Bestseller
O.P.Jindal Global UniversityO.P.Jindal Global UniversityO.P.Jindal Global University
  • 12 Months
WoolfWoolfMaster of Science in Computer Science
  • 18 Months
New
Liverpool John Moores University Liverpool John Moores University MS in Machine Learning & AI
  • 18 Months
Popular
Golden Gate UniversityGolden Gate UniversityDBA in Emerging Technologies with concentration in Generative AI
  • 3 Years
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (AI/ML)
  • 36 Months
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDBA Specialisation in AI & ML
  • 36 Months
Golden Gate University Golden Gate University Doctor of Business Administration (DBA)
  • 36 Months
Bestseller
Ecole Supérieure de Gestion et Commerce International ParisEcole Supérieure de Gestion et Commerce International ParisDoctorate of Business Administration (DBA)
  • 36 Months
Rushford, GenevaRushford Business SchoolDoctorate of Business Administration (DBA)
  • 36 Months
Liverpool Business SchoolLiverpool Business SchoolMBA with Marketing Concentration
  • 18 Months
Bestseller
Golden Gate UniversityGolden Gate UniversityMBA with Marketing Concentration
  • 15 Months
Popular
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Corporate & Financial Law
  • 12 Months
Bestseller
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Intellectual Property & Technology Law
  • 12 Months
Jindal Global Law SchoolJindal Global Law SchoolLL.M. in Dispute Resolution
  • 12 Months
IIITBIIITBExecutive Program in Generative AI for Leaders
  • 4 Months
New
IIIT BangaloreIIIT BangaloreExecutive Post Graduate Programme in Machine Learning & AI
  • 13 Months
Bestseller
upGradupGradData Science Bootcamp with AI
  • 6 Months
New
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
KnowledgeHut upGradKnowledgeHut upGradSAFe® 6.0 Certified ScrumMaster (SSM) Training
  • Self-Paced
upGrad KnowledgeHutupGrad KnowledgeHutCertified ScrumMaster®(CSM) Training
  • 16 Hours
upGrad KnowledgeHutupGrad KnowledgeHutLeading SAFe® 6.0 Certification
  • 16 Hours
KnowledgeHut upGradKnowledgeHut upGradPMP® certification
  • Self-Paced
upGrad KnowledgeHutupGrad KnowledgeHutAWS Solutions Architect Certification
  • 32 Hours
upGrad KnowledgeHutupGrad KnowledgeHutAzure Administrator Certification (AZ-104)
  • 24 Hours
KnowledgeHut upGradKnowledgeHut upGradAWS Cloud Practioner Essentials Certification
  • 1 Week
KnowledgeHut upGradKnowledgeHut upGradAzure Data Engineering Training (DP-203)
  • 1 Week
MICAMICAAdvanced Certificate in Digital Marketing and Communication
  • 6 Months
Bestseller
MICAMICAAdvanced Certificate in Brand Communication Management
  • 5 Months
Popular
IIM KozhikodeIIM KozhikodeProfessional Certification in HR Management and Analytics
  • 6 Months
Bestseller
Duke CEDuke CEPost Graduate Certificate in Product Management
  • 4-8 Months
Bestseller
Loyola Institute of Business Administration (LIBA)Loyola Institute of Business Administration (LIBA)Executive PG Programme in Human Resource Management
  • 11 Months
Popular
Goa Institute of ManagementGoa Institute of ManagementExecutive PG Program in Healthcare Management
  • 11 Months
IMT GhaziabadIMT GhaziabadAdvanced General Management Program
  • 11 Months
Golden Gate UniversityGolden Gate UniversityProfessional Certificate in Global Business Management
  • 6-8 Months
upGradupGradContract Law Certificate Program
  • Self paced
New
IU, GermanyIU, GermanyMaster of Business Administration (90 ECTS)
  • 18 Months
Bestseller
IU, GermanyIU, GermanyMaster in International Management (120 ECTS)
  • 24 Months
Popular
IU, GermanyIU, GermanyB.Sc. Computer Science (180 ECTS)
  • 36 Months
Clark UniversityClark UniversityMaster of Business Administration
  • 23 Months
New
Golden Gate UniversityGolden Gate UniversityMaster of Business Administration
  • 20 Months
Clark University, USClark University, USMS in Project Management
  • 20 Months
New
Edgewood CollegeEdgewood CollegeMaster of Business Administration
  • 23 Months
The American Business SchoolThe American Business SchoolMBA with specialization
  • 23 Months
New
Aivancity ParisAivancity ParisMSc Artificial Intelligence Engineering
  • 24 Months
Aivancity ParisAivancity ParisMSc Data Engineering
  • 24 Months
The American Business SchoolThe American Business SchoolMBA with specialization
  • 23 Months
New
Aivancity ParisAivancity ParisMSc Artificial Intelligence Engineering
  • 24 Months
Aivancity ParisAivancity ParisMSc Data Engineering
  • 24 Months
upGradupGradData Science Bootcamp with AI
  • 6 Months
Popular
upGrad KnowledgeHutupGrad KnowledgeHutData Engineer Bootcamp
  • Self-Paced
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Bestseller
KnowledgeHut upGradKnowledgeHut upGradBackend Development Bootcamp
  • Self-Paced
upGradupGradUI/UX Bootcamp
  • 3 Months
upGradupGradCloud Computing Bootcamp
  • 7.5 Months
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 5 Months
upGrad KnowledgeHutupGrad KnowledgeHutSAFe® 6.0 POPM Certification
  • 16 Hours
upGradupGradDigital Marketing Accelerator Program
  • 05 Months
upGradupGradAdvanced Certificate Program in GenerativeAI
  • 4 Months
New
upGradupGradData Science Bootcamp with AI
  • 6 Months
Popular
upGradupGradFull Stack Software Development Bootcamp
  • 6 Months
Bestseller
upGradupGradUI/UX Bootcamp
  • 3 Months
PwCupGrad CampusCertification Program in Financial Modelling & Analysis in association with PwC India
  • 4 Months
upGradupGradCertificate Course in Business Analytics & Consulting in association with PwC India
  • 06 Months
upGradupGradDigital Marketing Accelerator Program
  • 05 Months

Decision Tree in Machine Learning Explained [With Examples]

Updated on 23 September, 2022

11.33K+ views
9 min read

Introduction

Decision Tree Learning is a mainstream data mining technique and is a form of supervised machine learning. A decision tree is like a diagram using which people represent a statistical probability or find the course of happening, action, or the result. A decision tree example makes it more clearer to understand the concept.

The branches in the diagram of a decision tree shows a likely outcome, possible decision, or reaction. The branch at the end of the decision tree displays the prediction or a result. Decision trees are usually used to find a solution for a problem which gets complicated to solve manually. Let us understand this in detail with the help of a few decision tree examples.

A decision tree is one of the popular as well as powerful tools which is used for prediction and classification of the data or an event. It is like a flowchart but having a structure of a tree. The internal nodes of the trees represent a test or a question on an attribute; each branch is the possible outcome of the question asked, and the terminal node, which is also called as the leaf node, denotes a class label.

In a decision tree, we have several predictor variables. Depending upon these predictor variables, try to predict the so-called response variable.

Enrol for the Machine Learning Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

Related Read: Decision Tree Classification: Everything You Need to Know

Decision Tree in ML

By representing a few steps in the form of a sequence, the decision tree becomes an easy and effective way to understand and visualize the possible decision options and the potential outcomes from the range. The decision trees are also helpful in identifying possible options and weighing the rewards and risks against each course of action that can be yielded. 

A decision tree is deployed in many small scale as well as large scale organizations as a sort of support system in making decisions. Since a decision tree example is a structured model, the readers can understand the chart and analyse how and why a particular option may lead to a corresponding decision. The decision tree example also allows the reader to predict and get multiple possible solutions for a single problem, understand the format, and the relation between different events and data with the decision.  

Each result in the tree has a reward and risk number or weight assigned. If you ever use a decision tree, then you will have every final result with a possible drawback and benefit. To conclude your tree properly,  you can span it as short or as long as needed depending on the event and the amount of data. Let us take a simple decision tree example to understand it better. 

Consider the given data which consists of the details of people like: whether they are drinker, smoker, their weight, and the age at which these people died.

Name Drinker Smoker Weight Age (Died)
Sam Yes Yes 120 44
Mary No No 70 96
Jonas Yes No 72 88
Taylor Yes Yes 55 52
Joe No Yes 94 56
Harry No No 62 93

Let us try to predict if the people will die at a younger age or older age. The characteristics like drinker, smoker, and the weight will act as a predictor value. Using these, we will consider age as a response variable.

Let us label that people who died before the age of 70 died “young” and people who died after the age of 70 died “old”. Let us now predict the response variable based on the predictor variable. Given below is a decision tree made after learning the data.

The decision tree above explains that, if a person is a smoker, they die young. If a person is not a smoker, then the next factor considered is if the person is a drinker or not. If a person is not a smoker and not a drinker, the person dies old.

If a person is not a smoker and is a drinker, then the weight of the person is considered. If a person is not a smoker, is a drinker, and weighs below 90 kg, then the person dies old. And lastly, if a person is not a smoker, is a drinker, and weighs above 90 kg, then they die young.

From the data given let’s take Jonas’ example to check if the decision tree is classified correctly and if it predicts the response variable correctly. Jonas is not a smoker, is a drinker, and weighs under 90 kg. According to the decision tree, he will die old (age at which he dies>70). Also, according to the data, he died when he was 88 years old, this means the decision tree example has been classified correctly and worked perfectly.

But did you ever wonder about the basic idea behind the working of a decision tree? In a decision tree, the set of instances is split into subsets in a manner that the variation in each subset gets smaller. That is, we want to reduce the entropy, and hence, the variation is reduced and the event or instance is tried to be made pure.

Let us consider a similar decision tree example. Firstly, we consider if the person is a smoker or not. 

Here, we are uncertain about the non-smokers. So, we split it into drinker and nondrinker.

We can see from the diagram given below that we went from a high entropy having large variation to reducing it down to a smaller class in which we are more certain about. In this manner, you can incrementally build any decision tree example.

Let us construct a decision tree using the ID3 Algorithm. What is more important in the decision tree is a strong understanding of Entropy. Entropy is nothing but the degree of uncertainty. It is given by:

(At times, it is also denoted by “E”)

If we apply it to the above example, it will go as follow:

Consider the case when we don’t have people split into any category. It is a worst-case scenario (high entropy) when both types of people have the same amount. The ratio here is 3:3.

Similarly, for people who do not drink, have 1:1 ratio and the entropy would be 1. Thus, it needs a further split due to uncertainty. For people who do not drink, the ratio is 2:0. Hence, the entropy is 0.

Now, we have computed the entropy for the different cases and hence, we can calculate the weighted average for the same. 

For the first branch, E=661=1 

For the Smoker class, E=260+ 460.811=0.54

For the smoker and drinker class, E=260+ 261+260=0.33

The diagram below will help you in quickly understanding the above calculations.

Finally, the information gain:

Class Entropy Information gain (E2-E1)
People 1 0.46
Smoker 0.54 0.21
Smoker+Drinker 0.33

Conclusion

We have successfully studied decision trees in-depth right from the theory to a practical decision tree example. We also constructed a decision tree using the ID3 algorithm. If you found this interesting, you might love to explore data science in detail.  

If you’re interested to learn more about decision trees, machine learning, check out IIIT-B & upGrad’s PG Diploma in Machine Learning & AI which is designed for working professionals and offers 450+ hours of rigorous training, 30+ case studies & assignments, IIIT-B Alumni status, 5+ practical hands-on capstone projects & job assistance with top firms.

Frequently Asked Questions (FAQs)

1. What are decision trees?

Decision trees are used to visually organize and organize decision making information. The trees are drawn such that the root is at the top and the leaves are at the bottom. The decision trees are read from the bottom up, moving from left to right. Each level of the tree is a base for further testing and the decisions at each level will narrow the scope until the question is answered. A decision tree breaks a problem or decision into multiple sub-decisions and follows the logical path to the root, which is the primary goal. Decision trees are used to analyze the business environment, to prioritize and to provide insight, in order to make decisions on what direction to take.

2. What are the issues in decision tree learning in machine learning?

Decision trees can be used as a basis for testing new strategies or to explain strategies to others. A decision tree explains what will happen under a given set of assumptions. They can also be used to evaluate the performance of a strategy that was used in the past. Decision trees are known to be too susceptible to errors because of all their branches. Decision trees are not always accurate because, sometimes, they don’t take into account all possible variables, and the person analyzing the decision tree might not be experienced in all the aspects of the particular situation.

3. What kind of data is best for Decision Trees?

Decision Trees help you find patterns in data using a flow chart like structure. The best type of data would be qualitative, categorical and numerical. Although Decision Trees work with all types of data, they work best with numerical data. They must be able to have values that are numbers or there should be a way to translate them into numbers. Decision Trees are heavily dependent on the type of data as well as the quantity. If the number of data points is more than 100, Decision Trees would be a good model.