Decision Tree Classification: Everything You Need to Know

Introduction

Many analogies could be driven from nature into our real lives; trees happen to be one of the most influential of them. Trees have made their impact on a considerable area of machine learning. They cover both the essential classification and regression. When analyzing any decision, a decision tree classifier could be employed to represent the process of decision making.

So, basically, a decision tree happens to be a part of supervised machine learning where the processing of data happens by splitting the data continuously, all the while keeping in mind a particular parameter.

What are decision trees made of?

The answer to the question is straightforward. Decision trees are made of three essential things, the analogy to each one of them could be drawn to a real-life tree. All three of them are listed below:

1. Nodes: This is the place where the testing for the value happens. In a node, the value of a specific attribute is passed, and it’s checked and tested against the values to make a decision.  
2. Edges/Branches: This portion corresponds to the outcome of any test result. Edges/Branches are also responsible for linking two different nodes or leaves.
3. Leaf Nodes: These are the nodes that are typically found at the terminals. Leaf nodes are responsible for predicting the outcome.

Get Machine Learning Certification from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

Decision tree classification

The decision trees can be broadly classified into two categories, namely, Classification trees and Regression trees.

1. Classification trees

Classification trees are those types of decision trees which are based on answering the “Yes” or “No” questions and using this information to come to a decision. So, a tree, which determines whether a person is fit or unfit by asking a bunch of related questions and using the answers to come to a viable solution, is a type of classification tree.

These types of trees are usually constructed by employing a process which is called binary recursive partitioning. The method of binary recursive partitioning involves splitting the data into separate modules or partitions, and then these partitions are further spliced into every branch of the decision tree classifier.

2. Regression Trees

Now, a regression type of decision tree is different from the classification type of decision tree in one aspect. The data that has been fed into the two trees are very different. The classification trees handle the data, which is discreet, while the regression decision trees handle the continuous data type. A good example of regression trees would be the house price or how long a patient will typically stay in the hospital.

Learn more: Linear Regression in Machine Learning

How are the decision trees created?

Decision trees are created by taking the set of data that the model has to be trained on (decision trees are a part of supervised machine learning). This training dataset is to be continuously spliced into smaller data subsets. This process is complemented by the creation of an association tree that incrementally gets created side by side in the process of breaking down the data. After the machine has finished learning, the creation of a decision tree based on the training dataset that has been provided concludes, and this tree is then returned to the user.

The central idea behind using a decision tree is to separate the data into two primary regions, the region with the dense population (cluster) or the area, which are empty (or sparse) regions.

Decision Tree classification works on an elementary principle of the divide. It conquers where any new example which has been fed into the tree, after going through a series of tests, would be organized and given a class label. The algorithm of divide and conquer is discussed in details below:

Divide and conquer

It is apparent that the decision tree classifier is based and built by making use of a heuristic known as recursive partitioning, also known as the divide and conquer algorithm. It breaks down the data into smaller sets and continues to do so. Until it has determined that the data within each subset is homogenous, or if the user has defined another stopping criterion, that would put a stop to this algorithm.

How does the decision tree classifier work?

1. The divide and conquer algorithm is used to create a decision tree classifier. By making the use of the algorithm we always begin at the root of the tree and we also split the dataset to reduce the uncertainty in the final decision.
2. It happens to be an iterative process. So, we repeat this process at every node. This process is repeated until the time we don’t have the nodes of the purity we desire.
3. Generally, to avoid overfitting we set a limit of purity to be achieved. This means the final result might not be 100% pure.

Basics of the divide and conquer algorithm:

1. First comes choosing or selecting a test for the root node. Then begins the process of creating branches. The branches are designed with keeping in mind each possible outcome of the trial that has been defined.
2. Next comes the splitting of the instances of data into smaller subsets. Each branch would have its own splice, which is connected to the node.
3. This process then has to be repeated for each branch by using just the instances which come to the branch in question.
4. This recursive process should be stopped if all the instances belong to the same class.

Advantages of using decision tree classification

1. It does not require a tremendous amount of money to construct.
2. It is a swift process of classification of records that are new or unknown.
3. It can be very easily interpreted, especially if the tree is small in size.
4. The accuracy of prediction using the decision tree classifier is comparable to other methods of prediction or classification.
5. It also has the capability to exclude the features which are unimportant. This process of eliminating irrelevant features is done automatically.

Read: How to create perfect decision tree?

Disadvantages of using the decision tree classifier

1. Overfitting the dataset is very easy in this case.
2. The boundary of the decision has a restriction. It can only be parallel to the axes, which contains the attributes.
3. Models based on decision trees often have biased-on splits that have a massive number of levels.
4. Any small changes made on the dataset can have a significant impact on the logic which governs the decision.
5. Lager trees are challenging to understand because sometimes they might feel very counter-intuitive.

Also read: Decision Trees in Machine Learning

Learn data science courses from the World’s top Universities. Earn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Conclusion

Decision trees come in handy while we are faced with problems that cannot be handled with linear solutions. From observations, it has been noted that tree-based models can easily map the non-linearity of the inputs and effectively eliminate the problem at hand. Sophisticated methods like random forest generation and gradient boosting are all based on the decision tree classifier itself.

Decision trees are a potent tool which can be used in many areas of real life such as, Biomedical Engineering, astronomy, system control, medicines, physics, etc. This effectively makes decision tree classification a critical and indispensable tool of machine learning.