How does Decision Tree Algorithm works in AI?

In this post we shall aim to explain the Working of Decision Algorithm from the Machine Learning perspective.

The problem statement here would be to Classify the Vegetables based on their features, the tool that ought to be used here should be Decision Tree.

The Data set is not organized and the Entropy is high for the given Data set.

Foremost, we ought to define the training Data set, which is illustrated below.

Here our focus is the Color of the Vegetable along with its weight, and based on that, its classification would be achieved.

It is important to note here that the, Conditions ought to be suitably designed so as to obtain the Maximum information gain. It is imperative to understand here that any gain will symbolize a decrease in Entropy.

The Entropy is calculated by the following formula

The Vegetables are placed in table form, as shown herein.

The Entropy is then Calculated using the following formula.

Now we need to choose a condition that would give us the maximum
gain. This should be used to make the first split.

The Condition that we would use here would be the Color = Red

Based on this condition, all Vegetables that are Red in color would fall to your Left, and that would include, Tomatoes and Chilies, While those vegetables which are not Red in color will fall to the Right, which would include, Yam, Pumpkin and Cucumber.

We will further classify the Vegetables, based on their weights, as shown in the figure below. Vegetables whose weights are lesser than 3 grams would fall to our Left (Chilies in this case), while those which are greater would be fall to our Right (Tomatoes in this case).

Similarly, Vegetables which are greater than 75 grams, would fall to our Right, that would be Yam and Pumpkin, while those which returns false, will be the ones on the left, that would be Cucumber.

The final Classification would have an acceptable Entropy.

In later Posts, we would follow the Concept of Decision Algorithm with varied other illustrations.