What Is a Decision Tree? The Machine Learning Model You Already Understand

If you have ever decided whether to carry an umbrella by asking yourself "is it cloudy, and if yes, is rain forecast" then you have already built a decision tree in your head. That is the whole idea behind one of the most beginner friendly models in machine learning.

The 20 questions game

Think of a decision tree like the game 20 questions. You do not guess the answer straight away. You ask one yes or no question, the answer splits the possibilities into two smaller groups, then you ask the next question inside that smaller group, and so on, until you land on an answer.

A decision tree does exactly this with data instead of guesses. It looks at your data and asks questions like "is income above 50000" or "is age above 40." Each answer splits the data into smaller groups. Keep splitting, and eventually each small group is mostly one outcome, like "approve loan" or "reject loan." Draw all those questions and splits as a flowchart, and you get a tree shape, with a trunk at the top and branches leading to leaves at the bottom.

That is why it is called a decision tree. It genuinely looks like one when you draw it out, upside down, with the first question at the top and the final answers sitting at the tips of the branches.

Where this shows up at work

Banks use decision trees to decide whether to approve a loan. The tree might first ask about credit score, then income, then existing debt, narrowing down to approve or reject at the end. You can literally show this tree to a loan officer and they can follow the logic line by line, which is rare for a machine learning model.

Hospitals use decision trees to help triage patients. Retailers use them to predict whether a customer will return an item, asking about product category, price, and return history. In every case, the appeal is the same: someone without a data science background can look at the tree and understand exactly why it made a decision.

The honest catch: overfitting

Here is the nuance beginners usually miss. If you let a decision tree keep asking questions forever, it will keep splitting the data into smaller groups until each group has just one or two data points in it. At that point the tree has not learned a general pattern. It has just memorized the exact training data, including its noise and its flukes.

This is called overfitting, and it is the single most common mistake with decision trees. An overfit tree looks perfect on the data it was trained on and then falls apart the moment it sees a new customer it has not seen before, because it never learned the real pattern, just the specific examples.

The fix is to limit how deep the tree is allowed to grow, or require a minimum number of data points before it can split again. This is called pruning, and it is the difference between a tree that generalizes and a tree that just memorizes.

Decision tree vs random forest, in one line

Once you understand a single decision tree, the next term you will hit is random forest. A random forest is just many decision trees built on slightly different slices of the same data, all voting on the final answer together. One tree can be wrong in its own quirky way, but a few hundred trees voting together tend to cancel out each other's mistakes. You lose some of the readability of a single tree, but you gain accuracy. Most companies using decision trees in production actually run a random forest, and keep one single tree around mainly for explaining the logic to non technical people.

What this means for you

If you are new to machine learning, decision trees are a genuinely good place to start, not just an easy one. The logic maps to how people already think, the visuals are easy to build with a few lines of scikit-learn code, and the concept of overfitting you learn here will follow you into every other model you touch later, including random forests and neural networks. Learn this one well, and a lot of machine learning stops feeling like a black box.

Where to go from here

If you want to see how your current skills stack up before diving into models like this, Flexing Data's free AI Readiness Assessment is a good place to start. Or explore the free Labs to build and prune your first decision tree on real data.