ID3

ID3 is a simple, intuitive method to generate a decision trees from a given data-set.

The algorithm automatically selects attributes with higher discriminative power to be put higher up into the tree, such that smaller trees are learnt. (based on for instance information gain)
As the method will continue to add branching nodes to the tree until all instances in the dataset are correctly classified (if possible), the method tends to overfit.

• The method is simple to use.
  
• It is not guaranteed that the smallest decision tree is learnt:
• As the method is a greedy method, it can be stuck in local optima.
• ID3 only works on categorical data.
  
• As the generated trees are available to the users and clear to interpret, such method provides high level of explainability.
  
• In order to reduce overfitting, trees can be pruned into smaller trees, which tend to better generalize.
  

C45

C45 is a method to generate a decision tree from a given dataset. It is an extension of the earlier proposed ID3 method. Compared to ID3:

• it also works on continuous data;
• it provides pruning of the generated tree to reduce the risk of overfitting;
  
• it can work with missing values.
  

Just like ID3, this method is easy to use and the generated decision trees are available to the users, can be interpreted
and it can be motivated why the algorithm ordered the nodes in the tree in the given way. Hence, this method provides a high level of explainability.

Return to the Introduction