Decision-trees: keywords

Discretization, iterated binary segmentation, misclassification ...

Decision-trees, cont'd

Purity, Entropy, Information Gain, root-to-leaf ...

A UCI dataset on Bank note authentication

Four numerical values from analysis of Wavelet transformation:

• variance, skewness, curtosis and

• entropy of image.

One (integer) classification value: class

Decision-tree Classification

• As an alternative to Entropy, we use Gini index for assessing purity of segments (a further example is here.)

Decision-tree Classification, II

• As an alternative to Entropy, we use Gini index for assessing purity of segments (a further example is here.)

Decision-tree Classification, III

• As an alternative to Entropy, we use Gini index for assessing purity of segments (a further example is here.)

Exercise

1. Dowload and inspect a Decision-tree baseline code: dtree-baseline.py

2. Lay out and code functions that segment the data according to the best Gini values available.

def get_split(dataset):
    b_index, b_value, b_score, b_groups = 999, 999, 999, None
    # TODO: Find the best possible place to split the dataset
    #
    # TODO: assign datapoints to 'left' and 'right' segments
    # using the test_split(index, value, dataset)
    # function.
    # TODO: define a gini_index(groups, classes)
    # func. to construct a branch of the tree
    return {'index':b_index, 'value':b_value, 'groups':b_groups}

Remember: Gini=0 is the best scenario

Exercise, cont'd

• Calculate the Gini index for a split dataset

def gini_index(groups, classes):
    total_gini = 0.0
    # TODO : For each group calculate the gini index. 
    return total_gini