Skip to content

Random base selection

You can validate your model in a few simple steps:

  • Set up a random base selection as above, but specifying a percentage, say 60%. Call this the Random Training Base. Build the Decision Tree as usual.

  • Drag off your top nodes to create a selection

You should see the selection rules contain the Random Training Base ANDed together with the nodes you selected.

Right click on and delete the Random Training Base part of the selection, so that you are just left with the individual nodes (see below).

  • Overlay a cube on to this selection. The cube, being based on the selection of the top nodes, represents people we have predicted will be Swedish Holiday makers. We will be able to see how many of these people actually are.
  • Drag the analysis selection which you used to build the Decision Tree (e.g. Swedish holiday makers) to be one of the dimensions of the cube, and drag the Random Training Base selection you used to create the Decision Tree to be the other dimension.

  • Now run the cube and view the Row % statistics in the “Yes” column for being in the Swedish Holiday maker selection.

    In the row which is Yes for the Random Training Base, you get the same Analysis count and percentage as you got when you built the cube (see below). In the example below, the Analysis % is 6.90%. This is the proportion of Swedish holiday makers in the top nodes within the data used to create the Decision Tree model.

    The row which is No represents a holdout sample – people not used in the creation of the model. The proportion of Swedish Holiday makers in this sample is 6.98%.

Interpret the results

  • These two figures should be very close, if the model works against new data. The fact that the percentage is higher for the holdout sample is just due to random variation. The important thing is that they are virtually identical.