Sequence analysis wizard

The Sequence Analysis wizard provides a mechanism for creating on the fly, pattern match aggregation expressions that allow you to look for particular patterns of transactions, or for longest sequences of transactions with a particular property - for example, the same holiday destination, repeat purchases of a particular product, changes in donation type, etc.

The wizard simplifies and guides you through the process of creating the aggregation, offering all the same options that are available directly through the expression editor, with the result of the wizard being an on the fly aggregation expression which you can then use in a selection, or as a column in a data grid, for example.

To get started, from the Analysis Wizards section of the Wizards ribbon bar, select Sequence Analysis.

For an overview of each of the wizard steps, start with step 1 - Transactional Selection.

There are three possible pattern types to select from - click the links for a worked Holidays example of each:

1. Selector variable only - when you create a sequence only from the pattern within your chosen selector variable.
2. Numeric variable only - when you create a sequence only from the pattern within your chosen numeric/currency variable.
3. Selector and numeric variables - when you create a sequence from the pattern within your chosen selector and numeric/currency variables together.

Transactional selection¶

In the first step of the wizard you should identify the transactional records to be used. You can right click and select, or drag and drop the required selection or table.

Once selected, click Next.

Grouping table¶

At the second step, the possible grouping tables are displayed in the drop-down menu.

Transaction grouping¶

This is an optional step that works only when looking for patterns within a selector variable. The grouping variable must also be a selector.

When you specify a grouping variable, transactions are split into these category groups before the pattern match is then applied to each of the groups individually. For example, you might want to look for transactional patterns for a particular department within the business, or perhaps look at items with a specific response code.

Simply drag your grouping selector variable onto the drop-box.

Transaction ordering¶

You need to give FastStats a variable with which it can order the transactions that are being pattern matched.

For date and datetime variables you can choose whether the ordering should be earliest to latest, or vice versa, and optionally define the maximum number of days between transactions, if necessary. For example, in the context of holidays, you might set this to 400 days to investigate patterns relating approximately to a person's annual holiday bookings.

For numeric/currency variables, you can order the records from smallest to largest, or vice versa, and optionally set a maximum difference between the transaction values to be returned.

Pattern type¶

There are three possible pattern types to select from - click the links below for a worked Holidays example of each:

1. Selector variable only - when you create a sequence only from the pattern within your chosen selector variable.
2. Numeric variable only - when you create a sequence only from the pattern within your chosen numeric/currency variable.
3. Selector and numeric variables - when you create a sequence from the pattern within your chosen selector and numeric/currency variables together.

Selecting your pattern type will take you to the next relevant step(s) - Pattern Definition or Value Definition.

Pattern definition¶

In this step, drag and drop your selector variable and choose between a predefined Longest sequence or manually define the required sequence.

When selecting a 'longest' pattern, you can optionally create a list of specific categories to include or exclude. In the Pattern for Destination dialogue, double-click or left-click and drag categories into the Pattern Code List column.

To manually define the pattern:

1. Select Manual.
2. Click on the Set pattern button to open the Define pattern dialogue.
3. Click on the Value column drop-downs to select the variable categories you wish to include in the pattern.

The following positional wildcards are also supported:

See Aggregations On The Fly - Pattern Match Positional Wildcards for more information and examples.

In the case of the above defined settings:

• Value 1 should be a booking to Australia.
• Value 2 can be a booking to any destination.
• Value 3 should be a booking to Greece.

To add an include or exclude list for a manual sequence, use the icons at the bottom of the Define pattern for dialogue:

Once created, grey highlighting indicates that a list is in effect:

Click OK to apply the pattern definition and then Next.

Value definition¶

In this step of the wizard you drag and drop your selected pattern numeric variable and then define the pattern.

You can select from the predefined 'longest' definitions or choose to set your pattern manually.

When manually defining a pattern, the pattern elements you can use to create sequences on a numeric variable are as follows.

Using booking Cost as the value variable in these examples:

• Fixed numeric values - F200 to find transactions which are exactly £200 in value.
• Fixed numeric ranges - F>100, F<1000, F500-800 to find bookings which cost more than £100, less than £1000 and between £500-800 respectively.
• Relative numeric ranges - R10-30 to compare transactions and look for a transaction that is between £10-30 more than the previous transaction.
• Percentage difference numeric ranges - P10, P10-20 to calculate that the next value in the list is 10%, or 10-20% bigger than the previous transaction.
• Ongoing sum values - >600, S>1200, S>1800 to find a transaction which cost more than £600, followed by two further transactions which return an ongoing sum value of £1200 and £1800 respectively.
• Ongoing mean values - >300, M>300, M>300 to find a transaction which cost more than £300, followed by two further transactions, each returning an ongoing mean value of £300.

It is possible to use the above in conjunction with one another to create a pattern, when appropriate:

In the case of the above defined settings:

• Value 1 cost should be between £500-1000.
• Value 2 cost should be between £50-100 more than value 1.
• Value 3 cost should be more than 10% higher than value 2.

A number of positional wildcards are supported.

Sequences using a selector and numeric together must be consistent with each other and you should ensure that:

1. There are the same number of patterns in each of the selector and numeric pattern definitions.
2. The patterns have the same name.
3. The patterns have the same number of elements.
4. Any * elements in the pattern are in the same position in both the selector and numeric patterns.

Pattern return value¶

In this step you choose the pattern and the property from that pattern that you are interested in. The options presented in the Return the drop-down reflect the settings you define earlier in the wizard.

In the example below, the following variables have been used:

• Booking Date for the transaction ordering
• Destination and Longest Any for the pattern definition
• Cost and Longest Increasing for the value definition

Based on these, and the values being returned on the grouping table, the results of each return option are displayed in the following data grid, with the table below offering a key for the highlighted person, as follows:

Once you have defined your return information, click Next.

Name¶

In this step of the wizard you should add an appropriate name for the expression that will be generated. This may be based on organisational practice and could include, for example, your own name or initials to indicate who the creator is.

If you don't enter your own description, a name based on your pattern match settings is auto-generated.

Confirm¶

In this final step, you have the opportunity to review your pattern match settings.

You can go back and edit the settings if necessary or click Finish to create the pattern match aggregation expression.

Click on the tab at the bottom of the display to view the pattern match settings:

Numeric variable only¶

Scenario¶

Find the longest sequence of holidays a person has made where the cost is decreasing. find the start and end numeric values for each sequence.

To get started, launch the Sequence Analysis wizard, then:

1. Transactional Selection - right click and select the Bookings table as the transactional records to analyse.

   

2. Grouping Table - set People as the table to group the bookings up to.

   

3. Transaction Grouping - this is an optional step that allows additional grouping into the categories of an identified selector variable. No grouping variable is required here - simply click Next.

   

4. Transaction Ordering - in this example:

   1. Drag and drop the Booking Date variable onto the drop-box.
   2. Leave the default order as Earliest to Latest.

   No maximum gap between transactions is necessary here - click Next.

   

5. In the Pattern Type step, select Numeric Variable Only.

   

   As this is numeric only pattern matching, you move directly to the Value Definition step.

6. Drag and drop the Cost variable onto the drop-box, then set the pattern definition type as Longest Decreasing.

   

7. Use the Return the drop-down options to define the information you want to return. Here:

   1. Return the Pattern Length
   2. Return as a property on the Grouping Table

8. Give your pattern match aggregation an appropriate description.

   

9. You have the opportunity to review and, if necessary, edit your settings before creating the pattern match expression.

   

10. When you click Finish, the pattern match aggregation expression is generated.

    

For this scenario, to create the two additional pattern match aggregation expressions, you can either use the wizard two more times to follow the same steps but define the return information as Start Numeric Value and End Numeric Value:

Or you can make a copy of this first expression and then:

1. Click into the aggregation tab at the bottom of the editor window.
2. Edit the Return the setting.
3. Rename the aggregation.

4. Rename the expression.

   

The expressions can be added as columns on a data grid to display the results for each person.

Selector and numeric variable¶

Scenario¶

Find people who have been to Australia, then the United States, and then Greece, where the cost of the Australian booking is greater than £1000, with a more than 10% increase in the cost of the US booking and then the Greece booking.

To get started, launch the Sequence Analysis wizard, then:

1. Transactional Selection - right click and select the Bookings table as the transactional records to analyse.

   

2. Grouping Table - set People as the table to group the bookings up to.

   

3. Transaction Grouping - this is an optional step that allows additional grouping into the categories of an identified selector variable. No grouping variable is required here - simply click Next.

   

4. Transaction Ordering - in this example:

   1. Drag and drop the Booking Date variable onto the drop-box.
   2. Leave the default order as Earliest to Latest.

   No maximum gap between transactions is necessary here - click Next.

   

5. In the Pattern Type step, select Selector and Numeric Variables.

   

6. In the Pattern Definition step:

   1. Drag and drop the Destination variable as the selector variable to use.
   2. Select Manual from the Pattern definition type drop-down options.

   3. Click Set pattern and define the categories you wish to pattern match - here:

      

   4. Click OK and move onto the next step.

7. In the Value Definition step:

   1. Drag and drop the Cost variable as the numeric variable to use.
   2. Select Manual from the Pattern definition type drop-down options.

   3. Click Set pattern and define the numeric values you wish to pattern match - here:

      

      Note

      For more on the pattern elements you can include when creating sequences on selector and numeric variables see Pattern Definition and Value Definition.

   4. Give the pattern a description - here Aus-USA-Greece then OK and Next.

      Note

      The pattern name of the numeric value must exactly match that used for the selector.

8. In the Pattern Return Value step, leave the default settings to select the First - By Start Date and return the Pattern Name on the Grouping Table.

   

9. Give your expression a meaningful description.

   

10. In the final step of the wizard, you can review the defined settings. If necessary, navigate back through the steps and edit, or click Finish to create the on the fly aggregation expression.

    

You can now use the pattern match expression to continue your analysis. For example:

1. Drag and drop the expression onto a new People level selection and enter the pattern Aus-USA-Greece.

   

   Building the selection returns a count of 7 people who currently match the defined pattern

2. Drop a data grid onto the selection and add Booking Date, Destination and Cost.

3. Group by Person URN and build.

4. Sort into ascending order by booking date.

   

You can see that each person satisfies the pattern by:

• Having a sequence of bookings that is Australia followed by the United States and then Greece;
• The cost of the Australian booking being more than £1000;
• The cost of the US booking being more than 10% higher than the Australian booking;
• The cost of the Greece booking being more than 10% higher than the US booking,

Selector variable only¶

Scenario¶

Find the longest sequence of holidays a person has made to the same destination.

To get started, launch the Sequence Analysis wizard, then:

1. Transactional Selection - right click and select the Bookings table as the transactional records to analyse.

   

2. Grouping Table - set People as the table to group the bookings up to.

   

3. Transaction Grouping - this is an optional step that allows additional grouping into the categories of an identified selector variable. No grouping variable is required here - simply click Next.

   

4. Transaction Ordering - in this example:

   1. Drag and drop the Booking Date variable onto the drop-box.
   2. Leave the default order as Earliest to Latest.

   No maximum gap between transactions is necessary here - click Next.

   

5. Leave the default - Selector Variable Only - as the pattern type to use.

   

6. First:

   1. Drag and drop Destination as the pattern variable to use.

      You can now use the Pattern definition type drop-down to select either a 'longest' sequence option, or define your own pattern by selecting Manual.

   2. Select Longest Same.

      It is possible to specify categories to include or exclude, but this is not necessary here.

      

   As this is selector only pattern matching, you move directly to the Pattern Return Value step.

7. Use the Return the drop-down options to define the information you want to return. Here:

   1. Select the First
   2. Return the Pattern Length

   3. Return as a property on the Grouping Table

      

8. Give your pattern match aggregation an appropriate description.

   

9. You have the opportunity to review and, if necessary, edit your settings before creating the pattern match expression.

   

Once you click Finish, a FastStats expression window opens. Switch to the on the fly aggregation tab to see the settings defined using the wizard.

To visualise the results, drag and drop the expression onto a data grid: