How machine learning predicted traitors on Game of Thrones

So how can you predict when they are going to defect (or buy out)? Our first challenge was to transform the data we needed for training - a list of various behaviors of people with different personality traits - into personality trait data that could be processed by the model. Then, we used these activity data that could represent their personal personality traits to make predictions about them.

From another perspective, we might only focus on recent activities. This does help us keep our data up to date and address the negative impact of old data on our evaluation. But what if a character hasn't done anything recently? We'll still evaluate him based on his past actions. And we'll still keep some of his recent historical activity data, because some seemingly one-off activities in the past can turn into a significant pattern and affect future decision making.

We can benefit from a hybrid approach. For example, we can combine the past activities of the person in the model with the recent activities. In addition, we can use a series of different interface windows to treat recent activities differently. This way we can remember the actions that happened three weeks ago, but we can use different standards to measure the actions that happened yesterday.

You can see that in August, based on his recent behavior, our model thought he would defect (buy the product) during this period. Although this was expected, he remained loyal for several months. Of course, his defection did happen subconsciously. Since people's emotional state (whether they are ready to defect) changes over time, our model also needs to know whether someone is about to defect, so that we can know the exact time to strike back.

Another problem will arise when the scores stay at a peak before returning to a level. The spread of misleading predictions may make us temporarily skeptical of some extremely loyal characters, so we must ensure that the model's evaluation function is based on all scores over the entire period. When we retrain the model, we must penalize these incorrect scores and compare them to see which model is relatively better.

To evaluate whether a model is performing well, we simply need to consider the score we get each time we evaluate a character (daily or weekly), and observe how well the model predicts their behavior the following week. If at the beginning of a week, we assert that a character is very likely to defect, and they defect on Thursday of that week, then our model is working well and is giving positive and correct recommendations. However, if this character still does not defect in the predicted week, but does not defect until Thursday of the following week, then our model's recommendation is wrong. In this case, we have to consider how to score this character the following week.

If you follow all the tips above, your behavioral scoring model will bring valuable machine learning to your business needs. Knowing what people do and when is the key to predictive success.