How Netflix's crackdown on password sharing could use analytics

[#1]

Recently, I came across an article on BBC World News titled “Netflix considers crackdown on password sharing” that sparked my interest and got me thinking. Apparently, some Netflix subscriptions are being shared by multiple households in violation of the terms of service.

This issue isn’t unique to Netflix; most streaming platforms allow for simultaneous viewings from multiple devices by “allowing users to create multiple profiles,” as the article describes and which anyone who’s used a streaming service in the last few years could confirm. What users aren’t supposed to do is let friends or neighbors who don’t live at the same address use their subscription.

I have to say that I’m a huge admirer of Netflix, as well as a customer. My account has three profiles: one used by me, one by my wife, and one shared amongst our three kids. It has always amazed me how well the shows recommended on my profile align with my personal tastes and how they differ from those displayed more prominently in the other profiles.

As far as analytics is concerned, Netflix is an extremely mature company, and I have no doubt it is employing many of the same techniques used in recommending shows to me as it is in identifying terms of service violations. However, having no insights into its inner workings beyond this 300-word BBC article and my own experiences using the platform, I can only imagine what those techniques might be. Here’s what I’ve come up with:

• One option that clearly won’t work is to confirm sign-ins using local IP addresses that are linked to the physical addresses of the accounts. In the days before the pandemic, I used to travel quite frequently and would often sign in to Netflix using my profile from that very same iPad from hotels in various cities or even countries. I’m sure there are others who have multiple residences from which they watch Netflix. I’ve seen people watching Netflix with their phones on public transportation. This is all allowed under the terms of service, and taking steps to prevent it would certainly aggravate many loyal customers.
• The current trial, which uses an email or text message to verify an account, may equally annoy some people. I can easily imagine scenarios where I am unreachable just at the moment my 3-year-old tries to watch Peppa Pig and is asked to verify her account through a text sent to my phone. The screaming that would ensue (not really, she’s a great and understanding kid, but stay with me on this one) would be placated by immediately switching to Disney+, and perhaps, never switching back.

There are situations that are more likely than not to be actual violations, such as consistent log-ins to the same account during prime time at several repeating geographic locations within the same city, or daily simultaneous use in different countries or continents. There may exist isolated cases of legitimate use even within these occurrences, but they are hard to imagine and would certainly be in the minority.

According to the BBC article referenced earlier that forms the entirety of my research into this topic, Netflix “now has more than 200 million subscribers around the world.” If it is going to make any meaningful attempt at tackling this issue, it will have to turn to analytics for help. A really great place to start may be one of several unsupervised learning algorithms available in JMP, such as hierarchical clustering.

Unsupervised learning allows for users with similar viewing habits to be classified into clusters without having to identify those habits in advance. In the example below, 15,100 internet users are grouped into five clusters using 51 usage metrics. As can be seen in Figure 1, not all clusters contain the same number of users. In fact, Clusters 2, 3 and 5 contain only 25 users between them. If this were my data set, I would take a closer look at these clusters to try to understand how they differ from the remaining 15,075 people.

Figure 1: Counts for five clusters for data set of internet usage habits of 15,100 (left) and dendrogram produced by the hierarchical clustering platform in JMP.

That information may be found in the parallel cord plot below in Figure 2, where each line represents a cluster. Looking at the leftmost extreme of the plot, it seems the three users in Cluster 5, on average, had a larger tot_HO (total hours online) than the users in any of the other clusters, on average. This may be one of the metrics that distinguishes this cluster from the others. There are others as well, as indicated in the portions of the plot where the lines deviate.

Figure 2: Parallel cord plot, showing how the usage habits differ for each of the five clusters.

The next step would be to consider these differences and think logically about which clusters are composed of users who are most likely violating their service agreements. The nice thing about JMP is that initially, if nothing concrete emerges, the platforms are designed to let analysts keep exploring the data until something meaningful pops out at them. Once that happens, they'll have a good idea of the accounts to target the warnings.

From there, a streaming platform like Netflix could monitor these accounts to find out whether there are differences in their usage habits. Doing so might indicate that some of the accounts had multiple households sharing passwords, and the warnings were heeded. A clear outcome would be to do this as a test in a localized region and see if there is a spike in subscriptions in the days or weeks that followed.  

All of this could be the foundation for a training data set and the development of a model to estimate the chance that a user is committing infractions. Likelihoods greater than 50% may suggest violations, but the company may choose to be more conservative in its approach and only consider users above, say, a 70% chance as potential violators. This can be done by adjusting the probability threshold to optimize the confusion matrix – these options and others are found throughout JMP. The outcomes of this model could even be binned according to probability, with different actions taken on these bins. For example, users with habits indicating a 99% chance of violations may have their accounts suspended, while those above 80% may be sent warnings.   

This is where a streaming platform that wants to stop password-sharing needs to make some decisions about what it’s hoping to achieve, where its priorities lie, and what it would absolutely like to avoid. Without careful consideration, it may use this process to inadvertently develop models that unfairly and incorrectly target minority populations. Or, bad guesses may lead to scores of legitimate users cancelling their service and switching to a competitor. This highlights the importance of including human oversight in every step of this process, including, and especially, having humans evaluate the models using logic and common sense. There are many examples of organizations learning the hard way what machines will do to their brand if left to their own devices.

Perhaps streaming companies have ways of monetizing viewership apart from the subscription fees, such as product placement, and would rather err on the side of more viewers rather than fewer. Of course, how many viewers gained versus subscription fees lost would be the type of information needed before making those decisions. JMP can be used at every step of this process. Our software enables domain experts to wade through large and complex data sets until insights emerge that only they, being experts, can understand. It further allows those experts to distill the insights and present them in a way that is easily digested by stakeholders, thus ensuring any resulting actions align with the strategic needs of the business.

It'll be interesting to see what actions the various streaming services take to address this issue moving forward. One thing is certain: They, like many other organizations, will have leveraged the power of analytics to inform and augment the decisions driving them.