Personal Information: How much is Too Much

March 26, 2010

News Round Up for the Week Ending 26th March 2010

There has been a lot of interesting news in the world of analytics this week.  Lets see what some of that means for us as analytics professionals.

Have you ever been a little taken aback by the recommendations for you at Amazon or the advertisements that pop up on your mail server? Write to your friend in Dubai about whether she is happy in her job, and on the right- hand side of the page you will notice a little link for “UAE Recruitment Consultant”. The computer, tagging words like “jobs” and “Dubai” together, is attempting to out-think you!

In the last couple of years, the analytics industry has been struggling with the complicated questions of how much information is too much information.  When does the concern for privacy outweigh any potentially useful application of additional personal details? As time passes, and we leave behind a bigger, more indelible track in our online lives between our multiple mails, social media interactions and browser history, it is time to take a step back and re-think some of these considerations.

In the last fifteen days two companies have done just that.

  1. Netflix – a popular US based video mail services runs an annual competition among leading statisticians to improve its recommendation services. Put simply, what this means is that if someone has rented Dilwale Dulhania Le Jayenge three times, is he more likely a Shahrukh Khan fan or a fan of romantic movies or just someone who was born in the 1980s. The better you understand this, the more likely are you to recommend the next best selection to him. And hundreds of amateur and professional statisticians worked on existing Netflix user data to improve these algorithms. On the 12th of March this year, Netflix decided to stop sharing this data for further analysis amidst fears that it was illegal to share private customer information such as video preferences
  2. GoogleOpt Out For long people have worried that between our search histories and emails Google knows everything about us. Facing increasing censure about how it could potentially use this information, Google has recently announced that it will soon announce an opt-out feature, clicking on which will ensure that our data will not be used by Google Analytics. Radical change or lip service? Only time will tell.

As analysts, we can do our bit to share that we don’t violate any of the written (or unwritten) codes of data privacy. Here is how:

  1. Analyze- Don’t Speculate: Remember the time you looked at those credit card transactions and wondered how much that person really earned?  The first rule of ethical data management is to analyze and not speculate, look for patterns and not individual cases and outliers, and use only the information that is actively required for analysis
  2. Don’t share information, don’t share anecdotes: While most analysts respect data privacy and don’t share it with any body else, just as unethical is the cocktail hour anecdote about the information at your disposal. Respect your unnamed client’s privacy and avoid talking about it
  3. Question Yourself. Is My Analysis Really Helping the Customer? When in doubt, stop. And start again. Any analysis that leads to meaningless non actionable insight is nothing more than an intrusion on privacy

Data mining and analysis are extremely powerful tools. But as our venerable friend Mr. Spiderman once said, “With Great Power Comes Great Responsibility” Be responsible with the data you have, and don’t ever spend a sleepless night again!


One Response to “Personal Information: How much is Too Much”

  1. Krishna Prasad Says:

    There is a mild difference between how much is required and how much is too much!!! Because companies across the globe have always been trying to gain deeper understanding about their customers, but did they ever thought about how much is sufficient to get “Best Insights” about their customers, products, their competitors…..! And intense cut throat competition dynamically varies the difference between how much is required and how much is too much! As rightly said “With Great Power Comes Great Responsibility” but “With great information come great insights and Great Profits” so what is too much information? As every criminal leaves a clue every customer in the world leaves a clue to gain insights into his taste and preference and etc….! so every clue lead to new information some time personal information and some time non personal….but a clue is a clue that always needs to be explored!!!

Comments are closed.

%d bloggers like this: