2 min read
Cate Farrall - Jul 4, 2013

Different Shades of Big Data

The ethics of competitive intelligence monitoring

For anyone whose job includes collecting and storing information, it’s crucial to know where to draw the line between what is ethically acceptable and what is plain illegal.  Here’s how to conduct competitive intelligence with a clear conscience:


White information 

White information is public data. It is published, shared, and openly communicated to  and by anyone. The internet is full of this kind of information:

  • Corporate websites

  • International, national, and local press

  • Public social media: blogs, public Facebook pages, Twitter, Instagram, YouTube, etc.


Black Information

Black information is not legally available to the general public. This kind of information is usually well-hidden on the internet, most often encrypted and/or password protected:

  • Credit card details on Amazon or any other e-retailer

  • Facebook pages with privacy settings

  • Company intranets only accessible with an inhouse connection


The Shades of Grey

Apart from the obvious black and white information, there are various shades of grey, often referred to as the invisible web. This kind of information is available on the internet, but not easily accessible from a basic search engine, such as Google, Yahoo!, Bing, etc. Some examples are:

  • Newpapers and journals with a subscription, usually protected by a login and password

  • Information indexed in search engines embedded on other sites

  • Blogs and forums not listed by major search engines


What should I track?

This really depends on the objectives behind your CI project. Your objectives will help prioritise the type of sources you will need to follow.

If your aim is to monitor your online reputation, it’s best to track information that your consumers, staff, and other stakeholders can easily find, as this is what will most influence their opinion of you. You’ll need to track openly available white information, such as Google search suggestions, the Tweetstream, national and local press, etc.

On the other hand, if you want to find out about your competitors’ plans for launching new products or opening new stores, then you’ll need to include some grey matter. Consider looking at job postings and patent registrations. This kind of information is often contained on search engines embedded in other sites (LinkedIn for example) or on specialised search engines.

And the black information? Best to leave that to the likes of James Bond or Jack Bauer.


Written by Cate Farrall

Cate is our Training Manager at Digimind's Paris office.