Different Shades of Big Data
The ethics of competitive intelligence monitoring
For anyone whose job includes collecting and storing information, it’s crucial to know where to draw the line between what is ethically acceptable and what is plain illegal. Here’s how to conduct competitive intelligence with a clear conscience:
White information is public data. It is published, shared, and openly communicated to and by anyone. The internet is full of this kind of information:
International, national, and local press
Public social media: blogs, public Facebook pages, Twitter, Instagram, YouTube, etc.
Black information is not legally available to the general public. This kind of information is usually well-hidden on the internet, most often encrypted and/or password protected:
Credit card details on Amazon or any other e-retailer
Facebook pages with privacy settings
Company intranets only accessible with an inhouse connection
The Shades of Grey
Apart from the obvious black and white information, there are various shades of grey, often referred to as the invisible web. This kind of information is available on the internet, but not easily accessible from a basic search engine, such as Google, Yahoo!, Bing, etc. Some examples are:
Newpapers and journals with a subscription, usually protected by a login and password
Information indexed in search engines embedded on other sites
Blogs and forums not listed by major search engines
What should I track?
This really depends on the objectives behind your CI project. Your objectives will help prioritise the type of sources you will need to follow.
If your aim is to monitor your online reputation, it’s best to track information that your consumers, staff, and other stakeholders can easily find, as this is what will most influence their opinion of you. You’ll need to track openly available white information, such as Google search suggestions, the Tweetstream, national and local press, etc.
On the other hand, if you want to find out about your competitors’ plans for launching new products or opening new stores, then you’ll need to include some grey matter. Consider looking at job postings and patent registrations. This kind of information is often contained on search engines embedded in other sites (LinkedIn for example) or on specialised search engines.
And the black information? Best to leave that to the likes of James Bond or Jack Bauer.
Written by Cate Farrall
Cate is our Training Manager at Digimind's Paris office.