Or is Big Data simply one of two “camps” that exist within the business intelligence community as Eric Lebson and Ian Christopher McCaleb point out in their Holmes Report piece “Speaking Intelligently On Business Intelligence”. They rightly look at more traditional business intelligence being modeled more after law enforcement or journalistic intelligence gathering and show that digging in to big data is a much different type of effort.
They see some value in what they describe as input that is more based on human intelligence where it is ultimately data gather from human interaction. This is juxtaposed with the another government intelligence gathering effort, the NSA, where huge metadata sets are crunch looking for patterns that can lead to conclusions.
In their piece they write: “One distinct camp, the “big data” camp, focuses on the analysis of large quantities of data to develop analytic products. Huge reams of data are swept up by “bots” and crunched by any variety of software applications to provide products such as bench-marking, targeted reporting, data and text mining, performance measures, prescriptive or predictive analysis, statistical modeling, operational assessments and the like. Think, for instance, of a major beverage company researching the advertising purchasing patterns of a competitor to try to divine where that rival is focusing its growth strategy.
Public opinion polling too factors into this camp’s work as a means of making sense of the attitudes of large or specifically identified demographics.
When polls and data analytics are cross-referenced, a company or client may gain a greater context for understanding and operating within their market. When the term ‘intelligence’ is applied to this approach, it is often in terms of ‘market intelligence’ and the ‘intelligence’ in question is akin to what the NSA does when they look for patterns and linkages among huge volumes of communications meta-data.”
As this “camp” grows in influence within organizations some fear that without careful oversight even marketers can overstep in their quest for an audience “secret”. The U.S. government is concern enough to have convened a MIT workshop to study the rapidly emerging impact of big data efforts. White House adviser John Podesta, head of the presidential study on the future of privacy describes it this way, “We’re undergoing a revolution in the way that information about our purchases, our conversations, our social networks, our movements and even our physical identities are collected, stored, analyzed and used.” He goes on to say, “On Facebook there are some 350 million photos uploaded and shared every day,” he said. “On YouTube 100 hours of video is uploaded every minute, and we’re only in the very nascent stage of the Internet of things, where our appliances will communicate with each other and sensors will be nearly ubiquitous.”
Regardless of concerns this is a train that has already left the station and the data is only getting more abundant and more critical to the future of most businesses than ever. Healthcare stands to gain as the information silos there get broken down and by crunching all the structured and unstructured data there could lead to better patient outcomes. None of us can go back as our own “data footprints” are deep, wide and everywhere.
If you are conducting business intelligence and you are not considering your needs as far as utilizing what’s buried in that gigantic pile of data…you may be left behind. If you are not incorporating business intelligence into your day to day organizational guidance at all, you are flat out doomed. This is why many companies are turning to outside experts in these fields to create short and long term strategies for warehousing and then mining that data pile.
Many IT departments are stressed as the technology needed to do the work is either new to them or they simply need a more objective look at their business intelligence systems. Some already are stretched thin and taking on a very IT intensive effort to bring big data insights to analysts and decision makers may be more than they have manpower to accomplish. Both can easily outsource aspects of the planning, implementation and support to qualified firms and may find even cost savings overall by doing so.
Either way “Big Data” still remains a subset of business intelligence and not the total picture, yet. Given more time it is not unlikely that big data will join artificial intelligence and the Internet of Things as being a major part of our collective business efforts in the near future.
Author’s Note: This is an article written for a client in the IT and Application Development industry.
