Analysing large data sets – big data as it is known – will become a key basis of competition, underpinning new waves of productivity, growth and innovation according to research by McKinsey.
In fact, managing and manipulating big data has become such a critical component of a business’s ability to compete that a whole genre has emerged, that of data scientists.
Yet despite growing evidence to show how using, and analysing, big data can give an instant boost to business performance, many smaller-sized firms are still failing to latch onto this new and growing discipline. As Ben Rooney from the Wall Street Journal explains, mid-sized businesses need to identify and overcome the hurdles to implementing big data… or face being left behind.
Big Data is slowly moving into the enterprise, and where it has been used, has resulted in impressive performance improvement, he said. However, while there is increasing adoption of the technology, executives still see obstacles and organisations lack the skills and the tools to handle the data, it seems.
The Capgemini report, conducted by the Economist Intelligence Unit, said that on average where companies have applied Big Data techniques they are seeing a 26% performance improvement.
“This is quite substantial,” said Richard Brown, Global Program Leader of Business Information Management at Capgemini. “But you don’t see companies out-performing the market by 26%, so what does this mean?
“We rationalised this by saying organisations are tackling this business process, by business process. It is about looking at a particular area, so for example tackling fraud, or driving a marketing campaign.”
The report shows a growing appetite among organisations for data and data-driven decisions, despite their struggles with the enormous volumes being generated. Just over half of executives surveyed said management decisions based purely on intuition or experience are increasingly regarded as suspect, and two-thirds insist that management decisions are increasingly based on “hard analytic information”.
The biggest obstacles to the wide-ranging adoption of big data was the “silo-ing” of data within organisations, while lack of skills was also a key inhibitor – some 85% of respondents said it was the ability to handle the data and act on it in real time was a challenge.
The gap between the demand and supply of qualified data analysts was perceived to be the highest for retail and consumer goods companies. Two-thirds of respondents from these sectors cite access to talent as the toughest obstacle to data-driven decision making.
Furthermore most executives believe that big data management is not viewed strategically at their firm, and that they do not have enough of a “big data culture”.
Organisations struggle to make effective use of unstructured data for decision-making. Most business people are familiar with spreadsheets and relational databases, but less familiar with the tools used to query unstructured data, such as text analytics and sentiment analysis. Some 40% of respondents said they have too much unstructured data.
The Economist Intelligence Unit conducted a survey, completed in February 2012, of 607 executives. Participants hailed from across the globe, with 38% based in Europe, 28% in North America, 25% in Asia-Pacific and the remainder coming from Latin America and the Middle East and Africa.
The sample was senior, 43% of participants being C-level and board executives and the balance, other high-level managers such as vice-presidents, business unit heads and department heads. Respondents worked in a variety of different functions and hailed from over 20 industries. Of the latter, the best represented were financial services, professional services, technology, manufacturing, healthcare and pharmaceuticals, and consumers goods and retail.
What do you see as the biggest hurdle to enterprises adopting big data: The silo effect, lack of skills, not having a ‘big data culture’?
To discuss this and other articles please visit The Business Technology Forum group on LinkedIn.