by Scott Yara of Greenplum, a division of EMC
Pioneering organizations are now working with Big Data, data that is either too voluminous or too unstructured to be managed using traditional database technologies and are employing analytical tools to analyze and make decisions on it. Because of this, the role of a “data scientist” has emerged to describe individuals who have sufficient business acumen, analytical, and computer science skills to work effectively with Big Data.
The largest-ever global survey of data scientists reveals that two-thirds of these professionals lack confidence in their company’s ability to make business decisions based on new data. This means that in a highly competitive economy, when almost every business needs a differentiating edge, the people who touch, use, analyze and scrutinize data say their companies are sitting on gold mines of new information that are going untapped or underutilized.
The challenge, however, lies in the fact that there are gaps between the volume and formats of new data that organizations have at their fingertips and the new tools and skills and business processes required to derive actionable insights from it. Data is growing faster than Moore’s Law, and with the explosion of data generated by social media, video, mobile sensors and the like, organizations of all sizes and from all industries are having a hard time keeping up with it. Until recently, data analysis was focused almost exclusively on structured data. Now, it’s about analyzing the unstructured data streams rushing into organizations from all angles and in all kinds of formats.
Because, organizations are under pressure to increase the speed and effectiveness of their decision-making, increasingly, expanded analytic capabilities are the means to achieve this goal. That requires more than just a Big Data analytics platform, however. Organizations need to update the skills of their teams, foster an environment that enables analytic productivity, and develop a mindset towards data that values it as the new “fuel” for organizational productivity.
Ironically, in the era of Big Data, human skills such as creativity, communications and leadership are becoming more valuable than ever.
The recent study sponsored by EMC Corp. in cooperation with research partner Toluna describes data science as an emerging field that “applies advanced analytical tools and algorithms to generate predictive insights and new product innovations that are a direct result of the data.” Unlike traditional business intelligence (BI), which analyzes structured data to understand what just happened, Big Data analytics looks at richer and more varied volumes of data across an organization to discern patterns, identify anomalies and help decision makers be more proactive and predictive.
According to research firm Gartner, “Through 2015, more than 85 percent of Fortune 500 organizations will fail to effectively exploit Big Data for competitive advantage.”
This is a missed opportunity of staggering proportions that should make business leaders take notice.
Leveraging Big Data for business advantage requires more than just thinking through technology issues. Much of the Big Data productivity imperative falls upon IT leaders, who must ensure adequate levels of bandwidth, storage and processing power. But an even greater challenge lies with C-level executives, who must address the cultural and organizational issues associated with Big Data as they depend upon it for strategic decision-making and new value creation. All of this calls for an integrated and holistic approach to the opportunity that Big Data presents and this is just starting to register at the C-level.
Just as computer science emerged as a new category 30 years ago and BI 10 or so years after that, now is the time for data science as a discipline to stand on its own because all sectors of the business from marketing to finance need help extracting value from massive amounts of data.
The call to action is not just for companies to make room for this new breed of knowledge worker but to provide them with an environment that encourages data exploration. Data scientists need the funding and entrepreneurial freedom to run experiments on data so that they can tap into their own intellectual interests and discover potential business opportunities that others haven’t seen. This is the challenge for all businesses that want to make effective and competitive use of their gold mines of information.
Scott Yara, co-founder and senior vice president of Products for Greenplum, a division of EMC and a Big Data appliance and platform vendor, is responsible for Greenplum's overall product development and go-to-market efforts, including engineering, product management, and marketing.