Pentaho Survey Finds Technology, Data Complexity Hinder Big Data Adoption

With 50 percent of respondents in production or planning big data projects, survey identifies emerging applications and use cases in response to business needs

February 21, 2013, — Delivering the future of business analytics, Pentaho Corporation today announced results of a survey that found nearly 50 percent of respondents are either in production with a big data project or planning to deploy one in the next 18 months.  The poll also revealed three key trends with big data deployments, according to Pentaho:

• Big data talent crisis is exacerbated by overly complex Hadoop technologies combined with the expectations of data analysts for access to and self-service for big data sources.
• Data diversity prevails and new types and volumes of data present considerable data management problems.
• Revenue and customer big data applications demonstrate immense value to companies on the front lines with big data who are solving real problems and uncovering new business opportunities. 

Detailed findings from the survey include:  

Big Data Talent Crisis –The primary consumers of big data are either business analysts (55%) or data analysts (42%) with some serving both roles, which demands an understanding of analytics and a deeper technical acumen. Only 20 percent of respondents identify themselves as data scientists. The survey also pinpointed the top three challenges in deploying big data as: resource availability (45%), skills (43%), and complexity of big data (38%). Clearly, respondents are struggling with programming-intensive data analytics technologies and complex data sources. Continued improvement in the simplification, automation, visualization, interactivity and self-service functionality of big data solutions are needed.

Data Diversity Rules – With multiple data sources the norm, over 50% of survey respondents cited both managing data and manipulating data as big bottlenecks.  Furthermore, more than 60 % of respondents report that they are managing three or more data sources (32% with five or more).  Almost half (47%) say their organizations are using standard relational databases as well as Hadoop or NoSQL.

With the growing prevalence of data environments that include legacy relational databases and big data sources, organization must be able to quickly and cost-effectively reconcile existing database and data warehousing infrastructure with their big data stores,” said Richard Daley, founder and chief strategy officer at Pentaho. “An effective, holistic big data analytics platform such as Pentaho’s includes not only analytics, but also data integration, data quality and data manipulation capabilities.” 

First Big Data Winners: Revenue & Customer Applications – Emerging big data applications and use cases can reveal highly vital insights. According to the survey, customer transaction analysis (43%) is the most popular application in use today, followed by application log analysis (37%) and e-commerce transactions analysis (20%).

Among the early market adopters, half are visualizing their big data via reports/dashboards and interactive analysis. Finally, regardless of application type, the majority of the respondents said they are deploying on-premise, with only one-third using the cloud. 

Conducted by Pentaho, the survey polled approximately 270 respondents on big data-related questions ranging from deployment stage, infrastructure and software type, to data sources, target users, problem areas and early application uses. More than half of the survey participants were developers, ETL developers, or BI developers; while around one third were business analysts.

Pentaho Business Analytics for big data enables companies to realize business value from large volumes of diverse data by dramatically reducing the time and complexity required to design, develop and deploy a big data analytics solution.  Pentaho’s solution covers the entire big data lifecycle, from data extraction and preparation to data discovery and predictive analytics.

• Download Pentaho Business Analytics for big data and get started today with the instant and interactive capabilities.
• Visit Pentaho’s booth at the O'Reilly Strata Conference, Feb. 26-28, 2013 in Santa Clara, CA.
• Hear David Henry, senior vice president of enterprise solutions at Pentaho, and Benjamin Lloyd, senior software development manager at NetApp, co-present “Dotting the I’s with Hadoop on Eseries” at 2:20 p.m. on Feb. 28, 2013, at Strata.
• Join the conversation – follow @Pentaho, join our LinkedIn and Facebook groups.

About Pentaho, a Hitachi Group company
Pentaho, a Hitachi Group company, is a leading data integration and business analytics company with an enterprise-class, open source-based platform for diverse big data deployments. Pentaho’s unified data integration and analytics platform is comprehensive, completely embeddable and delivers governed data to power any analytics in any environment. Pentaho’s mission is to help organizations across multiple industries harness the value from all their data, including big data and IoT, enabling them to find new revenue streams, operate more efficiently, deliver outstanding service and minimize risk. Pentaho has over 15,000 product deployments and 1,500 commercial customers today including ABN-AMRO Clearing, BT, EMC, NASDAQ and Sears Holdings Corporation. For more information visit www.pentaho.com.