A 5-Minute Guide to Big Data
- By Jonathan Buckley
- September 24, 2015
Big data: Ask 5 average people what it is and you’re bound to get several different answers—and at least one glazed over expression. That’s not surprising. When the term was first being tossed around in the analytics field several years ago, a lengthy debate began about what “big data” was all about.
Today that debate is over. Big data and analytics are transforming organizations, industries, and to a big extent, our lives. For those who are unclear about how this game-changing technology can benefit their organizations, here’s a 5-minute guide to big data.
What Big Data Is
Not long ago, data was neat and structured. It could be gathered, organized into tables containing tidy columns and rows, and analyzed by traditional means. Then came the Digital Revolution and new technologies such as the Internet, smartphones, and web-enabled devices that ushered in the Information Age.
As a result, the world is awash in massive volumes of data, aka “big data”, flowing in constantly from multiple channels and in many different forms.
What makes today’s data unique is that it is no longer structured and orderly. In fact, 80 percent of today’s data is unstructured and chaotic, making it very difficult to store, manage and analyze with traditional relational database management systems. What’s more, industry experts tell us that 90 percent of the world’s data was created over the last two years, and that worldwide data volume is set to double every two years.
For organizations, big data is another term for their data—the data generated by customer transactions, web traffic logs, sensors, online videos, social media interactions, web-enabled devices—the list goes on.
It may seem that the definition of big data stops here. But the term also frequently refers to the sophisticated tools and technologies, i.e. the open source Hadoop big data analytics platform, which can be used by organizations to capture, manage and mine mountains of unstructured data for valuable insights that can boost profits and competitive advantage.
What Big Data Is Not
The rapid rise of big data’s popularity has led many organizations to assume that big data and analytics is the end-all solution to their big data problems. But that is simply not the case, as there are a number of things that big data is not.
For starters, big data is not easy: Capturing, storing and analyzing massive volumes of disparate data flowing in from numerous channels to solve complex problems is not a simple task. It takes a considerable investment of time and money—not to mention skilled data scientists and analysts—to pull off a successful big data initiative.
Big data is not just about “lots of data”: Organizations may be faced with massively large data sets flowing in from one channel, but that in and of itself doesn’t make them candidates for a big data strategy. Big data is about running analytics on multiple disparate data sets streaming in from multiple sources.
Big data is not about traditional problem solving: Big data implementations are not designed to offer solutions to problems within the bounds of traditional perspective. In fact, instead of merely answering existing questions, big data analytics is more about positing new questions about patterns, relationships, and causations that were heretofore unknown.
Big data is not for every organization: Caught up in the hype, many organizations want to jump right on the big data bandwagon. But the truth is, big data isn’t for everyone. This is especially true of companies that expect positive, quantifiable results from their big data initiative right away. For these companies, a better approach would be to first become proficient with data management and business analytics. There are a number of powerful BI tools outside the realm of big data that organizations can use to make better decisions, optimize operations, identify trends, drive new revenues and gain an advantage over their competitors. Once they’ve gotten a firm grasp on these tools, organizations will have the expertise they need to successfully transition into a big data strategy.
Why is Big Data Important?
Analysis of large unstructured datasets can reveal hidden insights that can benefit organizations in almost any industry. The analysis of multi-source weather data can reveal patterns that can be used to predict future events and potentially save lives. Patient data generated by hospital charts, monitors, sensors, doctor visits and pharmacy records can be analyzed to help healthcare organizations and pharmaceutical companies develop better treatments, reduce hospital stays and improve outcomes. Business organizations can use big data analytics to tap into a wealth of insights about customer habits, behaviors, likes, dislikes and buying patterns—insights that drive better decisions and inform new and better products and services.
The age of big data has arrived. As organizations gain a greater understanding of what big data is, and how tools such as cloud-based Hadoop can help them to leverage their data for profit and competitive advantage, the possibilities of big data seem endless.
Interested in learning more? Grab our 5 minute guide to big data tools.