White Papers

The Evolving Role of the Data Engineer

Issue link: https://www.qubole.com/resources/i/1243713

Contents of this Issue


Page 6 of 63

Foreword Today's exponential growth in data velocity, volume, and variety is forcing enterprises to look for a platform that addresses their imme‐ diate needs and scales to meet their requirements. Modern organiza‐ tions have data of multiple types and variable velocity, consumed by multiple personas on a continuous basis. The organizations are also accountable, legally and ethically, to their customers for how they collect and store personal information. What's more, modern interactive analytics and machine learning demand continuous data processing as users experiment and iterate with different data types to arrive at the insights these tools can pro‐ vide their businesses. Successful companies also implement financial governance controls that lead to cost savings when taking advantage of public clouds. This reduces guesswork and avoids difficult cost-controlling exercises. Enterprises want to do more with their data lakes: they see the bene‐ fits of leveraging inexpensive storage, the flexibility to support a great diversity of data, and advanced analytics that offer meaningful insights. At Qubole, we've created an Open Data Lake Platform that addresses all of this complexity as well as potential future requirements. It sup‐ ports open industry-standard ACID transactionality as a built-in feature to deliver on data privacy and right-to-be-forgotten require‐ ments. We've also made sure to support continuous data engineering and streaming analytics in addition to data exploration, ad hoc ana‐ lytics, and machine learning. v

Articles in this issue

view archives of White Papers - The Evolving Role of the Data Engineer