ADCG Explainer: How Data Lakehouses Can Help Your Compliance Scheme

When it comes to storing your data, organization is important. But many companies aren’t that organized, and as a result, data often ends up in what’s known as a data lake—or, as Venture Beat calls it, “a broader repository that stores data in its raw or natural format.”

Data lakes, by definition, can be messy and difficult to make sense of. This is why organizations will often opt to utilize a “data lakehouse.” This organizational method, which Forbes calls a hybrid mechanism, solves this by adding “layers of optimization to make the data more broadly consumable for gathering insights.”In other words, a lakehouse takes the highly-organized reporting and data analysis tools of a data warehouse platform and applies them to unstructured data in a lake format.

Data that has been processed by a data lakehouse may be used to provide an organization with enhanced flexibility, scalability, cost savings and exploration capabilities when compared to legacy architecture. That’s according to VentureBeat, which also notes that such a scheme can promote an ease of use for “applications such as artificial intelligence and machine learning[,]” and permit the organization to utilize that data for “real-time analysis, data democratization, and improved business outcomes via data-driven decisions.”

But while there are many benefits to a data lake house, there are struggles that an organization could face when implementing these data processing structures.

According to VentureBeat, organizations with existing architectures for storing data face the difficult task of migrating data in a legacy format to a new data lakehouse. This can be costly, prolonged, and disruptive to business operations.

To avoid this,  Adrian Estala, field chief data officer at Starburst, told VentureBeat that the use of a “phased migration approach” developed by your organization “should minimize business disruption and prioritize data assets based on your analytics use cases[.]”

A phased migration starts, according to Estala, by establishing a “virtualization layer across existing warehouse environments, building virtual data products that reflect the current legacy warehouse schemas.” Then, these products can be used “to maintain existing solutions and ensure business continuity.”

Once the existing data processes are secured, your organization should “prioritize moving datasets based on cost, complexity or existing analytics use cases.” Ronthal agreed with Estala’s recommended processes, and further promoted a “continuous assessment and testing” approach that begins with migrating the most complex data to ensure that the new data lakehouse is established in accordance with your organization’s expectations and needs.

To learn more about data lakehouses and assess the value that one could add to your organization, you can download the comprehensive five-part eBook on the ItProToday website.

* * * * * * *

To read our news alerts discussing: Florida and Montana’s privacy bills, projected growth in the data privacy software market, and a newly-unveiled encryption tool, click here.

This week’s breach report covers the following organizations: The American Bar Association, CFPB, Jewel Osco. Click here to find out more.

Jody Westby hosts our podcast, ADCG on Privacy & Cybersecurity, bringing together leaders in the privacy and cybersecurity arenas to discuss a wide range of issues ranging from the proposed federal and state regulations to best practices and standards for compliance. Episodes can be enjoyed on many platforms including Spotify and Apple Podcasts. Don’t forget to subscribe!

Our most recently released episodes:

90 | AdTech Meets Privacy Laws (with Guest Susan Israel)

89 | Quantum Technologies: What is Possible, Where We Are Headed & Policy Issues to Consider (with guests Berit Anderson, and Evan Anderson)

88 | TikTok: A Path for Election Interference and Open Source Intelligence? (with guest Chris Hoofnagle)

To browse our previously published articles and news alerts, please visit our website, and don’t forget to subscribe to receive free weekly Data and Cyber Governance news and Breach Reports directly to your email.