Data lake architectures without the vertigo

In the previous post, I explained what a data lake is and its main business benefits. This time around we will have a brief look at related and most commonly adopted data architectures.

The data lake has become a mainstream solution for businesses that want to stay on top of their data. Digital transformation was only the beginning: data creation is forecast to grow many times over between 2020 and 2035 (source: Statista). I believe the value extracted from data grows in line with its volume, and data platform implementations are following suit. It is no surprise that new data architecture patterns are appearing and the data lake has gone through its first evolutions. There are now three distinct implementations and architectures: centralised (the data lake itself), decentralised (data mesh) and a third that integrates a data warehouse and a data lake, called the lakehouse.

When we talk about a data lake we usually mean a centralised architecture. Having all data in one place has clear benefits and works well for organisations with small to medium data footprints. Organisations producing large volumes of data across the business may need a decentralised architecture. Enter data mesh, which organises data by business domain (for example marketing and sales), giving more ownership to the producers of a dataset and a better connection with data consumers.

Finally, organisations that already use both a data warehouse and a data lake can adopt lake house architecture, which integrates the two. Operationally, this avoids the problems of data gravity, namely unnecessary data movement and redundancy, and for the business it means reliable analytics and a faster time to market.

Having covered the architecture patterns related to the data lake, upcoming posts will look at each one in more detail.

For any data platform or AI needs, contact DataPhoenix.

Image source AWS.

Want to learn more? Once a pattern is chosen, the definitive guide to data lakes on AWS covers how to build it, and lake house architecture on AWS looks at the most common hybrid. For an independent view on which pattern fits your organisation, see our data platform consulting.