Lake House Architecture on AWS: Combining the Data Lake and the Data Warehouse

What a lake house is

A data lake holds raw data of every kind at low cost and at scale. A data warehouse holds curated, structured data and answers business questions quickly. Many organisations run both, and pay for it in copied data, duplicated pipelines and reports that disagree. Lake house architecture removes that split. It unifies the lake and the warehouse at the storage and catalogue layers, so the same data can be processed and consumed from either side without being moved between them. This article summarises the approach AWS describes in the source linked above.

On AWS, that means fewer redundant copies and less ETL code to build and maintain, and faster processing because data no longer travels between systems. The business gets one consolidated view of its data, quicker onboarding of new sources and analytics it can share across the organisation.

How it works

The data warehouse and the data lake are integrated at the storage and catalogue layers and present a single interface to the processing and consumption layers. Tools in those layers, such as SQL engines and Spark, can read data across both the warehouse and the lake. Native ETL or ELT pipelines in the warehouse can combine flat relational data with complex, hierarchical data held in the lake.

The layers on AWS

Storage sits underneath everything, with Amazon S3 holding the lake and Amazon Redshift the warehouse. Four layers build on it.

Ingestion. This layer collects data from databases, applications and IoT devices. On AWS, Amazon Kinesis handles streaming data and Amazon MWAA (managed Apache Airflow) handles batch workloads, with consistency, security and privacy designed in.

Catalogue. The central metadata repository holds schema definitions, lineage and other metadata, which supports discovery and governance. The AWS Glue Data Catalog integrates with Amazon S3, Amazon Redshift and Amazon Athena to provide one catalogue for the lake and the warehouse.

Processing. This layer transforms, cleans and enriches data for analysis. AWS Glue provides serverless processing, Amazon EMR runs big data workloads on Apache Spark, and Amazon Redshift handles warehousing tasks.

Consumption. This is where people use the data. Amazon Athena offers serverless querying, Amazon Redshift Spectrum extends warehouse queries into the lake, and Amazon QuickSight builds interactive dashboards, so different roles in the organisation can analyse data in the way that suits them.

Why it matters

Together, these layers give organisations a scalable, cost-effective data management platform that serves analytics, reporting and AI from the same governed data. DataPhoenix designs and builds lake house architectures on AWS, from first design to handover. For any data platform or AI needs, contact DataPhoenix.

Want to learn more? The lake house pattern builds on the foundations described in the definitive guide to data lakes on AWS and on the zones in AWS data lake layers explained. If you are deciding between AWS native services, Databricks and Snowflake for the warehouse half, see choosing the right data platform. Our data lake services cover both halves. For search over the lake, see OpenSearch and Elasticsearch for AWS data lakes.