Here you'll find cool data tech articles and our learnings from working on data solutions.
Searching an AWS data lake for ‘Elasticsearch’ now means Amazon OpenSearch Service. What a search engine adds to S3 and Athena, where it does not belong, a reference setup, the cost traps, and a checklist for deciding if your lake needs one.
Publishing an illustrated, narrated series used to take a team. Now it takes one person with the idea and one who knows agentic workflows. How DataPhoenix worked as the forward-deployed engineer for a travel series: map the work, build inside the client’s operation, ship, stay while it breaks, hand over.
Picking a data platform is one of the most expensive decisions a data team makes. AWS native, Databricks and Snowflake aren’t three flavours of the same thing: one gives you building blocks, one a complete lakehouse, one a managed SQL engine. Here’s how to tell which one fits before you commit…
AI products that impress in a demo have a habit of going quietly wrong in production: stale answers, numbers that won’t reconcile, PII flowing unmasked into a model. The engineers aren’t at fault. The data underneath was never treated as a first-class engineering concern, and that’s a data platform problem, not an AI one…
Told to “get it into the lake”? Here’s what actually happens between that conversation and the point where an analyst or an AI agent can trust the data: the medallion delivery framework, Bronze to Silver to Gold, and the gates a new source has to pass at each layer…
In our data-centric world, organizations are embracing contemporary data architectures to manage, process, and analyze vast volumes of information. The Lake House Architecture, a merger of data lake and data warehouse, is one such model. This article delves into the advantages of AWS-based Lake House Architecture and how it empowers businesses to fully harness their data…
In the previous post, I explained what a data lake is and its main business benefits. This time around we will have a brief look at related and most commonly adopted data architectures…
In this article, we will overview the different layers of a well-built data lake, as well as highlight common pitfalls and challenges to consider when implementing a data lake architecture…
When I tackled my first data lake project it was difficult for me to understand what it actually is. However, I finally grasped the concept with the help of understanding that there are three types of data — structured, semi-structured, and unstructured. In layman’s term, it is a data management solution that stores and process raw data at scale…
This guide aims to empower you with the fundamental knowledge needed to understand what a standard data lake is, it’s main architecture elements and how you can utilise AWS services for managing your data securely and effectively. With base-level understanding, you can build your expertise in data lakes and implement them across various business functions.
If you’ve been working with me for a while, you’ll probably know some of the great work I’ve been involved in in the realms of data solutions like data lakes, data mesh, AWS and DevOps…
Ready to put AI to work on your data? Get in touch with the DataPhoenix team.