DataPhoenix logo
DataPhoenix
menu
more detail

Secure data platform

National utilities operator

Creating a more secure & performant environment

Contact Us

Address performance and security concerns

Outcome. One data lake on AWS instead of several environments maintained in different ways. Access follows least privilege, new data sources go through a defined approval process, and requests to view or delete personal data can be actioned with confidence. The platform is maintained through infrastructure as code, Agile delivery and CI/CD, so the operational teams that were overloaded now have one estate to run.

A UK energy business was struggling to manage its data platform. Its operational teams were overloaded with multiple data environments, sources, processing methodologies and storage options. Because the solution was maintained in different ways by different teams, no one had a comprehensive picture of the entire estate. Adding new features and data sources was slow.

The company engaged DataPhoenix, which surveyed the platform. Besides the performance issues, the survey raised several security concerns. Access policies didn’t follow a ‘least privilege’ approach. The new data source approval process was not well defined. Furthermore, GDPR requests to view or delete personal information could not be actioned with confidence.

DataPhoenix helped develop a sustainable, high-performing data lake that met the security requirements and delivered significant improvements. The solution is now far easier to maintain, thanks to best practice including cross-team collaboration, infrastructure as code, Agile and CI/CD.

Want to learn more? This platform was built on the approach described in our AWS data lake services, with the personal-data handling from data classification and PII governance. The definitive guide to data lakes on AWS explains the architecture in general terms.

Goals

  • Unify the data platform and converge operational teams
  • Simplify operational workflow used by Data Engineers and Data Scientists
  • Keep data in check and stay compliant with regulatory requirements
  • Share and sell mined data with other business units in a controlled manner

Solution

  • Modern implementation: AWS, Terraform, Git, Atlantis
  • Data streamed from all sources into a single Data Lake repository
  • Easy-to-maintain modular approach with Infrastructure as Code
  • Self-service workflow and secure approval process
  • Incorporated ‘least privilege by design’ in the solution's code

Benefits

  • Transparent, indexed and secure data all in one place
  • New features can be delivered rapidly
  • Data Engineers and Data Scientists can work efficiently by scripting platform components
  • Reliable environments with testing in CI/CD pipelines
  • Data shared securely across business units, breaking down silos and improving efficiency
  • Reduced risk of brand damage and compliance-related fines
  • Easier to recruit operational staff thanks to market-standard tooling

Talk to DataPhoenix about your data

Tell DataPhoenix about your data and AI goals and we will work with you on where to start and what to build first.

Contact Us

Click for learn
More About us

Contact Us  

Ready to put AI to work on your data? Get in touch with the DataPhoenix team.