DataPhoenix logo
DataPhoenix
menu
more detail

Secure data platform

National utilities operator

Creating a more secure & performant environment

Contact Us

Address performance and security concerns

Outcome. One data lake on AWS instead of several environments maintained in different ways. Access follows least privilege, new data sources go through a defined approval process, and requests to view or delete personal data can be actioned with confidence. The platform is maintained through infrastructure as code, Agile delivery and CI/CD, so the operational teams that were overloaded now have one estate to run.

A UK-based business in the energy sector was experiencing ongoing issues related to managing its platform. Its operational teams were overloaded with multiple data environments, sources, processing methodologies and storage options. Because the solution was maintained in different ways by different teams, no one had a comprehensive picture of the entire estate. Adding new features and data sources was painfully slow.

The company engaged the Data Phoenix team and we conducted a platform survey. In addition to the identified performance issues, we raised a number of security concerns. Access policies didn’t follow a ‘least privilege’ approach. The new data source approval process was not well defined. Furthermore, GDPR-type requests to view or delete personal information or deletion couldn’t be actioned with confidence.

Data Phoenix helped to develop a sustainable and highly performant Data Lake solution that fulfilled necessary security requirements and delivered significant improvements. Moving forward, the solution is far easier to maintain thanks to the implementation of best practice approaches including cross-team collaboration, IAC, Agile and CI/CD.

Want to learn more? This platform was built on the approach described in our AWS data lake services, with the personal-data handling from data classification and PII governance. The definitive guide to data lakes on AWS explains the architecture in general terms.

Goals

  • Unify data platform and converge Operational teams
  • Simplify operational workflow used by Data Engineers and Data Scientists
  • Keep data in check stay compliant with regulatory requirements
  • Ability to share & sale mined data with other business units in controlled manner

Solution

  • State-of-the-art implementation: AWS, Terraform, Git, Atlantis
  • Data streamed from all sources into a single Data Lake repository
  • Easy-to-maintain modular approach with Infrastructure as Code
  • Self-service workflow and secure approval process
  • Incorporated ‘least privilege by design’ in the solution's code

Benefits

  • Transparent, indexed and secure data all in one place
  • New features can be delivered rapidly
  • Data Engineers and Data Scientists can work efficiently by scripting platform components
  • Reliable environments with testing in CI/CD pipelines
  • Data shared securely across different business units – breaking down silos and improving business efficiency
  • Reduced risk of brand damage and compliance-related fines
  • Easier to find operational staff thank to use of market-standard tooling

Unleash the Power of Your Data

If you’d like to know more about how we can help you architect and implement a solution that helps you unleash the value in your data, please get in touch.

Contact Us

Click for learn
More About us

Contact Us  

Ready to put AI to work on your data? Get in touch with the DataPhoenix team.