Business Challenge
When we were brought on, this NGO’s network of websites were in trouble. Our AWS DevOps Consultants performed an extensive audit that revealed a slew of major security issues and dozens of critical-but-untracked AWS resources (RDS, EC2, IAM, ElastiCache, S3, Route53, SES, SNS, VPN ). The codebase had been untouched for over two years, and there was no documentation about the building or deployment process.
AWS DevOps Transformation
Critical security updates and patches were made on the codebase, all secrets and configs which had been hard-coded were pulled out during application dockerization. A CI/CD pipeline was developed using Gitlab that automatically built and deployed each application for testing as commits were made to the development branch. Existing AWS resources were migrated without downtime to a new version-controlled and Terraform-managed VPC. Infrastructure, application code, and configuration are all now checked into version control, allowing easy rollbacks to occur. Security is now granular, with user and application-specific permissions to cloud and database resources conforming to the principle of least privilege.
The websites are now self-healing and easily scalable. Full logging, monitoring, and alerting allow the engineering team to pinpoint the source of problems quickly and nip problems in the bud. Hundreds of DNS entries were audited and migrated to a new CDN provider which has helped eliminate malicious traffic.
Results
Repository changes in Gitlab start a build process, creating a new docker image stored in the EC2 Container Registry, which is automatically deployed to a testing environment. Once tests pass and the team is satisfied, the image is promoted to production. Regressions are captured in the pipeline and stopped at the door prior to getting to production.
We provide 24/7 managed AWS support to the infrastructure and monitoring the applications allows engineers the ability to instantly detect issues with their deployments and alert the team. Prometheus, Alertmonitor, and Grafana provide metrics and alerting to the team.
The engineering team is able to Build, Test, Deploy, and Monitor their work without having to be involved in the infrastructure at all. This is because the tooling allows them to access the items they need to ensure the applications are behaving correctly and consistently performant.
Non-production environments are spun down each night, and are instantiated as developers need them, keeping costs absurdly low. When a new environment is spun up teams with access can start using it immediately, as it mirrors the existing production environment. Standing up new environments takes less time than making a pot of coffee.
Conclusion
Our DevOps Engineers are able to build, test, and deploy new features automatically. Entire environments can be instantiated at the push of a button. Promotion, rollbacks, backing up, and restoring state can be done without breaking a sweat. The team is able to ship changes to the applications into different environments during development & testing and efficiently deploy them to the cloud at scale. The team can focus on delivering much-needed new features.