Building a Scalable, Resilient, and Cost-Effective Integration Platform at The Warehouse Group
to scale up
The integration platform is cloud agnostic, enabling flexibility and minimal downtime.
The integration architecture at TWG is deployed on a series of virtual machines across multiple environments. This shared solution hosted integrations for several projects, so any problem or updates to one component often affected all teams. Any changes required had to be completed manually; they were time-consuming and subject to error.
Given these complexities, TWG wanted to build a scalable, robust integration layer to seamlessly connect their systems, ensure consistent configurations, and troubleshoot issues easily.
Country: New Zealand
About The Warehouse Group
The Warehouse Group (TWG) is one of the largest retail groups in New Zealand and consists of 6 household retail brands: The Warehouse, Warehouse Stationery, Noel Leeming, Torpedo7, 1-day, and TheMarket.
TWG has over 260 retail stores throughout New Zealand at present. Corporate citizenship plays a key role in TWG’s business philosophy and over the years, TWG has contributed to community-development initiatives and advocates for sustainable retail practices.
For TWG it was crucial to scale the platform quickly and cost-effectively. TWG wanted to move to auto-scaling in a container deployment with WSO2 on Kubernetes.
Moving the WSO2 platform to Kubernetes enabled TWG to scale the platform easily. In contrast to the traditional VM based deployments, the business anticipated a high level of operational flexibility and substantial operational cost reduction. Since Kubernetes is backed by a strong observability eco-system, TWG was able to collect and monitor performance metrics related to the whole platform. WSO2 provided Docker images and Helm charts, which were instrumental in this process.
The architecture at TWG is based on WSO2 Enterprise Integrator running on Docker with AWS Elastic Kubernetes Services performing the container orchestration. Run-time metadata is mounted on Amazon Relational Database Service (RDS), and Amazon Elastic File System (EFS). CI/CD processes run on Jenkins. Two separate container groups handled sync and async integration flows to support their unique integration and scaling requirements. Containerizing the solutions has enabled the team at TWG to easily test and deploy across multiple environments, with a significant reduction in cost and time spent doing so.
- TWG now has a resilient integration platform that is able to elastically scale in a cost-effective manner. This has enabled TWG to meet their goals pertaining to isolation, scalability, resilience, and automation.
- Each team’s solutions are now deployed in isolation, so that in the case of an issue there is no impact across teams.
- Services now operate with no, or minimal, downtime as new containers are deployed before old ones are stopped.
- Patches, servers, and maintenance are no longer worries for the team at TWG.
- Shorter onboarding times for new teams - 2 hours nowadays compared to the 3 - 5 days needed for building a new environment previously.
- Elasticity too has improved - it now also takes a mere 60 seconds to scale up pods and 300 seconds to scale up nodes.
(Case study contributors: Dassana Wijesekara and David Hymers from WSO2 and The Warehouse Group)
Containerizing WSO2 has allowed us to take our existing WSO2 platform and easily scale it to meet demand and improve resiliency. This has all been done whilst being cloud agnostic, thereby affording us flexibility in our cloud strategy going forward.”
Tim Duly, GM Engineering at The Warehouse Group
What are your enterprise challenges?
We can help!