Introduction
A leading Telecom client engaged ProCogia to perform an end-to-end review of their all their adhoc data processing solutions that were running on on-premise clusters or EC2 instances. ProCogia advised the client that it could optimize its data processing by migrating the ad-hoc computations to AWS Lambda.
The departments Management shifted towards our recommendations to move any ad-hoc analytics/data processing to serverless computing using AWS Lambda, which provided the required scalability, availability, and fault tolerance without maintenance overhead and without having to share any limited dedicated resources.
The Challenge
A leading telecom client’s ad-hoc analytics and data processing jobs were running on an on-premise cluster or on dedicated EC2 instances in AWS. It resulted in increased maintenance overhead and costs. Additionally, there were scheduling challenges that inevitably came along with the limited resources that had to be shared.
The challenge was to scale as the number of jobs increased, while ensuring that those jobs ran as efficiently as possible and infrastructure costs were kept low.
Procogia’s Approach
- Collaborated with stakeholders to understand the current ad-hoc processing jobs & predict future demand based on team fluctuations.
- Implemented the ad-hoc jobs in AWS Lambda, a serverless computing engine which enabled teams to run multiple ad-hoc data analysis scripts and schedule them via CloudWatch.
- Lambda bills for usage in increments of 100ms. These were more cost effective over EC2 instances or dedicated servers running adhoc scripts. ProCogia ensured the Lambda costs were low by building efficient Lambda functions which utilized Lambda’s permissible parallel computing power as needed.
- We helped users and teams to schedule their scripts without worrying about resource contention.
The Results
- ProCogia provided the client with a scalable solution which optimized the use of AWS resources and led to cost savings.
- The leading telecom client is now able to process any number of new ad-hoc scripts using serverless computing capabilities of AWS Lambda without having to worry about increased costs or maintenance overhead
Services Used
Cloud Data Engineering
We partner with all major cloud providers, allowing us to adopt a data-agnostic approach focused on delivering tailored game-changing solutions.
Data Consultancy
We provide data consultancy to organizations to optimize your investment in people, processes, and technology. This is typically through data strategy engagements, roadmaps, transformations, and independent technology advice.
Data Operations (DataOps)
We build robust, scalable, and enterprising data environments that enhance the collaboration between Data Science teams.
Related Blogs
Let’s connect
What can we help you with?
T: +1 425-624-7532
Alternatively, simply fill in this form and we’ll be in touch.
