Business need
Leading energy technology company had Big Data Green plum clusters are responsible for running all ingestion, analytic and data science jobs on a different Individual clusters, clusters are running the scheduled jobs and the ad hoc jobs based on business needs. With this current setup license cost is way too high and doesn’t want to spend money on license and asked us to provide the alternative solution. As part of New Architecture, suggested to migrate the Green plum to Data Bricks and Talend to Pyspark.
Approach & solution
Wissen Implemented the new architecture, replaced the Green Plum to Databricks and Talend to Pyspark. Successfully migrated all the applications data into new environment.
With this approach, there is no reduction in performance and we have reduced overcall cost.
Benefit :
By migrating to Open-Source ETL and hosting on AWS customer has reduced 15% of the cost and saving 2M USD per year.
