Business Need:
Aviation Industry has a wide range of Aircraft engines and has very huge data set to processed on day to day basis. The current Greenplum instances are costly and will soon lack sufficient support. Performance is also one of the challenge in Greenplum. Some Cybersecurity Maturity Model Certification (CMMC) controls are more sensible to implement in AWS Redshift.
Solution and Approach:
AWS Redshift comes with high reliability and high availability, along with the best performance compared to the GreenPlum DB. As part of the solution, all the objects in Greenplum database have to be rearchitected and migrated along with the data copy, also all the ingestion jobs/integrations have to connect to source and load the data into S3 bucket and then a process to load the data from S3 bucket to Redshift. Dataiku jobs process the data and load into Redshift. Downstream applications and data analytics platforms fetch the consume the data from Redshift.
Below is the high level architecture of the solution implemented.
Benefits:
AWS Redshift gave 50% improvement in performance compared to Greenplum.
AWS Redshift is very high reliability and high availability of ~99.99%.
Very minimal failures in the functions compared to Greenplum.
Migration Volume:
Schema Count: 770
Function Count: 5390
View Count: 5700
Spotfire Reports: 953
Performance Improvement: 50%
Reliability and Availability: 99.99%
