Data Engineer - Redshift , Pyspark

Minimum 4+ years of experience on developing data ingestion pipelines using Pyspark(Batch and Streaming) AND with designing tables in Redshift(distribution key, compression etc., vacuuming,etc.)

Analyze and understand customers use case and data sources and extract, transform and load data from multitude of customers enterprise sources and ingest into Companys Experience Platform.

Design and build data ingestion pipelines into the platform using PySpark.

Ensure ingestion is designed and implemented in a performant manner to support the throughout and latency needed.

Develop and test complex SQLs to extract\analyze and report the data ingested into Companys Experience platform.

Ensure the SQLs are implemented in compliance with the best practice to they are performant.

Migrate platform configurations, including the data ingestion pipelines and SQL, across various sandboxes.

Debug any issues reported on data ingestion, SQL or any other functionalities of the platform and resolve the issues.

Support Data Architects in implementing data model in the platform.

Contribute to the innovation charter and develop intellectual property for the organization.

Present on advanced features and complex use case implementations at multiple forums.

Attend regular scrum events or equivalent and provide update on the deliverables.

Work independently across multiple engagements with none or minimum supervision.