Data Engineer - Chennai - Immediate Joiners- Product Company

What Youll Do

• Design, implement, and maintain scalable data processing platforms used for real-time analytics and exploratory data analysis.
• Manage our various types of data from ingestion through ETL to storage and batch processing.
• Automate, test and harden all data workflow
• Architect logical and physical data models to ensure the needs of the business are met.
• Collaborate with our analytics teams, while applying best practices.
• Architect and develop systems and algorithms for distributed real-time analytics and data processing.

What Were Looking For

●Minimum of 5 years of experience working with high volume data infrastructure.

● Must have deep experience with both Databricks and AWS cloud computing platforms ● Proficient programming in Python

• Proficiency with SQL and ability to optimize querie
• Experience with large-scale data processing using Spark and/or Presto/Trino.
• Experience with a data transformation framework like dbt.
• Experience with job orchestration tooling like Airflow
• Experience with data ingestion frameworks like Fivetran.
• Experience in data modeling and database design.
• Experience in modern development lifecycle including Agile methodology, CI/CD, automated deployments using Terraform, GitHub Actions etc.