Innover - Senior Data Engineer - ETL/Python (5-6 yrs)

About the Role :

- We are looking for a skilled Data Engineer to join team and contribute to the development and maintenance of a customer-facing reporting and analytics platform.

- This role involves designing and implementing data models, ETL pipelines, and BI solutions while ensuring scalability and performance.

Key Responsibilities :

- Design, develop, and implement data models, ETL pipelines, and BI solutions using Spark, SQL, and Python.

- Translate complex technical requirements into efficient and scalable application code, aligning with cloud architecture principles.

- Collaborate with cross-functional teams, including product owners, business analysts, and quality assurance, to deliver high-quality solutions.

- Adopt a DevOps mindset, promoting continuous integration and development process improvements.

- Identify and resolve performance bottlenecks, ensuring the platform meets customer SLAs and scales for large data volumes.

- Work in a fast-paced, agile environment, demonstrating a passion for continuous learning and innovation.

Required Skills :

- Experience in developing Spark applications, including performance tuning and debugging.

- Proficient in Python development, with a focus on robust and optimized code.

- Experience building production-ready ETL pipelines for data warehouses or operational data stores.

- Hands-on experience with relational databases like Oracle, DB2, Teradata, Netezza, or Redshift.

- Strong SQL and data modeling skills.

- Excellent analytical, problem-solving, and troubleshooting abilities.

Preferred Qualifications & Skills :

- Experience with AWS stack (EMR, Redshift).

- Proficiency in Linux and shell scripting.

- Familiarity with ETL orchestration tools (Airflow, Control-M, AWS Step Functions).

- Experience optimizing ETL code for high-volume data processing.