Senior Data Engineer - Hadoop/Hive/Python (5-8 yrs)

Job Title : Senior Data Engineer (Snowflake, Azure/AWS, Hadoop, Hive, Python, Databases)

Experience : 5 to 8 years

Location : Pune and Chennai, India

Job Description :

We are seeking an experienced and talented Senior Data Engineer to join our dynamic team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines, working with both cloud-based and on-premise data solutions. This role requires strong expertise in Snowflake, Azure/AWS cloud platforms, Hadoop ecosystem, Hive, Python, and relational/non-relational databases.

Key Responsibilities :

- Design, develop, and optimize scalable data pipelines using Snowflake, AWS, or Azure cloud technologies.

- Build and maintain ETL processes for large-scale data integration from various sources.

- Implement data solutions on cloud platforms (Azure/AWS) ensuring best practices for security, scalability, and performance.

- Develop and optimize SQL queries, data models, and schemas within Snowflake and other database technologies.

- Leverage Hadoop and Hive technologies for big data processing and analytics.

- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.

- Ensure data quality, consistency, and integrity across all pipelines and datasets.

- Perform regular performance tuning and optimization of data processes.

- Contribute to data governance and ensure compliance with industry standards.

- Troubleshoot and resolve issues related to data engineering pipelines and systems.

- Lead and mentor junior data engineers, offering guidance on best practices and technical solutions.

Required Skills & Experience :

- 5 to 8 years of experience in Data Engineering, with a focus on cloud platforms (Azure or AWS).

- Expertise in Snowflake data warehousing platform.

- Strong experience working with big data technologies such as Hadoop, Hive, and other related frameworks.

- Proficiency in programming languages like Python for building scalable data solutions.

- Hands-on experience with SQL and NoSQL databases.

- Familiarity with ETL tools and data pipeline orchestration tools (e.g., Apache Airflow, Talend).

- Experience with data modeling, data integration, and data transformation techniques.

- Strong knowledge of cloud platforms (Azure or AWS), including data storage, compute, and orchestration services.

- Understanding of version control systems (Git) and CI/CD pipelines.

- Strong analytical skills and attention to detail.

- Excellent communication and collaboration skills, with the ability to work cross-functionally with teams.

Preferred Skills :

- Certifications in Snowflake, AWS, or Azure are a plus.

- Knowledge of machine learning or AI integrations with data platforms.

- Familiarity with Data Lake and Data Warehouse concepts.

Education : Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.