Senior Data Engineer - Python/SQL (6-10 yrs)

Role : Senior Data Engineer

Experience : 6+ years

Location : Bangalore / Hybrid

Job Type : Full-time

Role Overview :

We are looking for a Senior Data Engineer with 6+ years of experience in designing and building scalable data pipelines. The ideal candidate should have strong hands-on experience in Python and SQL, expertise in Databricks and Snowflake, and proficiency in streaming tools like Kafka. The role involves collaborating with teams in the USA and UK for requirement gathering and implementation.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines for processing large volumes of structured and unstructured data.

- Work extensively with Python and SQL to build ETL/ELT workflows.

- Implement data lakehouse solutions using Databricks and data warehousing solutions in Snowflake.

- Optimize data pipelines for performance, scalability, and reliability.

- Work with real-time data streaming tools like Kafka for event-driven architecture.

- Collaborate with cross-functional teams across the USA and UK to understand business requirements and translate them into technical solutions.

- Ensure data governance, quality, and security in all data processes.

- Automate workflows and monitor data pipelines to minimize failures and optimize efficiency.

- Troubleshoot and resolve issues in data pipelines and infrastructure.

Required Skills & Experience :

- 6+ years of experience in data engineering, building data pipelines, and working with large-scale data systems.

- Strong expertise in Python and SQL for data processing and transformation.

- Hands-on experience with Databricks for building scalable data solutions.

- Proficiency in Snowflake for data warehousing and analytics.

- Experience working with streaming platforms like Kafka for real-time data processing.

- Experience in optimizing ETL/ELT workflows for efficiency and cost-effectiveness.

- Ability to work with teams across different time zones, especially in the USA and UK.

- Strong problem-solving skills and ability to work in a fast-paced environment.

Preferred Skills :

- Experience with cloud platforms (AWS, Azure, or GCP) for data storage and processing.

- Familiarity with orchestration tools like Airflow.

- Experience with CI/CD for data engineering workflows.

- Knowledge of data modeling and governance best practices.