Valueleaf - Data Engineer - Big Data (7-9 yrs)

Experience Required : 7+ Years.

Location : Bengaluru.

Employment Type : Full-Time.

Industry : Fintech/Adtech/Messaging

Job Summary :

We are seeking a highly experienced and motivated Data Engineer to oversee the design, development, and management of scalable, high-performance data pipelines and platforms.

The ideal candidate will have a strong background in data engineering, cloud technologies, and big data frameworks, combined with leadership skills to guide a team of data engineers in delivering cutting-edge fintech solutions.

Key Responsibilities :

Leadership & Strategy :

- Lead and mentor a team of data engineers to deliver high-quality, scalable, and reliable data solutions.

- Define and implement the data engineering roadmap in alignment with business goals and analytics requirements.

- Collaborate with cross-functional teams, including data science, analytics, and product teams, to design data-driven solutions.

Data Architecture :

- Architect, build, and maintain robust data pipelines for batch and real-time processing.

- Design data warehouses and lakes using best practices for data modeling and governance.

- Ensure data systems are scalable, maintainable, and meet business performance SLAs.

Technology & Development :

- Implement and optimize ETL/ELT workflows using tools like Apache Spark, Airflow, or Talend.

- Work with big data technologies such as Hadoop, Kafka, Clickhouse and Snowflake.

- Develop and optimize SQL queries and scripts for data transformation.

- Ensure robust data security, integrity, and compliance with regulatory standards.

Stakeholder Management :

- Act as a liaison between technical teams and business stakeholders to translate business needs into technical solutions.

- Present progress, challenges, and key insights to senior management.

Innovation & Best Practices :

- Evaluate and adopt emerging technologies to enhance data processing capabilities.

- Establish and enforce data engineering best practices, including coding standards, testing, and documentation.

Key Requirements Technical Skills :

- Big Data : Proficiency in Hadoop, Apache Spark, Kafka, and other distributed computing frameworks.

- Programming : Strong coding skills in Python, Java, or Scala.

- Cloud Platforms : Experience with AWS (S3, Redshift, EMR, Athena, Glue).

- Database Management : Expertise in SQL, NoSQL databases (i.e., MongoDB, Cassandra), and relational databases (PostgreSQL, MySQL).

- ETL Tools : Hands-on experience with tools like Apache Airflow, Talend, or Informatica.

- Data Warehousing : Proficiency in building and managing data warehouses like Snowflake, BigQuery, or Redshift.

- DevOps : Familiarity with CI/CD pipelines, containerization (Docker, Kubernetes), and infrastructure-as-code (Terraform, CloudFormation).

Leadership Skills :

- Proven ability to lead and grow a high-performing data engineering team.

- Strong problem-solving and decision-making skills.

- Excellent communication and stakeholder management abilities.

Experience :

- 7+ years of experience in data engineering, with at least 2 years in a leadership role.

- Experience working in a fintech/Adtech/Messaging domain is a strong plus.