Data Engineer - Scala/Spark (5-7 yrs)

Job Title : Data Engineer

Duration : 6 Months (Contract)

Location : Bangalore, India

Timings : Full-Time (As per company timings)

Notice Period : Within 15 days or immediate joiners

Experience : 5+ years of relevant experience

About the Role :

We are seeking a highly skilled and motivated Data Engineer to join our team for a 6-month contract. In this role, you will play a crucial part in designing, developing, and implementing data-driven solutions that power our business. You will work with cutting-edge technologies and collaborate with cross-functional teams to deliver high-quality data products.

Key Responsibilities :

- Design and develop robust data ingestion frameworks, real-time processing pipelines, and data transformation frameworks utilizing open-source technologies like Kafka, Apache Spark (SQL, Scala, Java), and Python.

- Operationalize open-source data analytics tools for enterprise-wide use, ensuring seamless integration and optimal performance.

- Guarantee adherence to stringent data governance policies by implementing or validating data lineage, quality checks, and classification mechanisms.

- Adhere to the company's established development lifecycle for efficient development, deployment, and delivery of data solutions.

- Conduct in-depth root-cause analysis of data-related issues to identify and resolve problems effectively.

Key Skills & Requirements :

Strong proficiency in :

- Kafka

- Apache Spark (SQL, Scala, Java)

- Python

- Hadoop Platform

- Hive

- Airflow

- Cloud Composer

- PySpark

Experience with :

- Data warehousing and data lake architectures

- Agile development methodologies

- Cloud platforms (preferably GCP)

- Data quality and validation techniques

- Data security and privacy best practices

- Excellent communication and collaboration skills

Preferred Qualifications :

- Google Cloud Platform (GCP) certifications (e.g., Professional Data Engineer)

- Proven experience in data governance and DevOps practices

- Experience with containerization technologies (e.g., Docker, Kubernetes)