Home
Communities
Companies
- Companies
  
  Discover best places to work
- Compare Companies
  
  Compare & find best workplace
- Add Office Photos
  
  Bring your workplace to life
- Add Company Benefits
  
  Highlight your company's perks
Reviews
- Company reviews
  
  Read reviews for 6L+ companies
- Write a review
  
  Rate your former or current company
Salaries
- Browse salaries
  
  Discover salaries for 6L+ companies
- Salary calculator
  
  Calculate your take home salary
- Are you paid fairly?
  
  Check your market value
- Share your salary
  
  Help other jobseekers
Interviews
- Company interviews
  
  Read interviews for 40K+ companies
- Campus placements
  
  Interviews questions for 1K+ colleges
- Share interview questions
  
  Contribute your interview questions
Jobs
Awards

RATE NOW!
- ABECA 2025
  
  RATE NOW!
  
  AmbitionBox Employee Choice Awards - 4th Edition
- ABECA 2024
  
  AmbitionBox Employee Choice Awards - 3rd Edition
- AmbitionBox Best Places to Work 2022
  
  2nd Edition
- AmbitionBox Best Places to Work 2021
  
  1st Edition

Add office photos

Employer? Claim Account for FREE

Milestone Technologies

Compare

3.6

based on 51 Reviews

5 Milestone Technologies Jobs

Lead ETL Developer

Covestic (a Milestone company)

3.6

based on 51 Reviews

3-8 years

Hyderabad / Secunderabad

1 vacancy

Lead ETL Developer

Milestone Technologies

posted 16d ago

Job Role Insights

Flexible timing

Key skills for the job

Data Analytics Python Consulting SQL Automation Testing Information Technology

+ 3 more

Job Description

Job Overview

Senior Data Engineer

Position Overview:

We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for extensive data processing, analytics, and business intelligence functions. Demonstrating profound knowledge in cloud-native tools and platforms including AWS, Azure, Oracle, Snowflake, and Databricks, the Senior Data Engineer will also excel in deploying advanced data integration techniques utilizing cloud-native ETL tools (e.g., AWS Glue, Azure Data Factory) and Oracle Data Integrator (ODI).

Key Responsibilities:

Lead the architecture and development of scalable data solutions across both batch and streaming platforms, optimizing the use of cloud technologies (AWS or similar) to meet business and technical demands, including real-time data processing.
Analyze complex data elements and systems, engaging in the development of conceptual, logical, and physical data models, along with the validation and execution of ETL/ELT mappings and transformation logic.
Craft and peer review data models and schemas, enhancing new and current data sources for the data warehouse, ensuring efficiency and scalability.
Conduct thorough unit, integration, and system testing of data sources to ensure data integrity against source systems, aiming for continuous performance optimization.
Innovate with Snowflake s Snowpark and Snowpipe to architect scalable data management solutions, facilitating real-time data ingestion.
Leverage Apache Spark for comprehensive data processing and analytics, enhancing data workflow efficiency and scalability.
Implement Databricks Delta Live Tables to automate and ensure data ETL processes, maintaining data quality and consistency for real-time analytics.
Maintain high-performance data pipelines, making extensive use of AWS, Azure, and Databricks, along with seamless integration with Oracle Data Integrator (ODI).
Utilize cloud native ETL tools to streamline data integration and transformation across diverse data platforms.
Develop sophisticated data transformations and machine learning models within Snowflake using Snowpark to boost analytics capabilities.
Ensure the availability of real-time data and analytics through efficient configuration and optimization of Snowpipe.
Collaborate with business stakeholders and IT departments to transform business needs into technical specifications and practical data models.
Uphold data governance, security, and compliance standards across all data platforms and tools.
Remain abreast of emerging trends and technologies in data engineering, promoting innovative solutions that propel the data strategy.

Qualifications:

Bachelor s or master s degree in computer science, Information Technology, Engineering, or a related field.
Proven expertise in data processing and analytics using Apache Spark, with hands-on experience in Databricks Delta Live Tables for managing real-time data.
Proficiency in Snowflake, including mastery of Snowpark and Snowpipe, paired with experience in cloud native ETL tools (AWS Glue, Azure Data Factory, Google Cloud Dataflow).
Comprehensive background in Oracle, Snowflake, Databricks, and Oracle Data Integrator (ODI) for in-depth data integration and processing.
Strong programming skills in SQL, Python, Java, or Scala, with the capability to develop intricate data models and algorithms.
In-depth knowledge of data warehousing, data modeling, and data lake concepts within a cloud environment.
Exceptional problem-solving skills, complemented by the ability to work both independently and collaboratively.
Outstanding communication skills, with proficiency in engaging with both technical and non-technical team members.
Experience in managing and leading team

Compensation

Estimated Pay Range: Exact compensation and offers of employment are dependent on circumstances of each case and will be determined based on job-related knowledge, skills, experience, licenses or certifications, and location.

Our Commitment to Diversity & Inclusion

Back Share

Apply Now

Employment Type: Full Time, Permanent

Read full job description

Prepare for ETL Developer Lead roles with real interview advice