AmbitionBox
Jobs
- Reviews
- Salaries
- Interview Questions
- About Company
- Benefits
- Jobs
- Office Photos
- Community
Add office photos
i
UST
Compare
All Filters
1027 UST Jobs
Data Engineering Lead - AWS Glue & PySpark Specialist
UST
·
5-7 years
Bangalore / Bengaluru
1 vacancy
Data Engineering Lead - AWS Glue & PySpark Specialist
UST
posted 3mon ago
Job Role Insights
Flexible timing
Key skills for the job
Job Description
- Spark PySpark Development : Design and implement scalable data processing pipelines using Apache Spark and PySpark to support large-scale data transformations.
- ETL Pipeline Development : Build, maintain, and optimize ETL processes for seamless data extraction, transformation, and loading across various data sources and destinations.
- AWS Glue Integration : Utilize AWS Glue to create, run, and monitor serverless ETL jobs for data transformations and integrations in the cloud.
- Python Scripting : Develop efficient, reusable Python scripts to support data manipulation, analysis, and transformation within the Spark and Glue environments.
- Data Pipeline Optimization : Ensure that all data workflows are optimized for performance, scalability , and cost-efficiency on the AWS Cloud platform.
- Collaboration : Work closely with data analysts , data scientists , and other engineering teams to create reliable data solutions that support business analytics and decision-making .
- Documentation Best Practices : Maintain clear documentation of processes, workflows, and code while adhering to best practices in data engineering , cloud architecture , and ETL design .
- Expertise in Apache Spark and PySpark for large-scale data processing and transformation.
- Hands-on experience with AWS Glue for building and managing ETL workflows in the cloud.
- Strong programming skills in Python , with experience in data manipulation, automation, and integration with Spark and Glue.
- In-depth knowledge of ETL principles and data pipeline design, including optimization techniques.
- Proficiency in working with AWS services , such as S3 , Glue , Lambda , and Redshift .
- Strong skills in writing optimized SQL queries , with a focus on performance tuning.
- Ability to translate complex business requirements into practical technical solutions .
- Familiarity with Apache Airflow for orchestrating data workflows.
- Knowledge of data warehousing concepts and cloud-native analytics tools.
Employment Type: Full Time, Permanent
Read full job descriptionPrepare for Data Engineering Lead roles with real interview advice
Top UST Data Engineering Lead Interview Questions
What people at UST are saying
What UST employees are saying about work life
UST Benefits
View more benefits
Compare UST with
Accenture
Wipro
Cognizant
Capgemini
Genpact
IBM
DXC Technology
Sutherland Global Services
Optum Global Solutions
FIS
Virtusa Consulting Services
CGI Group
GlobalLogic
Bosch Global Software Technologies
Eviden
Atos
Nagarro
NTT Data
Hewlett Packard Enterprise
Publicis Sapient
Data Engineering Lead - AWS Glue & PySpark Specialist
5-7 Yrs
Bangalore / Bengaluru
Lead II - Enterprise Solutions - SAP - Finance & Controlling
8-10 Yrs
Bangalore / Bengaluru
AS400 Lead developer
5-10 Yrs
Chennai, Bangalore / Bengaluru, Thiruvananthapuram
