15 Vision Excel Career Solutions Jobs
AWS Data Engineer - Python/Apache Airflow (6-12 yrs)
Vision Excel Career Solutions
posted 4mon ago
Key skills for the job
JD : AWS Data Engineer
Job location : Hyderabad
Must Have :
- Design and develop efficient data pipelines using Python and orchestrate workflows with Airflow, ensuring reliable data ingestion, transformation, and storage.
- Utilize Spark (Glue, EMR) for largescale data processing tasks, optimizing for performance and scalability.
- Implement and manage streaming data solutions with Kafka, managing high-volume data with proficiency in SQS and Event Bridge for even- driven architectures.
- Leverage AWS services extensively, including S3 for data storage, ensuring best practices in cloud-based data handling and security.
- Work closely with data architects, analysts, and other stakeholders, providing technical expertise and support in data-related projects and initiatives.
Good to Have :
- Knowledge of Data Warehousing (DW) concepts and Data Integration (DI) methodologies.
- Experience with data modelling and familiarity with ETL processes and tools.
- Experience in API integration, with a solid understanding of RESTful services.
- Developing and consuming APIs to facilitate seamless data exchange and system interoperability.
Qualifications :
- Bachelor's degree in computer science, Engineering, Data Science, or related field.
- 4-8 years of relevant experience in data engineering, with a proven track record of developing and managing data solutions.
- Strong proficiency in Python and experience with Airflow for data pipeline management.
- Expertise in Spark for big data processing, with experience in Glue and/or EMR.
- Demonstrated experience with Kafka for real-time data streaming and familiarity with AWS messaging services like SQS and Event Bridge.
- In-depth knowledge of AWS cloud services, especially S3, and best practices in cloud security and data management.
Functional Areas: Other
Read full job description