Coditas - Lead AWS Data Engineer - Python/Apache Spark (4-10 yrs)

Job Description :

We are looking for data engineers who have the right attitude, aptitude, skills, empathy, compassion, and hunger for learning.

Build products in the data analytics space. A passion for shipping high-quality data products, interest in the data products space; curiosity about the bigger picture of building a company, product development and its people.

Roles and Responsibilities :

- Understand Business requirements and implement scalable solutions.

- Translate complex technical and functional requirements into detailed designs.

- Develop a highly scalable, reliable, and high-performance data processing pipeline to extract, transform and load data from various systems to the Enterprise Data warehouse/Data Lake/Data Mesh.

- Provide research, high-level design and estimates for data transformation and data integration from source applications to end-use.

- Investigate alternatives for data storing and processing to ensure implementation of the most streamlined solutions.

- Create end to end data products and productionize them in cloud/in-house servers.

Technical Skills :

- Minimum 6-10 years of progressive experience building solutions in Big Data environments.

- Have a strong ability to build robust and resilient data pipelines which are scalable, fault tolerant and reliable in terms of data movement.

- Hands-on experience of Apache Spark with Python for batch and stream processing.

- Should have knowledge of experience in batch and stream data processing.

- Exposure working on projects across multiple domains.

- Minimum 6 months of hands-on experience in Apache Kafka.

- Strong hands on capabilities on SQL and NoSQL technologies.

- Minimum 3+ years of experience with AWS services like S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK etc is must have or similar services of either Azure/GCP.

- Strong analytical/quantitative skills and comfortable working with huge sets of data.

- Excellent written and verbal communication skills.