Lead Data Engineer - Amazon Athena (5-10 yrs)

Qualifications :

- 5+ years experience delivering data platforms at scale

- 5+ years experience architecting, building and releasing production-grade data solutions within a SaaS environment

- Demonstrated, hands-on mastery of the AWS suite of data engineering tools including Athena, Glue, Kinesis, RedShift, S3 etc.

- Extensive experience with DBT for the development of data processing pipelines

- Experience with Infrastructure as Code (IaC) using Terraform to define and deploy infrastructure related to data solutions on AWS

- Experience with Apache Iceberg, Apache Hudi or Delta Lake

- Deep experience building batch and data streaming architectures

- Ability to assert and defend opinions on the merits of differing solutions

- Ability to clearly explain technical concepts to non-technical audiences

- Familiarity with a variety of third-party data engineering tools and common architectures

- Excellent communication skills necessary to manage business requests and work with different teams across different geographies and time-zones; experience working with remote and distributed teams will be an added advantage

- Hands-on working knowledge and experience is required in:

- ELT Processes (DBT, Spark, etc.)

- Relational Databases (PostgreSQL, MySQL, SQL, PLSQL, etc.)

- NoSQL Databases (Redis, Elastic Search, Mongo, etc.)

- Architecture patterns such as Lambda/Kappa/Delta

- Programming Languages (Python, Scala, Spark, etc.)

- Streaming Technologies (Kafka, Kinesis, etc.)

- Agile Methodologies (Scrum, TDD, BDD, etc.)

- Cloud Platforms (AWS, Azure, GCP), preferably AWS

- Development of Complex Big Data Solutions like Spark, RedShift, EMR, etc. and Open-Source Platforms like Hudi, etc.

- Experience with several of the following tools/technologies is desirable:

a. Amazon QuickSight

b. Git, Jira, GitHub, Jupyter Notebooks

b. Data Structures and Machine Learning Models

- Knowledge of the following technologies is a plus:

a. Unix/Linux environments

b. Continuous Integration and Continuous Delivery Tools like GitHub Actions, etc.

c. Containerization Technologies like ECS, Docker and Kubernetes