Lead Data Engineer

• Take responsibility for modernization, migration/transformation to a cloud data platform
• Design and develop reliable data infrastructure with leading privacy and security techniques to safeguard data using AWS technologies
• Architect secure, low latency, resilient and cost-effective solutions for enabling predictive analytics across the organization
• Design and architect frameworks to operationalize ML models through serverless architecture and support unsupervised continuous training models
• Work with the team to scale the company s data models (Tableau, Dynamo DB, Kibana)
• Communicate data-backed findings to a diverse community of internal and external stakeholders
• Build frameworks for data ingestion pipeline in both real-time and batch using best practices in data modeling
• Participate in technical decisions and collaborate with talented peers
• Review code and give meaningful feedback that assists others in building highly scalable solutions
• Drive tech direction and choices of technologies by making recommendations based on experience and research

Job Requirements:

• Bachelor s/Master s degree in Engineering, Computer Science (or equivalent experience)
• At least 10+ years of relevant experience as a data engineer
• Solid understanding of ETL/ELT processes
• Hands-on experience working in a public cloud environment and on-prem infrastructure
• Experience in Columnar Databases like Redshift Spectrum, Time Series data stores like Apache Pinot, and the AWS cloud infrastructure
• Experience with in-memory, serverless, streaming technologies and orchestration tools such as Spark, Kafka, Airflow, Kubernetes
• Current hands-on experience in IT platform implementation
• Nice to have experience in AWS Certified Big Data
• Experience designing and implementing AWS big data and analytics solutions in large digital and retail environments is desirable
• Advanced knowledge and experience in online transactional (OLTP) processing and analytical processing (OLAP) databases, data lakes, and schemas
• Experience with AWS Cloud Data Lake Technologies and operational expertise with Kinesis/Kafka, S3, Glue, and Athena
• Experience with any of the message/file formats: Parquet, Avro, ORC
• Design and development experience on a streaming service, EMS, MQ, Java, XSD, File Adapter, and ESB-based applications
• Experience in distributed architectures such as Microservices, SOA, RESTful APIs, and data integration architectures
• Hands-on experience migrating On-Prem data solutions to the cloud
• Prior experience managing On-prem Enterprise Data Warehouse solutions like Netezza is a plus
• Expert in commonly used AWS services (S3, Lambda, Redshift, Glue, EC2, etc.)
• Expertise in Python, PySpark, or similar programming languages
• Experience with BI tools (Tableau, Domo, MicroStrategy) is a plus
• Expertise in AWS Compute such as EC2, Lambda, Beanstalk, or ECS
• Experience in AWS Management and Governance suite of products such as CloudTrail, CloudWatch, or Systems Manager
• Prolific experience in Amazon Web Services (AWS) offerings, development, and networking platforms
• Experience with AWS Analytics such as Athena, EMR, or Glue
• Proficiency in Oracle, MYSQL, and Microsoft SQL Server databases is a plus
• Solid understanding of continuous integration / continuous delivery
• Experience in Jenkins
• Fluent in verbal and written English