Lead II - DevOps Engineering

Job Summary We are seeking a skilled and motivated DevOps Engineer with a strong background in AWS big-data solutions and expertise in implementing CI/CD pipelines using Harness.io. As a key member of our dynamic team, you will play a crucial role in designing, developing, and maintaining robust, scalable, and secure data pipelines and infrastructure for our big-data applications. Accountabilities AWS Big-Data Solutions: Collaborate with cross-functional teams to design, deploy, and manage AWS-based big-data solutions, including data storage, processing, and analytics services. Leverage AWS services such as Amazon S3, Amazon EMR, Amazon Redshift, and AWS Glue to architect efficient and scalable data workflows. Harness.io Implementation: Lead the adoption and utilization of Harness.io for the continuous integration and continuous deployment (CI/CD) pipelines. Design, configure, and automate CI/CD workflows to streamline the development, testing, and deployment processes of big-data applications. Security Validation: Integrate robust security practices into the CI/CD pipelines and build/release processes. Implement security checks, vulnerability scanning, and compliance validation to ensure data privacy, integrity, and protection at every stage of the pipeline. Infrastructure as Code (IaC): Champion the IaC approach for managing infrastructure resources. Use tools like AWS CloudFormation or Terraform to provision and manage AWS resources, ensuring consistency and reproducibility. Automation and Orchestration: Drive automation initiatives to increase operational efficiency. Automate repetitive tasks, infrastructure provisioning, and configuration management using scripting languages and tools like Ansible. Collaboration and Knowledge Sharing: Foster a culture of collaboration, knowledge sharing, and continuous improvement within the DevOps and broader engineering teams. Mentor junior team members and participate in peer code reviews. Best Practices and Innovation: Stay up-to-date with the latest trends, tools, and technologies in the AWS and big-data domain. Introduce innovative solutions and best practices to enhance the performance, reliability, and security of our data infrastructure. Basic Qualifications: Bachelors degree in Computer Science, Engineering, or related field (or equivalent experience). Proven experience in designing and deploying AWS big-data solutions, leveraging services like S3 and Glue. Hands-on expertise in implementing CI/CD pipelines using Harness.io or similar tools for big-data applications. Strong knowledge of security principles and experience integrating security checks into CI/CD pipelines (eg Sonarqube, Checkmarx). Proficiency in infrastructure automation using tools like AWS CloudFormation, Terraform, or similar. Solid scripting skills in languages such as Python, Bash, or PowerShell. Experience with IaC, configuration management tools (e.g., Ansible), and version control systems (e.g., Bit Bucket). Strong problem-solving skills, ability to troubleshoot complex issues, and an eye for detail. Excellent communication and teamwork abilities, with a focus on fostering a collaborative and inclusive work environment.