Lead Software Engineer - Python/Spark

As a Lead Software Engineer at JPMorgan Chase within the Asset and Wealth Management, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm s business objectives.

Job responsibilities

• Lead the design, development, and implementation of scalable data pipelines and ETL batches using Python/PySpark on AWS.
• Execute standard software solutions, design, development, and technical troubleshooting
• Use infrastructure as code to build applications to orchestrate and monitor data pipelines, create and manage on-demand compute resources on cloud programmatically, create frameworks to ingest and distribute data at scale.
• Add to team culture of diversity, equity, inclusion, and respect.
• Manage and mentor a team of data engineers, providing guidance and support to ensure successful product delivery and support.
• Collaborate proactively with stakeholders, users and technology teams to understand business/technical requirements and translate them into technical solutions.
• Optimize and maintain data infrastructure on cloud platform, ensuring scalability, reliability, and performance.
• Implement data governance and best practices to ensure data quality and compliance with organizational standards.
• Monitor and troubleshoot application and data pipelines, identifying and resolving issues in a timely manner.
• Stay up-to-date with emerging technologies and industry trends to drive innovation and continuous improvement.

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Demonstrable hands-on experience in Python and PySpark.
• Bachelors or Masters degree in Computer Science, Information Technology, or a related field.
• Proven experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Good understanding of data modeling, data architecture, ETL processes, and data warehousing concepts.
• Experience with big data technologies and services like AWS EMRs, Redshift, Lambda, S3.
• Excellent communication skills to work effectively with stakeholders, partner teams, and to translate technical concepts into business terms.
• Proven experience with efficient Cloud DevOps practices and CI/CD tools like Jenkins/Gitlab, for data engineering platforms.
• Good knowledge of SQL and NoSQL databases, including performance tuning and optimization.
• Strong analytical skills to troubleshoot issues and optimize data processes, working independently and collaboratively.
• Experience in leading and managing a team of engineers, with a proven track record of successful project delivery.

• Knowledge of machine learning concepts, language models and cloud-native MLOps pipelines and frameworks is a big plus.
• AWS Certifications in data engineering and machine learning is a plus.
• Familiarity with data visualization tools and data integrations.