Altysys - Data Engineering Manager - Python/Spark (8-12 yrs)

About Us :

Altysys is a dynamic and forward-thinking technology solutions company focused on leveraging data to drive business transformation. We specialize in creating scalable and efficient solutions for our clients, ensuring the best outcomes through cutting-edge technology and expertise.

Job Overview :

We are looking for a highly skilled and motivated Data Engineer Manager to join our team. The ideal candidate will have deep expertise in AWS, Python, GitHub, Spark, and SQL, and will be responsible for leading our data engineering team, building robust data pipelines, and ensuring seamless integration across our data architecture.

If you are passionate about data engineering and enjoy working in a collaborative, fast-paced environment, we'd love to hear from you.

Key Responsibilities :

- Lead and manage a team of data engineers, providing mentorship, guidance, and technical expertise.

- Design, develop, and optimize scalable and reliable data pipelines using AWS

- Implement data processing workflows using Apache Spark for big data transformation and analysis.

- Develop, maintain, and improve data integration processes using SQL and Python to ensure efficient data flow between systems.

- Collaborate with cross-functional teams to understand data requirements and deliver solutions that support business analytics, data science, and machine learning initiatives.

- Oversee code management and version control practices using GitHub, ensuring code quality and adherence to best practices.

- Monitor and troubleshoot data pipelines to ensure data quality, reliability, and performance.

- Maintain up-to-date knowledge of industry trends and best practices, ensuring that the data engineering team uses modern and efficient tools and technologies.

- Establish and enforce data governance and security protocols to ensure compliance with industry standards.

Technical Skills Required :

- AWS Cloud Services : Expertise in S3, Redshift, EMR, Glue, Lambda, and other relevant AWS tools.

- Python : Proficient in Python for data manipulation, ETL development, and integration with cloud services.

- Apache Spark : Strong experience with Spark for large-scale data processing and transformation.

- SQL : Advanced knowledge of SQL for data modeling, querying, and database management.

- GitHub : Hands-on experience with version control and collaborative development using GitHub.

Preferred Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.

- 5+ years of experience in data engineering with a focus on cloud-based solutions.

- Proven track record of managing and leading data engineering teams.

- Strong problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.

- Experience with DevOps practices, CI/CD pipelines, and data orchestration tools like Airflow is a plus.

What We Offer :

- Competitive salary and performance-based incentives.

- Comprehensive health and wellness benefits.

- Opportunities for career growth and professional development.

- A collaborative, inclusive, and dynamic work environment