Particle Black - Senior Data Engineer - Spark/Python (8-12 yrs)

Role : Senior Data Engineer

Exp. Range : 8-12 yrs

We are seeking a seasoned Data Engineer with 8+ years of experience in IT Services, exhibiting proven expertise in managing complex data processes, frameworks, and end-to-end project delivery. The ideal candidate will bring strong technical acumen, accountability, and leadership skills, ensuring the successful completion of key data engineering projects on time and to specification.

Key Responsibilities :

- Take ownership of assigned work and timelines, demonstrating proactive problem-solving, effective task management, and leadership in guiding teams through project milestones.

- Develop accurate project estimates and detailed plans, breaking down work for the team while setting clear expectations.

- Design, implement, and manage data pipelines, data transformations, and optimized ETL processes, applying in-depth knowledge of complex SQL queries, data engineering principles, and metadata-driven frameworks.

- Execute tasks across the full software lifecycle, from coding and unit testing to pull requests and code reviews, ensuring high quality deliverables.

- Performance and load testing will be integral to ensuring that all data processes meet scalability and efficiency benchmarks.

- Conduct functional and regression testing to validate that the code aligns with business requirements, supporting overall system stability.

- Provide technical consulting, troubleshooting, and support for production environments, ensuring data integrity and minimal downtime.

- Lead within Agile frameworks, driving continuous integration and delivery (CI/CD) practices to ensure rapid, iterative development and deployment.

Required Skills & Certifications :

- Mastery of Azure Synapse Spark, Python, complex SQL, and data file processing (Parquet, Delta) for optimized data handling and storage solutions.

- Demonstrated experience with metadata-driven frameworks, Agile methodologies, and technical documentation best practices.

- Proficiency with Azure DevOps for source control, pipeline automation, and release management.

- Expertise in ETL optimization and performance tuning, ensuring data workflows are efficient and scalable.

About company :

Software company helping government agencies achieve better outcomes through artificial intelligence and the strategic use of data. Our solutions engineered for health and human services programs deliver enhanced efficiency and improve the lives of citizens. Together with our agency customers, we achieve remarkable outcomes for families.

Our built-for-government suite of case management solutions employ configurable modules, cloud-based tech, a powerful AI assistant, an intuitive interface, and data-driven workflows on a low-code platform that enables agencies to modernize faster, and with greater confidence, than custom development projects

Educational Qualifications :

- Undergraduate Degree : B.Tech/B.E. or BCA in any specialization.

- Postgraduate Degree : M.Tech or MCA in any specialization.

This is a full-time, permanent position within our Engineering - Software & QA Department. The role is ideal for a candidate

passionate about data engineering who thrives on technical challenges and delivering impactful solutions within IT Services and Consulting.