Data Ingestion Lead/Specialist - Apache NiFi (8-16 yrs)

Job locations : Bangalore/ Hyderabad/ Pune/ Greater Noida

Experience : 8-16 Yrs

We are seeking an experienced NiFi Data Ingestion Engineer to design, develop, and maintain scalable data ingestion pipelines using Apache NiFi. The successful candidate will work closely with data architects, engineers, and stakeholders to ensure seamless data flow from various sources to target systems. The role requires strong understanding of data integration, ETL processes, and NiFi architecture.

Responsibilities :

1. Design and Develop Data Ingestion Pipelines : Create and optimize NiFi data flows to ingest data from various sources (e.g., databases, files, APIs) into target systems (e.g., data warehouses, lakes, or processing engines).

2. Configure NiFi Components : Set up and configure NiFi components, including processors, controllers, and services, to ensure efficient data processing and movement.

3. Implement Data Transformation and Enrichment : Apply data transformation and enrichment techniques using NiFi's built-in processors and custom scripts (e.g., Groovy) to prepare data for target systems.

4. Integrate with Other Systems : Integrate NiFi with other systems, such as data warehouses, lakes, or processing engines, to ensure seamless data flow.

5. Monitor and Troubleshoot : Monitor NiFi data flows and troubleshoot issues, ensuring high availability and performance.

6. Collaborate with Stakeholders : Work closely with data architects, engineers, and stakeholders to understand requirements, design data ingestion pipelines, and ensure successful implementation.

7. Maintain and Update Existing Pipelines : Update and maintain existing NiFi data ingestion pipelines to ensure they remain efficient, scalable, and compliant with changing requirements.