Apache NiFi Engineer - Configuration Management (5-8 yrs)

Role Description :

This full-time remote involves working with Apache Nifi to handle day-to-day tasks related to data flow management, system migrations, integrating systems, and monitoring data processes.

Join our dynamic team and be part of driving technological solutions to new heights with MLE Systems.

Project Overview :

- Support Apache NiFi, Establish workstream and preparation work to complete data transfers for legal day 1.

On Site Schedule :

- Fully remote OK, resources should be OK traveling to the US if needed.

Responsibilities :

- Design, implement, and manage data flows within Apache NiFi, ensuring efficient and reliable data movement across systems.

- Build scalable workflows for data ingestion, transformation, enrichment, and delivery.

- Configure processors, connections, and controllers to optimize data throughput and performance.

- Regularly monitor and maintain NiFi clusters, ensuring high availability and fault tolerance.

- Implement appropriate security measures, including user access control, data encryption, and SSL configurations within NiFi.

- Ensure compliance with organizational data governance policies and regulations.

- Work on integrating NiFi with other systems such as databases, cloud services (e., GCP, AWS, Azure) and other big data or messaging platforms.

- Ensure seamless connectivity between NiFi and external data sources or destinations.

- Implement monitoring tools and set up alerts to detect failures or performance issues.

- Generate reports on data flow metrics and performance for stakeholders.

- Stay up to date with the latest NiFi releases and patches, ensuring that the environment is using the latest stable versions.

- Optimize the performance of data transfer workflows by configuring NiFi's backpressure settings, load balancing, and ensuring high availability across NiFi clusters.

- Monitor file transfer and API performance, troubleshooting bottlenecks and optimizing workflows as needed for both throughput and latency.

- Collaborate with cross-functional teams, including data engineers, security teams, and IT infrastructure teams from both entities, to ensure smooth data flow and integration.

- Provide detailed documentation on data flow architecture, security protocols, and API configurations to ensure that the integration process is well-understood and maintainable by both organizations post-acquisition.

- Ensure that the data transfer mechanisms built are scalable and sustainable for long-term use.

- Support the ongoing data synchronization between systems while moving toward full integration of IT ecosystems.

Requirements :

- 5+ years of experience in Apache NiFi for managing data flows, file transfers, and API integrations.

- Knowledge of secure file transfer protocols (SFTP, FTPS, HTTPS) and encryption methods.

- Experience integrating systems using APIs and working with open systems (cloud, on-premise, hybrid).

- Strong understanding of data governance, compliance, and security practices, particularly in high-stakes M&A scenarios.

- Excellent troubleshooting and optimization skills for large-scale data migration and integration projects.

- Excellent English communication skills both written and verbal.

- Apache NiFi is an open-source data integration tool that allows users to design and manage data flows between systems, databases, and cloud services.

- It is a Java program that runs within a Java virtual machine on a server.