Data Engineer - Python Programming (5-6 yrs)

Requirements :

Programming Languages :

- Proficiency in Python for automating processes and supporting services.

- Shell scripting for task automation and system-level troubleshooting.

ETL Tools :

- Experience with ETL tools like Informatica Cloud.

- Ability to manage and optimize data extraction, transformation, and loading (ETL) processes.

SQL and Data Querying :

- Strong SQL skills for querying data, creating custom checks, and monitoring services.

- Experience managing and monitoring data pipelines in various environments (upstream, batch, streaming).

- Familiarity with pipeline orchestration tools like Luigi or Airflow.

- Experience providing operational support, including troubleshooting and resolving incidents.

- Knowledge of cloud platforms like AWS, GCP, or Azure.

- Experience with OpenShift and Kubernetes environments.

- Proficiency in using Prometheus, Grafana, or similar tools for monitoring service health.

- Ability to set up monitoring dashboards and perform anomaly detection.

- Knowledge of user access management, including account setup and permissions validation.

- Ability to understand and document data flows and dependencies between different data services.

- Familiarity with DevOps principles and Agile environments.

- Proficiency in using Google Docs and Jira for documentation and incident tracking.

- Strong communication skills for working with cross-functional teams.

- Experience in incident management and root cause analysis.

- Ability to identify and fix common failure points.

- Experience with Streaming Platforms,

- Knowledge of Apache Kafka and experience with streaming services.

- Awareness of data governance principles for maintaining data quality and compliance.