Big Data Engineer - ETL/Python (5-6 yrs)

We are looking for a Big Data Engineer to build scalable data pipelines and support enterprise analytics applications by integrating and processing large datasets.

Roles and Responsibilities :

- Develop and manage data pipelines for integrating, consolidating, and cleansing data.

- Utilize ETL tools and REST APIs for data integration and management.

- Plan, document scope, and estimate work effort for data-driven solutions.

- Develop and debug complex big data solutions, ensuring best practices.

- Build reusable frameworks to enhance data processing efficiency.

- Provide technical guidance for issue resolution, project assessments, and reporting.

- Develop standard data analytics services to identify risk reduction opportunities.

Skills and Qualifications Required :

- Experience in identity & access management, infrastructure technology, or security event monitoring.

- Bachelor's Degree in Software Engineering, Computer Science, Mathematics, or equivalent experience.

- Proficiency in Scala, Python, and SQL.

- Hands-on experience with big data technologies: Spark, Iceberg, HBase, NiFi, Kafka, Hadoop, HDFS, Avro, Trino, and Starburst.

- Strong knowledge of NoSQL technologies (Solr, Elasticsearch) and MPP stores (Greenplum)