1 Celeritas AI Job
Senior Data Engineer - Java/Scala/Python (7-8 yrs)
Celeritas AI
posted 29d ago
Flexible timing
Key skills for the job
Job Description :
- Spark development with batch and streaming experience
- Experience with Pyspark and Data Engineering
- ETL implementation and migration to spark
- Programming knowledge on scala, java & python
- Experience with kafka and Spark streaming (Dstream and Structured Streaming)
- Experience with using Jupyter notebooks or any other developer tool
- Flink and Kudu streaming
- Informatica workflow migration
- Automation of workflows CI/CD
- Nifi streaming and transformation
- Experience with Airflow or other workflow engines.
- Experience in BFSI industry, using cloud Technologies(Azure/AWS/Google Cloud)
- 7-8 years using PySpark, Apache Spark, Flink, Nifi, Kafka & other Data Engineering Technologies.
- Experience in SQL, RDBMS, Hadoop Technologies.
- Experience with Cloudera Tech Stack, like Hive, Impala, HDFS and related tools.
Functional Areas: Software/Testing/Networking
Read full job description