Data Engineer

Understanding of how to design technological solutions to complex data problems, developing testing modular, reusable, efficient and scalable code to implement those solutions
1. Understand, implement, and automate ETL pipelines with better industry standards
2. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, design infrastructure for greater scalability, etc.
3. Developing, integrating, testing, and maintaining existing and new applications
4. Design, and create data pipelines (data lake / data warehouses) for real world energy analytical solutions
5. Expert-level proficiency in Python (preferred) for automating everyday tasks
6. Strong understanding and experience in distributed computing frameworks, particularly Spark, Spark-SQL, Kafka, Spark Streaming, Hive, Azure Databricks etc
7. Limited experience in using other leading cloud platforms preferably Azure.
8. Hands on experience on Azure data factory, logic app, Analysis service, Azure blob storage etc.
9. Ability to work in a team in an agile setting, familiarity with JIRA and clear understanding of how Git works