Polaris Smart Metering - Data Engineer - AWS/Spark (1-8 yrs)

Job Description :

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will have extensive experience with AWS, MapReduce, PostgreSQL, NoSQL databases, Spark, Kafka, and the design of data lakes and data warehouses. The primary responsibility of this role is to design and implement data pipelines and build data visualizations to support our meter data management system.

Key Responsibilities :

- Design, develop, and maintain scalable and efficient data pipelines using AWS services, Spark, and Kafka.

- Create and manage data lakes and data warehouses to support large-scale data storage and retrieval.

- Implement data processing solutions using MapReduce and Spark to handle large volumes of meter data.

- Optimize and manage PostgreSQL and NoSQL databases to ensure high performance and reliability.

- Develop and implement data integration solutions to ingest and process data from various sources.

- Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions.

- Design and build data visualizations and dashboards to provide actionable insights for stakeholders.

- Ensure data quality, integrity, and security in accordance with industry best practices and regulatory requirements.

- Monitor and troubleshoot data pipelines and systems to ensure continuous data flow and minimal downtime.