11 N Human Resources & Management Systems Jobs

Responsibilities :

Data Pipeline Development :

- Design, develop, and maintain efficient and scalable ETL/ELT processes using Python, PySpark, and Azure Data Factory.

- Implement data integration solutions to ingest data from various sources into Azure Data Lake and data warehouses.

- Build and manage data orchestration workflows using tools like Azure Data Factory or similar.

Cloud Data Management (Azure) :

- Develop and manage data solutions on Azure, including Azure Databricks, Azure Synapse Analytics, Azure SQL Database, and Azure Data Lake Storage.

- Implement data modeling and data warehousing solutions using Snowflake or similar technologies.

- Migrate on-premises data to cloud-based solutions on Azure.

Data Processing and Analysis :

- Utilize Apache Spark and PySpark for large-scale data processing and analysis.

- Implement real-time data processing solutions using relevant frameworks.

- Ensure data quality and consistency throughout the data lifecycle.

Data Architecture and Design :

- Contribute to the design and implementation of data lakes and data warehouses to support operational and business intelligence.

- Apply data modeling principles to create efficient and effective data structures.

Collaboration and Communication :

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.

- Document data processes and solutions.

- Communicate effectively with both technical and non-technical audiences.

Required Skills & Experience :

- Minimum 3 years of experience in a Data Engineering role.

- Strong proficiency in Python, SQL, and Apache Spark/PySpark.

- Extensive experience with Azure cloud services, including Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure SQL.

- Experience in data modeling and data warehousing (Snowflake or similar).

- Proven experience in developing ETL/ELT processes.

- Experience with data orchestration and workflow management.

- Experience migrating data from on-premises to cloud (Azure).

- Strong understanding of data integration principles.

- Excellent written and verbal communication skills