Hiring For Data Developer - Pyspark

Project Role : Data Developer - Pyspark (Azure Databricks)

Project Role Description: As azure data developer the person will be responsible for Design and develop new solutions on the Azure Cloud Platform specifically for Azure Databricks, Azure Data Factory, Analysis Services, Azure Data Lake Store / Blob.

Work Experience :5 to 10 years IT Experience

Work location: Pune

Graduate or Post-Graduate in Computer Science/ Information Technology/Engineering.

Must Have Skills: At least 5 years strong professional experience of development in Azure Data Factory (ADF), Azure Databricks(pyspark), SQL, DW/Data Lake Design, Data Modeling, Data Analysis, Data Quality.

Also, must have Telecom Industry Experience.

Job Requirements:

• Hands-on experience working in complex data warehouse implementations using Azure Databrick, PySpark, Azure SQL Data warehouse, Azure Data Factory ETL and Azure SQL Database.
• Experience in building and delivering data marts, data warehouses, data lake and delta lake solutions.
• Development of data ingestion pipelines and robust ETL frameworks.
• Hands-on with end-to-end Azure Data brick implementation and good understanding with ADLS Gen2 as well as blob storage account.
• Must understand Data bricks command line interface with full automation
• Design, build, test, and support the project using Databricks, ADF, Azure Data Lake, PySpark
• In depth experience in writing complex SQL queries using joins, aggregates, set operators, window functions etc.
• Strong hands-on experience in analysing data on large datasets.
• Extensive experience in Dimensional data modelling including complex entity relationships and historical data entities
• Implementation of data cleansing and data quality features in ETL pipelines.
• Experience in documenting low level ETL designs
• Implementation of data streaming solutions
• The person Should be flexible to work independently or in a team
• Good analytical and problem-solving skills with Platform tasks such as creation and testing of data bricks implementation

Mandatory Technical Skills:

- Extensive Data Engineering experience using Python and PySpark Skills

- Understanding of Spark architecture

- Spark SQL and Performance Tuning

- Develop Databricks Notebooks and create Delta Lake tables with optimization techniques

- Delta Lake

- Spark Structured Streaming

- Hands on experience developing modern data platforms using

• Azure Databricks
• Azure Data Factory
• Azure Data Lake
• Azure DevOps

- Hands on experience parsing responses generated by API's (REST/XML/JSON)

- Experience in using CI/CD techniques, DevOps/Git repository code promotion and release/change management