Senior Data Engineer - Python/PySpark (5-8 yrs)

Job Title: Senior Data Engineer (Azure):

No. of position: 2 Positions

Location : Offshore (Remote from India)

Need Immediate joiners - max 15days

- 5-8 years of experience in IT Industry

- 3+ years of experience with Azure Data Engineering Stack (Data Factory, Databricks, Event Hub, Cosmos DB, Synapse)

- 3+ years of experience with Python, Pyspark

- Excellent knowledge of SQL and coding skills

- Good understanding of other Azure services like Azure Data Lake Analytics , Azure SQL DW , Azure key Vault , Azure Storage and Azure Functions

- Good Understanding of Modern Data warehousing concepts

- Experience with scripting languages such as shell.

- Excellent analytical and organization skills.

- Effective working in a team as well as working independently.

- Experience of working in Agile delivery

- Knowledge of software development best practices.

- Strong written and verbal communication skills.

- Azure Data Engineer certification is added advantage

- Working knowledge of CI/CD tools and concepts

- ADF (ETL pipeline experience on using FTP, DB, API, Azure Databricks), Understanding of Integration Runtime, GIT, Understanding of lookup, CDC

- ADB - Performance Optimization, Integration

- PySpark Programming building transformation pipeline on source data in format of CSV, Complex JSON, Parquet, DELTA

- Spark Optimization and architecture

- Python programming - Algorithmic questions on string manipulation, collection operation

- SQL programming with strong knowledge of CTE, JOINS, Indexes