Senior Software Engineer RR/1311/2024

Exp: 5 to 12 Years

Job Description:

Please consider Python as ETL tool to read data from source perform all datatype & data transformation, error handling ,logging within python and load to data bricks managed delta tables.
Anyone we are hiring should be very good at python along with very good knowledge in Databricks & PySpark so that they can evaluate and perform transformation within Databricks as well. Candidate should be able to do almost everything in python including class ,objects, methods, dictionaries ,loops, Wheel files ,packages, should be able to connect to any source data including any DBs, streaming data.
Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake

The ideal candidate has extensive experience in data engineering, working with large-scale data infrastructures, and optimizing ETL processes
Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
At least 6+ years of experience in data engineering, with a focus on building data pipelines
Proven experience with Spark, Scala, and other related technologies
Familiarity with Could Warehouses, data lakes, storage formats.
Strong knowledge of data modeling and ETL processes
Experience leading data engineering projects.
Proven experience with Databricks and Apache Spark
Strong proficiency in Python
Experience in Unix shell scripting and scheduling tools