Azure Data Engineer

Company Name: Credenca Data Solutions Pvt. Ltd.

Position: Azure Data Engineer

Experience Bucket: 3-5 Years

Description of position:

Credenca Data Solutions Pvt Ltd is looking for an experienced Azure Data Engineer with minimum 3 to 5 years experience. The candidate should have extensive hands-on experience or knowledge of ADF, Azure Databricks, Synapse,fabric,Azure Data Lake Gen 2, Azure SQL Server.

Desired skills and experiences

• Proven experience in assembling large, complex sets of data that meets non-functional and
functional business requirements

• Good exposure in working with Azure data bricks Pyspark, Spark SQL, Scala (Lazy evaluation
and delta tables, Parquet formats, Working with large and complex datasets)

• Experience/Knowledge in ETL, data pipelines, data flow techniques using Azure Data Services

• Leverage Databricks features such as Delta Lake, Unity Catalog, and advanced Spark
configurations for efficient data management.

• Debug Spark jobs, analyze performance issues, and implement optimizations to Ensure
pipelines meet SLAs for latency and throughput.

• Implement data partitioning, caching, and clustering strategies for performance tuning.

• Good understanding about SQL, Databases, NO-SQL DBs, Data Warehouse, Hadoop and various
data storage options on the cloud.

• Develop and manage CI/CD pipelines for deploying Databricks notebooks and jobs using tools
like Azure DevOps, Git, or Jenkins for version control and automation.

• Experience in development projects as Data Architect

• Must Need Skills- Data factory, Databricks, Databricks Architecture , Synapse, Py Spark / Python
and SparkAzure DB: Azure SQL / Cosmos DB

• Integrate data validation frameworks like Great Expectations & Implement data quality checks
to ensure reliability and consistency.

• Build and maintain a Lakehouse architecture in ADLS / Databricks.

• Manage access controls and ensure compliance with data governance policies using Unity
Catalog and Role-Based Access Control (RBAC).

• Experience integrating different data sources.

• Good Experiences in Snowflake, added advantage.

• Experience in supporting BI and Data Science teams in consuming the data in a secure and
governed manner

• Create and maintain comprehensive documentation for data processes, procedures, and
architecture designs

Work Location: Bengaluru,Mumbai,Pune

Requirment Fulfilment: Immediate to 15days