Senior Data Engineer - SQL/Python (7-10 yrs)

Job Description :

Technical Skills :

- 5+ years of experience in SQL, Python,.Net, SSIS, SSAS

- 2+ years of experience in Azure cloud services esp - SQL Server, ADF, Azure Databricks, ADLS, Key vault, Azure Functions, Logic Apps Etc. Highlighting Databricks.

- 2+ years of experience in using GIT and deploying code using CI/CD approach.

Soft Skills :

- Strong analytical and problem-solving skills.

- Excellent communication and interpersonal skills.

- Ability to work independently and as part of a team.

- Attention to detail and a commitment to quality.

Responsibilities :

- Create and manage scalable data pipelines to collect, process, and store large volumes of data from various sources

- Data Integration : Integrate data from multiple data sources, ensuring data consistency, quality, and reliability.

- Database Management : Design, implement, and optimize database schemas and structures to support data storage and retrieval.

- ETL Processes : Develop and maintain ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently moved between systems.

- Data Warehousing : Build and maintain data warehouses to support business intelligence and analytics needs.

- Performance Optimization : Optimize data processing and storage performance, ensuring efficient use of resources and quick data retrieval.

- Documentation : Create and maintain comprehensive documentation for data pipelines, ETL processes, and database schemas.

- Monitoring and Troubleshooting : Monitor data pipelines and systems for performance and reliability, troubleshooting and resolving any issues that arise.

- Technology Evaluation : Stay up-to-date with emerging technologies and best practices in data engineering, evaluating and recommending new tools and technologies as appropriate.