CGI - Azure Databricks Engineer - Python Programming (2-10 yrs)

Job Description :

Location : Bangalore, Chennai

Years of experience : 2 to 10 years

Role : Software Engineer / Senior Software Engineer / Lead Analyst

Looking for experienced AWS / Azure Developer / Lead who is very proficient in Databricks, adept at architecting and deploying scalable serverless applications. Specialized in real-time data processing, data warehousing, and API development, with a strong focus on security, optimization, and collaborative problem-solving.

Responsibilities :

- Design, develop, and deploy scalable and secure applications on the AWS / Azure platform.

- Utilize AWS / Azure services to implement robust solutions.

- Architect and implement serverless applications using AWS Lambda for efficient and cost-effective computing.

- Implement real-time data processing solutions using AWS for streaming and analytics.

- Design, implement, and optimize data warehousing solutions using Amazon Redshift for large-scale data storage and analytics.

- Implement event-driven architectures using AWS services to enable asynchronous communication between microservices.

- Build and optimize ETL processes to transform and load data into Redshift for reporting and analysis.

- Ensure the security and compliance of AWS applications, implementing best practices for data protection and access control.

- Identify and implement optimizations for performance, scalability, and cost efficiency in AWS environments.

- Collaborate with cross-functional teams, including software developers, system administrators, and data engineers, to deliver integrated solutions.

- Create and maintain technical documentation for AWS architectures, configurations, and processes.

- Implement CI/CD pipelines for automated testing, deployment, and continuous integration of AWS applications.

- Troubleshoot and resolve issues related to AWS services, ensuring timely resolution of production incidents.

- Strong skills in designing, implementing, and optimizing data warehousing solutions.