Lead Pyspark Developer

Work Experience: 8 to 13 Years

Work location: Bengaluru

Work Mode: Hybrid

Must Have Skills: Python, PySpark, Azure

Good to have Skills: Azure Databricks, PostgreSQL, Clinical Data

Job Overview:

Looking for senior developer in Big Data Processing Platform for Bangalore Location. The role involves design, development, enhanced software programs or cloud applications or proprietary products.

Technical Skills:

• Very good understanding of ETL and ELT methodology in Data Processing using cloud technologies, preferably expertise in Microsoft Azure.
• Extensive experience in Analysis, Design, Development, Implementation, Deployment and maintenance of Business Intelligence applications.
• Hands on expertise in MS Azure Databricks using python and pyspark.
• Hands on expertise in management of configuration based ETL development.
• Handling complex json, spark data frame, manipulation of key and values.
• Sound knowledge in secret management, file storage in Azure space.
• Expertise in any SQL Database, capable of writing complex sql for analysis.
• Automation in the area of Unit Testing, Deployment.
• Knowledge in Healthcare domain is added advantage.
• Knowledge in version control of code is added advantage.

Key Responsibilities:

• Minimum 10+ years of experience in IT industry; should have worked on a large-scale client implementation project.
• Minimum 6 years of experience in python and spark coding, must have handled complex json parsing.
• Must be capable of performing proof of concepts in the area of automation.
• Works collaboratively with architects and testers across regions.
• Reviews business requirements with regards to reporting, advises on the pros and cons of the possible technical solutions.
• Translates those business needs into technical specifications and provides input to the requirements for the development of a solution.
• Explores data the data to discover patterns, relationships, anomalies and trends.
• Writing relational and multi-dimensional database queries.
• Perform code reviews and optimization of existing queries.
• Comprehensively and coherently documents work in Confluence.
• Tracks and defines work within JIRA.

Competencies:

• Strong adaptability and problem-solving skills in an agile work setting.
• Extensive hands-on experience with key Azure services such as Databricks, Azure Function and SQL scripting.
• Ability to develop and implement effectively.
• Proven analytical and troubleshooting skills.
• Excellent written, Oral communication.

Qualifications: Graduate: B-Tech/ BE/ MCA