Databricks Administrator

Key Responsibilities :

Implement, setup, configure, monitor, and maintain various Databricks solutions including :

- Databricks access control & Provide Databricks User access, define roles

- Databricks Cost Management

- Databricks Performance Management.

- Databricks Backup (IaC based)

- Databricks SQL

- Provide Databricks User access, define roles

- Provision of ec2 instance and maintenance

- Databricks Cluster setup and management

- Capacity management and monitoring

- Setting up security perimeter(using Privecera) and mange user access un alignment to current security policies

- Provide On Demand technical Support to project team

- Triage and resolve issue faced by project team and work with Corning AWS Infrastructure team towards resolution

- Scheduled Report Monitoring & Back up Monitoring

- Performance Management

- Supervise Clusters Health check

- Disaster Recovery Deploy and maintain PaaS & SaaS Databricks platform for Machine Learning (ML) development, QA, staging, and production environments hosted on cloud infrastructure.

- Develop Infrastructure as Code(IaC) to automate databricks environment deployments.

- Develop a library of scripts to automate databricks MLOps environments.

- Identify and generate solutions to current problems in the organization.

- Aid developers and DevOps teams with implementing automation scripts that follow the company standards.

- Collaborating with DevOps team members as well as all of engineering Ensure that solutions exhibit high levels of

performance, security, scalability, maintainability, appropriate reusability, and reliability upon deployment.

- Partner with Cloud solution architect and development team to fully understand product requirements and configure the

- Databricks environments which meet these requirements.

- Manage Databricks infrastructure and collaborate with various teams to ensure it integrates seamlessly with all components including network, security, messaging, and remote access.

- Troubleshoot and resolve Databricks environment performance issues, connectivity issues, security issues.

- Research, adopt and relate new technologies to solve issues that arise while adhering to company security and standards.

- Solid experience in the administration and performance tuning of Databricks is required for success at this position.

- Solid scripting skills is required for success at this position.

- Experience in working with formal Change Control Systems.

- Mentor, coach, and share knowledge with other team members throughout the organization.

Other Responsibilities (If Any) :

- Availability during month-end Deck generation, may be sometime during week-end/holidays.

Eligibility Criteria for the Job :

Education : B.E/B.Tech in any specialization, BCA, M.Tech in any specialization, MCA

Age : No Bar

Work Experience :

- 3-4+ years of experience in Implement, setup, configure, monitor, and maintain various Microsoft AWS Databricks solutions

Primary Skill :

- Must have orchestrated reporting platform on cloud

- Databricks Admin & Application

Technical Skills :

- Configuring and installing libraries, connection with RDS Glue and other AWS components

- Configuring modifying underlying EC2 instances on which Databricks runs Databricks hands on administration

- Handling user request around systems access, Spark, data manipulation, data frames etc AWS administration EC2, IAM roles, appstream, bastion hosts, S3Active Directory Adding users, creating groups, integrating with Okta

Management Skills :

- Ability to handle given tasks and projects simultaneously in an organized and timely manner.

- Daily project updates and should be able identify the project risks & mitigations.

- Manage fortnightly updates for the Sr. Management of both the sides

Soft Skills :

- Good communication skills, verbal and written.

- Attention to details.

- Positive attitude and confident