Sr Data Engineer/Kapil/Veera/8 To 10 Years

Job Summary:

We are seeking a highly skilled and experienced Databricks Engineer to lead the development and implementation of advanced data solutions. The successful candidate will play a pivotal role in our data engineering team, designing and managing data pipelines and leveraging tools like Databricks, Delta Live Tables, Snowflake, and the Medallion Architecture.

Data Pipeline Design and Development:

• Architect, build, and maintain Medallion Architecture-based data pipelines using Databricks.
• Handle raw data ingestion using Delta Live Tables, ensuring data quality and low access latency.

Data Management:

• Manage complex data and metadata changes within the Databricks environment.
• Utilize expertise in Snowflake to provide insights into data warehousing and management practices.

Collaboration and Implementation:

• Work closely with data architects to align Databricks solutions with organizational objectives.
• Collaborate with data science teams to implement complex statistical models.

Security and Compliance:

• Ensure data security and regulatory compliance in Databricks and Snowflake environments.

Automation and Performance:

• Develop and deploy ETL workflows emphasizing automation and scalability.
• Optimize performance, troubleshoot issues, and debug within the Databricks platform.

Continuous Learning:

• Stay updated on industry best practices and emerging trends in Databricks technologies.

• Education:
• Bachelor s degree in Computer Science, Information Technology, or a related field.
• Experience and Expertise:
• Minimum 8 years of experience in Databricks platform and data engineering.
• Expertise in Delta Live Tables and implementing Medallion Architecture in a data lake environment.
• Proficiency in Snowflake for data warehousing.
• Strong understanding of ETL workflows and data pipeline automation.
• Deep knowledge of data structures, data modeling, and database schemas.
• Technical Skills:
• Highly proficient in SQL, Python, or Scala.
• Experienced in performance tuning, troubleshooting, and debugging in Databricks.
• Soft Skills:
• Outstanding problem-solving abilities with an innovative approach to complex challenges.
• Exceptional communication skills, with the ability to bridge gaps between technical and non-technical stakeholders.
• Proactive, collaborative, and passionate about data technologies.