Databricks Engineer - PySpark/Scala (3-5 yrs)

We are seeking a highly skilled Databricks Consultant to work remotely with one of our esteemed clients in New York City.

In this role, you will leverage your expertise in Databricks and data engineering to build scalable solutions, optimize data pipelines, and enable advanced analytics capabilities for the client.

About thinkbridge.

thinkbridge is how growth-stage companies can finally turn into tech disruptors.

- They get a new way there - with world-class technology strategy, development, maintenance, and data science all in one place.

- But solving technology problems like these involves a lot more than code.

- That's why we encourage think'ers to spend 80% of their time thinking through solutions and 20% coding them.

- With an average client tenure of 4+ years, you won't be hopping from project to project here - unless you want to.

- So, you really can get to know your clients and understand their challenges on a deeper level.

- At thinkbridge, you can expand your knowledge during work hours specifically reserved for learning.

- Or even transition to a completely different role in the organization.

- It's all about challenging yourself while you challenge small thinking.

thinkbridge is a place where you can :

What is expected of you ?

- As part of the job, you will be required to.

- Design and implement data pipelines using Databricks to process and transform large-scale datasets.

- Collaborate with the client's technical teams to gather requirements and create customized data solutions.

- Optimize Databricks clusters for performance, reliability, and cost efficiency.

- Implement ETL processes to ensure data quality, security, and accuracy.

- Provide guidance on best practices for leveraging Databricks for analytics and data engineering.

- Troubleshoot and resolve issues in data workflows and platform configurations.

- Stay updated with the latest Databricks features and industry trends to recommend innovative solutions.

Requirement :

Must-Have :

- Bachelor's degree in computer science, Data Engineering, or a related field.

- 3+ years of experience in data engineering and pipeline development.

- Expertise in Databricks, Spark, and PySpark.

- Hands-on experience with cloud platforms such as AWS, Azure, or Google Cloud.

- Proficiency in programming languages such as Python or Scala.

- Strong knowledge of ETL processes, data integration, and data warehousing concepts.

- Exceptional problem-solving and analytical skills.

- Strong communication skills to effectively collaborate with cross-functional teams.

Good to Have :

- Databricks certifications (e.g , Databricks Certified Data Engineer Associate or Professional).

- Experience with data visualization tools such as Tableau or Power BI.

- Familiarity with data governance frameworks and security best practices.

Other Details :

- 3-6 months Contract (Renewable based on performance).

- Working hours will follow the EST time zone.