Senior Data Engineer - Databricks Specialist

As a Senior Data Engineer at Quantiphi, you will solve big data problems and will develop solutions for migration, storage, and processing

Must have skills:

• 3+ years of hands-on experience in delivering data solutions
• Experience with Databricks implementations on AWS/Azure - developing data pipelines using PySpark
• Experience in Databricks Workspaces,Notebooks, Delta Lake, APIs
• Experience in Python, Scala
• Experience in advanced SQL and Query optimization
• Experience in implementing data integration projects using ETL
• Experience building and deploying cloud-based solutions at scale by ingesting data from sources like DB2.

Good to have skills :

• Experience with AWS services - S3,Redshift,Secrets Manager
• Experience in implementing data integration projects using ETL (preferably Qlik Replicate & QlikCompose)
• Experience in using Airflow or Step Functions for orchestration
• Exposure to IaC tools like Terraform and to CI/CD tools
• Prior experience in migrating (on-prem to cloud) and processing large amounts of data
• Experience in implementing data lake and data warehouse on cloud
• Experience in implementing the industry's best practices

For more details visit our website or our LinkedIn Page