106 Bay Area Technology Solutions Jobs
6-9 years
Bay Area Technology Solutions - Senior/Lead Engineer - Data Engineering (6-9 yrs)
Bay Area Technology Solutions
posted 2mon ago
Key skills for the job
Qualifications :
- 5+ years of experience in data engineering with significant expertise in Databricks and Apache Spark.
- Proficient in Unity Catalog for managing data lineage, security, and governance within the Databricks ecosystem.
- Experience of estimating and migrating legacy data warehouse workloads to Azure/Hybrid Cloud.
- Proficient in Unity Catalog for managing data lineage, security, and governance within the Databricks ecosystem.
- Experience building and optimizing ETL pipelines using tools like Azure Data Factory, Informatica, or similar.
- Strong understanding of CI/CD practices with experience in Git for version control and integration with Databricks.
- Expertise in SQL development and performance tuning for large-scale datasets.
- Knowledge of the Azure ecosystem, including data services like Azure Data Factory, Azure Data Lake and Azure Storage.
- Ability to work with both batch and streaming data processing pipelines.
- Experience with data modeling and dimensional design (e.g., star schema).
- Good understanding of data governance, compliance, and security best practices.
- Excellent communication and problem-solving skills, with the ability to manage multiple priorities.
- Ability to stay current on Databricks innovations and proactively introduce new features and capabilities to the team.
Functional Areas: Other
Read full job description6-9 Yrs
5-10 Yrs
Bangalore / Bengaluru
3-8 Yrs
Mumbai, Delhi/Ncr, Bangalore / Bengaluru