i
VAYUZ Technologies
54 VAYUZ Technologies Jobs
VAYUZ Technologies - Lead Data Engineer (15-18 yrs)
VAYUZ Technologies
posted 11d ago
Flexible timing
Key skills for the job
Responsibilities and accountabilities :
Data Collection and Integration :
- Collect data from various sources, including databases, APIs, external data providers, NoSQL, unstructured data and streaming sources. Design and implement efficient data pipelines to ensure a smooth flow of information into the data warehouse or storage system.
Data Storage and Management :
- Choose appropriate database systems, optimize data schemas, and ensure data quality and integrity. Consider scalability and performance to handle large volumes of data.
ETL (Extract, Transform, Load) Processes :
- Design ETL pipelines to transform raw data into a format suitable for analysis. This involves data cleansing, aggregation, and enrichment, ensuring the data is usable.
Big Data Technologies and Cloud Computing :
- Big data and OLAP technologies such as Google Cloud Big Query, Hadoop and Spark to efficiently process and analyze massive datasets.
Enterprise Data Governance :
- Define and ensure adherence to the Enterprise Data Governance framework for data practices, policies, and standards both at a technical and a functional level. Work closely with various data owners to develop and implement Data Governance Framework.
Distributed Systems :
- Handle huge data volumes and ensure fault tolerance. Understanding how distributed systems work is essential for data engineers.
Streaming Data (CDC) :
- Work with streaming technologies like Qlik Replicate, Informatica CDC, Apache Kafka to handle and analyze data as it flows in.
Qualification and experience :
- 15+ years of hands-on experience in the field of Data Integration and Governance; out of which at least 10 years of leading a team of Data Engineers and Data Governance specialists.
- Previously held roles leading Data Engineering teams for dynamic, growing organizations.
- Natural enthusiasm to keep up to date on emerging technologies and methodologies to continually improve and advance data solutions where possible.
- Knowledge and experience in building end-to-end Data Pipelines.
- Knowledge and experience in building Enterprise Data Governance Lifecycle based on the Maturity Model.
- Experience in implementing Data Availability, Data Quality, Data Catalogue, Data Lineage & Data Security initiatives across the Enterprise. Having worked in a regulated industry would be a plus.
- Experience in defining and enforcement of Data Engineering and Governance principles and policies across the Enterprise.
- Knowledge of Relational and NoSQL databases. Hands-on experience in T-SQL / PL-SQL programming.
- Bachelor's degree in IT, Computer Science, or a related field; Master's degree will be a plus.
Functional Areas: Software/Testing/Networking
Read full job descriptionPrepare for Lead Data Engineer roles with real interview advice