The Family Office Company BSC - Senior Data Engineer - Data Pipeline (5-9 yrs)

Job Summary :

- Responsible for designing, developing, and maintaining data pipelines, integrating APIs, and managing data-driven solutions within our organization.

- Collaborating with cross-functional teams to ensure data availability, quality, and usability for analysis and business insights.

- Proficiency in Python programming and a strong willingness to learn and grow in a dynamic environment.

- Sense for ownership and drive for improvements.

Responsibilities :

- Design, develop, and optimize data pipelines and ETL processes using Azure Data Factory (ADF) and Azure Databricks to extract, transform, and load data from various sources into data platforms.

- Integrate with external APIs and build data connectors to retrieve and ingest data from third-party systems, ensuring data accuracy and reliability.

- Preparing SQL views and procedures for other teams in the company.

- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and ensure the availability of relevant datasets for analysis and reporting purposes.

- Implement and enforce data governance practices, including data quality, data privacy, and data security measures, following industry best practices and regulatory requirements.

- Troubleshoot and resolve data-related issues, performance bottlenecks, and data pipeline failures, ensuring minimal disruption to data workflows and business operations.

- Continuously monitor and optimize data processing and storage infrastructure for improved performance, scalability, and cost efficiency.

- Ownership of various projects across the engineering team and drive to delivery in time.

- Stay updated with the latest trends, technologies, and best practices in data engineering, Azure services, APIs, and data management to propose and implement innovative solutions.

Qualifications :

Work experience :

- Proven work experience as a Data Engineer or a similar role, focusing on data pipeline development, API integration, database design, and data management.

- Strong expertise in Azure cloud services, particularly Azure Data Factory (ADF), Azure SQL Database, Azure Blob Storage, Azure Databricks.

- Proficient in SQL and Python programming and experience working with relational databases, data warehousing, and data modeling concepts.

- Experience with API integration, including RESTful APIs, data extraction, and data ingestion from external systems.

- Solid understanding of ETL processes, data integration, and data transformation techniques.

- Proficiency in Python programming and experience with data manipulation, automation, and scripting tasks.

- Strong problem-solving and analytical skills, with the ability to identify and resolve complex data-related issues.

- Experience with leadership and guidance of small team.

- Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.

- A keen desire to learn and stay updated with emerging technologies, tools, and methodologies in the data engineering field.

Education :

- Minimum of bachelor's degree from a recognized university or equivalent experience.

Skills & Competencies Required :

- Creative thinking.

- Desire to work in an information system environment.

- Excellent communication (written and oral) and interpersonal skills.

- Excellent organizational, multi-tasking, and time-management skills.

Interpersonal skills :

- Proven ability to build, manage, and foster a team-oriented environment.

- Proven ability to work creatively and analytically in a problem-solving environment.

- Proven ability to work independently.