Design, develop, and maintain efficient, reliable,and scalable data pipelines using modern data engineer tools and bestpractices.
Build and optimise data flows between variousinternal and external data sources and destination Implement andmaintain data quality checks, monitoring, and alerting systems.
Write efficient and maintainable code followingbest practices and coding standards
Participate in code reviews and contribute totechnical documentation.
Troubleshoot and resolve data pipeline issues andperformance bottlenecks.
Required Technical Skills:
Strong proficiency in SQL and data modelling
Hands-on experience with Python or Javaprogramming.
Proven experience with Apache Spark forlarge-scale data processing.
Experience with Apache Airflow for workfloworchestration
Expertise in working with modern data platformsand warehouses, particularly Databricks and/or Snowflake (good to haveSnowflake / databricks certification).
Experience working with at least one major cloudprovider (AWS, GCP, or Azure).
Strong understanding of data engineering concepts,ETL processes, and data warehousing principles.