Showcase strong analytical and problem-solving capabilities by consulting with clients and other stakeholders to understand business problems.
Responsible for designing, storing, processing, and maintaining of large-scale data and related infrastructure.
Is a developer who is responsible to build data pipelines, implement data quality, build data model, monitor, and optimize data pipelines to minimize runtime, and adhere to customer and MathCo. set standards.
Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing.
Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
Understanding of Medallion architecture pattern.
Has worked on at least one cloud platform.
Has executed multiple end-to-end data engineering projects.
Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.
Required Tech Stack
Strong proficiency in SQL.
Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP.
Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.).
Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java).
Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables).
Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc.
Has worked on code and SQL query optimization.
Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery.
Required Non-Tech Stack
Understanding of code optimization and security best practices.
Strong problem-solving skills.
Able to analyze problems and quickly develop effective and actionable strategies.
Proficient in written and verbal communication.
Exercises independent judgment to determine the best course of action within established practices and procedures.