Senior Data Engineer - SQL/Python (5-9 yrs)

Job Description :

The data engineering role requires creating and managing technical solutions on the modern data stack (Azure, Snowflake, fivetran), architecting, building, and managing data flows/pipelines and construct data models within Snowflake to be used for other downstream processes or for analysis purposes.

Role Responsibility :

- Design, develop, and maintain scalable data pipelines and analytics solutions

- Collaborate with stakeholders to gather requirements and translate business needs into technical solutions

- Develop efficient code with unit testing and code documentation

- Ensuring accuracy and integrity of data and applications through analysis, coding, documenting, testing, and problem solving

- Optimize and fine-tune data models, SQL queries, and transformations for performance and scalability

- Design, develop, and maintain scalable data models and transformations in conjunction with Snowflake, ensure the effective transformation and load data from diverse sources into data warehouse or data lake.

- Integrate FiveTran connectors to streamline data ingestion from various sources into Snowflake

- Develop custom Python scripts and functions to automate data workflows and enhance system capabilities

- Collaborate with data engineers to automate and orchestrate data pipelines.

- Provide technical expertise and support to resolve data-related issues and troubleshoot system failures

- Collaborate with the API development team to integrate data pipelines with external systems and applications

- Implement techniques for query optimization, caching, and workload management.

- Perform regular audits and troubleshooting to maintain system performance.

- Implement and manage data security measures, including access controls, encryption, and data masking.

- Ensure compliance with data governance policies and regulatory requirements.

- Communicate with all the project stakeholders on the project status

- Lead and participate in initiatives to enhance data platform capabilities and performance.

- Stay updated on emerging trends and technologies in data engineering, cloud computing, and analytics domains

Role Requirements :

- Bachelor's degree in Computer Science, Engineering, or a related field

- Proven experience of at least 5 years as a data engineer, ETL developer, or similar role, with a focus on Python and Snowflake

- Strong proficiency in SQL and database concepts, with hands-on experience in Snowflake data warehouse

- Strong proficiency in programming languages such as Python for data manipulation, automation, and scripting.

- Expertise in Azure durable functions

- Knowledgeable in relational databases, nonrelational databases, data pipelines (ELT/ETL), and file stores

- Knowledgeable in performance tuning and optimization

- Experience with cloud platforms like Azure and tools like Azure OCR for data extraction and processing

- Familiarity with data integration tools like FiveTran

- Knowledge of API development principles and experience integrating data pipelines with external systems

- Familiarity with CI/CD pipelines and version control systems (e.g., Git).

- Proficient in written, verbal and presentation communication (English)

- Ability to work effectively in a fast-paced, collaborative environment and manage multiple priorities

- Excellent analytical, problem-solving, and communication skills

- Ability to work effectively in a fast-paced, collaborative environment and manage multiple priorities

- Snowflake certification (SnowPro Core, SnowPro Advanced)