Senior Data Engineer (5-8 yrs)

Job Description :

Responsibilities :

- Contribute, as part of a team, in designing, testing, and implementing sophisticated data pipeline technologies to extract and transform data from source systems

- Perform data modeling on target systems to store data and support analytic querying

- Communicate with other development teams to gather and document requirements

- Work in Agile methodology and participate in design and review meetings

- Follow best practices for the software development life cycle including coding standards, reviews, source management, build, and testing

- Collaborate with other engineers in the team to implement best practices around large-scale data processing

Requirements :

- 5+ years of experience as a Data Engineer or in a similar role working with large data sets and ETL processes

- 7+ years of industry experience in software development

- Knowledge and practical use of a wide variety of RDBMS technologies such as MySQL, Postgres, SQL Server, or Oracle

- Strong SQL experience with an emphasis on analytic queries and performance

- Experience with various "NoSQL" technologies such as MongoDB or Elasticsearch

- Familiarity with either native database or external change-data-capture technologies

- Practical use of various data formats such as CSV, XML, JSON, and Parquet

- Use of Data flow and transformation tools such as Apache Nifi or Talend. PDI experience is desirable

- Implementation of ETL processes in languages such as Java, Python or NodeJS

- Use of large shared data stores such as Amazon S3 or Hadoop File System

- Thorough and practical use of various Data Warehouse data schemas (Snowflake, Star, OLAP cube)

- Use of cloud-based data warehouse technologies including Snowflake, and AWS RedShift.