RxDataScience - Data Engineer II - Scala/Python (3-5 yrs)

Job Description :

A Data Engineer is responsible for developing and maintaining data processing software like databases. Able to work with Data Analysts and other professionals to create unique data infrastructure, run tests on their designs to isolate errors, and update systems to accommodate changes in company needs.

Duties :

- Assembling large, complex sets of data that meet non-functional and functional business requirements.

- Identifying, designing, and implementing internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.

- Building required infrastructure for optimal extraction, transformation, and loading of data from various data sources using Azure and SQL technologies.

- Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics, including operational efficiency and customer acquisition.

- Working with stakeholders including data, design, product, and executive teams and assisting them with data-related technical issues.

- Supporting data infrastructure needs while assisting with data-related technical issues.

Qualifications :

- Bachelor's degree in computer science, Engineering, Mathematics, or a related field.

- 3+ years of experience in a Data Engineer role.

- Experience with big data technologies and databases.

- Experience with Azure cloud services ADF, AML, Data Lakes, Data Briks

- Experience with object-oriented/object function scripting languages Python, Scala, KDB_/qSQL etc.

- Experience with data pipeline and workflow management tools: Azure Data Factory.

- Experience with relational SQL and NoSQL databases, including pySpark, DataBriks and KDB+/qSQL

- Experience with data modeling, ETL development, and data warehousing.

- Experience with Agile or Scrum software development methodologies.

- Understanding of Clinical research data is preferable