Data Engineer

Primary Job Functions 

• Responsible for full development cycle of Data Engineering & Warehousing concepts, including requirements gathering, architecture, design, implementation, and maintenance.
• Identify and resolve data issues to ensure the quality and consistency of data; determine and implement opportunities for automation.
• Continuously develops expertise in the data and data models within the data warehouse and an understanding of supported business domains.
• Communicate effectively, both orally and written, to varied levels of the organization to include technical personnel, business managers, and senior leadership, regarding data inquiries and request.
• Conducts and may facilitate internal testing & user acceptance testing.
• Develop technical specifications and design documents.
• Ensure that quality processes are carried out through all phases of the Development Lifecycle.
• Design and implement efficient and effective functional design solutions.
• Work closely with the Business and Business Intelligence team to ensure requirements are met.
• Build robust data pipelines and integrate with multiple components and data sources.
• Build & maintain highly scalable and secure data environments.
• Design and architect new product features, champion the use of cutting-edge technologies and mentor the team in the adoption of these new technologies.
• Collaborate with the Data Science team on complex machine learning models to ensure that data processing, structure and accessibility maximizes model performance.
• Applies domain technical expertise when providing solutions to the business and its operations.

Required Qualifications

• Bachelor's Degree in computer science, Information Technology, Management Information Systems, or related field, Required. Technical MS Preferred.
• 5 Years of professional experience, in data engineering/data warehousing enterprise or related experience, Required. 
• Advanced skills with data warehouse architectures, dimensional models, star schema designs and/or in-memory/columnar databases
• SQL & T-SQL skills
• Proficient in programming languages such as Spark, Spark SQL, R, Python, Java, and Scala 
• Knowledge of Delta Lake
• Proficient in: SQL Server syntax, Microsoft/Azure services like Data Factory, & Azure Analysis Services (Tabular). Familiarity with associated syntaxes (DAX, MDX)
• Proficient with Microsoft Azure Cloud services in design and management, monitoring, security, and privacy of data using the full stack of Azure data services
• Strong experience with Database Management Systems (DBMS), specifically SQL and NoSQL.
• Demonstrated success working within Azure Synapse Analytics and Azure Blob Storage
• Working experience with API / RESTful data services
• Proficient with big data technologies such as Spark, Databricks, Hadoop, Hive,  etc.
• Knowledge of popular data discovery, analytics and BI (Business Intelligence) software tools like Power BI for semantic-layer-based data discovery
• Knowledge with containerization techniques such as Docker, Kubernetes