Develop, construct, test and maintain optimal data pipeline/ETL architectures
Map out data systems/objects and define/design required integration, ETL, BI AI systems/processes
Work closely within the team to prepare data for predictive and prescriptive modeling
Optimize AWS and Azure data delivery infrastructure for greater scalability
Utilize SQL as well as big data tools and frameworks to optimize data acquisition and preparation from enterprise data lake and data warehouse
Work with Cloud Architecture teams to strive for greater functionality in our data systems
Develop architecture required to return data to data warehouse for front-end product utilization
Curate data models in the data warehouse to be used by front-end advanced analytics designers
Provide production level code reviews for the team
Design, maintain and implement quality assurance and testing approaches
Deploy scripts and architectures to production via Jenkins
Skills/Experience required
Bachelor s Degree in Computer Science, Mathematics, Engineering, Management Information Systems, or related field
5+ years of experience working with enterprise data platforms, building, and managing data lakes and using data technologies
3-5 years of experience designing and implementing data warehouse solutions
2+ years of experience with Spark using Python/Scala. Experience with Spark streaming, building real time data pipelines is preferred
2+ years of experience working with AWS and or Azure platform. Experience with solutioning on AWS infrastructure using services like AWS S3, Lambda, EMR, Redshift (or Snowflake)
Experience with automating and orchestrating jobs on a big data platform using Oozie, Airflow, Jenkins or something similar
Good understanding and experience working with various products in the Big data ecosystem like Hive, HDFS, Presto, NoSQL databases like Cassandra, DynamoDB
Experience with setting up and using Kafka for real time streaming is a big plus
Prior experience with working in a SQL server-based environment and using SSIS, SSRS, TSQL is a plus.
Prior experience with traditional ETL tools like Talend Open Studio, Pentaho, or something similar is a plus
Advanced SQL and data design concepts
Drive innovation and efficiency through new approaches
Ability to work in a team environment that promotes collaboration