Architect - Data Engineering

Responsibilities

• Work with product owners, scrum masters and technical committee to define the 3 months road-map for each program increment (sprint wise)
• Manage and scale data pipelines responsible for ingestion and data transformation.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Prototype new approaches and build solutions at scale.
• Research in state-of-the-art methodologies.
• Create documentation for learnings and knowledge transfer.
• Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.
• Responsible for sustainability of already live pipelines in production environment.

• Bachelors degree in Computer Science, MIS, Business Management, or related field
• 8 + years experience in Information Technology
• 4 + years of Azure, AWS and Cloud technologies
• Good written and verbal communication skills along with collaboration and listening skills
• Experience dealing with multiple vendors as necessary.
• Hands on experience of writing complex SQL queries
• Big Data (Hadoop, HBase, MapReduce, Hive, HDFS etc.), Spark/PySpark
• Sound skills and hands on experience with Azure Data Lake, Azure Data Factory, Azure Data Bricks ,Azure Synapse Analytics, Azure Storage Explorer
• Proficient in creating Data Factory pipelines for on-cloud ETL processing; copy activity, custom Azure development etc.
• Knowledge on language such as Python, Scala

• Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
• Experience with data profiling and data quality tools
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.

Mandatory Non-Technical Skills

• Excellent remote collaboration skills
• Experience working in a matrix organization with diverse priorities.
• Enthusiast for learning functional knowledge specific to finance business
• Ability to work with virtual teams (remote work locations); within team of technical resources (employees and contractors) based in multiple global locations.
• Participate in technical discussions, driving clarity of complex issues/requirements to build robust solutions
• Strong communication skills to meet with delivery teams and business-facing teams, understand sometimes ambiguous, needs, and translate to clear, aligned requirements.