Data Architect

• Implement scalable processes for large-scale data processing
• Develop robust data processing pipelines for various scenarios such as bulk Ingestion, real-time streaming applications, and ML processes
• Develop data pipelines that provide low latency and are highly scalable in a complex environment
• Collaborate with data scientists/product owners and be accountable for data processing needs from development to production
• Evaluate and note down processes that improve data quality, data discoverability, data lineage, and data governance
• Express technical and business risks to those concerned
• Adapt to changing business conditions and new information quickly
• Actively influence the design and shape of the ML roadmap within this scope
• Develop strong awareness of organizational health
• Identify and make recommendations to solve complex issues
• Deliver constructive feedback to team members

Job Requirements:

• Bachelor s/Master s degree in Engineering, Computer Science (or equivalent experience)
• At least 4+ years of relevant experience as a software engineer
• Demonstrated ability to ship complex large-scale data processing systems with low latency
• Prolific experience with Azure Databricks
• Demonstrable experience in building and developing ETL code in Python, PySpark, etc
• Prior experience with SQL and NoSQL to process data, preferably in cloud databases such as Snowflake, Redshift, etc.
• Knowledge of data processing testing frameworks such as Great Expectations
• Extensive experience working with Docker and Linux (preferably in Kubernetes)
• Hands-on experience with data pipeline orchestration tools such as Airflow, Dagster, Prefect, etc.
• Working experience in Agile and test-driven methodologies
• Strong verbal and written communication skills
• Analytical skills and passion for answering questions with data
• Fluent in verbal and written English