Senior Data Engineer

Role & responsibilities

• Language: Proficiency in Java or Python or Scala. Prior experience in Oil & Gas, Titles & Leases, or Financial Services is a must have.
• Databases: Expertise in relational and NoSQL databases like PostgreSQL, MongoDB, Redis, and Elasticsearch.
• Data Pipelines: Strong experience in designing and implementing ETL/ELT pipelines for large datasets.
• Tools: Hands-on experience with Databricks, Spark, and cloud platforms is a must have
• Data Lakehouse: Expertise in data modeling, designing Data Lakehouses, and building data pipelines.
• Modern Data Stack: Familiarity with modern data stack and data governance practices.
• Data Orchestration: Proficient in data orchestration and workflow tools.
• Data Modeling: Proficient in modeling and building data architectures for high-throughput environments.
• Stream Processing: Extensive experience with stream processing technologies such as Apache Kafka.
• Distributed Systems: Strong understanding of distributed systems, scalability, and availability.
• DevOps: Familiarity with DevOps practices, continuous integration, and continuous deployment (CI/CD).
• Problem-Solving: Strong problem-solving skills with a focus on scalable data infrastructure.

Key Responsibilities:

• This is a player-coach role, with high expectations of hands on design and development.
• Design and development of systems for ingestion, persistence, consumption, ETL/ELT, versioning for different data types e.g. relational, document, geospatial, graph, timeseries etc. in transactional and analytical patterns.
• Drive the development of applications related to data extraction, especially from formats like TIFF, PDF, and others, including OCR and data classification/categorization.
• Analyze and improve the efficiency, scalability, and reliability of our data infrastructure.
• Assist in the design and implementation of robust ETL/ELT pipelines for processing large volumes of data.
• Collaborate with cross-functional scrum teams to respond quickly and effectively to business needs.
• Work closely with data scientists and analysts to define data requirements and develop comprehensive data solutions.
• Implement data quality checks and monitoring to ensure data integrity and reliability across all systems.
• Develop and maintain data models, schemas, and documentation to support data-driven decision-making.
• Manage and scale data infrastructure on cloud platforms, leveraging cloud-native tools and services.

Preferred candidate profile

Perks and benefits