Data Engineer with Azure and Palantir Foundry Experience

Job Summary (Immediate Hiring):

We are seeking a highly skilled Data Engineer to join our team within weeks and design, develop, and optimize robust backend pipelines and intuitive frontend solutions. The role requires expertise in Azure cloud services, Palantir Foundry, and modern development frameworks. You will play a pivotal role in building scalable data systems and delivering actionable insights through seamless integration of backend and frontend platforms.In this role, you will be ingesting, creating, and maintaining data pipelines end-to-end and Ontology objects, ensuring data integrity, and optimizing data workflows. You will also be creating and maintaining Pipelines and functions utilizing TypeScript.

Key Responsibilities:

• Develop and maintain scalable ETL pipelines for ingesting and transforming large datasets using Azure Data Factory, Databricks, and Palantir Foundry.
• Implement efficient data models, workflows, and integrations within Palantir Foundry.
• Create, deploy, and monitor Azure cloud services, including Data Lake, Blob Storage, and Azure Functions.
• Design, implement, and maintain scalable Functions
• Collaborate with cross-functional teams to understand data requirements and translate
• them into robust and efficient solutions.
• Develop and optimize ETL processes to ensure timely and accurate data delivery to various
• business applications.
• Implement data governance and security best practices to safeguard sensitive
• information.
• Troubleshoot and resolve issues related to data pipelines, ensuring continuous data availability and reliability.
• Collaborate with data scientists and analysts to facilitate data-driven decision-making processes.
• Collaborate with stakeholders to build frontend solutions that visualize data insights effectively.
• Optimize the performance of backend pipelines and ensure seamless frontend integration.
• Implement data security best practices, ensuring compliance with industry standards.
• Debug and resolve technical issues related to data workflows and visualization tools.
• Stay updated on advancements in Azure cloud services, Palantir Foundry, and data engineering best practices.

Qualifications:

Technical Skills:

• Proficiency in Azure services: Azure Data Factory, Azure Data Lake, Blob Storage, and Azure Functions.
• Extensive experience with Palantir Foundry, including pipeline creation and data workflow optimization.
• Solid understanding of backend development using Python, Scala, or Java.
• Experience with frontend frameworks like React.js, Angular, or Vue.js.
• Strong knowledge of database technologies: SQL, NoSQL (e.g., CosmosDB, MongoDB).
• Familiarity with CI/CD tools like Azure DevOps or Jenkins.
• Tools and skills you will use in this role:
• Azure Functions

1. Palantir Foundry
2. SQL
3. Python
4. PySpark
5. JavaScript
6. TypeScript
7. Version Control Git and Git Hub

• Hands-on experience with containerization tools like Docker and orchestration platforms like Kubernetes.

Soft Skills:

• Strong problem-solving and analytical abilities.
• Excellent communication and teamwork skills.
• Ability to work in a fast-paced, collaborative environment.

Preferred Qualifications:

• The ideal candidate should have a strong background in cloud technologies, data architecture, and a passion for solving complex data challenges
• Knowledge of machine learning pipelines and AI integration.
• Bachelor's Degree with 6 years experience; or Master's Degree with 4 years experience
• 6+ years of experience in data engineering, preferably within the pharmaceutical or life
• sciences industry
• Proven experience as a Data Engineer with a focus on cloud-based solutions.
• Strong proficiency in SQL, Python, and PySpark
• Hands-on experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google
• Cloud Dataflow).
• Expertise in data modeling, database design, and optimization.
• Familiarity with Big Data technologies (e.g., Apache Spark, Hadoop) is a plus.
• Excellent problem-solving and analytical skills.
• Effective communication and collaboration skills.
• Previous experience working with JavaScript and TypeScript
• Experience in data visualization tools like Power BI, Tableau, or Palantir Foundrys built-in visualization features.
• Experience in Agile/Scrum methodologies.

What We Offer:

• Competitive salary.
• Opportunities to work with cutting-edge technology.
• Flexible working arrangements and a dynamic team culture.
• Continuous learning and career development.