Principal Data Engineer

Principal Data Engineer

Experience: 15+ Years
Salary : INR 2,00,000-2,50,000 / year
Preferred Notice Period: Within 30 Days
Shift: 10:00 AM to 6:00 PM IST
Opportunity Type: Remote
Placement Type: Contractual
Contract Duration: Full-Time, 06 Months

(*Note: This is a requirement for one of Uplers' Partners)

What do you need for this opportunity
Must have skills required :
BigQuery, Cloud Functions, Cloud run, Dataform, Dataflow, Kubernetes, Airflow, Dataproc
Good to have skills :
Datadog, Docker, PubSub, DataOps principles, Grafana, Data Mesh architecture

Our Hiring Partner is Looking for:
Principal Data Engineer who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you.

Role Overview Description
Principal Data Engineer
Role Overview
We are seeking a dynamic and highly skilled Principal Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game. The ideal candidate will contribute significantly to platform development with diverse skillset while also being very hands-on coding and actively shaping the future of our data ecosystem.

What We Are Looking For:

Responsibilities:
● As a principal engineer, you will be responsible for ideation, architecture, design and development of new enterprise data platform. You will collaborate with other cloud and security architects to ensure seamless alignment within our overarching technology strategy.
● Architect and design core components with a microservices architecture, abstracting platform, and infrastructure intricacies.
● Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
● Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.
● Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.
● Architect and design the best security patterns and practices.
● Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
● Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
● Design and develop advanced analytics and machine learning capabilities on the data platform.
● Design and develop observability and data governance frameworks and practices.
● Stay up to date with the latest data engineering trends, technologies, and best practices.
● Drive the deployment and release cycles, ensuring a robust and scalable platform.

Requirements:
● 15+ of proven experience in modern cloud data engineering, broader data landscape experience and exposure and solid software engineering experience.
● Prior experience architecting and building successful enterprise scale data platforms in a green field environment is a must.
● Proficiency in building end to end data platforms and data services in GCP is a must.
● Proficiency in tools and technologies: BigQuery, Cloud Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, Airflow, PubSub.
● Experience with Microservices architectures - Kubernetes, Docker and Cloud Run.
● Experience building Symantec layers.
● Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.
● Solid experience with architecting and implementing metadata management including data catalogues, data lineage, data quality and data observability for big data workflows.
● Hands-on experience with GCP ecosystem and data lakehouse architectures.
● Strong understanding of data modeling, data architecture, and data governance principles.
● Excellent experience with DataOps principles and test automation.
● Excellent experience with observability tooling: Grafana, Datadog.

Nice to have:
● Experience with Data Mesh architecture.
● Experience building Semantic layers for data platforms.
● Experience building scalable IoT architectures.

How to apply for this opportunity

1. Register or login on our portal
2. Click 'Apply,' upload your resume and fill in the required details.
3. Post this click ‘Apply Now' to submit your application.
4. Get matched and crack a quick interview with our hiring partner.
5. Land your global dream job and get your exciting career started!

About Our Hiring Partner:
Edstem Technologies is a dynamic and fast-growing software company, with service offerings in custom software development, mobile app development, managed services, offshore augmentation, and software consulting. We translate the ideas and requirements of our clients into smart solutions in cutting-edge technologies, by employing agile tools and practices while involving them in every step of development. Our delighted clientele spanning the globe in multiple domains is a testimony both to the quality of our deliverables and to the technical prowess of our highly skilled team. We have a talent pool that is well-equipped to work on the latest technology and highly demanding projects, positioning us also as a key player in supplying technology manpower for clients across the globe.

About Uplers:
Our goal is to make hiring reliable, simple, and fast. Our role will be to help all our talents find and apply for relevant opportunities and progress in their career. We will support any grievances or challenges you may face during the engagement. You will also be assigned to a dedicated Talent Success Coach during the engagement.

(Note: There are many more opportunities apart from this on the portal. Depending on the assessments you clear, you can apply for them as well).

So, if you are ready for a new challenge, a great work environment, and an opportunity to take your career to the next level, don't hesitate to apply today. We are waiting for you!