Add office photos
All Filters
7 Thinkproject Jobs
Data Engineer
Thinkproject
posted 2mon ago
Job Role Insights
Flexible timing
Key skills for the job
Job Description
- ETL Pipeline Development: Design, build, and maintain ETL pipelines using Matillion to extract data from SQL databases, transform it according to business rules, and load it into Snowflake
- Data Transformation in Snowflake: Write and manage transformations directly within Snowflake using Snowflake Tasks, Streams, and Stored Procedures to handle specific business logic or real-time data requirements
- Porting ETL to Snowflake & Spark: Migrate existing ETL pipelines from Matillion into Snowflake and Spark, utilizing the native features of both platforms to optimize data processing, storage, and analytics
- Collaborate with Stakeholders: Work closely with data architects, business analysts, and other teams to understand data requirements and ensure that data pipelines are aligned with business needs
- Performance Optimization: Continuously optimize ETL and data transformation processes in both Matillion and Snowflake for enhanced performance and reduced operational costs
- Monitoring & Troubleshooting: Monitor the health of data pipelines and Snowflake transformations, troubleshooting issues, and implementing solutions to ensure data accuracy and availability
- Data Preparation: Implement data cleansing, aggregation, and enrichment processes within Snowflake to prepare data for downstream reporting and analytics
- Snowflake Management: Manage and optimize Snowflakes data architecture, ensuring best practices in data modeling, partitioning, and indexing to support analytics and reporting
- Compliance & Governance: Ensure all ETL processes and transformations comply with data governance, privacy, and security requirements
What you need to fulfill the role
- Expertise in Snowflake: Strong understanding of Snowflake architecture, including experience writing and managing Snowflake Data Warehouse, Snowflake Tasks, Streams, and Stored Procedures for real-time data processing and transformations
- Additionally, experience optimizing Snowflake billing and credit consumption
- SQL Databases: Proven experience working with SQL databases (eg, MySQL, PostgreSQL, Microsoft SQL Server) and writing complex SQL queries
- ETL Design & Optimization: Ability to design scalable ETL processes and optimize the existing ETL processes
- Data Transformation: Experience in data cleansing, aggregation, and transformation Performance Tuning: In-depth knowledge of performance optimization techniques, ensuring efficient data processing
- Experience: 3-5 years Good to have skills Proficiency in Matillion, Kafka and Spark are plus ML engineering or ML operations is preferred
- Cloud Platforms: Familiarity with Power BI and Azure is an added advantage
- Programming Languages: Experience with languages such as Python or Java is a plus
Employment Type: Full Time, Permanent
Read full job description What people at Thinkproject are saying
What Thinkproject employees are saying about work life
based on 1 employee
Flexible timing
Monday to Friday
No travel
Thinkproject Benefits
View more benefits
Compare Thinkproject with
Aconex
Compare
Procore
Compare
Autodesk
Compare
Trimble
Compare
Bentley Systems
Compare
Neumec Group
Compare
Asite Solutions
Compare
Hexagon AB
Compare
Accenture
Compare
Jio
Compare
IBM
Compare
Genpact
Compare
Capgemini
Compare
Cognizant
Compare
Infosys
Compare
Wipro
Compare
Tech Mahindra
Compare
Teleperformance
Compare
HDFC Bank
Compare
Larsen & Toubro Limited
Compare
Similar Jobs for you
Share an Interview