AmbitionBox
Search
Jobs
- Reviews
- Salaries
- Interview Questions
- About Company
- Benefits
- Jobs
- Office Photos
- Community
Add office photos
All Filters
7 Thinkproject Jobs
Senior Data Engineer
Thinkproject
posted 5mon ago
Job Role Insights
Key skills for the job
Job Description
- ETL Pipeline Development: Design, build, and maintain ETL pipelines using Matillion to extract data from SQL databases, transform it according to business rules, and load it into Snowflake.
- Data Transformation in Snowflake: Write and manage transformations directly within Snowflake using Snowflake Tasks, Streams, and Stored Procedures to handle specific business logic or real-time data requirements.
- Porting ETL to Snowflake & Spark: Migrate existing ETL pipelines from Matillion into Snowflake and Spark, utilizing the native features of both platforms to optimize data processing, storage, and analytics.
- Collaborate with Stakeholders: Work closely with data architects, business analysts, and other teams to understand data requirements and ensure that data pipelines are aligned with business needs.
- Performance Optimization: Continuously optimize ETL and data transformation processes in both Matillion and Snowflake for enhanced performance and reduced operational costs.
- Monitoring & Troubleshooting: Monitor the health of data pipelines and Snowflake transformations, troubleshooting issues, and implementing solutions to ensure data accuracy and availability.
- Data Preparation: Implement data cleansing, aggregation, and enrichment processes within Snowflake to prepare data for downstream reporting and analytics.
- Snowflake Management: Manage and optimize Snowflakes data architecture, ensuring best practices in data modeling, partitioning, and indexing to support analytics and reporting.
- Compliance & Governance: Ensure all ETL processes and transformations comply with data governance, privacy, and security requirements.
What you need to fulfill the role
- Proficiency in Matillion: Extensive experience in building ETL pipelines and transformations using Matillion, extracting data from SQL databases, and loading it into Snowflake.
- Expertise in Snowflake: Strong understanding of Snowflake architecture, including experience writing and managing Snowflake Data Warehouse, Snowflake Tasks, Streams, and Stored Procedures for real-time data processing and transformations. Additionally, experience optimizing Snowflake billing and credit consumption.
- SQL Databases: Proven experience working with SQL databases (e.g., MySQL, PostgreSQL, Microsoft SQL Server) and writing complex SQL queries.
- ETL Design & Optimization: Ability to design scalable ETL processes and optimize the existing ETL processes.
- Data Transformation: Experience in data cleansing, aggregation, and transformation both in Matillion and directly within Snowflake.
- Performance Tuning: In-depth knowledge of performance optimization techniques in both Matillion and Snowflake environments, ensuring efficient data processing.
- Cloud Platforms: Familiarity with Power BI and Azure is an added advantage.
- Programming Languages: Experience with languages such as Python or Java is a plus.
- Experience: 5-10 years
Employment Type: Full Time, Permanent
Read full job description What people at Thinkproject are saying
Thinkproject Benefits
View more benefits
Compare Thinkproject with
Accenture
Compare
Wipro
Compare
Infosys
Compare
Amazon
Compare
Capgemini
Compare
IBM
Compare
Tech Mahindra
Compare
Genpact
Compare
HDFC Bank
Compare
Teleperformance
Compare
Cognizant
Compare
WNS
Compare
LTIMindtree
Compare
iEnergizer
Compare
HCLTech
Compare
Bharti Airtel
Compare
Axis Bank
Compare
HDB Financial Services
Compare
Jio
Compare
TCS
Compare
Share an Interview