Senior Data Engineer

Roles & Responsibilities

Job Summary We are looking for a highly skilled Senior Data Engineer with a minimum of five years of experience to join our data team. The ideal candidate will have extensive experience in creating data pipelines, managing database servers, and working with cloud databases. Proficiency in data transformation and data modeling is essential, along with hands-on experience with Salesforce, AWS, Snowflake, MS SQL Server, DBT and specific technologies such as Apache Iceberg, data lakes, and lake houses.


Responsibilities

• Design and Implement Data Pipelines: Develop, maintain, and optimize batch and real-time data pipelines to ensure efficient data flow and processing.
• Data Transformation: Create and manage ETL/ELT processes to extract, transform, and load data from various sources, ensuring data quality and integrity.
• Database Management: Oversee and optimize database servers, including MS SQL Server and Snowflake, ensuring high availability and performance.
• Cloud Infrastructure: Deploy and manage data solutions on cloud platforms such as Snowflake, AWS ensuring scalability and security.
• Data Lake and Lakehouse Management: Implement and maintain data lakes and lake house architectures, utilizing Apache Iceberg for efficient data storage and querying.
• Salesforce Integration: Integrate data from Salesforce and other systems into data lakes and Snowflake for comprehensive analytics and reporting.
• Collaboration: Work closely with data scientists, analysts, and business stakeholders to gather requirements and ensure data solutions meet business needs.
• Monitoring and Troubleshooting: Monitor data pipelines and systems for performance issues, troubleshoot problems, and implement solutions to enhance efficiency.
• Documentation and Best Practices: Document data processes, maintain data dictionaries, and establish best practices for data management and governance.

Requirements

Must have hands-on experience in Snowflake, DBT, AWS, Salesforce (highly preferable), Tableau or any other BI tool, SQL Server

Must have hands-on experience in Data Modeling, Data Analysis and transformation, Design and development of OLAP Data Warehousing

• Good Education background - Bachelors degree in Computer Science, Engineering, or a related field.

• Minimum of 3 years of experience in data engineering or a similar role.
• Proficient in designing and implementing data pipelines using tools like Apache Airflow or AWS Data Pipeline.
• Strong SQL skills and experience with data transformation using Python, Spark, or similar technologies.
• Experience with data lakes and lake house architectures, particularly using Apache Iceberg.
• Proven ability to integrate data from Salesforce and other cloud systems.
• Strong analytical and problem-solving skills, with attention to detail.
• Excellent communication skills and the ability to work collaboratively in a team environment.

Preferred Skills

• Familiarity with data modeling and dimensional modeling techniques.
• Knowledge of data warehousing concepts.
• Experience with data visualization tools such as Tableau, Power BI, or Looker.
• Understanding of data security best practices and compliance requirements.
• Experience with containerization and orchestration tools like Docker and Kubernetes.

If you are passionate about data engineering and meet the qualifications outlined above, we encourage you to apply for this exciting opportunity to join our team and contribute to our data-driven initiatives!

Interested Candidates share CV at careers@rcgdigitalsystems.com