Sr. Data Engineer

Position: Senior Data Engineer

Experience: 4-7 years

Location: Remote

Employment Type: Full-Time

Overview: We are seeking a talented and motivated Senior Data Engineer to join our growing team. The ideal candidate will have extensive experience with data engineering tools such as Spark, Databricks, SQL, and DBT, along with proficiency in Python. You will be responsible for building scalable data pipelines, optimizing performance, and collaborating with various teams to deliver high-quality data solutions.

Responsibilities:

• Design, develop, and maintain robust ETL/ELT pipelines using Spark, Databricks, and DBT.
• Build and optimize data models for efficient querying and analysis.
• Collaborate with data scientists, analysts, and stakeholders to understand data needs and deliver scalable solutions.
• Write efficient and reusable Python scripts to automate data processes and workflows.
• Ensure data quality, reliability, and performance through testing and optimization.
• Monitor and troubleshoot data pipelines and address data issues in a timely manner.
• Contribute to code reviews, documentation, and best practice implementation across the team.

Minimum Qualifications:

• 4-6 years of experience in data engineering or a related field.
• Strong hands-on experience with Spark and Databricks for building and optimizing data pipelines.
• Proficiency in SQL for querying and manipulating large datasets.
• Experience with DBT for data modeling and transformation.
• Solid knowledge of Python for scripting and automation.
• Experience with data warehouse and big data environments.
• Strong problem-solving skills and ability to troubleshoot complex data engineering issues.
• Excellent communication skills and ability to collaborate effectively with cross-functional teams.

Good to Have:

• Experience with AWS cloud services, including S3, Lambda, IAM, and Kinesis.
• Knowledge of CI/CD pipelines and DevOps practices in a cloud environment.
• Familiarity with data lake architectures and big data tools.