37 Bluebyte Technologies Jobs
Lead Data Engineer - ETL/Python (8-10 yrs)
Bluebyte Technologies
posted 1d ago
Key skills for the job
Job Description :
- Lead and mentor a team while remaining hands-on with development and problem-solving.
- Work closely with stakeholders to understand business requirements and translate them into technical solutions.
- Design, develop, and optimize complex SQL queries for data extraction, transformation, and analysis.
- Utilize Python for automation, scripting, and data processing.
- Leverage Google Cloud Platform (GCP) services for data storage, processing, and analytics.
- Participate in code reviews, ensure best practices, and optimize data workflows.
- Troubleshoot and resolve performance bottlenecks in SQL queries and data pipelines.
Required Skills :
- Strong communication skills - ability to interact with stakeholders, lead discussions, and present insights.
- Expertise in SQL - deep understanding of database design, query optimization, and performance tuning.
- Hands-on leadership - ability to guide the team while contributing actively to development.
- Proficiency in Python - for scripting, data processing, and automation tasks.
- Good to have Knowledge of GCP services - BigQuery, Cloud Storage, Dataflow, Pub/Sub, etc.
- Experience in migration projects (preferred) - especially from on-prem to cloud.
Responsibilities :
- Lead and manage the design, development, and implementation of scalable data pipelines and data models for both batch and real-time processing.
- Develop and optimize ETL processes to collect, transform, and store data from multiple sources into the company's data warehouse or data lake.
- Work with teams to ensure the efficient integration of data sources, providing end-to-end solutions that support analytics and reporting needs.
- Design and implement strategies for data governance, security, and quality assurance to ensure compliance and data integrity.
- Build and manage data infrastructure (data lakes, data warehouses, cloud platforms like AWS, GCP, Azure) ensuring optimal performance, scalability, and cost-efficiency.
- Mentor and guide a team of data engineers, fostering a collaborative environment and promoting best practices in coding, architecture, and development.
- Optimize database queries and data processing pipelines to enhance performance and reduce costs.
- Troubleshoot and resolve data issues, collaborating with data scientists, product managers, and IT teams.
- Implement and maintain continuous integration/continuous deployment (CI/CD) pipelines for data engineering projects.
- Stay up to date with industry trends and best practices, driving innovation in data engineering practices.
Requirements :
- 8+ years of experience in data engineering, data architecture, or a related field.
- Strong proficiency in programming languages such as Python, Java, or Scala.
- Expertise in designing and optimizing large-scale data pipelines and ETL processes.
- Hands-on experience with cloud platforms (AWS, Google Cloud, Azure) and big data technologies (e.g., Hadoop, Spark, Kafka).
- Advanced knowledge of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
- Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
- Strong understanding of data modeling, data governance, and data security practices.
- Experience with version control systems (e.g., Git) and CI/CD tools.
- Proven leadership experience, with the ability to mentor junior team members and collaborate effectively with cross-functional teams.
- Excellent problem-solving, analytical, and communication skills.
Functional Areas: Software/Testing/Networking
Read full job description