Python Data Engineer

We are looking for a Data Engineer with expertise in Python, C#, and SQL (preferably MySQL/SingleStore) to work on optimizing data pipelines. The role involves analyzing and improving an existing C#-based parser and loader, ensuring seamless data transformation and ingestion into SingleStore. The ideal candidate will have strong experience in data processing, ETL pipelines, and database optimization.

Responsibilities

• Analyze the existing C#-based parser that generates multiple CSV files.
• Adapt or convert the C# parser to Python, optimizing data transformation processes.
• Review and enhance the existing data loader, ensuring efficient integration with SingleStore pipelines.
• Implement and optimize procedures and data pipelines for high-performance data ingestion.
• Identify opportunities to shift logic from loading to parsing, improving data processing efficiency.
• Collaborate with stakeholders to ensure reliable and scalable data workflows.

Required Skills & Experience

• Strong Python skills (data parsing, transformation, and automation).
• Experience with C#/.NET (ability to analyze and adapt existing code).
• Proficiency in SQL, particularly MySQL or SingleStore.
• Hands-on experience with data pipelines, ETL processes, and stored procedures.
• Experience handling CSV file processing and structured data workflows.
• Strong analytical and problem-solving skills in data pipeline architecture.

Preferred Qualifications

• Familiarity with database performance tuning and optimization.
• Experience migrating C# applications to Python for data engineering tasks.
• Understanding of high-performance data processing architectures.
• Knowledge of SingleStore or other distributed SQL databases.