Data Engineer with AI/ML-Python

Key Responsibilities
Data Pipeline Development: Design, develop, and optimize robust data pipelines to efficiently collect, process, and store large-scale datasets for AI/ML applications.

ETL Processes: Develop and maintain Extract, Transform, and Load (ETL) processes to ensure accurate and timely data delivery for machine learning models.

Data Integration: Integrate diverse data sources (structured, unstructured, and semi-structured data) into a unified and scalable data architecture.

Data Warehousing & Management: Design and manage data warehouses to store processed and raw data in a highly structured, accessible format for analytics and AI/ML models.

AI/ML Model Development: Collaborate with Data Scientists to build, fine-tune, and deploy machine learning models into production environments. Focus on model optimization, scalability, and operationalization.

Automation: Implement automation techniques to support model retraining, monitoring, and reporting.

Cloud & Distributed Systems: Work with cloud platforms (AWS, Azure, GCP) and distributed systems to store and process data efficiently, ensuring that AI/ML models are scalable and maintainable in the cloud environment.

Data Quality & Governance: Implement data quality checks, monitoring, and governance frameworks to ensure the integrity and security of the data being used for AI/ML models.

Collaboration: Work cross-functionally with Data Science, Business Intelligence, and other engineering teams to meet organizational data needs and ensure seamless integration with analytics platforms.

Required Skills and Qualifications
Bachelor's or Masters Degree in Computer Science, Engineering, Data Science, or a related field.
Strong proficiency in Python for AI/ML and data engineering tasks.
Experience with AI/ML frameworks such as TensorFlow, PyTorch, Scikit-learn, and Keras.
Proficient in SQL and working with relational databases (e.g., MySQL, PostgreSQL, SQL Server).
Strong experience with ETL pipelines and data wrangling in large datasets.
Familiarity with cloud-based data engineering tools and services (e.g., AWS (S3, Lambda, Redshift), Azure, GCP).
Solid understanding of big data technologies like Hadoop, Spark, and Kafka for data processing at scale.
Experience in managing and processing both structured and unstructured data.
Knowledge of version control systems (e.g., Git) and agile development methodologies.
Experience with data containers and orchestration tools such as Docker and Kubernetes.
Strong communication skills to collaborate effectively with cross-functional teams.
Preferred Skills
Experience with Data Warehouses (e.g., Amazon Redshift, Google BigQuery, Snowflake).
Familiarity with CI/CD pipelines for ML model deployment and automation.
Familiarity with machine learning model monitoring and performance optimization.
Experience with data visualization tools like Tableau, Power BI, or Plotly.
Knowledge of deep learning models and frameworks.
DevOps or MLOps experience for automating deployment of models.
Advanced statistics or math background for improving model performance and accuracy.