Python Developer (Contractual Role)

Overview:

We are seeking an experienced Python Developer to adapt and deploy existing web scraping scripts to run automatically in a scalable, cloud-based environment. Your role will be to enhance the system for scalability, automation, and long-term data storage solutions.

Project Goals:

• Automate Python web scraping scripts for deployment on Google Cloud Run.
• Optimize the current infrastructure for scalability and manageability across multiple containers.
• Design and implement a system to build efficiency.

Responsibilities:

• Adapt and optimize existing Python scripts (using Selenium) for scalable cloud deployment.
• Implement cloud automation using Google Cloud Platform, particularly with Google Cloud Run.
• Improve containerization strategy to automate the deployment of multiple web scraping scripts using Docker.
• Establish methods for efficient data storage (currently using Google Cloud Storage) and handling of large datasets.
• Collaborate with stakeholders to identify project needs and propose optimal solutions.
• Ensure long-term manageability and scalability for the deployment of multiple containers with different scripts.

Key Technologies:

• Python: Experience with Selenium for web scraping.
• Google Cloud Platform (GCP): Compute Instances, Google Cloud Run, Google Cloud Storage.
• Containerization: Docker, Google Cloud Run, Kubernetes (optional).
• Cloud Automation: Automating deployments using Google Cloud services.
• Data Handling: Experience working with large datasets and storing data in cloud storage solutions like Google Cloud Storage.

Skills & Qualifications:

• Strong proficiency in Python, particularly in using Selenium for browser automation.
• Experience with containerization using Docker and deploying applications to GCP.
• Experience automating cloud deployments and scaling web scraping operations.
• Experience in the logistics domain, especially with shipment tracking and handling shipment data.
• Familiarity with Flask and Gunicorn for managing web scraping scripts in a containerized environment.
• Understanding of data storage solutions in the cloud, including Google Cloud Storage.
• Ability to troubleshoot, debug, and optimize existing Python scripts and cloud infrastructure.
• Excellent problem-solving skills and an ability to work independently.

Preferred Experience:

• Experience with cloud automation tools and managing cloud-based workflows.
• Knowledge of handling data pipelines and storing large-scale web scraping data efficiently.
• Experience in managing and scaling web scraping scripts in cloud environments.
• Previous experience in the logistics or supply chain industry is highly preferred.