Python Web Scraping & Automation Engineer

Job Summary:

We are looking for a skilled Python Web Scraping & Automation Engineer with expertise in web crawling, API integrations, SQL databases, and process automation.

The ideal candidate will be responsible for developing efficient web crawlers, automating data collection processes, and managing data storage in SQL databases.

Key Responsibilities:

Develop and maintain web scrapers using Python libraries such as Scrapy, BeautifulSoup, Selenium, and Requests.

Automate data extraction, processing, and storage pipelines.

Work with SQL databases (MySQL, PostgreSQL, etc.) for efficient data storage and retrieval.

Handle API integrations for third-party data collection and automation.

Implement proxy management, CAPTCHA bypass, and session handling to ensure smooth data extraction.

Monitor and optimize scraper performance for speed, accuracy, and efficiency.

Ensure compliance with legal and ethical web scraping practices.

Collaborate with data analysts and developers to integrate extracted data into applications.

Troubleshoot and resolve issues related to web crawling and automation scripts.

Required Skills & Qualifications:

Proficiency in Python and experience with web scraping libraries (Scrapy, BeautifulSoup, Selenium, Requests).

Knowledge or willingness to learn PHP/Laravel.