3 Gmware Jobs

Role : Python Lead Engineer - Web Scraping & Crawling

Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.

Responsibilities :

- Develop and maintain service that extracts websites data using scrapers and APIs across multiple sophisticated websites

- Having strong understanding with Python Programming language

- Extract structured / unstructured data and manipulate data through text processing, image processing, regular expressions etc.

- Writing reusable, testable, and efficient code

- Build code that is easily readable, properly documented, and follows key coding standards.

- Use Beautiful Soup and other scraping tools to clean and process data for analysis

- Scrap data from multiple websites like Amazon, Flipkart and Mantra

- Lead and manage a team of web scraping experts to ensure the timely and accurate collection of data from various online sources

Requirements :

- Five years of work experience in Python based web scraping

- Sound understanding and knowledge of Python and good experience in any of the web crawling tools like requests, scrapy, BeautifulSoup, Selenium etc.

- Experience running large scale web scrapes.

- Design and build web crawlers to scrape data and URLs.

- Ability to clean the scraped data to make it ingestible in the database.

- Knowledge of Libraries like Selenium, Requests, Scrappy, BeautifulSoup etc.

- Good Understanding and Implementation of Scrapy, Selenium based crawlers which are highly scalable and reusable

- Identify and resolve any issues or challenges that may arise during the web scraping process

- Stay updated with the legal and ethical considerations of web scraping and ensure compliance