Python Developer (Kolkata)

Overview

We are seeking a highly experienced Senior Python Developer with a strong emphasis on web scraping and data extraction technologies, including Scrapy and AI-powered scraping techniques.

The ideal candidate will have a minimum of 5 years of hands-on experience developing robust scraping solutions, implementing AI-driven strategies to bypass anti-bot measures, and managing large-scale data pipelines.

This role demands a deep understanding of web scraping best practices, advanced Python expertise, and familiarity with AI/ML frameworks to enhance scraping capabilities.

You will work collaboratively with a cross-functional team to design, implement, and optimize scraping systems that deliver high-quality, structured data for critical business needs.

Key Responsibilities

Web Scraping Development

• Build and maintain efficient, scalable, and robust web scraping solutions using Scrapy and similar frameworks.
• Implement AI-driven techniques to overcome CAPTCHA, bot detection, and other anti-scraping measures.
• Extract, process, and structure unstructured data from diverse web sources into usable formats.
• Automate scraping processes with an emphasis on efficiency and reliability.

AI-Powered Scraping Techniques:

• Utilize AI/ML models to predict and adapt to anti-bot behaviors.
• Implement NLP (Natural Language Processing) for extracting meaningful insights from text-heavy web pages.

Data Pipeline Management

• Design and optimize ETL pipelines for large-scale data scraping and processing.
• Manage data storage solutions using NoSQL databases like MongoDB or cloud-based systems like AWS S3.

Code Quality and Maintenance

• Write clean, maintainable, and well-documented Python code.
• Implement automated testing frameworks to validate scraping accuracy and reliability.

Required Skills and Experience

Technical Expertise

• 5+ years of professional experience in Python development with a focus on web scraping.
• Proficiency in using Scrapy, BeautifulSoup, Selenium, and other scraping frameworks.
• Strong understanding of HTML, CSS, JavaScript, and browser behavior.
• Experience with AI/ML frameworks such as TensorFlow, PyTorch, or Scikit-learn for scraping-related challenges.
• Expertise in handling APIs (RESTful and GraphQL) for data extraction.
• Proficiency in database systems like MongoDB, PostgreSQL, or Elasticsearch.
• Strong knowledge of version control systems like Git and collaboration platforms like GitHub.

Key Attributes

• Strong problem-solving and analytical skills, with a focus on efficient solutions for complex scraping challenges.
• Excellent communication skills, both written and verbal.
• A passion for data and a keen eye for detail.

Why Join Us?

• Work on cutting-edge scraping technologies and AI-driven solutions.
• Collaborate with a team of talented professionals in a growth-driven environment.
• Opportunity to influence the development of data-driven business strategies through advanced scraping techniques.
• Competitive compensation and benefits.

We are also accepting applications for this profile from Python Developer, Software Developer, Python Django Developer, Backend Developer.