Walkin Interview For Sr Python Developer - Immediate Joiner

About Us: We are a dynamic and innovative team specializing in building scalable and efficient web scraping solutions for complex e-commerce platforms and diverse web environments. Our team is proficient in a wide range of technologies and advanced web scraping techniques, ensuring high-quality data extraction and delivery

Job Summary: We are seeking a talented Web Scraping Engineer / Data Extraction Specialist to join our growing team. The ideal candidate will have a strong background in web scraping, data extraction, and backend technologies. You will be responsible for designing, developing, and maintaining robust web scraping solutions, handling dynamic content, and overcoming anti-crawling measures.

Responsibilities:

● Develop and maintain scalable web scraping scripts using Python, JavaScript, and related frameworks (e.g., Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js).

● Implement advanced web scraping techniques, including API interception, sitemap parsing, and handling dynamic content.

● Design and build data pipelines for efficient data extraction, processing, and storage.

● Manage and optimize data extraction workflows, ensuring high speed and accuracy.

● Implement anti-crawling solutions, including IP rotation, proxy management, and CAPTCHA bypassing.

● Collaborate with cross-functional teams to gather requirements and deliver complex data solutions.

● Utilize backend technologies such as Flask, FastAPI, Django, Node.js, Spring Boot, and relational databases (PostgreSQL, MySQL) for data storage and API development.

● Work with cloud platforms like Azure and leverage services such as AzureML and ADLS GEN2.

Employ data processing techniques using libraries like NumPy and Pandas.

● Use tools like Postman, MITM, and DevTools for API testing and network traffic analysis.

● Apply machine learning and NLP techniques for data analysis and processing (e.g., sentiment analysis, content classification).

● Set up and manage server-side scraping environments.

● Monitor and troubleshoot scraping scripts to ensure optimal performance.

Required Skills:

● Strong proficiency in Python and/or JavaScript.

● Experience with web scraping libraries and frameworks (Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js).

● Knowledge of backend technologies (Flask, FastAPI, Django, Node.js, Spring Boot).

● Experience with relational databases (PostgreSQL, MySQL).

● Understanding of HTTP/HTTPS protocols and API communication.

● Familiarity with cloud platforms (Azure).

● Ability to handle dynamic content and JavaScript-heavy websites.

● Experience with anti-crawling techniques (IP rotation, proxies, CAPTCHA bypassing).

● Data processing and analysis skills (NumPy, Pandas).

● Experience with API testing tools (Postman).

● Knowledge of machine learning and NLP concepts is a plus.

● Strong problem-solving and debugging skills.

● Excellent communication and collaboration skills.

Experience:

● 3+ years of professional experience in web scraping or related fields. Education:

● Bachelor's degree in Computer Science, Engineering, or a related field