Web Scrapping & Chatbot Project

This project consists of two main components:

1. Web Scraper: A Scrapy-based spider that extracts job listings from LinkedIn.
2. Chatbot: A TensorFlow/Keras-based chatbot that allows users to query the scraped job data.

Prerequisites

• Python 3.9+

Getting Started

1. Clone the project: git clone https://github.com/NightFury2415/Web-Scrapping.git
2. Create and activate a Python Virtual Environment:
   • python3 -m venv venv
   • source venv/bin/activate (or venv\Scripts\activate on Windows)
3. Install dependencies: pip install scrapy nltk tensorflow numpy scrapeops_scrapy_proxy_sdk scrapy-user-agents
4. Run the scraper: cd basic_scrapy_spider scrapy crawl linkedin_jobs
5. Run the chatbot: cd .. python chatbot/chatbot.py