JJSR is a custom-built web crawler and indexer designed for efficiency and privacy. It is developed entirely within a mobile environment (Termux), proving that you don't need a high-end workstation to build powerful data tools.
- Ethical Crawling: Uses
robotparserto strictly followrobots.txtrules and honor website owner preferences. - Resource Management: Designed to operate within a daily limit of 500 URLs to ensure stability on mobile hardware.
- Automated Indexing: Features a custom cycle that refreshes data every 26 hours.
- SQLite Backend: A fast, serverless database that stores titles, meta-descriptions, and page content.
- Simple Flask UI: A clean, no-bloat interface for querying your local index.
- Language: Python 3.x
- Web Framework: Flask
- Parsing: BeautifulSoup4 & Requests
- Database: SQLite3
- Dev Environment: Termux (Android)
JJSR/jjsr/src/jjsr/
├── __init__.py # Flask Web Interface
├── crawler.py # The Indexing Engine
├── db.py # Schema and DB initialization
├── search_data.db # SQLite Database (Auto-generated)
└── README.md # Project Documentation