Important Limitations: This is a **frontend-only** crawler. It relies on the public `allorigins.win` proxy due to browser security restrictions (CORS). This has drawbacks:

**Reliability:** The proxy can be slow, rate-limited, or blocked.
**JavaScript Rendering:** It cannot run JavaScript on the target site, potentially missing dynamic content.
**Anti-Blocking:** True anti-blocking (like IP rotation) is not possible from the frontend.
**Use a proper backend crawler (Python/Node.js with libraries like Scrapy, Playwright, or crawl4ai) for serious scraping.**

Herholdts Frontend Crawler

Ready UA: Chrome

Basic crawler using CORS proxy for herholdts.co.za

Start URL

Max Pages

Crawl Delay

Options

Rotate User Agents (sent to proxy) Attempt Respect robots.txt

Note: Proxies cannot be used effectively from the frontend.

Actions

Crawled

Products

Images

Success Rate

Current UA

N/A

Crawling Progress 0/0 pages

Crawler Logs

System initialized. Ready to start crawling.

Extracted Products (0)

No products extracted yet

Crawler UI Template | Use responsibly and respect website terms of service.

Made with DeepSite - 🧬 Remix