Wigolo
The go-to web for your AI coding agent
...The fetch router can escalate from ordinary HTTP requests to browser automation when it encounters dynamic pages or anti-bot challenges. Crawling supports breadth-first, depth-first, sitemap, and mapping modes while respecting robots.txt and domain rate limits. Results, embeddings, and configuration are stored locally, and the core tools require no account or paid API key. Optional language models can synthesize cited reports, while Python and TypeScript integrations connect Wigolo to popular agent frameworks.