Node Crawler Files

Web Crawler/Spider for NodeJS + server-side jQuery

This is an exact mirror of the Node Crawler project, hosted at https://github.com/bda-research/node-crawler. SourceForge is not affiliated with Node Crawler.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
2.1.0 source code.tar.gz	2026-06-04	117.4 kB	0
2.1.0 source code.zip	2026-06-04	131.0 kB	0
README.md	2026-06-04	8.1 kB	0
Totals: 3 Items		256.4 kB	0

What's Changed

spelling typo fixes and JSDOM reference case consistency. by @prashantghimire in https://github.com/bda-research/node-crawler/pull/191
Update README.md by @suhaibmujahid in https://github.com/bda-research/node-crawler/pull/197
Inject JQuery in xhtml documents by @suhaibmujahid in https://github.com/bda-research/node-crawler/pull/198
Remove unused 'url' import by @alextes in https://github.com/bda-research/node-crawler/pull/206
fix typo in README.md by @bryanchriswhite in https://github.com/bda-research/node-crawler/pull/212
request not requests by @tleb in https://github.com/bda-research/node-crawler/pull/224
Recognize all XML MIME types to inject jQuery by @nolandg in https://github.com/bda-research/node-crawler/pull/245
Allow options to specify the Agent for Request by @spectral-sys0p in https://github.com/bda-research/node-crawler/pull/246
fix code block languages by @brad811 in https://github.com/bda-research/node-crawler/pull/253
python 3 and more recent node-stretch by @gardsted in https://github.com/bda-research/node-crawler/pull/252
Fix typo in readme by @lahaxearnaud in https://github.com/bda-research/node-crawler/pull/268
Allow to retry when skipDuplicates is enabled by @thomas-hilaire in https://github.com/bda-research/node-crawler/pull/269
Added examples documentation by @sunnyhuang2008 in https://github.com/bda-research/node-crawler/pull/272
fix the ";" problem in content-type of type-is by @h2012c106 in https://github.com/bda-research/node-crawler/pull/270
Added filestream require to download section by @swosko in https://github.com/bda-research/node-crawler/pull/278
20180611 updatetestwithnock by @Dong-Gao in https://github.com/bda-research/node-crawler/pull/280
Add formData arg to requestArgs by @humanik in https://github.com/bda-research/node-crawler/pull/300
Replacement of istanbul with nyc by @kossidts in https://github.com/bda-research/node-crawler/pull/304
Remove dependency of charset-parser. fix [#305] by @cuixiping in https://github.com/bda-research/node-crawler/pull/307
Fix a typo by @jumpalottahigh in https://github.com/bda-research/node-crawler/pull/309
docs: fix typo by @Jason-Cooke in https://github.com/bda-research/node-crawler/pull/314
add support for removeRefererHeader request option to preserve referer during redirects by @petskratt in https://github.com/bda-research/node-crawler/pull/329
Update README.md by @DanielHabenicht in https://github.com/bda-research/node-crawler/pull/336
comment Adding support for Https socks5. Agent is imported directly … by @djpavlovic in https://github.com/bda-research/node-crawler/pull/338
Release automate by @mike442144 in https://github.com/bda-research/node-crawler/pull/353
Remove stale vendored jQuery version by @pzmarzly in https://github.com/bda-research/node-crawler/pull/363
Fix some typos by @pzmarzly in https://github.com/bda-research/node-crawler/pull/364
add http2 functionality by @BeijingProtoHuman in https://github.com/bda-research/node-crawler/pull/367
Unify formatting in README.md by @rauno56 in https://github.com/bda-research/node-crawler/pull/374
Circleci project setup by @mike442144 in https://github.com/bda-research/node-crawler/pull/386
Fixed typo in README.md by @psh0502 in https://github.com/bda-research/node-crawler/pull/395
Fast stack buffer sync spider crawl example by @j-mendez in https://github.com/bda-research/node-crawler/pull/420
Update README.md by @kxxxo in https://github.com/bda-research/node-crawler/pull/437
Crawler V2 Beta is able to merge by @Miniast in https://github.com/bda-research/node-crawler/pull/471
fix readme by @Miniast in https://github.com/bda-research/node-crawler/pull/472
Crawler v2.0.1 is ready by @Miniast in https://github.com/bda-research/node-crawler/pull/475
New features on v2.0.2 by @Miniast in https://github.com/bda-research/node-crawler/pull/476
Some promotes and test update by @Miniast in https://github.com/bda-research/node-crawler/pull/477
feat: Add test for binary data stream. by @Zhubaoheng in https://github.com/bda-research/node-crawler/pull/480
Update v2.0.3-beta.2 by @Zhubaoheng in https://github.com/bda-research/node-crawler/pull/482
Deps upgrade and http2 autoselectfamily by @mike442144 in https://github.com/bda-research/node-crawler/pull/484
fix node version required by @mike442144 in https://github.com/bda-research/node-crawler/pull/485
Run CI on npm instead of pnpm by @mike442144 in https://github.com/bda-research/node-crawler/pull/486
Require Node.js 22 in README by @mike442144 in https://github.com/bda-research/node-crawler/pull/487

New Contributors

@prashantghimire made their first contribution in https://github.com/bda-research/node-crawler/pull/191
@suhaibmujahid made their first contribution in https://github.com/bda-research/node-crawler/pull/197
@alextes made their first contribution in https://github.com/bda-research/node-crawler/pull/206
@bryanchriswhite made their first contribution in https://github.com/bda-research/node-crawler/pull/212
@tleb made their first contribution in https://github.com/bda-research/node-crawler/pull/224
@nolandg made their first contribution in https://github.com/bda-research/node-crawler/pull/245
@spectral-sys0p made their first contribution in https://github.com/bda-research/node-crawler/pull/246
@brad811 made their first contribution in https://github.com/bda-research/node-crawler/pull/253
@gardsted made their first contribution in https://github.com/bda-research/node-crawler/pull/252
@lahaxearnaud made their first contribution in https://github.com/bda-research/node-crawler/pull/268
@thomas-hilaire made their first contribution in https://github.com/bda-research/node-crawler/pull/269
@sunnyhuang2008 made their first contribution in https://github.com/bda-research/node-crawler/pull/272
@h2012c106 made their first contribution in https://github.com/bda-research/node-crawler/pull/270
@swosko made their first contribution in https://github.com/bda-research/node-crawler/pull/278
@Dong-Gao made their first contribution in https://github.com/bda-research/node-crawler/pull/280
@humanik made their first contribution in https://github.com/bda-research/node-crawler/pull/300
@kossidts made their first contribution in https://github.com/bda-research/node-crawler/pull/304
@cuixiping made their first contribution in https://github.com/bda-research/node-crawler/pull/307
@jumpalottahigh made their first contribution in https://github.com/bda-research/node-crawler/pull/309
@Jason-Cooke made their first contribution in https://github.com/bda-research/node-crawler/pull/314
@petskratt made their first contribution in https://github.com/bda-research/node-crawler/pull/329
@DanielHabenicht made their first contribution in https://github.com/bda-research/node-crawler/pull/336
@djpavlovic made their first contribution in https://github.com/bda-research/node-crawler/pull/338
@mike442144 made their first contribution in https://github.com/bda-research/node-crawler/pull/353
@pzmarzly made their first contribution in https://github.com/bda-research/node-crawler/pull/363
@BeijingProtoHuman made their first contribution in https://github.com/bda-research/node-crawler/pull/367
@rauno56 made their first contribution in https://github.com/bda-research/node-crawler/pull/374
@psh0502 made their first contribution in https://github.com/bda-research/node-crawler/pull/395
@j-mendez made their first contribution in https://github.com/bda-research/node-crawler/pull/420
@kxxxo made their first contribution in https://github.com/bda-research/node-crawler/pull/437
@Miniast made their first contribution in https://github.com/bda-research/node-crawler/pull/471
@Zhubaoheng made their first contribution in https://github.com/bda-research/node-crawler/pull/480

Full Changelog: https://github.com/bda-research/node-crawler/compare/0.4.1...v2.1.0

Source: README.md, updated 2026-06-04

Other Useful Business Software

Custom VMs From 1 to 96 vCPUs With 99.95% Uptime Icon

Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.

Try Free

Secure File Transfer for Windows with Cerberus by Redwood Icon

Secure File Transfer for Windows with Cerberus by Redwood

Protect and share files over FTP/S, SFTP, HTTPS and SCP with the #1 rated Windows file transfer server.

Cerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.

Try for Free

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.

Get Started Free

Recommended Projects

File System Crawler for Elasticsearch
Elasticsearch File System Crawler (FS Crawler)
PhantomJS-Node
PhantomJS integration module for NodeJS
Spatie Crawler
An easy to use, powerful crawler implemented in PHP
X-Crawl
Flexible Node.js AI-assisted crawler library
gocrawl
Polite concurrent web crawler library for Go with flexible hooks