found 2,227 packages in 13ms

crawler17.3kMIT2.0.2

Crawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.

bda-researchabout 2 months agocrawler, javascript, spider, scraper

A library for efficiently walking a directory recursively

nodelib8 months agocrawler, NodeLib, fs, FileSystem

fdir93mMIT6.4.6

The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s

thecodrrabout 1 month agocrawler, util, os, sys

isbot4.9mUnlicense5.1.28

🤖/👨‍🦰 Recognise bots/crawlers/spiders using the user agent string.

omrilotan2 months agocrawlers, bot, spiders, googlebot

pdf-parse4.2mMIT1.1.1

Pure javascript cross-platform module to extract text from PDFs.

autokentabout 3 years agopdf-crawler, pdf-parse, xpdf, pdf.js

Packages

Lightweight debugging utility for Node.js and the browser

Lodash modular utilities.

React is a JavaScript library for building user interfaces.

A querystring parser that supports nesting and arrays, with a depth limit

Higher-order functions and common patterns for asynchronous code

Babel compiler core.