found 172 packages in 12ms

robots-parser5.7mMIT3.0.1

A specification compliant robots.txt parser with wildcard (*) matching support.

samclarkeover 2 years agorobots.txt, parser, user-agent, scraper

generate-robotstxt487.1kMIT8.0.3

Awesome generator robots.txt

itgalaxyabout 3 years agorobotstxt, robots.txt, robots-txt, generate

gatsby-plugin-robots-txt341.7kMIT1.8.0

Gatsby plugin that automatically creates robots.txt for your site

mdreizinover 2 years agogatsby, gatsby-plugin

sitemapper135.8kMIT4.0.2

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

seantomburke2 months agorobots.txt, parse, sitemap, xml

npmGitHubHomepage

express-robots-txt117.1kMIT1.0.0

Express middleware to serve and generate robots.txt

modoscabout 3 years agorobots.txt, express, robots, robots

‹
1
2
3
4
5
6
7
›

Packages

Lightweight debugging utility for Node.js and the browser

debug, log, debugger

Lodash modular utilities.

modules, stdlib, util

React is a JavaScript library for building user interfaces.

A querystring parser that supports nesting and arrays, with a depth limit

querystring, qs, query, url

Higher-order functions and common patterns for asynchronous code

async, callback, module, utility

babel babel-core

Babel compiler core.

6to5, babel, classes, const