Resource Blocking

Enable blockResources to abort unnecessary network requests during crawling. This can cut page load time by 50-80% for content-only crawls.

Quick Start

const result = await crawler.crawl("https://example.com", {
  blockResources: "fast",
});

Named Profiles

Feedstock ships with three built-in profiles that cover the most common use cases.

"fast" — Images, Fonts & Media

Blocks heavy assets while keeping CSS and JavaScript intact. Best for general-purpose crawling where you still need the page to render correctly.

const result = await crawler.crawl("https://example.com", {
  blockResources: "fast",
});

"minimal" — Everything Except HTML & JS

The most aggressive profile. Blocks stylesheets, images, fonts, media, and all other sub-resources. Only HTML documents and JavaScript are allowed through.

const result = await crawler.crawl("https://example.com", {
  blockResources: "minimal",
});

"media-only" — Images, Video & Audio

A lighter profile that only blocks images, video, and audio. CSS, fonts, and JavaScript all load normally.

const result = await crawler.crawl("https://example.com", {
  blockResources: "media-only",
});

What's Blocked

Custom Config

For fine-grained control, pass an object with patterns and/or resourceTypes:

const result = await crawler.crawl("https://example.com", {
  blockResources: {
    patterns: ["**/*.woff2", "**/*.gif", "**/ads/**"],
    resourceTypes: ["font", "image"],
  },
});

• patterns — glob patterns matched against the request URL.
• resourceTypes — Playwright resource types to block (e.g. "image", "stylesheet", "font", "media", "fetch", "xhr").

Both fields are optional. When both are provided, a request is blocked if it matches either a pattern or a resource type.

Backward Compatibility

Passing true still works and maps to the "fast" profile:

// These are equivalent
blockResources: true
blockResources: "fast"

Environment Variable

Set the profile via the FEEDSTOCK_BLOCK_RESOURCES environment variable:

# Use a named profile
FEEDSTOCK_BLOCK_RESOURCES=fast bun run crawl.ts

# Other profiles
FEEDSTOCK_BLOCK_RESOURCES=minimal bun run crawl.ts
FEEDSTOCK_BLOCK_RESOURCES=media-only bun run crawl.ts

How It Works

Resource blocking uses Playwright's context.route() to intercept and abort matching requests at the browser context level. This is more efficient than page-level routing when reusing sessions.

Service workers are also blocked (serviceWorkers: 'block' on every context) to ensure route interception works on sites that use service workers for caching.

Combine with Navigation Strategy

For maximum speed, pair with navigationWaitUntil: "commit":

const result = await crawler.crawl("https://example.com", {
  blockResources: "minimal",
  navigationWaitUntil: "domcontentloaded", // or "commit" for fastest
});

Deep Crawl with Resource Blocking

const results = await crawler.deepCrawl("https://example.com", {
  blockResources: "fast",
}, {
  maxDepth: 3,
  maxPages: 100,
});