Rust library

Embed conversion and acquisition in your own program.

Library conversion requires no browser process. to_blocks scans UTF-8 HTML into Markdown blocks; render writes them out, omitting any block whose stored frequency meets the template threshold.

let blocks = webfold::to_blocks("<h1>Hello</h1><p>World</p>", "https://example.com/");
let (markdown, kept, dropped) = webfold::render(&blocks, None, 1);

Fetching

use webfold::fetch::native::NativeFetcher;
use webfold::Webfold;

let w = Webfold::new(NativeFetcher::new());
let md = w.scrape("https://example.com/").await.unwrap();

Webfold::scrape calls its FetchBackend, decodes the response and converts the HTML. NativeFetcher is the default backend. Webfold::scrape_tiered is the path the URL scrape command uses, with an action budget.

Template counting across pages

Template::fold counts each block signature at most once per page. Pass the folded template to render to drop chrome shared across the set. Signatures are 64-bit and non-cryptographic, so collisions are possible, and counts wrap modulo 2^32.