Files
Tony Li 0c91f0068e feat(parser): add Firecrawl fallback for paywalled and bot-protected content
Integrates the Firecrawl API into GenericParser. If native HTTP requests fail
(e.g., 403 Forbidden) or returning nothing due to paywalls, we automatically
fallback to Firecrawl to extract Markdown content using pseudosearch engine features.

Requires FIRECRAWL_API_KEY environment variable to activate.
2026-02-22 15:41:34 +01:00
..