Agent-Reach is the #17 project on GitHub's weekly trending chart, giving AI agents the ability to "read the whole web." As of late July 2026 it sits at 61,391 stars, 4,971 forks, MIT license, pure Python, with the first commit on February 24, 2026 and the latest release v1.5.0, authored by Panniantong. What it does in one sentence: let coding agents like Claude Code, Cursor, and OpenClaw read Twitter, Reddit, YouTube, Bilibili, Xiaohongshu, and GitHub from a single CLI with zero API fees and a built-in MCP server. The topics tags declare its intent outright: twitter-scraper, reddit-scraper, youtube-transcript, xiaohongshu, bilibili, mcp, free-api.
What Pain It Solves
Anyone who's built an agent workflow has hit this: the LLM writes code and edits docs like a champ, but ask it to fetch something off the web and it goes blind-no YouTube transcript, Twitter API wants a paid plan, Reddit's anonymous endpoint 403s, Xiaohongshu demands login, Bilibili's anti-bot walls kill generic downloaders, generic web fetches come back as a pile of HTML tags nobody can read. Every platform has its own trap: paywalled APIs, blockades to bypass, accounts to log into, data to clean. Just getting an agent to read a single tweet can eat half a day-installing libraries, tweaking configs. Agent Reach compresses this into one line: paste the README's install.md link to your agent, say "install Agent Reach," and minutes later it's set up-Twitter searchable, Reddit readable, YouTube transcripts pullable, Xiaohongshu browsable, with the agent reading SKILL.md to know which upstream tool to call. The core shift is from "you pick tools and triage each platform yourself" to "a capability layer does the selection, install, and health check for you."
Multi-Platform Reading: One CLI for the Whole Web's Front Door
Supported platforms cover the mainstream Chinese and English channels. Zero-config, works out of the box: web pages (Jina Reader reads any URL), YouTube (yt-dlp pulls subtitles + video search), RSS (feedparser parses any feed), GitHub (gh CLI reads public repos + search), Bilibili (bili-cli does login-free search + video details), V2EX (hot posts + nodes + details), Xueqiu (stock quotes + trending posts). Unlocked after config: Twitter (search tweets + browse timeline + read threads), Reddit (search posts + read comments), Facebook/Instagram (search + Profile + Feed), Xiaohongshu (search + read + comments), LinkedIn (Profile + Company + Jobs), Xiaoyuzhou Podcast (Whisper transcribes audio to text), full-web semantic search (Exa via MCP; note: Exa moved to per-call X402 pricing in mid-2026, ~$0.007/search, 100 free trial uses then paid). The design tradeoff is pragmatic: whatever can be read anonymously gets a zero-config path; whatever needs login state (Twitter/Xiaohongshu/Reddit etc.) goes through "tell the agent: help me configure XXX," and the agent walks you step by step through exporting cookies or reusing a browser session-installed only when you name it.
Zero API Fees + Multi-Backend Routing: Auto-Swap When Blocked
This is the fundamental split from "yet another single-platform CLI." Agent Reach positions itself as a capability layer, not another tool-it only handles selection, install, health check, and routing; the actual reading is done by the agent calling upstream tools directly, with no wrapper layer. Each platform is an ordered backend list: primary + fallback. Bilibili goes bili-cli ▸ OpenCLI ▸ search API (yt-dlp was retired in June 2026 after Bilibili's anti-bot 412'd it to death), Twitter goes twitter-cli ▸ OpenCLI, Xiaohongshu goes OpenCLI ▸ xiaohongshu-mcp ▸ xhs-cli, LinkedIn goes linkedin-scraper-mcp ▸ Jina Reader. When one path dies, the author swaps to the next and you upgrade without noticing-in March 2026 a batch of single-platform CLIs went unmaintained, the author rerouted, users did nothing. agent-reach doctor tells you which backend each platform is currently using, which is online, which is broken, and how to fix it. All upstream tools are open source, and nearly all APIs are free (one exception: Exa search moved to per-call X402 pricing in mid-2026, $0.007/search); the only other thing that might cost you is a server proxy ($1/month), not needed on a local machine.
MCP and Agent Orchestration
It's not another SaaS; it's a local CLI + MCP service sitting next to your agent. The install itself runs through the agent: paste "install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md" to Claude Code/Cursor/OpenClaw, and the agent pip-installs itself, sets up system infrastructure (Node.js, gh CLI, mcporter), configures the Exa search engine, and registers SKILL.md into the agent's skills directory-after that, the agent automatically knows which upstream tool to call for "web research," "search Twitter," "watch this video" requests. The MCP part goes through mcporter into Exa semantic search (note: Exa moved to per-call X402 pricing in mid-2026, so the no-Key claim no longer holds-100 free trial uses, then ~$0.007/search), callable by any MCP-capable client. Compatibility covers Claude Code, OpenClaw, Cursor, Windsurf-any agent that can run shell commands. OpenClaw users need to enable exec permissions first (openclaw config set tools.profile "coding"). On security, cookies live only in ~/.agent-reach/config.yaml locally (permission 600), never uploaded or transmitted; --safe mode skips auto-installing system packages and just lists them; --dry-run previews every operation without doing anything; if you don't trust a component, swap the corresponding channel file, no impact on the rest.
Three-Minute Setup
# 1. Install (Python >= 3.10)
pip install agent-reach
# 2. One-shot install (default mode for personal machines)
agent-reach install --env=auto
# Or safe mode (production servers): agent-reach install --env=auto --safe
# Or preview only: agent-reach install --env=auto --dry-run
# 3. Health check
agent-reach doctor
# Outputs each platform's current backend, connectivity, fix prescriptions
# 4. Use (any command-line-capable Agent works)
# Tell Claude Code directly:
# "Summarize what this YouTube video says: URL"
# "Search Twitter for what people think of this product"
# "Search the whole web for the latest LLM framework comparison"
# Agent reads SKILL.md and calls the right upstream tool itselfEven easier: paste the README's install.md link to your agent, and it handles package install, infrastructure, MCP config, and skill registration itself-you just give orders. Updates are one line too: paste the update.md link. agent-reach uninstall wipes all tokens/cookies, skill files, and MCP config in one shot; --keep-config preserves config for reinstalls.
Who It's For + Five Pitfalls
For: people running coding agents for full-web research, social listening, competitor monitoring; developers building agent workflows who don't want to wire up each platform's API separately; cost-sensitive individuals and small teams unwilling to pay monthly platform API fees; privacy scenarios requiring cookies to stay on-device.
Five pitfalls. One, cookie ban risk: on platforms logged in via cookie (Twitter, Xiaohongshu, Reddit etc.), script/API calls can be detected by the platform and get the account banned-always use a dedicated side account, never your main one; a cookie equals full login permission, and a side account limits blast radius if it leaks. Two, Twitter cookie handling is special: agent-reach configure xhs-cookies does not inject cookies into OpenCLI/Chrome; after saving, the Twitter cookie is only used by doctor to check config completeness-before running the upstream twitter command directly, you still need to explicitly set TWITTER_AUTH_TOKEN and TWITTER_CT0 in the current process environment. Three, platform ToS: scraping each platform's data depends on their terms; bulk scraping and anti-bot bypassing sit in a legal gray area-read the terms before commercial use. The tool is MIT, but your usage may not be. Four, backend retirement and switching: yt-dlp was killed by Bilibili's anti-bot in June 2026 and retired; platform anti-bot escalation is the norm-when a path dies, wait for the author to reroute, or hang your own proxy. Local agent-reach doctor always tells you which path is currently in use. Five, server deployment: a local machine needs no proxy; only server deployments do (~$1/month). On servers run agent-reach install --env=auto --safe, not the default full-auto mode-shared machines especially should go through safe mode.
vs. the Competition
Against each platform's official API, the essence is "zero-fee + multi-channel aggregation" vs. "paid + single-platform": official APIs are stable and compliant but Twitter API starts at hundreds of dollars a month, Reddit's official API is approval-gated, and Xiaohongshu has no public API at all-Agent Reach compresses all of this into a zero-fee open-source layer, at the cost of using cookies/browser login state in a gray area and carrying your own compliance risk. Against Jina Reader, Jina only reads web pages (one URL to markdown); Agent Reach adds a layer of social-platform reading on top-and Jina is actually one of its upstream backends (the web channel's primary). Against firecrawl, firecrawl is a paid SaaS for structured web extraction, strong at batch-crawling whole sites and outputting clean markdown, but it only covers web pages and never touches social platforms-Agent Reach is an open-source local CLI covering social + web + search, positioned as the whole-web front door for agents, not a web-data ETL tool. Against single-platform CLIs (twitter-cli, bili-cli each on their own), Agent Reach adds selection + health check + multi-backend routing + automatic SKILL.md registration; when a single-platform CLI goes unmaintained, it reroutes, you don't have to babysit it. In one line: to let a coding agent read the whole web from one sentence, with zero API fees and automatic follow-up on platform anti-bot escalation-Agent Reach.
References
- Agent-Reach GitHub repo (61,391 stars, MIT, Python): https://github.com/Panniantong/Agent-Reach
- Official README (Chinese / English / 日本語 / 한국어 multilingual): https://github.com/Panniantong/Agent-Reach
- GitHub Trending weekly rank #17 (this week): see repo Trending status
- Upstream tools list (Jina Reader / yt-dlp / gh CLI / bili-cli / Exa / OpenCLI / twitter-cli / rdt-cli / xiaohongshu-mcp / feedparser / linkedin-scraper-mcp): see README Acknowledgements section
- Install and update docs: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md