2026-10-12

Evaluating REA claims without the hype

Chinese survey work in ringozzt/rea-research (created 2026-10-11; snapshot 3★ on 2026-10-12 fetch) maps primary vs secondary sources, npm-vs-star gaps, and “not a sandbox.” This post rewrites insights into rea.run voice — it does not paste the report wholesale and drops unverified sensational media claims.

Why this page exists

rea.run keeps getting traffic from viral “reverse anything” posts. Operators need a slow checklist that survives hype cycles: evidence, authorization, pins, and known dual-use limits. The rea-research repo is useful as a secondary survey (interactive report + mindmap); we keep primary CTAs on this site’s compliance and install paths.

Verifiable meta (fetched 2026-10-12 via GitHub API for orientation only): ringozzt/rea-research showed 3 stars and language HTML — stars are discovery, not quality.

Evaluation framework

Lens	Question	Healthy signal	Red flag
Evidence contract	Are claims tied to envelopes/addresses/unknowns?	Files + limits	Vibes-only screenshots
Local ≠ private	Does laptop MCP inherit host trust?	Disposable VM for untrusted bins	“Local means safe”
Stars ≠ installs	Do social metrics prove your host works?	`doctor` JSON green	Star-count as SLA
npm gap	Is the published package what demos used?	Pin `rea-agents@VERSION`	Floating `latest` folklore
Not a sandbox	Can process capture escape the story?	Explicit sandbox note	Implied isolation
Dual-use	Did authorization exist before the agent?	Legal-boundary first	“Agent lowers barrier = permission”

Ten-point checklist

Authorize the target in writing (own / open / clean-room / contract).
Pin rea-agents version; save doctor --json.
Prefer Evidence files over chat prose.
Mark unknowns explicitly; do not fill gaps with guesses.
Keep untrusted binaries in disposable VMs.
Treat stars and X likes as discovery only.
Compare npm package behavior to the demo host before citing demos.
Refuse crack/DRM/unpaid commercial how-tos — /faq/legal-boundary/.
Prefer /showcase/ for community case studies.
Re-read /blog/rea-agent-security-posture/ before production MCP registration.

Community pushback worth keeping

Server-side / closed-repo overclaims: client-only surfaces stay client-only (@0xLiquidation).
Do not fully trust AI conclusions without Evidence (@sharesfromhmf).
Calm local-MCP framing remains a tone model (@linuxuser1996).

Granularity limits

Even strong demos (DX-Ball-class on this site’s showcase) have reconstruction ceilings. Document what you could not recover. Star charts and interactive mindmaps from research repos are orientation — not a substitute for your Evidence.

Compliance

Avoid amplifying bank-attack sensationalism beyond “agents lower barriers → authorization still required.”
Keep /compliance/ and legal-boundary FAQ primary.
Footnote rea-research; no wholesale paste.

Sources & next

Worked example: reading a showcase card

Open any /showcase/ entry. Check: (1) compliance blurb, (2) rating limits, (3) footnotes to GH only, (4) whether the workflow needs BYO engines. If a social post omits those four, it is marketing — not an evaluation.

Cross-read /blog/rea-agents-6-4-0/ for version pins and /blog/rea-linux-startup-benchmarks/ for how to cite community timings without inventing rows.

Claim → evidence map

Claim type	Ask for	Reject if
Startup speed	Node + command + raw seconds + commit	Only “feels fast”
Tool count	Exact list/doctor output	Rounded marketing
Android success	Authorized device log + first-task wall clock	Store APK crack story
Evidence quality	Schema field names + example JSON	Screenshot of chat only

Curriculum pack (one sitting)

Read evaluating page.
Open #1998 and copy two rows into your notes.
Run doctor --json once.
Skim 6.4.0 release notes for ADB / JEB / compact schema only.
Write a three-line “what we will / will not claim” for your team.