Verdict Changes: What Moved After Our 19-Agent Validation Sprint

## Verdict Changes: What Moved After Our 19-Agent Validation Sprint

At [Your Firm], we pride ourselves on rigorous, data-driven investment models. But no model is perfect. Last month, we completed a **19-agent validation sprint**—a systematic audit where 19 independent agents cross-checked every data point, source, and assumption in our scoring engine. The results were humbling. We uncovered significant data errors—some from LLM hallucinations, others from stale or misinterpreted sources. This post details every major verdict change, why it happened, and what we learned.

### Why We Did This Sprint

Our scoring model uses a weighted combination of on-chain metrics, financial ratios, qualitative signals, and market data. Over time, we noticed that certain scores didn’t align with fundamentals. For example, a token with declining revenue still held a “CORE” rating. A project with an SEC probe was rated higher than peers. These anomalies triggered a full audit. The 19-agent sprint involved:

- **5 data agents** verifying on-chain metrics (TVL, revenue, fees) - **4 financial agents** checking P/S ratios, ARR, and insider transactions - **3 qualitative agents** reviewing news, regulatory filings, and team actions - **7 cross-validation agents** comparing outputs across LLMs (GPT-4, Claude, DeepSeek, Gemini)

What we found was a mix of human oversight and AI hallucination. The most egregious: DeepSeek hallucinated prices by **43x to 12,000x** for some small-cap tokens. That alone would have broken any model.

### Major Downgrades

#### 1. LINK: HOLD CORE → HOLD (-17 points)

**What was wrong:** Our model assumed Chainlink’s oracle revenue accrued to LINK token holders. In reality, Chainlink Labs captures that revenue through node operations and enterprise contracts. Token holders only benefit from staking rewards, which are a fraction of total revenue.

**Correction:** We removed the “revenue-to-token-holders” multiplier. LINK’s score dropped 17 points, moving from HOLD CORE to HOLD. The token remains a solid infrastructure play, but the revenue narrative was overstated.

#### 2. PENDLE: HOLD → CAUTIOUS HOLD (-15 points)

**What was wrong:** Two errors: (1) Revenue was reported as $12M/month, but it had collapsed 88% to $1.4M/month after the airdrop farming frenzy ended. (2) Insider selling data was missing—we later found 3 team members sold 40% of their unlocked tokens in Q1.

**Correction:** Updated revenue and added insider selling as a negative signal. Score dropped 15 points. PENDLE’s yield-trading thesis is still valid, but the risk/reward shifted.

#### 3. SOL: HOLD CORE → HOLD (-6 points)

**What was wrong:** Our model scored Solana highly for ecosystem breadth, but it overweighted “meme coin activity” as a positive. After the sprint, we reclassified meme coin dependency as a risk factor—high volatility, low retention, and regulatory scrutiny.

**Correction:** Adjusted the ecosystem quality weight. SOL dropped 6 points. Still a HOLD, but no longer CORE.

#### 4. GEOD: Score Inflated (No Rating Change Yet)

**What was wrong:** GEOD’s score was inflated by a revenue projection that assumed linear growth. In reality, revenue declined 12% QoQ for two consecutive quarters. The model also used a stale “ATH distance” metric that didn’t account for the token’s 80% drawdown.

**Correction:** We replaced projections with actual QoQ data. GEOD’s score is under review; a formal downgrade is likely next week.

#### 5. SYRUP: ARR Was a Projection, Not Achieved

**What was wrong:** Our model listed SYRUP’s ARR as $8.5M, sourced from a tweet. The tweet was a *projection* for 2025, not actual revenue. The real ARR was $0 (protocol not yet live).

**Correction:** Replaced ARR with $0. SYRUP’s score dropped below the threshold for any HOLD rating. It’s now under WATCH.

### Major Upgrades

#### 1. TIA: WATCH → CAUTIOUS HOLD (+8 points)

**What was wrong:** TIA’s supply data was wrong. Our model used a circulating supply of 200M tokens, but the actual was 350M after a scheduled unlock. This made the P/S ratio look 1.75x lower than reality.

**Correction:** Updated supply to 350M. The lower P/S ratio (still reasonable) plus strong staking demand moved TIA from WATCH to CAUTIOUS HOLD.

#### 2. CRV: Revenue Growing, ATH Distance Corrected (+6 points)

**What was wrong:** Two errors: (1) Revenue was listed as flat, but it grew 22% QoQ after the crvUSD launch. (2) “ATH distance” used the all-time high from 2021, ignoring the token’s 2024 high. This made the distance look larger than it was.

**Correction:** Updated revenue to QoQ growth and ATH distance to the 2024 high. CRV moved from HOLD to HOLD CORE.

#### 3. ONDO: SEC Probe Closed, Fee Switch Imminent (+10 points)

**What was wrong:** Our model still penalized ONDO for an active SEC probe, but the probe was closed in January 2025 with no action. Additionally, the team announced a fee switch mechanism that would direct protocol revenue to token holders—but our model hadn’t priced it in.

**Correction:** Removed the SEC penalty and added a “fee switch imminent” positive signal. ONDO moved from CAUTIOUS HOLD to HOLD CORE.

#### 4. JUP: Securitize Equities Live (+5 points)

**What was wrong:** Our model missed the launch of Securitize equities on Jupiter’s DEX. This is a major catalyst—it brings real-world asset trading to the platform, increasing volume and fees.

**Correction:** Added the Securitize integration as a positive weight. JUP moved from HOLD to HOLD CORE.

#### 5. NEAR: P/S Ratio Was 8x Overstated (+7 points)

**What was wrong:** NEAR’s P/S ratio was reported as 120x, but the actual was 15x. The error came from using “total value locked” instead of “protocol revenue” in the denominator. TVL is not revenue.

**Correction:** Used actual protocol revenue ($45M/year) instead of TVL ($360M). The P/S ratio dropped from 120x to 15x, making NEAR look significantly undervalued. Score increased 7 points.

### The DeepSeek Hallucination Problem

Our most alarming finding: DeepSeek hallucinated token prices by **43x to 12,000x** for 6 small-cap tokens. For example:

- Token A: Actual price $0.12 → DeepSeek reported $5.20 (43x) - Token B: Actual price $0.003 → DeepSeek reported $36.00 (12,000x)

These hallucinations came from DeepSeek scraping outdated or fake coin listings. If we hadn’t cross-validated with on-chain data, those tokens would have been wildly mis-scored. **Lesson: Never trust LLM-generated data without verification.** We now require at least two independent sources for every price point.

### Key Takeaways

1. **Data provenance matters.** A single hallucinated price can break an entire model. 2. **Revenue attribution is tricky.** Many projects claim “protocol revenue” that doesn’t flow to token holders. 3. **Insider selling is a leading indicator.** We now weight it 2x in our scoring. 4. **Always verify ARR claims.** If it’s a projection, treat it as zero until achieved. 5. **ATH distance must be dynamic.** Use the most recent cycle high, not the all-time high from years ago.

### What’s Next

We’re implementing a **continuous validation pipeline**—every week, 5 agents will randomly audit 10% of our data. We’re also open-sourcing our validation methodology so other teams can learn from our mistakes.

**Trust is earned by transparency.** We’ll publish all future verdict changes with full explanations. If you see something off in our scores, email us. We’ll investigate and publish the results.

*— The Research Integrity Team*