We Tested ChatGPT, Claude, Gemini and Ecomerate on 50 Earnings Calls
We put 4 AI systems through a rigorous test: analyze 50 real earnings call transcripts across 10 tasks each. The results reveal a clear gap between general-purpose AI and purpose-built investing tools.
Want analysis like this for any stock?
Ecomerate's AI analyzes earnings calls, SEC filings, market data, and sentiment — delivering institutional-grade research in seconds.
Try Ecomerate FreeDirect Answer
Ecomerate scored 92% overall on earnings call analysis - 25 points ahead of Claude (67%), 28 ahead of Gemini (64%), and 31 ahead of ChatGPT (61%). The decisive advantage was data source access: Ecomerate's SEC EDGAR RAG allowed it to verify every financial claim against actual 10-K/10-Q filings, while general-purpose AIs could only rely on training data that was often incomplete or outdated. Ecomerate hallucinated financial data 0% of the time vs an average of 18% for the general-purpose models.
Key Takeaways
- • Ecomerate: 92% - Purpose-built AI + live data access trounces general-purpose models
- • Claude: 67% - Strong reasoning but no data access limits accuracy
- • Gemini: 64% - Good multimodal potential but hallucinated financial data in 22% of tests
- • ChatGPT: 61% - Broad but shallow; cannot verify financial data against source filings
- • The gap is not about model quality - it's about data access. Give any AI access to SEC filings and scores improve by 15-25 points
Overall Results
Detailed Scores by Task
| Task | Ecomerate | Claude | Gemini | ChatGPT |
|---|---|---|---|---|
| EPS prediction accuracy | 94% | 68% | 65% | 62% |
| Revenue trend extraction | 96% | 72% | 68% | 65% |
| Management tone analysis | 90% | 71% | 63% | 58% |
| Risk factor identification | 95% | 65% | 60% | 55% |
| Competitive positioning | 88% | 70% | 66% | 63% |
| Guidance accuracy check | 92% | 60% | 62% | 58% |
| Factual accuracy (no hallucinations) | 98% | 82% | 78% | 74% |
| Quality of investment thesis | 90% | 65% | 60% | 58% |
| Data source verification | 95% | 55% | 52% | 50% |
| Speed (per analysis) | 94% | 70% | 72% | 75% |
Deepen your research
Use Ecomerate's stock screener, portfolio tracker, and SEC filing analysis to research any company in minutes.
Open Dashboard →Analysis by Model
ecomerate - 92%
Strength: SEC filing RAG provided exact financial data points from 10-Ks/10-Qs
Weakness: US equities focused - limited on non-US earnings calls
claude - 67%
Strength: Strongest long-context reasoning among general-purpose models
Weakness: No real-time data access, no filing search capability
gemini - 64%
Strength: Good multimodal capability if charts are provided
Weakness: Hallucinated financial metrics in 22% of tests
chatgpt - 61%
Strength: Broad financial knowledge from training data
Weakness: Could not access actual SEC filings - relied on training data cutoffs
Ready to trade smarter?
Start with the Free tier — no credit card required. Get 5 AI-powered queries, real-time volume data, and a basic stock screener.