Out of Ideas

The Problem

Everyone benchmarks LLM accuracy and speed. Nobody measures creative exhaustion — when a model starts recycling ideas. This matters for real-world use cases like brainstorming, content generation, and ideation.

The Solution

Built a 5-stage pipeline (generate → quality → verify → embed → deduplicate) that measures how many truly unique ideas each model produces before repeating. Used HNSW vector search for semantic deduplication and Perplexity for web-grounded verification.

Technical Highlights

• 14 LLM providers integrated — direct APIs + OpenRouter with auto-shadowing (direct keys override routing)
• Union-Find deduplication with survivor promotion — cluster-based, best entry per cluster survives
• HNSW vector search (Voyage AI embeddings, 768-d cosine) for semantic similarity
• 3 meta-AI loops: seed evolver, seed gatekeeper (Claude Opus), mobile rewriter
• Per-model cost tracking, budget caps, and stratified batch sampling
• Zod schema validation at every I/O boundary

Tech Stack

Key Numbers