A few days ago we ran a quiet experiment. We took the pandas repository — one of the most complex, widely-used Python libraries in existence — and asked three codebase intelligence tools the same two questions. Then we had three separate AI evaluators score the answers blind, without knowing which tool produced which output.
The evaluators scored 0 out of 6. Every QEEK answer was attributed to ChatGPT or Claude.
The blind test result
When asked to identify which tool produced which answer, AI evaluators (Grok, GPT, Claude) attributed QEEK's outputs to ChatGPT and Claude — the benchmark models for output quality. None guessed QEEK correctly. That's not a failure of the evaluators. That's a signal about where QEEK's output quality sits.
The test
We ran the pandas repository through QEEK, Google Code Wiki, and Codex and asked two progressively harder questions — one orientation, one deep architecture.
“Where is the core DataFrame class implemented? Show the main file and key methods.”
Orientation question — finding your way into a large codebase
“Explain the internal BlockManager architecture in pandas/core/frame.py. How does it enable pandas' performance and mixed-type support? Show key related files and methods.”
Deep architecture question — understanding how the system actually works
QEEK
Our tool
Google Code Wiki
Google's codebase tool
Codex
OpenAI
Question 1: “Where is the core DataFrame class?”
The orientation test. Can the tool find the right file and give you a useful map?
Clean, well-organized method catalog with tables grouped by category — construction, indexing, manipulation, aggregation, reshaping, merging, IO. Directly answers the question. Brief BlockManager mention at the end. Best structure of the three.
Selective method coverage with precise file paths and line numbers. Goes beyond listing — explains that .loc/.iloc are NOT in frame.py, they come from IndexingMixin. The deepest architectural insight of the three.
Contains useful details but delivered as a prose-heavy wall of text with no markdown formatting. The substance is there; the UX is not. Hard to extract value from in practice.
Question 2: “Explain the BlockManager architecture”
The deep architecture test. This is where exploration pain lives — a complex internal system most developers never fully understand.
Standout feature: an ASCII diagram of the BlockManager architecture that instantly communicates the block-based storage concept. Well-structured sections for performance and mixed-type support. Code snippets for key mechanisms. The most accessible answer across all audiences — juniors, PMs, and seniors.
Deepest architectural trace. Follows the complete construction path: __init__ → dict_to_mgr → arrays_to_mgr → create_block_manager_from_column_arrays → _form_blocks. The insight 'interleaving step is exactly the cost pandas avoids internally' crystallizes the entire design rationale.
Conceptually correct but focuses on peripheral files instead of core implementation. Hedges with 'not fully detailed in the provided file' — signs of limited source access. Misses key architectural elements: blknos/blklocs, consolidation mechanics, blockwise operations.
Who each tool is actually for
Quality isn't just about score — it's about fit. The evaluators assessed each answer across different user types.
| Persona | QEEK | Codex | Google Code Wiki |
|---|---|---|---|
| Junior developer | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐ |
| Product Manager | ⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐ |
| Senior engineer | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ |
| New team member | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ |
Persona ratings based on composite scoring across Grok, GPT, and Claude evaluations.
The twist: 0 out of 6
After scoring the answers, we revealed which tool produced which output. The evaluators had been guessing blind. Here's how they did:
| Answer | Evaluator guessed | Actual tool | Correct? |
|---|---|---|---|
| Q1 — clean method tables | ChatGPT | QEEK | |
| Q1 — prose wall of text | Gemini / Google | Google Code Wiki | |
| Q1 — line-level precision | Cursor | Codex | |
| Q2 — ASCII diagram + structure | Claude | QEEK | |
| Q2 — section headers | QEEK | Google Code Wiki | |
| Q2 — full call-chain trace | Cursor | Codex |
What this actually means
The evaluators attributed QEEK's best outputs — the clean method tables and the ASCII architecture diagram — to ChatGPT and Claude. Those are the models people use as the benchmark for output quality. Being confused with them isn't a mistake. It's a positioning statement.
What we're not claiming
Codex won on the deepest architectural tracing. When the task rewarded following complete call chains — __init__ → dict_to_mgr → arrays_to_mgr → BlockManager — Codex produced the most technically precise output. That's a real gap and we know it.
Two questions on one repo isn't a definitive benchmark. Google Code Wiki may win on narrow factual lookups or extremely large-scale monorepos. Codex has natural advantages on implementation-tracing tasks that suit its design.
Generation consistency matters. We got strong outputs in this run. Staying reliable across different repos, team sizes, and question types is the ongoing work.
Where QEEK is headed
The data needed to match Codex's call-chain tracing already exists in our embeddings. Pre-computing call graphs from import/dependency data and adding trace queries at answer time — “what calls this? what does this call?” — would combine QEEK's superior readability with Codex's architectural depth. That's the next step.
What this means for teams
Codex and Cursor are execution tools. They're at their best when you already know exactly what you want to build and you need precise, implementation-level output. They assume deep context. When that context is missing, they hallucinate — and if you don't understand the system yourself, you can't tell when they've gone wrong.
QEEK is the layer before that. The understanding layer. You explore the codebase, clarify what you want to build, map the architecture — and then hand off to Cursor or Codex with real context behind the prompt. Because you've done the understanding work first, you can spot when the execution goes sideways.
Most of the exploration pain in a software team isn't experienced by senior engineers. It's experienced by everyone around them — the people who need answers but don't have the deep context to find them without help.
That's what this test confirmed. QEEK scored highest with junior developers, PMs, and new team members — the people currently paying the Exploration Gap tax every single day. Not because we're better than Codex at tracing call chains. Because we make the whole system legible to everyone who needs to work with it.
Try it on your own repo
We tested on pandas because it's complex, public, and has a well-understood architecture — a fair challenge. Connect your own repo and see how QEEK handles the questions your team is actually asking.