pingdotgg / pingdotgg/t3code

T3 Code is significantly slower than Codex for the same coding task (20+ min vs ~3m24s)

Open
#695 24 comments 28 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Summary

T3 Code appears to be significantly slower than Codex for the same repository investigation task, using the same model configuration.

With the prompt below, Codex completed the task in 4m35s, while T3 Code took 15+ minutes.

This report is about end-to-end task completion time, not answer quality.

Benchmark prompt
Inspect this repository and answer these questions without editing anything:

1. Where is authentication implemented?
2. Which files handle login/session/token validation?
3. What middleware, guard, or server-side check protects authenticated routes?
4. Where are authorization rules enforced?
5. If a user loses access to a team/organization/project, what code path would prevent them from seeing protected data?

Requirements:
- Read-only investigation only
- Do not edit files
- Do not propose a refactor unless necessary
- Cite the exact files and relevant functions/classes
- End with a concise summary of the authentication flow

But I also tried several other prompts, and this does not seem to be specific to this exact prompt.

Settings
  • GPT 5.4
  • High
  • Fast: On
Results
  • Codex: 4m35s
  • T3 Code: 15+ minutes
Expected behavior

For the same prompt and comparable visible model settings, T3 Code should be much closer to Codex in total completion time, or at least not several times slower.

Notes

This task does not require editing, building, or running the project. It mainly involves repository search, file reading, code-path tracing, and synthesizing a final answer.

Because of that, the slowdown seems more likely related to orchestration/tooling/app overhead than to the model itself.

Environment
  • T3 Code version: 0.0.9 (4b6a0c433cb8)
  • Hardware: Apple M1 Pro, 16 GB RAM
  • macOS: Tahoe 26.2

If useful, I can provide exact timestamps, a screen recording, repeated comparisons, or rerun the benchmark with diagnostics enabled.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the benchmark in T3 Code 0.0.9 on the stated macOS and hardware, comparing it with Codex using the supplied read-only repository-investigation prompt. Collect diagnostics, exact timestamps, and repeated runs to isolate orchestration, tooling, or app overhead; done means identifying the bottleneck and substantially reducing end-to-end completion time.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
devtools, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.