anthropics / anthropics/claudes-c-compiler

Hunting for traces of `chibicc` in `Claude's C compiler`

Aperta
#232 10 commenti 53 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Rust
Stelle
2.8k
Fork
247
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

I'm in a somewhat unique position of having extensively worked on a fork of the compiler [chibicc](https://github.com/rui314/chibicc) and kept around a [list of its bugs](https://github.com/fuhsnn/slimcc/issues?q=is%3Aissue%20label%3Aupstream-chibicc). As this project basically speed-ran my hobby of past three years in two weeks, I got the perfect villain motive to discredit this blasphemy to the 50 years legacy of C! In a desperate attempt I went through those old issues searching for chibicc's "quirks" that this compiler coincidentally exhibits, and to my not surprise, there are indeed plenty!

Here are the issues curated in compiler-explorer for your amusement. Some of these can also be observed in `TinyCC`, we may see them as common mistakes for amateur compilers, or perhaps TinyCC's source was also in the training set weighting towards wrong behavior?

https://godbolt.org/z/YoWee7EPv Choke on `sizeof compound-literal` (chibicc)
https://godbolt.org/z/oeo8MhKhs Choke on `typeof(function-type) *` (chibicc)
https://godbolt.org/z/6rdfTK3vG Choke on `int *_Atomic` (chibicc)
https://godbolt.org/z/eMfoejznx Choke on accessing nested struct expression (chibicc)
https://godbolt.org/z/64K7Gv1j7 Same initialization bug (6 and 7 misplaced, second value is junk) (chibicc)
https://godbolt.org/z/xYbcv4von Qualifier of point-to type not considered for compatibility (chibicc)
https://godbolt.org/z/nETas1xrd Duplicated _Generic type not detected (chibicc)
https://godbolt.org/z/57zGanb4M Missing 32-bit truncation in constant evaluation (chibicc)
https://godbolt.org/z/KjKWGsTM7 Cast-to-bool ignored in constant evaluation (chibicc)
https://godbolt.org/z/W15bbre5v Cast-to-bool ignored with relocation pointer (chibicc)
https://godbolt.org/z/KcP31h1Po Direct assignment of const global struct omitted without error (chibicc)
https://godbolt.org/z/4ssc4hWjK Wrong result from `(boolean-bitfield)++` (chibicc)
https://godbolt.org/z/xbWxdqed7 Same struct alignment bug (chibicc)
https://godbolt.org/z/jEG17n4MP Same struct alignment bug 2 (chibicc, tcc)
https://godbolt.org/z/xsdccMe9E Same x87 ABI bug (chibicc, tcc)
https://godbolt.org/z/P4eKWdsYd Missing array-to-pointer decay (chibicc)
https://godbolt.org/z/jhd9cnWMW Missing array-to-pointer decay 2 (chibicc, tcc)
https://godbolt.org/z/jPqx8TGE1 Missing int promotion for statement-expresion (chibicc, clang)
https://godbolt.org/z/GcvT9P4P1 -E output doesn't preserve #pragma (chibicc)
https://godbolt.org/z/h775n3jWc File name from line directive not in debug info (there should be `.file N "foo"`) (chibicc)
https://godbolt.org/z/G6joW6Eeb Wrong behavior of GCC `,##__VA_ARGS__` extension (chibicc, tcc)
https://godbolt.org/z/dPGa7Ys3M Flexible struct can be created on stack (chibicc)
https://godbolt.org/z/n7G8oes6r Pointer to incomplete type can be indexed (chibicc)
https://godbolt.org/z/dKfWsKzoM #ifdef (number) silently evaluate to false (chibicc)
https://godbolt.org/z/GTfT1h7zK Ranged array initializer executed multiple times (chibicc)
https://godbolt.org/z/oz71jMqYP Function parameter of void type allowed and can be returned (chibicc)
https://godbolt.org/z/rbbxKsWhj `__func__` not counted as keyword (chibicc, tcc)
https://godbolt.org/z/41h9Yv4jx _Alignas() wrongly accept non-power-of-two (chibicc)
https://godbolt.org/z/j16bKG1jz Duplicated names wrongly allowed with identical shadowing characteristic (chibicc)
https://godbolt.org/z/Pd6Mj75f5 Macro argument tokens wrongly preserve newlines (chibicc)
https://godbolt.org/z/PeTfKsG3h `##` in object-like macro leak into output (chibicc)
https://godbolt.org/z/bos8fhddz Floating-point evaluation wrongly allowed in `#if` expression (chibicc)
https://godbolt.org/z/vPf8o7e9c Wrong `__LINE__` after backslash (chibicc)
https://godbolt.org/z/z5eeddsbG Function prototype mismatch not detected (chibicc)
https://godbolt.org/z/8YEofP4vh Duplicated case not detected (chibicc)
https://godbolt.org/z/dndExW6nE Wrong compatibility of `char` (chibicc)
https://godbolt.org/z/3WbhcW9Pn Wrong compatibility of string literal (chibicc)
https://godbolt.org/z/q4GGxehfK VLA silently allowed in file scope (chibicc)
https://godbolt.org/z/vEEGK9sjK Struct variable silently decay to its address (chibicc)

---

Also found an interesting fact @anthropics's blog didn't mention: although the agents "did not have internet access", in the list of test projects there are actually many small C compilers, including chibicc:
https://github.com/anthropics/claudes-c-compiler/blob/6f1b99acb2f4ec2414592136c2009fe7713deec3/ideas/new_projects_myasm.txt#L15
https://github.com/anthropics/claudes-c-compiler/blob/6f1b99acb2f4ec2414592136c2009fe7713deec3/ideas/new_projects_myasm.txt#L80
https://github.com/anthropics/claudes-c-compiler/blob/6f1b99acb2f4ec2414592136c2009fe7713deec3/ideas/new_projects_myasm.txt#L123
https://github.com/anthropics/claudes-c-compiler/blob/6f1b99acb2f4ec2414592136c2009fe7713deec3/ideas/new_projects_myasm.txt#L186-L190
https://github.com/anthropics/claudes-c-compiler/blob/6f1b99acb2f4ec2414592136c2009fe7713deec3/ideas/new_projects_myasm.txt#L237

According to this analysis https://vizops.ai/blog/agent-scaling-laws/:

> Some interesting takeaways from this visualization are that within the first 4 hours, the LLM had already written the first 10K lines of code focusing almost exclusively on the parser. Borrowing an analogy from chess, it felt like the LLM was blitzing through well known theory.

We know the agents must be able to "read" test projects' source in order to debug compilation, could these small C compilers in the test projects be the "well known theory"? That's a fun point to speculate.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.