[clang-repl][ORC][Windows] Investigating COFFPlatform integration, data imports, and emulated TLS

## Summary

I have been trying to understand why `clang-repl` on Windows can execute simple expressions and some C-style library or Win32 calls, but often fails as soon as code depends on the MSVC C++ runtime or standard library. I reproduced the failures locally and followed them through the interpreter, ORC, JITLink, the COFF runtime, and the MSVC runtime interfaces.

Initially, headers such as `` parsed, but common MSVC C++ runtime and standard-library operations did not work end to end. Manually loading selected CRT archives made `std::cout` print, but `std::cin` failed on an unresolved `__ImageBase`; threading, function-local statics, locale state, and `thread_local` exposed more unresolved symbols and JIT-linking failures.

I later found the same user-visible failure in #187040 and the earlier design discussion in #127468. That discussion suggested making emulated-TLS support available to the JIT instead of unconditionally disabling it. The experiment had independently taken that direction; it fixes the original TLS failure, but also exposes more COFFPlatform and MSVC runtime integration requirements.

I now have an experimental stack that runs in-process Windows `clang-repl` with `COFFPlatform` and passes a much broader set of tests:

[Experimental branch and commit stack](https://github.com/Sp0tless/llvm-project/tree/clang-repl-windows-coff)

This is a diagnostic prototype, not a proposed final architecture. I put changes at the nearest integration points to get a working system: the COFF LLJIT setup is currently in `IncrementalExecutor.cpp`, and the MSVC TLS adaptation is an IR rewrite in `OrcIncrementalExecutor.cpp`. Both may belong elsewhere in a production implementation.

@vgvassilev and @lhames, I would appreciate your guidance on the intended architecture and on turning this prototype into production-quality patches. In particular, I am unsure about the ownership boundaries, patch order, and test coverage across the interpreter, ORC/JITLink, and the Windows runtime. If some parts are better handled by other maintainers, I would also appreciate suggestions on whom to ask.

## Test Setup

All tests used:

```text
clang-repl -Xcc -fno-delayed-template-parsing
```

Delayed template parsing is already unsupported by the incremental interpreter. These failures still occur with it disabled, so it is separate from the fixes discussed here.

When I first enabled `COFFPlatform`, I supplied the matching runtime explicitly with `-orc-runtime=...`. The experimental branch now discovers the installed Windows ORC runtime automatically.

## Import-library Code and Data Symbols

After enabling `COFFPlatform` with the matching ORC runtime, bootstrap crashed while calling through:

```text
__imp___orc_rt_jit_dispatch_ctx
```

`__orc_rt_jit_dispatch_ctx` is a data symbol containing the dispatch context pointer. `DLLImportDefinitionGenerator` treated every imported symbol as callable and generated both an IAT slot and a pointer-jump/PLT-style thunk. The thunk was then used as a data pointer, causing an access violation during bootstrap.

This appears to be the case anticipated by a FIXME added in August 2022:

```cpp
// FIXME: check PLT stub of data symbol is not accessed
```

The prototype reads `COFF::ImportType` from COFF import-library members and propagates it into `DLLImportDefinitionGenerator`. Code imports receive a callable thunk; data imports receive only an IAT pointer slot. This fixes the bootstrap crash and resolves MSVC data imports such as `std::cout`, `std::cin`, and locale facet IDs.

This differs from the function-only `COFFAutoImportGenerator`: this path reads an actual import library and therefore has authoritative code/data metadata that should not be discarded.

## TLS and Runtime Integration

After resolving the import issue, emulated TLS exposed several additional problems:

- The Windows ORC runtime did not contain `__emutls_get_address`, requiring an external private archive during the initial experiment.
- MSVC's external `_Init_thread_epoch` is native TLS and cannot be globally lowered as an ordinary emulated TLS variable.
- Dynamically initialized TLS variables need their initializer functions and per-thread guards to remain usable across incremental PTUs.

The prototype:

- Includes emutls in the Windows ORC runtime.
- Adds a bridge for `_Init_thread_epoch`.
- Preserves dynamic TLS initializer functions and per-thread guards across PTUs.

Static and dynamically initialized `thread_local` variables now have distinct addresses and initialize once per thread, including across PTUs.

## SelfEPC Bootstrap Symbols

While reducing the commit stack, I found an independently reproducible bootstrap issue after #213265 moved the former direct EPC calls to SPS wrappers.

The `orc_rt_ci_sps_*` wrappers are registered by the target-process bootstrap used by `SimpleRemoteEPC`, but not by `SelfExecutorProcessControl`. Removing only the SelfEPC registration commit causes the rebuilt `clang-repl` to fail before accepting user input:

```text
clang-repl: Symbols not found: [ orc_rt_ci_sps_call_int32_void ]
```

Calling `rt_bootstrap::addTo(BootstrapSymbols)` in SelfEPC restores the missing wrappers. They are generic SPS controller-interface functions; COFFPlatform appears to be the current in-tree path that exposes the missing registration.

## COFF Image-relative Allocation Range

After the import-type fix allowed the required archive members and ORC runtime objects to be materialized, the investigation was blocked by a different JITLink failure:

```text
section .pdata: relocation target ... is out of range of Pointer32 fixup
```

On x86-64 COFF, `.pdata` and other image-relative references use 32-bit RVAs, including `IMAGE_REL_AMD64_ADDR32NB`. If the synthetic image header, code, data, and unwind information are allocated too far apart, the relocation cannot be represented.

To continue debugging, the prototype configures the Windows COFF `ObjectLinkingLayer` with a `MapperJITLinkMemoryManager` using a 64 MiB contiguous reservation. This keeps the relevant allocations close enough and allowed the standard-library, threading, exception, and TLS tests to proceed.

The fixed 64 MiB value only demonstrates the placement requirement. Once it is exhausted, the memory manager may reserve an unrelated range. A production design needs a scalable per-JITDylib colocating policy in the appropriate ORC or JITLink layer.

## Other Experimental Changes

Once this blocker was bypassed, broader tests exposed several integration and runtime-lifecycle problems.

While replacing early hard-coded DLL loading with normal import-library and runtime discovery, I found that an unresolved DLL-import probe could stop lookup before later generators, including static archives, had a chance to satisfy it. The prototype keeps these probes weak so lookup can continue.

Shutdown tests using `std::atexit` and global objects then showed that the COFF runtime ran callbacks in registration order instead of reverse order. The LIFO fix is now the separately approved #214053.

Higher-concurrency and first-use tests exposed an initializer re-entry deadlock. An initializer starts a worker, waits for it, and the worker re-enters the runtime through `atexit`; if the initializer still holds the JITDylib state mutex, the threads deadlock. The prototype extracts pending initializers under the lock, runs them after releasing it, and then reacquires it. A focused test passes, but the concurrent-update and lifetime semantics still need review.

The SelfEPC bootstrap registration, unresolved-import generator chaining, `atexit` ordering, and initializer re-entry findings appear separable enough for focused patches and tests.

## Reproducible Build and Tests

I added a manually triggered GitHub Actions comparison:

[Windows clang-repl COFF comparison](https://github.com/Sp0tless/llvm-project/actions/runs/31229459073)

The workflow builds the experimental branch and an unmodified upstream baseline, runs a small functional test set, and retains the inputs and full transcripts. It also uploads runnable Release bundles with `clang-repl.exe`, matching Clang resources, and the ORC runtime. Baseline failures remain visible in the summary and transcripts.

The experimental branch has also been tested locally in Debug and RelWithDebInfo builds with:

- Standard streams, including `cout`, `cin`, `cerr`, `clog`, and `wcout`.
- MSVC function and data imports.
- Exceptions and RTTI across incremental PTUs.
- Templates, COMDATs, inline functions, and function-local statics.
- `std::thread`, mutexes, and native `CreateThread` callbacks into JIT code.
- Static and dynamically initialized TLS variables on multiple threads.
- Dynamic DLL loading and calls through Kernel32, User32, Advapi32, BCrypt, and Winsock, including failed lookup, `%undo`, and successful retry.
- A small concurrent IOCP echo-server example.
- Global constructors, initializer re-entry, and LIFO shutdown callbacks.

Beyond short snippets, the same build ran an in-memory workload based on the SQLite amalgamation and a small Dear ImGui Win32 application loaded through the REPL. The latter created a working interactive window.

The SQLite experiment also exposed a separate incremental-C Sema bug where function-scope names leaked into later PTUs. Its fix is an independent top-of-stack commit, not part of the COFF/runtime diagnosis.

These are compatibility experiments, not conformance, stability, or performance claims. They do show that the stack can run nontrivial third-party code rather than only isolated library calls.

## Remaining Issues

The TLS prototype keeps an executor-wide list of dynamic initializer symbols without associating them with their PTU's `ResourceTracker`. Undoing or unloading such a PTU may therefore leave stale state.

Some platform and TLS checks use build-host `_WIN32` or all COFF triples, while the demonstrated COFFPlatform support is x86_64 only. Final checks should use the JIT target and execution mode.

Thread-local object destruction is not implemented. For example, the destructor of:

```cpp
thread_local TLSObject object;
```

does not run when a worker thread exits. MSVC normally uses `__tlregdtor` and PE TLS callbacks, but a JIT allocation is not a loader-managed PE image. ORC therefore needs an explicit model for thread exit, object ownership, unloading, and `ResourceTracker` interaction.

The Windows ORC runtime test configuration also appears to need work before these paths can receive reliable in-tree automated coverage.

## Related Cling Observation

I also reproduced related Windows TLS limitations in Cling. An [emutls prototype](https://github.com/Sp0tless/cling/tree/cling-windows-emutls) improves basic constant TLS, and a [manual Actions comparison](https://github.com/Sp0tless/cling/actions/runs/31202722003) records the result. Cling uses RuntimeDyld, GenericLLVMIRPlatform, and extra incremental IR transformations, and it has additional dynamic-TLS and MSVC alias/COMDAT failures. I have paused that work rather than assuming the `clang-repl` changes apply directly.

## Questions

The main decisions I need before replacing the prototype shortcuts with production-quality patches are:

1. Is COFFPlatform the intended foundation for in-process Windows `clang-repl`, and where should its LLJIT and COFF image-allocation configuration live?
2. For import libraries, what is the preferred API for propagating `COFF::ImportType` into DLL stub generation, should the stub-building implementation be shared with `COFFAutoImportGenerator`, and is unresolved-import probing the appropriate way to allow later generators to satisfy a symbol?
3. What allocation model should guarantee the required COFF image-relative address range per JITDylib, rather than relying on the prototype's fixed reservation granularity?
4. Is emulated TLS the intended model for JIT-owned Windows TLS, and where should `_Init_thread_epoch`, dynamic TLS initialization, `ResourceTracker` ownership, and thread-exit destruction be handled?
5. Should SelfEPC expose the complete generic `rt_bootstrap` symbol set, or only the SPS proxy wrappers currently required by COFFPlatform?
6. Is extracting pending initializers under the JITDylib state lock and running them after releasing the lock the intended re-entry model, and what concurrent-update semantics need to be preserved?
7. Would the following patch split be reasonable: SelfEPC bootstrap, import-library symbol types, unresolved-import generator chaining, Windows ORC emutls runtime, COFF initializer re-entry, and interpreter integration? Which of these would be the best first patch, and what automated coverage would be expected for it?

I am also unsure whether these changes conflict with downstream clients that provide their own import libraries, emutls implementation, ORC runtime, or out-of-process executor.

For now, I am focusing on getting the diagnosis and fixes right on `main`. If some of the fixes are confirmed, merged, and prove sufficiently self-contained, could they potentially be considered for backporting to a release branch later? I am not requesting a backport at this stage, and I will adjust or split the patches based on feedback.

Assisted-by: OpenAI Codex

Start with IncrementalExecutor.cpp and OrcIncrementalExecutor.cpp, then trace COFFPlatform, DLLImportDefinitionGenerator, and SelfExecutorProcessControl. Run clang-repl -Xcc -fno-delayed-template-parsing using the Windows comparison setup and reproduce the import, bootstrap, image-range, and TLS failures. Done requires an agreed production architecture with focused in-tree tests, including correct ResourceTracker and thread-lifecycle behavior.

Written by the indexing model from the issue text.