Proposal: Reflect Type Transfer for the State Codec
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start with Save and Load in internal/state/state.go, then read encodeState.findType and decodeState.findType in internal/state/encode.go and internal/state/decode.go. Review writer.put and reader.get in internal/state/memory.go plus the existing nativeReflectType bridge. Done means the state codec discovers and transfers the described type table and restores object-graph type relationships without changing the listed out-of-scope components.
Written by the indexing model from the issue text.
Description
Proposal: Reflect Type Transfer for the State Codec
1. Summary
Discover types from Go's reflect caches, describe them using references to other types, and preload them into the state codec's own type table. Encode that table before the object graph in caller-owned shared memory. The guest reconstructs its type table first, then uses it to allocate and restore values, including values that themselves represent a reflect.Type.
There is one TypeID namespace per transfer. Cache discovery and object traversal use the same table. Type records are written in TypeID order; object traversal order does not determine how the guest interprets those IDs.
This is a design proposal, not a description of completed support. It applies to the imported gVisor state code in internal/state. It does not change Sentry, the shared-library bridge, or ixgo in this increment. Source details below were checked against this worktree and Go 1.26.6.
2. User Stories / Motivation
- A caller passes an object containing a dynamically constructed type and values of that type. Both must refer to the same reconstructed type in the guest.
- A caller passes several interface values with the same dynamic type. The guest must resolve their type consistently without manual
state.Registercalls. - Later ixgo integration needs ordinary guest
reflect.Typeobjects that reflection can use directly. This proposal supplies type reconstruction; interpreter state and function reconstruction remain separate work.
The common need is to transport type identity and construction information alongside the existing object graph, without copying runtime type descriptors as ordinary heap objects.
The following example is used throughout the proposal:
recordType := reflect.StructOf([]reflect.StructField{
{Name: "Count", Type: reflect.TypeOf(int(0)), Tag: `json:"count"`},
})
sliceType := reflect.SliceOf(recordType)
record := reflect.New(recordType).Elem()
record.Field(0).SetInt(42)
items := reflect.MakeSlice(sliceType, 1, 1)
items.Index(0).Set(record)
root := struct {
Type reflect.Type
Value any
Items any
}{
Type: recordType,
Value: record.Interface(),
Items: items.Interface(),
}
After restoration, these relationships must hold:
reflect.TypeOf(restored.Value) == restored.Type
reflect.TypeOf(restored.Items).Elem() == restored.Type
reflect.ValueOf(restored.Value).Field(0).Int() == 42
reflect.ValueOf(restored.Items).Index(0).Field(0).Int() == 42
The host and guest type descriptor addresses may differ. Their relationship is established through the transferred type table.
3. Current Workaround
The imported state implementation resolves registered types through StateTypeName, StateFields, and a global name-to-type database populated by Register. The native closure extension also has a nativeType path that looks up static types in executable DWARF, and a closureType path for capture layouts derived from a PC.
This supports existing registered types and some executable types, but does not describe arbitrary standard-reflect dynamic types. A StructOf result cannot acquire the StateTypeName and StateFields methods required by the existing registration API. Passing a type as a value also requires recognizing what that value represents, rather than recursively copying its private *reflect.rtype implementation.
The current entry points are Save and Load, encodeState.findType, and decodeState.findType. The current byte-slice transport is already writer.put and reader.get.
4. Goals
- Enumerate the four standard reflect type caches and recursively collect the types referenced by their entries.
- Use one state-owned TypeID table for discovered types, types encountered while encoding values, and restored types.
- Reconstruct standard reflect dynamic types and preserve references to static types from the same executable.
- Encode a
reflect.Typevalue as a reference to that table. - Reuse the existing object graph machinery for values, shared references, interior references, and cycles.
- Keep the shared-memory transport as
w.putandr.get, without introducing JSON, gob, RPC, or a second serializer.
5. Out of Scope
- Copying raw runtime type descriptors,
runtime.reflectOffs, GC metadata, method code, or reflect's cache backing storage. - Discovering every type ever created through third-party unsafe runtime manipulation.
- Reconstructing arbitrary dynamically named types, arbitrary dynamic interface types, or arbitrary method sets through standard reflect constructors.
- Transferring channel contents, blocked goroutines, package globals, external resources, or interpreter execution state.
- Implementing
reflect.Valuetransfer orreflect.MakeFunccallback transfer. Their type references can use this table, but their values require separate handling. - Broadening the existing native closure ABI and platform support, changing Sentry, or connecting
internal/stateto productionsandbox.Runas part of this document.
6. Proposal
6.1 Design Rule and Ownership
The codec owns the type table. The cache reader supplies reflect.Type objects to that table; it does not maintain a separate ID allocator. An object encoder requests the ID of a type from the same table. A decoder resolves that ID from the transferred records.
Host reflect caches ----> state type table <---- object traversal
|
ordered type records
|
caller-owned memory
|
guest state type table
|
restored object graph
| Owner | Responsibility |
|---|---|
| Reflect cache reader | Read cache entries and expose their represented types; do not modify caches. |
| State type table | Deduplicate types, assign IDs, describe dependencies, and resolve guest types. |
| State object codec | Encode and decode values and object references using those IDs. |
| Shared-memory caller | Own the buffer and keep it valid while Save or Load uses it. |
| Guest reflect/runtime | Construct and retain valid local type descriptors and their runtime metadata. |
The type table is local to one Save or Load operation. TypeID 7 in one transfer has no relationship to TypeID 7 in another. ObjectID is a separate namespace: it identifies an object, not its type.
6.2 Find Types in Reflect Caches
Go 1.26.6 stores standard reflect constructor results in four caches. Cache keys and hash buckets are lookup machinery; they are not transferable TypeIDs.
| Symbol | Actual stored values | Extraction |
|---|---|---|
reflect.ptrMap |
*abi.PtrType values in a sync.Map |
Obtain the pointer with reflect.ValueOf(value).UnsafePointer(); its initial abi.Type is at the same address; convert with the existing reflect.toType bridge. |
reflect.lookupCache |
Values implementing reflect.Type in a sync.Map |
Assert value.(reflect.Type). This covers ArrayOf, ChanOf, MapOf, and SliceOf. |
reflect.funcLookupCache.m |
Buckets of []*abi.Type |
Iterate each bucket through reflection and convert each pointer with reflect.toType. |
reflect.structLookupCache.m |
Buckets of []reflect.Type |
Iterate each bucket directly. |
The function and struct caches have an outer sync.Mutex followed by a sync.Map. A linkname declaration must mirror that layout, including the mutex. For example, the already explored discovery shape is:
// These declarations depend on the checked Go version's private layout.
// The declaring file imports reflect, sync, and unsafe.
//go:linkname pointerCache reflect.ptrMap
var pointerCache sync.Map
//go:linkname compositeCache reflect.lookupCache
var compositeCache sync.Map
//go:linkname functionCache reflect.funcLookupCache
var functionCache struct {
sync.Mutex
Entries sync.Map
}
//go:linkname structCache reflect.structLookupCache
var structCache struct {
sync.Mutex
Entries sync.Map
}
Access to these private cache variables requires -ldflags=-checklinkname=0 with the checked toolchain. This permits linking; it does not establish layout compatibility with another Go version.
Reuse nativeReflectType, the existing linkname declaration for reflect.toType; another conversion bridge is unnecessary. Enumeration uses Range on each map and iterates every element of collision buckets. For example:
pointerCache.Range(func(_, value any) bool {
collect(nativeReflectType(reflect.ValueOf(value).UnsafePointer()))
return true
})
compositeCache.Range(func(_, value any) bool {
collect(value.(reflect.Type))
return true
})
functionCache.Entries.Range(func(_, value any) bool {
bucket := reflect.ValueOf(value)
for i := 0; i < bucket.Len(); i++ {
collect(nativeReflectType(bucket.Index(i).UnsafePointer()))
}
return true
})
structCache.Entries.Range(func(_, value any) bool {
for _, typ := range value.([]reflect.Type) {
collect(typ)
}
return true
})
Here collect means append the type if it is not already in a map[reflect.Type]bool. It does not mean assign an independent cache ID. Collect actual reflect.Type references, not only uintptr addresses, so the discovery result holds valid Go references until encoding finishes.
Complete the cache reads before processing the collected roots. Do not call type constructors from the Range callbacks: some constructors create more cached types, including internal map layout types.
The enumeration reads all entries it observes. It is not a single atomic snapshot across caches. Go's function and struct cache buckets are append-only in this version, but sync.Map.Range does not guarantee a consistent snapshot under concurrent writes. Object traversal must still add referenced types absent from the discovery result. Strict enumeration of every cache entry at one instant requires quiescent type creation; the synchronization policy is an open question in section 12.
These caches are not a complete registry of all static types. For example, PointerTo can return a compiler-provided PtrToThis before touching ptrMap. They also do not contain all types created by third-party unsafe constructors. Static dependencies and value-reachable types are collected through the same type table as encoding proceeds. runtime.reflectOffs is not a substitute: it is a separate mapping that also contains names and other offsets.
6.3 Assign IDs Once, Inside State
Keep the existing concept of a state-owned map[reflect.Type].... Equality of reflect.Type identifies a type; Type.String() is not a unique identity key.
The logical algorithm is:
reserve(t):
if the state table already contains t:
return its existing ID
assign the next nonzero ID
store t -> ID immediately
reserve the record slot for that ID
queue t for description
return ID
preload:
for each deduplicated cache type t:
reserve(t)
describe queued types:
for each type t:
construct its record
for each referenced type child:
use reserve(child) as the reference
put the completed record in t's reserved slot
reserve is pseudocode for the table operation, not a proposed exported API. Reserve before describing children, so repeated references reuse an ID. Newly discovered types during object encoding go through the same operation. Finish describing queued types before writing the stream.
For the running example, suppose the relevant discovered roots arrive as [sliceType, recordType]. In a minimal table with these roots, the result is:
Preload roots: sliceType -> 1, recordType -> 2
Describe ID 1: Slice { Elem: 2 }
Describe ID 2: Struct { Count: reserve(int), Tag: json:"count" }
Add dependency: int -> 3
Describe ID 3: Builtin { Kind: Int }
Real cache enumeration can contain unrelated entries, so the numeric IDs may differ. Correctness depends on the recorded references, not these particular numbers or deterministic cache iteration.
Now state reaches root.Value before root.Items:
root.Value has recordType -> lookup returns 2
root.Items has sliceType -> lookup returns 1
State does not renumber recordType to 1 just because that value was visited first. Both the cache reader and the value encoder use the same table.
6.4 Decompose Each Type
The input is a reflect.Type t whose ID has already been reserved. The output is its semantic type record in that ID's slot. Every reserve(child) below uses the same state table from section 6.3; it returns the child's ID and queues any newly discovered type for description. Guest notation types[id] means the result of resolving that dependency, not a requirement that records arrive in dependency order.
Apply the identity checks before switching on Kind: recognize an actual builtin, otherwise preserve an executable static type, otherwise reject an unsupported non-static named type, then describe a supported dynamic type. This prevents type Count int from becoming int or a named struct from becoming an anonymous struct.
| Input type / classification | Exact host extraction | Semantic record stored in this TypeID slot | Guest reconstruction | Example / boundary |
|---|---|---|---|---|
| Nil type value | Check t == nil before asking for Kind or reserving a type |
No type record; the value codec preserves nil | Restore the nil value | ID 0 is not assigned to a real type. Invalid is not a valid type descriptor to reconstruct. |
| Predeclared bool | Confirm type identity with the builtin bool type | Builtin { Kind: Bool } |
Retrieve guest builtin bool | No child TypeID. |
| Predeclared signed integers | Match the builtin int, int8, int16, int32, or int64 type by identity | Builtin { Kind: t.Kind() } |
Retrieve that exact guest builtin | Preserve Int versus Int64, even on a 64-bit ABI. |
| Predeclared unsigned integers | Match the builtin uint, uint8, uint16, uint32, uint64, or uintptr type by identity | Builtin { Kind: t.Kind() } |
Retrieve that exact guest builtin | byte is an alias for uint8, not another type entry. |
| Predeclared floats and complex numbers | Match float32, float64, complex64, or complex128 by identity | Builtin { Kind: t.Kind() } |
Retrieve that exact guest builtin | The Kind distinguishes each precision. |
| Predeclared string | Confirm type identity with the builtin string type | Builtin { Kind: String } |
Retrieve guest builtin string | String contents belong to value encoding. |
| unsafe.Pointer | Confirm identity with the unsafe.Pointer type | Builtin { Kind: UnsafePointer } |
Retrieve guest unsafe.Pointer type | Reconstructing this type does not make arbitrary pointer values transferable. |
| Any remaining type in the current executable's static type region | Obtain the descriptor address and subtract host runtime.types |
Static { Offset: descriptorAddress - typeRegionBase } |
Resolve the validated descriptor at the corresponding guest offset | Includes named types, static interfaces such as error, recursive named types, and unnamed static types; their methods remain part of the original descriptor. |
| Non-static named type | After builtin/static checks, test t.Name() != "" |
Unsupported-type error; no anonymous replacement | No standard reflect constructor for an arbitrary named type | Name and PkgPath alone cannot recreate its identity or attach its methods. |
Dynamic pointer *T |
elemID = reserve(t.Elem()) |
Pointer { Elem: elemID } |
reflect.PointerTo(types[elemID]) |
The type record contains no pointee object address. |
Dynamic slice []T |
elemID = reserve(t.Elem()) |
Slice { Elem: elemID } |
reflect.SliceOf(types[elemID]) |
Length, capacity, and backing storage describe a slice value, not its type. |
Dynamic array [N]T |
length = t.Len(); elemID = reserve(t.Elem()) |
Array { Len: length, Elem: elemID } |
reflect.ArrayOf(length, types[elemID]) |
[3]int and [4]int have different type records. |
Dynamic map map[K]V |
keyID = reserve(t.Key()); elemID = reserve(t.Elem()) |
Map { Key: keyID, Elem: elemID } |
reflect.MapOf(types[keyID], types[elemID]) |
Key comparability follows from the restored key type; guest MapOf builds its own hasher and internal layout. |
| Dynamic channel | direction = t.ChanDir(); elemID = reserve(t.Elem()) |
Chan { Dir: direction, Elem: elemID } |
reflect.ChanOf(direction, types[elemID]) |
Preserve BothDir, SendDir, or RecvDir. Buffer capacity and queued values are not type metadata. |
| Dynamic function signature | For i = 0..NumIn()-1, append reserve(t.In(i)); for i = 0..NumOut()-1, append reserve(t.Out(i)); read t.IsVariadic() |
Func { In: inputIDs, Out: outputIDs, Variadic: flag } |
reflect.FuncOf(resolvedInputs, resolvedOutputs, flag) |
For func(string, ...int) error, In refers to string and []int; Out refers to static error; Variadic is true. No PC or captured environment is stored here. |
| Dynamic anonymous struct | For i = 0..NumField()-1, get f = t.Field(i) and preserve f.Name, f.PkgPath, reserve(f.Type), string(f.Tag), and f.Anonymous in that order |
Struct { Fields: [{ Name, PkgPath, TypeID, Tag, Anonymous }, ...] } |
Resolve each TypeID, build the ordered []reflect.StructField, then call reflect.StructOf(fields) |
Preserve the complete tag, package path, field order, and embedding flag; obey StructOf's field and method restrictions. |
| Non-static interface | Kind is Interface after the static-type check | Unsupported-type error | There is no standard reflect.InterfaceOf constructor |
Method names and signatures are inspectable, but that does not provide a way to construct the interface. |
| Other non-static type outside these supported cases | No matching identity or supported constructor description | Unsupported-type error | No implicit conversion to an approximate type | Covers types requiring third-party runtime-specific reconstruction. |
The record names and field labels above describe the format's meaning; they do not assign binary tag numbers. The complete decomposition algorithm is:
describe(t):
require t != nil and t already has an ID
if t equals one of the actual builtin types:
return Builtin { Kind: t.Kind() }
if t's descriptor belongs to the current executable's static type region:
return Static { Offset: descriptorAddress(t) - typeRegionBase }
if t.Name() != "":
fail unsupported non-static named type
switch t.Kind():
Pointer:
return Pointer { Elem: reserve(t.Elem()) }
Slice:
return Slice { Elem: reserve(t.Elem()) }
Array:
return Array { Len: t.Len(), Elem: reserve(t.Elem()) }
Map:
return Map { Key: reserve(t.Key()), Elem: reserve(t.Elem()) }
Chan:
return Chan { Dir: t.ChanDir(), Elem: reserve(t.Elem()) }
Func:
inputs = []
outputs = []
for i from 0 up to, but not including, t.NumIn():
inputs.append(reserve(t.In(i)))
for i from 0 up to, but not including, t.NumOut():
outputs.append(reserve(t.Out(i)))
return Func { In: inputs, Out: outputs, Variadic: t.IsVariadic() }
Struct:
fields = []
for i from 0 up to, but not including, t.NumField():
f = t.Field(i)
fields.append({
Name: f.Name,
PkgPath: f.PkgPath,
TypeID: reserve(f.Type),
Tag: string(f.Tag),
Anonymous: f.Anonymous,
})
return Struct { Fields: fields }
Interface:
fail unsupported non-static interface type
otherwise:
fail unsupported type
while the description queue is not empty:
t = take next queued type
id = stateTypeIDs[t]
records[id - 1] = describe(t)
# reserve(child) may have appended more types to this same queue.
For the running example, describing recordType at ID 2 produces the following complete field record:
t.Field(0) gives:
Name="Count", PkgPath="", Type=int, Tag=`json:"count"`, Anonymous=false
reserve(int) -> 3
records[1] = Struct { # records[1] is the slot for TypeID 2
Fields: [{
Name: "Count",
PkgPath: "",
TypeID: 3,
Tag: `json:"count"`,
Anonymous: false,
}],
}
Offset and Index from a StructField are not encoded construction inputs; guest StructOf computes them. Size, alignment, pointer metadata, and equality machinery are also derived locally. Do not copy host addresses for type names, Equal/Hasher callbacks, GC bitmaps, or reflectOffs entries.
The static branch preserves a compiled type Node struct { Next *Node } as one reference instead of recursively rebuilding its named definition. As specified in section 12, validating the guest's static descriptor target remains unresolved; the pseudocode's host classification does not authorize casting an arbitrary guest offset to a runtime type pointer.
6.5 Encode Type Values and Ordinary Values
Use the same type table for two different references:
Interface { DynamicType: 2, Value: ... }
means: this ordinary value has type 2
TypeValue { ReferencedType: 2 }
means: this value represents type 2
The distinction is necessary for root.Type versus root.Value. A value implementing reflect.Type must be recognized before ordinary traversal follows the implementation pointer into runtime metadata. This also applies when that type value is carried through an any interface or used as a map key.
For the example, the logical object data is:
root.Type = TypeValue(2)
root.Value = Interface(DynamicType=2, Value=StructFields[42])
root.Items = Interface(DynamicType=1, Value=Slice(length=1, capacity=1, ...))
slice backing data = [StructFields[42]]
These are semantic diagrams. The existing object codec continues to emit ObjectIDs and references for the root, slice backing storage, and other objects. The diagram does not introduce field names into every object record or replace state pointer tracking.
After a dynamic interface type is resolved, ordinary fields still use the existing reflection traversal. Static field types generally follow from the parent object's type, so a separate TypeID need not be added to every scalar field value.
Native closure values continue through the existing PC and captured-environment path. Any types needed for their values must resolve consistently through the type table. Synthetic capture layouts derived from DWARF retain their existing special handling; this proposal does not establish new capture layouts or make callback migration automatic.
6.6 Write the Shared-Memory Stream
The current state encoder builds its object graph before writing it. Keep that sequencing:
1. Collect cache roots.
2. Preload state's type table and describe their dependencies.
3. Traverse the root object graph, reusing or adding type IDs.
4. Finish pending type descriptions and freeze the table for this write.
5. Write the existing object-count header.
6. Write type records in ascending TypeID order through w.put.
7. Write object IDs and their encoded values through w.put.
The type section uses the existing position-based ID convention:
shared memory
object-count header
first type record = Slice(Elem=2) -> ID 1
second type record = Struct(Count: TypeID=3, json:"count") -> ID 2
third type record = Builtin(Int) -> ID 3
remaining type records for the complete graph, if any
object ID, object data
object ID, object data
...
There is no separate cache-number table and no redundant explicit ID field required in each type record. The first record is ID 1, the second is ID 2, and so on. A description that recursively finishes before its parent must still be written in its reserved ID slot. Appending records in description-completion order would corrupt references.
Type records and object IDs already have distinct record kinds. Keep the existing object-count header: it counts objects, not type records. The guest can collect the leading type records until it reads the first object ID, retaining that first ID for subsequent object decoding. Numeric record tags and the exact new payload encoding remain implementation choices to settle before code changes; this proposal specifies their meaning and ordering.
The writer fills the caller-owned byte slice. Buffer exhaustion returns the existing short-buffer error; it does not allocate a replacement mapping. Descriptors are encoded data in shared memory, not live runtime type objects placed at agreed virtual addresses.
6.7 Decode Descriptions, Then Reconstruct Types
The guest must not immediately construct each type as its record arrives. Forward references are valid: record 1 above depends on record 2, which depends on record 3.
read record 1 -> descriptions[1] = Slice(Elem=2)
read record 2 -> descriptions[2] = Struct(Count: TypeID=3, ...)
read record 3 -> descriptions[3] = Builtin(Int)
read remaining type records
read first object ID -> the type prelude has ended
Only then resolve the type graph. A per-ID status distinguishes not started, resolving, and complete. A completed entry returns its stored reflect.Type; a dependency outside the table is an error. A cycle requiring construction of an unfinished dynamic type is unsupported by the standard constructors and must not recurse indefinitely.
For the example:
resolve(1)
needs resolve(2)
needs resolve(3)
types[3] = guest int
types[2] = StructOf([{Name:"Count", Type:types[3], Tag:`json:"count"`}])
types[1] = SliceOf(types[2])
For static records, compute the candidate from the guest's executable type-region base and the encoded offset, then resolve it through validated static-type handling. Matching executable and ABI are required; an arbitrary in-range address is not proof that it is a valid type descriptor. The current DWARF index is not exhaustive, so completing that validation path is an explicit open question rather than an assumed unsafe cast.
Standard reflect constructors perform their own runtime integration and cache insertion. There is no additional public reflect Register step, and host cache IDs or reflectOffs IDs are never installed into the guest runtime.
6.8 Decode Values Using the Reconstructed Table
Once the table is ready, continue with the saved first object ID and the existing object decoding loop:
root.Type:
TypeValue(2)
-> obtain types[2]
-> set root.Type to that reflect.Type value
root.Value:
Interface(DynamicType=2, ...)
-> reflect.New(types[2]).Elem()
-> decode Count = 42
-> set root.Value to the resulting value
root.Items:
Interface(DynamicType=1, ...)
-> restore the slice and backing objects using the existing graph decoder
-> element type is types[1].Elem(), which is types[2]
Repeated references to ID 2 always resolve to the same entry. Independently, repeated ObjectIDs preserve shared objects. Equal types do not imply identical objects, and two separate values of the same type must not be merged.
The guest reconstructs types and values in its own Go runtime. The shared buffer contains their descriptions and data. It is not installed as the runtime type heap. The per-load table keeps references while decoding, and the resulting values and runtime caches retain the types they use.
6.9 Concrete Integration Points in State
| File / existing boundary | Required change |
|---|---|
state.go: Save and Load setup |
Create the per-transfer table, preload discovered types before object traversal, and prepare guest types before decoding values. Preserve caller-owned buffer behavior. |
types.go: encode/decode type databases |
Use one ID allocator and ordered descriptor slots; resolve guest types from records instead of requiring the global name registry. |
encode.go: findType and pending types |
Return the preassigned ID when present; append newly encountered types through the same allocator; emit descriptors by ID. |
encode.go: value dispatch |
Recognize values representing reflect.Type before traversing their private implementation. |
decode.go: Load, findType, interface decoding |
Read the full type prelude, resolve dependencies, then reuse the resulting reflect.Type objects when allocating values. |
object.go: type and object records |
Encode richer type descriptions and the distinct type-value reference through the existing object serialization machinery. |
native.go: executable type lookup |
Supply static-type resolution without conflating type reconstruction with native closure capture discovery. |
memory.go |
Retain the existing byte-slice writer and reader; no alternate transport is needed. |
The cache reader is a Go-version-specific part of this type-handling boundary, not a new public subsystem. No second copy of object graph traversal is introduced.
Removing manual Register as a type-discovery requirement does not automatically remove the semantics of StateSave, StateLoad, field reconciliation, or AfterLoad callbacks. They may describe logical state differently from Go struct fields. Their retention and migration policy must be decided explicitly; they must not be silently bypassed as a side effect of switching type tables.
6.10 Boundary Cases and Concurrency
| Case | Meaning for this design |
|---|---|
| Cache order differs between runs | IDs can differ; each stream carries its own ordered type definitions. |
| Guest cache already contains the type | Constructors may return the existing canonical type; install that result under the stream ID. |
| A value references a type missed by cache enumeration | Add it through the same state type table during graph traversal. |
| A static named type has the same Kind as a builtin | Preserve the static type identity; do not replace it with the builtin. |
| A reflect.Type appears in an interface or map key | Encode the represented type reference, not *reflect.rtype storage. |
| A type record describes a channel or function | Reconstruct the type only; existing limitations on channel and function values still apply. |
| StructOf encounters fields or promoted methods it does not support | Preserve the constructor boundary and report failure; do not silently drop fields or methods. |
| Another goroutine creates types while caches are enumerated | Reads are concurrent map reads, not an atomic inventory; reachable types are supplemented during encoding. |
| Another goroutine mutates the values being saved | This proposal supplies no atomic value snapshot or new STW mechanism; the caller must provide a stable graph for Save. |
| Multiple transfers run concurrently | Their tables and IDs are independent. They read global caches and may populate guest reflect caches through normal constructors. |
Scanning all observed cache types means work and stream size grow with the process's accumulated cache contents, including types unrelated to the current root. Guest reconstruction can also retain those types in reflect's global caches. This cost follows from the selected full-cache discovery approach; a future reachable-only optimization must not be assumed here.
7. Error Handling
| Condition | Behavior |
|---|---|
| Unknown or out-of-range TypeID | Return a decoding error before using it to construct or allocate a value. |
| Truncated stream or insufficient output buffer | Preserve the reader/writer error through the existing state error boundary. |
| Type records appear after object decoding has begun | Reject the stream; all type definitions belong to the prelude. |
| Unsupported dynamic named/interface type or construction cycle | Return an unsupported-type error identifying the relevant record or type. |
| Reflect constructor rejects its arguments | Return a type-reconstruction error; do not substitute a different type. |
| Executable, Go ABI, or static-type reference cannot be validated | Stop before constructing values with that type. |
| A value is incompatible with its referenced type | Return a decoding error; do not force an unsafe assignment. |
Concrete diagnostics and binary tag values are not frozen by this proposal. Validation of shared-memory lengths, type references, and static descriptors belongs at the decoding boundary. Checking an ELF address range alone does not validate a runtime type object.
8. Compatibility
The semantic type record format changes, so encoder and decoder must be updated together. This proposal does not introduce a cross-version stream compatibility layer. The exact format-version treatment is listed as an open question before implementation.
The repository currently targets Go 1.26.6. Its native closure path requires the same non-PIE executable, capture DWARF, and Linux amd64 or arm64. Cache enumeration was investigated separately from that path; its use of private Go symbols is also version dependent. This proposal does not turn a cache enumeration result into proof of full closure transfer support on another platform.
No production sandbox or Sentry API change is required to write the type table into the existing shared-memory buffer. Connecting this refactored internal codec to the production transfer path remains a separate integration step.
9. Alternatives Considered
9.1 Keep Manual State Registration for Every Type
The registration contract requires methods on the registered type. Runtime-created anonymous types cannot supply those methods, so caller-side registration alone cannot implement this feature.
9.2 Allocate Cache IDs Separately from State IDs
This creates two meanings for an ID and requires translation on every type reference. Preloading state's own table removes that duplication.
9.3 Copy Runtime Type Memory or reflectOffs
Runtime descriptors reference other descriptors, names, code, and runtime-managed metadata. reflectOffs is neither a contiguous type region nor a complete registry. Reconstructing standard dynamic types through their constructors gives the guest valid local metadata.
9.4 Number Types Again in Guest Cache Order
Cache iteration order is not stable or shared between processes. The stream's ordered type records define the IDs; guest cache order has no role in decoding.
9.5 Serialize Only Type Names
Type.String is not a unique identity key, and a name does not contain the construction inputs for a dynamic composite type. Named static types also need their original identity and methods, not an anonymous approximation.
10. Testing Strategy
These are acceptance tests for implementation, not checks claimed to have passed for this proposal.
| Test | Required assertion |
|---|---|
| Enumerate all four cache shapes | Find constructor-created pointer, slice, array, map, channel, function, and struct types; visit every hash-bucket entry. |
| Cache-first numbering, reverse value order | Preload B then A, encode A then B, and restore the right types without renumbering. |
| Forward type references | Reconstruct Slice #1 -> Struct #2 -> Int #3 after reading all descriptions. |
| Repeated type identities | All uses of a type share one TypeID; restored Type values equal the dynamic types of restored values. |
| Builtins versus named types | Preserve a static type Count int as distinct from int; preserve aliases according to Go type identity. |
| Seven dynamic constructor categories | Check element/key relationships, array length, channel direction, function parameter order and variadic flag, and struct field metadata. |
| Struct layout | On the supported ABI, verify reconstructed Size, Align, field Offset, tags, and embedded-field status against the source type. |
| Static interfaces and recursive named types | Preserve the original type identity and method set; do not reconstruct them as anonymous types. |
| Type values in interfaces, maps, and repeated fields | Round-trip represented types without walking runtime descriptors. |
| Object graph regressions | Existing alias, cycle, interior-reference, and native-closure tests continue to preserve object relationships. |
| Concurrent cache growth | No unsafe cache access; missed but value-reachable types are added before the stream is written. Do not assert an atomic cache snapshot. |
| Consecutive and concurrent transfers | No cross-transfer TypeID leakage or dependency on prior per-transfer tables. |
| Guest reconstruction and GC | Restored values remain valid after the decode table is released and GC runs. |
| Malformed records and short buffers | Fail through the state error boundary, including invalid IDs, invalid constructors, unsupported cycles, and invalid static references. |
| Existing custom state hooks | Verify the separately agreed retention/migration policy before replacing their lookup path. |
| Linux amd64 and arm64 with the checked Go version | Validate the codec path on both existing native-closure platforms; do not infer either result from cache enumeration alone. |
11. Summary of Changes
| Area | Proposed result |
|---|---|
| Discovery | Read standard reflect caches and supplement their entries with referenced and encountered types. |
| Type identity | One state-owned, per-transfer TypeID table, preloaded before ordinary object traversal. |
| Shared memory | Ordered type descriptions followed by existing object records, written with w.put. |
| Guest reconstruction | Read descriptions first, resolve type dependencies, then decode values. |
| reflect.Type values | Encode references to represented types instead of copying runtime descriptor objects. |
| Register | Remove it as a prerequisite for type lookup; explicitly decide the remaining custom state semantics before deleting related code. |
| Sentry and ixgo | No changes in this increment; later adapters can consume restored guest reflect.Type objects. |
12. Open Questions Before Implementation
- Static descriptor validation. The existing native metadata indexes types found in DWARF, which is not a complete inventory of static types. The static-offset scheme needs a concrete way to accept valid executable types missing from that index while rejecting offsets that are not type descriptors. This proposal does not approve an unchecked cast.
- Custom state hooks. Decide whether StateSave/StateLoad and logical field reconciliation remain supported, and how their metadata shares the single type table. Type reconstruction alone is not authorization to remove those behaviors.
- Strict cache snapshot semantics. Decide whether the requirement is all entries observed during enumeration plus all value-reachable types, or an atomic inventory of all caches. The latter needs coordination of type creation; repeated Range calls do not prove completeness.
- Binary format details. Choose concrete record tags and the incompatible-format handling before editing the codec. Preserve the agreed implicit TypeID ordering; do not add a second numbering scheme.
- Types outside the standard constructors. Classify any ixgo/xtype-created named types before claiming this path covers them. Reconstructing a standard anonymous type does not establish equivalence to a third-party named type.
13. Source References
Repository references are relative to this proposal. Standard-library paths refer to the checked Go 1.26.6 source tree, not an assumed stable private API.
| Source | Evidence used |
|---|---|
internal/state/types.go |
Current ID allocation, positional descriptor registration, global name registry, and custom type reconciliation. |
internal/state/encode.go |
Type lookup, interface encoding, graph traversal before writing, and type records preceding objects. |
internal/state/decode.go |
Descriptor consumption, type resolution, interface allocation, and deferred object reconstruction. |
internal/state/object.go |
Current type descriptions, type specifications, object tags, and binary serialization. |
internal/state/memory.go |
Caller-owned buffer, w.put/r.get, short-buffer and truncated-input behavior. |
internal/state/native.go |
Existing reflect.toType bridge, executable metadata, PC/capture handling, and ABI restrictions. |
src/reflect/type.go: ptrMap, lookupCache, funcLookupCache, structLookupCache |
Cache ownership, entry shapes, constructor metadata, field information, and cache insertion. |
src/reflect/map.go: MapOf, groupAndSlotOf |
Actual map constructor, local hasher construction, and additional cached internal layout types. |
src/reflect/makefunc.go: makeFuncImpl, MakeFunc |
Function signature metadata is separate from callback implementation and environment. |
src/runtime/type.go: reflectOffs |
Offset mappings are not a complete dynamic type registry. |
src/sync/map.go: Range |
Enumeration does not provide a consistent snapshot under concurrent updates. |
- Dominant language
- Go
- Stars
- 0
- Forks
- 1
- Avg merge
- 6h 36m
- Merged PRs (30d)
- 18
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from xgo-dev/sandbox
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100