oxidecomputer / oxidecomputer/serde_tokenstream
Serde -> TokenStream serializer
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 77
- Forks
- 3
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 11
Description
Hi!
Have you considered implementing a reverse operation: serialization to TokenStream as part of this library?
Having a round-trip operation could be very useful in some contexts: one is a wrapper around macros using deserialization with serde_tokenstream. Because Rust macros are expanded recursively, one could write a macro such as:
#[simpler_api(param = "value")]
struct S(...);
that could expand to:
#[complex_api(
param1 = 42,
param2 = ["v", "a", "l", "u", "e"],
)]
struct S(...);
Of course, generating the parameters could be done manually, but having a complete round-trip operation as part of a single library could enable complex macro shenanigans. For instance, the macro_magic crate enables exporting tokens at one location, and then receiving them by a proc macro at completely different place (even across file and crate boundary). This is done with generation of decl macros doing callbacks to proc macros with a token tree. Combining this idea with de/serialization to valid token trees, we could have macros emit data-objects to be consumed by other proc-macros, effectively enabling sound communication across proc-macros invocation.
Collapsed: example data-macros expansion.
This is how one may implement data passing across various proc macros by using serialization to a TokenStream. In this example we do just a simple analysis that could be done using existing macro_magic: dump_analysis proc macro generates some data that is consumed by another proc-macro handler.
// A call such as
#[dump_analysis]
pub struct ApiCall {
field: u64,
}
// would generate a decl-macro like:
macro_rules! __api_call__data {
($callback:path, $extra:tt) => {
$callback!( {$extra},
// Data serialized with serde_tokenstream::serialize
{ n_fields = 1, field_names = [ field ] }
}
}}
}
// A user writes a handler like
#[handler(path::to::ApiCall)]
pub fn handle(request: path::to::ApiCall) { ... }
// which, first, expands to:
path::to::__api_call__data!{
handler_inner,
{pub fn handle(request: path::to::ApiCall) { ... }}
}
// this expands again to
handler_inner! {
{pub fn handle(request: path::to::ApiCall) { ... }},
{ n_fields = 1, field_names = [ field ] }
}
// At which point, implementation of handler_inner uses serde_tokenstream::deserialize to get the data back.
Have you thought about implementing this? Alternatively, would you accept such a contribution?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing serde_tokenstream deserialization API and its proc_macro/proc_macro2::TokenStream support. Define the intended reverse operation and round-trip behavior from the issue's examples, then verify that serialized data can be consumed by deserialize in proc-macro contexts. Done means the library provides an accepted serialization contribution with coverage for the described token-tree data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100