rust-lang / rust-lang/libs-team
ACP: give access to the inner iterator of `std::iter::Peekable`
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 178
- Forks
- 28
- Avg merge
- 15m
- Merged PRs (30d)
- 1
Description
Proposal
Problem statement
Some iterators carry some additional state that's not encompassed by the Iterator trait, e.g. an additional method or a field, but after wrapping them in Peekable, there's no more access to anything in them besides their Iterator impl.
Motivating examples or use cases
There's a Lexer struct, that implements Iterator by handing out Lexeme objects. One way to return lexing errors could be to yield Result<Lexeme, LexingError> via its Iterator impl, but that makes it very inconvenient to create processing pipelines out of this iterator, having to check for an error on every iteration. Instead, the Lexer struct yield just Lexeme, and stores the error in itself, which is to be retrieved after exhausting the iterator with Lexer::finish
struct LexingError { /* ... */ }
struct Lexeme { /* ... */ }
struct Lexer {
res: Result<(), LexingError>,
// ...
}
impl Iterator for Lexer {
type Item = Lexeme;
fn next(&mut self) -> Option<Self::Item> {
todo!()
}
}
impl Lexer {
fn finish(self) -> Result<(), LexingError> {
self.res
}
}
It could be useful to wrap the Lexer in Peekable to create a parser that tries various paths depending on the next token, but then one wouldn't be able to retrieve the error, and have to opt for the clumsier approach with type Item = Result<Lexeme, LexingError>;
Solution sketch
impl<I> Peekable<I> {
fn inner(&self) -> &I;
fn into_inner(self) -> I;
}
No inner_mut as it'd allow for scrambling the order of the items yielded by the peekable iterator
Alternatives
-
Don't put any additional state into iterators that are to be made
Peekable
That's not always possible since one doesn't always have control over how the state is stored by various objects. -
Create your own
Peekablethat allows for accessing the inner iterator
That's what I did in my own project, but this approach leads to a ton of boilerplate & likely won't allow for an implementation as performant & flexible as the one provided by the standard library
Links and related work
std::path::Components, std::path::Iter & std::str::Chars all have methods that return the remainder of the data to be processed.
What happens now?
This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.
Possible responses
The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):
- We think this problem seems worth solving, and the standard library might be the right place to solve it.
- We think that this probably doesn't belong in the standard library.
Second, if there's a concrete solution:
- We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
- We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the std::iter::Peekable API context, the Lexer::finish motivating example, and the linked feature lifecycle. No repository files or tests are named; completion depends on libs-api review of the proposed API and a later implementation decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100