Confirm the `ReadableBuffer.writeTo()` and `WritableBuffer.readFrom()` contracts
- Dominant language
- Java
- Stars
- 4.1k
- Forks
- 2k
- Avg merge
- 3d 56m
- Merged PRs (30d)
- 48
Description
**Jetty version(s)**
13.0.x
subtask of https://github.com/jetty/jetty.project/issues/14865
**Enhancement Description**
`ReadableBuffer.writeTo()` and `WritableBuffer.readFrom()` are the only ways to bridge the abstraction with the NIO buffers.
Since there are many use-cases for this bridging, the contract must be carefully tailored to fit them all.
#### The default implementation
All implementations of `ReadableBuffer` and `WritableBuffer` must implement `writeTo()` and `readFrom()` respectively, and support at least the default `Target` and `Fount` interfaces:
```java
public interface ReadableBuffer extends Retainable
{
/**
* @return the number of bytes written
/*
long writeTo(Target target) throws IOException;
interface Target
{
/**
* @param byteBufferToWrite the buffer to be written
*/
void write(ByteBuffer byteBufferToWrite) throws IOException;
}
}
```
With the above contract, the `nio.ByteBuffer`s abstracted by the `ReadableBuffer` instance are iterated to call `Target.write()`. As soon as a `write()` call does not fully consume all remaining bytes of the `nio.ByteBuffer`, the iteration stops and `writeTo()` returns the number of consumed bytes.
```java
public interface WritableBuffer extends Retainable
{
/**
* @return the number of bytes read
/*
long readFrom(Fount fount) throws IOException;
interface Fount
{
/**
* @param byteBufferToReadInto the buffer to read into
* @return true if EOF was reached while reading, false otherwise
*/
boolean read(ByteBuffer byteBufferToReadInto) throws IOException;
}
}
```
With the above contract, the `nio.ByteBuffer`s abstracted by the `WritableBuffer` instance are iterated to call `Fount.read()`. As soon as a `read()` call does not produce at least one byte or returns true, the iteration stops and `readFrom()` returns the number of produced bytes, or -1 if that number was 0 and `read()` returned true indicating EOF was reached.
The above contract must:
- be usable in all cases requiring nio buffers: reading and writing from/to TCP/UDP sockets, transformation from cleartext to cipher bytes and vice-versa, inflating and deflating...
- be implementable for all `ReadableBuffer` / `WritableBuffer` implementations: backed by a single buffer, aggregating backed by multiple buffers, accumulating backed by multiple buffers, backed by a NIO FileChannel...
#### The optimized implementations
Two optimizations have been identified: gathering writes/scattered reads and `FileChannel.transferTo()`. These can be introduced by sub-interfacing `Target` or `Fount`. Here is the `Target` for example:
```java
interface GatheringTarget extends Target
{
void write(ByteBuffer[] byteBuffersToWrite) throws IOException;
}
interface TransferToTarget extends Target
{
long write(FileChannel channelToTransferFrom) throws IOException;
}
```
then the `ReadableBuffer.writeTo()` and `WritableBuffer.readFrom()` supporting such optimizations could test if the passed target supports them too, or fallback to the default `Target` contract otherwise:
```java
class FileChannelReadableBuffer implements ReadableBuffer
{
private FileChannel channel;
public long writeTo(Target target) throws IOException
{
if (Target instanceof TransferToTarget ttTarget)
{
return ttTarget.write(channel);
}
// fallback: loop over filling a ByteBuffer and calling Target.write()
}
}
```
```java
SocketChannel destination = ...;
ReadableBuffer backedByFileChannel = ...;
backedByFileChannel.writeTo(new TransferToTarget()
{
public long write(FileChannel channelToTransferFrom) throws IOException
{
return channelToTransferFrom.transferTo(position(), remaining(), destination);
}
public void write(ByteBuffer byteBufferToWrite) throws IOException
{
destination.write(byteBufferToWrite);
}
});
```
The above contract must:
- support all optimizations we want to have: gathering writes, scattering reads, FileChannel.transferTo()
- be implementable for all `ReadableBuffer` / `WritableBuffer` implementations: backed by a single buffer, aggregating backed by multiple buffers, accumulating backed by multiple buffers, backed by a NIO FileChannel...
#### Helpers and sibling implementations
Are there enough similarities between `Target` and `Fount` implementations that a standard set of helpers should be provided?
`readFrom()` and `writeTo()` throw `IOException`, which is a natural thing when dealing with real IO (actually reading and writing from/to a socket) but less so when doing TLS encryption/decryption and compression/decompression.
Should a pair of sibling methods be added that have an identical behavior but do not throw? I.e.:
```java
long mapFrom(Fount);
long mapTo(Target);
```
Should we also provide methods that take one (or many) arguments to pass to the `Target` / `Fount` lambda to avoid having to create allocating lambdas? I.e.:
```java
long writeTo(OneArgTarget target, T arg) throws IOException;
long writeTo(TwoArgTarget target, TA argA, TB argB) throws IOException;
```
Contributor guide
Research direction
Start with the ReadableBuffer and WritableBuffer interfaces and inspect every implementation, including NIO ByteBuffer and FileChannel-backed variants. Confirm contracts that cover partial consumption, EOF, gathering writes, scattered reads, and transferTo optimizations, with tests demonstrating the required behavior across the listed socket, TLS, compression, and file use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100