decompressobj inefficiency and work-around, patch included.

Greetings!

I'm working with a network protocol which is essentially a zstandard-compressed stream of newline-delimited lines of text. A Twisted protocol for receiving this data is given a slice of compressed data which can span frames. It's simple to feed chunks of data to decompressobj.decompress until it yields uncompressed data, but it's not clear how to determine how much trailing input data didn't contribute to the uncompressed output and should be processed with a new instance of decompressobj. The only solution I've found in terms of python-zstandard 0.8.1 has been to feed data to decompressobj.decompress one byte at a time and roll to a new decompressobj each time uncompressed data is produced. This is pretty slow. If I've missed something, I'd appreciate advice.

Meanwhile, I've privately replaced decompressobj.decompress with a new function decompressobj.decompress2 and re-implemented decompressobj.decompress as a trivial wrapper around this new function. The new function returns a tuple consisting of (1) the uncompressed result and (2) the value of input.pos before the final call to ZSTD_decompressStream. Using this interface, my protocol's dataReceived method looks like this:

    def dataReceived(self, bytes):
        decompressed, remaining = self.__dobj.decompress2(bytes)
        if len(decompressed) > 0:
            self.__json_receiver.dataReceived(decompressed)
            self.__dobj = self.__dctx.decompressobj()
            if remaining > 0:
                self.dataReceived(bytes[remaining:])

If I haven't missed something in the current python-zstandard API, would you consider a solution like mine for inclusion in the next release? I'm attaching the delta to decompressobj.c for reference.

Thanks!

decompressobj.c.diff.txt

Esamina decompressobj.c e il file allegato decompressobj.c.diff.txt, iniziando dall’implementazione esistente di decompressobj.decompress e dalle relative chiamate a ZSTD_decompressStream. Riproduci il caso di stream segnalato con chunk che attraversano più frame, quindi conferma che il comportamento proposto della posizione di input preservi l’API esistente e consenta ai chiamanti di identificare l’input rimanente.