Problems reading GZIPPED file with libz-sys > 1.1.5 and zlib-ng-compat

Problems reading GZIPPED file with libz-sys > 1.1.5 withzlib-ng-compat.
With zlib and it works fine. With libz-sys == 1.1.5andzlib-ng-compat` it also works fine.

See: https://github.com/pola-rs/polars/issues/3895

Problematic gzipped CSV file: https://temp.aertslab.org/.tsv/atac_fragments.head40000000.tsv.gz

import polars as pl

df = pl.read_csv(
    'atac_fragments.head40000000.tsv.gz',
    skip_rows=52,
    has_headers=False, 
    sep="\t",
    use_pyarrow=False
)

Cargo.toml

[package]
name = "polars_read_csv"
version = "0.1.0"
edition = "2021"


[dependencies]
polars-core = { path = "../polars/polars/polars-core", default-features = false }

[features]


[dependencies.polars]
path = "../polars/polars"
default-features = false
features = [
  "simd",
  "strings",
  "csv-file",
  "performant",
  "decompress-fast",
]


[profile.release]
target-cpu = "native"
debug = 1

src/main.rs

use polars::prelude::*;

fn main() -> Result<()> {
    let df = CsvReader::from_path(
        "atac_fragments.head40000000.tsv.gz",
    )?
    .with_delimiter(b'\t')
    .with_skip_rows(52)
    .finish()?;

    println!("{:?}", df.shape());
    println!("{}", df.tail(Some(30)));

    Ok(())
}

Get polars:

cd ..
git clone --maxdepth 100 https://github.com/pola-rs/polars

Ttry dffirent flate2 backends in polars-core by changing: decompress-fast

❯ git diff polars/polars-io/Cargo.toml
diff --git a/polars/polars-io/Cargo.toml b/polars/polars-io/Cargo.toml
index 3461c37e7d..a39b89b06c 100644
--- a/polars/polars-io/Cargo.toml
+++ b/polars/polars-io/Cargo.toml
@@ -28,7 +28,9 @@ dtype-categorical = ["polars-core/dtype-categorical"]
 csv-file = ["csv-core", "memmap", "lexical", "polars-core/rows", "lexical-core"]
 fmt = ["polars-core/fmt"]
 decompress = ["flate2/miniz_oxide"]
-decompress-fast = ["flate2/zlib-ng-compat"]
+#decompress-fast = ["flate2/zlib"]
+#decompress-fast = ["flate2/zlib-ng"]
 temporal = ["dtype-datetime", "dtype-date", "dtype-time"]
 partition = ["polars-core/partition_by"]
 # don't use this

# Compile each version with differnt decompress-fast settings:
cargo build --release

# with "flate2/zlib-ng-compat"
$ target/release/polars_read_csv 
Error: ComputeError("invalid utf8 data in csv")

# with "flate2/zlib-ng"
$ target/release/polars_read_csv 
Error: ComputeError("invalid utf8 data in csv")

# with "flate2/zlib"
$ target/release/polars_read_csv
(39999947, 5)

Start with the Cargo.toml feature configuration and src/main.rs reproduction, using the linked gzipped TSV file. Build with each flate2 backend and compare the reported CSV error or resulting shape; done means the problematic file reads correctly without invalid UTF-8 data.