[Ahmottie]

Problem

FileIO.readFileGZip copied the entire decompressed file into memory before returning the first line. Memory use grew with the uncompressed file size, and files larger than 2 GB uncompressed failed with System.IO.IOException: Stream was too long.

Verification on real data

SRR001666_1.fastq.gz from ENA. Normal size and concatenated three times (2.77 GB uncompressed, 84,572,016 lines):

Memory before reading: 240 MB
First line after 0.0 s, peak memory 241 MB
Read 84572016 lines in 18.6 s, peak memory 786 MB