microsoft / microsoft/STL

<fstream>: seekg(tellg()) fails for text files containing unix line endings, <cstdio> likely also affected

Open
#1,784 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
11.1k
Forks
1.7k
Avg merge
4d 15h
Merged PRs (30d)
22

Description

Describe the bug
When opening a text file stream containing Unix line endings (\n instead of \r\n) a supposedly no-op seek leads to discarding characters due to buffering.
In extension a seek to a previously saved location runs into the same problem.

Debugging into the C stdlib shows that the "un-conversion" of \n happens in ftell/fgetpos already, where the converted buffer is searched for \n characters and makes the assumption that each of them was exactly \r\n in the underlying file.

FTR: I discovered this in my custom implementation of std::file_buf based on cstdio primitives

Command-line test case

C:\Temp>type repro.cpp
#include <fstream>
#include <iostream>
int main(){
  const char* filename = "foo.txt";
    {
        std::ofstream f(filename, std::ios::binary);
        f << "Line 1\n";
        f << "Line 2\n";
        f << "Line 3\n";
    }
    std::fstream f(filename);
    std::string line1, line2, line3;
    getline(f, line1);
    std::cout << "1: '" << line1 << "'\n";
    f.seekg(f.tellg()); // Problem line
    getline(f, line2);
    std::cout << "2: '" << line2 << "'\n";
    getline(f, line3);
    std::cout << "3: '" << line3 << "'\n";
}

C:\Temp>cl .\repro.cpp

C:\Temp>.\repro.exe
1: 'Line 1'
2: 'ne2'
3: 'Line 3'

Expected behavior
No characters are omitted.

STL version
Visual Studio 2019 version 16.9.1

Additional context
I checked other stdlibs for similar issues. On libcxx f.sync() leads to the same behavior when using \r\n line endings and libstdcxx has the same behavior when using f.seekg(f.tellg()); and \r\n line endings. A similar case in libstdcxx occurs when switching from Read to Write at the mentioned line where the write would lead to "LiNewline" (i.e. 2 chars too late)
And finally on MinGW _ftelli64 and ftello/ftell returned different values for the current file position, likely due to the same newline (mis-)handling

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the repro.cpp command-line test case and run it against the reported Visual Studio 2019 16.9.1 implementation. Inspect the fstream and cstdio seek, tell, and buffering behavior around Unix line endings, then verify that seekg(tellg()) does not omit characters and that the related cstdio behavior is understood.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.