<fstream>: seekg(tellg()) fails for text files containing unix line endings, <cstdio> likely also affected
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 11.1k
- Forks
- 1.7k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 22
Description
Describe the bug
When opening a text file stream containing Unix line endings (\n instead of \r\n) a supposedly no-op seek leads to discarding characters due to buffering.
In extension a seek to a previously saved location runs into the same problem.
Debugging into the C stdlib shows that the "un-conversion" of \n happens in ftell/fgetpos already, where the converted buffer is searched for \n characters and makes the assumption that each of them was exactly \r\n in the underlying file.
FTR: I discovered this in my custom implementation of std::file_buf based on cstdio primitives
Command-line test case
C:\Temp>type repro.cpp
#include <fstream>
#include <iostream>
int main(){
const char* filename = "foo.txt";
{
std::ofstream f(filename, std::ios::binary);
f << "Line 1\n";
f << "Line 2\n";
f << "Line 3\n";
}
std::fstream f(filename);
std::string line1, line2, line3;
getline(f, line1);
std::cout << "1: '" << line1 << "'\n";
f.seekg(f.tellg()); // Problem line
getline(f, line2);
std::cout << "2: '" << line2 << "'\n";
getline(f, line3);
std::cout << "3: '" << line3 << "'\n";
}
C:\Temp>cl .\repro.cpp
C:\Temp>.\repro.exe
1: 'Line 1'
2: 'ne2'
3: 'Line 3'
Expected behavior
No characters are omitted.
STL version
Visual Studio 2019 version 16.9.1
Additional context
I checked other stdlibs for similar issues. On libcxx f.sync() leads to the same behavior when using \r\n line endings and libstdcxx has the same behavior when using f.seekg(f.tellg()); and \r\n line endings. A similar case in libstdcxx occurs when switching from Read to Write at the mentioned line where the write would lead to "LiNewline" (i.e. 2 chars too late)
And finally on MinGW _ftelli64 and ftello/ftell returned different values for the current file position, likely due to the same newline (mis-)handling
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the repro.cpp command-line test case and run it against the reported Visual Studio 2019 16.9.1 implementation. Inspect the fstream and cstdio seek, tell, and buffering behavior around Unix line endings, then verify that seekg(tellg()) does not omit characters and that the related cstdio behavior is understood.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100