hPutBuf/writeFile fails on OSX for bytestrings over ~2^31 bytes in size
- Dominant language
- Haskell
- Stars
- 301
- Forks
- 144
- Avg merge
- 7d 22h
- Merged PRs (30d)
- 1
Description
Writing more than ~2GB at once to a file appears to fail on OSX. This may be related to a platform issue also encountered in this python ticket: https://bugs.python.org/issue24658
## Repro:
The following code fails on OSX, emitting an exception:
```module Main where
import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as LBS
main :: IO ()
main = do
bigBS <- pure $ BS.replicate (2^32) 1
BS.writeFile "testFile" bigBS
```
`*** Exception: testFile: hPutBuf: invalid argument (Invalid argument)`
## Alt approaches
Manually creating a file handle and using hPut directly makes no difference
```module Main where
import System.IO
import qualified Data.ByteString as BS
main :: IO ()
main = do
bigBS <- pure $ BS.replicate (2^32) 1
fileHandle <- openFile "testFile" WriteMode
BS.hPut fileHandle bigBS
```
Writing a lazy bytestring also makes no difference
```
module Main where
import System.IO
import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as LBS
main :: IO ()
main = do
bigBS <- pure $ BS.replicate (2^32) 1
fileHandle <- openFile "testFile" WriteMode
LBS.hPut fileHandle (LBS.fromStrict bigBS)
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.