apache / apache/arrow

Timezone logic on Linux and Mac

Open
#47,347 0 comments 1 reaction 0 assignees View on GitHub
Component: C++ Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

In pyarrow, I've been getting `pyarrow.lib.ArrowInvalid: Cannot locate timezone 'UTC': discover_tz_dir failed to find zoneinfo` despite my timezone appearing to be properly setup. I seem to have tracked down the problem due to a mixup in the timezone discovery logic (introduced by 67850befa604ce27c2ea8f37132a5ccf73fe68d9?)

As of now, [cpp/src/arrow/vendored/datetime/tz.cpp](https://github.com/apache/arrow/blob/67850befa604ce27c2ea8f37132a5ccf73fe68d9/cpp/src/arrow/vendored/datetime/tz.cpp) contains such logic to find zoneinfo:

```
discover_tz_dir()
{
...
# elif !defined(__APPLE__)
CONSTDATA auto tz_dir_default = "/usr/share/zoneinfo";
CONSTDATA auto tz_dir_buildroot = "/usr/share/zoneinfo/uclibc";

// Check special path which is valid for buildroot with uclibc builds
if(stat(tz_dir_buildroot, &sb) == 0 && S_ISDIR(sb.st_mode))
return tz_dir_buildroot;
else if(stat(tz_dir_default, &sb) == 0 && S_ISDIR(sb.st_mode))
return tz_dir_default;
else
throw runtime_error("discover_tz_dir failed to find zoneinfo\n");
# else // __APPLE__
# if TARGET_OS_IPHONE
# if TARGET_OS_SIMULATOR
return "/usr/share/zoneinfo";
# else
return "/var/db/timezone/zoneinfo";
# endif
# else
CONSTDATA auto timezone = "/etc/localtime";
```

As far as I can tell, macOS contains zoneinfo in `/usr/share/zoneinfo` and Linux contains zoneinfo in somewhere like `/etc/zoneinfo` symlinked from `/etc/localtime`. However, according to the above logic, zoneinfo is searched for in `/usr/share/zoneinfo` if the system is NOT Apple (i.e. Linux).

In my Python application, I've managed to workaround this issue for the time being by making a cgroup with `/etc/zoneinfo` bind mounted to `/usr/share/zoneinfo` using bubblewrap.

### Component(s)

Python, C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.