IronLanguages / IronLanguages/ironpython3

IronPython.SQLite not working on macOS/Linux

Open
#1,483 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

linux macOS
Dominant language
C#
Stars
2.8k
Forks
316
Avg merge
1d 9h
Merged PRs (30d)
1

Description

While testing some PyPI packages on a recent ipy 3.4 I have discovered that the sqlite3 package bundled with IronPython is not working correctly. After looking into the source code and some debugging I've discovered two major issues ans a few smaller ones.

Major issue #1 is that there is only one "platform compatibility" layer: os_win_c.cs, which, according to the embedded docs is Windows-specific. Nevertheless, it is being unconditionally compiled on all platforms. The compilation succeeds, but the code compatibility alternatives are limited to Windows 95, Windows CE, Windows RT, Windows Phone, Silverlight, and Windows NT (maybe I have missed some other Windows variant...). When run on macOS/Linux, the compatibility methods run out of all platform detection checks and return an error. I can't see how it ever was working on macOS/Linus in any serious way. When I hacked the code to treat macOS as WinNT, it went a little further, but still hit another snag.

Major issue #2: the code uses .NET API that is not supported on macOS. Warnings about that have been recently disabled (#1330). This pertains file locking, which is essential functionality, effectively making sqlite3 unable to open any file. An in-memory database should work though.

The warning itself mentions only problems on macOS, so presumably Linux should be OK (after some hacks). I haven't tested it on Linux yet, but if I find no more snags, I will submit the changes to get at least this platform supported. There is also a possibility to use Mono.Unix compatibility layer on macOS to get partial file locking, but I am not sure if this is fully implemented; the Mono documentation is lacking.

On a wider note, the SQLite implementation that IronPython uses looks very old, unmaintained, supporting obsolete platforms, and not supporting the platforms that IronPython and .NET currently target. So I wonder whether instead of spending time trying to get this code to work, it wouldn't be better to replace it with a modern implementation that is actively maintained (like SQLite.raw).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the SQLite platform compatibility layer in os_win_c.cs and the file-locking warning referenced by issue #1330. Reproduce the sqlite3 behavior on macOS and Linux, then determine whether the existing implementation can support both platforms or needs replacement; done means file-backed SQLite works on the supported platforms.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, sqlite
Domain
databases, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.