OvertureMaps / OvertureMaps/schema

PySpark test suite is not self-contained

Open
#585 0 comments 0 reactions 1 assignee View on GitHub

@sethfitz is already working on this.

Since Jul 22, 2026.

Dominant language
Python
Stars
213
Forks
22
Avg merge
3d 3h
Merged PRs (30d)
31

Description

Summary

The uv workspace is meant to give an identical, reproducible test environment,
but the PySpark test suite depends on machine env state in two ways:

  1. Spark/JVM configuration leaks in from the shell. SPARK_HOME and a
    family of related variables are honored by PySpark's launcher and the JVM,
    so a system-wide Spark install or exported JVM flags change or break
    every test session.
  2. The JVM itself is not part of the venv. The pyspark wheel bundles
    Spark's jars and launcher scripts but no Java runtime; the tests silently
    rely on whatever java the machine resolves.

Both are the same underlying problem: the uv venv does not fully own the test
runtime.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.