Unexpected Y-axis translation when using `ortho()` and `resetMatrix()`

[!NOTE]
This issue is a repost of https://github.com/processing/processing/issues/6175 by @usuallyannoyed, updated and edited for tone and clarity.

Description

The ortho() function does not behave as expected.

Expected Behavior

ortho(0, width, 0, height) with a cleared modelview matrix (resetMatrix) should allow users to draw within the specified bounds (e.g., 0 to width on the x-axis and 0 to height on the y-axis).

Current Behavior

Instead, the drawing area behaves incorrectly, requiring adjustments (e.g., translating -height on the y-axis) to achieve the expected results. This behavior makes it unintuitive for users who expect the specified bounds to align with the visible viewport.

Steps to Reproduce

void setup() {
  size(500, 500, P3D);
}

void draw() {
  ortho(0, width, 0, height);
  resetMatrix(); // clear modelview matrix
  fill(255);
  
  // Uncomment the line below to 'fix' the behavior
  // translate(0, -height); 
  
  beginShape(TRIANGLES);
  vertex(640, 640);
  vertex(0, 640);
  vertex(0, 0);
  endShape();
}

Observed Result

The triangle only becomes visible when compensating for the unexpected offset by translating the y-axis.

Expected Result

The triangle should appear within the specified ortho bounds without additional transformations.

Environment

• Operating System: macOS 13.4
• Processing Version: 4.3.1

Possible Causes / Solutions

The issue appears to stem from the following block of code in the Processing source:

https://github.com/processing/processing4/blob/937f528c1638fa96bc8fde5a67483dad9de8b5c9/core/src/processing/opengl/PGraphicsOpenGL.java#L4482-L4486

Despite the comment saying that The minus sign is needed to invert the Y axis., this implementation does not fully invert the y-axis as intended. Instead, it flips the y-axis around zero, resulting in all the y-coordinates being offset into negative space.

Suggested Fix

To invert the y-axis properly in normalized device coordinates (NDC), pre-multiply the projection matrix with a -1 scale for the y-axis. This can be achieved by modifying the projection matrix calculation as follows:

// Add a pre-multiplied -1 y scale to flip the y axis in NDC.
    projection.set(x,  0, 0, tx,
                   0, -y, 0, -ty,
                   0,  0, z, tz,
                   0,  0, 0,  1);

This approach ensures the y-axis behaves as expected without requiring additional transformations.

Additional comments

This behavior stems from Processing's choice to have the Y-axis increase downward, consistent with traditional computer graphics. While reasonable, the implementation relies on ad-hoc adjustments throughout the code. Centralizing it at a lower level (NDC or viewport) and exposing it as an optional flag would provide more clarity and flexibility.

Beginne mit der Reproduktionsskizze und prüfe den Code für die Projektionsmatrix in core/src/processing/opengl/PGraphicsOpenGL.java im Bereich der Zeilen 4482–4486. Achte dabei insbesondere darauf, wie ortho() mit resetMatrix() interagiert. Überprüfe das Verhalten mit dem angegebenen Processing 4.3.1-Setup und vergleiche die gerenderten Koordinaten mit den angeforderten Grenzen. Als erledigt gilt die Aufgabe, wenn das Dreieck ohne die ausgleichende y-Verschiebung innerhalb der ortho-Grenzen erscheint.