Floating point / HDR texture support

Processing is tightly coupled to ARGB 8-bit textures via the PImage class hierarchy. When the project started, the only crossplatform drawing lib available in the JDK was AWT/Java2D. Those APIs expose pixels as int in ARGB (TYPE_INT_ARGB/TYPE_INT_RGB), and java.awt.Color wraps 0–255 channels. That layout provides zero copy access, blits via MemoryImageSource, and compatibility with other AWT stuff that's used pervasively throughout the codebase.

Every part of Processing assumes pixel data is stored as 32‑bit ints with 8‑bit channels. That assumption is baked into our public API (the "pixels" array) and is all over the place inside PGraphics, image filters, tessellators, OpenGL and future WebGPU backends, and font/shape caches, etc. Because there is no abstraction around "pixel format" or "color storage", any attempt to add floating point textures would require touching practically every file, amounting in an effective soft-fork.

Deep coupling

Here are some more specific issues:

• int[] pixels is exposed as public state and could be used from user sketches all the way down to renderer internals. Every consumer assumes 4 bytes per pixel with the same layout. Public fields are particularly problematic in Java as they can't really be overridden, just shadowed.
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/core/PImage.java#L92
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/core/PApplet.java#L222
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PGraphicsOpenGL.java#L5379-L5386

• Color math is hardcoded to obtuse bit manipulation and 0–255 ranges eg colorCalc packs/unpacks ints, PShape duplicates this, and image blending/filtering is doing lots of >>> 24 & masks.
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/core/PGraphics.java#L7631-L7638
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/core/PGraphics.java#L7680-L7735
  https://github.com/processing/processing4/blob/main/core/src/processing/core/PShape.java#L3514
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/core/PImage.java#L2471-L2480

• GPU texture uploads/downloads are always int[] → IntBuffer → GL_RGBA/GL_UNSIGNED_BYTE. See Texture.set/setNative, the texSubImage2D(...PGL.RGBA, PGL.UNSIGNED_BYTE...) calls, the rgbaPixels conversions that only handle ARGB/RGB/ALPHA ints and the updatePixelBuffer IntBuffer path.
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/Texture.java#L303-L305
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/Texture.java#L346-L347
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/Texture.java#L1000-L1061
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/Texture.java#L792-L795

• Moving data between CPU pixels and GPU buffers is just "reinterpret int as native RGBA". readPixels() reads RGBA/UNSIGNED_BYTE and immediately calls PGL.nativeToJavaARGB. The reverse path (drawPixels) calls PGL.javaToNativeARGB before writing to the FBO.
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PGraphicsOpenGL.java#L5389-L5411
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PGL.java#L1646-L1653
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PGraphicsOpenGL.java#L5437

• GL resources themselves are fixed to 8‑bit RGBA, renderbuffers/textures are always allocated as RGBA8, and vertex colors/materials are uploaded as UNSIGNED_BYTE attributes.
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/FrameBuffer.java#L392
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PGL.java#L2854
  https://github.com/processing/processing4/blob/11f1a1e666c7cec6ddf483c5c8fffb5b9009a137/core/src/processing/opengl/PShapeOpenGL.java#L5517-L5520

• Even our new WebGPU backend just forwards the existing fillR/fillG/fillB/fillA values that originate from the same 8‑bit colorCalc method flow, so it inherits every limitation and would otherwise have to reimplement basically all of the existing API.
  https://github.com/processing/processing4/blob/1ee012e7942f0cfa9029151c6d7a1b992c044d0c/core/src/processing/webgpu/PGraphicsWebGPU.java#L56

Why is this an issue / What this limits

At a high level, while 8-bit textures may have been the norm in the early 2000s, the assumption has basically flipped. Modern GPUs execute most shading math in 32-bit lanes regardless of the source format, so even when sampling from an 8-bit texture the ALUs are still getting their parallelism over 32-bit registers. Unless you're explicitly using packed instructions (which is rare for generalist graphics work), the only real thing sthat 8-bit gets is memory bandwidth, which mostly only matters on low-end mobile. Everywhere else the industry is moving toward mixed precision (e.g. FP16 tensor cores for ML matmul ops) because compute cost is no longer tied to per-channel bit depth.

However, specifically relevant to art, the lack of floating point textures is severe limitation to the following techniques.

Installation art

Stuff like motion tracking, heatmap accumulation, or any "paint with time" piece depends on adding and accumulating tiny deltas every frame. In 8-bit those changes round away, so small movement never shows up and trails die instantly. Float buffers let artists integrate small movement over minutes or hours and only quantize when sending to the projector. Multi-projector setups also have problems, feathering overlaps, gamma correcting different units, and warping/projection mapping content through multiple correction stages all require smooth ramps. Quantizing each stage to 256 buckets produces visible steps, which is especially problematic in dark rooms typical for these kinds of installs. Hardware sensorb ased work (think depth cameras, environmental sensors, etc) typically rely on filters and other post-processing to be useful for producing a final render for the projector.

Generative art

Feedback is super common in generative art, ie read the last frame, do some texture processing, write it back, repeat. In 8-bit every read-modify-write cycle loses precision, so the loop collapses after a few iterations and colors wander off due to rounding and look bad if they work at all. Float textures keep temporal buffers alive and stable which is why applications like TouchDesigner or vvvv typically default to them. Popular algos like reaction-diffusion, fluid sims, and cellular automata also add/subtract tiny gradients every step. Quantization steps in 8-bit makes patterns fall apart. Floats match the reference equations. Even simple particles need floats to ensure the motion looks and particle sim parameters work from frame to frame. Layering hundreds of low-alpha sprites in 8-bit will jump between "invisible" and "too strong" because the intermediate alpha values simply don't exist. For people who care about color grading or LUT work, esp to reference stuff like mockups made by designers which is important in professional settings working on a team, you need headroom. Doing the math in 8-bit crushes blacks and create midtone bands which will make your designers cry.

Modern 3D graphics techniques

In the more game oriented world, TAA and other AA techniques assume you can accumulate subpixel jitter over many frames without the values collapsing. When you use 8-bit components, the first couple of blends push dark tones to zero and you're left with banding or ghost trails. Modern engines accumulate in 16/32-bit floats and only clamp when tonemapping out to the display. The same is true for HDR stuff like bloom, tone mapping, or PBR shading that need to push energy past 1.0, do postprocessing, then compress the result. With 0-255 you just clip to white and the math breaks. Deferred renderers and screenspace effect passes also suffer when encode normals, roughness, or depth in 8-bit and SSAO/SSR immediately reveal the quantization steps. Backends like Metal/Vulkan assume you can request RGBA16F, R11G11B10F, depth-only attachments, etc. which the user may want to display / debug. There's interesting Processing style techniques here that simply aren't possible atm.

Conclusion

Changing Processing's texture format and color math is a breaking change at basically all levels of the existing codebase but is crucially important for modern graphics techniques.

Commencez par retracer les hypothèses concernant les pixels et les couleurs dans PImage.java, PGraphics.java, PShape.java, PGraphicsOpenGL.java, Texture.java, PGL.java, FrameBuffer.java, PShapeOpenGL.java et PGraphicsWebGPU.java. L’issue ne spécifie ni conception concrète, ni limite d’implémentation, ni test ; le travail ne serait terminé qu’une fois qu’une approche définie prendrait en charge les formats à virgule flottante ou HDR sur tous les chemins concernés sans casser l’API existante.