google-research / google-research/tabfm

JAX and PyTorch constructors disagree on identical config: same value builds on one backend, raises on the other

Open
#95 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.6k
Forks
270
Avg merge
1d 7h
Merged PRs (30d)
1

Description

Following on from #92 (PyTorch constructor) — I ran the same matrix against the JAX constructor, and the two backends disagree on identical inputs, in both directions.

Both at `0`, on `b15593e4c1111ddb5f4f30dd2957df2edbaa04ca`:

| parameter | PyTorch | JAX |
|---|---|---|
| `embed_dim` | builds OK | `ZeroDivisionError` |
| `max_classes` | builds OK | `ZeroDivisionError` |
| `row_num_cls` | builds OK | `ZeroDivisionError` |
| `ff_factor` | builds OK | `ZeroDivisionError` |
| num frequencies | builds OK | `ZeroDivisionError` |
| `col_num_blocks` | builds OK | builds OK |
| `feature_group_size` | builds OK | builds OK |

And the asymmetry reverses for negatives — `feature_group_size=-1` **builds a model on JAX**, while PyTorch raises `RuntimeError: zeros: Dimension size must be non-negative`.

So the same `config.json` can produce a working model on one backend and an exception on the other, and which one is stricter depends on the parameter. Since `convert_and_upload.py` converts a JAX checkpoint to PyTorch and runs a parity check, I think the intent is that the two agree.

The `-1` failures on the JAX side are also worth a look on their own — `col_num_blocks=-1` surfaces as `MLIRError: Invalid type`, which is a long way from telling the caller that a block count cannot be negative.

None of this bites the default shapes. It matters if you consider the two backends interchangeable, because right now a configuration that is valid on one is not necessarily valid on the other.

I would guess the fix is the same shared validation suggested in #92 and #94, applied once and used by both constructors, rather than patching each backend separately — but that is your call on the architecture.

Disclosure: I used an AI assistant to help find this. I ran both matrices myself.

Contributor guide

Open the contributing guide

Research direction

Compare the JAX and PyTorch constructors discussed in the issue, then inspect convert_and_upload.py and its parity check. Reproduce the listed zero and negative-value matrices against the referenced commit, and verify that both constructors consistently accept or reject each configuration with clear errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.