Hyper-parameters to reproduce DeltaNet results are in the paper but not in the repo

Hello,

First of all, thanks a lot to the authors for making this repository publicly available---I found this very helpful!

I'm leaving a note here in case it's useful to others interested in reproducing the (DeltaNet) results from the paper.

The paper’s appendix provides a clear set of hyperparameters (another big thanks to the authors), but these are not reflected in the example commands in the README.

In my attempt, running the commands in the README file as-is did not reproduce the modular arithmetic results reported in the paper. *However*, using the hyperparameters listed in the appendix did work.

For instance, the appendix indicates that a successful modular arithmetic configuration for DeltaNet uses 3 layers (compared to 2 for the parity task), and a batch size of 1024. These are not mentioned in this repo.

I'm not submitting a pull request since I've been working with a different model implementation, but I thought it might still be useful to share this here in case somebody else is having a problem reproducing results.

PS: as a side note, in my experience with a model closely related to DeltaNet (but slightly different) implemented using `fla` (https://github.com/fla-org/flash-linear-attention). I found that setting `initializer_range` (an `fla` model parameter) to 0.006 (rather than the default 0.02) improved stability. Also, training for 20,000 steps (with batch size 1024) was sufficient to solve modular arithmetic (without brackets), as opposed to the 100,000 steps reported in the paper.

Read the README example commands and compare their DeltaNet modular-arithmetic settings with the hyperparameters listed in the paper appendix, especially the layer count and batch size. Update the relevant README commands so the documented configuration matches the reproducible setup, then verify that the commands reflect the reported modular-arithmetic results.

Written by the indexing model from the issue text.