video_encode_cosmos.py pipeline broken: stale cosmos1 import, numpy/protobuf version conflicts, semadedup/semdedup typo

Summary

Following the curation env setup in the README exactly, the video-encoding → semantic-dedup pipeline (curation/video_encoding/video_encode_cosmos.py → curation/semdedup/*.py) does not run as released. I hit four separate problems while trying to reproduce it; filing them together since they're all on the same code path. Environment: python==3.11.0, installed per README's mamba create -n curation ... + pip install -r requirements.txt + pip install -e ./dlimp + pip install -e ..

---

1. cosmos1.models.tokenizer.inference.video_lib no longer exists upstream

video_encode_cosmos.py and video_encode_cosmos_hdf5.py both do:

from cosmos1.models.tokenizer.inference.video_lib import CausalVideoTokenizer

NVIDIA/Cosmos (the repo this import path implies) has since been repurposed for "Cosmos 3" — current main has no cosmos1/ directory at all, and the only tag is Cosmos3. The old "Cosmos 1.0" tokenizer module this code was written against no longer exists in that repo's history.

The same CausalVideoTokenizer class (identical constructor signature: checkpoint_enc=..., same ._enc_model attribute) now lives in the standalone repo NVIDIA/Cosmos-Tokenizer as cosmos_tokenizer.video_lib.CausalVideoTokenizer.

Suggested fix: update both imports to:

from cosmos_tokenizer.video_lib import CausalVideoTokenizer

and add NVIDIA/Cosmos-Tokenizer as an install step in the README (pip install -e . from a clone of that repo).

---

2. Installing cosmos_tokenizer's requirements breaks the curation env's numpy/tensorflow/matplotlib compatibility

Cosmos-Tokenizer/requirements.txt pulls in mediapy==1.1.6 / einx==0.1.3 / loguru, and installing them naively (pip install ... with no --no-deps) upgrades numpy to 2.x, which breaks the already-installed tensorflow==2.15.0 (numpy<2.0,>=1.23.5) and scipy==1.10.1 (numpy<1.27,>=1.19.5).

Pinning numpy straight back to whatever requirements.txt implies (we tried 1.23.0, matching tensorflow's floor) then breaks matplotlib==3.11.0 (already present in the env from another dependency), which requires numpy>=1.25 and is imported unconditionally by mediapy/__init__.py (so it's hit just by importing cosmos_tokenizer.video_lib, not only by visualization code).

Working fix: numpy==1.26.4 is the actual intersection of all four constraints (tensorflow<2.0,>=1.23.5 / scipy<1.27,>=1.19.5 / matplotlib>=1.25 / cosmos-tokenizer has no hard numpy pin). The README/requirements.txt should pin numpy explicitly to avoid this resolution puzzle for new installs.

---

3. tensorflow_datasets import fails: protobuf too old for tensorflow_metadata's generated pb2 files

Pre-existing in the env as installed from requirements.txt (tensorflow-datasets==4.9.2, no protobuf pin):

File ".../tensorflow_metadata/proto/v0/anomalies_pb2.py", line 9, in <module>
    from google.protobuf import runtime_version as _runtime_version
ImportError: cannot import name 'runtime_version' from 'google.protobuf'

Installed protobuf==4.25.9 doesn't ship runtime_version (added in later protobuf releases); tensorflow-metadata==1.21.0's prebuilt _pb2.py files were compiled against a protoc new enough to require it. This blocks import tensorflow_datasets, and therefore blocks video_encode_cosmos.py entirely (it does import tensorflow_datasets as tfds at module level).

Suggested fix: pin a compatible protobuf version in requirements.txt alongside tensorflow-datasets==4.9.2 / tensorflow-metadata.

---

4. Typo: curation.semadedup vs actual directory curation/semdedup — the entire dedup pipeline is unimportable

All four core dedup scripts import from a module that doesn't exist on disk:

curation/semdedup/clustering.py:15:from curation.semadedup.loader import EmbeddingLoader
curation/semdedup/clustering.py:10:from curation.semadedup.utils import get_logger
curation/semdedup/sort_clusters.py:27:from curation.semadedup.utils import get_logger
curation/semdedup/sort_clusters.py:28:from curation.semadedup.loader import EmbeddingLoader
curation/semdedup/semdedup.py:21:from curation.semadedup.loader import EmbeddingLoader
curation/semdedup/dendedup.py:21:from curation.semadedup.loader import EmbeddingLoader

The actual directory is curation/semdedup/ (no a after sem). There is no curation/semadedup/ anywhere in the repo. Every one of clustering.py, sort_clusters.py, semdedup.py, dendedup.py will raise ModuleNotFoundError: No module named 'curation.semadedup' immediately on import, regardless of environment setup — this looks like a search-and-replace typo introduced at some point (semdedup → semadedup) that wasn't applied consistently to the directory name itself.

Suggested fix: sed -i 's/curation\.semadedup/curation.semdedup/g' curation/semdedup/*.py (or rename the directory, whichever matches the intended public API).

---

Environment for reference

python 3.11.0 (mamba env `curation`, installed per README)
torch 2.12.1
tensorflow 2.15.0
tensorflow-datasets 4.9.2
tensorflow-metadata 1.21.0
numpy 1.26.4 (after working around #2)
protobuf 4.25.9 (still broken per #3)

Happy to send a PR for #1 and #4 (small, mechanical fixes) if that's useful — wanted to confirm the intended fix direction first since #1 depends on which upstream repo you want to pin to.

Start with the README and requirements.txt, then inspect curation/video_encoding/video_encode_cosmos.py, video_encode_cosmos_hdf5.py, and the four scripts under curation/semdedup/. Reproduce the documented environment failures, verify the tokenizer and dependency constraints, and check that all deduplication modules import successfully. Done means the video-encoding and semantic-dedup pipeline can be imported and run using the documented setup.