occasional LaTeX footnotes
- Ngôn ngữ chính
- Python
- Star
- 19.5k
- Fork
- 1.6k
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
### 🐛 Describe the bug
Hello,
Thanks so much for sharing this work and the new FP8 model. Apologies if this is a mistake on my end.
I. As mentioned in some previous issues: The --markdown flag but the resulting .md files tend to not have headings parsed (with ## or so).
II. Moreover, the resulting markdown files seem to have superfluous latex code such as for footnotes
- \textsuperscript{11}
- \(^{12}\)
- \(^{16}\)
I'm running the standard pipeline in a container (Singularity/Apptainer), with the latest image as of last week in a university cluster. The compute node runs the following
```bash
apptainer exec --nv /home/me/containers/olmocr_latest.sif \
python -m olmocr.pipeline /home/me/datasets/mydata/v0/documents \
--pdfs /home/me/datasets/mydata/v0/documents/*/*.pdf \
--markdown
```
J
### Versions
Python 3.12.11
accelerate==1.10.1
aiofile==3.9.0
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
annotated-types==0.7.0
anyio==4.11.0
astor==0.8.1
attrs==25.3.0
awscrt==0.28.1
beaker-py==2.5.0
beautifulsoup4==4.14.2
bitsandbytes==0.48.1
blake3==1.0.7
bleach==6.2.0
blinker==1.9.0
boto3==1.40.45
botocore==1.40.45
build==1.3.0
cached_path==1.8.0
cachetools==6.2.0
caio==0.9.24
cbor2==5.7.0
certifi==2025.8.3
cffi==2.0.0
chardet==4.0.0
charset-normalizer==3.4.3
click==8.3.0
cloudpickle==3.1.1
cmake==4.1.0
compressed-tensors==0.11.0
cryptography==46.0.2
cuda-bindings==12.8.0
cuda-python==12.8.0
cufile-python==0.1.1
cupy-cuda12x==13.6.0
dbus-python==1.2.18
-e git+https://github.com/deepseek-ai/DeepEP@e3908bf5bd0cc6265bcb225d15cd8c996d4759ef#egg=deep_ep
deep_gemm @ file:///tmp/tmp.jdeoLXViT2/deepgemm/dist/deep_gemm-2.0.0+ea9c5d9-cp312-cp312-linux_x86_64.whl
defusedxml==0.7.1
Deprecated==1.2.18
depyf==0.19.0
dill==0.4.0
diskcache==5.6.3
distro==1.9.0
distro-info==1.1+ubuntu0.2
dnspython==2.8.0
einops==0.8.1
email-validator==2.3.0
eval_type_backport==0.2.2
fastapi==0.118.0
fastapi-cli==0.0.13
fastapi-cloud-cli==0.3.0
fastjsonschema==2.21.2
fastrlock==0.8.3
filelock==3.19.1
flashinfer-python @ file:///vllm-workspace/flashinfer
Flask==3.1.2
frozendict==2.4.6
frozenlist==1.7.0
fsspec==2025.9.0
ftfy==6.3.1
fuzzysearch==0.8.0
gguf==0.17.1
google-api-core==2.25.2
google-auth==2.41.1
google-cloud-core==2.4.3
google-cloud-storage==2.19.0
google-crc32c==1.7.1
google-genai==1.41.0
google-resumable-media==2.7.2
googleapis-common-protos==1.70.0
greenlet==3.2.4
grpcio==1.75.1
h11==0.16.0
hf-xet==1.1.10
hf_transfer==0.1.9
httpcore==1.0.9
httplib2==0.20.2
httptools==0.6.4
httpx==0.28.1
huggingface-hub==0.35.3
humanize==4.13.0
idna==3.10
img2pdf==0.6.1
importlib-metadata==4.6.4
interegular==0.3.3
invoke==2.2.0
itsdangerous==2.2.0
jeepney==0.7.1
Jinja2==3.1.6
jiter==0.11.0
jmespath==1.0.1
jsonschema==4.25.1
jsonschema-specifications==2025.9.1
jupyter_client==8.6.3
jupyter_core==5.8.1
jupyterlab_pygments==0.3.0
keyring==23.5.0
lark==1.2.2
lazr.uri==1.0.6
lingua-language-detector==2.1.1
llguidance==0.7.30
llvmlite==0.44.0
lm-format-enforcer==0.11.3
lmcache==0.3.7
lxml==6.0.2
markdown-it-py==4.0.0
markdown2==2.5.4
MarkupSafe==3.0.3
mdurl==0.1.2
mistral_common==1.8.5
mistralai==1.9.11
mistune==3.1.4
modelscope==1.30.0
more-itertools==8.10.0
mpmath==1.3.0
msgpack==1.1.1
msgspec==0.19.0
multidict==6.6.4
nbclient==0.10.2
nbconvert==7.16.6
nbformat==5.10.4
networkx==3.5
ninja==1.13.0
nixl==0.6.0
numba==0.61.2
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cudnn-frontend==1.14.1
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-ml-py==12.575.51
nvidia-nccl-cu12==2.27.3
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvshmem-cu12==3.4.5
nvidia-nvtx-cu12==12.8.90
nvtx==0.2.13
olmocr @ file:///build
openai==2.1.0
openai-harmony==0.0.4
opencv-python-headless==4.12.0.88
orjson==3.11.3
outlines_core==0.2.11
packaging==25.0
pandocfilters==1.5.1
partial-json-parser==0.2.1.1.post6
pikepdf==9.11.0
pillow==11.3.0
platformdirs==4.4.0
playwright==1.55.0
-e git+https://github.com/ppl-ai/pplx-kernels@c336fafe1d416c3d0c16e19cace590953b4f21b3#egg=pplx_kernels
prometheus-fastapi-instrumentator==7.1.0
prometheus_client==0.23.1
propcache==0.3.2
proto-plus==1.26.1
protobuf==6.32.1
psutil==7.1.0
py-cpuinfo==9.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.2
pybase64==1.4.2
pycountry==24.6.1
pycparser==2.23
pydantic==2.11.9
pydantic-extra-types==2.10.5
pydantic_core==2.33.2
pyee==13.0.0
Pygments==2.19.2
PyGObject==3.42.1
PyJWT==2.3.0
pynvml==12.0.0
pyparsing==2.4.7
pypdf==6.1.1
pypdfium2==4.30.0
pyproject_hooks==1.2.0
python-apt==2.4.0+ubuntu4
python-dateutil==2.9.0.post0
python-debian==0.1.43+ubuntu1.1
python-dotenv==1.1.1
python-json-logger==3.3.0
python-magic==0.4.27
python-multipart==0.0.20
PyYAML==6.0.3
pyzmq==27.1.0
RapidFuzz==3.14.1
ray==2.49.2
redis==6.4.0
referencing==0.36.2
regex==2025.9.18
requests==2.32.5
rich==13.9.4
rich-toolkit==0.15.1
rignore==0.7.0
rpds-py==0.27.1
rsa==4.9.1
runai-model-streamer==0.14.0
runai-model-streamer-s3==0.14.0
s3transfer==0.14.0
safetensors==0.6.2
scipy==1.16.2
SecretStorage==3.3.1
sentencepiece==0.2.1
sentry-sdk==2.39.0
sequence_align==0.3.0
setproctitle==1.3.7
setuptools==79.0.1
setuptools-scm==9.2.0
shellingham==1.5.4
six==1.17.0
smart_open==7.3.1
sniffio==1.3.1
sortedcontainers==2.4.0
soundfile==0.13.1
soupsieve==2.8
soxr==1.0.0
starlette==0.48.0
sympy==1.14.0
syntok==1.4.4
tabulate==0.9.0
tenacity==9.1.2
tiktoken==0.11.0
timm==1.0.20
tinycss2==1.4.0
tinyhost==0.4.18
tokenizers==0.21.4
torch==2.8.0+cu128
torchaudio==2.8.0+cu128
torchvision==0.23.0+cu128
tornado==6.5.2
tqdm==4.67.1
traitlets==5.14.3
transformers==4.55.2
triton==3.4.0
typer==0.19.2
typing-inspection==0.4.2
typing_extensions==4.15.0
ubuntu-pro-client==8001
unattended-upgrades==0.1
urllib3==2.5.0
uv==0.8.22
uvicorn==0.37.0
uvloop==0.21.0
vllm @ file:///vllm-workspace/dist/vllm-0.11.0-cp38-abi3-linux_x86_64.whl
wadllib==1.3.6
watchfiles==1.1.0
wcwidth==0.2.14
webencodings==0.5.1
websockets==15.0.1
Werkzeug==3.1.3
wheel==0.45.1
wrapt==1.17.3
xformers==0.0.32.post1
xgrammar==0.1.25
yarl==1.20.1
zipp==1.0.0
zstandard==0.25.0
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.