How much VRAM does vlm option granite_vision use?

### Question

How much VRAM does vlm option granite_vision use? Is it fp16? Maybe a good idea to mention in the docs? 12GB of VRAM is not enough?

> docling --pipeline vlm --vlm-model granite_vision crawl/LIBIS000000309336.pdf
> 2025-10-15 17:39:22,709 - INFO - Loading plugin 'docling_defaults'
> 2025-10-15 17:39:22,711 - INFO - Registered ocr engines: ['auto', 'easyocr', 'ocrmac', 'rapidocr', 'tesserocr', 'tesseract']
> 2025-10-15 17:39:22,741 - INFO - paths: [PosixPath('/tmp/tmpqwhf1ocz/LIBIS000000309336.pdf')]
> 2025-10-15 17:39:22,741 - INFO - detected formats: []
> 2025-10-15 17:39:22,767 - INFO - Going to convert document batch...
> 2025-10-15 17:39:22,768 - INFO - Initializing pipeline for VlmPipeline with options hash 744865fb6fc3bc38cf7bf5be3238ce87
> 2025-10-15 17:39:22,773 - INFO - Loading plugin 'docling_defaults'
> 2025-10-15 17:39:22,774 - INFO - Registered picture descriptions: ['vlm', 'api']
> 2025-10-15 17:39:22,806 - INFO - Accelerator device: 'cuda:0'
> Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
> /home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/transformers/models/auto/modeling_auto.py:2284: FutureWarning: The class `AutoModelForVision2Seq` is deprecated and will be removed in v5.0. Please use `AutoModelForImageTextToText` instead.
> warnings.warn(
> Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:15<00:00, 7.84s/it]
> 2025-10-15 17:39:41,307 - INFO - Processing document LIBIS000000309336.pdf
> 2025-10-15 17:39:42,489 - WARNING - Encountered an error during conversion of document 77b43422cb21115b6efafce6f135d9d4fab76921f4c511d0705980bde077b812:
> Traceback (most recent call last):
>
> File "/home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 230, in _build_document
> for p in pipeline_pages: # Must exhaust!
>
> File "/home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 195, in _apply_on_pages
> yield from page_batch
>
> File "/home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/docling/models/vlm_models_inline/hf_transformers_model.py", line 182, in __call__
> predictions = list(self.process_images(images, user_prompts))
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> File "/home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/docling/models/vlm_models_inline/hf_transformers_model.py", line 257, in process_images
> inputs = {k: v.to(self.device) for k, v in inputs.items()}
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> File "/home/kvb/.local/pipx/venvs/docling/lib/python3.11/site-packages/docling/models/vlm_models_inline/hf_transformers_model.py", line 257, in
> inputs = {k: v.to(self.device) for k, v in inputs.items()}
> ^^^^^^^^^^^^^^^^^
>
> torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 68.00 MiB. GPU 0 has a total capacity of 11.63 GiB of which 45.44 MiB is free. Including non-PyTorch memory, this process has 11.57 GiB memory in use. Of the allocated memory 11.09 GiB is allocated by PyTorch, and 203.43 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
>
> 2025-10-15 17:39:42,490 - INFO - Finished converting document LIBIS000000309336.pdf in 19.75 sec.
> 2025-10-15 17:39:42,490 - WARNING - Document /tmp/tmpqwhf1ocz/LIBIS000000309336.pdf failed to convert.
> 2025-10-15 17:39:42,490 - INFO - Processed 1 docs, of which 1 failed
> 2025-10-15 17:39:42,493 - INFO - All documents were converted in 19.75 seconds.