docling-project / docling-project/docling
Problem running on google collab
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Question
I am running the docling with granite model on google collab with T4 GPU, it's taking alot of time for 16 pages of pdf.
I have attached the code below. Please take a look and help me out here. Thanks
from docling.datamodel import vlm_model_specs
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
VlmPipelineOptions,AcceleratorOptions, AcceleratorDevice
)
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline
import torch
# Convert a public arXiv PDF; replace with a local path if preferred.
source = ""
###### USING SIMPLE DEFAULT VALUES
# - GraniteDocling model
# - Using the transformers framework
if not torch.cuda.is_available():
raise RuntimeError("No GPU detected! Please enable a GPU in Colab: "
"Go to Runtime > Change runtime type > Hardware accelerator > GPU.")
# Print GPU information
gpu_name = torch.cuda.get_device_name(0)
print(f"GPU detected: {gpu_name}")
print("threads:,", torch.cuda.device_count())
# Set AcceleratorOptions to use GPU
accelerator_options = AcceleratorOptions(
num_threads=8, # Use the number of available CUDA devices
device=AcceleratorDevice.CUDA
)
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
),
}
)
doc = converter.convert(source=source).document
print(doc.export_to_markdown())
###### USING MACOS MPS ACCELERATOR
# Demonstrates using MLX on macOS with MPS acceleration (macOS only).
# For more options see the `compare_vlm_models.py` example.
pipeline_options = VlmPipelineOptions(
vlm_options=vlm_model_specs.GRANITEDOCLING_TRANSFORMERS,
)
pipeline_options.accelerator_options = accelerator_options
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
pipeline_options=pipeline_options,
),
}
)
doc = converter.convert(source=source).document
print(doc.export_to_markdown())
...
Contributor guide
Assessment
This issue has not been assessed yet.