docling-project / docling-project/docling

Problem running on google collab

Open
#2,303 3 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Question
I am running the docling with granite model on google collab with T4 GPU, it's taking alot of time for 16 pages of pdf.
I have attached the code below. Please take a look and help me out here. Thanks

from docling.datamodel import vlm_model_specs
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
VlmPipelineOptions,AcceleratorOptions, AcceleratorDevice
)
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline
import torch
# Convert a public arXiv PDF; replace with a local path if preferred.
source = ""
###### USING SIMPLE DEFAULT VALUES
# - GraniteDocling model
# - Using the transformers framework

if not torch.cuda.is_available():
raise RuntimeError("No GPU detected! Please enable a GPU in Colab: "
"Go to Runtime > Change runtime type > Hardware accelerator > GPU.")

# Print GPU information
gpu_name = torch.cuda.get_device_name(0)
print(f"GPU detected: {gpu_name}")
print("threads:,", torch.cuda.device_count())
# Set AcceleratorOptions to use GPU
accelerator_options = AcceleratorOptions(
num_threads=8, # Use the number of available CUDA devices
device=AcceleratorDevice.CUDA
)

converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
),
}
)

doc = converter.convert(source=source).document

print(doc.export_to_markdown())

###### USING MACOS MPS ACCELERATOR
# Demonstrates using MLX on macOS with MPS acceleration (macOS only).
# For more options see the `compare_vlm_models.py` example.

pipeline_options = VlmPipelineOptions(
vlm_options=vlm_model_specs.GRANITEDOCLING_TRANSFORMERS,
)
pipeline_options.accelerator_options = accelerator_options
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
pipeline_options=pipeline_options,
),
}
)

doc = converter.convert(source=source).document

print(doc.export_to_markdown())
...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.