docling-project / docling-project/docling
When called vlm (granite-docling), got <section_header_level_1><loc_0><loc_0><loc_499><loc_500> as output
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
I am using granite-docling-258M model, hosted remotely using HP Endpoint.
When i tried to use docling with this model, i get on this as output:
I have tried troubleshooting but to no avail.
### Codes
def openai_compatible_vlm_options(
model: str,
prompt: str,
format: ResponseFormat,
hostname_and_port,
temperature: float = 0.7,
max_tokens: int = 4096,
api_key: str = "",
skip_special_tokens=False,
):
headers = {}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
options = ApiVlmOptions(
url=f"{hostname_and_port}/v1/chat/completions", # LM studio defaults to port 1234, VLLM to 8000
params=dict(
model=model,
max_tokens=max_tokens,
skip_special_tokens=skip_special_tokens, # needed for VLLM
),
headers=headers,
prompt=prompt,
timeout=90,
scale=2.0,
temperature=temperature,
response_format=format,
)
print(f"VLM options: {options}")
return options
def main():
logging.basicConfig(level=logging.INFO)
data_folder = Path(__file__).parent / "./FinDDR2025"
input_doc_path = data_folder / "MeisinLee__Cover_Letter_Bain.pdf"
print(f"Converting {input_doc_path}")
# Configure the VLM pipeline. Enabling remote services allows HTTP calls to
# locally hosted APIs (LM Studio, Ollama) or cloud services.
pipeline_options = VlmPipelineOptions(
enable_remote_services=True # required when calling remote VLM endpoints
)
pipeline_options.vlm_options = openai_compatible_vlm_options(
model="ibm-granite/granite-docling-258M", # For VLLM use "ibm-granite/granite-docling-258M"
hostname_and_port="huggingface endpoint", # LM studio defaults to port 1234, VLLM to 8000
prompt="OCR the full page to markdown.",
format=ResponseFormat.MARKDOWN,
api_key="",
)
# Create the DocumentConverter and launch the conversion.
doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options,
pipeline_cls=VlmPipeline,
)
}
)
result = doc_converter.convert(input_doc_path)
#print(result.document)
print(result.document.export_to_markdown())
Contributor guide
Assessment
This issue has not been assessed yet.