cactus-compute / cactus-compute/cactus

fix: resolve Python FFI buffer limits and single-quote escaping in tool calling

Open
#760 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6k
Forks
501
Avg merge
1d 18h
Merged PRs (30d)
4

Description

### Description

While working with the Cactus codebase, I encountered three related bugs/limitations in the FFI bindings and tool call parsing logic:

#### 1. Invalid JSON generated during single-quoted parameter parsing in tool calls
LiquidAI models (`LFM2`/`LFM2.5`) format tool calls as Python-like arguments (e.g., `my_tool(param='val')`).
* **The issue**: In `cactus-engine/src/chat_tools.h`, the parser function `py_literal_to_json` translates escaped single quotes (`\'`) literally as `\'` inside the resulting double-quoted JSON string (producing `"hello \'world\""`).
* **Why it fails**: Per RFC 8259, `\'` is an invalid escape sequence in JSON. Standard JSON parsers (e.g. C++ `picojson` and python's `json` module) fail with a syntax error, causing tool execution to abort.
* **Proposed fix**: Modify the escape sequence parser to unescape `\'` to `'` in the double-quoted JSON string since single quotes do not require escaping.

#### 2. `RuntimeError` when retrieving documents > 4KB from index
* **The issue**: The vector database supports indexing documents up to 65,535 bytes (`Index::validate_documents`). However, the Python FFI binding `cactus_index_get` uses a hardcoded buffer limit `_INDEX_DOC_BUF_SIZE = 4096`.
* **Why it fails**: When retrieving a document that exceeds 4KB, the C++ code returns `-1` (buffer too small), causing the Python wrapper to raise `RuntimeError: Failed to get from index`.
* **Proposed fix**: Bump `_INDEX_DOC_BUF_SIZE` to `65536` in the Python FFI bindings to accommodate the maximum supported document size.

#### 3. RAG query response buffer size limitation (64KB)
* **The issue**: The Python binding `cactus_rag_query` allocates a `65536` byte buffer.
* **Why it fails**: RAG queries returning multiple chunks (up to `top_k = 5`) can easily exceed 64KB, causing query failures.
* **Proposed fix**: Increase the RAG query buffer to `1 << 20` (1MB), matching the completion buffer size.

---

### Verification
* Created a new C++ test suite `cactus-engine/tests/test_chat_tools.cpp` to verify parsing logic.
* Verified that all tests compile and pass successfully.

I have these changes ready on a local branch. Please let me know if you would like me to submit a Pull Request!

Contributor guide

Open the contributing guide

Research direction

Start with cactus-engine/src/chat_tools.h and the Python FFI bindings, then run cactus-engine/tests/test_chat_tools.cpp. Verify that escaped single quotes produce valid JSON, index retrieval supports the documented 65,535-byte maximum, and RAG queries can return responses beyond 64KB. Done means the new parser tests pass and all three buffer-limit failures are covered without breaking existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
ai, backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.