cactus-compute / cactus-compute/cactus
fix: resolve Python FFI buffer limits and single-quote escaping in tool calling
- Dominant language
- C++
- Stars
- 6k
- Forks
- 501
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
### Description
While working with the Cactus codebase, I encountered three related bugs/limitations in the FFI bindings and tool call parsing logic:
#### 1. Invalid JSON generated during single-quoted parameter parsing in tool calls
LiquidAI models (`LFM2`/`LFM2.5`) format tool calls as Python-like arguments (e.g., `my_tool(param='val')`).
* **The issue**: In `cactus-engine/src/chat_tools.h`, the parser function `py_literal_to_json` translates escaped single quotes (`\'`) literally as `\'` inside the resulting double-quoted JSON string (producing `"hello \'world\""`).
* **Why it fails**: Per RFC 8259, `\'` is an invalid escape sequence in JSON. Standard JSON parsers (e.g. C++ `picojson` and python's `json` module) fail with a syntax error, causing tool execution to abort.
* **Proposed fix**: Modify the escape sequence parser to unescape `\'` to `'` in the double-quoted JSON string since single quotes do not require escaping.
#### 2. `RuntimeError` when retrieving documents > 4KB from index
* **The issue**: The vector database supports indexing documents up to 65,535 bytes (`Index::validate_documents`). However, the Python FFI binding `cactus_index_get` uses a hardcoded buffer limit `_INDEX_DOC_BUF_SIZE = 4096`.
* **Why it fails**: When retrieving a document that exceeds 4KB, the C++ code returns `-1` (buffer too small), causing the Python wrapper to raise `RuntimeError: Failed to get from index`.
* **Proposed fix**: Bump `_INDEX_DOC_BUF_SIZE` to `65536` in the Python FFI bindings to accommodate the maximum supported document size.
#### 3. RAG query response buffer size limitation (64KB)
* **The issue**: The Python binding `cactus_rag_query` allocates a `65536` byte buffer.
* **Why it fails**: RAG queries returning multiple chunks (up to `top_k = 5`) can easily exceed 64KB, causing query failures.
* **Proposed fix**: Increase the RAG query buffer to `1 << 20` (1MB), matching the completion buffer size.
---
### Verification
* Created a new C++ test suite `cactus-engine/tests/test_chat_tools.cpp` to verify parsing logic.
* Verified that all tests compile and pass successfully.
I have these changes ready on a local branch. Please let me know if you would like me to submit a Pull Request!
Contributor guide
Research direction
Start with cactus-engine/src/chat_tools.h and the Python FFI bindings, then run cactus-engine/tests/test_chat_tools.cpp. Verify that escaped single quotes produce valid JSON, index retrieval supports the documented 65,535-byte maximum, and RAG queries can return responses beyond 64KB. Done means the new parser tests pass and all three buffer-limit failures are covered without breaking existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ai, backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100