bytedance / bytedance/Dolphin

Line breaks in code blocks are lost during Markdown generation

Open
#67 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9k
Forks
775
PR merge metrics
No merged PRs in 30d

Description

### **Description**

When processing a PDF, the system correctly identifies multi-line code blocks and preserves the newline characters (`\n`) in the intermediate JSON representation. However, during the final conversion to a Markdown file, these newline characters are stripped, causing the entire code block to be rendered as a single, unformatted line.

### **Steps to Reproduce**

1. **Source Content:** A PDF file containing a formatted, multi-line code block. (See original screenshot).

![Image](https://github.com/user-attachments/assets/c967d83d-b17e-40b2-bfbe-950ead50395c)

2. **Intermediate JSON:** The system (Dolphin) correctly parses this and generates a JSON object. Note that the `text` field contains `\n` characters, preserving the line breaks:

```json
{
"label": "code",
"bbox": [96, 609, 606, 806],
"text": "for _, tc:= range testCases {\ntestCase:= tc\nt.Run(testCase.name, func(t *testing.T) {\nt.Parallel()\nt.Logf(\"Test case %s with inputs %d and %d should produce\n%d\",\ntestCase.name, testCase.a, testCase.b,\ntestCase.result)\nresult:= Add(testCase.a, testCase.b)\nif result!= testCase.result {\nt.Errorf(\"Adding %d and %d doesn't produce %d,\ninstead it produces %d\",\ntestCase.a, testCase.b, testCase.result, result)",
"reading_order": 4
}
```

3. **Final Output:** The process that generates the Markdown file from this JSON produces incorrect output.

### **Expected Behavior**

The final Markdown file should render the code block with its original line breaks, properly formatted inside a code fence.

```go
for _, tc := range testCases {
testCase := tc
t.Run(testCase.name, func(t *testing.T) {
t.Parallel()
t.Logf("Test case %s with inputs %d and %d should produce %d",
testCase.name, testCase.a, testCase.b,
testCase.result)
result := Add(testCase.a, testCase.b)
if result != testCase.result {
t.Errorf("Adding %d and %d doesn't produce %d, instead it produces %d",
testCase.a, testCase.b, testCase.result, result)
}
})
}
```

### **Actual Behavior**

The code block is flattened into a single line, losing all formatting and readability.

```
for _, tc:= range testCases { testCase:= tc t.Run(testCase.name, func(t *testing.T) { t.Parallel() t.Logf("Test case %s with inputs %d and %d should produce %d", testCase.name, testCase.a, testCase.b, testCase.result) result:= Add(testCase.a, testCase.b) if result!= testCase.result { t.Errorf("Adding %d and %d doesn't produce %d, instead it produces %d", testCase.a, testCase.b, testCase.result, result)
```

### **Conclusion**

The bug lies in the final stage of Markdown generation. The process responsible for converting the JSON `text` field into Markdown content is not respecting the `\n` newline characters.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue does not name a file or test; start by locating the final Markdown-generation path that converts the JSON text field into Markdown. Reproduce the problem with a PDF containing a multi-line code block and verify that the completed output preserves its newline characters inside a code fence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.