Kaggle / Kaggle/kaggle-cli

Kaggle MCP save_notebook silently drops all data-source fields (competition/dataset/kernel/model)

Open
#1,140 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
7.5k
Forks
1.4k
Avg merge
3d 15h
Merged PRs (30d)
11

Description

# Kaggle MCP `save_notebook` silently drops all data-source fields (competition/dataset/kernel/model)

## Summary

The hosted Kaggle MCP server (`https://www.kaggle.com/mcp`) exposes a
`save_notebook` tool. When you call it with any of the data-source arrays
(`competitionDataSources`, `datasetDataSources`, `kernelDataSources`,
`modelDataSources`, or their `...Setter` variants), the call **returns success**
with a valid `version_number`/`kernel_id`, but the sources are **never attached to
the notebook**. The resulting kernel runs with an empty `/kaggle/input`, and the
committed metadata shows no attached sources.

The equivalent public REST endpoint — `POST /api/v1/kernels/push`, which the
`kaggle-api` CLI uses under the hood — accepts the **same field names** and attaches
the sources correctly. Both paths map to the **same backend RPC**
(`ApiSaveKernelRequest` → `POST /api/v1/kernels/push`), so the defect appears to be
in the **MCP tool's field forwarding**, not in the backend.

I'm filing here because `kaggle-api` owns the `kernels push` code path and the
`ApiSaveKernelRequest` proto that the MCP `save_notebook` tool wraps. If the MCP
server lives in a different (non-public) repo, please redirect — but the reproduction
and the "what works" contrast below should let the right team pinpoint it quickly.

## Impact

For **notebook-only / code competitions**, this makes MCP `save_notebook` unusable
end-to-end: without the competition data mounted, the notebook cannot read the input
files, so it errors at runtime (`FileNotFoundError` on the competition data), never
produces `submission.csv`, and therefore cannot be submitted. Agentic/MCP workflows
that rely solely on the documented MCP tools hit a hard dead end here.

## Environment

- Kaggle MCP server: `https://www.kaggle.com/mcp`
- Auth: token authentication via `Authorization: Bearer KGAT_...` (not OAuth)
- Transport: JSON-RPC over HTTP (`tools/list`, `tools/call`)
- MCP client: tested from an MCP-capable client **and** with hand-built `curl`
requests directly against the endpoint (identical result — see below).

## The `save_notebook` tool schema (from `tools/list`)

The tool advertises both a plain field and a `...Setter` variant for every
data-source array, e.g. (abridged):

```
save_notebook.request:
slug: string
text: string
language: string
kernelType: string
kernelExecutionType: string
isPrivate: boolean
enableInternet: boolean
competitionDataSources: array|null
competitionDataSourcesSetter: array|null
datasetDataSources: array|null
datasetDataSourcesSetter: array|null
kernelDataSources: array|null
modelDataSources: array|null
...
```

## Reproduction

### 1) A minimal diagnostic notebook that just lists `/kaggle/input`

```python
from pathlib import Path
for p in sorted(Path('/kaggle/input').rglob('*'))[:50]:
print('FOUND:', p)
print('DONE')
```

### 2) Call MCP `save_notebook` with a competition (or dataset) source attached

`tools/call` payload (competition example; a public dataset like
`kaggle/meta-kaggle` reproduces the same way):

```json
{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/call",
"params": {
"name": "save_notebook",
"arguments": {
"request": {
"slug": "USER/diag-mount",
"newTitle": "Diag Mount",
"text": "from pathlib import Path\nfor p in sorted(Path('/kaggle/input').rglob('*'))[:50]:\n print('FOUND:', p)\nprint('DONE')\n",
"language": "python",
"kernelType": "script",
"isPrivate": true,
"enableInternet": false,
"kernelExecutionType": "SaveAndRunAll",
"competitionDataSources": [""]
}
}
}
}
```

**Result:** the call succeeds:

```json
{ "ref": "/code/USER/diag-mount", "url": "...", "version_number": N, "kernel_id": ... }
```

…but the run log shows **only**:

```
DONE
```

i.e. `/kaggle/input` is **empty** — nothing was mounted.

### 3) Variants tried (all fail identically)

- `competitionDataSources: [...]` only
- `competitionDataSourcesSetter: [...]` only
- both `competitionDataSources` and `competitionDataSourcesSetter` together
- `datasetDataSources: ["kaggle/meta-kaggle"]` (rules out anything competition-specific)
- minimal payloads vs. fully-populated payloads

In **every** case, the notebook runs with an empty `/kaggle/input`, and
`get_notebook_info` for the committed version returns metadata with **no**
`*_data_sources` present.

### 4) Rule out client-side serialization

The same request was sent as a **hand-built JSON array** directly to the MCP
endpoint with `curl` (no MCP-client library in the path). The data source is still
dropped. This rules out client-side array serialization and points to the **MCP
server** handler.

## What DOES work (the contrast that isolates the bug)

The classic public REST endpoint attaches the sources correctly with the **same
field name** and the **same token**:

```bash
curl -s -X POST "https://www.kaggle.com/api/v1/kernels/push" \
-H "Authorization: Bearer KGAT_..." \
-H "Content-Type: application/json" \
--data '{
"slug": "USER/diag-mount",
"newTitle": "Diag Mount",
"text": "from pathlib import Path\nfor p in sorted(Path(\"/kaggle/input\").rglob(\"*\"))[:50]:\n print(\"FOUND:\", p)\nprint(\"DONE\")\n",
"language": "python",
"kernelType": "script",
"isPrivate": true,
"enableInternet": false,
"competitionDataSources": [""]
}'
```

Response includes `"invalidCompetitionSources": []` (source accepted), and after the
run the log shows the data mounted, e.g.:

```
FOUND: /kaggle/input/competitions//train_...
FOUND: /kaggle/input/competitions//test_...
FOUND: /kaggle/input/competitions//valid_...
DONE
```

### Why this proves it's the MCP wrapper, not the backend

- In `kaggle-api`, `kernels_push()` builds an `ApiSaveKernelRequest` and sets
`request.competition_data_sources` before calling `save_kernel(request)`.
- `ApiSaveKernelRequest`'s own endpoint is `POST /api/v1/kernels/push` (method
`POST`) — i.e. the MCP `save_notebook` tool and the CLI `kernels push` target the
**same backend RPC** and the **same proto field** (`competition_data_sources`).
- Since the backend clearly honors the field when called via REST, the loss must
happen in the **MCP tool's request construction/forwarding** (e.g. the
`...Setter` vs. plain field mapping not being applied to the outgoing
`ApiSaveKernelRequest`).

## Additional working notes (useful for anyone hitting this)

These aren't bugs, but they surprised us and may help triage / help others:

- **Two data-mount paths exist.** Competition data can appear at
`/kaggle/input//` **or** `/kaggle/input/competitions//`. Code should
locate files via `Path('/kaggle/input').rglob(name)` rather than hard-coding a path.
- **First run after newly attaching a source may not mount it.** Even via the
working REST path, the *first* run right after a source is added sometimes still
shows an empty `/kaggle/input`; pushing the identical body a second time mounts it
and the run succeeds. (Reported here in case it's related to the same attachment
pipeline.)
- **The rest of the MCP competition flow works fine.** Once a notebook version has
run to `COMPLETE` and produced `submission.csv`, the MCP tools
`create_code_competition_submission` (with the committed `kernelVersion`) and
`get_competition_submission` (poll until `status: COMPLETE` + `public_score`)
work as documented. Only `save_notebook`'s data-source attachment is broken.
- `get_notebook_info` / `get_notebook_session_status` return "Not found" until a
version has actually committed/completed (private drafts and failed-only kernels
aren't queryable), which is easy to misread as an auth problem.

## Expected behavior

MCP `save_notebook` should forward the provided data-source arrays to the backend so
that the resulting notebook mounts them under `/kaggle/input`, matching the behavior
of `POST /api/v1/kernels/push` / `kaggle kernels push`.

## Actual behavior

MCP `save_notebook` accepts the data-source arrays, returns success, but produces a
notebook with **no** data sources attached and an **empty** `/kaggle/input`.

## Suggested fix direction

Ensure the MCP `save_notebook` handler maps its `*DataSources` / `*DataSourcesSetter`
tool arguments onto the corresponding repeated fields of the outgoing
`ApiSaveKernelRequest` (`competition_data_sources`, `dataset_data_sources`,
`kernel_data_sources`, `model_data_sources`) before dispatching to
`POST /api/v1/kernels/push`. A round-trip check (call `save_notebook`, then read the
committed metadata and assert the sources are present) would catch regressions.

Contributor guide

Open the contributing guide

Research direction

Start at the save_notebook handler and compare its request construction with kaggle-api's kernels_push(), especially the ApiSaveKernelRequest data-source fields. Verify completion by calling save_notebook, checking committed metadata with get_notebook_info, and confirming the source is mounted under /kaggle/input; a round-trip regression check should cover competition, dataset, kernel, and model arrays.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend-api-design, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.