dolthub / dolthub/dolt

Dolt push / clone with depth params takes up an excessive amount of RAM

Open
#11,240 5 comments 0 reactions 0 assignees View on GitHub
bug customer issue no repro performance version control
Dominant language
Go
Stars
24.4k
Forks
873
Avg merge
1d 5h
Merged PRs (30d)
108

Description

## Issue
DB file size: 195GB
Dolt version: 2.1.7
We use `dolt push` to remote repository by `files://`. It crashed many times, each one we add some memory until memory up to 128GB.
When we clone from the same repository with depth 1, we found 128GB memory of the server which execute `dolt clone` was not enough, and crashed when clone at 30minutes after we execute command. It work successfully on the second time, and used 98% of the memory

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported `dolt push` to a `files://` remote and `dolt clone` with depth 1 using the stated 195GB repository and Dolt 2.1.7. Measure memory during both commands and compare the successful and failed clone runs; done means the operations complete without excessive memory growth.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.