luarocks / luarocks/luarocks

`source.url = XXX.tar.gz` 852 times slower than `cli untar`

Open
#1,892 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Lua
Stars
3.7k
Forks
479
PR merge metrics
No merged PRs in 30d

Description

### Please read the following before submitting:
- Please do NOT submit bug reports for questions. Ask questions in the [Gitter
chat-room](https://gitter.im/luarocks/luarocks).
- Please do NOT submit duplicate bug reports. Look through the [issue
tracker](https://github.com/luarocks/luarocks/issues) before submitting.

### Please fill out the following:
- **Platform:**
- Linux.

ArchLinux + nixpkgs

- **LuaRocks version:**
- 3.13.0

- **Configuration file:**
- Please try to produce with the default configuration.
- If you cannot reproduce with the default configuration, please try to find
the minimal configuration to reproduce.
- Upload the config to a pastebin such as gist.github.com.

- **LuaRocks output from when the issue occurred:**
- Use the `--verbose` flag.

- **Description:**
- The steps you took in plain English to reproduce the problem.

# bug

`source.url = XXX.tar.gz` 852 times slower than `cli untar`

## unexpected

`source.url = XXX.tar.gz` takes 28.9s, which is

- 51.571174377224196 times slower than `source.url = 'git+http://XXX'`

- 56.27766990291262 times slower than `source.url = 'XXX.zip'`

- 852.4411764705882 times slower than `cli untar`

```lua
source = {
url = 'http://example.com/repo/archive/commit.tar.gz',
...
}
```

```console
$ rm -rf ~/.luarocks
$ time luarocks install xxx
...
real 0m28.983s
user 0m10.908s
sys 0m17.665s
```

```lua
function tar.untar(filename, destdir)
...

-- FIXME: 28s
while true do
...
end

...
```

## expected

- `source.url = 'git+http://XXX'` takes 0.562s

- `source.url = 'XXX.zip'` takes 0.515s

- `cli untar` takes 0m0.034s

## git

```lua
source = {
url = 'git+http://example.com/repo.git',
...
}
```

```console
$ rm -rf ~/.luarocks
$ time luarocks install xxx
...
real 0m0.562s
user 0m0.488s
sys 0m0.193s
```

## zip

```lua
source = {
url = 'http://example.com/repo/archive/commit.zip',
...
}
```

```console
$ rm -rf ~/.luarocks
$ time luarocks install xxx
...
real 0m0.515s
user 0m0.167s
sys 0m0.177s
```

## tgz

```console
$ du -sh /tmp/commit.tar; time tar -xf /tmp/commit.tar -C "$(mktemp -d)"
12M /tmp/commit.tar

real 0m0.034s
user 0m0.008s
sys 0m0.026s
```

## workaround

Use commit.zip instead of commit.tar.gz

## suggestion

Try `cli untar` first.

## related

Use commit tarball as workaround for

https://github.com/luarocks/luarocks/issues/569

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Linux LuaRocks 3.13.0 timings with `luarocks install xxx`, then inspect the `tar.untar(filename, destdir)` function mentioned in the issue. Compare its extraction path with `cli untar` and verify that `.tar.gz` installation approaches the command-line extraction time without breaking archive handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.