Calling glob() on a directory of 200k files produces a RangeError: Maximum call stack size exceeded
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5.9k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
Thank you 🙇♀ for wanting to create an issue in this repository. Before you do, please ensure you are filing the issue in the right place. Issues should only be opened on if the issue relates to code in this repository.
- If you have found a security issue please submit it here
- If you have questions about writing workflows or action files, then please visit the GitHub Community Forum's Actions Board
- If you are having an issue or question about GitHub Actions then please contact customer support
If your issue is relevant to this repository, please include the information below:
Describe the bug
Calling globber.glob on a directory with 200k files produces this error:
Run actions/github-script@v6.4.1
RangeError: Maximum call stack size exceeded
at DefaultGlobber.globGenerator_1 (/home/runner/work/_actions/actions/github-script/v6.4.1/dist/index.js:3333:27)
at globGenerator_1.next (<anonymous>)
at resume (/home/runner/work/_actions/actions/github-script/v6.4.1/dist/index.js:3222:44)
Error: Unhandled error: RangeError: Maximum call stack size exceeded
at fulfill (/home/runner/work/_actions/actions/github-script/v6.4.1/dist/index.js:3224:31)
To Reproduce
Here is a workflow that reproduces the issue:
name: Reproduce globbing issue w/ too many files
on:
pull_request:
branches:
- main
jobs:
reproduce-glob-issue-too-many-files:
name: Reproduce globbing issue w/ too many files
runs-on:
- ubuntu-latest
steps:
- name: Generate 100k files
run: |
mkdir -p ${{ runner.temp }}/test
for i in {1..100000}
do
echo $i > ${{ runner.temp }}/test/f$i.txt
done
# This one is fine.
- name: Glob 100k files
uses: actions/github-script@v6.4.1
with:
script: |
const path = `${process.env.RUNNER_TEMP}/test/*.txt`
const globber = await glob.create(path)
const files = await globber.glob()
console.log(files.length)
- name: Generate additional 100k files
run: |
mkdir -p ${{ runner.temp }}/test
for i in {100000..200000}
do
echo $i > ${{ runner.temp }}/test/f$i.txt
done
# This one fails with the RangeError.
- name: Glob 200k files
uses: actions/github-script@v6.4.1
with:
script: |
const path = `${process.env.RUNNER_TEMP}/test/*.txt`
const globber = await glob.create(path)
const files = await globber.glob()
console.log(files.length)
Here is a public example where I've run this workflow and it has failed:
- PR: https://github.com/alexklibisz/github-script-glob-range-error/pull/1
- Workflow: https://github.com/alexklibisz/github-script-glob-range-error/actions/runs/6393726981/job/17353558090?pr=1
Expected behavior
Calling .glob should not fail with a RangeError, or a known limit should be documented, or a workaround should be documented.
Screenshots
Desktop (please complete the following information):
Not relevant, see additional context for relevant details
Smartphone (please complete the following information):
Not relevant, see additional context for relevant details
Additional context
I've been able to reproduce this on public Github using github-script 6.4.1 and on Github Enterprise using github-script 6.3.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the globber.glob entry point and the DefaultGlobber.globGenerator stack frame shown in the report. Run the supplied GitHub Actions workflow to compare globbing 100k and 200k files. Done means the 200k-file case no longer raises RangeError, or the supported limit or workaround is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, typescript
- Domain
- ci-cd, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100