Add client retry or throttling/backoff
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- ruby
- Domain
- api, cli, networking
Research direction
Start with the bundle exec neocities push ./_site entry point and reproduce the failure using the CircleCI configuration and a site containing about 100 small files. Confirm the behavior around the reported connection reset after rapid uploads; done means CI deployments complete reliably when uploads trigger throttling or transient disconnects.
Written by the indexing model from the issue text.
Description
Hey there!
Thank you for creating this tool! I've noticed when I use a hosted CI service like CircleCI that uploads a Jekyll site that sometimes it fails, and the client gets disconnected. When it does this, it looks like the push is screaming through (like >30 uploads in a second) and the client might be getting cut off and rate limited.
Example deploy command:
bundle exec neocities push ../_site
Error coming out, and retries seem to fail the same way after about 30 uploads that went through really fast:
Uploading tech/year/month/day/blah.html ... bundler: failed to load command: neocities (/home/circleci/project/neocities/vendor/bundle/ruby/3.1.0/bin/neocities)
/home/circleci/project/neocities/vendor/bundle/ruby/3.1.0/gems/httpclient-fixcerts-2.8.5/lib/httpclient/session.rb:813:in `rescue in block in parse_header': HTTPClient::KeepAliveDisconnected: Connection reset by peer (HTTPClient::KeepAliveDisconnected)
I haven't been able to replicate this issue from my laptop, where uploads of individual files are uploaded slower. If it is server-side rate limiting, it'd be nice if the client could jive with it. This doesn't fail every time with CI services. If things are a bit slower uploading from a CI service, it will sometimes work.
Steps to replicate
- Have a repo with ~100 small files that are part of a static website (ie, Jekyll, Hugo).
- Add a Gemfile that looks like this in the root directory of the repo:
source 'https://rubygems.org'
gem 'neocities'
- Setup the repo to work with CircleCI
- Add an environment variable to CircleCI project config called
NEOCITIES_API_KEYwith your API key - Add a CircleCI config to the repo
.circleci/config.ymlthat looks something like this (assumes static assets are in a directory called_sitelike with Jekyll):
---
version: 2.1
orbs:
ruby: circleci/ruby@2.0.0
ruby_image: &ruby_image cimg/ruby:3.1
workflows:
build_and_deploy:
jobs:
- deploy_neocities
jobs:
deploy_neocities:
resource_class: large
docker:
- image: *ruby_image
steps:
- checkout
- ruby/install-deps
- run:
name: Deploy Jekyll from _site directory to neocities
command: |
bundle exec neocities push ./_site
With failed tasks, you can retry with SSH to log in with CircleCI to manually run the deploy command and experiment.
- Dominant language
- Ruby
- Stars
- 70
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from neocities/neocities-ruby
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
neocities/neocities-ruby#80 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
neocities/neocities-ruby#81 ·
-
OpenSSL conflict Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
neocities/neocities-ruby#79 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 64/100
neocities/neocities-ruby#78 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
neocities/neocities-ruby#77 · 1 comment ·
All issues in neocities/neocities-ruby
Similar issues
-
バグ
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
voxpupuli/puppet-epel#186 · 1 comment ·
-
external_created_at is no longer used for the message timestamp since the new message UI (v4.4.0) OpenBug Frontend
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
TheOdinProject/curriculum#31402 · 1 comment ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100