Roadmap (tentative)

__Current tasks:__
- [ ] prototype bloom points system @borzunov (#6 )
- [x] local tensor parallelism ( #143 , using [BlackSamorez/tesnor_parallel](https://github.com/BlackSamorez/tensor_parallel) by @blacksamorez and @IaroslavLisniak )
- [x] increase default max sequence length (from #146 )
- [x] allow running a server without open ports @Vahe1994
- [ ] option to download pre-quantized blocks (@mryab )
- [x] improved routing (@justheuristic )
- [x] newest-latest libp2p - @Vahe1994
- [ ] touch up fine-tuning examples, make sure they work in reasonable time ( @justheuristic )
- [ ] a way to temporarily shutdown petals server
- suggested by @craffel : when running a petals server on a machine that is often in use, people should be able to shut off petals servers while running their experiments
- suggested behavior: shut down asap, restart once gpus are not in use for T minutes
- __Wanna contribute?__
- go to [our discord server](https://discord.gg/D9MwApKgWa) and ask around!
- always in demand:
- contribute examples (recommended but not required: create an issue / draft first, before you code them)
- OS support / hardware support ( e.g. see #147 )
- more models: OPT-175B, switch-XXL, whatever comes into fashion
- host a server! (see README)

__End of december:__ cover more use cases
- [x] tutorial: generation notebook
- [x] tutorial: prompt-tuning notebook
- [x] PreLayer prompt-tuning - mentioned as one of the baselines in https://arxiv.org/abs/2106.09685 - DONE
- [x] inference with prompt-tuning ( https://github.com/learning-at-home/bloom-demo/issues/13 by @artek0chumak)
- [x] advanced inference: beam search, constraints/fusion, LoRA/AdaMix ( @artek0chumak , https://github.com/learning-at-home/bloom-demo/issues/13 )
- [x] some kind of hub for tutorials, e.g. a minimalistic website
- [x] alpha test: let more people play with 176B model (where? no-brainer: bigscience, stability, discord)
- [x] rich inference interface for designing custom generation algorithms (by: @artek0chumak )
- [x] let servers run requests with different priorities ( https://github.com/learning-at-home/bloom-demo/issues/8 by: @greenfatguy )
- By this point, we must answer the main questions: (1) will people use it? (2a) what for? (2b) why not?

__End of ~july~ august:__ make it reliable, test with early adopters
- [x] make it so that servers cannot be _killed_ by a bad client request ( https://github.com/learning-at-home/hivemind/issues/3 by: @justheuristic)
- [x] find the best way to reduce the size of 176B model ( https://github.com/learning-at-home/bloom-demo/issues/4 by: @TimDettmers )
- [x] let servers automatically find and serve the most in-demand layers ( @borzunov )
- [x] implement popular non-beam inference types ( https://github.com/learning-at-home/bloom-demo/issues/13 by @artek0chumak )
- [x] compress the activations sent between client and server nodes ( by: @mryab )
- [x] find enough hardware to run the 176B model ( https://github.com/learning-at-home/bloom-demo/issues/14 by: @justheuristic )
- [x] pre-alpha test: once it is stable enough, let some (trusted) folks play with it and get their feedback
- [x] submit a EMNLP system demonstration proposal ( https://2022.emnlp.org/calls/System_Demonstrations/ )
- begin investigating: tutorials, documentation, examples

__End of june:__ build a proof-of-concept
- [x] agree on the user interface (see https://github.com/learning-at-home/bloom-demo/issues/5#issuecomment-1159776494 )
- [x] run simple (__but correct!__) inference with a smaller model (for generation)
- [x] do simple (__but correct!__) forward/backward with frozen layers (for prompt tuning)
- [x] client can dynamically choose which remote servers to use for inference ( by: @justheuristic )
- [x] create basic correctness tests for later
- [x] check if 8-bit compression is remotely feasible ( by: @TimDettmers )
- it's okay if the code is not super reliable for now
- it's okay if servers have to be set up manually for now
- begin investigating: quantized weights, quantized communication, automatic server allocation, "bloom points"

__Important, but not urgent:__
- multiplicative adapters from https://wandb.ai/learning-at-home/LM_OWT/reports/Parameter-sharing-revisited--VmlldzoxOTAxNjcx ?
- non-critical performance improvements ( #11 )
- better finetuning methods: LoRA, AdaMix, PreLayer (see LoRA/AdaMix), whatever is SoTA at the time of building
- fully decentralized point system