LivelyKernel / LivelyKernel/lively.next

Bringing back `lively.app`

Open
#988 3 comments 0 reactions 2 assignees Claimed by @linusha View on GitHub
✨ enhancement 🏥 help wanted 💬 ideas welcome
Dominant language
JavaScript
Stars
90
Forks
20
PR merge metrics
No merged PRs in 30d

Description

In the past, we ran into situations where a missing rebuild or an outdated version of `git` lead to problems. This happened multiple times and with technically-able users. As `lively.next` has the explicit goal of fostering collaboration between designers and programmers, I think this constitutes a call for action on our side:

**Retrieving a working `lively.next` setup and keeping it up to date needs to become easier.** Now, it is a bit ironic that we only recently scrapped the attempts of a working `docker` setup we had. But in any way I do not think `docker` constitutes a good solution to this problem (and it never meant to), as the end result would again be people needing to copy-paste some `docker` commands they do not understand.

I though about two ways to tackle this:

1. Bringing back a lively executable, that takes care of installing/updating lively and spins up a server.
2. Provide a stable lively server that people can just connect to with their browser.

---

# Server Driven Approach

I think we would actually have everything ready for this to function. Actually promoting this would probably require us to spend (significant?) amounts of time thinking about user separation and other security measures, but for a select group of users that would be of less importance, I'd think.
However, I see some major downsides to this approach:

1. It would require us to actually administer a server, which might or might not be much work. With the current velocity with which we proceed, it would definitively be something that we would need to think of often.
2. What I think is more problematic: It actually distances the users quite a bit from what they actually do, most significantly from their file-system. As `lively.project` is still in its infancy, I have worries that users might loose data and cannot easily restore/save/backup parts of their work when `lively.project` falls flat under them.

Stuff like spontaneous builds would also become much harder, forcing each retrieval of an artifact to go through GitHub. I have automagical actions for this nearly ready, but it would still constitute an annoyance.

# `lively.app`

This leaves us with the option of a prepared lively application that can just be installed/executed like any other program.
Concretely, this would mean providing an electron app or something similar (more on that later).

## Dependencies

The most obvious hurdle to take is that, at the moment, we require quite a some dependencies. As these are to be installed as native applications, we cannot just bundle them up (at least not with far too much involvement, i.e., thinking about platforms, following release cycles,...). Let's consider our list from the `README`:

- node.js v18.12.1
- git

**These are actually important. More on that below.**

- sed or gsed on MacOs
- ss or netstat on MacOs
- perl
- python3 with sultan installed

These are only used inside of various scripts (mainly the bulk testing script and the artifact checker). I'd argue that these are not important for the average user. The overall goal of this undertaking should not be to move everyone onto the app, but the workflow of us as core developers will probably remain unchanged. T**hus, we can just ignore those, when thinking about app support.**

- brotli

Here, I am unsure. Looks like the freezer already uses this via a npm library - correct, @merryman?

- aspell

We provide spelling correction via a command inside of lively. While I think this feature is overall quite bare-bones at the moment and could use some love in general, we do not have any special requirements that I am aware of, so we will be able to switch to some JS library (there are plenty of options).

## How to Create an Executable

As we need to bundle a `node` application, we will need to use something like `electron`, `nw.js` or `pkg`. There are other tools available, but these are the ones I looked at. Using any one of these will take care of our node dependency.

### Which tool to use?

From looking at it, `electron` seems to be the best documented of the ones listed here. I'd prefer using that, to be honest. An additional benefit of using `electron` would be that we would probably come quite far with the `lively.app` basis that we already had in the past. The only advantage I would see with using `nw.js` is that we were thinking about bundling a complete lively app at one point, that utilizes the option of calling the `node` API in the browser, to get an improved debugging experience inside of lively. Pursuing this idea would then require another switch.

### `git`

This leaves us with our dependency on `git`, which has become quite the dealbreaker since the introduction of `lively.project`. While I think installing this would be alright, even for non-technical users (we could still improve our installation instructions), we could also experiment with getting rid of calls to a native `git` application from within the client.
For this, I looked at the available `JS`/`node` libraries for git. The best option we have (since `nodegit` seems to be a nightmare in with regards to interoperability) seems to be [`isomorphic-git`](https://isomorphic-git.org). With that it is possible to operate directly on a git repository in a `git` compatible way, i.g. actual files get manipulated, which is what we'd want. I identified the following blockers:

1. `isomorphic-git` does not provide all operations `git` provides and the output is obviously different. Before proceeding we thus should:
- [ ] check all git operations we need and see if we can map them
- [ ] if we were to proceed with this, we would need to spend some time enhancing the code where ever we use git in order to account for the different output
2. `isomorphic-git` needs a FileSystem in order to operate. On `node` side, one can just use `fs`. This is the easier option, but it would mean that we would need to use `evaluateOnServer` quite often and I am undecided how desirable that is. The other option would be to experiment with writing our `ShellClientResource` in a way that it fulfills the requirements that `isomorphic-git` has with regards to a file system. The list of operations that one needs to support is the following (taken from [here](https://isomorphic-git.org/docs/en/fs)):

```
Using the "promise" API (preferred)

A "promise" fs object must implement the same set functions as a "callback" implementation, but it implements the promisified versions, and they should all be on a property called promises:

[fs.promises.readFile(path[, options])](https://nodejs.org/api/fs.html#fs_fspromises_readfile_path_options)
[fs.promises.writeFile(file, data[, options])](https://nodejs.org/api/fs.html#fs_fspromises_writefile_file_data_options)
[fs.promises.unlink(path)](https://nodejs.org/api/fs.html#fs_fspromises_unlink_path)
[fs.promises.readdir(path[, options])](https://nodejs.org/api/fs.html#fs_fspromises_readdir_path_options)
[fs.promises.mkdir(path[, mode])](https://nodejs.org/api/fs.html#fs_fspromises_mkdir_path_options)
[fs.promises.rmdir(path)](https://nodejs.org/api/fs.html#fs_fspromises_rmdir_path)
[fs.promises.stat(path[, options])](https://nodejs.org/api/fs.html#fs_fspromises_stat_path_options)
[fs.promises.lstat(path[, options])](https://nodejs.org/api/fs.html#fs_fspromises_lstat_path_options)
[fs.promises.readlink(path[, options])](https://nodejs.org/api/fs.html#fs_fspromises_readlink_path_options) (optional [¹](https://isomorphic-git.org/docs/en/fs#footnote-1))
[fs.promises.symlink(target, path[, type])](https://nodejs.org/api/fs.html#fs_fspromises_symlink_target_path_type) (optional [¹](https://isomorphic-git.org/docs/en/fs#footnote-1))
[fs.promises.chmod(path, mode)](https://nodejs.org/api/fs.html#fs_fspromises_chmod_path_mode) (optional [²](https://isomorphic-git.org/docs/en/fs#footnote-2))

¹ readlink and symlink are only needed to work with git repos that contain symlinks.

² Right now, isomorphic-git rewrites the file if it needs to change its mode. In the future, if chmod is available it will use that.
```
3. There seems to be different experiences with regards to how well SSH works with `isomorphic-git`. We should
- [ ] check whether we actually rely on SSH or if we are good to go with HTTPS only (my notes are inconclusive about this :clown_face:)

4. Using `isomorphic-git` for pushing and pulling makes the use of a proxy mandatory. `isomorphic-git` provides one that can be used for "small projects and testing". It might also be that `makeProxied` suffices.
- [ ] investigate this!
---

Summing up, I think a good approach would be to provide an electron app and utilize `isomorphic-git` in order to not need a native `git` installation for `lively.project` to work.
However, I understand this post merely as an (extensive) starting point for a discussion on how to proceed. I deem this problem to be quite complex and would like to minimize the risk of quick shots.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.