nextcloud / nextcloud/recognize

Modularization

Open
#680 3 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
PHP
Stars
699
Forks
68
Avg merge
1d 4h
Merged PRs (30d)
5

Description

Describe the feature you'd like to request

I think it would be very desirable to make use of the great structure of this app.

I wrote some issue (https://github.com/matiasdelellis/facerecognition/issues/210) some time ago. Many of my ideas from then match this app and I simply love this work.

Temporary worker:
It would be great to have some kind of separation between the app and the "workers". I think of a simple page with JavaScript, WASM and Web Workers API. If a (eg. admin) browser is opened at a workstation with a strong CPU/GPU the work would be done in the browser and make use of the computing power of this workstation. It is possible that every user can help to process its data by opening some special browser tab.

Permanent worker:
The same approach could be used by some Docker containers to provide a permanent workers at workstations (servers) with strong CPU/GPU. Users could decide to use their workstation with a strong GPU to process their photos more quickly.

Special workers:
If there is a interface definition there would be a easy way to use AI accelerators like Googles Coral USB TPU to do the work. If connected to a Raspberry Pi the app could make use of such hardware in form of a compliant docker container.

I think the existing approach with native and WASM workers builds a perfect base for this system.

Describe the solution you'd like

Define some clear interface. The interface could be used and provide processing power. Depending on the permissions the data is user specific (worker provided by a user) or global (server wide worker).

Describe alternatives you've considered

TensorFlow Serving is a simple REST API definition which would provide some kind of external inference solution. It might be a good way but would lack app specific "inside knowledge" which might be necessary.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by locating the existing native and WASM worker implementations and the media-processing entry point, then review TensorFlow Serving's REST interface as a reference for a worker contract. Done means a clearly documented interface covering worker permissions, user-specific or global data, and supported worker types.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, javascript, php, raspberry-pi, tensorflow, wasm
Domain
backend, devops, distributed-systems, embedded-iot, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.