beetbox / beetbox/beets

Beets History Logging

Open
#1,392 4 comments 0 reactions 0 assignees View on GitHub
discussion
Dominant language
Python
Stars
15.7k
Forks
2.1k
Avg merge
4d 21h
Merged PRs (30d)
31

Description

Hi there.

So I've been looking to implement a beets work history log. This will essentially log all the actions beets completes, along with any supplementary information. A user then is able to see this via the beet interface and have a more granular view of the updates beets makes and to what. It will in the future allow the user to rollback changes (within reason) to a specific point in time. This means users can "test" commands, and commit changes. In addition if a feature to run beets commands in the background is implemented, a log of work can be recorded.

I've penned up an initial design to this functionality, with the hopes others can contribute and we can work as a community to see this implemented. This is a work in progress so anyone can suggest changes, etc.

I think this development should happen in 2 phases. They are as follows:
- **Phase 1** - Implement the database and logging of beet's actions
- **Phase 2** - Implement the rollback functionality
# Phase 1
#### > Database Design

I believe the following columns will be required for the new history database.
- id - auto incremented ID number
- action - This will hold the action that beets is currently doing. This will be discussed more in the Logs section
- description/path - will hold the the item which the action is acting on
- active - this is a flag to indicate which row is active. This will come into effect, when we implement phase 2, which has the ability to rollback to a previous history. Rolling back, will invalidate all the logs below it, if a new command is run from a rolled back position. Further thought will need to go into then on implementation.
#### > Logs

The logs will be able to log the following actions:

File Actions
- Move - A move of a file
- Copy - A copy of a file
- Link - Creation of hard or soft link
- Unlink - Remove of a hard or soft link
- Rename - Physical rename of a file
- Delete - The physical deletion of a file - This will unable to be reversed!

Library Actions
- Add - Addition of item in beets
- Delete - Deletion of an item
- Write - Essentially every change to meta data is a write. The addition of lyrics, is a write, but the removal of lyrics is still a write of blank text

General Actions
- Command - The history should be able to log the command that was executed with beets. The description will contain the command and the flags that were presented to it.

Each of these actions can and should be logged, and with each (except delete) a reversal process exists, which can be used to rollback changes.

With each action we would also needs to store a serialised version of the item, before the change is made. e.g. before a `beet write` writes to a file, we should log the full data of the item, and then run the command. This means when we rollback, we have have the information to rollback to.
#### > Config

For this phase I see only the following configuration items being relevant at the minute. Happy to accept more
- `history -> enabled` - A boolean flag to enable the logging functionality
- `history -> loglimit` - A number defining the maximum number of logs to hold. Database will delete any older logs. It may be beneficial to also either limit logs by DB size ( which can be difficult ) or date, but again that introduces its own issues. Number of rows should be sufficient.
#### > Process

The process in which this would work would be simple. For each write to the database, the required information is stored. As the transaction class is abstracted, this write to the history can be implemented in a central location. It is important to note that there will be some slowdown in writing due to each request being logged. However this slowdown, is the tradeoff for implementing such feature.

As the logs go in, a process will be required to remove old logs, via a SQL command.
#### > UI

I believe the history UI should be fairly simple, called with something similar to `beet history` it will display a table of information, which will be pretty much a dump of the new history table. This will allow the user to see what beets has done.
# Phase 2

I wont discuss much of phase 2 at the moment, but the premise is to effectively pass beets a number of the history item, and beets will start a process to run though every log from the present back to the specified number, and reverse every action to get beets back to that specific point in time.

Further details and design of this will be created at a later point.
# Conclusion

I would like to hear all of your thought. Also there are some architectural challenges to face, like how we can add another table to beets but separate it from the library. Right now item and album are subclasses of Database. And we really need to figure out the best method of sorting and accessing the logs.

Contributor guide

Open the contributing guide

Research direction

The issue names no files or tests; start by reading the transaction abstraction and the Database hierarchy for item and album, then inspect how the beet command interface is organized. The proposal describes a two-phase history system, but implementation scope and acceptance criteria are not settled beyond logging actions and exposing them through a beet history command.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sql
Domain
cli, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.