Event logging

Verification

• I searched for similar feature request and found none was relevant.

Pitch us your idea!

Storing history of events in flash memory

Description

Background

Hi there, I'm embedded software engineer and I like to use PineTime watch. It is my first created feature for open source project, so let me know if I do something wrong with this ticket. I would like to help with adding lots of different features like showing step counter history (#788), showing battery percentage history (on watch), storing ble debug events, etc.

But I found that most of them require storing different events to non volatile memory. I have experience with using event log for these purpose. I know that this feature is not visible for end-user, but it is helpful for implementing lots other features.

Feature description

It is possible to store lots of events on flash memory using ring buffer (i.e. old events will be overwritten by new events).

Usage:

1. History of battery voltage

We can store battery voltage value each an hour, then we can build a plot for showing battery discharging during some period of time.

2. History of step counter

It is the same idea as with battery voltage, but we should store step counter. We can store it once a day or more often.

3. Crash dump

We can use event log for storing events with reason of fault in case if hard fault was detected.

4. Debugging

I found it useful to store some events like BLE connection, low battery level, unexpected state of some subsystem, etc to eventlog. Then this data can be used for signalizing some errors or unexpected behavior.

4.5 Storing startup data

For debugging purpose we can store some startup up data, like FW version, battery voltage level, reset reason, etc. then this data can be used for signalizing some issues as it provides all history of FW updating, reason of rebooting and other.

5. Capture user data for getting some statistics

It is possible to add some test mode which allows to save some user data, like accelerometer, HRM, etc. Then these data might be used for improving step counting algorithm or some other subsystem.

Implementation idea (generic part)

4byte event alignment

We have a few X kb pages for storing Eventlog (at least 2). Each event takes at least 4 bytes for storing it. First bit of each 4 bytes is used as flag if it is last part of this event or not (like UTF-8). (Most events should use only 4 bytes, later we can allocate more ranges for storing long events). The rest bits of first byte is used for selecting event type. And the rest bytes are used for event data.
Examples:

• 80 00 00 05 // Event type = 0, value = 50ms. Timestamp. This type of events uses the rest 3 bytes for storing time (10 ms units) which elapsed from last event. So we should save this event before storing each another new event in case if elapsed more than 10ms.
• 81 00 00 00 // Event type = 1, value = 0. Single event. This type of event uses the rest 3 bytes (24kk combinations) for storing a single events like BLE client connected, BLE client disconected or Battery voltage was dropped less than 2 volt. Usually I prefer to split 24kk values to some ranges (normal events, warning events, debug events)
• 02 00 00 00 80 00 00 00 // Event type = 2, value = 0 (6 bytes + 7 bit) this is example of long event. You can see that it uses 1 bit of each 4 bytes as sing of end of the event. (like UTF-8).
• 82 00 00 02 // Event type = 2, value = 2. Eventlog page swapping event. This event is used after swapping page of eventlog. data of this event shows how many page swapping occurs for all time. It also used for detecting the last event page on initialization.
• ff ff ff ff // Event type 7f, value ff ff ff. This is a fake event which means that the rest area of eventlog page is free.

Ring buffer usage

Ring buffer is used for erasing older events and storing newer events. Example: we use 6 pages 4 kB each of them. It means that we are able to store 1024 events per each page (Actually I reserve 24 events so we have 1000 events for storing per page). On initialization we read first event of each event page to see event with page swapping which have highest value for latest eventlog page. Then we search event 0xffffffff which means that the rest events of this page are free. In case if we add 1000th event we start eventlog procedure which should erase oldest eventlog page, then write eventlog page swapping event to this page. Now we are able to use this page for storing new events. All events which occurs during page swapping should be stored to the reserved area of last eventlog page (24 events).

Reserve a few events at the end.

Usually I reserve some space at the end of each page for storing crash dump which might occurring during page swapping when new page is not available for storing eventlog.

Option for implementation:

1. Using a few pages of internal nrf52 micro flash memory.

I have experience of working with this uC and usually I use a few pages of internal flash memory for storing eventlog. I know that you are using MCU bootloader which probably erase all this memory during FW updating. Probably it is fine, but it means that we can't receive history of FW updating versions, etc.

Cons:

1. Eventlog might be erased by MCU bootloader on each FW update
2. Part of internal flash memory will be reserved for storing eventlog.

2. Using some low level API for storing events in SPI memory.

Cons:

1. it might be not possible for storing fault events.
2. Part of SPI flash memory will be reserved for storing eventlog.

3. Use existing FS API for storing eventlog to the file

Cons:

1. I'm not sure if ring buffer logic is make sense for this approach.
2. it might be not possible for storing fault events.

No source files, tests, or entry points are named. Start by reviewing the existing internal flash, SPI-memory, and filesystem APIs described in the issue, then determine which storage path can support a ring buffer and fault-time writes. Done means an agreed scope, storage design, and validation plan for event history, crash data, and page rotation.