osquery / osquery/osquery

Send watcher logs through an IPC to osquery main process

Open
#7,562 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

blueprint core daemon
Dominant language
C++
Stars
23.6k
Forks
2.6k
Avg merge
6d 7h
Merged PRs (30d)
14

Description

Blueprint

The origin of this issue: https://github.com/osquery/osquery/issues/6790

Correct, the "watchdog process" is intentionally as small as possible, meaning it has no logger functionality.

The way to implement this is to include a small IPC between the watchdog and osquery. I have wanted to implement this before but there is no quick and easy IPC that spans the platforms we support. I think this is possible but it's a small or medium sized project someone has to work on (looking for volunteers!).

Originally posted by @theopolis in https://github.com/osquery/osquery/issues/6790#issuecomment-748553883

First of all, this is the first time I colaborate in this project, so I tried to do this blueprint the best way I could. Also, I don't know osquery in deep yet.

I'm suggesting to use message_queue from the boost library. It creates a shared memory area to exchange messages between a sender and a listener on different processes.

I've tested it on Linux and Windows, works fine. However, it creates temporary files on Windows and other OS that doesn't support POSIX shared memory objects. More details.

The idea is to send the messages from the Watcher process, receive them in osquery and use the standard LOG API. This way, the IPC logs will be created as the rest of the logs.

Message_queue needs to reserve a static memory area for a number of messages with a fixed length. Support variable length and resizing the area is possible, but requires a lot of work and the code will be more difficult to maintain. I suggest to reserve enough area for the longest message (i.e. 1024 characters) and a queue of 4 messages, would be configurable by flags.

sequenceDiagram
    participant Watcher
    participant message_queue
    participant IPC_logger
    participant Osquery

    Watcher->>+Watcher: Initializes
    Watcher->>+message_queue: Init
    message_queue->>+message_queue: Creates fixed length shared memory
    Osquery->>+Osquery: Initializes
    Osquery->>+IPC_logger: Launches thread
    IPC_logger->>+IPC_logger: Opens fixed length shared memory
    loop
        IPC_logger->>+IPC_logger: Listens
        Watcher->>+Watcher: Serializes message
        Watcher->>+message_queue: send(msg, sizeof(msg), severity);
        alt Free space available
            message_queue->>+message_queue: Move to shared memory
        else No free space available
            message_queue->>+message_queue: Keep it on local until free space
        end
        message_queue->>+IPC_logger: Notify
        IPC_logger->>+IPC_logger: Read most prioritary message
        IPC_logger->>+Osquery: LOG(severity) << msg
    end

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named; begin with origin issue #6790 and the Boost interprocess message_queue documentation. Done means watcher messages reach the osquery main process through a cross-platform IPC and are emitted through the standard LOG API.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.