Run FastFlowLM as a systemd service

Open
#541 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
linux
Domain
devops, operating-systems

Research direction

Start by reviewing the proposed /etc/fastflowlm/fastflowlm.conf and /etc/systemd/system/fastflowlm.service files, then check the systemd EnvironmentFile and ExecStart behavior. Done should mean FastFlowLM starts reliably as a service, supports the needed configuration, and accepts client connections, but the issue does not name repository files or tests.

Written by the indexing model from the issue text.

Description

Hi!

I want to run FastFlowLM on my local AI provider machine. Since I could not find the config files, I wrote some on my own.

I added a file

/etc/fastflowlm/fastflowlm.conf

with the content

Configuration for the FastFlowLM AI provider

Model to use

Model=gemma4-it:e4b

IP address and port clients connect to

Host=192.168.23.52
Port=52625

(There should be more variables here, but it seems systemd has problems to include them in the actual service file?)

and another file

/etc/systemd/system/fastflowlm.service

with the content

[Unit]
Description=FastFlowLM
After=network.target

[Service]
EnvironmentFile=/etc/fastflowlm/fastflowlm.conf
ExecStart=/opt/fastflowlm/bin/flm --host ${Host} -p ${Port} serve ${Model}
WorkingDirectory=/opt/fastflowlm/bin
User=andreas
Group=andreas
MemoryAccounting=true
MemoryHigh=16G
MemoryMax=16G
LimitAS=16G
LimitMEMLOCK=infinity
Restart=always
RestartSec=60

[Install]
WantedBy=multi-user.target

Run the service with

systemctl daemon-reload
systemctl start fastflowlm

Or enable it, so it runs at every boot

systemctl enable fastflowlm

I can run the service at least for now. There might be bugs in my config, since I am not an admin.

Have to write some code, so my own client can connect to it. Will do further testing then.

Sorry, if this has been discussed before.

Thanks a lot for the great project! So happy, that I can use the NPU in my strix point machine!

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/FastFlowLM

All issues in ROCm/FastFlowLM

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.