ArduPilot / ArduPilot/MAVProxy

Chat module issues & enhancements list

Open
#1,281 5 comments 0 reactions 0 assignees View on GitHub
bug enhancement
Dominant language
Python
Stars
595
Forks
773
Avg merge
2d 6h
Merged PRs (30d)
18

Description

This is a consolidated list of issues and enhancements for the MAVProxy chat module added with PR https://github.com/ArduPilot/MAVProxy/pull/1278

Bugs:

- [ ] error when reloading modules ([issue](https://github.com/ArduPilot/MAVProxy/issues/1342))
- [ ] typing "chat show" into the MAVProxy produces an innocuous error. We should either remove that completely or make it re-show the chat GUI in case the user has closed it

Enhancements:

- [ ] Store thread-id and allow users to re-use the thread (perhaps this should be the default)
- [ ] Add version support so that chat module can find compatible assistants (e.g. append "VerX.Y" to the assistant name and use during init)
- [ ] Add function to allow AI to send arbitrary mavlink messages to the vehicle
- [ ] Support multiple vehicles

References:

- [MAVProxy wiki is here](https://ardupilot.org/mavproxy/docs/modules/chat.html)
- ArduPilot "ai" discord channel is a good place for real-time chatting
- OpenAI's API reference: https://platform.openai.com/docs/api-reference/assistants

Resolved items:

- [x] Voice-to-text recording is currently hard-coded to 5sec. We should change this to a "Listen". So when the user pushes the button it will stay depressed and record when the volume is over a given threshold. Once it detects the end of a sentence (e.g. volume drops below the threshold for maybe 2 or 3 seconds) it sends to openai. https://github.com/ArduPilot/MAVProxy/pull/1435
- [x] get around OpenAI unreliability in accessing files.
- [x] Adding a Cancel button to cancel a run that's been sent to the assistant.
- [x] Support "streaming" which is now available from OpenAI ([see docs here](https://platform.openai.com/docs/api-reference/assistants-streaming))
- [x] pass parameter units, range, human-readable-text with each parameter request? PR: https://github.com/ArduPilot/MAVProxy/pull/1306
- [x] try to reduce lag by lowering the "max_tokens" value sent with requests ([see reference here](https://platform.openai.com/docs/guides/production-best-practices/improving-latencies)) -- seems out-of-date
- [x] Handle OpenAI API Errors and display a better warning to the user ([see docs here](https://platform.openai.com/docs/guides/error-codes/python-library-error-types)). Common errors include AuthenticationError and RateLimitError
- [x] Allow AI to retrieve and set parameters (may require uploading vehicle specific parameter description file)
- [x] Add function to allow AI to retrieve the latest mavlink message(s) sent by the vehicle (probably using something like, mpstate.master().messages.get('HEARTBEAT', None)) -- completed https://github.com/ArduPilot/MAVProxy/pull/1286
- [x] Improve reliability of takeoff command and "please fly here". -- see PR https://github.com/ArduPilot/MAVProxy/pull/1283
- [x] investigate occasional freezing where the Input text box remains greyed out probably waiting endlessly for a reply from the Assistant that never comes. We should probably add a status bar to the bottom of the screen and display the "latest_run.status". -- not seeing anymore, perhaps was a server side issue
- [x] Under Windows the user input text is grey instead of red meaning it is slightly more difficult to see what is your text and what is the chatbot's reply. -- fixed by https://github.com/ArduPilot/MAVProxy/pull/1288

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the MAVProxy chat module introduced by PR #1278 and read the chat module wiki page and the linked issue or PR for the specific unchecked item you select. Define one enhancement or bug fix with its own acceptance criteria; this consolidated list is done only when that selected item is implemented and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.