ArduPilot / ArduPilot/MAVProxy
Chat module issues & enhancements list
- Dominant language
- Python
- Stars
- 595
- Forks
- 773
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 18
Description
This is a consolidated list of issues and enhancements for the MAVProxy chat module added with PR https://github.com/ArduPilot/MAVProxy/pull/1278
Bugs:
- [ ] error when reloading modules ([issue](https://github.com/ArduPilot/MAVProxy/issues/1342))
- [ ] typing "chat show" into the MAVProxy produces an innocuous error. We should either remove that completely or make it re-show the chat GUI in case the user has closed it
Enhancements:
- [ ] Store thread-id and allow users to re-use the thread (perhaps this should be the default)
- [ ] Add version support so that chat module can find compatible assistants (e.g. append "VerX.Y" to the assistant name and use during init)
- [ ] Add function to allow AI to send arbitrary mavlink messages to the vehicle
- [ ] Support multiple vehicles
References:
- [MAVProxy wiki is here](https://ardupilot.org/mavproxy/docs/modules/chat.html)
- ArduPilot "ai" discord channel is a good place for real-time chatting
- OpenAI's API reference: https://platform.openai.com/docs/api-reference/assistants
Resolved items:
- [x] Voice-to-text recording is currently hard-coded to 5sec. We should change this to a "Listen". So when the user pushes the button it will stay depressed and record when the volume is over a given threshold. Once it detects the end of a sentence (e.g. volume drops below the threshold for maybe 2 or 3 seconds) it sends to openai. https://github.com/ArduPilot/MAVProxy/pull/1435
- [x] get around OpenAI unreliability in accessing files.
- [x] Adding a Cancel button to cancel a run that's been sent to the assistant.
- [x] Support "streaming" which is now available from OpenAI ([see docs here](https://platform.openai.com/docs/api-reference/assistants-streaming))
- [x] pass parameter units, range, human-readable-text with each parameter request? PR: https://github.com/ArduPilot/MAVProxy/pull/1306
- [x] try to reduce lag by lowering the "max_tokens" value sent with requests ([see reference here](https://platform.openai.com/docs/guides/production-best-practices/improving-latencies)) -- seems out-of-date
- [x] Handle OpenAI API Errors and display a better warning to the user ([see docs here](https://platform.openai.com/docs/guides/error-codes/python-library-error-types)). Common errors include AuthenticationError and RateLimitError
- [x] Allow AI to retrieve and set parameters (may require uploading vehicle specific parameter description file)
- [x] Add function to allow AI to retrieve the latest mavlink message(s) sent by the vehicle (probably using something like, mpstate.master().messages.get('HEARTBEAT', None)) -- completed https://github.com/ArduPilot/MAVProxy/pull/1286
- [x] Improve reliability of takeoff command and "please fly here". -- see PR https://github.com/ArduPilot/MAVProxy/pull/1283
- [x] investigate occasional freezing where the Input text box remains greyed out probably waiting endlessly for a reply from the Assistant that never comes. We should probably add a status bar to the bottom of the screen and display the "latest_run.status". -- not seeing anymore, perhaps was a server side issue
- [x] Under Windows the user input text is grey instead of red meaning it is slightly more difficult to see what is your text and what is the chatbot's reply. -- fixed by https://github.com/ArduPilot/MAVProxy/pull/1288
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the MAVProxy chat module introduced by PR #1278 and read the chat module wiki page and the linked issue or PR for the specific unchecked item you select. Define one enhancement or bug fix with its own acceptance criteria; this consolidated list is done only when that selected item is implemented and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100