arduino / arduino/Arduino

How to use an (avr) core from git?

Open
#9,237 3 comments 0 reactions 0 assignees View on GitHub
arduino-builder arduino-cli Component: Core question
Dominant language
Java
Stars
14.6k
Forks
7k
PR merge metrics
No merged PRs in 30d

Description

I often work on the AVR core files, so I have a checkout of the ArduinoCore-avr repository on my system that I want to usually use. Previously, I just had a symlink from `~/Arduino/hardware/arduino/avr` to the `ArduinoCore-avr` checkout and that was used as the AVR core by the IDE.

However, I've just updated to the latest git (I'm also running the IDE from git) and this no longer works. I'm not sure what change broke it exactly, since reverting to (what I think was) the previous version does not unbreak it (so maybe I had some extra local changes before in e.g. the `build/linux` folder, or maybe a custom arduino-builder version too?).

The problem I'm seeing now is that:
- Arduino-builder gets `-hardware` options in order "build/linux/work/hardware", "~/.arduino15/packages", "Sketchbook/hardware": https://github.com/arduino/Arduino/blob/6116a8e9206124329e0c8b58e828eb5b83b6055a/arduino-core/src/cc/arduino/Compiler.java#L241-L243

It seems that this also determines the priority, meaning that any bundled AVR-core, or any boards-manager installed core is used before my core from git in my sketchbook. I do not have any boards-manager installed core, so to use my custom core I removed the `build/linux/work/hardware/arduino` folder, which makes arduino-builder use the sketchbook core, but then the IDE seems to no longer know about avrdude:

java.io.IOException: Cannot run program "{runtime.tools.avrdude.path}/bin/avrdude": error=2, No such file or directory

This is confirmed by `--get-pref`, running `arduino --get-pref 2>/dev/null |grep avrdude` returns nothing.

I guess this makes sense: Tools are defined by `.json` files, but I have none (for avr) in my `~/.arduino15` directory. I do have `build/linux/work/hardware/package_index_bundled.json` which defines an avrdude tool, but (I guess) since there is no corresponding AVR core directory (I deleted it), the tool is not loaded.

I tried:
- Putting a symlink to the git avr core in `build/linux/work/hardware/arduino/avr`, which works, but only if the version number in the `package_index_bundled.json` file matches the version number in the core's `platform.txt`. If not, the avrdude is not made available. Since I sometimes jump around different versions, this is not really convenient.
- Keeping the original `build/linux/work/hardware/arduino/avr` directory, but giving my core from git a higher version number in its platform.txt (hoping that arduino-builder would use the highest version number), but that did not work (avrdude works, but arduino-builder uses the bundled avr core).
- Installing an avr core with the boards manager and replace `build/linux/work/hardware/arduino/avr` with a symlink to the git core. I thought this woujld let the board-manager installed version make its tools available (so avrdude would work), but the symlink in `build/linux/...` has higher priority for arduino-builder, so that core is used. However, I now get `java.io.IOException: Cannot run program "___REMOVE___/bin/avrdude": error=2, No such file or directory`, so apparently it does recognize the tool, but it [has a null path](https://github.com/arduino/Arduino/blob/master/arduino-core/src/processing/app/BaseNoGui.java#L702-L706) somehow...
- Keeping the original bundled core and linking to the git core as `~/Arduino/hardware/arduino-git/avr`. This changes the core name and lets the core be installed alongside the regular avr core rather than overriding it (which causes all boards to show up in the boards menu twice). This needs a change in the git core's name in platform.txt to be able to distinguish them and makes the board menu *even longer*, but I guess this is a reasonable way to do this.

What is noteworthy is that this problem does not seem to occur in arduino-builder, which has no trouble finding `avr-gcc` (which is also a tool and is also referenced through `runtime.tools.avr-gcc.path`), so it seems the IDE and arduino-builder load their tools differently (but I've never really dug in to this stuff to really understand it...).

I guess it comes down to the handling of cores that do not have a corresponding `.json` file to define their dependencies. I guess this also goes for other third-party cores installed directly into the sketchbook, though those typically use the tools from the e.g. the official AVR or SAM cores. Since these third-party cores have a distinct name and offer boards distinct from the official boards, the priority used by arduino-builder is not a problem for them. My `~/Arduino/hardware/arduino-git` approach above essentially turns the git core into a distinct core as well, which is why that works.

So I wonder: Are there better ways to support this usecase? If not, is there anything we should or could change to make this easier?

Contributor guide

Open the contributing guide

Research direction

Start with arduino-core/src/cc/arduino/Compiler.java around the -hardware options and arduino-core/src/processing/app/BaseNoGui.java around lines 702-706; compare how the IDE and arduino-builder resolve core tools. Done would be a documented or implemented reliable way to use a git AVR core while retaining avrdude and avoiding bundled-core conflicts.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.