Stop Sam Local from rebuilding layers on every invoke

Currently it seems that `sam local start-api` will build any lambda layers on **every** request. This ends up creating a very slow dev experience.

Since `sam build` seems to be able to determine if a layer changes or not the local api behaviour not just assume the layer hasn't changed. If it does change `sam build` will rebuild it anyway?

it seems like passing the `--warm-containers=EAGER` as mentioned in this thread: https://github.com/Envek/aws-sam-typescript-layers-example/issues/11#issuecomment-1036057433 stops the layers from building each time but this isn't really a nice solution since a lot of times during testing I want to just test a single lambda I don't want to have to wait for all the lambda's to build layers again (each time I make a change to the code).

Is there another option I'm missing?