On using hard links: discussion on the choice and on alternative approaches

Motivation

Recently I encountered an issue that hard links in docker images may not be handled properly by Google Cloud. In some specific scenario the files may also go missing. There is a detailed issue report I created on this problem: https://github.com/starrify/google-cloud-run-docker-hard-link-issue

In the actual project of mine, the missing file was a build outcome by node-gyp. The service failed at run time since Node.js was unable to load the already-missing file.

In my example it is surely not node-gyp to blame for the mishandling of hard links. Yet I'm still creating this ticket for discussing the usage of hard links by node-gyp, and possibly better ways to that.

Current State: Prefer hard links over copies

As of the current version (b6e1cc71279092552f9e224be245bf91e6d0c981), it is observed that node-gyp prefers hard links when copying the building outcome. Here is code example for make:
https://github.com/nodejs/node-gyp/blob/b6e1cc71279092552f9e224be245bf91e6d0c981/gyp/pylib/gyp/generator/make.py#L401

The above code is found to have been originally introduced in 972780bd:

- cmd_copy = rm -rf "$@" && cp %(copy_archive_args)s "$<" "$@"
+ cmd_copy = ln -f "$<" "$@" 2>/dev/null || (rm -rf "$@" && cp %(copy_archive_args)s "$<" "$@")

Previously: Hard links to soft links..

By searching through the project history I noticed commit 2d948276, whose commit message was simply don't hardlink. In this commit it changed from making hard links to soft links:

- cmd_copy = ln -f "$<" "$@" 2>/dev/null || (rm -rf "$@" && cp -af "$<" "$@")
+ cmd_copy = ln -sf "$<" "$@" 2>/dev/null || (rm -rf "$@" && cp -af "$<" "$@")

Previously: ..then to no link at all

I noticed as well a commit 9049241f saying don't use links at all, just copy the files instead:

- cmd_copy = ln -f "$<" "$@" 2>/dev/null || (rm -rf "$@" && cp %(copy_archive_args)s "$<" "$@")
+ cmd_copy = rm -rf "$@" && cp %(copy_archive_args)s "$<" "$@"

Point of discussion

I feel rather curious about the reasons behind these decisions. For example, why hard links were deemed as inappropriate and got removed the other day? Is any of those reasons still apply today?

I guess it might be good to list all benefits and potential drawbacks of each approach (hard links, soft links, copying, etc.) somewhere for comparison.

Plus, assuming we are to keep the current approach unchanged, which is to prefer hard links over copying, would it be good enough to introduce some optional switch to alter the behavior. For example (pseudo code):

if has_env_var(NODE_GYP_COPY_COMMAND_FORCE_ONLY_COPY) then
  cmd_copy = cp
else
  cmd_copy = ln || cp
endif

Begin with gyp/pylib/gyp/generator/make.py around line 401 and review the cited commits that changed the copy command. Compare the hard-link, soft-link, and copy approaches described in the issue. Done would be an agreed, scoped decision about the behavior, with any selected change and its validation documented.