Proposal: resolve node module dependencies during bsc (ropm-style)

BACKGROUND

We're attempting to build a Rush monorepo, which uses PNPM workspaces with symlinks for node_modules.

We can have "application using libraries" using built-in bsc file mapping functionality, for example:

apps
  roadrunner/
    src/
      source/
        Main.brs
/libraries/
  dynamite/
    src/
      source/
        Dynamite.brs
      components/
        Dynamite.xml
      images/
        explosion.gif

{
  "files": [
    "manifest",
    "source/**/*.*",
    "components/**/*.*",
    "images/**/*.*",
    {
      "src": "../node_modules/@acme/dynamite/src/source/**/*.*",
      "dest": "source/dynamite"
    },
    {
      "src": "../node_modules/@acme/dynamite/src/components/**/*.*",
      "dest": "components/dynamite"
    },
    {
      "src": "../node_modules/@acme/dynamite/src/images/**/*.*",
      "dest": "images/dynamite"
    }
  ]
}

The roadrunner app can now use the dynamite library in a reasonable way, but there are some caveats:

• No automatic re-namespacing, like ropm -- you have to make sure each individual library doesn't introduce any conflicts with each other.
• If you want App depending on Library A depending on Library B, the App needs to list all the remaps for all the libraries (including transitive dependencies). That's not ideal!

PROPOSAL

What if bsc itself could detect and remap node modules folders.

For example, I imagine this syntax: <package-name>:/path/to/file (<package-name> replaces pkg).

The desired behavior would be that in my source folder, I might type:

import "@acme/dynamite:/source/Dynamite.bs"

Then brighterscript would automatically do a few magical things:

• First, it understands that @acme/dynamite prefix means ./node_modules/@acme/dynamite/
• Inside that folder is a src/ folder, and in there is source/Dynamite.bs.
• Now that we've found the file, we need to automatically remap it
  • It should act just like we've added an entry to bsconfig.json, even though we haven't
  • We could mimic ropm: source/roku_modules/acme_dynamite/Dynamite.bs

With this scheme, transitive dependencies are automatically handled!

If LibraryA depends on LibraryB, then the same logic will handle it without issue:

• e.g.: ./node_modules/LibraryA/src/source/LibraryA.bs depends on LibraryB:/source/LibraryB.bs
• During compile it knows this file is at ./node_modules/LibraryA/node_modules/LibraryB/src/source/LibraryB.bs
• The final output is:
  • source/roku_modules/LibraryA/LibraryA.brs
  • source/roku_modules/LibraryB/LibraryB.brs

What about other types of URLs, like images and resources?

Because this new capability is a URI prefix, it can be used for any source code that can reference pkg:, like references to images or JSON files!

Once again, a reference to @acme/dynamite:/images/explosion.gif should be modified to pkg:/images/roku_modules/dynamite/explosion.gif, and the file copied to that location in the output.

What about components?

This is where I need the most help -- I'm not sure how I can reference components.

I have one idea, which is a variation on the above: maybe if you import anything using a package reference, it automatically detects and generates the mappings for all subfolders, including components.

In this case, if any .bs file imports "@acme/dynamite:/source/anything.bs", that would trigger the entire package (every subfolder it might have, like images, resources, components) to be copied into the appropriate roku_modules structures, even if you don't reference them.

(I'm open to other approaches.)

Namespacing?

All of the above really nicely solves the copying-and-remapping mechanic, and gives you an experience that's close to TypeScript's require or import. But, it doesn't solve the namespacing problem, e.g.: you have two libraries that raw-define function add(). How can you "namespace" the code using the library name?

I think we could take ropm's approach and default to prefixing every function with acme_dynamite_ (etc.). If these names got long, then in theory we could support custom remapping in bsconfig:

{
    "packages": {
        "@acme/dynamite": {
            "namespace": "Dynamite"
        }
    }
}

You shouldn't need to list transitive dependencies here, because unless you are actually calling them, you won't need to know what they are named and so custom remapping isn't required.

Next steps

I realize this is a wall of text, hoping that I've explained what we are trying to do (work within the symlinking / pnpm folder structure and not use ropm's "copy files" approach). Very open to poking holes in the design and additional discussion!

Start at the bsc file-mapping and bsconfig.json handling entry points, then trace how node_modules paths and package-reference URIs could be resolved. Review the proposed transitive dependency, resource, component, and namespacing cases. Done would require an agreed design and corresponding compiler behavior, including tests for the supported package-reference syntax.