Include non-imported assets in deployments

Describe the problem

A reasonably common pattern is for an endpoint to depend on files in the project directory that aren't visible to Vite because there's no import declaration involved — for example database.sqlite or documentation/getting-started.md. These endpoints work during development and in local preview, but as soon as the app is deployed they break, because the files are not included in the build.

Describe the proposed solution

Something like this:

import marked from 'marked';

// src/routes/documentation/[slug].js
export async function get({ params }) {
  const markdown = await import.meta.load(`documentation/${params.slug}.md`);

  return {
    body: marked(markdown.toString())
  };
}

This uses import.meta.load because it's easy to statically analyse and replace with an environment-appropriate implementation. In this case, we see that it's called with the expression `documentation/${params.slug}.md`, and we can easily turn that into a list of files in the project directory that match /documentation\/.+\.md/. An adapter creating lambdas can simply include those files in the function directory and provide a promisified fs.readFile implementation; an adapter creating a Cloudflare Worker could put the file in a KV store and provide an implementation that reads from it.

In some cases, an adapter might not have a way to read those files, and would be able to throw an error at build time rather than failing cryptically at runtime.

We can restrict the expressions that import.meta.load accepts to template literals and string concatenations.

Slightly related: #3850

Alternatives considered

Next.js solves this problem with the (unfortunately-named!) nft package. It statically analyses the build output and figures out which files are needed, including dependencies in node_modules that weren't included in the bundle and assets that are read from the filesystem.

Since static analysis is inherently limited, code needs to be written a certain way for files to be correctly located. nft's static analysis is very good — you can do this...

const cwd = process.cwd();
const file = path.resolve(cwd, `documentation/${slug}.md`);

...and get a list of dependencies that includes all the .md files in documentation. But you can't do this...

const contents = fs.readFileSync(`documentation/${slug}.md`);

...or this...

import { cwd } from './shared.js';
const file = path.resolve(cwd, `documentation/${slug}.md`);

...or this:

import { read } from './utils.js';
const contents = read(`documentation/${slug}.md`);

In addition, since we're using things like path and process, this can only work in Node-like environments, when ideally a SvelteKit app should happily render in an edge function.

This leads me to the conclusion that we're better off using something less idiomatic that will only work if you adhere to the 'rules', and which could theoretically be used across different environments.

It might still be useful to use nft to find external node_modules (in particular native dependencies that can't be bundled by Vite), so that they can be included in lambdas without further configuration. But that's a separate issue.

---

Another possibility is to use some sort of explicit configuration. Aside from being something of a burden (that doesn't solve the cross-environment problem), it would be difficult to come up with an API that works with function splitting (and doesn't result in all your documentation/*.md files being bundled with unrelated routes).

---

Finally, because someone is going to suggest it, I don't think we should do this:

import { load } from '$app/fs';

const db = await load('database.sqlite');

This takes us into an uncanny valley where load is sort of a normal function but not really — it looks like you should be able to memoize it or pass it somewhere as an argument, but you can't (at least not without breaking the static analysis guarantees we need).

Start by tracing how Vite builds the example src/routes/documentation/[slug].js entry point and how deployment adapters receive its output. Compare the proposed import.meta.load expression analysis with the lambda and Cloudflare Worker cases; done means non-imported matching assets are included or a clear build-time error is produced across supported adapters.