Pushing data into workspaces from external data stores

We frequently want to push data into a workspace at the beginning of a project, and sometimes to refresh that data later on, possibly on a regular basis. We're currently doing that with a private endpoint in the workspace that connects to an external ADF instance, allowing us to push CSV and (soon) SQL tables too.

We also need to be able to push other data types eventually, such as large amounts of image data, which will go either to shared storage or to a storage account made accessible from inside the workspace.

I'm aware that other people have other implementations. E.g., @jonnyry has his ADF inside the TRE, IIRC, and pushes from there to the client workspaces. That will probably look similar to ours, but not completely.

Can we have a discussion here to see what the requirements are, what work is planned or done, how we might be able to standardise a solution for connectivity to external data stores? Is anyone else doing anything in this space?

Start by comparing the existing private workspace endpoint and the alternative ADF-inside-the-TRE approach described in the issue. Gather requirements for CSV, SQL tables, image data, shared storage, and regular refreshes, then document an agreed standard for connectivity and confirm the planned work or implementation path.