parse-community / parse-community/parse-server

Allow Parse.File creation from url without fetching with s3 protocols

Open
#8,801 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type:feature
Dominant language
JavaScript
Stars
21.4k
Forks
4.8k
Avg merge
7h 45m
Merged PRs (30d)
11

Description

New Feature / Enhancement Checklist
Current Limitation

Currently to create a Parse.File on Parse Server you need to upload the file directly one way or another which is a very processor , memory and bandwidth expensive operation, especially if you use an S3 adapter.
The file has to go from the client, to the parse server, and then to your storage provider

Feature / Enhancement Description

Providing upload urls is a common feature of many file storage providers, it allows us to not have to put the file through the parse server and directly from the client to the storage.
We can set up a Cloud Function that is used to generate an upload URL securely for the client, and then we should be able to create a Parse File in the client that has no data except the metadata, name and url of the file in the S3 storage.

Example Use Case

Here is my Cloud Function that gets an upload link from B2


// In your cloud code (cloud/main.js)
Parse.Cloud.define('requestB2UploadUrls', async (request) => {
  const { user, params } = request;
  if (!user) throw new Error('You must be logged in to upload files');
  const { numberOfFiles } = params;
  try {
    // Authorize with B2 to get the account authorization token
    const authResponse = await axios.get('https://api.backblazeb2.com/b2api/v2/b2_authorize_account', {
      auth: {
        username: accountId,
        password: applicationKey
      }
    });

    const authToken = authResponse.data.authorizationToken;
    const apiUrl = authResponse.data.apiUrl;

    // Get upload URLs
    const uploadUrls = [];
    for (let i = 0; i < numberOfFiles; i++) {
      const uploadUrlResponse = await axios({
        method: 'GET',
        url: `${apiUrl}/b2api/v2/b2_get_upload_url`,
        params: {
          bucketId
        },
        headers: {
          Authorization: authToken
        }
      });

      uploadUrls.push(uploadUrlResponse.data);
    }

    return uploadUrls;
  } catch (error) {
    throw new Parse.Error(500, error);
  }
});

And here is how my client code should be

// Client-side function to get upload URLs from Parse Server
async function getUploadUrls(numberOfFiles) {
  try {
    const uploadUrls = await Parse.Cloud.run('requestB2UploadUrls', { numberOfFiles });
    return uploadUrls;
  } catch (error) {
    console.error('Failed to get upload URLs:', error);
  }
}

/** Client-side function to upload a file to B2 and update Parse.File
 * @param {object} uploadUrlInfo
 * @param {Blob} fileBlob
 * @param {object} metadata
 *  */
async function uploadFileToB2(uploadUrlInfo, fileBlob, metadata) {
  try {
    // Upload the file blob to B2 using the upload URL
    const response = await axios.post(uploadUrlInfo.uploadUrl, fileBlob, {
      headers: {
        Authorization: uploadUrlInfo.authorizationToken,
        'X-Bz-File-Name': encodeURIComponent(fileBlob.name),
        'Content-Type': fileBlob.type,
        'X-Bz-Content-Sha1': 'do_not_verify', // You should calculate the SHA1 in a production app
        ...metadata
      }
    });

    // Update the Parse.File with the URL to the uploaded file on B2
    const b2FileUrl = `${b2UrlRoot}/${response.data.fileName}`;
    const parseFile = new Parse.File(response.data.fileName, { url:b2FileUrl  }, fileBlob.type);
    parseFile.setMetadata(metadata); // Set the metadata on the Parse.File object
    await parseFile.save();

    return parseFile;
  } catch (error) {
    console.error('Failed to upload file to B2:', error);
  }
}
Alternatives / Workarounds

Not using Parse.File altogether but then we need to handle all of the deletion, metadata etc... on our own

We could also try to create file items using mongoose but this means we don't get afterFileSave or beforeFileSave functionality

3rd Party References

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the Parse.File creation and save flow, then inspect the S3 adapter boundary; the issue names no repository files or tests. Reproduce the client flow shown in the issue and determine how a file with only its name, metadata, and URL should be handled without fetching. Done means the URL-backed Parse.File works while retaining the expected file metadata and lifecycle behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.