googleapis / googleapis/google-api-nodejs-client

Google Drive: Incorrect Forbidden 403

Offen
#2,968 2 Kommentare 2 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

priority: p2 type: bug
Vorherrschende Sprache
TypeScript
Sterne
12.3k
Forks
2k
Ø Merge
1 T. 9 Std.
Gemergte PRs (30 T.)
24

Beschreibung

We are migrating our video catalog to a new processing and hosting solution and as a part of that, we need to transfer files that are currently housed in GoogleDrive. After around 60 videos transferred (our videos are typically between 100MB and 300MB), we start to see errors when trying to transfer the files which give a 403 error code with `statusText` "Forbidden".

We are using the `drive.files.get` api to stream the file into S3

```
drive.files
.get(
{
fileId: driveId,
alt: 'media',
auth: driveApiKey,
},
{ responseType: 'stream' },
)
.then(response =>
s3
.upload({
Bucket: sourceBucket,
Key: `${jobId}/${fileName}`,
Body: response.data,
})
.promise(),
)
```

I was retrying the file transfers to debug the issue, but the issue persisted. Once the hour turned 6:00 PM today, I happened to retry a few more and they succeeded.

This hints at some hourly egress limit. I was unable to find any documentation for this limit and this was not covered by the cases described in

https://developers.google.com/drive/api/guides/handle-errors

This is not strictly a request rate limiting issue so perhaps a 429 would not be appropriate, but a more explicit `statusText` would at least allow for more reasonable debugging.

#### Environment details

- OS: Linux (AWS Lambda
- Node.js version: 14
- npm version: 6.14.16
- `googleapis` version: 39.2.0

#### Steps to reproduce

1. Programmatically transfer a number video files (seemingly around 10GB) to another location (S3 in my case) within one hour.
2. Observe 403 error with message "Forbidden" like

```
GaxiosError: [object Object]
at Gaxios. (/opt/nodejs/node_modules/gaxios/build/src/gaxios.js:73:27)
at Generator.next ()
at fulfilled (/opt/nodejs/node_modules/gaxios/build/src/gaxios.js:16:58)
at processTicksAndRejections (internal/process/task_queues.js:95:5) {
response: {
config: {
url: 'https://www.googleapis.com/drive/v3/files/?alt=media&key=',
method: 'GET',
responseType: 'stream',
paramsSerializer: [Function (anonymous)],
headers: [Object],
params: [Object: null prototype],
validateStatus: [Function (anonymous)]
},
data: PassThrough {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
[Symbol(kCapture)]: false,
[Symbol(kTransformState)]: [Object]
},
headers: {
'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"',
connection: 'close',
'content-length': '1103',
'content-type': 'text/html; charset=UTF-8',
date: 'Wed, 04 May 2022 00:56:53 GMT'
},
status: 403,
statusText: 'Forbidden'
},
config: {
url: 'https://www.googleapis.com/drive/v3/files/?alt=media&key=',
method: 'GET',
responseType: 'stream',
paramsSerializer: [Function (anonymous)],
headers: {
'Accept-Encoding': 'gzip',
'User-Agent': 'google-api-nodejs-client/0.7.2 (gzip)'
},
params: [Object: null prototype] {
alt: 'media',
key: ''
},
validateStatus: [Function (anonymous)]
},
code: '403'
}
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Start with the reported drive.files.get call and the Google Drive API error-handling documentation linked in the issue. Reproduce the transfer workload described, including the recurring 403 response after roughly 10GB in an hour. Done means identifying whether the response is caused by a documented Drive limit and determining whether the client library can expose a more useful error detail.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
nodejs, typescript
Bereich
api
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.