kodadot / kodadot/workers

[Capture] stress test fails

Open
#224 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
11
Forks
14
PR merge metrics
No merged PRs in 30d

Description

Calling capture service 20 times in parallel results in internal server error.
Currently I do not have any specific quick fix.
However will try to :)

```ts
import { getItemsFromCollection } from '@/query.ts'
import { consola } from 'consola'
import { $fetch, FetchError } from 'ofetch'
const CAPTURE_BASE_URL = ''
const IPFS_GW =

const api = $fetch.create({
baseURL: CAPTURE_BASE_URL,
})

export const makeScreenshot = async (url: string) => {
const value = await api('/screenshot', {
method: 'POST',
body: {
url,
},

}).catch((error: FetchError) => {
const search = new URL(url).searchParams.get('hash');
throw new Error(`[Capture::makeScreenshot] ${search} failed ${error.message}`)
})

return value
}

const items = await getItemsFromCollection('ahp', '40', {
limit: 100,
offset: 50,
});

consola.log(items.length);

const toFile = async (item: any): Promise => {
const sn = item.sn;
if (!item.media) {
consola.warn(`Skipping ${sn}`);
return null
}
const url = item.media.replace('ipfs://', IPFS_GW);
console.log(url)
const res = await makeScreenshot(url);
const file = new File([res], `${sn}.jpeg`, { type: 'image/jpeg' })
const x = await file.arrayBuffer();
await Deno.writeFile(file.name, new Uint8Array(x));
return file;
}

const files: Promise[] = []

for (const item of items) {
consola.start(`Processing ${item.sn}`);
const file = toFile(item);
consola.success(`Done ${item.sn}`);
if (file) {
files.push(file);
}

}

consola.start(`MAGIC ${files.length}`);
const x = await Promise.all(files);
consola.success(`MAGIC ${x.length}`);
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the TypeScript reproduction using the /screenshot capture endpoint and the makeScreenshot function, then test 20 parallel requests as shown. Investigate why concurrent calls produce internal server errors. Done means the same parallel stress test completes without internal server errors; no source file or test is identified in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.