quickwit-oss / quickwit-oss/quickwit
Lambda does not exit on errors, keeps running until timeout
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Describe the bug
I haven't tested it super thoroughly, but I saw Lambda runs till max timeout even thought it couldn't process anything. One example was sending gz files which I realized not supported
Steps to reproduce (if applicable)
Steps to reproduce the behavior:
- deploy indexer lambda
- send compressed file, test.json.gz
Expected behavior
It should exit immediately and not run until timeout
Configuration:
export class RustFunction extends Function {
constructor(scope: Construct, id: string, props?: Partial<FunctionProps>) {
const lambdaAssetPath = path.join(
__dirname,
"../../quickwit/quickwit",
"quickwit-lambda/deploy",
id
);
console.log("lambdaAssetPath", lambdaAssetPath);
super(scope, id+'_f', {
...props,
code: Code.fromAsset(lambdaAssetPath),
handler: id, // use id to specify either "indexer" or "searcher"
runtime: Runtime.PROVIDED_AL2,
architecture: Architecture.ARM_64,
logRetention: RetentionDays.ONE_DAY,
tracing: Tracing.DISABLED,
});
}
}
Binaries compiled with this command (I didn't use cross it was too slow on Mac)
LIBZ_SYS_STATIC=1 TARGET_CC=aarch64-linux-musl-gcc RUSTFLAGS="-C linker=aarch64-linux-musl-gcc -C link-arg=-static -C opt-level=z -C lto" cargo build --release --target aarch64-unknown-linux-musl
- Output of
quickwit --version
checked out v0.8.1
- The index_config.yaml
version: 0.7
index_id: test-index
doc_mapping:
field_mappings:
- name: website
type: text
- name: name
type: text
- name: founded
type: i64
indexed: true
fast: true
- name: size
type: text
- name: locality
type: text
- name: region
type: text
fast: true
- name: country
type: text
fast: true
- name: industry
type: text
fast: true
- name: linkedin_url
type: text
search_settings:
default_search_fields: [name]
indexing_settings:
split_num_docs_target: 2000000
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the quickwit-lambda/deploy entry points used for the indexer and searcher handlers, then reproduce the reported behavior by sending test.json.gz to the deployed indexer Lambda. Done means an unsupported compressed file causes the Lambda to exit immediately instead of running until its maximum timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100