aws-samples / aws-samples/aws-batch-with-ffmpeg

Large-scale parallel processing

Open
#15 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Dockerfile
Stars
65
Forks
13
PR merge metrics
No merged PRs in 30d

Description

With Step Functions, you can orchestrate large-scale parallel processing. These parallel workloads let you concurrently process large-scale data sources stored in Amazon S3.
A Map state set to Distributed is known as a Distributed Map state. In Distributed mode, the Map state allows high-concurrency processing. In Distributed mode, the Map state processes the items in the dataset in iterations called child workflow executions.

https://docs.aws.amazon.com/step-functions/latest/dg/use-dist-map-orchestrate-large-scale-parallel-workloads.html

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked AWS Step Functions documentation on Distributed Map states and then inspect the repository's existing AWS Batch and FFmpeg workflow. The issue does not identify files, entry points, tests, or a concrete completion condition, so the intended implementation remains undefined.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.