google / google/turbinia

Get Dependency Graph and Task Config for each Task

Open
#1,302 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
794
Forks
172
PR merge metrics
No merged PRs in 30d

Description

I have two feature requests for the API

### Allow to fetch a graph for how the Jobs and Evidences are connected.
At the moment i have no clue what kind of Jobs actually will get started if I process an evidence with a set of Jobs. It would be really cool to be able to know beforehand what Jobs will get triggered based on the Evidence-Type and the initial Jobs selected.

For now i do this with some sort of twisted reflection and create a graph to get an impression of what is going to happen:

```python
from turbinia.jobs.interface import TurbiniaJob
from turbinia.workers import TurbiniaTask
result = {}
for subclass in TurbiniaJob.__subclasses__():
evidence_in = []
evidence_out = []
tasks = {}
for name, class_ in inspect.getmembers(inspect.getmodule(subclass), inspect.isclass):
class_hierarchy = inspect.getmro(class_)
if TurbiniaJob in class_hierarchy:
evidence_in += [a.__name__ for a in class_.evidence_input]
evidence_out += [a.__name__ for a in class_.evidence_output]
elif TurbiniaTask in class_hierarchy:
tasks[class_.__name__] = class_.TASK_CONFIG
result[subclass.__name__] = {"evidence_in": evidence_in, "evidence_out": evidence_out, "tasks": tasks}
n = pp.Network(directed=True)
for jobname, data in result.items():
n.add_node(jobname, type="job")
for ev_out in data["evidence_out"]:
n.add_node(ev_out, type="evidence")
n.add_edge(jobname, ev_out)
for ev_in in data["evidence_in"]:
n.add_node(ev_in, type="evidence")
n.add_edge(ev_in, jobname)
for taskname, config in data["tasks"].items():
n.add_node(taskname, type="task", config=config)
n.add_edge(jobname, taskname)
```

A visiual representation looks something like this:
![image](https://github.com/google/turbinia/assets/23474309/057bc072-06c1-4fe5-8c9b-4aa0ebfd8809)

### Allow to fetch the Task Config for each Task
As it is possible for each evidence type to fetch the needed and possible parameters it would be amazing to be able to fetch the possible task parameters for each Task.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing TurbiniaJob in turbinia.jobs.interface and TurbiniaTask in turbinia.workers, including evidence_input, evidence_output, and TASK_CONFIG. Determine the API shape needed to expose the job/evidence graph and task configurations, then verify that both requested capabilities are available through the API.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.