google / google/grr

Flows get Stuck in GRR 3.2.3.2

Open
#627 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5.1k
Forks
796
PR merge metrics
No merged PRs in 30d

Description

Hi all,

we recognized some strange behaviour of the GRR Server in Version 3.2.3.2.
After executing flows on clients, some of the flows will be executed and suddenly some of them get stuck. After deploying the debug agents on this clients we can see that the client does not receive messages from the server (find attached log). We also figure out some problems with GRR Artifact Collection when we try to use the %%users.appdatalocal%% Knowledgebase, the flow gots killed:

`Traceback (most recent call last): File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/server/grr_response_server/flow_runner.py", line 587, in RunStateMethod direct_response=direct_response, request=request, responses=responses) File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/server/grr_response_server/flow.py", line 297, in Decorated res = f(*args[:f.func_code.co_argcount]) File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/server/grr_response_server/flows/general/collectors.py", line 128, in StartCollection self.Collect(artifact_obj) File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/server/grr_response_server/flows/general/collectors.py", line 175, in Collect self.GetFiles(source, self.GetPathType(), self.args.max_file_size) File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/server/grr_response_server/flows/general/collectors.py", line 226, in GetFiles ignore_errors=self.args.ignore_interpolation_errors)) File "/usr/share/grr-server/local/lib/python2.7/site-packages/grr/lib/artifact_utils.py", line 116, in InterpolateKbAttributes e)) KnowledgeBaseInterpolationError: Failed to interpolate %%users.localappdata%%\Microsoft\Windows\UsrClass.dat with the knowledgebase. users.localappdata `

The connection between the clients and the server works well.

We're currently using the following setup:
1x GRR-Server on Hardware:
1x HA-Proxy that distribute the Client connections between 3 GRR HTTP-Frontend server
6x Workers

GRR-Version 3.2.3.2
Datastore: Sqlite.

At the moment I have now clue what the problem is. Could you please provide some advice on debugging?

[agent.log](https://github.com/google/grr/files/2393737/issue.log)

Thx for your help.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.