facebookresearch / facebookresearch/fairo

[Turk][Bug Report] Some commands are logged twice

Open
#325 0 comments 0 reactions 1 assignee Claimed by @Rebecca-Qian View on GitHub
bug report P1
Dominant language
Jupyter Notebook
Stars
929
Forks
123
PR merge metrics
No merged PRs in 30d

Description

## Type of Issue

Select the type of issue:
- [x] Bug report (to report a bug)
- [ ] Feature request (to request an additional feature)
- [ ] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [ ] Documentation Ask

## Description

```
spawn an object of your choosing|{'dialogue_type': 'HUMAN_GIVE_COMMAND', 'action_sequence': [{'action_type': 'SPAWN', 'reference_object': {'filters': {'triples': [{'pred_text': 'has_name', 'obj_text': [0, [2, 2]]}]}, 'text_span': [0, [2, 5]]}}]}|semantic_parser|craftassist|1618310034.7754169
Spawn an object of your choosing.|{'dialogue_type': 'HUMAN_GIVE_COMMAND', 'action_sequence': [{'action_type': 'SPAWN', 'reference_object': {'filters': {'triples': [{'pred_text': 'has_name', 'obj_text': [0, [0, 0]]}]}, 'text_span': [0, [0, 0]]}}]}|semantic_parser|craftassist|1618310034.785497
```

## Current Behavior

Some commands are being logged twice in `nsp_outputs.csv` from Turk interaction sessions. Strangely, the command without processing also has model outputs where span values are wrong. Need to debug further whether model is queried twice (why), if so why commands with punctuation cause span values to break, and why this only happens on some commands.

## Expected Behavior

one record for each chat sent in the logs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.