Python UDF operator doesn't resume after fixing the runtime error
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
The python UDF operator pauses when there is a runtime error. After I fix the runtime error and click "Retry the failed tuple", it doesn't resume the execution.
Steps
1. Set `enable-transactional-reconfiguration = true` in `core/amber/src/main/resources/application.conf`
2. Run this workflow
```
{
"operators": [
{
"operatorID": "TextInput-operator-063c8ef7-9ea9-4c89-afee-bea2b1f12c0b",
"operatorType": "TextInput",
"operatorVersion": "25faefd4bf57f6fc1b0b99384eded40f593332c0",
"operatorProperties": {
"attributeType": "string",
"attributeName": "line",
"textInput": "1"
},
"inputPorts": [],
"outputPorts": [
{
"portID": "output-0",
"displayName": "",
"allowMultiInputs": false,
"isDynamicPort": false
}
],
"showAdvanced": false,
"isDisabled": false,
"customDisplayName": "Text Input",
"dynamicInputPorts": false,
"dynamicOutputPorts": false
},
{
"operatorID": "PythonUDFV2-operator-319229ed-5d86-49c3-b426-244ea3d08fce",
"operatorType": "PythonUDFV2",
"operatorVersion": "25faefd4bf57f6fc1b0b99384eded40f593332c0",
"operatorProperties": {
"code": "from pytexera import *\n\nclass ProcessTupleOperator(UDFOperatorV2):\n \n @overrides\n def process_tuple(self, tuple_: Tuple, port: int) -> Iterator[Optional[TupleLike]]:\n yield tuple_['a']",
"workers": 1,
"retainInputColumns": true
},
"inputPorts": [
{
"portID": "input-0",
"displayName": "",
"allowMultiInputs": true,
"isDynamicPort": false,
"dependencies": []
}
],
"outputPorts": [
{
"portID": "output-0",
"displayName": "",
"allowMultiInputs": false,
"isDynamicPort": false
}
],
"showAdvanced": false,
"isDisabled": false,
"customDisplayName": "Python UDF",
"dynamicInputPorts": true,
"dynamicOutputPorts": true
}
],
"operatorPositions": {
"TextInput-operator-063c8ef7-9ea9-4c89-afee-bea2b1f12c0b": {
"x": 449,
"y": 230
},
"PythonUDFV2-operator-319229ed-5d86-49c3-b426-244ea3d08fce": {
"x": 588,
"y": 225
}
},
"links": [
{
"linkID": "link-2def2bb4-2329-4978-b843-3fd82bab8d4c",
"source": {
"operatorID": "TextInput-operator-063c8ef7-9ea9-4c89-afee-bea2b1f12c0b",
"portID": "output-0"
},
"target": {
"operatorID": "PythonUDFV2-operator-319229ed-5d86-49c3-b426-244ea3d08fce",
"portID": "input-0"
}
}
],
"groups": [],
"commentBoxes": [],
"settings": {
"dataTransferBatchSize": 400
}
}
```
3. Execute the workflow (it will fail)
4. Press "Unlock for Logic Change"
5. Remove "['a']" from the python udf code
6. Press "Confirm Change"
7. Press "Retry the faulty tuple"
8. Still gives me an error
https://github.com/user-attachments/assets/bb4412fa-87a3-4f57-bc73-9d12ead14e79
Contributor guide
Assessment
This issue has not been assessed yet.