[Bug]: Move files - can't process files using previous results as args
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### Apache Hop version?
2.9.0
### Java version?
openjdk version "21.0.2" 2024-01-16
### Operating system
Linux
### What happened?
Steps to reproduce:
- create a pipeline that generates a recordset with the fields requested by the Move files action: you can simply add a Data grid (with the fields src_path, dst_path and wildcard, and respective values) and link it with a Copy rows to result
- create a workflow that uses the above pipeline and link it to a Move files action
- in the Move files action, check "Copy previous results to args" and "Destination is a file"
When dealing with local files, the action fails with the following error:
```
2024/06/12 10:01:33 - Move files - found [1] argument(s) from previous result
2024/06/12 10:01:33 - Move files - Processing row source File/folder source : [/work/hop/mount/iraiser/in/expIraiser_2024_06_07.csv] ... destination file/folder : [/work/hop/mount/iraiser/in/_deleteme_expIraiser_2024_06_07.csv]... wildcard : [null]
2024/06/12 10:01:33 - Move files - ERROR: Can not move file/folder [/work/hop/mount/iraiser/in/expIraiser_2024_06_07.csv] to [file:///work/hop/mount/iraiser/in/_deleteme_expIraiser_2024_06_07.csv]. Exception : [
2024/06/12 10:01:33 - Move files - Could not extract the file name from the URI: file:///work/hop/mount/iraiser/in/_deleteme_expIraiser_2024_06_07.csv
2024/06/12 10:01:33 - Move files - ]
2024/06/12 10:01:33 - Move files - =======================================
2024/06/12 10:01:33 - Move files - Total files in error : 1
2024/06/12 10:01:33 - Move files - Total files successfully processed : 0
2024/06/12 10:01:33 - Move files - =======================================
2024/06/12 10:01:33 - _stub-move-files - Finished action [Move files] (result=[false])
```
When dealing with remote files (like those in an sFTP server), Hop throws an Exception: it cannot print an error because a variable `destinationfilefolder` is null (see #3985 for the log trace).
In both cases, the Detailed log (example above) shows that the file names and the (empty) wildcard are correctly passed from the pipeline to the Move files action
### Issue Priority
Priority: 2
### Issue Component
Component: Actions, Component: Transforms
Contributor guide
Research direction
Start by reproducing the workflow with a Data grid, Copy rows to result, and the Move files action, testing both local and remote files. Trace how the Move files action handles previous-result arguments when "Destination is a file" is selected, including the URI and null-destination errors described here and in #3985. Done means both local and remote moves complete without errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100