internetarchive / internetarchive/heritrix3

${launchId} is not being replaced (sometimes)

Open
#495 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.3k
Forks
793
Avg merge
1d 8h
Merged PRs (30d)
8

Description

Hi,

I've observed in the code that the value "${launchId}" is expected to be replaced with a value I'm not sure what is. Anyway, I'm trying to understand the configuration file and I found that the disposition chain uses for the `directory` directive the value "${launchId}". If I'm not wrong, this value should create a directory with this value **replaced**. What it happens instead, not always but it happens, is that in the job directory there is a directory with the literal name "${launchId}". Is this expected? I've observed that there are other directives which uses this value, but I haven't checked out if this affects to these directives as well.

```xml



```

I'm using the [last version](https://github.com/internetarchive/heritrix3/tree/c5d9bbd0c09556d445d64ca252fe2220e59debc9) of Heritrix (last commit from master).

I think that the times this happened to me has been when the issue described in [this comment](https://github.com/internetarchive/heritrix3/issues/340#issuecomment-1233205765) happened:

```bash
ls -la /home/cgarcia/Documentos/heritrix3/build_1661889017/heritrix-3.4.0-SNAPSHOT/jobs/clashroyale/\$\{launchId\}/

# total 12
# drwxrwxr-x 3 cgarcia cgarcia 4096 ago 31 11:31 .
# drwxrwxr-x 8 cgarcia cgarcia 4096 ago 31 19:16 ..
# drwxrwxr-x 2 cgarcia cgarcia 4096 ago 31 11:31 reports

ls -la /home/cgarcia/Documentos/heritrix3/build_1661889017/heritrix-3.4.0-SNAPSHOT/jobs/clashroyale/\$\{launchId\}/reports/

# drwxrwxr-x 2 cgarcia cgarcia 4096 ago 31 11:31 .
# drwxrwxr-x 3 cgarcia cgarcia 4096 ago 31 11:31 ..
# -rw-rw-r-- 1 cgarcia cgarcia 280 ago 31 11:31 crawl-report.txt
# -rw-rw-r-- 1 cgarcia cgarcia 0 ago 31 11:31 seeds-report.txt
# -rw-rw-r-- 1 cgarcia cgarcia 13 ago 31 11:31 threads-report.txt
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how the disposition chain handles the `directory` value `${launchId}` and how launch identifiers are substituted. Reproduce the intermittent case described in the issue, including the literal `${launchId}` directory, and check whether other directives show the same behavior. Done means the relevant paths consistently use the resolved launch identifier rather than the literal placeholder.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.