jf scan Failed to index tar file as container image and exits with exit code 0

Describe the bug

jf scan failed to index tar file downloaded with skopeo as container image. I've tried that with oci and v2 format (see logs below). Also the command exits with Scan completed successfully. and exit code 0 wich is unexpected.

Background: I would like to scan container images without running docker cli and docker socket on my agent.

This may be related to jfrog/jfrog-cli-security#350 and jfrog/jfrog-cli-security#340.

Current behavior

I censored our private domain with <**censored**>. Here is the log output:

$ jf --version
jf version 2.51.1

$ skopeo copy --format oci docker://ubuntu:22.04 oci-archive:scan_target_oci.tar
Getting image source signatures
Copying blob aece8493d397 done  
Copying config e4c5895818 done  
Writing manifest to image destination
Storing signatures

$ JFROG_CLI_LOG_LEVEL=DEBUG jf scan scan_target_oci.tar
08:30:27 [Debug] JFrog CLI version: 2.51.1
08:30:27 [Debug] OS/Arch: linux/amd64
08:30:27 [Debug] Sending HTTP GET request to: https://<**censored**>/xray/api/v1/system/version
08:30:27 [Debug] Usage Report: Sending info...
08:30:27 [Debug] Sending HTTP GET request to: https://<**censored**>/artifactory/api/system/version
08:30:27 [Debug] Artifactory response: 200 
08:30:27 [Debug] JFrog Artifactory version is: 7.68.14
08:30:27 [Debug] Sending HTTP POST request to: https://<**censored**>/artifactory/api/system/usage
08:30:27 [🔵Info] JFrog Xray version is: 3.85.5
08:30:27 [Debug] Creating lock in: /home/vagrant/.jfrog/locks/xray-indexer
08:30:27 [Debug] Releasing lock: /home/vagrant/.jfrog/locks/xray-indexer/jfrog-cli.conf.lck.2563.1701156627985803506
08:30:27 [🔵Info] [Thread 2] Indexing file: scan_target_oci.tar
08:30:29 [🔵Info] 2023/11/28 08:30:28 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
2023-11-28T07:30:28.295Z [jfxia] [DEBUG] [] [filter:135                    ] [BackgroundContext               ] Initializing filtering service
2023-11-28T07:30:29.072Z [jfxia] [DEBUG] [] [archive_mgr:112               ] [BackgroundContext               ] Indexing standalone file scan_target_oci.tar using artifactory folder /tmp/jfrog.cli.temp.-1701156627-2066992780
2023-11-28T07:30:29.074Z [jfxia] [DEBUG] [] [archive_mgr:148               ] [UnnamedContext                  ] Local path: /tmp/jfrog.cli.temp.-1701156627-2066992780/2ba2eb66-dc17-4a9f-6428-3c42be273858/170115662907428581/scan_target_oci.tar
2023-11-28T07:30:29.074Z [jfxia] [DEBUG] [] [archive_mgr:156               ] [UnnamedContext                  ] Scanning file from Artifactory with mimetype 'application/x-gzip'
2023-11-28T07:30:29.232Z [jfxia] [DEBUG] [] [archive_mgr:199               ] [UnnamedContext                  ] Found archive file. Performing deep scan for file /tmp/jfrog.cli.temp.-1701156627-2066992780/2ba2eb66-dc17-4a9f-6428-3c42be273858/170115662907428581/scan_target_oci.tar
2023-11-28T07:30:29.234Z [jfxia] [WARN ] [] [docker_tar:74                 ] [UnnamedContext                  ] Failed to index tar file as container image, continue to generic tar indexer. Error: failed to analyze OCI tar archive
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/docker_tar.go:144 (DockerTarOpener.analyzeTarAsContainer) ---
Caused by: failed to parse and validate manifests list: index.json
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/oci_tar.go:53 (DockerTarOpener.handleIndexFile) ---
Caused by: manifest does not contain annotations
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/oci_tar.go:85 (DockerTarOpener.parseAndValidateManifestsList) ---
2023-11-28T07:30:29.234Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/c9cf959fd83770dfdefd8fb42cfef0761432af36a764c077aed54bbc5bb25368
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/e4c58958181a5925816faa528ce959e487632f4cfd192f8132f71b32df2744b4
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable oci-layout
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [archive_mgr:536               ] [UnnamedContext                  ] No classification found for scan_target_oci.tar, classified as generic
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [archive_mgr:543               ] [UnnamedContext                  ] scan_target_oci.tar was classified as Generic
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [utils:424                     ] [UnnamedContext                  ] total running time for indexing tree construction of  scan_target_oci.tar: 5.4963e-05 seconds
2023-11-28T07:30:29.237Z [jfxia] [DEBUG] [] [utils:424                     ] [UnnamedContext                  ] total running time for indexing scan_target_oci.tar: 0.00496765 seconds

08:30:29 [Debug] Sending HTTP POST request to: https://<**censored**>/xray/api/v1/scan/graph?scan_type=binary
08:30:29 [🔵Info] Waiting for scan to complete on JFrog Xray...
08:30:29 [Debug] Sending HTTP GET request to: https://<**censored**>/xray/api/v1/scan/graph/2d60f701-ea47-4632-65f0-89788c776f10?include_vulnerabilities=true

Vulnerable Components
┌───────────────────────────────────────────┐
│ ✨ No vulnerable components were found ✨ │
└───────────────────────────────────────────┘
08:30:30 [🔵Info] Scan completed successfully.

$ echo $?
0

$ skopeo copy --format v2s1 docker://ubuntu:22.04 oci-archive:scan_target_v2s1.tar
Getting image source signatures
Copying blob aece8493d397 done  
Writing manifest to image destination
Storing signatures

$ JFROG_CLI_LOG_LEVEL=DEBUG jf scan scan_target_v2s1.tar
08:31:36 [Debug] JFrog CLI version: 2.51.1
08:31:36 [Debug] OS/Arch: linux/amd64
08:31:36 [Debug] Sending HTTP GET request to: https://<**censored**>/xray/api/v1/system/version
08:31:36 [Debug] Usage Report: Sending info...
08:31:36 [Debug] Sending HTTP GET request to: https://<**censored**>/artifactory/api/system/version
08:31:37 [Debug] Artifactory response: 200 
08:31:37 [Debug] JFrog Artifactory version is: 7.68.14
08:31:37 [Debug] Sending HTTP POST request to: https://<**censored**>/artifactory/api/system/usage
08:31:37 [🔵Info] JFrog Xray version is: 3.85.5
08:31:37 [Debug] Creating lock in: /home/vagrant/.jfrog/locks/xray-indexer
08:31:37 [Debug] Releasing lock: /home/vagrant/.jfrog/locks/xray-indexer/jfrog-cli.conf.lck.2726.1701156697243170288
08:31:37 [🔵Info] [Thread 2] Indexing file: scan_target_v2s1.tar
08:31:38 [🔵Info] 2023/11/28 08:31:37 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
2023-11-28T07:31:37.453Z [jfxia] [DEBUG] [] [filter:135                    ] [BackgroundContext               ] Initializing filtering service
2023-11-28T07:31:38.173Z [jfxia] [DEBUG] [] [archive_mgr:112               ] [BackgroundContext               ] Indexing standalone file scan_target_v2s1.tar using artifactory folder /tmp/jfrog.cli.temp.-1701156697-3797983710
2023-11-28T07:31:38.173Z [jfxia] [DEBUG] [] [archive_mgr:148               ] [UnnamedContext                  ] Local path: /tmp/jfrog.cli.temp.-1701156697-3797983710/f176bf74-3064-475d-44fe-f4dfc0f13703/170115669817310139/scan_target_v2s1.tar
2023-11-28T07:31:38.173Z [jfxia] [DEBUG] [] [archive_mgr:156               ] [UnnamedContext                  ] Scanning file from Artifactory with mimetype 'application/x-gzip'
2023-11-28T07:31:38.320Z [jfxia] [DEBUG] [] [archive_mgr:199               ] [UnnamedContext                  ] Found archive file. Performing deep scan for file /tmp/jfrog.cli.temp.-1701156697-3797983710/f176bf74-3064-475d-44fe-f4dfc0f13703/170115669817310139/scan_target_v2s1.tar
2023-11-28T07:31:38.320Z [jfxia] [WARN ] [] [docker_tar:74                 ] [UnnamedContext                  ] Failed to index tar file as container image, continue to generic tar indexer. Error: failed to analyze OCI tar archive
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/docker_tar.go:144 (DockerTarOpener.analyzeTarAsContainer) ---
Caused by: failed to parse and validate manifests list: index.json
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/oci_tar.go:53 (DockerTarOpener.handleIndexFile) ---
Caused by: manifest does not contain annotations
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/oci_tar.go:85 (DockerTarOpener.parseAndValidateManifestsList) ---
2023-11-28T07:31:38.320Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/755196ffd48538b9af32c2f73566d6b180b484b6a02f733a2d34ee6ce0836fe9
2023-11-28T07:31:38.321Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
2023-11-28T07:31:38.321Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable blobs/sha256/aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e
2023-11-28T07:31:38.324Z [jfxia] [DEBUG] [] [fileutil:157                  ] [TODOContext                     ] checking if the file is supported executable oci-layout
2023-11-28T07:31:38.324Z [jfxia] [DEBUG] [] [archive_mgr:536               ] [UnnamedContext                  ] No classification found for scan_target_v2s1.tar, classified as generic
2023-11-28T07:31:38.324Z [jfxia] [DEBUG] [] [archive_mgr:543               ] [UnnamedContext                  ] scan_target_v2s1.tar was classified as Generic
2023-11-28T07:31:38.324Z [jfxia] [DEBUG] [] [utils:424                     ] [UnnamedContext                  ] total running time for indexing tree construction of  scan_target_v2s1.tar: 7.3909e-05 seconds
2023-11-28T07:31:38.324Z [jfxia] [DEBUG] [] [utils:424                     ] [UnnamedContext                  ] total running time for indexing scan_target_v2s1.tar: 0.003686863 seconds

08:31:38 [Debug] Sending HTTP POST request to: https://<**censored**>/xray/api/v1/scan/graph?scan_type=binary
08:31:38 [🔵Info] Waiting for scan to complete on JFrog Xray...
08:31:38 [Debug] Sending HTTP GET request to: https://<**censored**>/xray/api/v1/scan/graph/f81a8f38-d01b-41da-6354-a0c98bd769e8?include_vulnerabilities=true

Vulnerable Components
┌───────────────────────────────────────────┐
│ ✨ No vulnerable components were found ✨ │
└───────────────────────────────────────────┘
08:31:39 [🔵Info] Scan completed successfully.

$ echo $?
0

Reproduction steps

jf --version

skopeo copy --format oci docker://ubuntu:22.04 oci-archive:scan_target_oci.tar
JFROG_CLI_LOG_LEVEL=DEBUG jf scan scan_target_oci.tar
echo $?

skopeo copy --format v2s1 docker://ubuntu:22.04 oci-archive:scan_target_v2s1.tar
JFROG_CLI_LOG_LEVEL=DEBUG jf scan scan_target_v2s1.tar
echo $?

Expected behavior

First of all I expected the scan to work because the tar archieves are recognized as container images.
Then I expected the Failed to index tar file as container image to be an ERROR instead of an WARN.
Furthermore I expected the command to fail if such error occurs and exit with a non-zero exit code.

JFrog CLI version

jf version 2.51.1

Operating system type and version

Ubuntu 22.04.3 LTS

JFrog Artifactory version

{"version":"7.68.14","revision":"76814900"}

JFrog Xray version

{"xray_version":"3.85.5","xray_revision":"8ef6b2d"}

Start with the jf scan entry point and the tar-indexing path; the logs identify docker_tar.go:144 and oci_tar.go:53/85 as the failing stack locations. Reproduce with the provided skopeo OCI and v2s1 commands, then verify failed container indexing is reported as an error and produces a non-zero exit code rather than generic scan success.