holzschu / holzschu/a-shell

Escaping for a-Shell is broken?

Open
#564 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Perl
Stars
3.9k
Forks
213
Avg merge
1m
Merged PRs (30d)
1

Description

I've found a strange issue on a-Shell when using ffmpeg.
This command:

```
ffmpeg -y -loglevel repeat+info -i file:output.m4a -map 0 -dn -ignore_unknown -vn -acodec copy -write_id3v1 1 -metadata 'title=You'"'"'re Going to Miss It - Harry Gregson-Williams' -metadata date=20170728 -metadata 'description=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'synopsis=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'purl=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata 'comment=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata artist=Prologue -movflags +faststart file:output.temp.m4a

```
Is not parsed correctly by a-Shell, and returns the following ERROR:

```
[~/Documents]$ ffmpeg -y -loglevel repeat+info -i file:output.m4a -map 0 -dn -ignore_unknown -vn -acodec copy -write_id3v1 1 -metadata 'title=You'"'"'re Going to Miss It - Harry Gregson-Williams' -metadata date=20170728 -metadata 'description=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'synopsis=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'purl=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata 'comment=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata artist=Prologue -movflags +faststart file:output.temp.m4a
ffmpeg version n5.1.2-11-g30d432f205 Copyright (c) 2000-2022 the FFmpeg developers
built with Apple clang version 14.0.0 (clang-1400.0.29.202)
configuration: --cc=clang --arch=arm64 --disable-asm --enable-cross-compile --enable-pthreads --enable-videotoolbox --disable-audiotoolbox --enable-openssl --enable-libfreetype --enable-libzimg --enable-network --disable-debug --disable-gpl --disable-nonfree --enable-libmp3lame
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:output.m4a':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: M4A isomiso2
encoder : Lavf59.27.100
Duration: 00:09:15.21, start: 0.000000, bitrate: 126 kb/s
Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
[NULL @ 0x13901f0e0] Unable to find a suitable output format for '"'
": Invalid argument
[~/Documents]$
[~/Documents]$
```

It seems the escaping is not parsed correctly by the shell.
```'"'"'``` is the correct escaping for ```'``` on UNIX. But a-Shell parses it wrong, and ```"``` ends being left out.

Moreover: this bug in the parsing code for escaping characters can cause a-Shell to crash. Here is a command that ALWAYS **crashes a-Shell** (at least on my iPad):

```
ffmpeg -y -loglevel repeat+info -i file:output.m4a -map 0 -dn -ignore_unknown -vn -acodec copy -write_id3v1 1 -metadata 'title=You'"'"'re Going to Miss It - Harry Gregson-Williams' -movflags +faststart file:output.temp.m4a
```
And this command is just a shortened version of the one above.

Also, those kind of commands are usually emitted by youtube-dl and yt-dlp (this is where I got them) for postprocessing videos. And this bug seems to make both programs crash with ERROR. No such crash happens on my Debian machine.
In other words, at the moment many youtube URLs (all those with titles that need escaping) cannot be passed to those programs in a-Shell (unless you disable the postprocessing options).

You can reproduce the issue in yt-dlp with the following command:
```
[~/Documents]$ yt-dlp -x "https://www.youtube.com/watch?app=desktop&v=5jgQy5tMQQ8" --add-metadata -f m4a -o output.mp4
[youtube] Extracting URL: https://www.youtube.com/watch?app=desktop&v=5jgQy5tMQQ8
[youtube] 5jgQy5tMQQ8: Downloading webpage
[youtube] 5jgQy5tMQQ8: Downloading android player API JSON
[youtube] 5jgQy5tMQQ8: Downloading MPD manifest
[info] 5jgQy5tMQQ8: Downloading 1 format(s): 140
[download] Destination: output.mp4
[download] 100% of 8.42MiB in 00:00:00 at 25.51MiB/s
[FixupM4a] Correcting container of "output.mp4"
[ExtractAudio] Not converting audio output.mp4; the file is already in a common audio format
[Metadata] Adding metadata to "output.mp4"
ERROR: Postprocessing: Going: Invalid argument
[~/Documents]
[~/Documents]
[~/Documents]
```

And if you need the verbose output:
```
[~/Documents]$ yt-dlp -vU -x "https://www.youtube.com/watch?app=desktop&v=5jgQy5tMQQ8" --add-metadata -f m4a -o output.mp4
[debug] Command-line config: ['-vU', '-x', 'https://www.youtube.com/watch?app=desktop&v=5jgQy5tMQQ8', '--add-metadata', '-f', 'm4a', '-o', 'output.mp4']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.02.17 [a0a7c01] (pip)
[debug] Python 3.11.0 (CPython iPad8,11 64bit) - macOS-16.3.1-iPad8,11-arm-64bit (OpenSSL 1.1.1i 8 Dec 2020)
[debug] exe versions: ffmpeg n5.1.2-11-g30d432f205 (setts), ffprobe n5.1.2-11-g30d432f205, phantomjs
broken, rtmpdump broken
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, web
sockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1782 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2023.02.17)
[youtube] Extracting URL: https://www.youtube.com/watch?app=desktop&v=5jgQy5tMQQ8
[youtube] 5jgQy5tMQQ8: Downloading webpage
[youtube] 5jgQy5tMQQ8: Downloading android player API JSON
[youtube] 5jgQy5tMQQ8: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] 5jgQy5tMQQ8: Downloading 1 format(s): 140
[debug] Invoking http downloader on "https://rr1---sn-45nufxc-4jve.googlevideo.com/videoplayback?expire=1676999680&ei=oKf0Y72jDdCy1gK9zYHACQ&ip=2001%3Ab07%3A646e%3A5dca%3A2980%3Acfa0%3Af7d0%3A412a&id=o-AHSlh_FSkKKzr4GcZ-0xyysNiZhoqnqIm7ZRJrx9-OkS&itag=140&source=youtube&requiressl=yes&mh=Yw&mm=31%2C26&mn=sn-45nufxc-4jve%2Csn-hgn7rnls&ms=au%2Conr&mv=m&mvi=1&pl=51&initcwndbps=1357500&spc=H3gIhnhBotRV-MRblM1PQUOSDm8Wo1Y&vprv=1&svpuc=1&mime=audio%2Fmp4&gir=yes&clen=8825280&dur=555.212&lmt=1520923364209603&mt=1676977531&fvip=5&keepalive=yes&fexp=24007246&beids=24472384&c=ANDROID&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAKhHMcH3qDi2dBgGJiM-hMZMw0SRacZlmpUZRtURmiVvAiBkAHPgjytupySQes61Wv05cMo0ujC3WBhgJALhzWt-xQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAP1XiDCvicFI5r9KlQPYs4xuWRBuoOl31P9Cwwo1OH45AiEA0xZ-AxKigWNTEVl5S5Syy8khyf9G2YKGnypejC7AUn8%3D"
[download] output.mp4 has already been downloaded
[download] 100% of 8.40MiB
[debug] ffmpeg command line: ffprobe -show_streams file:output.mp4
[ExtractAudio] Destination: output.m4a
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:output.mp4 -vn -acodec copy -bsf:a aac_adtstoasc -movflags +faststart file:output.m4a
Deleting original file output.mp4 (pass -k to keep)
[Metadata] Adding metadata to "output.m4a"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:output.m4a -map 0 -dn -ignore_unknown -vn -acodec copy -write_id3v1 1 -metadata 'title=You'"'"'re Going to Miss It - Harry Gregson-Williams' -metadata date=20170728 -metadata 'description=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'synopsis=Music composed by Harry Gregson-Williams for the movie Spy Game.' -metadata 'purl=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata 'comment=https://www.youtube.com/watch?v=5jgQy5tMQQ8' -metadata artist=Prologue -movflags +faststart file:output.temp.m4a
[debug] ffmpeg version n5.1.2-11-g30d432f205 Copyright (c) 2000-2022 the FFmpeg developers
built with Apple clang version 14.0.0 (clang-1400.0.29.202)
configuration: --cc=clang --arch=arm64 --disable-asm --enable-cross-compile --enable-pthreads --enable-videotoolbox --disable-audiotoolbox --enable-openssl --enable-libfreetype --enable-libzimg --enable-network --disable-debug --disable-gpl --disable-nonfree --enable-libmp3lame
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:output.m4a':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: M4A isomiso2
encoder : Lavf59.27.100
Duration: 00:09:15.21, start: 0.000000, bitrate: 126 kb/s
Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
[NULL @ 0x104ee0b30] Unable to find a suitable output format for 'Going'
Going: Invalid argument

ERROR: Postprocessing: Going: Invalid argument
Traceback (most recent call last):
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3307, in process_info
replace_info_dict(self.post_process(dl_filename, info_dict, files_to_move))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3486, in post_process
info = self.run_all_pps('post_process', info, additional_pps=info.get('__postprocessors'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3468, in run_all_pps
info = self.run_pp(pp, info)
^^^^^^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3446, in run_pp
files_to_delete, infodict = pp.run(infodict)
^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/postprocessor/common.py", line 24, in run
ret = func(self, info, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/postprocessor/common.py", line 129, in wrapper
return func(self, info)
^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 705, in run
self.run_ffmpeg_multiple_files(
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 324, in run_ffmpeg_multiple_files
return self.real_run_ffmpeg(
^^^^^^^^^^^^^^^^^^^^^
File "/var/mobile/Containers/Data/Application/B3257773-9D80-4169-8C86-EA5754C70197/Library/lib/python3.11/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 362, in real_run_ffmpeg
raise FFmpegPostProcessorError(stderr.strip().splitlines()[-1])
yt_dlp.postprocessor.ffmpeg.FFmpegPostProcessorError: Going: Invalid argument
[~/Documents]$
[~/Documents]$
```

What do you think?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.