mltframework / mltframework/mlt
Color shift after first frame of output when using two composite transitions
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 1.9k
- Forks
- 387
- Avg merge
- 17h 28m
- Merged PRs (30d)
- 8
Description
Hi there! I'm diving into MLT to composite two videos next to each other at specific sizes and orientations. They may animate between different orientations, but for this issue, I kept it to just one to demonstrate the problem. The problem is that I often see a noticeable color shift right after the video starts. If there's no transition, I don't see a color shift in the output. (The color is slightly different from the original, but maybe close enough to get by. But what I mean is, it doesn't noticeably change from the first frame to the second.)
I can get rid of the shift if I switch the profile colorspace from 709 to 601, though I'm not sure if that means all but the first frame is output in 601 or if the smaller colorspace is just masking the color shift. The output does seem slightly darker than the original, at least in this example.
I was initially having this issue using the mlt library via swig, but I was also able to reproduce it with melt. The input XML in my example was created using the XML consumer from a ruby script, hence quite a bit of unnecessary attributes. But I included it all for completeness.
Ideally my output would be in the 709 colorspace and be as true to the original colors as possible while also having no discernible color shift in the output. Am I doing that correctly or is there a better way? What am I missing?
The input file: https://embed-ssl.wistia.com/deliveries/d2e5245e04b6bcdce7981a5bb550b602/pets000.mov?disposition=attachment
The output file (you can see the color shift here): https://embed-ssl.wistia.com/deliveries/0fa094f2a99c63b1253f01d7675082ef/output.mov?disposition=attachment
The command I'm using:
melt \
xml:input.xml \
-consumer \
avformat:output.mov \
vcodec=libx264 \
f=mov \
crf=23 \
x264opts=keyint=89:min-keyint=89:scenecut=-1 \
movflags=faststart \
strict=strict \
threads=0 \
width=1280 \
height=720
input.xml:
<?xml version="1.0"?>
<mlt LC_NUMERIC="en_US.UTF-8" version="6.23.0">
<profile description="DV/DVD PAL" width="1280" height="720" progressive="1" sample_aspect_num="1" sample_aspect_den="1" display_aspect_num="1" display_aspect_den="1" frame_rate_num="30" frame_rate_den="1" colorspace="709"/>
<producer id="producer0" in="0" out="114">
<property name="length">15000</property>
<property name="eof">continue</property>
<property name="resource">0x000000ff</property>
<property name="aspect_ratio">1</property>
<property name="mlt_service">colour</property>
<property name="colour">0x000000FF</property>
</producer>
<producer id="producer1" in="0" out="114">
<property name="length">358</property>
<property name="eof">continue</property>
<property name="resource">pets000.mov</property>
<property name="meta.media.nb_streams">2</property>
<property name="meta.media.0.stream.type">video</property>
<property name="meta.media.0.stream.frame_rate">23.976</property>
<property name="meta.media.0.stream.sample_aspect_ratio">0</property>
<property name="meta.media.0.codec.width">1280</property>
<property name="meta.media.0.codec.height">720</property>
<property name="meta.media.0.codec.rotate">0</property>
<property name="meta.media.0.codec.frame_rate">24000</property>
<property name="meta.media.0.codec.pix_fmt">yuv420p</property>
<property name="meta.media.0.codec.sample_aspect_ratio">0</property>
<property name="meta.media.0.codec.colorspace">709</property>
<property name="meta.media.0.codec.color_trc">1</property>
<property name="meta.media.0.codec.name">h264</property>
<property name="meta.media.0.codec.long_name">H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10</property>
<property name="meta.media.0.codec.bit_rate">5948147</property>
<property name="meta.attr.0.stream.language.markup">eng</property>
<property name="meta.attr.0.stream.handler_name.markup">Apple Video Media Handler</property>
<property name="meta.media.1.stream.type">audio</property>
<property name="meta.media.1.codec.sample_fmt">s16</property>
<property name="meta.media.1.codec.sample_rate">44100</property>
<property name="meta.media.1.codec.channels">2</property>
<property name="meta.media.1.codec.name">pcm_s16le</property>
<property name="meta.media.1.codec.long_name">PCM signed 16-bit little-endian</property>
<property name="meta.media.1.codec.bit_rate">1411200</property>
<property name="meta.attr.1.stream.language.markup">eng</property>
<property name="meta.attr.1.stream.handler_name.markup">Apple Sound Media Handler</property>
<property name="meta.attr.major_brand.markup">qt </property>
<property name="meta.attr.minor_version.markup">512</property>
<property name="meta.attr.compatible_brands.markup">qt </property>
<property name="meta.attr.encoder.markup">Lavf58.20.100</property>
<property name="seekable">1</property>
<property name="meta.media.sample_aspect_num">1</property>
<property name="meta.media.sample_aspect_den">1</property>
<property name="aspect_ratio">1</property>
<property name="audio_index">1</property>
<property name="video_index">0</property>
<property name="mute_on_pause">1</property>
<property name="mlt_service">avformat</property>
</producer>
<producer id="producer2" in="0" out="114">
<property name="length">358</property>
<property name="eof">continue</property>
<property name="resource">pets000.mov</property>
<property name="meta.media.nb_streams">2</property>
<property name="meta.media.0.stream.type">video</property>
<property name="meta.media.0.stream.frame_rate">23.976</property>
<property name="meta.media.0.stream.sample_aspect_ratio">0</property>
<property name="meta.media.0.codec.width">1280</property>
<property name="meta.media.0.codec.height">720</property>
<property name="meta.media.0.codec.rotate">0</property>
<property name="meta.media.0.codec.frame_rate">24000</property>
<property name="meta.media.0.codec.pix_fmt">yuv420p</property>
<property name="meta.media.0.codec.sample_aspect_ratio">0</property>
<property name="meta.media.0.codec.colorspace">709</property>
<property name="meta.media.0.codec.color_trc">1</property>
<property name="meta.media.0.codec.name">h264</property>
<property name="meta.media.0.codec.long_name">H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10</property>
<property name="meta.media.0.codec.bit_rate">5948147</property>
<property name="meta.attr.0.stream.language.markup">eng</property>
<property name="meta.attr.0.stream.handler_name.markup">Apple Video Media Handler</property>
<property name="meta.media.1.stream.type">audio</property>
<property name="meta.media.1.codec.sample_fmt">s16</property>
<property name="meta.media.1.codec.sample_rate">44100</property>
<property name="meta.media.1.codec.channels">2</property>
<property name="meta.media.1.codec.name">pcm_s16le</property>
<property name="meta.media.1.codec.long_name">PCM signed 16-bit little-endian</property>
<property name="meta.media.1.codec.bit_rate">1411200</property>
<property name="meta.attr.1.stream.language.markup">eng</property>
<property name="meta.attr.1.stream.handler_name.markup">Apple Sound Media Handler</property>
<property name="meta.attr.major_brand.markup">qt </property>
<property name="meta.attr.minor_version.markup">512</property>
<property name="meta.attr.compatible_brands.markup">qt </property>
<property name="meta.attr.encoder.markup">Lavf58.20.100</property>
<property name="seekable">1</property>
<property name="meta.media.sample_aspect_num">1</property>
<property name="meta.media.sample_aspect_den">1</property>
<property name="aspect_ratio">1</property>
<property name="audio_index">1</property>
<property name="video_index">0</property>
<property name="mute_on_pause">1</property>
<property name="mlt_service">avformat</property>
</producer>
<tractor id="tractor0" global_feed="1" in="" out="">
<track producer="producer0"/>
<track producer="producer1"/>
<track producer="producer2"/>
<transition id="transition0" out="114">
<property name="a_track">0</property>
<property name="b_track">1</property>
<property name="start">-213.33333333333337/120.0:853.3333333333335x480.00000000000006</property>
<property name="factory">loader</property>
<property name="aligned">1</property>
<property name="progressive">1</property>
<property name="mlt_service">composite</property>
<property name="end">-213.33333333333337/120.0:853.3333333333335x480.00000000000006</property>
<property name="distort">1</property>
</transition>
<transition id="transition1" out="114">
<property name="a_track">0</property>
<property name="b_track">2</property>
<property name="start">426.66666666666663/120.0:853.3333333333335x480.00000000000006</property>
<property name="factory">loader</property>
<property name="aligned">1</property>
<property name="progressive">1</property>
<property name="mlt_service">composite</property>
<property name="end">426.66666666666663/120.0:853.3333333333335x480.00000000000006</property>
<property name="distort">1</property>
</transition>
</tractor>
</mlt>
I compiled mlt from commit 0e756b692b4b1248d27254d124cd41f580878372--I upgraded from 16.6.0 where I had the same issue--with this configuration:
./configure --disable-decklink --disable-dv --disable-feeds --disable-frei0r --disable-gtk2 --disable-jackrack --disable-kdevlive --disable-kino --disable-linsys --disable-lumas --disable-motion_est --disable-ndi --disable-normalize --disable-oldfilm --disable-opencv --disable-opengl --disable-qt --disable-resample --disable-rtaudio --disable-sox --disable-swfdec --disable-vid.stab --disable-videostab --disable-vmfx --disable-vorbis --disable-xine --swig-languages=ruby
And I'm on Mac OS Catalina 10.15.7.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report with input.xml and the supplied melt command, comparing the first frames with and without the two composite transitions and with profile colorspace 709. Trace the color-processing path used by those transitions; done means output in 709 no longer has a visible shift after the first frame while preserving the reported composite layout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100