Transitions contain a/some "ghost" frames

[hvdwolf]

ppa Openshot 1.42 on Ubuntu 11.10 and Ubunti 12.04 32bits

EDIT: I changed the title from "Transitions contain a "stutter" frame" to "Transitions contain a/some "ghost" frames" as it better discribes the issue.

In my videos I (sometimes) use a set of images. I select the set, add them to the timeline, select random transitions of 2 secs, and add them.
Some transitions contain a "stutter" in the transition itself in the beginning.
I tried by slightly changing the transition time, both by making it slightly too long and slightly too short. No matter what, in some transitions the "stutter" remains.
I did this now numerous times and it's not one transition type but all transition types. Not always the second and fifth (for example) transitions, but sometimes the third, fifth and seventh.
In one set of images of a test run, some transitions are OK and some others contain the stutter.
When I simply start a new project, add the same or a slightly different set of images with random transitions, it happens again.
The transition type itself (dissolve, fractal, wipe, etc.) doesn't matter. In one run a wipe and dissolve don't work. In another run the blinds and clouds don't work. Not only the transitions are random, so is the stutter.

It is both displayed in the preview as well as the rendered movie.

Any clues to this?

[hvdwolf]

some more information.

When I put a video clip on the bottom track in front of the image "collection", with or without a transition to the first image, the stutter appears.
When I put the video clip on the top track 2, or I don't put a video track in front of the images, it works correctly.

See the attached images for explanation. I assume this is not a user error but a bug. Please confirm and I will post it as a bug.

[hvdwolf]

Even more information.

It only happens with some blender created and openshot rendered titles.
What I did is create a couple of animated titles by just playing with the settings. I rendered these titles (and the title only) to x.264 mp4 movies. This to be able to use them again and again in my "productions", but for the time being they are just experiments.
These same title movies I put in front of my images. Some of the created titles give the stutter effect, or maybe the title "ghost effect" or "ghost clips" is better, like it is used/described in this post. (So I updated the title of this post as well)

I already checked whether the mentioned video length of the video codec (h.264) is the same as the length of the video container (.mp4): It is correct (the same length) for all movie title movies.
Note also that all projects I use(d) have the exact same settings: always HD 720p 25fps

[Andy]

Did you use the 'Add to Timeline' option to add the videos/transitions? If not can you try it with that & see if the same thing happens.

It might be this bug:

https://bugs.launchpad.net/openshot/+bug/520941

[hvdwolf]

Did you use the 'Add to Timeline' option to add the videos/transitions? If not can you try it with that & see if the same thing happens.

It might be this bug:

https://bugs.launchpad.net/openshot/+bug/520941

I just dragged the video to the time line. Then I selected multiple images at once, right-clicked and used the 'Add to Timeline' function. This gives me the "automatic" positioning and automatic adding of transitions between the images. That's all and only what I did with regard to this issue.

And yes: It's exactly the same bug.
I already tried to analyze why it happens with one video clip and not with another video clip, when positioned before the image collection. As mentioned: (untill now) it only happens with animated titles generated via blender and rendered with openshot. I will also try with other videos generated by openshot and other movie editors.
I'm trying to analyze the videos, both the codecs and containers, to see what causes this.

By now I'm thinking it has to do with the h.264 codec. Due to the fact how the frames are built up (B/I frames and less relevant P-frames) it is extremely hard to do frame precise editing. Maybe this "un precision" is also the reason why openshot can't handle it and causes some frame inaccuracy in the rest of the video.
I will transcode the malfunctioning title movies to straight mpeg-4 ASP codec mp4s and/or mpeg2 mpgs instead of the current h.264 AVC codec mp4s.

If it is the h.264 codec way of handling, the next question comes to mind: how does openshot deal with this? Does openshot "cut" on the i-frames in h.264 (keyframes in asp mepg4) to force the frame accurate editing even if it is on a minor different place the use wants it? Or does it have some calculation algorithm to calculate frame precision in the B-frames?

[Andy]

I don't think it's codec specific, at least not to H.264 - I've run into this bug when just using images, when doing a slideshow type thing.

it's a really difficult bug to solve - although I've encountered it before, I can't reliably reproduce it. The reason I asked if you used the Add to Timeline option was because I found it was OK for me when I used that. If I placed the images/transitions manually then I (sometimes) encountered the bug.

What seems to happen is that Openshot places an extra 'invisible' transition which causes the 'ghost' transition to appear. You can see this transition if you open the sequence.mlt file in ~/.openshot. In other words this extra transition gets created within the XML that gets passed to MLT for processing, but it doesn't appear on the timeline.

I plan on looking into it again during our current development cycle, but probably need one of those elusive eureka moments to get to the bottom of it!

BTW: when checking the length of the video files, you really need to do it with ffplay and the 'melt' command line tool. The problem we had seen before with differing lengths was where MLT was reporting a different length to ffmpeg. That was suspected as the main cause of this bug for quite a while, but my recent experiences with images would suggest otherwise. Of course, there could be more than 1 cause that creates the same bug.

[hvdwolf]

I don't think it's codec specific, at least not to H.264 - I've run into this bug when just using images, when doing a slideshow type thing.

You are right. It has nothing to do with the codec. The same openshot blender generated title movie transcoded to another codec (mpeg4 asp and mpeg2) gives the same bug/issue.

However: I now found a work around!
As depicted in the first post of this thread I positioned a "malfunctioning" video in front of the image collection. If I trim this video (or another malfunctioning one) a little, even only a split second shorter, the ghosting/hidden transition disappears.
It must have something to do with the video clip length.

EDIT: The same doesn't work though for shifting the images and transitions (right-click first image and transition) a split second "to the left".

[Andy]

Can you check something in your project when you see this happen? Can you take a look in the Transition properties and Clip properties and see if the 'Position on Timeline' field is the same (for the clip that the transition transitions to).

[hvdwolf]

Can you check something in your project when you see this happen? Can you take a look in the Transition properties and Clip properties and see if the 'Position on Timeline' field is the same (for the clip that the transition transitions to).

Yes and No! I added 8-9 images and with three I had the ghosting. Two of them had the same timeline time. For one of them the image started 0.03 seconds earlier than the transition. I changed that to make it equal but that didn't change the behavior of the transition. So I set the image 0.03 seconds behind the transition: still no change.

What's more: When checking the other correctly "transitioning" images I found that some other images also started 0.03 seconds earlier than the transition but these worked correctly.

And again: No matter whether the clips start 0.03 seconds earlier or not, if I reduce the advancing video clip with 0.03 seconds (or so) the transition works.

Another observation which I should have done earlier: When I add the advancing video clip to the project files section, it simply mentions 00:00:09 seconds.
When dragging it to the time line and right-clicking it, it mentions 9.94 seconds. However, ffmpeg -i <video> or ffprobe <video> mentions that it is 9.96 seconds: a 0.02 second difference.
(Note also that I would expect the time to be rounded to 00:00:10 instead of 00:00:09 in the project files section, if the clip is either 9.94 or 9.96 seconds, but that aside).

Another "malfunctioning" video reports 9.98 in openshot clip properties and 10.00 via ffmpeg -i.
Both videos were created by openshot which means that either MLT or ffmpeg does something wrong (IMHO).

I checked some other videos that were created by Handbrake and/or iMovie (both on Mac OS X): ffmpeg -i and openshot report in about 50% of the cases the same length and in the other cases a difference of 0.01 seconds. Note that handbrake also uses ffmpeg as engine.

It seams that when compensating for the time difference of 0.02 seconds, being: making the video at least 0.03 seconds shorter, the issue is gone.
Maybe a very crude patch could be to make the videos that are dragged to the timeline alsways a bit shorter then they mention to be: somewhere between 0.03 and 0.10 seconds (Just guessing here)

[Andy]

That certainly ties in with one of our main thoughts on this bug, that it is MLT & ffmpeg reporting different lengths for certain clips. That probably explains why certain users are plagued by the bug and others aren't, as it is down to specific file types.

Not sure about that approach for a patch as that would unduly affect clips that are not affected by the bug.

Can you attach the output of ffmpeg -i for one of your malfunctioning files? And are the clips that work OK from the same source?

I'll try and take at a look at the rounding, I agree that something that is 9.94 should be rounded to 10.

[hvdwolf]

All tried clips are blender generated animated movie titles at HD 720p 25fps from the same ppa openshot 1.42 on Ubuntu 12.04. They are all in length somewhere between 3 and 10 seconds.
An ffmpeg -i output (skipped from all info in the top) for one of the movies is

Code: Select all

$ ffmpeg -i media/WBPublic/Shared Videos/openshot-title-examples/neonrondingen.mp4
 <...snip...>
  libavutil    51.  7. 0 / 51.  7. 0
  libavcodec   53.  6. 0 / 53.  6. 0
  libavformat  53.  3. 0 / 53.  3. 0
  libavdevice  53.  0. 0 / 53.  0. 0
  libavfilter   2.  4. 0 /  2.  4. 0
  libswscale    2.  0. 0 /  2.  0. 0
  libpostproc  52.  0. 0 / 52.  0. 0
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/media/WBPublic/Shared Videos/openshot-title-examples/neonrondingen.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 1970-01-01 00:00:00
    encoder         : Lavf53.3.0
  Duration: 00:00:09.96, start: 0.000000, bitrate: 6365 kb/s
    Stream #0.0(und): Video: h264 (High), yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 6363 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
At least one output file must be specified


This one is reported as 9.94 seconds in openshot.

To get the number of frames I used the command

Code: Select all

$ ffmpeg -i /media/WBPublic/Shared\ Videos/openshot-title-examples/neonrondingen.mp4 -vcodec copy -f rawvideo -y /dev/null 2>&1 | tr ^M '\n'
<...snip...>
frame=  249 fps=  0 q=-1.0 Lsize=       0kB time=9.96 bitrate=   0.0kbits/s   
video:7737kB audio:0kB global headers:0kB muxing overhead -100.000000%


ffmpeg tells me that it is 9.96 seconds and 249 frames (and we know it is at 25fps): 249 frames at 25 fps = (249/25=) 9.96 seconds. Why does MLT report that it's 9.94 seconds. That's not even possible at 25 fps. It should report either 9.96 seconds or on wrong frame count, 9.92 or 10.00.
The 9.94 seconds is not a multitude of the number of frames divided by the 25 fps.
MLT seems to be the culpit.

[Andy]

If you're happy to upload this clip somewhere for Dan to take a look at we can report a bug with MLT - but he will need an example file. And maybe 1 that MLT gets right for comparison purposes.

[hvdwolf]

Tell me where and I will upload the clips. I can also upload them to my own webserver or to mediafire.

[hvdwolf]

And another test drive:
I created (again) a neonroundings animated title, which is 250 frames (as you know frames are counted 0 to 249 in movies).
I put it on the time line and it gave me 9.96 seconds, which is the correct length.
Then I rendered it to ogg, mpeg, asp avi, avc avi, asp mp4 and avc mp4.
I did a plain "ffmpeg -i <movie>" (1) of the movie and a "ffmpeg -i <movie> -vcodec copy -f rawvideo -y /dev/null 2>&1 | tr ^M '\n'" (2).
Then I dragged them to the timeline in openshot and checked the properties.

Reported lengths:

Code: Select all

movie               (1)             (2)                     openshot
                   secs          frames/secs                 secs
asp.avi            10:18         252/10:08                   10:18
avc.avi            10:08         252/10:08                   10:08
asp.mp4            10:18         252/10:08                   10:18
avc.mp4            10:08         252/10:04                   10:08
.mpeg              10:00         252/10:04                   09.98
.ogg               09:99         249/09.96                   09.98

By now I get my doubts about the number of frames, but still I think they count from 0 to something.
Now if we count the 0 frame as 1 frame, we get (249+1)/25 = 9.96 secs, and we get (252+1)/25=10.12 secs.
On the other hand: 252/25=10.08 secs.
Nevertheless: When creating a 250 frames animation, where come the 252 or 253 frames from?
And yes: I checked the number of pngs in my blender created sequence. There are 250 pngs counted from 1 to 250.
So: obviously not only MLT screws up. ffmpeg screws up as well.
Some thoughts:
-the container (avi, mp4, mpeg, ogg) displays a different number of frames than the codec stream (mpeg4-asp, mpeg4-avc, mpeg2, theora) inside the container.
- Both report it wrong and the second ffmpeg command simply counts the number of frames processed.

And then the proof of the pudding:
I created a new movie where I placed the blender title sequence and all all the movies (one by one of course) in front of the image collection.
Only the avc (h.264) coded title movies, so both the avc.mp4 and the avc.avi movie, generated the "flicker" in the frame.

My final conclusion/question (at this moment ): The h.264 codec is too blame, either the encoding or the decoding. Does that mean that the x264 lib is the real screw up?
When doing an ldd to check which libs are linked for a certain executable I find:

Code: Select all

ubuntu@ubuntu:~$ ldd /usr/bin/ffmpeg | grep 264
   libx264.so.120 => /usr/lib/i386-linux-gnu/i686/sse2/libx264.so.120 (0xb6044000)
ubuntu@ubuntu:~$ ldd /usr/bin/melt
   linux-gate.so.1 =>  (0xb7740000)
   libmlt.so.4 => /usr/lib/libmlt.so.4 (0xb76fa000)
   libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb76df000)
   libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7539000)
   libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xb7534000)
   /lib/ld-linux.so.2 (0xb7741000)

(I grep-ped ffmpeg as it is linked against a huge amount of libraries.). Is melt statically linked to x264 in the ppa?

[Andy]

Ok, so I suggest uploading your files to somewhere like mediafire and then create a bug report here:

http://sourceforge.net/tracker/?atid=613414&group_id=96039&func=browse

Include the link to your files in your bug report, and provide info on your MLT & ffmpeg versions as well as a detailed description like you have here. Dan is out of town at the moment, so not sure how quick he will be able to respond.

I don't know whether melt is statically linked to x264, I'm not an expert when it comes to builds unfortunately.

[hvdwolf]

I created a bug ticket for Dan Dennedy: <https://sourceforge.net/tracker/?func=detail&aid=3523049&group_id=96039&atid=613414>