Working with Video

[HeatherHaze]

13 hours ago, scook said:

The is a limitation of the Media Foundation engine, One wonders if MS will ever update it.

To move video use the Directshow engine. See https://www.cakewalk.com/Documentation?product=Cakewalk&language=3&help=Troubleshooting.07.html

Here is a recent thread about this issue

 

 

Thanks for the reply.  I did see this thread first, and tried the various suggestions.  Nothing worked.  And not to put too fine a point on it, but it's not just a limitation of the Media Foundation engine.  Cakewalk (now Bandlab) has chosen to use this engine in spite of its limitations.  I know other DAWs that work in Windows don't have the same limitation (i.e. Cubase).  So I consider this a limitation of Cakewalk.  I don't care what engine they use.  I just expect it to work.  ? 

I don't mean to sound cynical.  I've been using Cakewalk since version 2.0 for DOS (read: a very long time).   It is far and away my favorite DAW, and I'd much prefer to do everything in Cakewalk.   That's largely why this little quirk is so frustrating to me.  

[Noel Borthwick]

1 hour ago, HeatherHaze said:

Thanks for the suggestion.  I've tried this and have had this setting active for some time.  It doesn't seem to help, the video is still using the Media Foundation engine.  

Right because directshow doesn't support mp4 out of box on windows. So catch 22.

Video support on windows is problematic unless you roll your own video support. Something we don't have the time or resources to do. Well loop back at some time and try and work with Microsoft on solving the MF issues which is really the only option available at this time.

[LarsF]

Don't know why MF would care if daw just used an offset where to start playing the video?

What decides if video plays and from where on timeline?

Why cannot Cakewalk just keep an offset where video starts, just puzzled over this.

As I recall you can turn off playing loaded video.

So why not offset in code, is my question as retired programmer. MF must get time from somewhere like Cakewalk now do relocate video as you might have moved playback cursor and then start playback. So I guess Cakewalk have an internal offset in timeline where this is to relocate.

 

I feel that StudioOne are as low on resources playing back video, like mp4, as Cakewalk, and StudioOne allow offset.

And StudioOne immediately locate video with transport stopped - if relocating project cursor - which Cakewalk does not - would be great if it did.

Not sure though what engine is used, but really good in my experience.

Cubase new video engine is horrible on resources and I used VidPlayVst I linked to instead - cpu lowered by 16%. Project went from 36% to 20% total using VidPlayVst instead of native video engine. I waited bleeding 18 months for Steinberg to fix a new video engine and when it came it were a resourcehog.

 

VidplayVst also allow to set a midi C-note anywhere as video is supposed to start. So if to make do with no thumbnails.

Based on some open source code, I think.

 

[HeatherHaze]

7 hours ago, Noel Borthwick said:

Right because directshow doesn't support mp4 out of box on windows. So catch 22.

Video support on windows is problematic unless you roll your own video support. Something we don't have the time or resources to do. Well loop back at some time and try and work with Microsoft on solving the MF issues which is really the only option available at this time.

Okay, I understand.  I appreciate your forthright reply.   I'm sure you realize this one minor quirk makes Cakewalk unattractive for film composing.  It's really a shame, because otherwise I think Cakewalk is absolutely unbeatable.  I'm deeply grateful for the fact that Cakewalk continues to exist, but I'm still worried about its future.   

[Noel Borthwick]

Our video support isn't the best, but I don't fully agree that not having an offset makes it unusable for use with film scoring, since there are options to make it work. I do a lot of video work myself and its a simple task to export a video from any video editing application that is aligned with the music in Cakewalk.  From that point on there is no need to offset the video in Cakewalk. We've explored many options with video and have yet to find something that works across the board.
Sadly video at least for desktop apps is one area that Windows is sorely lacking and Microsoft has yet to address it cohesively. This shouldn't be an area where companies have to reinvent the wheel. Video is enormously complex to implement from scratch and even worse to support with the moving targets of video formats. We have considered using FFMpeg  and may try and revisit that - it is also not a panacea for everything since it requires a lot of integration work.

[winkpain]

I concur with the frustrations expressed here by @HeatherHaze   I certainly look forward to a time when this issue can be improved, if it can be. I hope that it can be! I appreciate that there are these complications and that it's not really a front burner issue for CW. I am a lowly creator and not a coder so do not understand the complications involved, and I do tend to feel that if other applications on Windows can work with video successfully, then this one can too. For sure this is a simple minded response, but just sayin'.

But yes wow, the Catch 22 problem with the video engine and the corresponding offset issue is a bummer (this has plagued me quite a bit in the last year). Not being able to load multiple clips is a drag. My issue with .mov files continues to evade solution.

I work  a lot with video, usually from various sources, that I am then to compose and track to. In addition to scoring film and animations which are preferably finished edited pieces when I get them, frequently they are not or just working edits, a lot of what I get are videos of rehearsals (scoring for theater) and is stuff that is changing all the time during the work in progress. In these cases I'm just using them for necessary visual timing and relative hit points, etc. The videos themselves are not part of the final project, so having to edit and prep a full timeline's worth of clips before bringing it into CW is a time consumer for something (the video) that is just going to be discarded anyway. But it is a workaround.

CW is just not the right tool in these cases. Which is too bad, because WE LOVE IT SO and want to use it throughout the workflow.

I respectfully vote for improved video functions at some point!

Edited May 22, 2020 by winkpain

[LarsF]

I find Cakewalk video handling pretty good, but not dependent on that a piece of music is frame accurate when scenes shift.

I don't notice it's there, which is a good thing. Both Cubase and Reaper felt seriously sagging starting/stopping transport etc.

Since using mixed locked and non-locked markers though - it would be swell to be able to shift to previous/next locked and previous/next non-locked. So quickly jump to next scene(locked in time) or a musical spot(non-locked, on grid).

Maybe modifier key together with current previous/next marker or separate key binding to do either.

 

As retired programmer I don't know how you could not just subtract an offset in frames project from the place where video is located and run.

This frames offset is saved with project.

- timeline pos minus frame offset become negative - no video is running as transport runs

- timeline pos minus frame offset is positive - run video from that calculated spot when transport runs

When positioning cursor on timeline, somehow this translate into a pos into video. Playback starts and this location is found.

So why this call to start video cannot just as well adjust the pos by some frame offset, beats me.

Like you have a file position to tell where next file write goes and similar - isn't there same thing for a video?

Just seems so unlikely this does not exist - that one parameter is frame offset, or timestamp into video - or a separate call to move that position, which I assume is what happends when you put cursor in an arbitrary spot and then start playback. Video is synced to that.

 

Or some other solution which Heather referred to, if having black frames as start to work as offset into timeline, that she needed correct frame - for scene changes I assume. Can there be a calculation so such offset method could work for professionals doing score.

 

I have never used project offset - to tell that project bar 1 is somewhere else than far left( if Cakewalk has that).

Is there a solution with this?

 

Since i slide in some initializing clips before project, usually start on bar 5 or 10 or so, seems to create problems if I cannot go earlier than 

[Jean Corivo]

On 5/21/2020 at 12:46 AM, Noel Borthwick said:

Quicktime video definitely works in Cakewalk using the Media Foundation engine in Windows 10. See image where I am running a QT video in the latest release.
Keep in mind that the MOV extension is just a container for a video format so just looking at the file extension doesn't mean anything.

If stock Windows can play the video then Cakewalk, should be able to load it since we rely on Media Foundation for all our video support. For it to be compatible with Windows the video must be encoded in a compatible format such as MPEG4 or you must have the appropriate codec installed. Generally a thumb rule is if the Windows video player can play it then it should be ok to import.

Actually, the Quicktime .mov works well with the "Media Foundation"  video engine.
However, changing the starting point is impossible and the entry points and operate in an approximate manner.
You have to fall back on the old video engine, much less powerful, but which allows a better handling of .avi files. (mjpeg, DV and a series of old, mostly obsolete codecs)
The only way to get around the problem is to use an application like VidPlayMTC or a second computer and do the synchronization by midi timecode. However this slows down the production process, especially when you need to synchronize sound elements at specific sync points. An integration of a better video player would be highly appreciated.
However, Cakewalk is a fantastic creative tool and the multiple updates show that it is powered by a dynamic and energetic team. ! 

Edited May 23, 2020 by Jean Corivo

[LarsF]

35 minutes ago, Jean Corivo said:

However, changing the starting point is impossible and the entry points and operate in an approximate manner.
 

By that approximate you mean - when we enter a marker from left or from right I can see that playback does not always start on the same frame?

When you change an entry point, is that when you placed cursor on any pos in timeline and start playback?

What kind of offset are you giving MF calls for this?

Frames offset from start or timecode offset?

[LarsF]

Found this article

https://docs.microsoft.com/en-us/windows/win32/medfound/seeking--fast-forward--and-reverse-play

Two calls

 

HRESULT GetCurrentPosition(MFTIME *phnsPosition);

HRESULT SetPosition(MFTIME hnsPosition);

handle seeking in video.

So why cannot daw timeline offset itself to video and just set another time?

 

If using GetCurrentPosition you add internal offset in timeline to that.

When using SetPositon() you subtract offset from timeline and send to SetPosition().

 

Edited May 23, 2020 by LarsF

[Noel Borthwick]

Seeking to a different offset is not quite the same thing as actually trimming the vide because that is implemented at the video engine level and is reflected in video exports, looping, etc. Also you can relocate the trimmed video anywhere on the timeline and the thumbnails will show the correct video frames. If all we did was seek to a different frame this wouldn't happen automatically.
Media foundation is rather complex - all the video playback is handled internally in the MF engine. The host sets up a video track with the clips and trim points and then all the playback is handled by the engine which we slave to our audio engine clock.

The bug with trimming is Media foundation hangs whenever we set trim points. I had a lengthy thread with MS years ago and they were unable to figure it out at the time. I don't think they have many people who even know this stuff there! They seem to spend more time changing sdk's then actually fixing the stuff that has been broken forever. I'll follow up again and see how it goes

[LarsF]

I just found over the years that when automatic does not work, to have full control you go manual.

I assumed playback was handled by engine but where on timeline anything happends is up to devs. And adjusted seek would at least play back where timeline is. And if you slave to your clock that offset would remain was my attempt with this angle.

But export might have issues writing audio in the right spot, nothing automatic anymore. But everything up to export could be handled with these seek I believe.

But maybe as Corivo said - MF engine is just approximate positioning and maybe not what Heather and other professionals need.

So the work would be pointless if having other issues anyway. Just a thought.

 

I just do audio export, not video, and align audio in NLE. So not sure why professionals don't do that, why stick to the daw.

I found implementation really good as I made black frames at start. Just zooming in and getting frame numbers on thumbnails consecutive and where first nonblack appear, I set start marker and align bar 5 to that spot. and do audio export from that spot too.

 

So another route to solve it might be to handle presented frame numbers or something whatever HeatherHaze needed to know the spot even introducing offset frames at start. Not sure how she meant. If you find your actual video after blackframes start 500 frames into timeline, you subtract that value from what is presented - so where in original is alway at hand. Then she could use blackframes trick at start. Maybe she could chime in and tell exactly what are the needs.

 

Then for export one could introduce a checkbox saying something "substitute preview video in project, ask for original at export" or similar. So the black frame lead version in project is replaced for the actual export if writing project audio into it.

Then audio is replaced in that separate video. I guess if having 4k-8k video possibly, you would not want that for preview anyway. Unnecessary heavy load for just preview purposes.

 

Just another bunch of ideas....if this is a turnoff for those not happy with Cakewalk for video.

Start MS rant.....

About MS, they seems to be flooded with rookies in later years. The attempt to replace Xbox forum as one was a disaster and made by real beginners. And was unusable for a year until some other people took over. They destroyed and fully working forum, and not sure if recovered yet.

I stayed on VS2005 due to that they removed the fully integrated help and search system. Still use it. But that they maintained it with fixes to work on Windows 7 and 8 is nice. A load of patches, but still that they did.

On the good side, they are learning the customerbase still left. FLS issue was fixed etc. But that it even were an issue show how little they know how the pc is used out there - how daws load hundreds of plugins etc. Most households probably do well without a pc at all. All needs for internet, social media is handled by smartphones and pads - and MS missed that train all together. So they better listen to userbase with big ears if even having a pc market in future, the foundation for MS as a business.

 

Windows 10 is a mockup with all these updates turning the machine upside down just by a new update. I avoided getting new machine now for 4 years - I simply don't want an ever changing machine. They should move back to make Windows 11, 12 and so forth - and let people have fully functioning machines not interrupted by updates. It's just convenient for MS to maintain one platform. So they go cheap there too. Making a platform targeting people that surf internet mostly - which probably use pads instead anyway.

 

After all these years would be nice if MS could make a safe os to start with. Just adding stuff that make it vulnerable and have to constantly be fixed.

 

End rant....

[Millzy]

A timely post! Last week, for the first time ever, I imported a video file into cbb. I was asked to record a bass track to it as part of an ISO music project. I wondered why I couldn't slide the video clip along the timeline a bit to allow me more time between hitting record and starting to play.  Some good information here, thanks Noel for explaining some of the limitations you guys are up against. Let's hope there's a possibility of a workaround sometime that would allow one or two basic video functions to be added.      

[RichusRkr]

my old sonar platinum has a Video option in the View menu and plays and syncs my video no problem. My up to date bandlab has no Video option under the View menu even though it sucessfully imports my MP4 and does play the audio and displays the video thumbnails I cannot find a way to play and sync my video.  what am I missing?

 

[scook]

When a documented feature seems missing  such as the Video view chances are a workspace is running that does not have the feature.

Either create a custom workspace or set the workspace menu to None.

[Andres Medina]

I work a lot in projects involving video and syncing music to it.

No problems at all, providing that:

1. the video is encoded with mp4 codec and you have your windows codecs up to date.

2. You add a blank space at the beginning of the video to provide room for any pre roll you need for your music.

3. The exported video is not too heavy (for your computer configuration).

4. The other thing, for accurate frame syncing is to reexport your video file with as little as 1 frame in the Keyframe distance option of your video editing.

This ensure that when advancing your video frame by frame (in the video window of Cakewalk) , the video doesn´t jump several frames. 

Once you get the video right, just import it, and it should play smoothly,  using little resources.

If you get it right once, to get it right for ever.

For the music composition task, I find useful this:

1. Lock completely  the audio track imported along the video, to ensure that it doesn't  change position when you change your tempo.

2. Mark your video syncing spots using the markers feature and lock them, for the same reason.

Sure, it would be nice if someday this wonderful DAW could just move the video in the timeline and lock it...! 

Hope this helps!

 

[Andres Medina]

Just now, Andres Medina said:

I work a lot in projects involving video and syncing music to it.

No problems at all, providing that:

1. the video is encoded with mp4 codec and you have your windows codecs up to date.

2. You add a blank space at the beginning of the video to provide room for any pre roll you need for your music.

3. The exported video is not too heavy (for your computer configuration).

4. The other thing, for accurate frame syncing is to reexport your video file with as little as 1 frame in the Keyframe distance option of your video editing.

This ensure that when advancing your video frame by frame (in the video window of Cakewalk) , the video doesn´t jump several frames. 

Once you get the video right, just import it, and it should play smoothly,  using little resources.

If you get it right once, to get it right for ever.

For the music composition task, I find useful this:

1. Lock completely  the audio track imported along the video, to ensure that it doesn't  change position when you change your tempo.

2. Mark your video syncing spots using the markers feature and lock them, for the same reason.

Sure, it would be nice if someday this wonderful DAW could just move the video in the timeline and lock it...! 

Hope this helps!

 

... All the video related tasks described above refers to using a video editor to prepare your personal copy before importing into Cakewalk.

[Andres Medina]

2 hours ago, Andres Medina said:

I work a lot in projects involving video and syncing music to it.

No problems at all, providing that:

1. the video is encoded with mp4 codec and you have your windows codecs up to date.

2. You add a blank space at the beginning of the video to provide room for any pre roll you need for your music.

3. The exported video is not too heavy (for your computer configuration).

4. The other thing, for accurate frame syncing is to reexport your video file with as little as 1 frame in the Keyframe distance option of your video editing.

This ensure that when advancing your video frame by frame (in the video window of Cakewalk) , the video doesn´t jump several frames. 

Once you get the video right, just import it, and it should play smoothly,  using little resources.

If you get it right once, to get it right for ever.

For the music composition task, I find useful this:

1. Lock completely  the audio track imported along the video, to ensure that it doesn't  change position when you change your tempo.

2. Mark your video syncing spots using the markers feature and lock them, for the same reason.

Sure, it would be nice if someday this wonderful DAW could just move the video in the timeline and lock it...! 

Hope this helps!

 

... and a snapshot of the export video configurations that always works for me. H.264 codec is the best I came across.

Using this configurations I manage to work really smoothly with video inside Blb.

 

[Kev]

After exporting video to AVI format (uncompressed), I use Handbrake (which is free) for subsequent conversions of the AVI file to other formats. Converting takes less time than exporting, i.e. seconds rather than minutes.

[Jonathan Shapiro]

Noel:

Several folks here point out that having up-to-date CODECs is essential, and I concur that CODEC management is something of a black art in Windows - more so because you may have something work due to some random application installing a CODEC you don't even realize is there.  I also get that Bandlab has finite resources. I have two thoughts:

1. Would it be possible to assemble as some kind of tech note or help entry somewhere a list of suggested CODECs that people at Bandlab know to work and where they can be obtained?

2. MOV, as you say, is a container format. Does it have the ability to stream successive video segments from a single file? If so, would it be feasible to insert a leader segment into the MOV file directly and then let Media Foundation just play it? I have no idea if that would work; just spitballing here. It would certainly be a disgusting hack....