Controller Refresh Frequency

[norfolkmastering]

Amon, in the Bandlab support team, suggested I post this topic to reach out to the development team.

I have a fairly unusual application which needs a fast Controller Refresh Frequency, preferably 10ms.  This allows me to record into Cakewalk and simultaneously to an external analogue recording system, all sync'd and controlled from Cakewalk.

The 2020.08 release allowed the minimum Refresh Frequency to be reduced from 50ms to 25ms which was a really useful step.  At that time I was told that reducing the Refresh Frequency down to 10ms would be possible but via the Cakewalk.ini.  I have not been able to get this working.  Although the Cakewalk.ini can be edited to 10ms and displays as 10ms in Preferences\Control Surfaces, the actual refresh frequency is still operating at 25ms.  I have tested this very carefully,

It would be great if the development team could have a look to see what is blocking the 10ms setting from operating and hopefully remove the blockage.

I have a whole dialogue of how I test the refresh frequency which Amon has in an email.  Happy to reproduce here if it would help.

Regards

Robert

 

[msmcleod]

@norfolkmastering - I had a look at the refresh interval issue. It looks like some of the UI optimisations have had the effect of capping the refresh rate at 25ms.  Although the timer controlling the refresh is in its own thread, the refresh and any MIDI messages that are requested to be sent out by the control surface are synchronised to the main thread.

I did manage to find a way around this to support a refresh of 10ms, however its worth noting that any MIDI messages sent back out via Cakewalk will still be capped at 25ms.

The only way around this would be for your control surface DLL to deal with the MIDI messages outside of Cakewalk - either that, or not use MIDI at all for controlling the analog side.
 

[norfolkmastering]

Hi Mark

Thanks for checking this out.

I am using AZ Controller to configure and output the MIDI messages.  Does that count as 'outside of Cakewalk'?

If 'yes' then please let me know what would be required to get the 10ms refresh operational.

Best regards

Robert

[msmcleod]

2 hours ago, norfolkmastering said:

Hi Mark

Thanks for checking this out.

I am using AZ Controller to configure and output the MIDI messages.  Does that count as 'outside of Cakewalk'?

If 'yes' then please let me know what would be required to get the 10ms refresh operational.

Best regards

Robert

AFAIK, AZController uses Cakewalk to send it's MIDI messages, as it'll be using the control surface API.  So whilst the calls to refresh can be increased, the frequency of how quickly MIDI messages can be sent out from AZController can't as long as it's doing it via Cakewalk. 

What I think is required is for AZController to call midiOutShortMsg and midiOutLongMsg directly. This would require @azslow3 to make some modifications to AZController.  You'd also need to ensure that the MIDI out device was NOT enabled in Cakewalk so ensure that AZController had full control over it.  In other words, in Cakewalk, you'd configure your MIDI In device only, but the MIDI Out device would have to be set within AZController itself.

It's also worth noting that a refresh frequency of 10ms had an adverse affect on Cakewalk's performance.

I experienced a fair amount of dropouts unless all other applications were closed - just having a browser open would cause dropouts in Cakewalk.  I suspect this would be improved if AZController had full control over sending MIDI messages, but given that all control surface functions are performed in the main thread, I'm not sure this would be a significant improvement unless perhaps AZController used a separate thread for sending MIDI messages.  Any calls to Cakewalk must continue to be done in the main thread though, and doing this every 10ms will adversely affect performance.

 

[azslow3]

I don't think MIDI IO speedup is going to bring any improvement for the topic in this particular case. MIDI messages in question is reaction on parameter changes in Cakewalk.  So something has to be changed during 10ms and AZ Controller has to find that change. In Cakewalk  CS API that is always done by routine scanning, which by itself is going to take some time (primary in Cakewalk). Also current buffer size (I guess) is going to add its length as jitter to that process, since CS loop is not synced with RT audio.

BTW I have already proposed on my forum that smooth changes in parameters should be inter/extrapolated on the hardware side (unlike in VST3, there is no ramp info possible in CS API, so I understand it is not easy to decide how to do this).

[msmcleod]

3 hours ago, azslow3 said:

I don't think MIDI IO speedup is going to bring any improvement for the topic in this particular case.

That's good to hear

3 hours ago, azslow3 said:

...MIDI messages in question is reaction on parameter changes in Cakewalk.  So something has to be changed during 10ms and AZ Controller has to find that change. In Cakewalk  CS API that is always done by routine scanning, which by itself is going to take some time (primary in Cakewalk).

I don't think finding the change should take too long - assuming that you're not querying too many parameters, and the COM overhead isn't too high (it shouldn't be as by that point, you've already got the interface, and it's not inter-process).  Cakewalk already knows the state, and all you're doing is a parameter look up on that state. There's no messages involved, just function calls and a big switch statement.
 

3 hours ago, azslow3 said:

BTW I have already proposed on my forum that smooth changes in parameters should be inter/extrapolated on the hardware side (unlike in VST3, there is no ramp info possible in CS API, so I understand it is not easy to decide how to do this).

This was something I was going to suggest in my previous post. I built a couple of hardware projects that both eventually "failed" due to me relying on some sensible balance between the frequency of refresh rates, and what the hardware could realistically handle.  In my case, my displays were i2C based and the i2C bus couldn't cope with the throughput of updates (I was also multiplexing on the i2C bus, which added further complications with higher rates). However, a lower refresh rate meant parameter changes weren't smooth enough.  It's a pity, as it worked fine with a small number of knobs/displays, but just didn't scale up. In the end, I gave up and bought a Mackie C4

My suggestion for the OP however would be interpolation of the values in hardware. If continuous values are needed, it might need a phase-locked-loop based on a lower refresh rate - this could be implemented in software within the hardware's microcontroller.
 

[azslow3]

5 hours ago, msmcleod said:

I don't think finding the change should take too long - assuming that you're not querying too many parameters, and the COM overhead isn't too high (it shouldn't be as by that point, you've already got the interface, and it's not inter-process).  Cakewalk already knows the state, and all you're doing is a parameter look up on that state. There's no messages involved, just function calls and a big switch statement.

24 tracks x (track, sends, EQ and Comp parameters). So O(1000).  It takes at least 2-3 times more calls then parameter to get the information. Not sure when it start to take significant time, but there should be some limit even on modern computers ?

[msmcleod]

2 hours ago, azslow3 said:

24 tracks x (track, sends, EQ and Comp parameters). So O(1000).  It takes at least 2-3 times more calls then parameter to get the information. Not sure when it start to take significant time, but there should be some limit even on modern computers ?

hmm - 1000 parameters is a lot.

I don't think the CS API was ever designed for querying such a large amount of parameters. If that had been envisaged, it would probably have had a call to return all the common parameters in a single struct as the result of a single call.

[norfolkmastering]

Hi both

Thanks for exploring this topic so thoroughly.  I just wanted to clarify a few points re my intended use of the Cakewalk and AZ Controllers.

Remote control of parameters in my analogue mixer is only for routing and level/pan control.  All the other analogue processing (e.g. EQ and Compression) is manually controlled using inserts and outboard gear  As routing is primarily a session set up function, this leaves just level control which will happen 'in real time'.   In terms of processing overheads within Cakewalk, there will be no software plug-ins being used.

The primary reason I wanted to reduce the Cakewalk Controller Refresh Time below 25ms was for increased precision and time accuracy of fast crossfades between takes. 

As one example:-

I recently mixed a song with four takes of a technically very difficult piano piece.  I had to use fast crossfades to combine the best of the four performances.  In the Cakewalk digital domain this is reasonably easy to achieve.

In the future, when I am recording both to Cakewalk and to my 24 Track analogue tape machine, I will do all the mixdown rehearsal in Cakewalk and then do the final mixdown from the tape machine which is locked to Cakewalk using SMPTE (striped on to Track 24).  Cakewalk will control my audio mixer so that all the rehearsed level control can be implemented in the analogue domain.

If I was to use inter/extrapolated data processing within my audio mixer to gain more control precision, then I introduce another (albeit small) delay in the control chain between Cakewalk and my target VCAs.  I would prefer not to do this.  My VCA gain control elements have an integration time of around 7ms which takes care of zipper noise issues quite successfully.

So I would like to be able to experiment with Cakewalk Controller Refresh times below 25ms if this is possible to implement.

Regards

Robert

 

[azslow3]

12 hours ago, msmcleod said:

hmm - 1000 parameters is a lot.

I don't think the CS API was ever designed for querying such a large amount of parameters. If that had been envisaged, it would probably have had a call to return all the common parameters in a single struct as the result of a single call.

May be I have misunderstood the intention with EQ/Comp, it will be way less then.

By itself 1000 parameters under default timing is not a problem, there are some presets which use that amount. At the beginning I had worries, so my monitors have "speed" parameter to not ask every cycle. In practice, I have not hit significant CPU use nor audio glitches by monitoring every cycle. All my presets have all monitors in that mode.

But when requesting CPU time is absolute, let say 1ms per loop, that is just 1/75 of not RT processing by default. With 10ms cycle that is 1/10 and can start influence something.

[norfolkmastering]

Hi Alexey

I think you maybe misunderstood.  Only my mixer routing and level control will be controlled from Cakewalk.  (So I will not control EQ or Comp from Cakewalk).  In terms of 'dynamic' parameters, that means 24 tracks plus 8 buses, each with:

Gain, Pan, Level (so 96 in total)

in addition, what I call more 'static' parameters:

Track to Bus Routing, Solo, Cut, Rec arm, Interleave, Source routing (maybe around 600 in total)