Issue with multi processing and thread sync

[Henrizzle]

Hi David,

It all works perfectly with single processor. It actually works better when I offload more of the processing to UAD2. The problem is when there is more native processing. There is a threshold where the mixing threads stop getting lucky because there is thread sync object somewhere that is not multiprocessor safe. I have been doing digital audio software for over 20 years, I don't do the voodoo style troubleshooting. When this threshold is crossed there is only about 28% processor usage, so it has to be a mutex or semaphore that is not processor safe. In single processor case all the stuff gets serialized on the one processor there is no sync issue.

Also the random elements I mention would not cause this issue as they are random elements meant to dither while the data pipeline is the same everytime.

Well I don't agree with this stripping method of trouble shooting, this is voodoo. There is a cause and effect to everything.

Again the UAD is not the issue as the more native stuff I remove to put o UAD it pop and scrapes less.

Anyhow, thanks but this sort of voodoo trouble shooting it not useful in this thread, I have contacted Cakewalk support and they are investigating the issue.

Even if they can just get the option working where you specify no multi thread rendering for the export it would be an improvement as then you can be sure there arn't these artifacts that you notice later in the export. For now I am changing to single processor render to export then changing back to multiprocessor to do mix adjustments.

[David Baay]

Troubleshooting by the process of elimination is "voodoo"?  In the absence of access to the source code, it's one of the few avenues available for an end user to investigate.  The "binary" approach I outlined is just a way of speeding up the elimination process in a big project. If the problem is truly involving thread sync, and isn't involving any specific plugin(s),  this approach might not help, but it seems to me you're making a lot of assumptions about the root cause of the problem that are not justified by the evidence.

I'll be interested to see what the Bakers find.

[Noel Borthwick]

It is very unlikely that it is a general multiprocessing sync issue in the DAW. It has been extensively tested over decades and if there was even a tiny bug there there would be thousands of users running into it.

The more common problem is with plugins. Do the same test completely disabling or removing all plugins and use that as a process of elimination. You can export audio and choose to not include plugins. Definitely eliminate hardware UAD plugins in your test.

Though less common there may also be a disk throughput issue leading to inability to cache data in time. Some samplers have been known to have issues like this.

[Simon Payne]

Solved it Cakewalk. Make your plugin chainer apply the vsts in parallel as apposed to series. They can then properly distribute the processing over multiple cores and threads. Until then Blule Cats Patchwork dose the job. 

[David Baay]

14 minutes ago, Simon Payne said:

Make your plugin chainer apply the vsts in parallel as apposed to series.

If I understand correctly, you're conflating parallel processing  with parallel routing.  What  you're describing is parallel routing . Parallel routing of signals through FX  plugins is likely to sound vastly different from serial routing, and isn't generally desirable except for specific purposes like adding reverb and parallel compression. And parallel processing should not depend on (or benefit from) parallel routing.

 

[Simon Payne]

On 8/10/2023 at 7:21 PM, David Baay said:

If I understand correctly, you're conflating parallel processing  with parallel routing.  What  you're describing is parallel routing . Parallel routing of signals through FX  plugins is likely to sound vastly different from serial routing, and isn't generally desirable except for specific purposes like adding reverb and parallel compression. And parallel processing should not depend on (or benefit from) parallel routing.

 

no parallel routing in the vst bin is the gateway to parallel processing. Cakewalk implements neither that's the problem. 

Edited August 11, 2023 by Simon Payne

[Simon Payne]

[Glenn Stanton]

actually the latest version is 2.66 🙂 

[Max Arwood]

I guess I need to update and try it. I am certainly a plugin over user lol!

[Henrizzle]

Okay I have officially moved to StudioOne, was able to get the Artist version for $99 it has all the features I need. The same mixes using same tracks and same plugins works perfectly in StudioOne, as a matter of fact I can put on lots more without running into this issue. This is not an issue with something configured wrong, rather it is an issue with multiprocessor mix threads synchronization like I have stated. So when you are running below threshold, it is simply getting lucky most of the time (threads completed in correct order for the audio frame). As soon as the system approaches about 28% or so on my machine it stops getting lucky. So if the developers are seeing this please check for multiprocessor safe mutexes and semaphores AND look for a place where you actually forgot to put a sync object this is where it is getting lucky. 

[Quick Math]

I might be experiencing something similar as well.

When I export my mixes, I often sense something's not quite right, like a change in the sound or a harshness.

This mainly happened on larger projects, and I started to notice it about a year ago, though I wasn't sure.

Just yesterday, I exported a WAV file for a project with many plugins, and again, the exported file didn't sound right.

And then I tried both multi-core and single-core settings for my exports multiple times and compared them.

The single-core versions sounded natural to me.

The harshness I could hear from multi-core rendered versions could be internal errors or something.

While this could be just my perception as the difference seems to be very subtle, I try the single-core setting.

 

 

[Henrizzle]

3 hours ago, Quick Math said:

I might be experiencing something similar as well.

When I export my mixes, I often sense something's not quite right, like a change in the sound or a harshness.

This mainly happened on larger projects, and I started to notice it about a year ago, though I wasn't sure.

Just yesterday, I exported a WAV file for a project with many plugins, and again, the exported file didn't sound right.

And then I tried both multi-core and single-core settings for my exports multiple times and compared them.

The single-core versions sounded natural to me.

The harshness I could hear from multi-core rendered versions could be internal errors or something.

While this could be just my perception as the difference seems to be very subtle, I try the single-core setting.

 

 

Do yourself a favor and try another DAW. I spent $99 on StudioOne Artist version, has everything I need, did 3 mixes today. Super easy without issues and artifacts. So I am leaving Cakewalk after over 20 years of using this DAW. It is not being maintained, currently I am experiencing these issues:

- Artifacts with multiprocessing mix threads, also encountered artifacts once using single thread render, so there is some core sync issue with audio frames and tasks

- UAD2 plugins, playback studders then grinds to halt if you add plugin while running, sometimes even crashes or program is stuck in loop and unresponsive, adding these plugins while running works perfectly in StudioOne

- Some tracks with volume envelopes will disappear using Multiprocess render

- Config file setting don't work. For example, there is a setting to specify not to allow multiprocessor render on export, it does not work, instead you have to change to single thread in preferences to render without the artifacts and tracks disappearing

- Takes a very klong time sometimes to open project, this also points to threads out of sync and is waiting for something to happen, sometimes stuck forever

Hopefully these and other issues will get addressed when Cakewalk is no longer free. For now I realized I wasted lots of time with this and should have spent the $99 a lot sooner.

[Lord Tim]

Saying that this DAW is not being maintained is completely untrue. Another update is currently in the works, and a whole new evolution of the product line is about to be launched.

If these problems were as widespread as you say, there would be a lot more than one thread on here complaining about it, right?

The CTO replied to the post earlier - like the guy who knows the code and how it actually works rather than speculating - and had other opinions regarding what you're seeing.

If Studio One works for you, that's great - it's a fantastic bit of software and if the workflow suits you (which is the #1 thing people should be concerned about) then more power to you! But your experience isn't typical for most users here.

[Lord Tim]

5 hours ago, Quick Math said:

I might be experiencing something similar as well.

When I export my mixes, I often sense something's not quite right, like a change in the sound or a harshness.

This mainly happened on larger projects, and I started to notice it about a year ago, though I wasn't sure.

Just yesterday, I exported a WAV file for a project with many plugins, and again, the exported file didn't sound right.

And then I tried both multi-core and single-core settings for my exports multiple times and compared them.

The single-core versions sounded natural to me.

The harshness I could hear from multi-core rendered versions could be internal errors or something.

While this could be just my perception as the difference seems to be very subtle, I try the single-core setting.

 

Try this:

Export both versions, then load them into any DAW of your choice, Put one on Track 1, the other on Track 2, starting at exactly the same spot. Then flip the phase on one of the tracks.

Things like modulation and some convolution reverbs aside, which will change each mix no matter what, if any difference is well under -100dB then it's practically identical and you know it's your brain playing tricks on you.

[Quick Math]

1 hour ago, Lord Tim said:

Try this:

Export both versions, then load them into any DAW of your choice, Put one on Track 1, the other on Track 2, starting at exactly the same spot. Then flip the phase on one of the tracks.

Things like modulation and some convolution reverbs aside, which will change each mix no matter what, if any difference is well under -100dB then it's practically identical and you know it's your brain playing tricks on you.

Thanks, these randers didn't cancel out at all properly because there are modulation and also vst instruments in the project don't cancel out at all between any.  

Now, I tried blind tests multiple times with various combinations of 3 single core and 3 multi core renders. While there are differences as they don't cancel out and I sometimes preferred one over others on a certain section, while I pick different one on other sections.

I wasn't always picking single core versions among 3 tracks.

So, probably it was a wrong assumption that single core versions sound more natural.  I liked a certain sections of a certain version which happened to be single core versions when I was initially comparing them yesterday.

[Henrizzle]

It is unlikely these days that these 2 tracks would cancel out. Mainly because some plugins and the DAW as well is using random number generator to do a dither, rendering with slightly different bits on each pass.

Okay this is my third day on StudiOne, I'm convinced Cakewalk even when there are no oblivious artifacts has some issues. The reason is when I'm mixing the same tracks on StudioOne it just sounds smoother and more pleasant, making the mix go fast and easy. I can dial it in in a few hours, on Cakewalk it went day after day after day, into months, still doesn't sound 100% good.

[Henrizzle]

Another problem that leads to the unknown without seeing the source code. The dither has an issue. For example, if you record 24bit -> 24bit data, Cakewalk applies a dither on the data. Dither is intended to statistically yield detail in the bits that would otherwise be truncated. For example, if you are recording 24bit -> 16bit data there should be a dither to yield some detail in the 8bits of resolution you lost. So it seems either this feature is not well tested or the idea of dither is not properly understood by the software engineers doing the source code.

To see this simply record some silence from your 24 bit driver with the record data set to 24bits, blow up the track vertically. Next set the data to 32bits and you see there is no dither.

[Glenn Stanton]

the dither adds a noise component (not detail) to the LSBs (avoiding a "quantinization error" when truncating digital values) when you convert from (say) 24-bit to 16-bit (i.e. dropping the bottom 8 bits).  if you're doing final mix in prep for mastering - leave dithering off and send in same resolution you are mixing in. if you're trying to produce an MP3 (etc) directly off the mix, then pick one of the POW dithering options. note: the results will not be as good as say the Ozone or WaveLab etc dither options. S1 vs CW vs etc may likely have different dithering options and aliasing filters. 

it is interesting that no matter what we do - noise is always going to be part of audio 🙂 

some reference: What Is Dithering in Audio? (izotope.com) https://www.izotope.com/en/learn/what-is-dithering-in-audio.html

Edited August 20, 2023 by Glenn Stanton

[Will.]

Heres the thing: We sometimes forget to do oversampling in a plugin or virtual instrument on a track. If im correct here - ive watched a video on the latest release of STone where they said, the DAW does its own oversampling per track under the hood, where as CbB does this per project? 

The point is - it only takes one plugin to introdue these aliasing artifacts in the bounce/render stage. Sometimes oversampling are not needed, where you can fix this  with Lookahead if the plugin provides it. 

This can also be a wrong export setting issue, such as wrong buffer size, sample rate and dithering presets. If can also an import sample rate conversion of a sample that was used - even in a track bounce. 

[Henrizzle]

I just did another giant mix this morning in StudioOne, took about an hour and a half. Same mix as I have done in Cakewalk. In Cakewalk, it took weeks sometimes months because there seems to be this scratchy-ness that is mixed in and you can never get it to sound clear. I'm here to tell you, spend $99 on StudioOne, the cost of this free software is now your sanity. I'm outta here, no more Cakewalk madness for me. Good luck everyone and stop giving all the voodoo advise, the issue is plain and simple, this software is buggy and not maintained because it is free.