Diagnosing plug-in compatibility problems and crashes in Cakewalk

[Noel Borthwick]

Some of the most common and complex problems that arise when using a DAW (digital audio workstation) are issues relating to plug-in compatibility.  At Cakewalk we take plug-in compatibility problems seriously and all reported issues are investigated. It is however impossible for us to test thousands of plug-ins in Cakewalk. While we follow best practices for VST compatibility, handling plug-in's can be complex since many vendor's do not test their plug-ins in all hosts, and there can be subtle or not so subtle (crash) behavior changes across plug-ins running in different host's. Before drawing conclusions on the cause of a plug-in related fault, its important to properly collect information that can assist with diagnostics.

If you encounter an issue with a plug-in here are some tips to diagnose it.

Identify the kind of problem. There are a few kinds of issues:

• Compatibility issues. While this can be wide its important to try and isolate whether the problem appears within the plugin or the host. e.g do you notice the problem in the plug-in UI, does it occur with only certain presets, does the issue occur when loading the project or when exporting audio, etc.
• Crashes. This is self explanatory but its critical to observe whether the crash report indicates that the crash in within the host program or within the plug-in DLL. The crash dialog will typically list the module that the crash occurs in in the title. If a crash occurs please save the Minidump for the crash since it is the only way for developers to diagnose the issue.
• Hangs. In a few rare cases a plug-in can cause the host to hang. If a hang occurs you can still capture a dump file while the app is hangling, though it may be a lot larger.
• Project specific problems. These are cases that result in unexpected behavior only with certain project files.

Diagnostics and action

If the plug-in crash is listed as happening within the plug-in DLL the first course of action is to contact the vendor of the plug-in. The reason for this is in many cases the the issue is caused by the plug-in running into an unexpected operation from the DAW that can only be fixed by the plug-in vendor. Every DAW is different and unless the plug-in has been specifically tested with a DAW there is a potential for unexpected bugs being discovered.

If the crash occurs in the host program (Cakewalk.exe is shown in the crash title) then one possibility is that the host ran into an unexpected condition. This should be reported to Cakewalk. It's important to note that not all crashes in the host are caused by the host. Plug-ins share the same memory space as the host program and as a result of buffer overwrites can corrupt memory, leading to random crashes or app shut-downs. 

In all problem cases the best way to report it is to provide the following details to all vendors concerned.

• Write a detailed description of the problem
• Include a zip file containing the project in which the problem occurs
• Include all minidump's if the plug-in causes a crash. Please read this for more information on how to capture a minidump

In some cases Cakewalk will assist and work with the plug-in vendor to get a resolution to a problem. If you send a problem report to a third party vendor you may also refer them to contact us through our support channel at support@cakewalk.com should they have technical questions or require internal follow up with us.

[AxlBrutality]

One thing I want to mention just in general is that there is often an unwillingness on the part of plugin manufacturers to dig into problems that involve Cakewalk, when we contact them directly.  I really much prefer to go directly to you/CbB and have you deal with them these days.

I've had Slate (and the various manufacturers they partner with), MeldaProductions, and JST all very flatly point the finger back at Cakewalk and refuse to dig further into issues, even without having been given much information.  It is what it is.  I'm not sure what y'all's standing with some of these manufacturers/publishers is, but it doesn't always seem to be good.

I don't like getting stuck in the middle of a three-way finger-pointing session (It's particularly frustrating when you guys can't figure something out or believe it's an issue on the plugin companies' end, but they all vehemently state that it's an issue on CbB's end), and ending up having to do all the diagnostics on my own and just throw it at everyone and hope someone can solve it. Particularly when it's not my job and I'm not being paid for my troubles.

As mentioned, there are some manufacturers that don't seem to hold Cakewalk (or the previous Sonar iterations) in high regard.  In fact, JST (a pretty big player in the scene now, especially among rock/metal engineers and musicians) has flat out recently told me to switch DAWs (not just for troubleshooting purposes, but permanently) and that they don't recommend Cakewalk.  (So at that point, I end up just getting frustrated/mad at everybody, which admittedly doesn't help.)

Lastly, there are a lot of instances where it's not easy to identify whether it's the plugin or the host, so often I push concerns to you guys by default (for the above reasons and more).  I realize coming to you guys with everything may be cumbersome, but it really is better and easier that way on our end...and I'm quite sure that less experienced/knowledgeable users don't know any better or have any idea what they're doing.  As CbB continues to be free and expand into new userbases, you're going to deal with that much more often.

 

[Noel Borthwick]

@AxlBrutality I fully understand this. We have excellent relationships with many plugin vendors. Waves, NI, Celemony, Arturia and Steinberg are some big guys who come to mind but there have been numerous others who are extremely responsive when there are compatibility issues.  On the other hand others make assumptions without doing the due diligence testing and debugging. Doing a test that compares a plugin in host A and inferring that host B is to blame because it doesn't work there is flawed logic. Software is far more complex than that and most competent developers would never make that assumption. 
Just in the 09 release I communicated with Steinberg about verified bugs in plugins state management that we have to work around. This is an ongoing problem and not something we can solve directly since none of us have any control over third party vendor timelines and policies. 

I can say with 100% certainty that when a vendor reports a specific compatibility issue to us we bump that to the highest priority to solve as soon as we can and work with them directly to get to the problem. The bottom line is if a plugin vendor asks you to switch DAW's without supplying evidence that it is actually a DAW issue, its really your call to decide the value proposition of whether you would rather stick with that developer or abandon your DAW. One root issue I have seen is that some plugin developers do not test on Windows systems and can be quick to dismiss plugin bugs as being host specific without actual verification. Some others regurgitate obsolete out of date information. Yes in the past we had compatibility issues but the vast majority of plugin issues have been resolved over the years. Our VST3 implementation is second to none (even by Steinberg's admission).

[Starship Krupa]

I wish I could recall the links to the posts and articles, but if you trust my credibility at all as a 58-year-old musician/business owner who spent most of the '90's in the software industry, in everything from startups to Adobe, I'll tell you that the situation with plug-in specifications and compatibility  is probably not what you imagine.

Even (especially) with the latest, VST3, if anyone thinks that Steinberg and other industry players (or even plug-in vendors) sat down and worked out the details of how VST's and DAW's should work together, then wrote out a spec, went over it checking for omissions and errors, and finally published it, then as errors and inconsistencies were uncovered by other companies and plug-in vendors, Steinberg was alerted and those changes were incorporated into revisions of the spec, then I must now inform  you that the Tooth Fairy and the Easter Bunny were actually just your parents leaving you money and candy.

It probably went more like:

Scene: Offices of Steinberg, 3 Months before Musikmesse 1996

Sales: "I think we can sell more copies of our new pluggy-inny Cubase if we let other people make plug-ins for it too."

Marketing: "Great idea! The new version with the plug-ins! Let's let everyone make plug-ins for it! We'll be legends, it'll be the new MIDI!"

Management: "Hey, people who chose coding and not technical writing as their career, give us the specification for the plug-ins so that other people can write them."

Engineering: "That's insane, we can barely make our own that don't crash the host, and we don't have it all written down in one place, it's in the form of comments in the code!"

Marketing: "Ha, ha,   you geeks have a great sense of humor. The spec must be ready for our presentation at Musikmesse. We're calling it VST for 'Virtual Studio Technology.' And don't worry, the outside vendors' plug-ins will all be certified by our QA process."

Engineering: "You're insane, Musikmesse is in 3 months and furthermore our QA staff can barely handle testing our own products with the resources they have."

Management: "Just have that spec ready for the presentation at Musikmesse. Even if you have to write it on a cocktail napkin and hand it to them, it will be ready."

A couple of years pass....

Scene: offices of an unidentified other DAW company

Other company's marketing: "Cubase got such a huge head start on us with that VST thing that it's become the friggin' standard, we have to make our host support VST's because people have these huge libraries of them. And plug-in houses don't want to code for DX."

Other company's management: "Hey, engineering, this is the VST spec from Steinberg, we will be showing off how our product can host every VST ever coded by anyone at Winter NAMM this year."

Other company's engineering: "This is a series of German swear words written on the back of a cocktail napkin from Steinberg's Musikmesse booth a couple of years ago and Winter NAMM  is in 2 months."

OCMgmt.: "Ha ha, you geeks have a great sense of humor. It's the full VST spec. Steinberg says so."

OCEng.: "Steinberg has no incentive for our host to be able to run VST's. It's the opposite. There's nothing in this about crash protection, memory management, preset management, default UI, installation location, sidechaining, UI scaling, they barely tell you how to get audio in and out. People will blame us when the things fail to load and/or bring our host down. We're at the mercy of Steinberg and the plug-in developers. It'll be a testing and support nightmare forever."

OCMgmt.: "This is the VST spec. We'll be showing off how we can host every VST on the market. I have faith in you."

(I made up the foregoing drama based on reading KVR and Wikipedia and my own experiences as a QA engineer testing a photo editor that was advertised as being  "compatible with Photoshop plug-ins." There is a financial incentive for the company who creates the spec to make it difficult for their competitors to use it. Cakewalk and  DP and Samplitude and FLStudio and Mixcraft get reps for being "buggy" and having compatibility problems, bingo, less competition for Cubase. Poor Mixcraft couldn't run a VST3 without crashing to save its life until the most recent version, 9, and Mixcraft is a stable program. Sampletank 3 never has been able to run for more than about 10 minutes without crashing in either CbB or MX. I'll bet it works a treat in Cubase.)

From reports, it didn't get any better with VST3. Maybe they added plugin-based sidechaining to the German swear words. From all I can gather, VST3 is the "New Coke" of plug-in formats, for those of you old enough to remember that marketing debacle/accidental success.

As for some plug-in vendors being dix, well, some of them are virtually one-person operations, and being good at coding does not necessarily, or even greater than 50% of the time, in my experience, go hand-in-hand with being good at dealing with other people. They're not all as personable as Noel! I am a one-person operation myself, and one of the reasons for that is I don't want other people telling me how to do things I feel passionate about. If you're even an  average coder, you can make a fortune working for any number of companies and not have to hassle with the things you have to deal with as a business owner, and have retirement and health benefits as well. So why have your own company?

Many engineers are, as they say these days "on the spectrum," and one of the things that goes along with that is difficulty in reading and conveying emotions. Being on the spectrum is often an advantage when it comes to the main work of coding, but a disadvantage when it comes to things like hashing things out with engineers at other companies. They can read to neurotypical people as rude and abrupt. They want to focus on the important thing (which would probably be the signal processing algorithm) and get annoyed by "peripheral" things like host compatibility.

Larger companies can hire people to act as a buffer between the coding talent and the rest of the world. I've had that role at a couple of companies. I felt like the "Jive Lady" in Airplane! "Stewardess, I speak geek." One-person operations don't have that luxury, so it's the world talking directly to the programming genius.

In the end, what works for ensuring compatibility is if the plug-in vendors feel it is worth their resources to help ensure it. Whenever possible, on forms or questionnaires, I let the vendors know I use Cakewalk. I put it in my sig on recording forums so people can see. I use Cakewalk and I buy plug-in licenses!

Edited December 5, 2019 by Starship Krupa

[Starise]

To be fair, I see this as a more or less minority problem of plugins for Cakewalk. Maybe I have been one of the lucky ones or maybe it's because I mostly buy plug ins known to be solid on many DAWS.  My system scanned over 1100 plugins and NO crashes. Some of them are 32 bit.

 As a non coder but literate computer user it would seem that X plug should connect to Z if X is the same or allows for some flexibility and if Z is the host DAW. I know it's a lot deeper than that and X seems to have many sub connections that can cause issues in different DAWS. 

The fact that VST works in Cakewalk as well as it does with most plug ins says a lot. Buying plug ins from one man operations where they don't test with the same ferocity that others do is certainly no fault of Cakewalk. Plug ins attached to one DAW have certainly been tested in that DAW, or so one would think.

[Starship Krupa]

On 12/19/2019 at 11:54 AM, Starise said:

To be fair, I see this as a more or less minority problem of plugins for Cakewalk. Maybe I have been one of the lucky ones or maybe it's because I mostly buy plug ins known to be solid on many DAWS.  My system scanned over 1100 plugins and NO crashes. Some of them are 32 bit.

This is how I see it and close to my experience as well. I see plug-ins and hosts as two different programs from two different companies, and if they work together, that's great, and if not, I'm not quick to start blaming one or the other. The bigger the companies involved, and the more commercial, the more I'll say, hey, work it out. Meldaproduction and Waves would have no business failing to work with Digital Performer and Samplitude. The users of all would have invested good money and headspace.

But I also understand that my perspective is not everyone's, and I "get" how someone who switches around between three different DAW's, and the only one that won't run the new plug-in they just dropped $50 on is Cakewalk would draw the conclusion that it's Cakewalk's problem and only Cakewalk's. In most other situations, I would draw the same conclusion.

That's why I wrote out my comedy "history of the VST spec." I once experienced firsthand the challenges of developing a plug-in host that was not the original host the plug-ins were written for. VST was never created as an industry standard the way that for instance, MIDI was. It was (and remains) a proprietary Cubase add-on that got released into the wild. If they were called "Cubase" plug-ins instead of "VST's," people might be more understanding.

"This Cubase3 plug-in isn't working in Cakewalk!" Well, it's great that any of them work at all, considering that they're CUBASE plug-ins....

8 hours ago, pwalpwal said:

(old) feature request - sandbox plugins so if they crash they don't crash cakewalk

That is a fine idea in theory, but as Noel has pointed out, and you probably know too, there is no way to do this without slowing things down (adding latency at the plug-in level, which might be cumulative, adding to the time it takes to add a VST to a track, etc.). A step in the right direction was to allow sandboxing at scan time, so at least they don't crash or hang that process. It's a tradeoff that I guess they don't want to make. If people turn on sandboxing, perhaps it will get unacceptably sluggish. And the user still ends up not being able to use their plug-in.

And still, no matter how much sandboxing you do, a program inside a program can still probably find a way to make the host program fall down and go boom if it tries hard enough. ?

From my armchair, I like the approach of making CbB more bomb-resistant when a plug-in is found to crash it. That achieves the goal of compatibility as well as stability. After all, there is truth in the statement that if REAPER can run the plug-in then so can Cakewalk. It might indeed be a poorly-coded plug-in, but the fact that it runs without crashing in other hosts suggests that it's possible to "code around it" as the expression goes.

[Noel Borthwick]

All Sandboxing does is crash a different process  We've experimented with sandboxing - after all bitbridge is exactly that, except it only sandboxes 32 bit plugins. 
Doing it well is a very complex endeavor and potentially introduces a different set of problems. Workarounds are presently the only way most DAW's make plugins work but it takes cooperation and a willingness to work together between vendors.

[bitflipper]

Interoptability challenges can be a nightmare for developers, including the legions of folks who make Windows itself. And it's been that way for as long as I've been in the business, a span that predates not only VSTs and DAWs but even Windows, TCP/IP and the World Wide Web.

We take it for granted that you can watch Netflix on your phone or embed a picture of your cat in an email, but most users are unaware of the years of late night hair-pulling sessions and negotiations that ultimately birthed such technological miracles.

A great many musicians and DAW users are also software creators. You'll find the makings of a band in any office full of coders (as evidenced by Mr. Borthwick, who is himself a great picker). Consequently, many Cakewalk users and forum participants are also propellerheads, geeks and bit-flippers. And across that particular subset of CbB users you will find unanimous respect for Cakewalk, its processes, and its dedication to quality.

That should say something to the less-informed masses who continually whine about the most trivial bumps in the road. But sadly, it doesn't seem to.

 

 

 

[Ephraim Isaac]

It's unfortunate devs aren't always willing to cooperate.

A while back I made this post:

Can anybody here check this out?
 

Edited January 24 by Ephraim Isaac

[Robert Bone]

I am experiencing Cakewalk crashes any time I open the Arturia Modular V2 synth UI. It appears to be an access violation, and occurs when inserting into both existing projects where it used to work, and it crashes if inserted and its UI is opened in a new project. Modular V2 is a VST2 plugin. Its V3 version is a VST3 and opens. I have an existing project with the V2 VST2 version, that I need to see what preset is loaded, so I can swap in the V3 version which works.

 

A frustrated sad Bob Bone.

[sjoens]

My issue with Arturia was solved by a fresh install of Windows... not what you want to hear, but they worked one day and not the next, so Windows (or me) did something to cause it.  Several issues were cropping up over time so it was needed.  A week getting it all back up and running and now everything DAW and otherwise seems to work a lot better.

[Robert Bone]

UPDATE on my Arturia Modular V2 VST2 plugin crashing Cakewalk.

It turned out to be related to licensing, and is resolved.

I had created a Cakewalk project several years back, on a different PC, that had Arturia V Collection 3 installed, which included the Modular V2 VST2 plugin.  Since everything was properly licensed and activated, Modular V2 worked just fine on that PC.

That PC later died - the motherboard went belly up.  When I ported everything over to my laptop, that included a bunch of VSTi plugins, which included Modular V2.  What I didn't realize at the time (3:30 AM), was that I had to activate Arturia V Collection 3 on the laptop, to have that Modular V license properly ready for use.

It was only because I had opened up the old Cakewalk project from the now dead PC, that Modular V2 was loaded, and because it was not activated on the laptop, when I attempted to open the Modular V2 UI in the Cakewalk project, it purposefully killed itself with an Access Violation, which then crashed Cakewalk.

I have subsequently Activated Arturia V Collection 3 on the laptop, and reinstalled Modular V V2 from the V Collection 3, and it now happily works, in stand-alone mode and as a VSTi plugin in Cakewalk projects.

A SUGGESTION:  The above experience DID cause me to realize something that could be useful to anybody.  One of the things I tried to do was to open up the Modular V V2 plugin to see what preset I had loaded into it when it was working.  There is currently no way to determine that, if the plugin can not be opened for whatever reason.  SOOOOOO - moving forward - I will dilegently maintain a list in a text or Word document, of each preset loaded into each plugin, for each project, so that if at some point a plugin breaks and no longer can be opened, at least I will know what preset had been loaded when that plugin was working, which may help me aproximate the sound using a newer version of the same plugin, or give me some clue if I have to recreate that sound from some other preset or from scratch.

Bob Bone

[sjoens]

On 12/21/2019 at 7:17 AM, bitflipper said:

Interoptability

Easily confused with Interruptibility.  

[bitflipper]

You waited nine months to make that joke? 

[sjoens]

... something interrupted my train of thought.