To developers: possible BUG in Cakewalk!

[pulsewalk]

VSTi track automation data is linked relative to plugin .dll including its installation path, instead of relative to the plugin .dll itself. Why?

I discovered this during my problem described in this thread about orphaned automation envelopes.

Problem NOT solved even though project is saved as Cakewalk Bundle .cwb file.

Cakewalk PLEASE solve this, otherwise it will not be possible to do collaboration work on project files on different computers, unless the project receiving computer has an exact mirrored installation (of VSTi paths) of the "project sending"-computer, which I would think is not always the case or even many times or even often varies between users (not everyone put their VSTi installations in the same folders).

Please FIX this.

[azslow3]

Automation data are linked to plug-in parameters. I don't think plug-in installation path is ever stored in projects.

I suggest you to check you are using the same plug-in format (VST2 or VST3) on both computers. Cakewalk could automatically "replace" VST2 with VST3, in case VST3 is installed (and compatible with the procedure, at least from Cakewalk perspective). If such replacement happens, parameters list can be different and so existing automation data  can't be matched.

[pulsewalk]

2 hours ago, azslow3 said:

Automation data are linked to plug-in parameters. I don't think plug-in installation path is ever stored in projects.

I suggest you to check you are using the same plug-in format (VST2 or VST3) on both computers. Cakewalk could automatically "replace" VST2 with VST3, in case VST3 is installed (and compatible with the procedure, at least from Cakewalk perspective). If such replacement happens, parameters list can be different and so existing automation data  can't be matched.

Yes I use the same plug-in format and same 64-bits too. As a matter of fact, it was enough to just move the plugin .dll file to the same folder location as on the computer the project was created, and automation worked again. That indeed should mean that plug-in installation path is stored in projects, which is actually what I assume is a bug, because I can't think the developers would do something like that on purpose? I cannot see the logic in that, on the contrary, it would make project sharing and project transferring between computers/producers difficult and in same cases even impossible (say the initial computer where the project was created have a E: drive where the VSTi's are installed, and the project is sent to another producer that doesn't even have an E: drive but only a C drive or maybe only C + D, then what? having to create virtual drives or buy new harddrives and start reinstalling/moving VST's, and by that creating other problems instead and what not?).

Edited November 11, 2021 by pulsewalk

[Noel Borthwick]

It is not linked to the plugin dll and certainly not the install path. However there are some caveats with VST2 plugins. see below.

Plugins are assigned a 128 bit UUID when scanned which is used for all operations including loading plugins as well as automation.
For VST2 plugins the UUID is derived from the manufacturer assigned plugin ID as well as the plugin name (because ID's are not guaranteed to be unique). This is the legacy method used only for VST2. VST3 plugins already have unique UUID's .

For automation envelopes for VST2 plugins for legacy reasons the UUID utilizes the 8.3 plugin name. If you are loading a project on two machines and the 8.3 file name differs for the plugin you can run into orphaned envelopes. The most common reason for this is because the new machine has disabled 8.3 file names on the hard drive. This is a bad idea. You can check if there are different 8.3 file names on the two machines for the same plugin dll by running this from a dos box and replacing <plugin.dll> with the actual dll name.  If the dll has an 8.3 file name it will be listed. BOTH machines must have identical 8.3 names or automation will get orphaned potentially.

 dir /X  <plugin.dll>

Do this test and see if you find a difference with the plugins that are having orphaned envelopes. If there is no 8.3 name then you need to reenable support for it in the OS and re-install or copy the plugins again to let the OS generate them. This is not something we can change for VST2 without breaking compatibility with millions of projects.

This is a good reason to not use VST2 plugins when possible.

[Noel Borthwick]

2 hours ago, azslow3 said:

Automation data are linked to plug-in parameters. I don't think plug-in installation path is ever stored in projects.

I suggest you to check you are using the same plug-in format (VST2 or VST3) on both computers. Cakewalk could automatically "replace" VST2 with VST3, in case VST3 is installed (and compatible with the procedure, at least from Cakewalk perspective). If such replacement happens, parameters list can be different and so existing automation data  can't be matched.

This would be rare. We replace VST2 to VST3 based on rules provided by the VST3 spec. If a plugin vendor marks the plugin as being convertable then its unlkely they will change parameters because that would be silly.

[pulsewalk]

3 hours ago, Noel Borthwick said:

For automation envelopes for VST2 plugins for legacy reasons the UUID utilizes the 8.3 plugin name. If you are loading a project on two machines and the 8.3 file name differs for the plugin you can run into orphaned envelopes. The most common reason for this is because the new machine has disabled 8.3 file names on the hard drive. This is a bad idea. You can check if there are different 8.3 file names on the two machines for the same plugin dll by running this from a dos box and replacing <plugin.dll> with the actual dll name.  If the dll has an 8.3 file name it will be listed. BOTH machines must have identical 8.3 names or automation will get orphaned potentially.

 dir /X  <plugin.dll>

Do this test and see if you find a difference with the plugins that are having orphaned envelopes. If there is no 8.3 name then you need to reenable support for it in the OS and re-install or copy the plugins again to let the OS generate them. This is not something we can change for VST2 without breaking compatibility with millions of projects.

I've checked and there is actually a difference. On the first machine/computer 1 (where the project was created) the plugin dll does NOT have a 8.3 name. However, on the other newly installed computer (computer 2), the plugin dll DOES have a 8.3 file name.

Oddly enough, running: "fsutil 8dot3name query" gives a state 2: "The registry state is: 2 (Per volume setting - the default)." on BOTH machines on the same drive where the plugin dll is located.

I don't know how to strip a file from its 8.3 file name, but if that is possible, it might solve this problem for me. I'm not sure how more I could "reenable" 8.3 support as you suggest, as fsutil (obviously) states that 8.3 support is already a state 2 (with other words: enabled), on both machines on the drives where the plugin dll is located.

Edited November 11, 2021 by pulsewalk

[Xoo]

Try moving the dll out of the folder, check its 8.3 state in the new folder, then move it back?

[pulsewalk]

Alright. So just a quick update. On "computer 2" where I get the "orphaned automation envelopes"-problem, I get the problem when enabling the plugin dll which is on drive C. Now, this file actually have a 8.3 name too. However, when moving the same file to the D drive on the same computer, the 8.3 name disappears and thus the "orphaned automation envelopes"-problem also disappears.

Oddly enough, running: "fsutil 8dot3name query" gives a state 2: "The registry state is: 2 (Per volume setting - the default)." on BOTH the C and D drive!! I wonder why. 8.3 is enabled on both drives (not to mention also on computer 1), but still when moving the plugin dll to some locations the file loses its 8.3 name, and with that also creating the "orphaned automation envelopes"-problem.

I imagine I'm not the only one in the world having these problems, but now I at least know what could cause it and thus may be able correct it.

Edited November 11, 2021 by pulsewalk

[pulsewalk]

8 minutes ago, Kevin Perry said:

Try moving the dll out of the folder, check its 8.3 state in the new folder, then move it back?

Tried that, didn't work. It seems to be some central setting so the 8.3 file name is added no matter how I move it back and forth. And the same thing when moving it around in the other location, where the file does not have a 8.3 name, then it stays that way.

Another problem is however that it doesn't matter really what setting one choses 8.3 or not. If a Cakewalk project is already created with a VSTi with a certain dll file setting (either 8.3 on or off), that is it. Then I guess the entire project will have to be finished that way. I'm not sure if that could be corrected inside the project somehow. To somehow saving the automation data/project file. Then after that correct the 8.3 naming problem, and THEN continue on the project like nothing happened. I guess that's not possible.

So if the affected user have different projects using the different named (8.3 or not) plugin dll files, that user will have to continue to use the same file settings as long as he is working on the project and as long as he needs to reopen it.

That's bad news I'm afraid

Edited November 11, 2021 by pulsewalk

[Xoo]

46 minutes ago, pulsewalk said:

Alright. So just a quick update. On "computer 2" where I get the "orphaned automation envelopes"-problem, I get the problem when enabling the plugin dll which is on drive C. Now, this file actually have a 8.3 name too. However, when moving the same file to the D drive on the same computer, the 8.3 name disappears and thus the "orphaned automation envelopes"-problem also disappears.

Oddly enough, running: "fsutil 8dot3name query" gives a state 2: "The registry state is: 2 (Per volume setting - the default)." on BOTH the C and D drive!! I wonder why. 8.3 is enabled on both drives (not to mention also on computer 1), but still when moving the plugin dll to some locations the file loses its 8.3 name, and with that also creating the "orphaned automation envelopes"-problem.

I imagine I'm not the only one in the world having these problems, but now I at least know what could cause it and thus may be able correct it.

Default for non-system drives is off as of Windows 10 (I think).  So that makes sense.

[Xoo]

43 minutes ago, pulsewalk said:

Tried that, didn't work. It seems to be some central setting so the 8.3 file name is added no matter how I move it back and forth. And the same thing when moving it around in the other location, where the file does not have a 8.3 name, then it stays that way.

Another problem is however that it doesn't matter really what setting one choses 8.3 or not. If a Cakewalk project is already created with a VSTi with a certain dll file setting (either 8.3 on or off), that is it. Then I guess the entire project will have to be finished that way. I'm not sure if that could be corrected inside the project somehow. To somehow saving the automation data/project file. Then after that correct the 8.3 naming problem, and THEN continue on the project like nothing happened. I guess that's not possible.

So if the affected user have different projects using the different named (8.3 or not) plugin dll files, that user will have to continue to use the same file settings as long as he is working on the project and as long as he needs to reopen it.

That's bad news I'm afraid

I missed some steps - sorry!

1 - Move dll from current location

2 - Run VST scan in CbB

3 - Wait a bit (NTFS caches 8.3 names), maybe reboot

4 - Move dll back to old location

5 - Run VST scan in CbB

[Noel Borthwick]

1 hour ago, pulsewalk said:

Alright. So just a quick update. On "computer 2" where I get the "orphaned automation envelopes"-problem, I get the problem when enabling the plugin dll which is on drive C. Now, this file actually have a 8.3 name too. However, when moving the same file to the D drive on the same computer, the 8.3 name disappears and thus the "orphaned automation envelopes"-problem also disappears.

Oddly enough, running: "fsutil 8dot3name query" gives a state 2: "The registry state is: 2 (Per volume setting - the default)." on BOTH the C and D drive!! I wonder why. 8.3 is enabled on both drives (not to mention also on computer 1), but still when moving the plugin dll to some locations the file loses its 8.3 name, and with that also creating the "orphaned automation envelopes"-problem.

I imagine I'm not the only one in the world having these problems, but now I at least know what could cause it and thus may be able correct it.

Yes this is the unfortunate effect of 8.3 names with VST2. We have code to handle loading plugins irrespective of mismatches and we synthesize a better UUID that doesnt rely on short names for that. However for automation it still uses the old format for compatibility reasons. 
I'll think about it some more and see if there is a way to auto detect this and prevent envelopes getting orphaned in such scenarios but its not going to be easy.

[azslow3]

15 hours ago, Noel Borthwick said:

This would be rare. We replace VST2 to VST3 based on rules provided by the VST3 spec. If a plugin vendor marks the plugin as being convertable then its unlkely they will change parameters because that would be silly.

I have thought plug-in replacement is the only case when (instance) UUID+parameter (that is how I match automations when parsing projects) can be disrupted... 

10 hours ago, Noel Borthwick said:

Yes this is the unfortunate effect of 8.3 names with VST2. We have code to handle loading plugins irrespective of mismatches and we synthesize a better UUID that doesnt rely on short names for that. However for automation it still uses the old format for compatibility reasons. 
I'll think about it some more and see if there is a way to auto detect this and prevent envelopes getting orphaned in such scenarios but its not going to be easy.

I remembered changing dll name was not affecting project loading. And I have just checked again: after renaming TruePianos.dll into True.dll and re-scanning in Cakewalk, "new" dll was matched in the project and related automation was not orphaned.

[foldaway]

10 hours ago, Noel Borthwick said:

Yes this is the unfortunate effect of 8.3 names with VST2. We have code to handle loading plugins irrespective of mismatches and we synthesize a better UUID that doesnt rely on short names for that. However for automation it still uses the old format for compatibility reasons. 
I'll think about it some more and see if there is a way to auto detect this and prevent envelopes getting orphaned in such scenarios but its not going to be easy.

Hi Noel,

Would it be possible to implement a 'replace plugin' dialog for missing plugin IDs?  So that in the event that automatic fixup fails there is a mechanism by which
the user can select a replacement plugin to load in place of the missing plugin. I'm thinking specifically for VST2 plugins.

I've had situations where the same plugin .dll leads to a different UUID on another machine, meaning it cannot be loaded. I'm assuming this is related to the discussion above.
I've also had the situation where I've been unable to find the exact same version of a plugin used in an old project where but a later version (with bug fixes only) was available & these also have different UUIDs.  Not sure if the UUIDs include a hash of the file or if this is still related to the 8.3 name issue.

My thinking is that if a plugin has the same parameter layout & binary blob format then is should be able to load the project using that plugin & if it works, then this version of the project can be saved using the new UUID format avoiding the problem in the future.   In fact even if the replacement failed & led to a crash, I'd still be happy that I could at least try, rather than having to redo all the work from scratch.

[azslow3]

@foldaway from my experience,  feeding plug-ins with incompatible data can produce way more strange effects then just a crash... VST2 is identified by ID, not by file name. This ID supposed to be unique, registered by Steinberg. But who follow all rules... Unlike UUID, VST2 ID is short (4 characters) and so clashes are likely, forcing DAWs to use some (unspecified) method to distinguish physically different plug-ins with the same IDs.

As I have mentioned, it seems like Cakewalk in general match plug-in in the project with currently installed one. I mean it seems like 8.3 name issue is more glitch/bug then regular (also from OP and discussion, it happens for automations matching, not for plug-in matching). In case Cakewalk is unable to match some plug-in at all, most probably plug-in developer has done that on purpose and attempts to find a "back door" is not a good idea...  

[pulsewalk]

7 hours ago, azslow3 said:

I remembered changing dll name was not affecting project loading. And I have just checked again: after renaming TruePianos.dll into True.dll and re-scanning in Cakewalk, "new" dll was matched in the project and related automation was not orphaned.

Changing the plugin dll file name doesn't do anything here either. However, different 8.3 file names does. So this seems only be affecting the 8.3 file name part. Unfortunately, that cannot easilly be detected in Windows explorer by default and the user would need to have deeper knowledge about what to look for and how to check it. I guess many users have struggled with that and finally just scrapped the whole automation and built it up again. But that's not really an ideal option, so lets hope it can be fixed!

I'm not sure how other DAWs handle this but I've not experienced this before. I don't know if that is because other DAWs are handling the issue differently or because there simply have not been situations in my case with 8.3 file names being changed. At least now I know a workaround. Not the most ideal sort, but at least it works with not too much effort. It'll be a hell though, when I need to open other projects which will be looking for the other 8.3 file names, then it'll start all over again ?

Edited November 12, 2021 by pulsewalk

[foldaway]

3 hours ago, azslow3 said:

@foldaway from my experience,  feeding plug-ins with incompatible data can produce way more strange effects then just a crash... VST2 is identified by ID, not by file name. This ID supposed to be unique, registered by Steinberg. But who follow all rules... Unlike UUID, VST2 ID is short (4 characters) and so clashes are likely, forcing DAWs to use some (unspecified) method to distinguish physically different plug-ins with the same IDs.

As I have mentioned, it seems like Cakewalk in general match plug-in in the project with currently installed one. I mean it seems like 8.3 name issue is more glitch/bug then regular (also from OP and discussion, it happens for automations matching, not for plug-in matching). In case Cakewalk is unable to match some plug-in at all, most probably plug-in developer has done that on purpose and attempts to find a "back door" is not a good idea...  

I was mainly thinking of the issue where the plugin .dll is identical across machines but not recognised as idenitcal, meaning a project using the plugin wont  load correctly on one of the machines.  In this situation a replace dialog would be ideal.
From an implementation perspective, while certain safeguards could be put in place such as checking the VST ID, parameter list etc. I'm not sure it would be worth the time.  So I'd be happy with a 'this might go badly wrong, are you sure you want to try?' checkbox.
I wrote a VST plugin wrapper back in the Vista days, so I know what you mean about strange effects.  I had a lot of BSODs during development! but Windows is a lot more robust than it used to be.  Yeah, also familiar with the 4 chr id's that no-one bothered with!
As a side note, I seem to recall having similar issues with DX plugins.  ie. same plugin install on 2 machines but not recognised as being the same.

[azslow3]

I have tried to reproduce, but I have failed... put VST2 on a separate disk, played with 8.3 enable/disable, renaming long/short, etc. Every time Cakewalk was able to find the plug-in and its automations.

According to the (Windows) documentation, 8.3 related call just return 8.3 in case it exists and return original (possibly longer) name when not. I can only imagine there is no check when it returns more then 8.3 and that produce overflow (garbing something).

[Xoo]

You can break it by having 2 dll's with near identical long filenames that "collapse" to the same 8.3 filename (barring the number at the end: ~1, ~2 etc) and depending on the order you move the dll's into the scanned folder, the 8.3 filenames swap round.  Yes, I've had this happen, but it's easy to fix with some judicious file moving and renaming (I may have gone via a drive with 8.3 naming disabled on to ensure the file was "clean" - I forget).

[azslow3]

@Kevin Perry Thanks for the tip!

I still could not reproduce the issue. I have swapped ~1/~2 for 2 VSTs, so 8.3 names was swapped (while original long names kept). Cakewalk still could find everything...

VST scanning has reported 1 "new" plug-in and 0 removed (I don't think I have made any other changes, so expected 2/2). And so I can imagine how strange things can happened. UUIDs for synths in fact include 8 characters from 8.3 name + VST2 ID (not that I don't trust Noel, just wanted to check I also "see" them  ).

I link automations by other UUIDs, which do not include name/VST id, worked fine so far. And so the problem doesn't affect my code (that was my worry). But I am not Cakewalk...