Cakewalk notes events shifting issue

[Mark Bramnik]

Hi, I'm a newbie in a Cakewalk world, coming from Yamaha Home Arrangers. Recently I've started to learn how to create styles for these keyboards using a PC and thought about using Cakewalk for plain midi editing.

 I open a style in a specialized software and "strip" all the non-relevant things which are in use by Yamaha Keyboards (so called CASM section) - I end up with a plain midi file that I can open in Cakewalk. And here comes my issue:

Depending on a file content Cakewalk seems to "shift" notes by some arbitrary number of ticks, depending on the file. This happens when a file has a lot of controller/ system exclusive events. I literally see all notes shown with different ticks. I've asked on Yamaha arrangers forum and got a confirmation that this seems to be a bug in a Cakewalk itself.

Now, this never happens for "simple" midi files, only those "busy" with the aforementioned  data and the "gap" is always different from file to file. Usually I'm talking about 60-100 ticks  difference (at least thats what I've observed). I've tried to open these midi files in a "specialized" software (Style Magic, Mix Master) as well as in another DAW (Cubase, Ableton) and I haven't seen this issue.

So my question is whether this indeed is a known issue and if there is a way to submit an issue via a bug tracking system about that? 

 

Thanks a lot in advance,

Mark

Edited July 23, 2022 by Mark Bramnik

[msmcleod]

I suspect the events are being shifted to make room for the sysex events. IIRC large amounts of sysex events are spaced out so they don't flood the buffers with too much data at once.

Have you tried checking "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)  in preferences before loading the MIDI file?

[Bavi_H]

Could you provide or link to an example style file I could look at? (I can strip the CASM and following sections myself using a hex editor.)

Could you describe one example of a note that had its tick position shifted? What is the correct position you expect and the wrong position you're seeing?

Could you link to your question about this issue on Yamaha arrangers forum? I want to read what other people say about the issue.

 

I recently looked at a possibly similar issue where saving a MIDI file in Cakewalk caused all the notes to get moved 1 measure later. I found Cakewalk documentation that explains when Cakewalk sees a GM, GS, or XG Reset System Exclusive message at the beginning of the file, it will insert an initial "setup measure" when you save the MIDI file. You can add a line to a configuration file to disable this behavior.

[Mark Bramnik]

Hi Guys, First of all thanks for answering me and sorry for a delayed response, I wish I could dedicate to music more time Anyway I'll explain this issue as detailed as I can. 

So I've tried @msmcleod's answer - used Cakewalk with this flag - it has produced a wrong result. Here are step-by-step instructions that reproduces the issue

For example I took a file called BigRoom.T551.prs (attached). This is a Yamaha Factory Style from their last Arranger Genos - I've copied  it from the arranger itself "as is" and didn't make any changes in that file .  I've opened it in mixmaster tool (a specialized software that works  with Yamaha Styles) and checked where the first Markers (SFF2 and SInt) are:

 

So they're supposed to be at the first tick of the first bar (Yamaha arrangers have a resolution of 1920 ticks per quarter note).

Basically Markers are used to differentiate between the variations, but in this example they're just anchors which are "simple to track"/

Now From MixMaster I call Go To Function and it opens the file in the Cakewalk. Why Go To function? Because (in my understanding) Yamaha Style contains also a CASM section (Operational commands that the Keyboard can read and operate during the style playback - stuff like How to treat chord changes, etc). Since Cakewalk (as like as any other DAW) doesn't have any support for this kind of data, it automatically "strips" this CASM section and kind of splits the data into two different complemetary: the midi file and a CASM Data that doesn't get even loaded. It creates a temporary file called MMaster.sty and asks Cakewalk to load it instead of the original file. Then we can make changes in Cakewalk and save (again to the MMaster.sty), when we close Cakewalk, the MixMaster will merge back the edited data back with the CASM data. While the Cakewalk is opened, MixMaster "waits" in the background.

Ok, So I open Cakewalk with a GoTo Function and load the Markers View (Alt +Shift +4) - these markers have the correct 

time stamps:

Now I open a piano roll of 16th channel and move the first note I see one tone Up (this change is done just for the sake of illustration):

 

I save the file and close the cakewalk. At this point I get back to the Mix Master. 

And here the issue arises! When I open again the List View in the Mix Master - I see that the markers have been shifted (just like the other notes, again - the markers are easy to see because there is a relatively small amount of them) as you see it has been shifted 58 ticks right!

Given the fact that the only change I've done is changing 1 single note pitch, I don't understand why do we even have these kind of shifts in time.

 

 

A Link to MixMaster Program:

https://psrtutorial.com/MB/EV_Files/mixmaster.html

 

Thanks a lot for your time

BigRoom.T551.prs

Edited July 29, 2022 by Mark Bramnik
Attached Style File

[Bavi_H]

(This was a preview test post that I accidentally submitted.)

Edited July 30, 2022 by Bavi_H

[Bavi_H]

In general, when you open a MIDI file, Cakewalk seems to convert MIDI file events into various internal events and properties, and when you save as a MIDI file, it converts those internal items back into MIDI file events. This causes various MIDI file events to get rearranged or moved. I have found some settings that will improve the situation, but there are still events that get moved. Until we find workarounds (or Cakewalk is updated) to stop all the unwanted event movement, you might have to use other MIDI file editors.

Suggested settings so far:

• Uncheck "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)".
• Set GeneralMidiSMFs to 0.

UPDATE: See my later post for an additional setting to use.

More information

So far I have tested opening then saving the example MIDI file in Cakewalk, then looking at the MIDI file using other tools. I mainly looked at the events up to the first Note On event, investigating what Cakewalk changed, and trying to figure out why Cakewalk changed things, hoping to find workarounds to prevent unwanted changes. Here's what I know so far.

Cakewalk always makes the following changes:

• Changes the MIDI file resolution from 1920 ticks per quarter note to 960 ticks per quarter note.
• Events at the same tick position can be reordered.
• Modifies the tempo meta event from 472440 microseconds per quarter note to 472441 microseconds per quarter note (from 127.000254... quarter notes per minute to 126.999985... quarter notes per minute)
• Inserts a key signature meta event.
• Inserts a Data Entry LSB (CC38) event after every Data Entry MSB (CC6) event.
• Changes the Text meta event into a Marker meta event. (Cakewalk assumes any Text event in a Format 0 MIDI file is acting as a Marker.)
• Changes Note Off events (with any velocity) to Note On with velocity zero.

With the following options:

• "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)" option is checked
• GeneralMidiSMFs setting is 1

Cakewalk makes the following changes:

• All of the System Exclusive messages get moved to the beginning.
• The initial markers end up getting moved from 1:1:0 to 1:1:58 (in the 1920 ticks per quarter note resolution that MixMaster uses)
• The first note (on channel 10) gets moved from 2:1:0 to 3:1:0

With the following options:

• "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)" option is unchecked
• GeneralMidiSMFs setting is 0

Cakewalk makes the following changes:

• The first note (on channel 10) gets moved from 2:1:0 to 2:1:54 (in the 1920 ticks per quarter note resolution that MixMaster uses)

The movement of the first note from 2:1:0 to 2:1:54 seems to be related to the RPN messages (CC101, CC100, CC6, plus the CC38 that Cakewalk adds).

When Cakewalk opens a MIDI file, it combines the CC messages that make up an RPN command into one "RPN" event in its event list.
When Cakewalk saves a MIDI file, it splits an "RPN" event into four MIDI file events spread over 4 ticks:

"RPN" tick + 0: CC101 (Registered Parameter Number MSB)
"RPN" tick + 1: CC100 (Registered Parameter Number LSB)
"RPN" tick + 2: CC6 (Data Entry MSB)
"RPN" tick + 3: CC38 (Data Entry LSB)

If the CC events generated from the "RPN" event end up at a position that would overlap other events, Cakewalk normally tries to only adjust the overlapped events' positions to a minimum later position. But it sometimes seems to get confused, and shifts all the following events in the track by a constant number of ticks.

As a test, if I remove all the CC101, CC100, and CC6 events in the original MIDI file, then...

With the following options:

• "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)" option is unchecked
• GeneralMidiSMFs setting is 0

The first note (on channel 10) remains at 2:1:0.
 

Edited August 1, 2022 by Bavi_H

[Bavi_H]

On 7/23/2022 at 2:28 PM, Mark Bramnik said:

I've asked on Yamaha arrangers forum and got a confirmation that this seems to be a bug in a Cakewalk itself.

By the way, I was also interested in reading what other people said about this issue. Could you provide a link to your question about this issue on Yamaha arrangers forum?

[Mark Bramnik]

Hi Bavi_H, 

First of all, thanks a lot for your detailed answer. I really appreciate it. I'll have to understand what you've written in a first message because I'm a relatively newcomer in DAWs in general, but bottom line I think I'm suggesting to implement a setting in the Cakewalk that will activate a "non-intrusive" mode of operating the midi files that were already created somewhere else.  

From your answer I understand that such a setting doesn't exist, and this is an unfortunate situation. 

 

As for the arranger's forum. I've asked this question in the psrtutorial which is the best international forum for yamaha arrangers owners. The thread itself is available here:

https://www.psrtutorial.com/forum/index.php/topic,63571.0.html

Frankly I was hoping to get more answers here

Note that ckobu, a guy who runs an excellent youtube channel about advanced concepts in arrangers even made a video about this long time before I've discovered this issue by myself:

In the video he suggests to apply a series of "manual" conversions in the mixmaster after the modifications to the midi file that Cakewalk does. Probably he is familiar which which messages Cakewalk tends to add, and uses an another feature of Mix Master which allows to shift all the selected events to the fixed amount of ticks (specified in the UI of mix master). But to me it sounds as a workaround. I personally suggested to somehow automate this in the mix master itself - for example to memorize the position of mandatory markers, so that the program could calculate the "diff" but I'm not familiar with the code of MixMaster to estimate whether its possible or not. In addition probably it won't be the best possible solution because in future versions Cakewalk might start adding different messages so that it will be really hard to track how to resolve this "diff"...)

So all-in-all I believe a solution at the level of Cakewalk could be the best in this situation.

It could be great if someone from the Cakewalk Developement Team could answer whether it is something that might be considered for implementation or there is no chance that Cakewalk will implement something like this in future? 

In general MixMaster (which is by far the best known Yamaha Specialized Software) doesn't integrate with any DAW but the Cakewalk...  

Thanks a lot again,

Regards,

Mark

Edited July 30, 2022 by Mark Bramnik

[Bavi_H]

I found another setting that, together with the other settings I recommended in my previous post, will work around the problem. When you use these settings, the initial markers and the notes are no longer shifted. Be aware that Cakewalk still reorders and modifies other events, see the details in my earlier post. But using the following settings should make your style editing process useable without having to re-shift events back to their original positions.

Suggested settings:

• In Cakewalk Preferences, MIDI, Playback and Recording, uncheck "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)".
• In the TTSSEQ.ini [Options] section, set GeneralMidiSMFs to 0.
• In the TTSSEQ.ini [Options] section, set TranslateRPN to 0.

Detailed steps to change these settings:

1. In Cakewalk, go to the Edit menu and choose the Preferences command.
2. Under the MIDI category, click on "Playback and Recording".
3. At the bottom of the page, uncheck the box "Always Use Sysx Banks for MIDI Files (Instead of Sysx Data)", then click OK:
   
4. Close Cakewalk and open Notepad (or another text editor).
5. In Notepad, go to the File menu and choose the Open command.
6. In the File name box, put in the following name then click the Open button.
   %AppData%\Cakewalk\Cakewalk Core\TTSSEQ.ini
   Note: %AppData% is an environment variable that will take you to your App Data folder.
7. Under the [Options] section add the following lines:
   GeneralMidiSMFs=0
   TranslateRPN=0
   (If these lines already exist, change the values on the end to 0).
8. Save the file and close Notepad.
   

Now when you open your style MIDI file in Cakewalk and re-save it, the markers and notes are no longer shifted.

__________

Sources

I mentioned the source of the GeneralMidiSMFs setting in this previous thread: Saved MIDI file misaligned track starts. Specifically
from Special handling of GM, GS, and XG MIDI files:

Quote

When Cakewalk finds a GM, GS or XG reset message, it assumes that you would like to work within the GM Score Production guidelines. [...] If necessary, Cakewalk will insert blank setup measures at the start of your song.

and Cakewalk Help - TTSSEQ.INI:

Quote

Variables in the [Options] section [...]

GeneralMidiSMFs=<0 or 1> [...] This line controls whether or not Cakewalk creates a General MIDI setup measure when loading [actually when saving] Standard MIDI Files.

 

I read about the TranslateRPN setting in a PDF file called The Cakewalk Application Language Programming Guide for SONAR (Version 2.2. Febrary, 24, 2010). You can download it from the link in this post: Re: CAL manual or guide in PDF. In this document, section 5.9 "Details on RPN and NRPN Controller Events" describes how Cakewalk combines the CC events for RPN or NRPN commands into one event. On page 84 it says:

Quote

By adding a statement to the 'TTSSEQ.INI' file [...], we can instruct Cakewalk not to combine clusters of controller events into "RPN" and "NRPN" events. Open 'TTSSEQ.INI' in the Notepad editor and locate the heading [OPTIONS]. Under this heading add the line,

TranslateRPN=0

and save the file. Now restart Cakewalk and the "RPN" event will never darken your sequence again!

 

Edited August 1, 2022 by Bavi_H

[Mark Morgon-Shaw]

On 7/30/2022 at 8:30 PM, Mark Bramnik said:

So all-in-all I believe a solution at the level of Cakewalk could be the best in this situation.

Seems like a pretty niche/obscure issue - have you tried all the other DAWs ? How does Reaper handle it ?   

[Mark Bramnik]

Hi Guys,

First of all, @Bavi_H, thanks a lot for your answers. I'll definitely give them a try and will check on more styles, I'll also will try to use other modifications to test these settings and will update. 

@Mark Morgon-Shaw

I haven't really tried other DAWs I've tried Style Magic which is another specialized software for Yamaha arranger keyboards, and obviously it doesn't have issues like this.

In general, MixMaster integrates only with Cakewalk, and since it was free and "easy-to-understand" it has become an obvious choice for me (I'm not a musician and just study for my own joy and fun)... I'm not experienced with other types of DAWs - but I'll do my best to try and update.

In general the style handling in Yamaha arranger keyboards is not an easy and straightforward process, because unfortunately Yamaha do not provide any software to handle this task and the arrangers themselves do not have many on-board style modification options.

I can't estimate whether its a niche issue, to me it looks like there many people who own various yamaha keyboards (I believe its the most widespread company when it comes to arrangers), and some of them would like to create styles.  These people (including me) would be really glad to see a solution  for this issue, because IMHO, Cakewalk has the best midi editing capabilities - way better than anything available for Yamaha keyboards, of course its only my personal opinion...