FR: AI tempo recognition and quantize

[Teegarden]

I don't know if it already exists, but with the new wave of AI tools in audio I could imagine that the following would be a next logical option for a DAW which might not be so difficult to achieve (AFAIK mainly feeding enough source material in such tool to let it recognise patterns like they do with AI photo software):

• A tempo recognition option that scans the audio, and that identifies the main tempo as well as tempo changes of the main song as well as tempo/rhythm detection of solos. The latter especially, since there can be many time variations in solos
• Than the option to quantize (in particular handy for an instrument solo or voice line) according to what the quantize tool suggests based on music style, or any value between a hard quantize and human feel whereby all (unexpected) tempos of the notes are quantized in a way the musician intended with just one push of a button

Edited October 15, 2022 by Teegarden

[Byron Dickens]

Both are well documented.

http://www.cakewalk.com/Documentation/Home

[Teegarden]

12 hours ago, Byron Dickens said:

Both are well documented.

http://www.cakewalk.com/Documentation/Home

Yes, that's exactly what I would  like to see improved. Currently you have to check what you've been doing, set parameters accordingly and experiment until you finally have a satisfying result. And yes, I know you can make your own presets (and of course play more accurate, just saying before you come up with that?) etc. 

I want a one push of a button that automatically takes all the work out of my hands, just like e.g. the latest Luminar Neo does with a photo (or IZotope Neutron 4 does with a mix to get you much faster where you want to be). If you don't like the result you can very quickly adjust to your liking. Much faster than the traditional way of improving things.
Everything that speeds up my recording process is more than welcome... I know all the basics but never dreamed of becoming a studio engineer, I prefer to put most of my time in composing and playing music.

 

[Will.]

1 hour ago, Teegarden said:

I want a one push of a button that automatically takes all the work out of my hands

?? Well, you can always wait a 100 years before you make music. ??

AI coding will never EVER! be accurate. I've been in studios where they use sonible plugins and yet, always have to correct it. The human ears will always have an advantage over AI. Keep in mind: all these algorithms that these things use, are also just based on a "humans" knowledge on spectrum frequencies, and dynamic ranges. So, human error in these codes are inevitible. 

Every mic, vocal, guitar, keyboard, studio, reflection, accoustics, dynamics and locations are different - irrespective on whether it is the same song being played over and over. Every space you get is different. Meaning it will never give you an accurate result. 

It's the same with two producers in the same studio, working on the same song, using the same equipment. One will do a better job than the other, because we hear details better. So, an AI system will NEVER be as great. Music requires and demand human expressions. 

Though: I guess, there are maybe certain aspects where it might (using it losely) work for some people. After all theres no right or wrong way in music production. If it works for you - i mean, you're using it. 

Just my little humble opinion. 

Edited October 16, 2022 by Will.

[Teegarden]

@Will. I never said they are accurate. The idea is that they learn to interpret human error and sounds and the way they are being mixed and mastered in different styles by the great producers and implement that on a piece of music that you feed it to give you a better start point.

Plenty of professionals agree that AI tools do make life easier (and get better every day), even when you (like I indicated before) want/need to adjust things.
Main advantage is that you can start much closer to a desired end result. Just some quick tweaking often does the job.

As for the human error: as long as it is not to outrageous it can be a much desired thing. Without it everything would be (too) clean/synthetic. Just like many prefer analog modelled processors instead of pure digital ones, the analog distortion makes sounds warmer, more natural. 

What doesn't work for you doesn't mean that it doesn't work for others... For me and many others it certainly does work and with AI tools I usually get results that also professionals appreciate and they save me lots of time.

Anyway, we fully agree that (studio) acoustics and the ears are ultimately the crucial factors. That doesn't change the fact that AI tools can speed up things.  

So I definitely would like to see AI implementation all over the place (with quickly adjustable parameters for fine tuning, as the top AI software usually provides). 

 

NOTE: it could be an extra option in a menu, so no one would have to sacrifice his preferred "old-school" workflow 

[Will.]

46 minutes ago, Teegarden said:

Plenty of professionals agree that AI tools do make life easier 

Not to start any debate. I guess we all interpret"professionals" differently. 

1 hour ago, Teegarden said:

 I definitely would like to see AI implementation all over the place.

I dont.

[Teegarden]

Professionals IMHO: studio engineers that work in and/or own a sound studio, musicians that live well from making music, photographers that live well from photography, people that are being rereferred to in the field as experts,... Please let's not try to outsmart each other 

 

OK, I don't need or want a sampler at all.

You like to have one, so I support that because it is important to you and doesn't interfere with my workflow.. 

Would be nice if you could approach things that way as well...

 

Edited October 16, 2022 by Teegarden

[Byron Dickens]

We're a long way from having a one-button "suck every last drop of life out of my music"  feature.

 

 

Thank God.

 

Edited October 16, 2022 by Byron Dickens

[Will.]

21 hours ago, Teegarden said:

OK, I don't need or want a sampler at all.

LOL! Dude, come on - your feelings.

You want something that every user would not have an option to. You will have an option to not use the built-in sampler. I personally dont want to work with AI. You can use a sampler with AI tempo in it too. See . . .  options. 

FWIW: I work everyday with music. I work in a "professional" roam. I go the whole nine yards. I travel overseas. Thats why I said, we interpret the word "professional" differently. In todays world of music the word "professional" does not exist anymore. I even keep air quoting the word losely. 

You get 13 year old teenagers that mix better than most legends/icons of today. That's what i'm trying to tell you. Hits are being created on just a laptop and mouse with 4inch monitors. These rascals make millions - way more than what "professionals" make these days. 

So tell me what it means to be a "professional" these days in music. 

FWIW: Im not killing your request. I'm mainly just pointing out some real-life issues that comes with it.

Edited October 17, 2022 by Will.
Typos.

[Xoo]

Some questions...

- What would the AI do that the current (well-defined) algorithms don't achieve?

- Who would do the training of the AI?

- Who would build the inference models for the AI (this isn't traditional DSP-type knowledge needed)?

- How well would it run on a current/future processing (look at the specialised AI triaining/inference processors that Intel and nVidia make and compare to a normal CPU)?

Edited October 16, 2022 by Kevin Perry
Missed out a don't

[Promidi]

7 hours ago, Teegarden said:

So I definitely would like to see AI implementation all over the place 

Hell to the no!

[Byron Dickens]

 

[Teegarden]

20 hours ago, Byron Dickens said:

We're a long way from having a one-button "suck every last drop of life out of my music"  feature.

If well implemented the aim is to do the exact opposite of this...Hence the option to tweak afterwards to your personal liking...
Good models are better at pattern recognition and identifying details than humans, see some examples below in the post.

I know the possibilities of these new technologies well as well as the limitations, already years ago I initiated an international machine learning project which now is also supported by the EU. 

 

18 hours ago, Kevin Perry said:

Some questions...

- What would the AI do that the current (well-defined) algorithms don't achieve?

- Who would do the training of the AI?

- Who would build the inference models for the AI (this isn't traditional DSP-type knowledge needed)?

- How well would it run on a current/future processing (look at the specialised AI triaining/inference processors that Intel and nVidia make and compare to a normal CPU)?

1. Again: speed things up.  Right now I need to check the tempos and selected the kind of measure etc. Also, I need to experiment with the settings to get the desired human feel and with complicated solos you need to select parts with entirely different notes and speed to treat separately (in my experience, but maybe I'm missing something here). A well defined model would do most of it en would give me some fast parameters that I could use to make it perfect.
   
   I do admit that the work needed for my proposed feature might be too much in compared to the potential benefit. On the other hand, I could see it being developed along with several other deep learning features aimed at different parts of the audio enhancement all from the same feed used to train the models.
   There are probably many other DAW procedures that could benefit from AI solutions as well.
    
2. The training: good question. I don't know the exact scales and possibilities for and limitations of the bakers. There are many free AI systems being developed, so I could imagine perhaps partnering with one of these developers. Perhaps with GPU audio (see point 3).
    
3. I guess it could run extremely well if you implement technologies like developed by GPU audio. They are also aiming at machine learning implementations and are IMHO an important direction for the future of audio technology. They’ve now forged official partnerships with AMD and NVIDIA – the latter of which recently hosted inundated GPU Audio workshops at their Deep Learning Institute. The focus of these workshops is for developers to get their hands on GPU Audio’s groundbreaking software in readiness for an upcoming SDK release.

Software like iZotope is being used by many here on the forum. Part of it makes use of AI implementations. It usually runs well om modern computers AFAIK.

@Will. I guess from all you wrote you don't use iZotope's AI tools. Well, I just love that kind of tools and there clearly is a market or it. My feature request is intended to be seen as from a same perspective as how Neutron, RX, and Ozone implement it.
 

The point you guys seem to miss is the following:

• AI is intended to give you a starting point closer to where you usually go for a finished project
• It just saves time, you will most of the time need to tweak/fine-tune once a model has provided a suggestion
• Suggestions are not supposed to be static, just like with human error each time a slight variation will be presented even if you didn't change any parameter or piece of the audio.  

All I asked for is extra options, not replacing existing ones. You don't need to see the benefit, as long as others like me do see it. It's personal preference. You use it or you don't use it. Everything is fine.

I've seen AI (a misplaced term, it is machine learning - and its subset deep learning) being implemented around me in many different fields. The systems get better quickly and can take some (often boring, needlessly time consuming ) work out of your hands and depending on the application are sometimes already better than humans e.g. diagnosing diseases. Not just audio-visual implementations, but robotics, drug development etc.

I wouldn't come up with the suggestion if I wouldn't have had some experience myself with the implementation of these models and know the exceptional possibilities  (and limitations) if done well...

Some examples to illustrate how entirely different fields benefit from the latest developments:

•  AI peer reviewers unleashed to ease publishing grind “It doesn’t replace editorial judgement but, by God, it makes it easier,”
• Lip reading: Lipnet—deep learning network created by Oxford University with funding from Alphabet’s DeepMind— has achieved a 93% success in reading people's lips. The best of human lip readers have only a 52% success rate. A team at the University of Washington used lip sync to create a system that adds synthetic audio to an existing video.
• Diagnosing diseases:
  1) AI surpasses humans at diagnosing diseases: deep learning algorithms can correctly detect disease in 87% of cases, compared to 86% achieved by health-care professionals. The ability to accurately exclude patients who don’t have disease was also similar for deep learning algorithms (93% specificity) compared to health-care professionals (91%).
  2) IBM Watson’s accuracy rate for lung cancer is 90%, compared to a mere 50% of human physicians.
• Transcribing audio: Microsoft AI Beats Humans at Speech Recognition. The AI system had an error rate of 5.9 %, comparable to that of human transcribers employed by Microsoft. When researchers repeated the test, its error rate was 11.1%. This was virtually on par with the human result of 11.3 percent. 
• AI can be creative and make unique pieces: 
  1) Critics of AI nauseatingly argue that machines could never be creative (HMMMM, sounds familiar...?), or curious, or discover anything of significance — because they lack consciousness. Nevertheless, a team at Tufts have proved naysayers wrong. Intelligence does not need consciousness to discover new knowledge. By combining genetic algorithms with genetic pathway simulation the researchers created a system that was able to make the first scientific theory to be discovered by an AI: of how flatworms (or the species “planaria” to the initiated) regenerate body parts. The AI-generated theory will have a significant impact in human regenerative medicine.
  2) An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy. I love this picture, it could just as well have been done by an artist. They could have fooled me.  

 

 

  

[Will.]

22 minutes ago, Teegarden said:

Hence the option to tweak afterwards to your personal liking...

And the winner is . . . 

This beats the purpose of AI tools - doesnt it? So why bother with it?  

22 minutes ago, Teegarden said:

@Will. I guess you don't use AI tools. 

I thought this was perfectly clear in my first post. Also do not assume things. All the answers to your assumptions, has already been given. 

If you like working with presets, thats cool and all. Its basically what AI tools are "Live|Dynamically Presets" 

Edited October 17, 2022 by Will.

[Teegarden]

1 hour ago, Will. said:

This beats the purpose of AI tools - doesnt it? So why bother with it?  

No, not at al..............You simply ignore the main fact which I tried to explain in several ways which is that they save time, they help you get to your goal faster.  

I had hoped for a constructive discussion here with likeminded musicians that appreciate the latest innovation trends and like to see some of it implemented in their DAW.

Why waste time on a blog to keep repeating you don't like/want something which anyway will not get in the way of your workflow at all. If you don't appreciate it, why not just explain once why you are not interested and leave it at that. Coming back to it doesn't help anyone. At least @Kevin Perry asked some constructive questions.

Edited October 17, 2022 by Teegarden

[Bruno de Souza Lino]

On 10/16/2022 at 7:14 PM, Kevin Perry said:

Some questions...

- What would the AI do that the current (well-defined) algorithms don't achieve?

- Who would do the training of the AI?

- Who would build the inference models for the AI (this isn't traditional DSP-type knowledge needed)?

- How well would it run on a current/future processing (look at the specialised AI triaining/inference processors that Intel and nVidia make and compare to a normal CPU)?

Bandlab already does that in their online DAW. Drag any audio of bunch of audio there and the software figures which tempo and key it is. It's not perfect, but it's a start.

[Teegarden]

Thanks! Now in CbB please, so I don't have to drag? (Haven't tested it yet, so don't know if it's good enough). I've seen some other plugins try to achieve similar things well.

Sad thing is that many people don't understand the technology, they look at some not yet well implemented AI examples and decide it will bring nothing. 
They also seem to think that I will not save time and that you only depend on the decision the AI model makes, while a good implementation let's you craft the result in a way that it is the same or better than you would have achieved with your standard tools, while doing everything faster. 

AI is unstoppable and unescapable, they use it in most high tech sectors already to quickly find new drugs, new sophisticated batteries and so on.  There are exciting recent scientific breakthroughs that would never have been possible without AI.

The only question is, do you want to stick with your old school methods and being overtaken by the rest of the world (other DAWs are already experimenting with AI implementations at different levels), or do you want to try to see if you can have things improved maybe in a better way than the competition does (which means that you need to start experimenting).

[Glenn Stanton]

i'm a fan of AI - so long as it works offline as well as online... i like the idea of letting computers do a lot of work for me....

[marcL]

On 10/17/2022 at 8:52 PM, Teegarden said:

Again: speed things up.  Right now I need to check the tempos and selected the kind of measure etc. Also, I need to experiment with the settings to get the desired human feel and with complicated solos you need to select parts with entirely different notes and speed to treat separately (in my experience, but maybe I'm missing something here). A well defined model would do most of it en would give me some fast parameters that I could use to make it perfect.

IMHO the target is much to sophisticated!

Why is it so difficult to create a real drum part for a song with a VSTi that sounds so realistic as played by a human drummer? Me and 10000s of musicians tried to achieve this for years and I think it is nearly impossible to get the same feel! A drummer has not only "one feel" that is represented by a swing value and some random deviations! He delays/forwards the kick, snare, cymbals in the beat differently in every song, maybe even song part, often he has a different swing with the cymbals, ... And this is only the drummer! If you look at the tempo of a song by a whole band it gets even more fascinating! ?

Your AI software would have to recognize all this! If you have Melodyne, then just make some tests how it recognizes (or rather NOT) the tempo!

[marcL]

 

4 hours ago, Teegarden said:

Sad thing is that many people don't understand the technology, they look at some not yet well implemented AI examples and decide it will bring nothing. 

That's a fact that most AI (new) technology brings nothing but new problems. The positive examples are the exception. Just look at the last 10 years of software development and you can see a lot of trouble.

4 hours ago, Teegarden said:

that you only depend on the decision the AI model makes, while a good implementation let's you craft the result

If I look at the software that I use, then IMHO most software does not meet your "good implementation"! There are so many bad examples not only in music software!

4 hours ago, Teegarden said:

AI is unstoppable and unescapable,

That's true, unfortunately! Just if I compare news articles and private messages (e-mails, phone, ...) before and after the introduction of spell checking, then I am afraid of AI development!

Other example in aviation: If the flight situation is challenging, then often the pilotes have to go to manual control and this gets more and more difficult, because they lack routine caused by the AI (flight system) handling the usual cases. There are so many examples of catastrophies caused by technical analysis faults.

4 hours ago, Teegarden said:

The only question is, do you want to stick with your old school methods and being overtaken by the rest of the world

No, I do not reject AI generally! If something is really mature, then it may help (I would also like your tempo recognition/quantization, but I doubt it will be available in my life). I just think some healthful skepticism is better than just a blue-eyed view of this topic. ?