Physical Modeling Synthesis with household items (video)

[Chandler]

This is a video on how you can use MSoundfactory to create instruments such as xylophones using stuff you found around your house.  Keep in mind this is synthesis, not sampling, so you don't have to worry about wasting HD space or velocity layers.  Anyway I hope you find this useful and interesting.

 

 

[DaveClark]

Hi Chandler,

Thanks for this video.  The only problem I have with it is the title that includes "physical modeling."  This isn't physical modeling as those who actually do such things regard it.  Consider this from Stefan Bilbao's book (*):

"Physical modeling synthesis ... involves a physical description of the musical instrument as the starting point for algorithm design.  For most musical instruments, this will be a coupled set of partial differential equations, describing, for example, the displacement of a string, membrane, bar, or plate, or the motion of the air in a tube, etc."  He goes on to describe the correspondence between physical parts of an instrument and their representations in analogous locations in computer memory.  This latter is completely absent in the techniques you are illustrating.  The synthesis you are demonstrating is a frequency-domain operation.  I think it's great, but it's not physical modeling. 

Regards,

Dave Clark

(*) Stefan Bilbao.  Numerical Sound Synthesis: Finite Difference Schemes and Simulation in Musical Acoustics.  Wiley, 2009, page 8.

[Chandler]

Thanks Dave. This is actually physical modeling. This is a type called modal synthesis, which is commonly used in PM synths. In this case I didn't label things according to their physical analogues, but the same things are happening. You could label the resonance as "dampening" and the structure as "shape/material" if you like, but for this tutorial I chose not to as I feel its a waste of time. This is a more "under the hood" look at PM synthesis, but what is actually happening is the same.

[DaveClark]

Hi Chandler,

Sorry to have to say so, but it's neither modal synthesis nor physical modelling.  In the same reference as I cited above, modal synthesis is described on page 10.  The first step is to solve the eigenvalue problem for the system, considering the system of PDEs as well as "information about material properties and geometry, and associated boundary conditions." (Bilbao, page 10.)   You aren't doing that.  At about 4:30 in the video, you are moving the cursor about in a time-domain recording and getting very different representations in the frequency domain, then choosing one based on looks and sound.  You are leaving out the first step that involves detailed consideration of the physics of the system.

If you were doing physical modelling, you could reshape the object you were modelling and solve the new eigenvalue problem.  Your method requires you to get a new object, record the sound of it being hit by something in a way that is also not modelled at all, then fiddle around with the cursor to get some new representation of it in the frequency domain, construct a time-domain representation of it, then fiddle around with it some more.  You are also not physically modelling decay; you are imposing a decay function on top of the time-domain signal that would actually go on forever after your reconstruction of it from the frequency components, if not for a imposed decay.  You are also not modelling different sizes of a physical object to obtain different notes.  Instead you are manipulating the spectrum and/or the reconstructed signal with mathematical operations that often turn out to be poor approximations to physical changes.

Again, I think this stuff is great and I think people would find it useful, but also again, it's not physical modelling, nor modal synthesis as you are claiming.  You may be doing some of the latter steps of modal synthesis, but you are not starting at the beginning.  You left out the physics of the system.  Yes, some of what you do mathematically can be in part be justified by physical principles, but that doesn't make it physics.

Regards,

Dave Clark

PS: Modal synthesis involves use of a "shape matrix," projecting modal functions onto an observation state, does not involve temporal Fourier transforms, and generates the output waveform directly in the time domain.  I don't see any signs that you're doing any of that.  What you are doing looks to me like good ol' Fourier analysis followed by some editing, then constructing time-domain signals, then manipulating those.  Just regular DSP.  I would be interested to know if you are actually doing any of the steps of modal synthesis as Bilbao describes it.  In my own room modelling, I solve the PDEs in 3-D (wave equation in 3-D), obtaining the eigenfrequencies and amplitudes (in a statistical approximation sum), then switch to DSP to construct IRs.  Strictly speaking, this is not modal synthesis either, but I claim it is physical modelling.

[Chandler]

Hi Dave,

I think you’re confusing how physical objects are described mathematically and how physical modeling synthesis actually works. For something like a string of course it can be discribed using a mathematical formula, such as a PDE. In practice this isn’t how it is done using a computer. In practice something like the karplus-strong algorithm is done using combfilters. Modal synthesis is usually done using a filterbank of bandpass filters which is what I’ve done here.

I think you’re assuming this is some sort of additive synthesis. It isn’t and the signal can’t have an infinite decay because its generated through the excitation of filters not the generation of sine waves. Also the Fourier anaylsis isn’t generating the signal it only messures the frequency and amplitude of the partials. If you wanted to actually calculate these partials mathematically you enter them manually, you could certainly do that. IMO this is a waste of time for me and most people, so I skipped it. The material properties are contained in the analysis. Here is a paper discribing something similar to what I’m doing. https://pdfs.semanticscholar.org/4176/122b3826976473661130d7cb3a44bf667a8f.pdf

Of course you are right that I’m not correctly modeling every aspect of the physical object, however if you have to model every nuance perfectly and compute it in real-time then physical modeling doesn’t exist. If we’re going by that standard the karplus-strong algorithm isn’t physical modeling synthesis because it is modeling an ideal string instead of a real world physical object. Of course you’re free to have your own opinions on what constitutes PM and what doesn’t, but what is done in the video is consistent with what is generally known as modal synthesis and PM.

[DaveClark]

Hi Chandler,

Thanks for your reply and the reference which I have so far only briefly examined due to time constraints.  Before asking a simple question below that will probably clarify everything, at least for me, please allow me to make a few preliminary comments so that the question itself is clear.

To do physical modelling, one creates a mathematical or numerical analogy.  In the analogy there must be certain correspondences between the real object being modelled and the analogical model in order to qualify as a physical model.  These include:

1) Geometry, for example a mesh in the model.

2) Material parameters, e.g. the speed of sound in the relevant material.

3) Physical laws, e.g. the appropriate equations.

4) Boundary and initial conditions, e.g. the velocity of mesh points at the position of the strike at t = 0+.

The obvious advantage of true physical modelling, not something that is merely called “physical modelling” (*), is that one can ask the modelling person to directly adapt an existing model to other conditions, making the SAME CHANGES IN THE MODEL that one makes with real objects.  Take for example, your existing model of striking a glass with something and modify it in a fashion analogous to what one can do with real objects.

A second glass to be struck is half the height of the first glass, but has twice the value of all the diameters along its length.  The glassy material of the second glass has a speed of sound that is 15% higher than that of the first.  The second glass is struck 1.5 cm below the location of the strike of the first glass.  

Using a physical model, the modeller would be able to predict what this would sound like, even if not accurate due to incompleteness.  

Can you do that?

Please note that I’m not asking whether or not you can “by hook or crook” create an audio file that you think sounds like the situation described; I’m asking you if you can make the same analogous changes to the model of the glass being struck (geometry, material parameters, IC/BC) as you would to the real objects involved.  

If not, the model is not a physical model.  On the other hand, if MSoundFactory can do this, then your video has completely misled me about what MSoundFactory can do.  

I just viewed about half of it again, and I see absolutely nothing that convinces me that it can accomplish what I am asking.  In fact at 11:00 you discuss a glass harmonica.  Rather than physically model a finger rubbing a glass, you turn up the resonance of the modal filter!  In a physical model, you would, for example, disturb the mesh points of the boundary of the mesh and allow the physical model to behave like the real  object.  If MSoundFactory can do the latter, then your demonstration is highly misleading.

Thanks again, and I really do think that this stuff you are doing is great.  I looked briefly at Modalys and am interested in checking that out further when I have time.

Regards,
Dave

(*) Does calling a whale a fish make it true?  IMHO no.  If somebody calls a whale a fish, am I obligated to do so?  IMHO no.  Am I obligated to attempt to educate?  IMHO yes.

[msmcleod]

If you look at how Yamaha's VL synths work, they're really not that different from how MSoundFactory is being used in the video. 

In many ways, you're both correct here, but most of the implementations of Physical Model Synthesis I've seen don't normally give you access to the underlying algorithms. Instead, they offer you pre-canned models which you can build a more complex physical model with.

If you want to delve into "real" Physical Modelling, where you can go into the fine detail, then take a look at CSound.  I suspect however, you'll end up having to render your sounds offline if the math gets too computationally intensive.

[Chandler]

Hi Dave,

As for your question, the answer is no, although you could if you wanted to calculate those values yourself in another program or on paper.  If that is your personal definition of physical modeling synthesis, then this isn't it. AFAIK there is no physical modeling synth that does anywhere near what you're saying PM is. The majority PM synths don't calculate those things in real time and most of the time they don't calculate them at all. For example AAS Tassman, Chromaphone or String Studio don't do anything like what you're proposing. The SWAM strings are said to use PM synthesis, but you can't change the type of strings or increase the string length beyond the lowest note. The violin body isn't even modeled, it uses an IR.  These are still referred to as physical modeling, but if you don't think so that's fine. I guess you could call it resonance synthesis or something, but why make up a new word for something that is already understood. 

I appreciate your critique, but I feel like we're arguing over whether a cucumber is a fruit or a vegetable.  

[Starship Krupa]

Wow, you guys have gone into the stratosphere!

About 20 years ago I had a soundcard built around a Yamaha chip that had the Sondius XG  soundset built into it. Yamaha kind of dropped the ball with that, IMO, but I managed to find drivers and software that allowed me to access the voices and  articulations with the aftertouch on my keyboard controllers.

That stuff was some serious  modeling, especially with the woodwinds.

I'm not sure if Yamaha ever went further with it or if they let the licensing deal expire, but I thought it held a lot of promise just based on the audible results.

[abacab]

I'm still having enough trouble getting my head around FM synthesis.

[msmcleod]

4 hours ago, abacab said:

I'm still having enough trouble getting my head around FM synthesis.

These two articles were the breakthrough for me in FM programming:
 
Great article on how to approach programming an FM patch by focusing on each element of the sound:
http://msmcleod.co.uk/cakewalk/prgrmdx7.pdf
 
This page explores how to use modulator frequency ratios to create square wave, sawtooth etc:
http://web.archive.org/web/20170809204512/http://the-all.org:80/tx81z/fm_overview.html