RE: Convolve and TimeStretch Quarks

Hi Sam,

Thanks for this! Great work!

I tried for some time to figure out what would be the most convenient way (in the user perspective) to do basic NRT DSP on SuperCollider and it this is still a big question for me, e.g. use analyse/modify a soundfile using traditional DSP functions like those present on matlab and python (scipy, librosa).
As you said, the Signal DSP functions are way slower when compared with these frameworks. Moreover, using NRTserver is sometimes quite tricky due to the fact that it is an adaptation of the RT server, for instance, you can't write simple and short soundfile analysis code as you can do in python/matlab... in this approach you always need to think in RT execution translated to NRT world.

Is doing everything on the language side the most convenient alternative? What I am doing now is creating scripts in python and using SC to run them, but this is clumsy because you need to keep to IDEs open, debugging in the two... Would it be possible/desirable to ship SuperCollider along with these open source NRT DSP modules (python, C++) that are over the web to allow the client side usage?

All the best,
Fellipe Martins

From: [email protected]<mailto:[email protected]>
Sent: sexta-feira, 26 de fevereiro de 2021 07:40
To: [email protected]<mailto:[email protected]>
Subject: Re: [sc-users] Convolve and TimeStretch Quarks

It is a good question, and I will say I don’t know 100% of what I speak. What I do know is that the sclang implementations here are perfectly sample accurate. What I don’t know is if the server stuff isn’t, but it doesn’t sound as good, so I think that is the issue. Basically, the way both algorithms work is, they read a block audio from the SoundFile, compute the audio on the exact samples they are supposed to compute with, and write the data directly out to disk in the exact location where they go. For the TimeStretch, this means computing 520 ffts and 520 windowing envelopes based on correlation between frames per 32768 samples of audio output. By looking directly at sample blocks with Signal, I can guarantee that these are perfectly accurate.

FFT on the server is designed to deal with streams of audio, not blocks of samples. There is also an issue where you can only use sine and hann windows in the FFT and IFFT objects. JP Drecourt came up with a brilliant way around this with his PaulStretch implementation, and this is what my server side stuff is based on. The problem is, just using a window outside of the FFT and IFFT adds a “significant” amount of distortion to the output, because I don’t think things are totally lined up no matter how correct the code is. And thus, there is no way to get the crossfades completely correct when trying to compensate for uncorrelated frames. So, basically there are two kinds of jitter at play, which is probably minimal in most cases, but not so much in mine.

Doing this project revealed two things to me:

The language-side FFT that we use in Signal really needs to be updated where the Server side stuff is, as it is probably 4-8 times slower than it could be with newer libraries.

The Server side FFT could also use some love in two main categories:
1) allow for user defined windows
2) implementation of a Buffer-based FFT that deals with buffers instead of audio-streams.

I realize the best response to these thoughts is that I should probably do the work myself, as this is an open-source, user-supported environment. I agree, but sadly capitalism..ya know..

Sam


On Feb 25, 2021, at 8:53 PM, [email protected]<mailto:[email protected]> wrote:

Just curious what you mean by sample accurate convolution?

On Fri., Feb. 19, 2021, 4:47 a.m. , <[email protected]<mailto:[email protected]>> wrote:
Thanks Wouter. Uff. The old “changed how it worked and didn’t test it with a different duration” trick.

It should be fixed now. Tested on many different durMults.

Eli, thanks for that. I updated the help and actually gave TimeStretch2 a little tweak to place it in line with how TimeStretch functions. The silence and cut make sense. TS2 uses the server side fft, so in order to line up all the frames correctly, there has to be at least 65536 frames of silence at the beginning. The end of the track is a little trickier, and not perfect, but TimeStretch is really the way to go in all cases. I’ve spent a lot of time trying to get TS2 to sound like TS, but to no avail.

This is also true with the Convolve quark. The sample accurate convolution is a revelation. It is so clear and detailed compared to using something like ReaVerb with the same samples. It makes me want to only do NRT convolution reverbs in my mixes from now on…though I probably won’t.

Sam


On Feb 18, 2021, at 5:56 PM, [email protected]<mailto:[email protected]> wrote:

Sam, testing these out. Cool stuff, thanks for adding these quarks.

I noticed that all the examples in the TimeStretch2 help file involve the use of TimeStretch (rather than TimeStretch2), which looks like a mistake. TimeStretch.stretch(...) produces a "method not understood" error.

Otherwise, all seems to be working for me. Still tinkering with settings to hear the different results. TimeStretch2.stretch(...) produces a file that has a chunk of silence at the beginning (~3 sec), and the audio is a bit cut off at the end of the file. This might be from something I'm doing on my end, though.

Nice work!
Eli


_________________________
Eli Fieldsteel, DMA (he/him/his)
Director, University of Illinois Experimental Music Studios
Assistant Professor of Composition-Theory
Office: 217-300-0956
MB 4008, School of Music
1114 W Nevada Street
Urbana, IL 61801


On Thu, Feb 18, 2021 at 8:53 AM <flo.huth-HaK8Zq/[email protected]<mailto:flo.huth-HaK8Zq/[email protected]>> wrote:
thank you!
On 17.02.21 15:34, [email protected]<mailto:[email protected]> wrote:
Ah. I thought I could prevent download without preventing playback. I thought wrong. It is fixed now.

Sam


On Feb 17, 2021, at 3:05 PM, flo.huth-HaK8Zq/[email protected]<mailto:flo.huth-HaK8Zq/[email protected]> wrote:

hi,
im restricted by your google drive and cant download the file.
thx!
On 17.02.21 14:17, [email protected]<mailto:[email protected]> wrote:
Hey all,

I’ve added one quark of late and updated a second:

Major Update: TimeStretch

Super PaulStretch algorithm which separates the signal into 9 discrete "octaves", which are processed independently
Now runs completely in sclang, so it is very slow, but also sample accurate
On MacOS each channel can be processed in its own terminal window
Uses correlation analysis and custom windowing between frames to achieve a smooth stretch
Can time stretch resonant and transient elements with different settings, significantly improving the clarity of a stretch
Now works with files of any SampleRate
Python version by Alex Ness and Jem Altieri

Here is a little taste test with the Star Trek TNG theme song:

https://tinyurl.com/55ocdnm9

New Quark: Convolve

Some times you just want to convolve two sound files (and I have wanted to be able to do this in SC since I stopped using CSound in 2002). Uses the convolve method of the SignalBox Quark to convolve two files of any size. Runs completely in the language.

Let me know of any issues, and I hope these can help you make some covidtastic space jams.

Sam