Re: Convolve and TimeStretch Quarks

It is a good question, and I will say I don’t know 100% of what I speak. What I do know is that the sclang implementations here are perfectly sample accurate. What I don’t know is if the server stuff isn’t, but it doesn’t sound as good, so I think that is the issue. Basically, the way both algorithms work is, they read a block audio from the SoundFile, compute the audio on the exact samples they are supposed to compute with, and write the data directly out to disk in the exact location where they go. For the TimeStretch, this means computing 520 ffts and 520 windowing envelopes based on correlation between frames per 32768 samples of audio output. By looking directly at sample blocks with Signal, I can guarantee that these are perfectly accurate. 

FFT on the server is designed to deal with streams of audio, not blocks of samples. There is also an issue where you can only use sine and hann windows in the FFT and IFFT objects. JP Drecourt came up with a brilliant way around this with his PaulStretch implementation, and this is what my server side stuff is based on. The problem is, just using a window outside of the FFT and IFFT adds a “significant” amount of distortion to the output, because I don’t think things are totally lined up no matter how correct the code is. And thus, there is no way to get the crossfades completely correct when trying to compensate for uncorrelated frames. So, basically there are two kinds of jitter at play, which is probably minimal in most cases, but not so much in mine.

Doing this project revealed two things to me:

The language-side FFT that we use in Signal really needs to be updated where the Server side stuff is, as it is probably 4-8 times slower than it could be with newer libraries.

The Server side FFT could also use some love in two main categories:
	1) allow for user defined windows
	2) implementation of a Buffer-based FFT that deals with buffers instead of audio-streams.

I realize the best response to these thoughts is that I should probably do the work myself, as this is an open-source, user-supported environment. I agree, but sadly capitalism..ya know..

Sam

> On Feb 25, 2021, at 8:53 PM, [email protected] wrote:
> 
> Just curious what you mean by sample accurate convolution?
> 
> On Fri., Feb. 19, 2021, 4:47 a.m. , <[email protected] <mailto:[email protected]>> wrote:
> Thanks Wouter. Uff. The old “changed how it worked and didn’t test it with a different duration” trick.
> 
> It should be fixed now. Tested on many different durMults.
> 
> Eli, thanks for that. I updated the help and actually gave TimeStretch2 a little tweak to place it in line with how TimeStretch functions. The silence and cut make sense. TS2 uses the server side fft, so in order to line up all the frames correctly, there has to be at least 65536 frames of silence at the beginning. The end of the track is a little trickier, and not perfect, but TimeStretch is really the way to go in all cases. I’ve spent a lot of time trying to get TS2 to sound like TS, but to no avail.
> 
> This is also true with the Convolve quark. The sample accurate convolution is a revelation. It is so clear and detailed compared to using something like ReaVerb with the same samples. It makes me want to only do NRT convolution reverbs in my mixes from now on…though I probably won’t.
> 
> Sam
> 
>> On Feb 18, 2021, at 5:56 PM, [email protected] <mailto:[email protected]> wrote:
>> 
>> Sam, testing these out. Cool stuff, thanks for adding these quarks.
>> 
>> I noticed that all the examples in the TimeStretch2 help file involve the use of TimeStretch (rather than TimeStretch2), which looks like a mistake. TimeStretch.stretch(...) produces a "method not understood" error.
>> 
>> Otherwise, all seems to be working for me. Still tinkering with settings to hear the different results. TimeStretch2.stretch(...) produces a file that has a chunk of silence at the beginning (~3 sec), and the audio is a bit cut off at the end of the file. This might be from something I'm doing on my end, though.
>> 
>> Nice work!
>> Eli
>> 
>>  
>> _________________________
>> Eli Fieldsteel, DMA (he/him/his)
>> Director, University of Illinois Experimental Music Studios 
>> Assistant Professor of Composition-Theory
>> Office: 217-300-0956
>> MB 4008, School of Music
>> 1114 W Nevada Street
>> Urbana, IL 61801
>> 
>> 
>> On Thu, Feb 18, 2021 at 8:53 AM <flo.huth-HaK8Zq/[email protected] <mailto:flo.huth-HaK8Zq/[email protected]>> wrote:
>> thank you!
>> 
>> On 17.02.21 15:34, [email protected] <mailto:[email protected]> wrote:
>>> Ah. I thought I could prevent download without preventing playback. I thought wrong. It is fixed now.
>>> 
>>> Sam
>>> 
>>>> On Feb 17, 2021, at 3:05 PM, flo.huth-HaK8Zq/[email protected] <mailto:flo.huth-HaK8Zq/[email protected]> wrote:
>>>> 
>>>> hi,
>>>> 
>>>> im restricted by your google drive and cant download the file.
>>>> 
>>>> thx!
>>>> 
>>>> On 17.02.21 14:17, [email protected] <mailto:[email protected]> wrote:
>>>>> Hey all,
>>>>> 
>>>>> I’ve added one quark of late and updated a second:
>>>>> 
>>>>> Major Update: TimeStretch
>>>>> 
>>>>> Super PaulStretch algorithm which separates the signal into 9 discrete "octaves", which are processed independently
>>>>> Now runs completely in sclang, so it is very slow, but also sample accurate
>>>>> On MacOS each channel can be processed in its own terminal window
>>>>> Uses correlation analysis and custom windowing between frames to achieve a smooth stretch
>>>>> Can time stretch resonant and transient elements with different settings, significantly improving the clarity of a stretch
>>>>> Now works with files of any SampleRate
>>>>> Python version by Alex Ness and Jem Altieri
>>>>> 
>>>>> Here is a little taste test with the Star Trek TNG theme song:
>>>>> 
>>>>> https://tinyurl.com/55ocdnm9 <https://tinyurl.com/55ocdnm9>
>>>>> 
>>>>> New Quark: Convolve
>>>>> 
>>>>> Some times you just want to convolve two sound files (and I have wanted to be able to do this in SC since I stopped using CSound in 2002). Uses the convolve method of the SignalBox Quark to convolve two files of any size. Runs completely in the language.
>>>>> 
>>>>> Let me know of any issues, and I hope these can help you make some covidtastic space jams.
>>>>> 
>>>>> Sam
>>>>> 
>>>>> 
>>> 
>