Re: Spectral Editing is Not User Friendly

Steve and friends,

> I like the idea of being able to export the spectrogram, edit in an 
> image editor, and then import back into Audacity. I presume that what 
> you have in mind is similar to the software "Photosounder" 
> (http://photosounder.com/). That would be a totally new feature, with 
> only superficial similarity to the current "spectral edit" feature.

I don't think it is theoretically possible to edit the spectrogram in an 
image processor (even if it were a customized one for Audacity) and get 
a valid spectrogram of the processed signal from which the processed 
signal could be recovered.

First of all, the spectrogram has no phase information, so colors alone 
cannot represent the correct sound, even the original one. In other 
words, the spectrogram drawing is not a reversible operation, even if no 
transformation or effect is applied.

Second, the color map is a single-dimension one, defined by the linear 
interpolation of five colors which are just points on a bi-dimensional 
color map. Unless we could ensure the map color of the image processor 
is restricted to the colors contained in the piecewise linear path 
representing amplitudes, we would get meaningless spectrograms.

Third, there is a limitation in resolution given by the size of the 
window so spectrograms do not present a continuous spatial transition of 
colors. Instead there are small but noticeable rectangles whose areas 
depend on the time-frequency uncertainty principle. If the FFT size is 
large, they are very short (high frequency resolution) and wide (low 
time resolution); if FFT size is small, they are very tall and narrow 
instead.

A possibility would be to take advantage of the bi-dimensional nature of 
the full visible color map and represent the FFT complex numbers 
(retaining the phase information), which are also bi-dimensional, as 
colors. I guess several people have already tried some experiments on 
this (I mean the graphic representation; the concept of complex 
spectrogram is frequently used in signal processing). I also guess the 
spectrogram image would be stunning but far from being intuitive. 
Moreover, it would be very difficult to figure out the result of a given 
graphic effect, as well as what to do with the image to get the desired 
signal effect.

The only practical approach is to simulate the spectrogram editing for 
very simple operations, such as zero or one gain (or real gain or 
attenuation).

This is the approach we had presented (with Luciano). Our plugin allowed 
to draw closed regions of arbitrary shape to be either filtered out or 
retained . The plugin was based on overlapping time-varying FFT filter. 
Whatever the case, the retained part has gain 1 and the rejected part 
has gain 0. The idea could be extended to support drawing with an 
"attenuating pencil" which would apply a given gain (picked from a gray 
shade map).

You mentioned Photosouder. I skimmed through the user guide and there is 
no hint as to what is actually doing when importing an image. What I'm 
pretty sure is that when editing an already loaded or computed 
spectrogram it does not re-synthesize the sound from the image, but 
processes the signal and recalculates the spectrogram.

In other words, for an arbitrary image (say, a photo), it tries to 
represent it within the available color map and spectral and time 
resolution without concern for phase, but they have the advantage that 
we don't know a priori how the image "sounds"... And for a true 
spectrogram, it just simulates the image editing.

Regards,

Federico



On 06/10/2017 11:19, Steve the Fiddle wrote:
> Hi Nikita,
>
> Spectral Editing in Audacity is a fairly new, and not yet fully 
> developed, feature. Despite its shortcomings, it was considered to 
> still be useful enough to include in the Audacity release.
>
> I like the idea of being able to export the spectrogram, edit in an 
> image editor, and then import back into Audacity. I presume that what 
> you have in mind is similar to the software "Photosounder" 
> (http://photosounder.com/). That would be a totally new feature, with 
> only superficial similarity to the current "spectral edit" feature.
>
> The current feature is really a 'simulation' of editing the 
> spectrogram, rather than actually editing it. How it works is that the 
> selected spectral region sets two frequency parameters (high and low). 
> Specially crafted effects (the "Spectral" effects) can then use these 
> parameters to control their behaviour. The effects then process the 
> audio data as would any other audio effect. So the effect is not 
> actually acting on the spectrogram - it acts on the audio, and the 
> spectrogram is updated from the processed audio.
>
> Steve
>
> On 6 October 2017 at 13:38, Nikita Sadkov <[email protected] 
> <mailto:[email protected]>> wrote:
>
>     Hi! It was an amazing discovery, that Audacity now supports spectral
>     editing - arguably one of the most fundamental features, especially if
>     you want to synth custom sfx for your project or edit some badly
>     recorded material from https://freesound.org
>
>     My complain is that it is very obscure and tedious to use in its
>     current form. You have to separately enable spectral selection and use
>     "Spectral edit ..." tools under "Effect" menu. I.e. you can't just
>     press Del to clear the frequencies or use Draw Tool (F3) to paint your
>     sfx with mouse.
>
>     So I think it would make Audacity order of magnitude more useful, if
>     it copies UI from Sony SpectraLayers, together with layers, which
>     works just like a raster editor, but unfortunately lacks waveform
>     editing capabilities. Also, it would be nice feature to export
>     spectrum plot into png files, so people could edit it in GIMP and then
>     import back into Audacity.
>
>     ------------------------------------------------------------------------------
>     Check out the vibrant tech community on one of the world's most
>     engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>     *********** ASKING FOR HELP *************
>
>     When asking for help on this list, please include the following
>     information, so we can
>     help you properly:
>
>     * What operating system you are using (for example, Windows XP or
>     Mac OS X 10.5.1)
>
>     * Exactly what three digit version number of Audacity you are
>     using (Help > About
>        Audacity, or Audacity >  About Audacity on a Mac computer)
>
>     * If this is a recording problem, what equipment you are recording
>     with, and how is it
>        connected to the computer?
>
>     Mailing list: [email protected]
>     <mailto:[email protected]>
>     To UNSUBSCRIBE, use the form at the bottom of this web page:
>     https://lists.sourceforge.net/lists/listinfo/audacity-users
>     <https://lists.sourceforge.net/lists/listinfo/audacity-users>
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> *********** ASKING FOR HELP *************
>
> When asking for help on this list, please include the following information, so we can
> help you properly:
>
> * What operating system you are using (for example, Windows XP or Mac OS X 10.5.1)
>
> * Exactly what three digit version number of Audacity you are using (Help > About
>     Audacity, or Audacity >  About Audacity on a Mac computer)
>
> * If this is a recording problem, what equipment you are recording with, and how is it
>     connected to the computer?
>
> Mailing list: [email protected]
> To UNSUBSCRIBE, use the form at the bottom of this web page:
> https://lists.sourceforge.net/lists/listinfo/audacity-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

*********** ASKING FOR HELP *************

When asking for help on this list, please include the following information, so we can
help you properly:

* What operating system you are using (for example, Windows XP or Mac OS X 10.5.1)

* Exactly what three digit version number of Audacity you are using (Help > About
   Audacity, or Audacity >  About Audacity on a Mac computer)

* If this is a recording problem, what equipment you are recording with, and how is it
   connected to the computer?

Mailing list: [email protected]
To UNSUBSCRIBE, use the form at the bottom of this web page:
https://lists.sourceforge.net/lists/listinfo/audacity-users