CBR/ABR quality regressions for q0, q1, q2 and q3

Hello there,

it has been observed that with CBR and ABR, perceived audio quality 
increases when using faster settings than default.

A discussion regarding this over at HydrogenAudio: 
https://hydrogenaud.io/index.php/topic,126120.0.html

This is very easily noticeable with very low bitrates, but is also 
detectable with the default bitrate of 128 kbps. The discussion thread 
on HydrogenAudio includes samples.

I traced this down to cfg->noise_shaping_amp, which is set to 1 for 
-q3/-q2 and is set to 2 for -q1/-q0. Choosing a value of 0 (as for the 
faster quality settings) improves perceived audio quality.

I opened up a bug report a while back over at 
https://sourceforge.net/p/lame/bugs/516/ with trivial patches and an 
attempt at a theory of why quality might be affected negatively with 
noise_shaping_amp being set to 1 or 2. This, however, is merely 
speculative, as it's not quite clear to me what settings 1 and 2 are 
actually trying to achieve - presumably, those are intended to increase 
quality (and perhaps they do in VBR?), but I don't quite understand the 
reasoning behind those approaches.

While setting noise_shaping_amp to 0 is a trivial thing to do, it might 
also be (pure speculation, again) that the actual root cause is with the 
CBR and ABR encoding loops somehow not interacting properly with 
noise_shaping_amp 1/2 and that a "true fix" might involve tinkering with 
those encoding loops.

How would one progress on this issue? With -q3 being the default 
setting, rather regrettably CBR and ABR encodings are impacted 
negatively by default.

Another, but most likely minor thing: It appears that noise_shaping_stop 
is unused, yet set up by lame_init_qval. I assume this could be removed?

Best regards,
Maik