var create through mi mode on std::istream sometimes very slow

I need some help to further investigate a problem.
When using VSCode to debug using gdb, it unfortunately load the variables
for all the frames in current thread.
VSCode uses gdb's mi mode to get a vizualisation of these variables.

Most var-create takes 1 or 2 ms. But a few takes hundreds or a couple of
thousand ms.
The worst is std::istream taking around 2 seconds to be created, from the
log:
1: (51975) <-1343-var-create - * "is"
1: (54172)
->1343^done,name="var80",numchild="0",value="{...}",type="std::istream
&",thread-id="2",has_more="0"
1: (54172) ->(gdb)
1: (54172) 1343: elapsed time 2197

I can't say that I can reproduce it while running normal gdb, not using mi
mode. And I can't reproduce in a simple small test project, with just a
main and one function call with an std::istream, then the var-create is
down to around 1 ms.

When I test it now I run gdb without my gdbinit to avoid the influence of
our pretty-printers.
I did also test to register a pretty-printer for std::istream that just
returned a static text, it got the time down to around 1400 ms.

Any advice on how to further investigate this problem?

Running gdb 9.1 (same problem in 8.2)
Ubuntu 18.04
C++ (a rather big executable with around 100 threads)