help in tracking down 1481 memory leak (with reproduction steps)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi,

The past weeks I=E2=80=99ve been stumped trying to track down a severe memo=
ry
leak which prevents releasing 1481.

I have a hard time tracking down where and why exactly it happens,
therefore I=E2=80=99d be very grateful for your help.

The following shows how to reproduce the problem on GNU/Linux: Getting
Freenet 1481 to crash with an Out-of-Memory error within less than 30
minutes. The gist is: Upload a file.

tee freenet-1481-OOM-reproduction.sh << EOF

wget https://github.com/freenet/fred/releases/download/build01481/new_insta=
ller_offline_1481.jar
java -jar new_installer_offline_1481.jar
# click through the setup wizard and the in-browser first-run wizard, give =
Freenet high upload bandwidth (i.e. 164kiB/s)

# give freenet time to start the FCP server
sleep 180

# prepare a file to upload
INSERTFILE=3D"$(mktemp /tmp/insert.temp.XXXXXXXXXX)"
head -c 100M < /dev/urandom > "$INSERTFILE"
IDENT=3Dtestupload"${INSERTFILE##*.}"

# prepare the command to connect to freenet and upload the file
# connect with HELLO
TEMPFILE=3D"$(mktemp /tmp/insert.temp.XXXXXXXXXX)"
echo ClientHello > $TEMPFILE
echo "Name=3DUpload-Test${INSERTFILE##*.}" >> $TEMPFILE
echo ExpectedVersion=3D2 >> $TEMPFILE
echo End >> $TEMPFILE
echo >> $TEMPFILE

# upload with ClientPut
echo ClientPut >> $TEMPFILE
echo "DontCompress=3Dtrue" >> $TEMPFILE
echo "URI=3DCHK@/testupload" >> $TEMPFILE
echo "Identifier=3D$IDENT" >> $TEMPFILE
echo MaxRetries=3D-1 >> $TEMPFILE
echo UploadFrom=3Ddirect >> $TEMPFILE
echo DataLength=3D$(ls -l $INSERTFILE | cut -d " " -f 5) >> $TEMPFILE
echo Persistence=3Dforever >> $TEMPFILE
echo Global=3Dtrue >> $TEMPFILE
echo End >> $TEMPFILE
cat $INSERTFILE >> $TEMPFILE

# run the insert
(cat $TEMPFILE | nc 127.0.0.1 9481) &

# watch how long the node lives
for i in {1..100}; do curl 'http://127.0.0.1:8888/stats/?fproxyAdvancedMode=
=3D2' 2>/dev/null | grep -io nodeUptimeSession.*'<' | grep -io '[^;]*s<' | =
grep -io '.*s' ; curl 'http://127.0.0.1:8888/stats/?fproxyAdvancedMode=3D2'=
 2>/dev/null | grep -io '[^>]* java memory.*&' | grep -io '[^&]*'; sleep 5;=
 done

EOF

I hope this allows you to reproduce the problem =E2=80=94 and I would be ve=
ry
happy if you could find and fix the source of the problem! This has been
blocking the release of 1481 for far too long.

With 33 peers as target (but only up to 25 actually connected, my
connection isn=E2=80=99t that fast), this gets Freenet to die with an OOM w=
ithin
less than 15 minutes (last successful stats site at 14m17s).

Best wishes,
Arne
=2D-
Unpolitisch sein
hei=C3=9Ft politisch sein
ohne es zu merken

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE801qEjXQSQPNItXAE++NRSQDw+sFAlu6eUMACgkQE++NRSQD
w+uokg//WGXbvrX4aYALLiYtUokSB5h7SrZ0ZYIYdVNCXmH+WiTIbInULEZ/FYI5
S65hOX20LkOUNYknGXM2q/6I6L60GKstyiQWGcz3mUTPl4UX18bNrjqSz1nMOAXg
rAGluDrOCCuGss1te4q9DmXgiy9gCsjlikcI9oxKm/q9JzajWYzbP/hpBU8goXvZ
g8xySYf7a35/+94SQZoBS19qNvEwXkuoLgfer2I7jJUly10dkU7kjFXvO3wicX2C
w38I1Q8Tk90T543sNPg3OnJ3BFjdbBbBPk97ZX5sO9DEG/nJrJled8OhNZMmPWeL
Gu4hzTlSDwFRJK0NOZhFJ5Akv3YhNDr+CKzxBVSWavjoNK96fKaUdRnJgxAs9cHu
lFI91/zRBj//roh8XyWwAmLUyKrsKiu+4+76jj2pNPbvvRW6nZhKmGLdnqA1Xkux
9E4N36nVL16s0ufsuSc10bJLoYDIUheAmxN3mk9sdwa4z7LPVcZTjOOCdpdtFHmM
1QVU91/Pg5SDh1zEumGEBejLwqf8NOmBulNuOJxpl8PcatsGaVCgj5l51+7y4uzN
zDUndXmVRGouTJCmyQA23MqjLEn9YP0z/KLm9VHt8ZYYNjgoV8feBF7xlH4b7OPz
XKxaHq2ymLVVUtCe9jPA9x3PPuso7CBObJWvCtJuG7f9UHyMWx+IswQBAQgAHRYh
BN0ovebZh1yrzkqLHdzPDbMLwQVIBQJbunlDAAoJENzPDbMLwQVIBSgEAIstWCCT
pNCuuL8LojbMTlHhasHwjPVvyIFrTGyEg7DzHS+Sjf+qw9l1oe2McmFO4jij+KCT
DYH/G+4+leJDvCjNvN8atAUk5+nTnOQZvJIjx1EqnJ+UXW2XGVqZiLWy8RpBUhQ7
9KqRpmW4Nhv6xMu0rSMKx5jDibJfoh3o3gfR
=iRZj
-----END PGP SIGNATURE-----
--=-=-=--