Re: Performance since updating Ubuntu 18.04 to 22.04 and many drop lines

Hi Paul,

Thanks. Because it was becoming a problem on my updated servers (they couldn't keep up), I got the same idea last night.
I just implemented it this morning. Because the shorewall allow & drop commands are broadcasted frequently by an ansible (ssh) commands on a cluster of multiple servers, I had to improvise to test this on a specific server.

I've moved the current database to an ipset (and increase the default list-length-limit). I added this to the shorewall rules config.
I've switched the shorewall executable for a simple bash script (I'm sorry, I'm not with the bash syntax so I guess you'll laugh based on the quality).

#!/bin/sh

if [ "$1" = "drop" ]; then
    for ip in "$@"
    do
        if [ "$ip" != "drop" ]; then
            /usr/sbin/ipset add tfblocklist $ip
        fi
    done
    /usr/sbin/ipset save > /etc/iptables/ipsets
elif [ "$1" = "allow" ]; then
    for ip in "$@"
    do
        if [ "$ip" != "allow" ]; then
            /usr/sbin/ipset del tfblocklist $ip
        fi
    done
    /usr/sbin/ipset save > /etc/iptables/ipsets
else
    /usr/sbin/shorewallorig "$@"
fi

Seems to work alright, performance is great (even better than before). So I think I'll implement this method on the cluster.
Using ipset to solve this problem seems like an improvement as it still keep the enormous iptables -L -n list readable.

Thanks for you input and provided solution!!


Martijn Verhoef

Van: Paul Gear via Shorewall-users <[email protected]>
Verzonden: donderdag 4 juli 2024 08:59
Aan: [email protected]
CC: Paul Gear <[email protected]>
Onderwerp: Re: [Shorewall-users] Performance since updating Ubuntu 18.04 to 22.04 and many drop lines


Hi Martijn,

I've noticed similar things, although it's not a big deal on my system because the number of addresses is much lower.

Under recent Ubuntu (and Debian, and I'm sure many other distros) versions, iptables has become a compatibility wrapper around nftables.  My guess (only a guess, without any data to back it up) would be that this is the cause.

I'd try using ipsets instead to see if this improves your performance; something like:

DROP    net:+reject  $FW
REJECT    $FW    net:+reject

in your rules to implement the blocking, and:

ipset create -exist reject hash:ip counters hashsize 65536 maxelem 16777216  # tune these numbers to your liking

to create the set, and:

ipset add reject 1.2.3.4

to add something to the list.

I'd be interested to know how you fare with this...
On 4/7/24 05:10, Martijn Verhoef via Shorewall-users wrote:
Hi,

Since I updated Ubuntu, I've been experiencing performance problems when using the 'shorewall drop' command.
During the upgrade Ubuntu 18.04 to 22.04, shorewall updated from version 5.1.12.2  to 5.2.3.4

Based on a script, I update my firewall rules every few minutes using a 'shorewall drop <ip1> <ip2> ... && shorewall allow <ip1> <ip2> ...' command.

Since the upgrade, I see that it takes approximately 15 seconds per ip-address to process. On my other servers, it takes much less time.

Using the process manager, I found out the following 4 commands are executed and take approx. 3-4 seconds each. How is it possible that they take so much time since this update?
/sbin/iptables -D dynamic -s <ip> -j reject
/sbin/iptables -D dynamic -s <ip> -j DROP
/sbin/iptables -D dynamic -s <ip> -j logreject
/sbin/iptables -D dynamic -s <ip> -j logdrop
...