Do not load specific Linux kernel modules

Hi,

I'm trying to solve some possible SIP issues in my LAN, and I'd like
to temporarily disable SIP-related Linux kernel modules.
It seems that shorewall loads the modules according to the content of
/usr/share/shorewall/helpers. Instead of touching that file I'd rather
set DONT_LOAD in shorewall.conf.

# grep DONT_LOAD shorewall.conf
DONT_LOAD=nf_nat_sip,nf_conntrack_sip

However, a shorewall restart will not remove the following:

# lsmod | grep -i sip
nf_conntrack_sip       40960  2
nf_conntrack          176128  25
xt_conntrack,nf_nat_irc,nf_nat,nf_conntrack_tftp,nf_nat_ftp,nf_conntrack_pptp,nf_conntrack_netbios_ns,nf_conntrack_sane,xt_nat,nf_nat_tftp,nf_nat_amanda,nf_conntrack_sip,nf_conntrack_h323,nf_nat_pptp,nf_conntrack_broadcast,nf_conntrack_irc,nf_conntrack_amanda,nf_conntrack_netlink,xt_connmark,nf_conntrack_ftp,xt_CT,nf_nat_h323,nf_conntrack_snmp,nf_nat_snmp_basic,xt_REDIRECT

# modprobe -r nf_conntrack_sip
modprobe: FATAL: Module nf_conntrack_sip is in use.

Is there a way to unload nf_conntrack_sip without rebooting the OS?

Regards,

Vieri