Re: multiple qemu networking together and with host (Linux host)

On Fri, Dec 12, 2025 at 11:54 PM Andrew Randrianasulu
<[email protected]> wrote:
>
> On Fri, Dec 12, 2025 at 5:31 AM Andrew Randrianasulu
> <[email protected]> wrote:
> >
> > It was harder than I thought.
> >
> > Main setup:


/me crying in  copyasta ...: I failed to edit command  fully, so it
was mix of example and my own lines!

====
Main setup:

https://fo.ax/kali-kit/html_docs/86box.html

ip tuntap add dev tap-vde mode tap user "$USER"
ip link set tap-vde up
ip a add 10.1.0.1/16 dev tap-vde

vde_switch --mode 666 --numports 8 --tap tap-vde --mgmt
/tmp/vde.qemux0.mgmt --mgmtmode 666 -s /tmp/vde.qemux0 --daemon

Next, because I use wifi

sysctl -w net.ipv4.ip_forward=1

iptables -t nat -A POSTROUTING -s 10.1.0.1/16 -o wlan0 -j MASQUERADE

iptables -A FORWARD -i wlan0 -o tap-vde -m state --state
RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i tap-vde -o wlan0 -j ACCEPT


====

Now, with this setup I can't reach into qemu network from hosts on
192.x. network
So, now reading more networking/routing/iptables howtos ....
> >
> > https://fo.ax/kali-kit/html_docs/86box.html
> >
> > ip tuntap add dev tap-vde mode tap user "$USER"
> > ip link set 86box0 up
> > ip a add 10.1.0.1/16 dev tap-vde
> >
> >
> >
> > vde_switch --mode 666 --numports 8 --tap tap-vde --mgmt
> > /tmp/vde.qemux0.mgmt --mgmtmode 666 -s /tmp/vde.qemux0 --daemon
> >
> > Next, because I use wifi
> >
> > sysctl -w net.ipv4.ip_forward=1
> >
> > iptables -t nat -A POSTROUTING -s 10.1.0.1/16 -o wlan0 -j MASQUERADE
> >
> > iptables -A FORWARD -i eth0 -o tap-vde -m state --state
> > RELATED,ESTABLISHED -j ACCEPT
> > iptables -A FORWARD -i 86box0 -o wlan0 -j ACCEPT
> >
> > Install dhcpd (DHCP daemon)
> >
> > config it like
> >
> > cat /etc/dhcpd.conf
> > # dhcpd.conf
> > #
> > # Configuration file for ISC dhcpd (see 'man dhcpd.conf')
> > #
> > subnet 10.1.0.0 netmask 255.255.255.224 {
> > option routers 10.1.0.1 ;
> > range 10.1.0.10 10.1.0.15 ;
> > }
> >
> > whole config (ifconfig -a)
> >
> > root@slax:/dev/shm# ifconfig -a
> > eth7      Link encap:Ethernet  HWaddr A8:A1:59:9B:07:45
> >           UP BROADCAST MULTICAST  MTU:1500  Metric:1
> >           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> >
> > lo        Link encap:Local Loopback
> >           inet addr:127.0.0.1  Mask:255.0.0.0
> >           inet6 addr: ::1/128 Scope:Host
> >           UP LOOPBACK RUNNING  MTU:65536  Metric:1
> >           RX packets:340797 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:340797 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:235555020 (224.6 Mb)  TX bytes:235555020 (224.6 Mb)
> >
> > tap-vde   Link encap:Ethernet  HWaddr 86:2F:0D:6D:AC:96
> >           inet addr:10.1.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
> >           inet6 addr: fe80::842f:dff:fe6d:ac96/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:4709 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:4229 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:305415 (298.2 Kb)  TX bytes:260279 (254.1 Kb)
> >
> > wlan0     Link encap:Ethernet  HWaddr 1C:BF:CE:C6:BE:00
> >           inet addr:192.168.1.104  Bcast:192.168.1.255  Mask:255.255.255.0
> >           inet6 addr: fe80::1ebf:ceff:fec6:be00/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:3233241 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:2858540 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:1608028041 (1533.5 Mb)  TX bytes:845539101 (806.3 Mb)
> >
> > IPTABLES config:
> >
> > root@slax:/dev/shm# iptables -t nat -vnL
> > Chain PREROUTING (policy ACCEPT 1343 packets, 142K bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >
> > Chain INPUT (policy ACCEPT 1031 packets, 45964 bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >
> > Chain OUTPUT (policy ACCEPT 1736 packets, 297K bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >
> > Chain POSTROUTING (policy ACCEPT 1736 packets, 297K bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >    17  1052 MASQUERADE  all  --  *      wlan0   10.1.0.0/16          0.0.0.0/0
> >     0     0 MASQUERADE  all  --  *      wlan0   10.1.0.0/16          0.0.0.0/0
> >
> > =================
> >
> > root@slax:/dev/shm# iptables  -vnL
> > Chain INPUT (policy ACCEPT 86261 packets, 23M bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >
> > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >    56  8221 ACCEPT     all  --  tap-vde wlan0   0.0.0.0/0            0.0.0.0/0
> >    66 29463 ACCEPT     all  --  wlan0  tap-vde  0.0.0.0/0
> > 0.0.0.0/0            state RELATED,ESTABLISHED
> >
> > Chain OUTPUT (policy ACCEPT 82010 packets, 16M bytes)
> >  pkts bytes target     prot opt in     out     source               destination
> >
> > =======================
> >
> > qemu 1;
> > qemu-system-sparc -hda
> > /mnt/zip/home/guest/QEMU/sparc-netbsd-15g-compressed-r1.qcow2   -smp 1
> > -accel tcg,thread=multi,tb-size=256 -M SS-20 -net
> > nic,macaddr=52:54:00:02:12:23 -net vde,sock=/tmp/vde.qemux0
> >
> > qemu 2:
> > qemu-system-sparc -hda /dev/shm/sparc-netbsd-15g-compressed-r1.qcow2
> > -smp 1 -accel tcg,thread=multi,tb-size=256 -M SS-20 -m 256 -net
> > nic,macaddr=52:54:00:01:12:24 -net vde,sock=/tmp/vde.qemux0
> >
> > Both qemu NetBSD machines should get ip 10.1.0.xx
> >
> > You should be able to set /etc/resolv.conf to nameserver in your real
> > network (192.168.1.1 in my case - wifi router) and ping sites like
> > google.com etc
> >
> > I was able to get nc (netcat) sending and receiving across two qemu's
> > and see host ftp/ssh services.
> >
> > Next step is configuring distcc itself ;)
>
>
> I think I finally set it up, on server setting PATH to masquaraded compiler in
> /usr/pkg/lib/distcc/bin eraly in PATH
>
> then this command
>
> distccd --listen 10.1.0.10 --allow 10.1.0.11 --enable-tcp-insecure
> --log-stderr --verbose
>
> on client I set /etc/mk.conf as was suggested by this hackaday article:
>
> PKGSRC_COMPILER=distcc gcc
>  MAKE_JOBS=3
>  DISTCC_HOSTS=10.1.0.10:3632
>
> and finally set DISTCC_HOSTS to same 10.1.0.10 (I launched distcc
> server qemu first so it get IP earlier)
>
> at some point I was getting "Failed to distribute" errors, I think
> --enable-tcp-insecure, even if discouraged by man  was important?
>
>
>
>
> >
> > In theory vde2 should work on macos hosts, and may be freebsd ? Linux
> > is easy mode
> >
> >
> > It was bigger adventure than I initially hoped!