Xen in WiFi networks

tecr

　　From:
Xen in WiFi networks


　　
　　Successfully running Xen on a machine with a single WiFi card can be tricky. In my situation the network in
dom0 worked fine but in domU I could only see incoming packets. Outgoing ones were sent out to
xen-br0 but never transmitted over the wireless network.

Sample network configuration
　　There are two physical hosts on the network - Router and a Xen host running
dom0 and domU:

Host
IP address
Router
192.168.157.20
Xen dom0
192.168.157.21
Xen domU
192.168.157.41　　Xen dom0 network interface has the real MAC address of the
WiFi card, Xen domU MAC address is set in /etc/xen/domU config:

~# cat /etc/xen/domU
[...]
nics = 1
vif = ['eth0=00:60:00:00:00:01']

　　



Problem and solution
　　After some research it turned out that the culprit blocking outgoing traffic wasn't
Xen but the wireless card which refused to send out packets with a MAC address different from its own. Since Xen deploys
ethernet bridge to provide network for domU kernels the outgoing packets have a virtual source MAC address different from the real one of the
WiFi card. You don't need Xen to observe this problem - simple ethernet bridge constructed with brctl(8) between wired and wireless segments will not work with many wifi cards either. See section
It doesn't work with my Wireless card at
Linux Bridging FAQ.
　　The solution obviously is to change the source MAC address to the real one on all outgoing packets. At first I tried to set it in vif=['mac=...'] config option for
domU host, but it didn't work. Better solution turned out to be deployment of
ebtables - netfilter infrastructure similar to iptables but working with MAC addresses instead of IP addresses. The simplest ebfilter rule to make it work is:
　　



ebtables -t nat -A POSTROUTING -s 00:60:00:00:00:01 \
-j snat --to-source 00:60:aa:bb:cc:dd

　　

This rule rewrites the source MAC address of all packets from 00:60:00:00:00:01 to 00:60:aa:bb:cc:dd.You may want to add some more checks, e.g. for source interface name or alike. This is the bare minimum. I have executed this command in
dom0 but in theory it could work in domU as well.

Details
　　Surprisingly enough there doesn't have to be a corresponding reversed dnat rule. Ping from outside to
domU triggers this packet sequence:
　　



01:56:03.972501 00:02:11:22:33:44 > ff:ff:ff:ff:ff:ff,
ethertype ARP (0x0806), length 42:
arp who-has 192.168.157.41 tell 192.168.157.20
01:56:03.975205 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,
ethertype ARP (0x0806), length 60:
arp reply 192.168.157.41 is-at 00:60:00:00:00:01

　　

Interesting point is the ARP reply. Although it comes from WiFi real MAC address, it carries the domU
virtual MAC inside in the packet. Subsequently my workstation created an entry in its ARP table with the
virtual(!) address of domU and started sending all further communication to that MAC:
　　



01:56:03.975231 00:02:11:22:33:44 > 00:60:00:00:00:01,
ethertype IPv4 (0x0800), length 98:
IP 192.168.157.20 > 192.168.157.41:
icmp 64: echo request seq 1
01:56:03.982348 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,
ethertype IPv4 (0x0800), length 98:
IP 192.168.157.41 > 192.168.157.20:
icmp 64: echo reply seq 1

　　

Fortunately the Linux kernel on Router only updates its ARP table from the body of ARP replies and not from packet headers, which I believe is a correct behaviour. However I'm not sure that
all operating systems do like this.

Notes

• To get the MAC address of your WiFi card use command
  ip link show:
  

ip link show
# ip link show
[...]
2: eth0: <BROADCAST,MULTICAST,NOTRAILERS,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:60:aa:bb:cc:dd brd ff:ff:ff:ff:ff:ff

• The chipset on my WiFi card that caused these troubles is
  Intersil Corporation Prism 2.5 Wavelan chipset with firmware 1.3.6. Other chipsets may work just fine without these kludges.
  
• Xen version in use here is 2.0.6 and both dom0 and domU are running SuSE Linux Professional 9.2 with kernel 2.6.11.4, but it is fairly irrelevant as the problem is in hardware not in software.