Bridged network configuration


#1

I would like to bridge two ethernet interfaces on a resin device.

Router with DHCP — eno1 on resin device
eno2 on resin device — other device

I booted an ubuntu live USB image on the machine I had installed resin on and placed the three files below in /etc/NetworkManager/system-connections. I was then able to manually start them with

nmcli c reload
nmcli c up 495714fd-b741-491c-a22f-16bfe6fd36b4
nmcli c up 0c7117e7-a7e7-4840-a282-7e509a3fdf38
nmcli c up 3ec37956-8e2b-48f2-a784-e80677c072d3

This produced the network configuration I hoped, with a DHCP assigned address on bridge0 and a device plugged in to eno2 able to connect to acquire a DHCP lease and connect to the network as expected. I then copied all three files to /resin-boot/system-connections on the resin device and rebooted. This resulted in no network connectivity, either for the resin device or for the device plugged in to eno2. I suspect that not all connections are started. How do I ask NetworkManager to start these three connections on boot?

bridge0

[connection]
id=bridge0
uuid=495714fd-b741-491c-a22f-16bfe6fd36b4
type=bridge
interface-name=bridge0
permissions=
secondaries=

[ipv4]
dns-search=
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=auto

eno1

[connection]
id=eno1
uuid=0c7117e7-a7e7-4840-a282-7e509a3fdf38
type=ethernet
interface-name=eno1
master=bridge0
permissions=
secondaries=
slave-type=bridge

[ethernet]
duplex=full
mac-address-blacklist=

eno2

[connection]
id=eno2
uuid=3ec37956-8e2b-48f2-a784-e80677c072d3
type=ethernet
interface-name=eno2
master=bridge0
permissions=
secondaries=
slave-type=bridge

[ethernet]
duplex=full
mac-address-blacklist=


#4

Hello,

I would suggest creating a small script that runs on boot that activates the connections using the network manager command line tool.
For example:

# list connections
$ nmcli connection show
# bring connections up
$ nmcli connection up 495714fd-b741-491c-a22f-16bfe6fd36b4
$ nmcli connection up 0c7117e7-a7e7-4840-a282-7e509a3fdf38

#5

OK, that seems reasonable, but now I have an installation problem. I can add these configuration files to the resin install USB stick and they will be copied to the system. But without the script, the system cannot connect to resin.io to install an image containing the script. Is there some place to put scripts that execute before the system attempts to connect to the network?


#6

You can put connection files in the boot partition of a ResinOS image in the /system-connections directory. These connection files will be copied to the root partition. If these connections are set to autoconnect, they should start automatically when Network Manager starts.


#7

OK, I tried again with a slightly different configuration and got about the same result. The bridge is not automatically started, according to the documentation autostart=TRUE is the default. Manually setting this did not help, nor did setting autostart-slaves=1. When I try to start the bridge manually, I get an error. I’m a bit stumped now, any suggestions on the next thing to try? The full session interacting with the resin host OS is below:

=============================================================
Welcome to ResinOS
=============================================================
bash-4.3# nmcli c show
NAME UUID TYPE DEVICE
Wired connection 1 0b019836-371f-3dcc-9f0f-da538656fda5 802-3-ethernet eno3
Wired connection 2 ebdeeccb-628f-3a27-b3d0-b4f257e7ba82 802-3-ethernet –
br0 1f31f5d6-4536-48a9-9584-6f642787bcf4 bridge –
eno1 6a9af57b-bd12-45cf-a221-e953bda5e9ff 802-3-ethernet –
eno2 827e029e-edfc-49dd-9220-d2da61bad29a 802-3-ethernet –
enp0s20u1u1 915bc827-6514-4af6-aade-b7044997ca00 802-3-ethernet –
bash-4.3# nmcli c up br0
Error: Connection activation failed: Connection ‘br0’ is not available on the device br0 at this time.
bash-4.3# nmcli c up eno1
Error: Connection activation failed: Master device ‘eno1’ can’t be activated: Device unmanaged or not available for activation
bash-4.3# nmcli c up eno2
Error: Connection activation failed: Master device ‘eno2’ can’t be activated: Device unmanaged or not available for activation
bash-4.3# nmcli c up enp0s20u1u1
Error: Connection activation failed: Master device ‘enp0s20u1u1’ can’t be activated: Device unmanaged or not available for activation
bash-4.3# ifconfig br0
br0 Link encap:Ethernet HWaddr 1A:19:F2:9E:BB:1B
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

bash-4.3# ifconfig eno1
eno1 Link encap:Ethernet HWaddr AC:1F:6B:15:F6:8C
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:51 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6691 (6.5 KiB) TX bytes:0 (0.0 B)
Memory:fb220000-fb23ffff

bash-4.3# ifconfig eno2
eno2 Link encap:Ethernet HWaddr AC:1F:6B:15:F6:8D
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3078 (3.0 KiB) TX bytes:0 (0.0 B)
Memory:fb200000-fb21ffff

bash-4.3# ifconfig enp0s20u1u1
enp0s20u1u1 Link encap:Ethernet HWaddr 70:88:6B:82:9C:3A
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

The contents of the configuration files are:

br0

[connection]
id=br0
uuid=1f31f5d6-4536-48a9-9584-6f642787bcf4
type=bridge
interface-name=br0
permissions=

[ipv4]
dns-search=
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

eno1

[connection]
id=eno1
uuid=6a9af57b-bd12-45cf-a221-e953bda5e9ff
type=ethernet
interface-name=eno1
master=br0
permissions=
slave-type=bridge

[ethernet]
mac-address-blacklist=

eno2

[connection]
id=eno2
uuid=827e029e-edfc-49dd-9220-d2da61bad29a
type=ethernet
interface-name=eno2
master=br0
permissions=
slave-type=bridge

[ethernet]
mac-address-blacklist=

enp0s20u1u1

[connection]
id=enp0s20u1u1
uuid=915bc827-6514-4af6-aade-b7044997ca00
type=ethernet
interface-name=enp0s20u1u1
master=br0
permissions=
slave-type=bridge

[ethernet]
mac-address-blacklist=


#8

Hi @russel, thanks for providing this information. We’re looking into this and should have some more information for you soon. Thanks for your patience!


#9

@russel, just wanted to update you here and let you know that the ability to managed bridged networks should now be in the latest release of resinOS in our staging environment (v2.9.5 as of the time I’m writing). It’s not yet in production but you can kick the tires if you like by downloading the latest resinOS from dashboard.resinstaging.io. Assuming all the testing goes well this should be available in production soon.


#11

Thanks for the update, I finally got back to this today. The most recent available resinOS was v2.9.6+rev1(prod). I plugged in a USB-Ethernet adaptor which was assigned the name enp0s20u3u3. I then used the “Host OS” console to execute the following commands:

# nmcli c add ifname br0 type bridge con-name br0
# nmcli c add type bridge-slave ifname eno2 master br0 con-name eno2-bridge
# nmcli c add type bridge-slave ifname enp0s20u3u3 master br0 con-name adapter-bridge

This created the connection entries I expected. I then disabled the default connections for the two interfaces and tried to bring up the bridge:

# nmcli c down "Wired connection 2"
Connection 'Wired connection 2' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
# nmcli c down "Wired connection 3"
Connection 'Wired connection 3' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
# nmcli c up br0
Error: Connection activation failed: Connection 'br0' is not available on the device br0 at this time.

These commands resulted in files in /etc/NetworkManager/system-connections with the same options as when I tried manually creating the files in my other tests. It looks to me like I am doing something fundamentally wrong, hopefully someone here can spot the error.


#12

Hi @russel, it looks like the issue is that we are explicitly excluding any bridge names that start with balena* or br* . Could you please try with a different name and see if that works?

We’ll also take a note to update our docs to reflect that.


#13

Success! I changed the bridge name to cb0 and was able to enable the bridge. This allowed IPV6 traffic to flow, but IPV4 traffic including DHCP requests were still blocked. It looks like the default FORWARD firewall rule drops packets and exceptions need to be made. Adding a firewall rule matching the one for balena0 resulted in the connectivity I want

# iptables -A FORWARD -i cb0 -j ACCEPT
# iptables -A FORWARD -o cb0 -j ACCEPT

Does resin have a way to apply these firewall rules at system boot or do I need to do this in my application container?


#28

currently we have no firewall services on host. Is adding these rules in the container works well for you?


#29

currently we have no firewall services on host. Is adding these rules in the container works well for you?


#42

Yes, adding the firewall rules in my application container works. I now have the bridging configuration I hoped for working, thank you all for your help.


2 Ethernet cards, 2 networks
#43

Thanks for your feedback. We created an issue and we are going to investigate further. https://github.com/resin-os/meta-resin/issues/958


#45

I investigated the issue in detail and posted an update on github with my findings.

To summarise, iptable rules shouldn’t be needed for creating bridged networks in the current resinOS images (tested 2.12.6 as of this time).