Disabling NetworkManager DNS for shared connection


#1

Hello,

I’m trying to run ResinOS on an RPi3 configured as a hotspot and router. The setup works well using the hotspot example from the docs, but now I’d like to set up pi-hole for network-wide ad-blocking. The recently-released pi-hole 4.0 ships an official Docker container, so it should definitely run in a container.

Unfortunately, it won’t launch because port 53 is in use. I did some poking around, and it looks like NetworkManager launches a dnsmasq instance for shared connections. To disable DNS in this instance and only use DHCP, you’d likely edit a file in /etc/NetworkManager/dnsmasq-shared.d and set port = 0.

I’ve spent a couple hours trying to figure out a way to do this. I understand that the root partition is read-only, and that files in the state partition are whitelisted. Is there some other way to disable DNS on the dnsmasq instance NetworkManager manages for shared connections so I can run a pi-hole container?

Note that I don’t want to change what DNS value my DHCP server returns. I’m fine with it returning what it does. I just want to run my own DNS server at that location.

I’ve also tried systemctl stop dnsmasq, but this only affects the upstream dnsmasq, not the one NetworkManager launches for shared connections. I also found a thread about running pi-hole, but it seems to imply that you can bind containers to the interface IP, which I don’t think is the case when NetworkManager is sharing the connection.

Thanks for any help.


#4

WiFi Connect also spawns a dnsmasq instance in its container: https://github.com/resin-io/resin-wifi-connect/blob/master/src/dnsmasq.rs. It does not interfere with the one running in the host OS.

I used hostapd a few times as well and it needed a similar dnsmasq setup: https://github.com/resin-io-playground/hostapd-minimal

I have not used pi-hole, but I quickly saw now in its man page the following configuration option: -i, interface Specify dnsmasq's interface listening behavior. Will that be helpful?


#6

I don’t think the issue is that Pihole needs a different interface configuration. The issue is that when using the AP configuration on the network setup page, with method=shared, NetworkManager binds port 53 in a way that Resin won’t let me change. So ultimately I need to pass -p 53:53 to my Docker command line, and that won’t work regardless of what happens on Pihole’s end because Docker is doing the binding.

I have, however, made a bit of progress. I put my wireless interface in manual mode and connected to it directly, manually setting an IP on my laptop. When doing this, I can ping out from the device, but it of course isn’t routing packets from my laptop. So I tried bridging my wifi and ethernet interfaces. The ethernet interface gets an IP automatically via DHCP, while I want the wifi interface acting as an access point. Unfortunately, the wifi interface never comes up. Here’s what I have. First, the bridge:

[connection]
id=br0
uuid=69614cbf-5245-449d-819d-2496ca7efade
type=bridge
autoconnect=true
interface-name=br0
permissions=

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

Not sure if all these autoconnect=trues are needed. Next, the ethernet. This gets an address from my provider via DHCP.

[connection]
id=bridge-slave-eth0
uuid=56793758-6a0b-4613-b17b-6d2efaac7a6d
type=ethernet
autoconnect=true
interface-name=eth0
master=br0
permissions=
slave-type=bridge

[ipv4]
method=auto

And wlan0, which I just want to assign a manual address and have it route packets. With this in place, port 53 should be unblocked and I can run Pihole:

[connection]
id=bridge-slave-wlan0
uuid=bb421627-971e-4637-a50b-4ffe61a33e73
type=wifi
autoconnect=true
interface-name=wlan0
master=br0
permissions=
slave-type=bridge

[wifi]
band=bg
mac-address-blacklist=
mac-address-randomization=0
mode=ap
seen-bssids=
ssid=ICanHasWifi

[wifi-security]
group=
key-mgmt=wpa-psk
pairwise=
proto=rsn
psk=xxx

[ipv4]
address1=192.168.0.1/24
dns-search=
method=manual

Any idea what I might be missing?


#7

I also enabled persistent logging, plugged the Pi in for a couple minutes, then unplugged it. Nothing looked alarming in the review. All the interfaces appeared to be grought up, in the sense that NetworkManager said they were. But I never saw DHCP requests from the ethernet port, nor did my AP come up. I’m convinced I’ve configured this bridge wrong in a subtle way that I just can’t spot, but that is perfectly reasonable for NM to just bring up the devices and do nothing more. :slight_smile:
Thanks.


#10

You were right about NetworkManager spawning a new dnsmasq instance when setting connection mode to shared. This runs on the same IP that you need pi-hole’s dnsmasq instance to run, so the port will not be available indeed. Manual option is the correct one for this case.

My suggestions would be:

  1. For connection sharing there are two possibilities NAT and bridged. I think you will be better off for your use case with NAT. This is also what NetworkManager does in its shared mode.

  2. Install pi-hole in a recent Debian container. Usually I go with either Stretch or even the unstable Buster if I have issues on Stretch.

  3. I looked at pi-hole’s installer. It checks for nmcli availability. nmcli is the command line interface of NetworkManager. It communicates with NetworkManager through D-Bus. You may install NetworkManager inside the container, so that the installer invokes nmcli. However do not forget to mask the NetworkManager’s service: RUN apt-get update && apt-get install -y network-manager && systemctl mask NetworkManager.service. You will need also to export the correct DBUS_SYSTEM_BUS_ADDRESS address, so that nmcli communicates with the NM service running on the host OS. More information on this here: https://docs.resin.io/reference/OS/network/2.x/#changing-the-network-at-runtime

  4. If you are running a multicontainer setup you need to set docker compose network_mode to host.

  5. Investigate what the pi-hole installer does, as it may well set networking for your use case. I have not done that, so I am not sure whether it will do that or not.

  6. Here is another topic related to Internet connection sharing that you may find useful which includes a sample repo from a forum user: Route Wireless to Wired with resinOS 2.x and RPi 3

  7. You may find useful the create_ap script, which is for creating Access Points with Internet sharing (it uses hostapd instead of the lightweight built-in access point NetworkManager creates, but it can still be useful for creating the AP, or as a reference at least): https://github.com/oblique/create_ap

Hope that is helpful :wink:


#12

Thanks for all the help. I’m almost there! My setup does almost exactly what I want, but because of the read-only rootfs, a couple things won’t persist. I’ll document everything here in case anyone googles how to set up pihole on ResinOS and needs pointers.

First, the hotspot config:

[connection]
id=resin-hotspot
uuid=36060c57-aebd-4ccf-aba4-ef75121b5f77
type=wifi
autoconnect=true
interface-name=wlan0
permissions=
secondaries=

[wifi]
band=bg
mac-address-blacklist=
mac-address-randomization=0
mode=ap
seen-bssids=
ssid=ICanHasWifi

[wifi-security]
group=
key-mgmt=wpa-psk
pairwise=
proto=rsn
psk=supersecretpasswordhere

[ipv4]
address1=192.168.0.1/24
dns-search=
method=manual

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

That’s the only interface you need for my setup, which uses the Pi as my network router connected to a cablemodem which is assigned an IP via DHCP.

Next I assign 192.168.0.2/24 to my laptop’s network address, set the gateway to 192.168.0.1, and SSH to root@192.168.0.1:22222. I then run:

# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Now I set up a docker-compose.yml to launch pihole:

version: "2"

volumes:

  pihole:

  dnsmasq:

services:

  pihole:
    image: pihole/pihole:v4.0_armhf
    network_mode: host
    ports:
      - 53:53
      - 53:53/udp
      - 67:67/udp
      - 80:80
      - 443:443
    volumes:
      - pihole:/etc/pihole
      - dnsmasq:/etc/dnsmasq.d
    environment:
      ServerIP: 192.168.0.1
      WEBPASSWORD: super_secret_web_password
      DNSMASQ_LISTENING: local
    cap_add:
      - NET_ADMIN

Not sure if the NET_ADMIN is needed since I’ve set network_mode: host, but I just got this working and am hesitant to breathe on it wrong.

Before launching this, I need to disable dnsmasq:

# systemctl stop dnsmasq

Then, on my laptop:

$ DOCKER_API_VERSION=1.22 DOCKER_HOST=tcp://192.168.0.1:2375 docker-compose up -d

With this setup, you’re able to access http://pi.hole/admin/. You probably want to access Settings->DHCP, enable the DHCP server, set the range, etc. I think this sets up dnsmasq to listen on local interfaces, but couldn’t hurt to check under Settings->DNS.

Two problems with this setup. First, I need to set up the iptables rule on each boot. Next I need to either disable dnsmasq permanently, or just stop it again and again. Looks like the way to do this is to spin up other containers with the appropriate privileges to run these commands, and then make the pihole container depend on those starting up. I’ll try to integrate that later, but right now I’m going to just let this run and get back to work. :slight_smile:
Thanks a bunch for your help and pointers.


#13

Nice work! You may also create an example GitHub repo whenever you are ready and have some additional time for this so that other Resin users may try it out :wink: