On MC7304, MC7430 or MC7750 etc I can use AT!USBCOMP command to enable both RMNET0 and RMNET1 USB interfaces by the module, the Linux kernel driver, qcserial and qmi_wwan, will create two pairs of wwan0 and /dev/cdc-wdm0, wwan1 and /dev/cdc-wdm1 respectively.
When I am using the ModemManager’s Simple interface’s Connect method to setup data connection, most of time the wwan0 interface is activated, but sometimes wwan1 is used instead
The Connect method doesn’t take argument about which wwanX interface to be used for data connection, how should I do that ?
This would help pave the way for setting up multiple PDN connections via different wwan interfaces.
There’s no way to ask ModemManager to request one specific wwanX over another. When the modem is detected by ModemManager, it will detect both cdc-wdm0 and cdc-wdm1 as QMI-capable, and it will be setup to use both.
On a standard connection triggered by the Simple.Connect() method, you’ll ask ModemManager to get the modem connected as easy as possible, and that means it will try to use the first available cdc-wdm/wwan pair to setup the connection.
If you want to have multiple PDNs connected at the same time you cannot use the Simple interface. Instead, you should manually create bearer objects with different APN settings, and connect the bearers independently with the Bearer.Connect() method.
Therefore, if your modem exposes 2 different cdc-wdm/wwan pairs, and ModemManager detects both with the same modem, you should be able to do the following (e.g. using mmcli commands here and assuming modem is at index 0):
// create bearer for first APN
$ mmcli --modem 0 --create-bearer="apn=first.apn"
// bearer 0 created, now connect
$ mmcli --bearer 0 --connect
// bearer 0 connected, now show bearer info
$ mmcli --bearer 0
// bearer info will tell you which is the data port used, e.g. wwan0
// create bearer for second APN
$ mmcli --modem 0 --create-bearer="apn=second.apn"
// bearer 1 created, now connect
$ mmcli --bearer 1 --connect
// bearer 1 connected, now show bearer info
$ mmcli --bearer 1
// bearer info will tell you which is the data port used, e.g. wwan1
At this point, wwan0 is associated to the “first.apn” network, and wwan1 is associated to the “second.apn” network, and you would be able to setup custom routing rules or whatever.
Note, that you don’t need to take care of special link layer protocol settings, ModemManager does all that for you transparently (when using the dual cdc-wdm/wwan setup in those modems, one of the pairs is usually setup as raw-ip and the other as 802.3).
Also, note that you cannot use NetworkManager to setup the dual APN connection, as NM only uses the Simple interface, it doesn’t create separate bearer objects.
Unfortunately that won’t work. That approach is assuming that you get different modem objects for the two different cdc-wdm/wwan pairs (with a different primary port reported each) and that is not right. You’ll get one single modem object, owning both cdc-wdm/wwan pairs, and still one single cdc-wdm port reported as primary.
There is a nondeterministic on my device regarding which one of wwan0 or wwan1 is used when both of them are available. I would like to figure out whether it is possible get rid of this nondeterminitic. Most of time it is wwan0, but sometimes, not rare at all, wwan1 is used.
By " it will try to use the first available cdc-wdm/wwan pair to setup the connection.", do you mean when cellmodem was reset and probed by kernel usb drivers, whichever USB WDM interface is enumerated first would be manipulated by ModemManager?
As as matter of fact, I was wondering if this could be the case. ModemManager listens and handles uevent and device_added() callback function will handle a new tty / net / cdc-wdm device. In its source code, I see that it seems to save only one device into its ctx->device. I am pretty new to ModemManager source code, if I was right on this point, it seems that it could only handle the first device found on the cellmodem device. So if RMNET1 WDM USB interface is probed first, then wwan1 will be recorded by ModemManager. If this is still correct, then the uncertainty I run into actually arise from the nondeterministic which USB interface on cellmodem is enumerated by Linux kernel.
Another question, I see that the two wwan interfaces are set as raw-ip and 802.3 by default respectively, are both of them used when setting up any data connection? If this is the case, then I could not use AT!USBCOMP command to disable RMNET1 (wwan1 and cdc-wdm1) at all even if there is only one PDN connection is required.
The order of enumeration for the cdc-wdm interfaces in a single device is guaranteed, as they enumeration happens in the order of the USB interfaces. Now, for each enumerated cdc-wdm port, ModemManager performs some “port probing” mechanism, e.g. to truly validate that the cdc-wdm port can speak QMI. If the order of cdc-wdm0/cdc-wdm1 is non-deterministic, my assumption is that the port-probing could cause this; the first cdc-wdm port that finishes probing successfully is the one flagged as primary, and so the first one that will be used for connection.
That said, the “safe” way to handle this issue is to directly avoid fixing the interface names in any program that makes use of the logic provided by MM. Of course it’s always easier to just e.g. add iptables rules on wwan0 and assume the wwan0 is the interface that will be used, but that may not be true always.
One thing ModemManager could do is to, if there are more than one available control ports of the same type and all of them have the same characteristics, do first an alphabetical sort before selecting which is primary. That’s possibly an easy fix for the issue you have at hand, and it would remove the non-deterministic behavior you’re seen (as long as both cdc-wdm ports always probe right, but that’s another thing).
As said earlier, order of USB enumeration in a single device is guaranteed by the kernel enumerating the interfaces in the order of the USB interfaces. Also, MM dos not only handle the first port found in the device, it tracks all ports found in the device, probes all of them (either QMI, AT or MBIM) and then based on the amount of ports it successfully probed, it decides which kind of modem management to use. E.g. a modem that probes QMI and AT ports will be managed in QMI mode preferably. If the same modem runs on a system without qmi_wwan driver, or if for any reason QMI port probing fails, it will be managed in AT mode instead.
When you setup one single data connection, only one of the interfaces will be used. The wwan interfaces have a 1-to-1 relationship with the cdc-wdm ports; so if you connect a WDS client in cdc-wdm0 you’ll need to setup the wwan0 interface, and if you connect a WDS client in cdc-wdm1 you’ll need to setup the wwan1 interface. That’s why you can have 2 separate APNs connected at the same time in different interfaces, and even provide different routing rules for each in your host system, which is very useful.
The fact that one is raw-ip and the other 802.3 doesn’t matter much for ModemManager. If you’re using a recent enough MM and kernel, MM will not use the link layer protocol, it will use the one by default setup in the interface: it will instead tell the kernel which is the link layer protocol in use in the interface. So if you’re
setting up wwan1 which is raw-ip, MM will tell the kernel that the interface is raw-ip; and if you’re setting up wwan0 which is 802.3, MM will tell the kernel the interface is 802.3. The connection logic will also be slightly different for both cases in MM, because if the interface is raw-ip MM will request the upper layers to setup the interface with static IP addressing (MM queries which settings should be configured in the wwan); while if the interface is 802.3, MM requests the upper layers to setup the interface with DHCP instead (as all DHCP clients can work with network interfaces with ethernet headers, unlike network interfaces without them).
Use the network interface name provided in the MM bearer object and make absolutely no assumptions about any relationship with a particular /dev/cdc-wdmX device or anything else. That network interface is what’s connected by that specific bearer object. IP configuration, firewall rules, routing policies, or anything else needing the network device name, should all get it from the MM bearer.
Doing things this way will make things a lot easier when you move on to multiplexed sessions using either MBIM or QMUX. Then you can have a number of bearers referring to dynamically allocated network device names, all controlled by the same /dev/cdc-wdmX device. So there is no one-to-one relationship. Looking forward to this, BTW
Getting the current network interface name from the bearer object will work for sessions multiplexed on the host, sessions multiplexed on the modem and also for multiple modems. I guess we also should be prepared for a new driver generation supporting a PCIe modem interface. These will obviously not have any /dev/cdc-wdmX management device.
If you were to let MM guarantee a specific network device name for a bearer, then I’d prefer that to be an input to the bearer configuration along with APN etc. And then a userspace provided name that did not collide with anything allocated by the kernel/drivers. Similar to the systemd names…
Thanks a lot for Mork and Alexsander’s kind and informative reply, much appreciated!
Now I see that I’d better not to worry which cdc-wdmX control device MM is selected as the primary device, but just care about creating and connecting a bearer, then inquiring it what its bearer-interface is. When the time comes for me to support QMAP multiplexing, I will abandon using the MM’s simple interface.
Yes, currently in my bash scripts, I have inquired the bearer-interface on the fly and in my firewall rules, I simply used “wwan+” which covers wwan0 and wwan1
It makes sense to me MM will probe those found USB interfaces on a cellmodem to figure out their capabilitie. I will read more in this regard to understand how this undeterministic arises from.
