Configuring IPMP in Oracle Solaris 11
In this post, we are going to discuss how we will be configuring IPMP in Oracle Solaris 11. Oracle SPARC Virtual Machine supports link-based IP network multipathing which is known as IPMP with the help of virtual network devices.
Table of Contents
How to Check vswitch configuration on CDOM
# ldm ls -o net secondary NAME secondary MAC 00:14:4f:f9:cf:da VSW NAME MACADDRESS NET-DEV DVID|PVID|VIDs ---- ---------- ------- -------------- secondary-vsw0 00:14:4f:f9:01:8f net6 1|1|-- DEVICE :switch@0 ID :0 LINKPROP :phys-state MTU :1500 INTER-VNET-LINK :on MODE :-- VSW-RELAY-MODE :local How to Check NIC Configuration for the LDOM
Consider the LDOM server name is ldom01 just for demonstration purposes.
# ldm ls -o net ldom01 NAME Ldom01 MAC 00:14:4f:fb:44:8a NETWORK NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet0 primary-vsw0@primary 00:14:4f:fb:c7:4a 1|--|-- DEVICE :network@0 ID :0 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :--
Next, We will have to configure two virtual network devices vnet0 and vnet1 on Control Domain. Every domain is connected to a different virtual switch device on the service domain for using link-based IPMP.
Virtual Network devices get configured to obtain physical link state status. We already have vnet0 so we will modify existing devices via set-vnet command.
# ldm add-vnet linkprop=phys-state vnet1 secondary-vsw0 ldom01 # ldm set-vnet linkprop=phys-state vnet0 ldom01 # ldm ls -o net ldom01 NAME ldom01 MAC 00:14:4f:fb:44:8a NETWORK NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet0 primary-vsw0@primary 00:14:4f:fb:c7:4a 1|--|-- DEVICE :network@0 ID :0 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :-- NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet1 secondary-vsw0@secondary 00:14:4f:fb:4c:d1 1|--|-- DEVICE :network@1 ID :1 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :-- When we complete all the mentioned configurations, then we will proceed with IPMP configuration in the Logical Domain.
We are trying to create an IPMP group and then we will add both virtual nic(vnics) to that IPMP group.
Because Logical Domain has an existing NIC vnet0 (net0), it needs to add the secondary NIC vnet1 (net1) in the IPMP group.
Oracle has a concept of vanity name. In our case, net0 and net1 are the Oracle Solaris 11 vanity names for vnet0 and vnet1.
Check the existing network configuration: On LDOM
root@ldom01:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok — —
net0/v4 static ok — 10.175.176.47/23
root@ldom01:~#
Create additional nic net1 (vnet1).
root@ldom01:~# ipadm create-ip net1 root@ldom01:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 net0 ip ok -- -- net0/v4 static ok -- 10.175.176.47/23 net1 ip down -- -- root@ldom01:~#
Already existing nic net0 has got an IP address configured of LDOM, IP configuration has to be deleted to allow us to configure it under the IPMP group.
Remember connectivity to the LDOM server will be going to lost, therefore, below configurations needs to be done through the LDOM console.
root@ldom01:~# ipadm delete-addr net0/v4 root@ldom01:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 net0 ip down -- -- net1 ip down -- -- root@ldom01:~# Create IPMP group “ipmp0” with nics net0 and net1 grouped into it. Then assign the IP address to the IPMP group.
root@ldom01:~# ipadm create-ipmp ipmp0
root@ldom01:~# ipadm add-ipmp -i net0 -i net1 ipmp0
root@ldom01:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp down — —
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
root@ldom01:~# ipadm create-addr -T static -a 10.175.176.47/23 ipmp0/v4
root@ldom01:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp ok — —
ipmp0/v4 static ok — 10.175.176.47/23
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
root@ldom01:~#
The IPMP configuration is complete. The LDOM server should be accessible on the network now.
To verify the configuration, it is possible to list the NICs and network configuration and compare the MAC address with that of vnetX interfaces provisioned in CDOM for this LDOM server profile.
To list the network configuration:
root@ldom01:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp ok — —
ipmp0/v4 static ok — 10.175.176.47/23
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
root@ldom01:~#
To list the NICs on the LDOM server: root@ldom01:~# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 10000 full vnet0 net1 Ethernet up 10000 full vnet1
That concludes configuring IPMP in Oracle Solaris 11. This will be persistent across reboots. To make sure you can reboot the LDOM server and check configuration after reboot.
