Configuring IPMP in Oracle Solaris 11
In this post, we are going to discuss how we will be configuring IPMP in Oracle Solaris 11. Oracle SPARC Virtual Machine supports link-based IP network multipathing which is known as IPMP with the help of virtual network devices.
How to Check vswitch configuration on CDOM
# ldm ls -o net secondary NAME secondary MAC 00:14:4f:f9:cf:da VSW NAME MACADDRESS NET-DEV DVID|PVID|VIDs ---- ---------- ------- -------------- secondary-vsw0 00:14:4f:f9:01:8f net6 1|1|-- DEVICE :[email protected] ID :0 LINKPROP :phys-state MTU :1500 INTER-VNET-LINK :on MODE :-- VSW-RELAY-MODE :local How to Check NIC Configuration for the LDOM
Consider the LDOM server name is ldom01 just for demonstration purposes.
# ldm ls -o net ldom01 NAME Ldom01 MAC 00:14:4f:fb:44:8a NETWORK NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet0 [email protected] 00:14:4f:fb:c7:4a 1|--|-- DEVICE :[email protected] ID :0 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :--
Next, We will have to configure two virtual network devices vnet0 and vnet1 on Control Domain. Every domain is connected to a different virtual switch device on the service domain for using link-based IPMP.
Virtual Network devices get configured to obtain physical link state status. We already have vnet0 so we will modify existing devices via set-vnet command.
# ldm add-vnet linkprop=phys-state vnet1 secondary-vsw0 ldom01 # ldm set-vnet linkprop=phys-state vnet0 ldom01 # ldm ls -o net ldom01 NAME ldom01 MAC 00:14:4f:fb:44:8a NETWORK NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet0 [email protected] 00:14:4f:fb:c7:4a 1|--|-- DEVICE :[email protected] ID :0 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :-- NAME SERVICE MACADDRESS PVID|PVLAN|VIDs ---- ------- ---------- --------------- vnet1 [email protected] 00:14:4f:fb:4c:d1 1|--|-- DEVICE :[email protected] ID :1 LINKPROP :phys-state MTU :1500 MAXBW :-- MODE :-- CUSTOM :disable PRIORITY :-- COS :-- PROTECTION :-- When we complete all the mentioned configurations, then we will proceed with IPMP configuration in the Logical Domain.
We are trying to create an IPMP group and then we will add both virtual nic(vnics) to that IPMP group.
Because Logical Domain has an existing NIC vnet0 (net0), it needs to add the secondary NIC vnet1 (net1) in the IPMP group.
Oracle has a concept of vanity name. In our case, net0 and net1 are the Oracle Solaris 11 vanity names for vnet0 and vnet1.
Check the existing network configuration: On LDOM
[email protected]:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok — —
net0/v4 static ok — 10.175.176.47/23
[email protected]:~#
Create additional nic net1 (vnet1).
[email protected]:~# ipadm create-ip net1 [email protected]:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 net0 ip ok -- -- net0/v4 static ok -- 10.175.176.47/23 net1 ip down -- -- [email protected]:~#
Already existing nic net0 has got an IP address configured of LDOM, IP configuration has to be deleted to allow us to configure it under the IPMP group.
Remember connectivity to the LDOM server will be going to lost, therefore, below configurations needs to be done through the LDOM console.
[email protected]:~# ipadm delete-addr net0/v4 [email protected]:~# ipadm NAME CLASS/TYPE STATE UNDER ADDR lo0 loopback ok -- -- lo0/v4 static ok -- 127.0.0.1/8 lo0/v6 static ok -- ::1/128 net0 ip down -- -- net1 ip down -- -- [email protected]:~# Create IPMP group “ipmp0” with nics net0 and net1 grouped into it. Then assign the IP address to the IPMP group.
[email protected]:~# ipadm create-ipmp ipmp0
[email protected]:~# ipadm add-ipmp -i net0 -i net1 ipmp0
[email protected]:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp down — —
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
[email protected]:~# ipadm create-addr -T static -a 10.175.176.47/23 ipmp0/v4
[email protected]:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp ok — —
ipmp0/v4 static ok — 10.175.176.47/23
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
[email protected]:~#
The IPMP configuration is complete. The LDOM server should be accessible on the network now.
To verify the configuration, it is possible to list the NICs and network configuration and compare the MAC address with that of vnetX interfaces provisioned in CDOM for this LDOM server profile.
To list the network configuration:
[email protected]:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp ok — —
ipmp0/v4 static ok — 10.175.176.47/23
lo0 loopback ok — —
lo0/v4 static ok — 127.0.0.1/8
lo0/v6 static ok — ::1/128
net0 ip ok ipmp0 —
net1 ip ok ipmp0 —
[email protected]:~#
To list the NICs on the LDOM server: [email protected]:~# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 10000 full vnet0 net1 Ethernet up 10000 full vnet1
That concludes configuring IPMP in Oracle Solaris 11. This will be persistent across reboots. To make sure you can reboot the LDOM server and check configuration after reboot.