Solaris 11- How to Configure IPMP – Probe-Based ?

Lingesh

10 years ago

When your system required to handle huge amount of traffic, then you may need to combine one or more physical interfaces in to logical interfaces in various methods.1.Link Aggregation 2.Configuring IPMP. IPMP methods works in two methods to detect the failures. If the system is configured with default router, then it checks the connectivity with it.Otherwise it works in multicast mode by checking the connectivity with near by node.

Here we will see how to configure probe-based IPMP. The IPMP group can be configured with active-active interfaces or active-standby interfaces. IPMP provides network availability during interface failures.As you know Solaris 11 has changed lot from Solaris 10 and configuring IPMP also has no exception from that. In Solaris 11, you need to use “ipadm” command to configure the IPMP.

We will also see how to enable the transitive probing for oracle Solaris cluster in the end of the article.

1. Login to Solaris 11 host and make sure two or more than physical interfaces available.

root@UnixArena:~# dladm show-phys
LINK              MEDIA                STATE      SPEED  DUPLEX    DEVICE
net0              Ethernet             up         1000   full      e1000g0
net1              Ethernet             up         1000   full      e1000g1
net2              Ethernet             unknown    0      unknown   e1000g2
net3              Ethernet             unknown    0      unknown   e1000g3
root@UnixArena:~#

2.Here we are going to use net0 and net1 to configure the IPMP group.So make there is no IP exists on net0 & net1.If exists,remove it using ” ipadm delete-addr net0/XXXX”

root@Unixarena1:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
   lo0/v6         static     ok           --         ::1/128
net0              ip         ok           --         --
   net0/pubip     static     ok           --         192.168.2.32/24
net1              ip         down         --         --

3.Here IP exists on net0. Let me remove it from console.

root@Unixarena1:~# ipadm delete-addr net0/pubip
root@Unixarena1:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
   lo0/v6         static     ok           --         ::1/128
net0              ip         down         --         --
net1              ip         down         --         --

4.Now we are good to start configuring the IPMP. Let me create a new IPMP group.

root@UnixArena:~# ipadm create-ipmp ipmp0
root@UnixArena:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       down         --         --
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
   lo0/v6         static     ok           --         ::1/128
net0              ip         ok           --         --
net1              ip         ok           --         --
root@UnixArena:~#

5.Plumb the interface to configure IP if not already exists in ipadm list.

root@UnixArena:~# ipadm create-ip net0
ipadm: cannot create interface net0: Interface already exists
root@UnixArena:~# ipadm create-ip net1
ipadm: cannot create interface net1: Interface already exists
root@UnixArena:~#

6.Associate the network interfaces to ipmp group which we have created in step number 4. IP address has taken from /etc/hosts file with entry of UnixArena.

root@UnixArena:~# ipadm add-ipmp -i net0 -i net1 ipmp0
root@UnixArena:~# ipadm create-addr -T static -a UnixArena/24 ipmp0/pubip
root@UnixArena:~# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       ok           --         --
   ipmp0/pubip    static     ok           --         192.168.2.31/24
lo0               loopback   ok           --         --
   lo0/v4         static     ok           --         127.0.0.1/8
   lo0/v6         static     ok           --         ::1/128
net0              ip         ok           ipmp0      --
net1              ip         ok           ipmp0      --
root@UnixArena:~#
root@UnixArena:~# cat /etc/hosts |grep UnixArena
192.168.2.31    UnixArena
root@UnixArena:~#

7.For Active-Active IPMP configuration, no further changes needed. The default configuration works as Active-Active model.If the standby property is set to off,then its active-active model. You can verify the settings using below commands.

root@UnixArena:~#  ipadm show-ifprop -p standby
IFNAME      PROPERTY        PROTO PERM CURRENT    PERSISTENT DEFAULT    POSSIBLE
lo0         standby         ip    rw   off        --         off        on,off
ipmp0       standby         ip    rw   off        --         off        on,off
net0        standby         ip    rw   off        --         off        on,off
net1        standby         ip    rw   off        off        off        on,off
root@UnixArena:~#
root@UnixArena:~#  ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net0        yes     ipmp0       --mbM--   up        ok        unknown
net1        yes     ipmp0       -------   up        ok        ok
root@UnixArena:~#

8.For active-standby setup, you need to modify the interface property.Here my standby interface is net1.

root@UnixArena:~# ipadm set-ifprop -p standby=on  -m ip net1
root@UnixArena:~# ipadm show-ifprop -p standby net1
IFNAME      PROPERTY        PROTO PERM CURRENT    PERSISTENT DEFAULT    POSSIBLE
net1        standby         ip    rw   on         on         off        on,off
root@UnixArena:~#
root@UnixArena:~#  ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net0        yes     ipmp0       --mbM--   up        ok        unknown
net1        no      ipmp0       is-----   up        ok        ok
root@UnixArena:~#

9.Its time to test the IPMP setup. Disable one interface and check whether the standby interface is talking load of primary or not .Before proceeding to disable , make sure both interfaces are active in IPMP level.

root@UnixArena:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net1        no      ipmp0       is-----   up        disabled  ok
net0        yes     ipmp0       --mbM--   up        disabled  ok
root@UnixArena:~#

As per the above output, one interface is multicast mode and other one in transitive mode.None of the interface in disabled mode. So it’s good begin the test.

10.Disable the net0 and check the status.

root@UnixArena:~# ipadm disable-if -t net0
root@UnixArena:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net1        yes     ipmp0       -smbM--   up        ok        unknown
root@UnixArena:~# ipmpstat -a
ADDRESS                   STATE  GROUP       INBOUND     OUTBOUND
::                        down   ipmp0       --          --
UnixArena                 up     ipmp0       net1        net1
root@UnixArena:~#

Awesome …Its working fine.

11.Enable the net0 back.

root@UnixArena:~# ipadm enable-if -t net0
root@UnixArena:~# ipmpstat -a
ADDRESS                   STATE  GROUP       INBOUND     OUTBOUND
::                        down   ipmp0       --          --
UnixArena                 up     ipmp0       net0        net0
root@UnixArena:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net0        yes     ipmp0       --mbM--   up        ok        unknown
net1        no      ipmp0       is-----   up        ok        ok

Super stuff..System back to normal with two interfaces on IPMP.
That’s it.We have successfully configured the probe based IPMP and tested.

If you are going to use oracle Solaris cluster then we need to enable transitive probing.Here we will see how to enable transitive probing.
1.Check the existing transitive value,By default it will be false.

root@UnixArena:~#  svccfg -s svc:/network/ipmp listprop config/transitive-probing
config/transitive-probing boolean     false
root@UnixArena:~# ipmpstat -p
ipmpstat: probe-based failure detection is disabled
root@UnixArena:~#

2.Modify the transitive probing to true.

root@UnixArena:~# svccfg -s svc:/network/ipmp setprop config/transitive-probing=true
root@UnixArena:~#  svccfg -s svc:/network/ipmp listprop config/transitive-probing
config/transitive-probing boolean     true

3.Re-load the IPMP configuration by refreshing SMF service.

root@UnixArena:~# svcadm refresh svc:/network/ipmp:default
root@UnixArena:~# svcs svc:/network/ipmp:default
STATE          STIME    FMRI
online         23:08:28 svc:/network/ipmp:default
root@UnixArena:~#
root@UnixArena:~# ipmpstat -p
TIME      INTERFACE   PROBE  NETRTT    RTT       RTTAVG    TARGET
0.94s     net1        t4     0.58ms    0.59ms    0.55ms    
2.35s     net1        t5     0.51ms    0.52ms    0.55ms    
3.76s     net1        t6     0.47ms    0.47ms    0.54ms    
4.72s     net1        t7     0.53ms    0.53ms    0.54ms    
6.69s     net1        t8     0.50ms    0.50ms    0.53ms    
^C
root@UnixArena:~#

As per the above output,you can see probe-based failure detection is enabled.

Thank you for visiting UnixArena. Please leave a comment if you have any doubt on this.