Management Network Switch Rename

Any system moving from Shasta v1.3 to Shasta v1.4 software needs to adjust the hostnames and IP addresses for all switches to match the new standard. There is now a virtual IP ending in .1 which is used by spine switches. In Shasta v1.3, the first spine switch used the .1 address. In Shasta v1.4, the ordering of the switches has changed with spine switches being grouped first. The hostname for switches has changed from two digits to a dash and then 3 digits. All IPv4 data for the switches and switch naming comes from Cray Site Init (CSI).

From v1.3, this example system had these IP addresses and hostnames on the HMN network. Similar names and IP address numbers for the NMN and CAN networks as well. Also note that some systems may not have an agg or aggregation set of switches. This is ok. Simply follow the directions below, always paying attention to the CSI specified IPv4 addresses and skip the aggregation switch directions.

10.1.0.1        sw-spine01
10.1.0.2        sw-leaf01
10.1.0.3        sw-spine02
10.1.0.4        sw-leaf02
10.1.0.5        sw-agg01
10.1.0.6        sw-agg02
10.1.0.7        sw-cdu01
10.1.0.8        sw-cdu02

The desired settings for the HMN network would be more like these.

10.1.0.2        sw-spine-001
10.1.0.3        sw-spine-002
10.1.0.4        sw-agg-001
10.1.0.5        sw-agg-002
10.1.0.6        sw-leaf-001
10.1.0.7        sw-leaf-002
10.1.0.8        sw-cdu-001
10.1.0.9        sw-cdu-002

This system needs to do the renames in this order: do CDU switches (8 to 9, 7 to 8) and then leaf switches (4 to 7, but also 2 to 6), then aggregation switches (6 to 5, 5 to 4) and spine switches (3 to 3, and 1 to 2). These have IP address changes and name changes since we now have 3 digits instead of 2 for the switch hostname.

  1. Check switch IP addresses, names, and component names in /var/www/ephemeral/prep/${SYSTEM_NAME}/networks when booted from the LiveCD on ncn-m001.

    pit# export SYSTEM_NAME=eniac
    pit# cd /var/www/ephemeral/prep/${SYSTEM_NAME}/networks
    pit# vi NMN.yaml
    

    Excerpt from NMN.yaml

      ip_reservations:
      - ip_address: 10.252.0.2
        name: sw-spine-001
        comment: x3000c0h33s1
        aliases: []
      - ip_address: 10.252.0.3
        name: sw-spine-002
        comment: x3000c0h34s1
        aliases: []
      - ip_address: 10.252.0.4
        name: sw-cdu-001
        comment: d0w1
        aliases: []
      - ip_address: 10.252.0.5
        name: sw-cdu-002
        comment: d0w2
        aliases: []
      - ip_address: 10.252.0.6
        name: sw-leaf-001
        comment: x3000c0w38
        aliases: []
      - ip_address: 10.252.0.7
        name: sw-leaf-002
        comment: x3000c0w36
        aliases: []
    
    pit# vi HMN.yaml
    

    Excerpt from HMN.yaml

      ip_reservations:
      - ip_address: 10.254.0.2
        name: sw-spine-001
        comment: x3000c0h33s1
        aliases: []
      - ip_address: 10.254.0.3
        name: sw-spine-002
        comment: x3000c0h34s1
        aliases: []
      - ip_address: 10.254.0.4
        name: sw-cdu-001
        comment: d0w1
        aliases: []
      - ip_address: 10.254.0.5
        name: sw-cdu-002
        comment: d0w2
        aliases: []
      - ip_address: 10.254.0.6
        name: sw-leaf-001
        comment: x3000c0w38
        aliases: []
      - ip_address: 10.254.0.7
        name: sw-leaf-002
        comment: x3000c0w36
        aliases: []
    

    pit# vi MTL.yaml

    
    Excerpt from MTL.yaml
    

    ip_reservations:

    • ip_address: 10.1.0.2 name: sw-spine-001 comment: x3000c0h33s1 aliases: []
    • ip_address: 10.1.0.3 name: sw-spine-002 comment: x3000c0h34s1 aliases: []
    • ip_address: 10.1.0.4 name: sw-cdu-001 comment: d0w1 aliases: []
    • ip_address: 10.1.0.5 name: sw-cdu-002 comment: d0w2 aliases: []
    • ip_address: 10.1.0.6 name: sw-leaf-001 comment: x3000c0w38 aliases: []
    • ip_address: 10.1.0.7 name: sw-leaf-002 comment: x3000c0w36 aliases: []

    pit# vi CAN.yaml

    
    Excerpt from CAN.yaml showing the two spine switches. Most v1.3 systems would have had these as ending in .1 and in .3. Note these switches are not named "spine" or "agg" since the SHCD may specify differing exit points, but with either option the IPv4 address is specified.
    

    ip_reservations:

    • ip_address: 10.103.8.2 name: can-switch-1 comment: "" aliases: []
    • ip_address: 10.103.8.3 name: can-switch-2 comment: "" aliases: []
  2. Save the running-config from all switches before you start.

    Note: Mellanox switches require the “enable” command before doing “show running-config”

    pit# ssh admin@10.1.0.1 switch# show running-config switch# exit

    Save this information in a text file for later evaluation and comparison after all changes have been made. pit# vi before.sw-spine01.txt

    Repeat this for all of the switches. The example system has switches up to 10.1.0.8.

  3. Start moves with the highest numbered switch. In this case, that is sw-cdu02. Said another way, if a switch is in a pair, start with the second half of the pair (i.e.. 2 of 2).

    Move sw-cdu02 to sw-cdu-002 and increase IP addresses as specified in CSI output. It is a Dell switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2 and vlan 4 on a CDU switch.)

    pit# ssh admin@10.1.0.6
    sw-cdu02# configure terminal
    sw-cdu02(config)# hostname sw-cdu-002
    sw-cdu-002(config)# interface vlan2
    sw-cdu-002(conf-if-vl-2)# ip address 10.252.0.7/17
    sw-cdu-002(conf-if-vl-2)# interface vlan4
    sw-cdu-002(conf-if-vl-4)# ip address 10.254.0.7/17
    sw-cdu-002(conf-if-vl-4)# router ospf 1
    sw-cdu-002(config-router-ospf-1)# router-id 10.252.0.7
    sw-cdu-002(config-router-ospf-1)# exit
    sw-cdu-002(config)# exit
    sw-cdu-002# write memory
    

Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

sw-cdu-002# exit
pit# ssh admin@10.252.0.7
sw-cdu-002# configure terminal
sw-cdu-002(config)# interface vlan1
sw-cdu-002(conf-if-vl-1)# ip address 10.1.0.7/16
sw-cdu-002(conf-if-vl-1)# exit
sw-cdu-002(config)# exit
sw-cdu-002# write memory
sw-cdu-002# exit
pit#
  1. Move sw-cdu01 to sw-cdu-001 and increase IP addresses as specified in CSI output. It is a Dell switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2 and vlan 4 on a CDU switch.)

    pit# ssh admin@10.1.0.5
    sw-cdu01# configure terminal
    sw-cdu01(config)# hostname sw-cdu-001
    sw-cdu-001(config)# interface vlan2
    sw-cdu-001(conf-if-vl-2)# ip address 10.252.0.6/17
    sw-cdu-001(conf-if-vl-2)# interface vlan4
    sw-cdu-001(conf-if-vl-4)# ip address 10.254.0.6/17
    sw-cdu-001(conf-if-vl-4)# router ospf 1
    sw-cdu-001(config-router-ospf-1)# router-id 10.252.0.6
    sw-cdu-001(config-router-ospf-1)# exit
    sw-cdu-001(config)# exit
    sw-cdu-001# write memory
    

    Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

    sw-cdu-001# exit
    pit# ssh admin@10.252.0.6
    sw-cdu-001# configure terminal
    sw-cdu-001(config)# interface vlan1
    sw-cdu-001(conf-if-vl-1)# ip address 10.1.0.6/16
    sw-cdu-001(conf-if-vl-1)# exit
    sw-cdu-001(config)# exit
    sw-cdu-001# write memory
    sw-cdu-001# exit
    pit#
    
  2. Move sw-leaf02 to sw-leaf-002 and increase IP addresses as specified in CSI output. It is a Dell switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2, vlan 4, vlan 7, vlan 10) on a leaf switch.

    pit# ssh admin@10.1.0.4
    sw-leaf02# configure terminal
    sw-leaf02(config)# hostname sw-leaf-002
    sw-leaf-002(config)# interface vlan2
    sw-leaf-002(conf-if-vl-2)# ip address 10.252.0.5/17
    sw-leaf-002(conf-if-vl-2)# interface vlan4
    sw-leaf-002(conf-if-vl-4)# ip address 10.254.0.5/17
    sw-leaf-002(conf-if-vl-4)# interface vlan7
    sw-leaf-002(conf-if-vl-7)# ip address 10.103.8.5/24
    sw-leaf-002(conf-if-vl-7)# interface vlan10
    sw-leaf-002(conf-if-vl-10)# ip address 10.11.0.5/16
    sw-leaf-002(conf-if-vl-10)# router ospf 1
    sw-leaf-002(config-router-ospf-1)# router-id 10.252.0.5
    sw-leaf-002(config-router-ospf-1)# exit
    sw-leaf-002(config)# exit
    sw-leaf-002# write memory
    

    Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

    sw-leaf-002# exit
    pit# ssh admin@10.252.0.5
    sw-leaf-002# configure terminal
    sw-leaf-002(config)# interface vlan1
    sw-leaf-002(conf-if-vl-1)# ip address 10.1.0.5/16
    sw-leaf-002(conf-if-vl-1)# exit
    sw-leaf-002(config)# exit
    sw-leaf-002# write memory
    sw-leaf-002# exit
    pit#
    
  3. Move sw-leaf01 to sw-leaf-001 and increase IP addresses as specified in CSI output. It is a Dell switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2, vlan 4, vlan 7, vlan 10) on a leaf switch.

    pit# ssh admin@10.1.0.2
    sw-leaf01# configure terminal
    sw-leaf01(config)# hostname sw-leaf-001
    sw-leaf-001(config)# interface vlan2
    sw-leaf-001(conf-if-vl-2)# ip address 10.252.0.4/17
    sw-leaf-001(conf-if-vl-2)# interface vlan4
    sw-leaf-001(conf-if-vl-4)# ip address 10.254.0.4/17
    sw-leaf-001(conf-if-vl-4)# interface vlan7
    sw-leaf-001(conf-if-vl-7)# ip address 10.103.8.4/24
    sw-leaf-001(conf-if-vl-7)# interface vlan10
    sw-leaf-001(conf-if-vl-10)# ip address 10.11.0.4/16
    sw-leaf-001(conf-if-vl-10)# router ospf 1
    sw-leaf-001(config-router-ospf-1)# router-id 10.252.0.4
    sw-leaf-001(config-router-ospf-1)# exit
    sw-leaf-001(config)# exit
    sw-leaf-001# write memory
    

    Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

    sw-leaf-001# exit
    pit# ssh admin@10.252.0.4
    sw-leaf-001# configure terminal
    sw-leaf-001(config)# interface vlan1
    sw-leaf-001(conf-if-vl-1)# ip address 10.1.0.4/16
    sw-leaf-001(conf-if-vl-1)# exit
    sw-leaf-001(config)# exit
    sw-leaf-001# write memory
    sw-leaf-001# exit
    pit#
    
  4. Move sw-spine02 to sw-spine-002. It already has the .3 IP address so does not need to change. It is a Mellanox switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2, vlan 4, vlan 7, vlan 10) on a leaf switch.

    Note: The addresses for the spine switches in CAN.yaml shown above (ip_address: 10.103.8.2, name: can-switch-1) and (ip_address: 10.103.8.3, name: can-switch-2) were shown on the 10.103.8.0 subnet so the virtual-router address on interface vlan7 magp 7 should be 10.103.8.1.

    pit# ssh admin@10.1.0.3
    sw-spine02> enable
    sw-spine02# configure terminal
    sw-spine02(config)# hostname sw-spine-002
    sw-spine02(config)# no protocol magp
    sw-spine-002(config)# interface vlan 1 ip address 10.1.0.3/16 primary
    sw-spine-002(config)# interface vlan 2 ip address 10.252.0.3/17 primary
    sw-spine-002(config)# interface vlan 4 ip address 10.254.0.3/17 primary
    sw-spine-002(config)# interface vlan 7 ip address 10.103.8.3/24 primary
    sw-spine-002(config)# interface vlan 10 ip address 10.11.0.3/16 primary
    sw-spine-002(config)# router bgp 65533 vrf default router-id 10.252.0.3 force
    sw-spine-002(config)# router ospf 1 vrf default router-id 10.252.0.3
    sw-spine-002(config)# exit
    sw-spine-002# write memory
    

    Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

    sw-spine-002# exit
    pit# ssh admin@10.252.0.3
    sw-spine-002 [standalone: master] > enable
    sw-spine-002 [standalone: master] # configure terminal
    sw-spine-002 [standalone: master] (config) # no protocol magp
    sw-spine-002 [standalone: master] (config) # protocol magp
    sw-spine-002 [standalone: master] (config) # interface vlan 1
    sw-spine-002 [standalone: master] (config interface vlan 1) # no ip address
    sw-spine-002 [standalone: master] (config interface vlan 1) # ip address 10.1.0.3/16 primary
    sw-spine-002 [standalone: master] (config interface vlan 1) # exit
    sw-spine-002 [standalone: master] (config) #    interface vlan 1 magp 1
    sw-spine-002 [standalone: master] (config interface vlan 1 magp 1) # ip virtual-router address 10.1.0.1
    sw-spine-002 [standalone: master] (config interface vlan 1 magp 1) # ip virtual-router mac-address 00:00:5E:00:01:01
    sw-spine-002 [standalone: master] (config interface vlan 1 magp) # exit
    sw-spine-002 [standalone: master] (config) #    interface vlan 2 magp 2
    sw-spine-002 [standalone: master] (config interface vlan 2 magp 2) # ip virtual-router address 10.252.0.1
    sw-spine-002 [standalone: master] (config interface vlan 2 magp 2) # ip virtual-router mac-address 00:00:5E:00:01:02
    sw-spine-002 [standalone: master] (config interface vlan 2 magp 2) # exit
    sw-spine-002 [standalone: master] (config) #    interface vlan 4 magp 4
    sw-spine-002 [standalone: master] (config interface vlan 4 magp 4) # ip virtual-router address 10.254.0.1
    sw-spine-002 [standalone: master] (config interface vlan 4 magp 4) # ip virtual-router mac-address 00:00:5E:00:01:04
    sw-spine-002 [standalone: master] (config interface vlan 4 magp 4) # exit
    sw-spine-002 [standalone: master] (config) #    interface vlan 7 magp 7
    sw-spine-002 [standalone: master] (config interface vlan 7 magp 7) # ip virtual-router address 10.103.8.1
    sw-spine-002 [standalone: master] (config interface vlan 7 magp 7) # ip virtual-router mac-address 00:00:5E:00:01:07
    sw-spine-002 [standalone: master] (config interface vlan 7 magp 7) # exit
    sw-spine-002 [standalone: master] (config) #    interface vlan 10 magp 10
    sw-spine-002 [standalone: master] (config interface vlan 10 magp 10) # ip virtual-router address 10.11.0.1
    sw-spine-002 [standalone: master] (config interface vlan 10 magp 10) # ip virtual-router mac-address 00:00:5E:00:01:10
    sw-spine-002 [standalone: master] (config interface vlan 10 magp 10) # exit
    sw-spine-002 [standalone: master] (config) # exit
    sw-spine-002 [standalone: master] # write memory
    sw-spine-002 [standalone: master] # exit
    pit#
    
  5. Move sw-spine01 to sw-spine-001 and increase IP addresses as specified in CSI output. It is a Mellanox switch.

    Note: You can change many addresses in a single session, but not the one you used to connect. This first connection will skip vlan 1 and change all of the other vlans (vlan 2, vlan 4, vlan 7, vlan 10) on a leaf switch.

    Note: The addresses for the spine switches in CAN.yaml shown above (ip_address: 10.103.8.2, name: can-switch-1) and (ip_address: 10.103.8.3, name: can-switch-2) were shown on the 10.103.8.0 subnet so the virtual-router address on interface vlan7 magp 7 should be 10.103.8.1.

    pit# ssh admin@10.1.0.1
    sw-spine01> enable
    sw-spine01# configure terminal
    sw-spine01(config)# hostname sw-spine-001
    sw-spine-001(config)# no protocol magp
    sw-spine-001(config)# interface vlan 2
    sw-spine-001 [standalone: master] (config interface vlan 2) # no ip address
    sw-spine-001 [standalone: master] (config interface vlan 2) # ip address 10.252.0.2/17 primary
    sw-spine-001 [standalone: master] (config interface vlan 2) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 4
    sw-spine-001 [standalone: master] (config interface vlan 4) # no ip address
    sw-spine-001 [standalone: master] (config interface vlan 4) # ip address 10.254.0.2/17 primary
    sw-spine-001 [standalone: master] (config interface vlan 4) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 7
    sw-spine-001 [standalone: master] (config interface vlan 7) # no ip address
    sw-spine-001 [standalone: master] (config interface vlan 7) # ip address 10.103.8.2/24 primary
    sw-spine-001 [standalone: master] (config interface vlan 7) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 10
    sw-spine-001 [standalone: master] (config interface vlan 10) # no ip address
    sw-spine-001 [standalone: master] (config interface vlan 10) # ip address 10.11.0.2/16 primary
    sw-spine-001 [standalone: master] (config interface vlan 10) # exit
    sw-spine-001 [standalone: master] (config) # router bgp 65533 vrf default router-id 10.252.0.2 force
    sw-spine-001 [standalone: master] (config) # router ospf 1 vrf default router-id 10.252.0.2
    sw-spine-001 [standalone: master] (config) # exit
    sw-spine-001 [standalone: master] # write memory
    

    Logout of the switch and return using the new IP address for vlan 2 so that vlan 1 can be corrected.

    sw-spine-001# exit
    pit# ssh admin@10.252.0.2
    sw-spine-001 [standalone: master] > enable
    sw-spine-001 [standalone: master] # configure terminal
    sw-spine-001 [standalone: master] (config) # no protocol magp
    sw-spine-001 [standalone: master] (config) # protocol magp
    sw-spine-001 [standalone: master] (config) # interface vlan 1
    sw-spine-001 [standalone: master] (config interface vlan 1) # no ip address
    sw-spine-001 [standalone: master] (config interface vlan 1) # ip address 10.1.0.2/16 primary
    sw-spine-001 [standalone: master] (config interface vlan 1) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 1 magp 1
    sw-spine-001 [standalone: master] (config interface vlan 1 magp 1) # ip virtual-router address 10.1.0.1
    sw-spine-001 [standalone: master] (config interface vlan 1 magp 1) # ip virtual-router mac-address 00:00:5E:00:01:01
    sw-spine-001 [standalone: master] (config interface vlan 1 magp 1) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 2 magp 2
    sw-spine-001 [standalone: master] (config interface vlan 2 magp 2) # ip virtual-router address 10.252.0.1
    sw-spine-001 [standalone: master] (config interface vlan 2 magp 2) # ip virtual-router mac-address 00:00:5E:00:01:02
    sw-spine-001 [standalone: master] (config interface vlan 2 magp 2) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 4 magp 4
    sw-spine-001 [standalone: master] (config interface vlan 4 magp 4) # ip virtual-router address 10.254.0.1
    sw-spine-001 [standalone: master] (config interface vlan 4 magp 4) # ip virtual-router mac-address 00:00:5E:00:01:04
    sw-spine-001 [standalone: master] (config interface vlan 4 magp 4) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 7 magp 7
    sw-spine-001 [standalone: master] (config interface vlan 7 magp 7) # ip virtual-router address 10.103.8.1
    sw-spine-001 [standalone: master] (config interface vlan 7 magp 7) # ip virtual-router mac-address 00:00:5E:00:01:07
    sw-spine-001 [standalone: master] (config interface vlan 7 magp 7) # exit
    sw-spine-001 [standalone: master] (config) # interface vlan 10 magp 10
    sw-spine-001 [standalone: master] (config interface vlan 10 magp 10) # ip virtual-router address 10.11.0.1
    sw-spine-001 [standalone: master] (config interface vlan 10 magp 10) # ip virtual-router mac-address 00:00:5E:00:01:10
    sw-spine-001 [standalone: master] (config interface vlan 10 magp 10) # exit
    sw-spine-001 [standalone: master] (config) # exit
    sw-spine-001 [standalone: master] # write memory
    sw-spine-001 [standalone: master] # exit
    pit#
    
  6. Save the running-config from all switches after completion

    Note: Mellanox switches require the “enable” command before doing “show running-config”

    pit# ssh admin@10.1.0.2
    switch# show running-config
    switch# exit
    

    Save this information in a text file for comparison with the running-config saved before all changes were made. pit# vi after.sw-spine01.txt

    Repeat this for all of the switches. The example system has switches up to 10.1.0.7.

There are other changes needed, as described in Dell and Mellanox Changes for Shasta v1.3 to v1.4 Upgrades