Question : Problem: WTF's up with my iSCSI network config ???

Experts:

I just purchased an EMC AX4-5i dual-SP SAN appliance; two racks, one for SAS drives and the other with SATA drives. I'm just setting up the appliance and I'm stuck, hoping you all can help me figure something out.

If you look at the attached file you'll notice my vanilla setup: 1 server with 3 NICs connected to a pair of GigE switches configured in a meshed network connecting a pair of SP units, each with two iSCSI ports of their own.

The problem I'm having is that on the server i can only ping one of two switches and only two of four iSCSI ports

C:\Program Files\Support Tools>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : galapagos
   Primary Dns Suffix  . . . . . . . : xxx.local
   Node Type . . . . . . . . . . . . : Unknown
   IP Routing Enabled. . . . . . . . : Yes
   WINS Proxy Enabled. . . . . . . . : Yes
   DNS Suffix Search List. . . . . . : xxx.local


Ethernet adapter 192.168.253.98:

   Connection-specific DNS Suffix  . : xxx.local
   Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Server Adapter
   Physical Address. . . . . . . . . : 00-04-23-AB-6A-0B
   DHCP Enabled. . . . . . . . . . . : No
   IP Address. . . . . . . . . . . . : 192.168.253.98
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter 192.168.253.99:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Server Adapter #2
   Physical Address. . . . . . . . . : 00-04-23-AB-6A-0C
   DHCP Enabled. . . . . . . . . . . : No
   IP Address. . . . . . . . . . . . : 192.168.253.99
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter 192.168.10.25:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : HP NC3163 Fast Ethernet NIC
   Physical Address. . . . . . . . . : 00-50-8B-EB-15-1C
   DHCP Enabled. . . . . . . . . . . : No
   IP Address. . . . . . . . . . . . : 192.168.10.25
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.10.1
   DNS Servers . . . . . . . . . . . : 192.168.10.13
                                       192.168.10.25
   Primary WINS Server . . . . . . . : 192.168.10.13
   Secondary WINS Server . . . . . . : 192.168.10.25

C:\Program Files\Support Tools>ping 192.168.253.199

Pinging 192.168.253.199 with 32 bytes of data:

Reply from 192.168.253.199: bytes=32 time=3ms TTL=64
Reply from 192.168.253.199: bytes=32 time=2ms TTL=64
Reply from 192.168.253.199: bytes=32 time=1ms TTL=64
Reply from 192.168.253.199: bytes=32 time=2ms TTL=64

Ping statistics for 192.168.253.199:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 1ms, Maximum = 3ms, Average = 2ms

C:\Program Files\Support Tools>ping 192.168.253.198

Pinging 192.168.253.198 with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.253.198:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Program Files\Support Tools>ping 192.168.253.200

Pinging 192.168.253.200 with 32 bytes of data:

Reply from 192.168.253.200: bytes=32 time<1ms TTL=64
Reply from 192.168.253.200: bytes=32 time<1ms TTL=64
Reply from 192.168.253.200: bytes=32 time<1ms TTL=64
Reply from 192.168.253.200: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.253.200:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Program Files\Support Tools>ping 192.168.253.201

Pinging 192.168.253.201 with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.253.201:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Program Files\Support Tools>ping 192.168.253.202

Pinging 192.168.253.202 with 32 bytes of data:

Reply from 192.168.253.202: bytes=32 time=1ms TTL=64
Reply from 192.168.253.202: bytes=32 time<1ms TTL=64
Reply from 192.168.253.202: bytes=32 time<1ms TTL=64
Reply from 192.168.253.202: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.253.202:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 1ms, Average = 0ms

C:\Program Files\Support Tools>ping 192.168.253.203

Pinging 192.168.253.203 with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.253.203:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Program Files\Support Tools>

So before I go any further and start configuring iSCSI initiators or LUNs, etc...I wanted to clear up this networking mystery


Thanks,
juckyt

Answer : Problem: WTF's up with my iSCSI network config ???

Remove the default gateway from the 167.x.x.x card
If there are multiple subnets on the 167.x.x.x side that you need to reach, add them manually with the route add command

f.ex
if you have ip 167.10.10.11 mask 255.255.255.0 on your machine. All machines in 167.10.10.0 will be reachable even if you don't supply a default gateway. If there is also a 167.10.11.0 network you need to reach: (assuming gateway/router is 167.10.10.1)

route add 167.10.11.0 mask 255.255.255.0 167.10.10.1

It will disappear if you reboot - route add -p makes it permanent...
route print shows you your current routes, route del to remove.

You can add multiple gateways, but unless the gateway with the lowest metric is down, it will always be used...and if it's down, you will need to wait for timeouts.... You should therefore only have one default gateway. Two gateways is possible for failover reasons, and nothing else.