Tuesday, December 9, 2008

Connect a Sun Solaris SPARC box to HP storage

My experience with SUN Sparc servers has been pretty good except for the fact that Sun charges $$ for anything which does not qualify as support and hence does not fall under master support contract.

I was once setting up a SPARC box V4** series and wanted to connect it to Sun Storage. I was told by Sun that it would cost us to connect the server to storage, as it was a new addition and not an existing support, fair enough. So instead we decided to connect it to another HP storage frame in our data center. Here are the detailed activities, which I did in order to connect the SUN box to HP frame.

1. Shutdown the box.
2. Insert the HBA card into the box.
3. Reboot the server.
4. Install the HBA packages and SUN packages for Storage management (SUNWqlc, SUNWqlcx, SUNWsmapi, SUNWssadx, SUNWssaop, OB2-CC, OB2-CORE, OB2-DA) available online and are opensource.
5. Install SFS_base_packages.S8 for Solaris 8 storage management.
6. After installing the packages, the server needs to be rebooted to load the libraries for HBA card detection.

7. Once the server is up, run the following commands to find out the path to HBA:-

bash-2.03# luxadm -e port

Found path to 5 HBA ports

/devices/pci@8,700000/SUNW,qlc@3/fp@0,0:devctl NOT CONNECTED
/devices/pci@8,700000/SUNW,qlc@3,1/fp@0,0:devctl NOT CONNECTED
/devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl CONNECTED***
/devices/pci@8,600000/SUNW,qlc@1,1/fp@0,0:devctl NOT CONNECTED
/devices/pci@9,600000/SUNW,qlc@2/fp@0,0:devctl CONNECTED***

*** Indicates the HBA device ports available on the system, rest are the open PCI slots with no controller card.

Now confirm that the CONNECTED paths are HBA cards:-

# luxadm -e dump_map /devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 10e00 0 50001fe15000f3d8 50001fe15000f3d0 0xc (Array controller device)
1 10000 0 210000e08b07d5b4 200000e08b07d5b4 0x1f (Unknown Type,Host Bus Adapter)

Now check the dynamic configured devices

# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 CD-ROM connected configured unknown
c1 fc-private connected configured unknown
c1::500000e01129ff41 disk connected configured unknown
c1::500000e0112a0281 disk connected configured unknown
c2 fc connected unconfigured unknown
c3 fc connected unconfigured unknown
c4 fc-fabric connected configured unknown
c4::50001fe15000f3d8 array-ctrl connected unconfigured unknown
c5 fc connected unconfigured unknown

If the cfgadm command shows the HBA on controller 4 as unconfigured, connect the HBA to SAN through fiber cable, and run the command "cfgadm -o force_update -c configure c5" to configure the HBA channel. The good thing is that at this point you don’t have to actually present the vdisk, just ensure that the cable is in the HBA card so that link is UP..

# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 CD-ROM connected configured unknown
c1 fc-private connected configured unknown
c1::500000e01129ff41 disk connected configured unknown
c1::500000e0112a0281 disk connected configured unknown
c2 fc connected unconfigured unknown
c3 fc connected unconfigured unknown
c4 fc-fabric connected configured unknown
c4::50001fe15000f3d8 array-ctrl connected configured unknown
c5 fc connected unconfigured unknown

Important commands for HBA detection and configuration of LUNs are:-

luxadm -e port
luxadm -e dump_map
cfgadm -al
cfgadm -al -o show_FCP_dev
cfgadm -c configure c3
cfgadm -o force_update -c configure c3 ###Use this if the normal configure does not work.
luxadm probe
luxadm qlgc


8. Once the HBA is configured, create a vdisk on HP storage and assign it to the HBA card (using the WWN number).
9. Since my HP storage is connected to couple of switches, the switch needs to be configured to allocate the Path to HBA card.
10. After configuring the switch, enable the config on it (Note:- Never reboot the SAN switch, this is a common mistake and could lead to path revertion)
11. Once the config is enabled, the switch would be able to send out the LUNs to HBA card on the server.
12. Make sure that the Server Vdisk on the storage is set to "Path A/failover".
13. Begin detection of the disk drives on the server (following is the order of commands):-

luxadm probe ## initial probe for disk drives
luxadm qlgc ## initial probe of devices on Qlogic driver (HBA card vendor)
cfgadm -al ## Would display the new devices on the server

Sometime, cfgadm is unable to configure the controller so force it:-

"cfgadm -o force_update -c configure c3"

Once the controller is configured, the devices needs to be detected on the server:-

cfgadm -al -o show_FCP_dev ## this command will show all Fiber Channel Port devices coming from SAN.

14. Run the “format” command to see all the disk drives on the server.


Ideally, put in two HBA cards and then configure them. Afterwards, you could use the Volume manager software (like Veritas, it is free with Solaris 8 :D ). The LUNs multi-pathing can be done via DMP module of Veritas or any volume manager software.