Configuring JBOD with a LSI 2208 controller
PMEnableJBOD PDMakeGood PDMakeJBOD
Note: Although this procedure seems to work I found the machine was not stable.
How to configure disks for JBOD mode on a LSI 2208 controller on a Supermicro X9DRH-7TF motherboard. This is so the disks can be used in a ZFS Pool.
Procedure
This is done using the BIOS preboot CLI. In theory it should be done in any OS using the command line tools. However the NAS4Free distribution doesn't seem to have a command line tool installed by default.
JBOD Adapter property
Query the current state of the adapter 'enable JBOD' property (for all adapters)
$ AdpGetProp enablejbod -aALL Adapter 0: JBOD: Disabled Exit Code = 0x0
Enable JBOD
$ AdpSetProp EnableJBOD 1 -aALL Adapter 0: Set JBOD to Enable success. Exit Code = 0x0
List physical devices
List the available physical devices to get the enclosure number and drive identifier. Note that because the terminal emulation doesn't allow scrolling back and the amount of information well exceeds what fits on a page, the '-page24' providing basic paging support.
The PDList also provides field 'Firmware State'.
$ PDList -aALL -page24
Make good
Mark each of the devices as 'good'. Use the output of the PDList command to determine the enclosure and slot identifiers. The BIOS level command interface required a 'force' parameter to perform the action. The 'force' option had to go before the adpater option.
The devices can be done one at a time
$ PDMakeGood -PhysDrv[252:0] -Force -a0 Adapter: 0: EnclId-252 SlotId-0 state changed to Unconfigured-Good. Exit Code = 0x0
Or many at a time
$ PDMakeGood -PhysDrv[252:1,252:2,252:3,252:4,252:5,252:6,252:7] -Force -a0
Adapter: 0: EnclId-252 SlotId-1 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-2 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-3 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-4 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-5 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-6 state changed to Unconfigured-Good.
Adapter: 0: EnclId-252 SlotId-7 state changed to Unconfigured-Good.
Exit Code = 0x0
Make JBOD
Make each of the disks to be in JBOD mode.
$ PDMakeJBOD -PhysDrv[252:0] -a0
Adapter: 0: EnclId-252 SlotId-0 state changed to JBOD.
Exit Code = 0x0
Or many at a time
$ PDMakeJBOD -PhysDrv[252:1,252:2,252:3,252:4,252:5,252:6,252:7] -a0 Adapter: 0: EnclId-252 SlotId-1 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-2 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-3 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-4 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-5 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-6 state changed to JBOD. Adapter: 0: EnclId-252 SlotId-7 state changed to JBOD. Exit Code = 0x0
Supermicro X9DRH-7TF LSI 2208
When the motherboard reboots, the following output is provided by the LSI 2208 BIOS. The motherboard was delivered with v23.9.0-0015 firmware.
Links
- http://www.nexentastor.org/boards/1/topics/8033
- http://www.supermicro.com/manuals/other/LSI_2108_2208_SAS_MegaRAID_Configuration_Utility.pdf
- http://forums.servethehome.com/raid-controllers-host-bus-adapters/599-lsi-raid-controller-hba-complete-listing-plus-oem-models.html
- LSI 9266-8i Support Documents and Downloads
- http://www.freebsd.org/cgi/man.cgi?query=mps&sektion=4
- http://www.lsi.com/downloads/Public/Solid%20State%20Storage/WarpDrive%20SLP-300/LSI_SSS_P3/MPSLSI_FreeBSD_P3.txt
- http://wiki.cse.ucdavis.edu/support:general:megacli#export_all_megaraid_drives_for_linux_software_raid
- http://tools.rapidsoft.de/perc/perc-cheat-sheet.html
- Intel Integrated RAID Module RMS25CB080
Appendices
MegaCli command help
| Description | MegaCLI Command |
StorCLI Command |
|---|---|---|
| Set properties on the selected controllers |
Megacli -AdpSetProp <propertyname>-an|-a0,1,2|-aall The following properties can be set using this command:
|
/cx set <property1> The following properties can be set using this command:
|
| Set the drive state to JBOD | MegaCli –PDMakeGood -PhysDrv[E0:S0,E1:S1....] -aN|-a0,1,2|-aALL |
storcli /cx/ex/sx set good [force] or storcli /cx/ex/sx set jbod |
PDList sample
Enclosure Device ID: 252 Slot Number: 6 Enclosure position: N/A Device Id: 1 WWN: 50015178f35c3a61 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 93.160 GB [0xba52230 Sectors] Non Coerced Size: 92.660 GB [0xb952230 Sectors] Coerced Size: 92.200 GB [0xb866800 Sectors] Firmware state: JBOD Device Firmware Level: 0250 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221105000000 Connected Port Number: 1(path0) Inquiry Data: BTTV305002CZ100FGN INTEL SSDSC2BA100G3 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Link Speed: 6.0Gb/s Media Type: Solid State Device Drive Temperature : N/A PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No
Enclosure Device ID: 252 Slot Number: 5 Enclosure position: 0 Device Id: 7 WWN: Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 2.728 TB [0x15d50a3b0 Sectors] Non Coerced Size: 2.728 TB [0x15d40a3b0 Sectors] Coerced Size: 2.727 TB [0x15d3ef000 Sectors] Firmware state: JBOD Device Firmware Level: 0A80 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x4433221106000000 Connected Port Number: 6(path0) Inquiry Data: WD-WMC1T3031904WDC WD30EFRX-68AX9N0 80.00A80 FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive Temperature :33C (91.40 F) PI Eligibility: No Drive is formatted for PI information: No PI: No PI Drive's write cache : Enabled Drive's NCQ setting : Disabled Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No
FreeBSD /var/run/dmesg.boot
Nas4Free/FreeBSD sees the drive with 195371568 (0xba52230) sectors, which is the same as the 'Raw Size' (in sectors) above.
mfisyspd0 on mfi0 mfisyspd0: 95396MB (195371568 sectors) SYSPD volume mfisyspd0: SYSPD volume attached
diskinfo
# diskinfo -v /dev/mfisyspd1
/dev/mfisyspd1
512 # sectorsize
100030242816 # mediasize in bytes (93G)
195371568 # mediasize in sectors
0 # stripesize
0 # stripeoffset
12161 # Cylinders according to firmware.
255 # Heads according to firmware.
63 # Sectors according to firmware.
# Disk ident.
PDMakeGood Warnings
The command needs a 'force' option. The force option seems to need to go before the adpater option.
$ PDMakeGood -PhysDrv[252:0] -a0 To prevent data loss and adverse effect on system, please stop all the IOs to this drive.If it is a file system drive, please unmount it. use -force option if you are sure to perform this operation Exit Code = 0x1
If the physical drive is already good the following message is generated
$ PDMakeGood -PhysDrv[252:0] -Force -a0 Adapter: 0: Failed to change PD state at EnclId-252 SlotId-0. Exit Code = 0x1
