This post is a part of a general "fuck you" to Network Appliance Inc.

This post is copyrights 2008-2010 by vardomskiy. I reserve the right to be identified as the author of this work. I do not grant the license to copy or use the information here in to the NetApp Inc, Network Appliance Inc or it's subsidiaries, dependents or lawyers on it's retainer or staff. I do not grant the right to copy or re-distribute this post to any pro-DMCA organization.

To all other users I grant all rights that are usually granted by the Creative Commons Attribution Non-Commercial Share Alike license.

Oh, and I am not responsible for you doing anything I describe here. You WILL lose your data, set your pants on fire, and blow up your filer. You have been warned.

Ever been in this situation?

You have a NetApp filer, which is out of support. One of the disks in the DS14 shelf fails. What do you do? Like any self-respecting ghetto-admin you roll on to eBay, the Official Procurement Vehicle of Ghetto NOCs world-wide[TM], and search for NetApp disks. And you realize that NetApp branded 144 GB disks cost two or three times as much as identical generic Seagate disks. You decide that you are S-M-R-T SMART, and order a bunch of the 75$ 144 gig Seagate ST3146807FC, thinking that you are going to save some money. NetApp will update the firmware on the disk once you plug it in, right? Disks arrive, you put them into NetApp/Xyratex carriers, plug them into DS14 shelf, and... your filer recognizes them as 72 gig disks, unsupported, and tells you that it will run for 24 hours, and then shut down, since these are unsupported disks.

If you ever been in the above situation, or want to use generic disks in a NetApp filer, without paying NetApp prices, this is for you.

Basically you need to hookup the FC disks to a PC running Linux, and update firmware of the disk with NetApp firmware and reformat the disks 520 bytes per sector, so NetApp would recognize them as it's own. Then you need to plug the disks into NetApp, and format them under the version of OnTAP that you are running.

More specifically, you need the following:

• NetApp DS14Mk2 shelf you can take out of service for half a day (or FAS250 or 270 that you are not using 24/7 and can use for a few hours + ESH2 or DB9 module, or whatever it is you need to hook up DS14mk2 to a PC.) Alternatively you can use any other means of connecting an FC drive to a PC. If you use a generic FC disk enclosure, every time I mention DS14mk2, I mean the disk enclosure, so act appropriately


• A PC box with Linux and (most likely) PCI-X slot


• an FCAL adapter (You can pull the standard NetApp adapter, such as "Dual-channel, QLogic 2312 (2342)" from a filer for a few hours or buy a cheap one from eBay for this purpose)


• All the cabling to hook up DS14Mk2 to FCAL adapter in a PC


• NetApp filer to which you can either connect the above shelf, or into which you can plug in the disks, and which you can boot into maintenance mode.


• Generic Seagate disks, that would match the NetApp branded ones. I've used Seagate ST3146807FC in place of NetApp option X274 (144 gig), and Seagate ST373307FC in place of NetApp option X272 (72 gig)


• Carrier sleds for NetApp shelves (And hot damn, those sleds are bloody expensive on their own)

Find a spare computer into which you can plug the FCAL controller. If you already have a Linux box that you can use, plug the controller into it, otherwise grab a netinstall image from Debian ftp site, burn it, and install Debian onto nearest spare shitbox in the office. We don't need more then a couple of gigs of disk space (at most) and 400 Mhz CPU.

Once you have Debian (or your favorite brand, I am actually a Solaris kind of guy, so I am not about to get into a distribution holy war) on a PC, and FCAL adapter in a PC, boot up the thing. It should boot, and detect your FCAL adapter. I am assuming that your FCAL card is Qlogic chipset based, and you don't have the firmware for FCAL adapter yet. So go ahead, grab the contents of ftp://ftp.qlogic.com/outgoing/linux/firmware/ (or grep dmesg for "ISP", and figure out which specific firmware you need), and drop them all into /lib/firmware (or whatever your kernel module expects).

Now, try to insmod qla2xxx kernel module one more time. If the adapter initializes OK, rmmod qla2xxx, and wire up the DS14mk2 shelf to the Linux box. Make sure that the only disks plugged in are the disks you are about to convert to NetApp specification.

Next roll over to http://www.tc.umn.edu/~erick205/Projects/index.html, and get a copy of fwdl (In my experience you *can* try using SeaTools Enterprise too, but SeaTools was segfaulting on firmware flashing, while fwdl didn't). Unarchive, compile, be happy.

Next,

apt-get update && apt-get install sg3-utils libsgutils1

You will eventually use sg_format, part of sg3-utils to format the disk 520 bytes/sector in a bit, and you need libsgutils to actually talk to the devices.

Grab from your filer's /vol/vol0/etc/disk_fw (or wherever your boot disk is) the firmware files that correspond to the drives you want to flash. For ST3146807FC you need X274_SCHT6146F10.NA16.LOD, for ST373307FC you need X272_SCHT6073F10.NA16.LOD (Which are actually the same file since these drives are in the same Seagate family. Coincidentially you have 36 gig ST336607FC, they can be flashed as well into X270s, but that's not useful at all )



Insmod the sg kernel module, and then insmod qla2xxx, check that dmesg reports all of the disks you want to flash:

QLogic Fibre Channel HBA Driver

PCI: Enabling device 0000:00:0a.0 (0150 -> 0153)

qla2xxx 0000:00:0a.0: Found an ISP2312, irq 177, iobase 0xe88c0000

qla2xxx 0000:00:0a.0: Configuring PCI space...

qla2xxx 0000:00:0a.0: Configure NVRAM parameters...

qla2xxx 0000:00:0a.0: Verifying loaded RISC code...

qla2xxx 0000:00:0a.0: Allocated (412 KB) for firmware dump...

qla2xxx 0000:00:0a.0: Waiting for LIP to complete...

qla2xxx 0000:00:0a.0: Cable is unplugged...

scsi16 : qla2xxx

qla2xxx 0000:00:0a.0: 

 QLogic Fibre Channel HBA Driver: 8.01.07-k1

  QLogic QLA2342 - 133MHz PCI-X to 2Gb FC, Dual Channel

  ISP2312: PCI (33 MHz) @ 0000:00:0a.0 hdma-, host#=16, fw=3.03.27 IPX

PCI: Enabling device 0000:00:0a.1 (0150 -> 0153)

qla2xxx 0000:00:0a.1: Found an ISP2312, irq 185, iobase 0xe891e000

qla2xxx 0000:00:0a.1: Configuring PCI space...

qla2xxx 0000:00:0a.1: Configure NVRAM parameters...

qla2xxx 0000:00:0a.1: Verifying loaded RISC code...

qla2xxx 0000:00:0a.1: Allocated (412 KB) for firmware dump...

qla2xxx 0000:00:0a.1: Waiting for LIP to complete...

qla2xxx 0000:00:0a.1: LIP reset occured (f7f7).

qla2xxx 0000:00:0a.1: LIP occured (f7f7).

qla2xxx 0000:00:0a.1: LOOP UP detected (2 Gbps).

qla2xxx 0000:00:0a.1: Topology - (Loop), Host Loop address 0x0

scsi17 : qla2xxx

qla2xxx 0000:00:0a.1: 

 QLogic Fibre Channel HBA Driver: 8.01.07-k1

  QLogic QLA2342 - 133MHz PCI-X to 2Gb FC, Dual Channel

  ISP2312: PCI (33 MHz) @ 0000:00:0a.1 hdma-, host#=17, fw=3.03.27 IPX

  Vendor: SEAGATE   Model: ST3146807FC       Rev: 0006

  Type:   Direct-Access                      ANSI SCSI revision: 03

SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)

sdb: Write Protect is off

sdb: Mode Sense: ab 00 10 08

SCSI device sdb: drive cache: write through w/ FUA

SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)

sdb: Write Protect is off

sdb: Mode Sense: ab 00 10 08

SCSI device sdc: drive cache: write through w/ FUA

SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)

sdc: Write Protect is off

sdc: Mode Sense: ab 00 10 08

SCSI device sdc: drive cache: write through w/ FUA

 sdc: unknown partition table

sd 17:0:1:0: Attached scsi disk sdc

sd 17:0:1:0: Attached scsi generic sg2 type 0

Beauty.

Now, you can grab SeaTools from Seagate, and play around with them, but in my experience they segfault on firmware update, so they can be useful to show perty things, but otherwise of no utility (since you can find out device IDs corresponding to your drives using dmesg).

debian:/home/vardomskiy/Seagate# ./st -i /dev/sg1

        /dev/sg1

        Vendor = SEAGATE  

        Product = ST3146807FC      

        Version = 0006

        Serial Number = 3HY9X64L

        Copyright = Copyright (c) 2003 Seagate All rights reserved 

        SCSI Firmware = 03110006

        Servo RAM Release = 2002B907

        Servo ROM Release = 00000000

        Servo RAM Date = B907

        Servo ROM Date = 2002



        Blocksize = 512, Highblock = 286749487, Capacity = 143375 MB

        -this is a Seagate drive

        -this drive supports DST

                -short DST time = 120 seconds

                -long DST time = 3072 seconds

        -Mode Page Settings [current value (default)]:

                -WCE bit = 0 (1)

                -RCD bit = 0 (0)

                -AWRE bit = 0 (1)

                -ARRE bit = 0 (1)

                -DExcpt bit = 0 (0)

                -Number of cache segments = 32 (32)

                -PM  bit = 0 (0)

                -DLM bit = 0 (0)

                -DTOLI bit = 0 (0)

                -JIT bit 0 = 1 (1)

                -JIT bit 1 = 1 (1)

                -JIT bit 2 = 1 (1)

                -JIT bit 3 = 1 (1)



debian:/home/vardomskiy/Seagate# 

Upload the firmware using the fwdl tool:

debian:/home/vardomskiy/Seagate/fwdl-1.2.3# ./fwdl /dev/sg1 X274_SCHT6146F10.NA16.LOD 

Gathering inquiry data from the drive...done

Device Type:            Disk

Removable:              0

ISO Version:            3

Response Data Format:   12

Additional Length:      8b

Option Bits:            a

Vendor ID:              SEAGATE 

Product ID:             ST3146807FC     

Revision Level:         0006

Vendor Specific:                3HY9X64L



About to update drive firmware. This could render the drive unusable.



Are you certain you want to continue? [yN] y

done.

debian:/home/vardomskiy/Seagate/fwdl-1.2.3#

Once you did this for all the disks, rmmod the qla2xxx and power down the DS14Mk2 shelf. Wait a few seconds, and power it back on, then insmod the qla2xxx.

At this point the drives have NetApp firmware on them, but the dmesg will show that the system is very unhappy with them since they are unformatted

QLogic Fibre Channel HBA Driver

PCI: Enabling device 0000:00:0a.0 (0150 -> 0153)

qla2xxx 0000:00:0a.0: Found an ISP2312, irq 177, iobase 0xe88c0000

qla2xxx 0000:00:0a.0: Configuring PCI space...

qla2xxx 0000:00:0a.0: Configure NVRAM parameters...

qla2xxx 0000:00:0a.0: Verifying loaded RISC code...

qla2xxx 0000:00:0a.0: Allocated (412 KB) for firmware dump...

qla2xxx 0000:00:0a.0: Waiting for LIP to complete...

qla2xxx 0000:00:0a.0: Cable is unplugged...

scsi12 : qla2xxx

qla2xxx 0000:00:0a.0: 

 QLogic Fibre Channel HBA Driver: 8.01.07-k1

  QLogic QLA2342 - 133MHz PCI-X to 2Gb FC, Dual Channel

  ISP2312: PCI (33 MHz) @ 0000:00:0a.0 hdma-, host#=12, fw=3.03.27 IPX

PCI: Enabling device 0000:00:0a.1 (0150 -> 0153)

qla2xxx 0000:00:0a.1: Found an ISP2312, irq 185, iobase 0xe891e000

qla2xxx 0000:00:0a.1: Configuring PCI space...

qla2xxx 0000:00:0a.1: Configure NVRAM parameters...

qla2xxx 0000:00:0a.1: Verifying loaded RISC code...

qla2xxx 0000:00:0a.1: Allocated (412 KB) for firmware dump...

qla2xxx 0000:00:0a.1: Waiting for LIP to complete...

qla2xxx 0000:00:0a.1: LIP reset occured (f7f7).

qla2xxx 0000:00:0a.1: LIP occured (f7f7).

qla2xxx 0000:00:0a.1: LOOP UP detected (2 Gbps).

qla2xxx 0000:00:0a.1: Topology - (Loop), Host Loop address 0x0

scsi13 : qla2xxx

qla2xxx 0000:00:0a.1: 

 QLogic Fibre Channel HBA Driver: 8.01.07-k1

  QLogic QLA2342 - 133MHz PCI-X to 2Gb FC, Dual Channel

  ISP2312: PCI (33 MHz) @ 0000:00:0a.1 hdma-, host#=13, fw=3.03.27 IPX

  Vendor: NETAPP    Model: X274_SCHT6146F10  Rev: NA16

  Type:   Direct-Access                      ANSI SCSI revision: 03

sdb : READ CAPACITY failed.

sdb : status=1, message=00, host=0, driver=08 

sd: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

sdb: Write Protect is off

sdb: Mode Sense: ab 00 10 08

SCSI device sdb: drive cache: write through w/ FUA

sdb : READ CAPACITY failed.

sdb : status=1, message=00, host=0, driver=08 

sd: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

sdb: Write Protect is off

sdb: Mode Sense: ab 00 10 08

SCSI device sdb: drive cache: write through w/ FUA

 sdb:<6>sd 13:0:0:0: SCSI error: return code = 0x08000002

sdb: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

Info fld=0x0

end_request: I/O error, dev sdb, sector 0

Buffer I/O error on device sdb, logical block 0

sd 13:0:0:0: SCSI error: return code = 0x08000002

sdb: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

Info fld=0x0

end_request: I/O error, dev sdb, sector 0

Buffer I/O error on device sdb, logical block 0

 unable to read partition table

sd 13:0:0:0: Attached scsi disk sdb

sd 13:0:0:0: Attached scsi generic sg1 type 0

  Vendor: NETAPP    Model: X274_SCHT6146F10  Rev: NA16

  Type:   Direct-Access                      ANSI SCSI revision: 03

sdc : READ CAPACITY failed.

sdc : status=1, message=00, host=0, driver=08 

sd: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

sdc: Write Protect is off

sdc: Mode Sense: ab 00 10 08

SCSI device sdc: drive cache: write through w/ FUA

sdc : READ CAPACITY failed.

sdc : status=1, message=00, host=0, driver=08 

sd: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

sdc: Write Protect is off

sdc: Mode Sense: ab 00 10 08

SCSI device sdc: drive cache: write through w/ FUA

 sdc:<6>sd 13:0:0:0: SCSI error: return code = 0x08000002

sdb: Current: sense key: Medium Error

    Additional sense: Medium format corrupted

Info fld=0x0

[...]

Pages and pages of above cruft.

That's cool, you just need to format the disks at 520 bytes per sector.

NetApp uses the 8 extra bytes to keep the checksum of the data in the sector. If you plug the 512 byte/sector formatted disk into NetApp, it will show as having half the capacity because every second sector will have 8 byte checksum in it, and that's it. Coincidentially that's why A-SIS/dedup was "simple" to implement on WAFL - there is already an 8 byte checksum for every sector, so all the filer has to do is build a binary tree of the checksums, and for each new sector of data, check if there is one already that matches the checksum. If it does, do the byte by byte comparison, and in event of perfect match, repoint the FAT, and mark the sector clear. Sounds simple, but the devil is in the detailes.

Any way, format the disks 520 bytes per sector:

debian:/home/vardomskiy/Seagate# sg_format --format --size=520 --verbose /dev/sdb

    inquiry cdb: 12 00 00 00 24 00 

    NETAPP    X274_SCHT6146F10  NA16   peripheral_type: disk [0x0]

      PROTECT=0

    mode sense (10) cdb: 5a 00 01 00 00 00 00 00 fc 00 

    mode sense (10): requested 252 bytes but got 28 bytes

Mode sense (block descriptor) data, prior to changes:

  Number of blocks=280790184 [0x10bc84a8]

  Block size=520 [0x208]



A FORMAT will commence in 10 seconds

    ALL data on /dev/sdb will be DESTROYED

        Press control-C to abort

A FORMAT will commence in 5 seconds

    ALL data on /dev/sdb will be DESTROYED

        Press control-C to abort

    format cdb: 04 10 00 00 00 00 



Format has started

    test unit ready cdb: 00 00 00 00 00 00 

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 0 %

Format in progress, 0% done

    test unit ready cdb: 00 00 00 00 00 00 

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 0 %

Format in progress, 0% done

    test unit ready cdb: 00 00 00 00 00 00 

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 0 %

Format in progress, 0% done

    test unit ready cdb: 00 00 00 00 00 00 

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 1 %

Format in progress, 1% done

[....]

    test unit ready cdb: 00 00 00 00 00 00

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 97 %

Format in progress, 97% done

    test unit ready cdb: 00 00 00 00 00 00

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 97 %

Format in progress, 97% done

    test unit ready cdb: 00 00 00 00 00 00

test unit ready:  Fixed format, current;  Sense key: Not Ready

 Additional sense: Logical unit not ready, format in progress

  Field replaceable unit code: 2

  Progress indication: 97 %

Format in progress, 97% done

    test unit ready cdb: 00 00 00 00 00 00

FORMAT Complete

debian:/home/vardomskiy/Seagate# 

At this point you are almost done, but need the netapp to continue.

rmmod the qla2xxx, detach the disk shelf with the disks from a Linux box, and plug the shelf full of drives into a NetApp (Best to just use an FAS250 or FAS270 module in place of ESH2 in DS14Mk2, although you can connect just the shelf to a full head, or move the disks somewhere).

Now you need to boot the netapp in maintenance mode (ctrl-c on boot), and use disk assign to take over these disks. Halt the filer, and boot it in maintenance mode again, but this time choose to zero all owned disks (This is why you want only the newly NetApped disks in the filer!!!). Once the NetApp is done zeroing the disks, you get a new batch of spares.

In my experience, 14 generic Seagate 144 gig FC disks can be bought for ~1000 USD, which might be cost-effective if you have a shelf full of 36 giggers, or even 72 giggers.

Updated 2009-05-26 : I am glad that at least someone found the information above useful. Thomas Uphill notes that if you only work on a single disk, you can go ahead and use diskcopy in NetApp maintenance mode instead of zeroing the disks. This will put the correct disk label onto the disk. This poses a question: Maybe all I needed to do was format the disk at 520 bytes/sector, and then copy the disk over in a NetApp? In other words, if a NetApp sees correctly formatted disk, will it update firmware? Must test that next time I am upgrading a shelf full of disks.

Updated 2009-09-03 : Now there are instructions in the comments on how to convert 1 TB Seagate SATA disks to NetApp spec. So if you have a bunch of DS14MK2-AT shelves full of 250 or 320 GB disks, you can upgrade your disks and increase your disk space, or remove a shelf or two from that systems closet (and heck, eBay the shelves you no-longer need :-)

Updated 2009-10-05 : Based on the reader contributions, here is a partial list of generic flashable disks. I am sure there are more, but that's what we have for now. FC disks I verified myself, while information on the SATA based disks was contributed by various readers.

Generic Drive	NetApp Option	Firmware filename	Capacity	Interface
Seagate ST336607FC	X270	X270_SCHT6036F10.NA16.LOD	36 GB	FC
Seagate ST373307FC	X272	X272_SCHT6073F10.NA16.LOD	72 GB	FC
Seagate ST3146807FC	X274	X274_SCHT6146F10.NA16.LOD	144 GB	FC
Seagate Barracuda ES.2 ST3500320NS	X267	X267_SMOOST50SSX.NA01.LOD	500 GB	SATA
Seagate Barracuda ES.2 ST31000340NS	X269	X269_SMOOS01TSSX.NA01.LOD	1TB	SATA

Updated 2010-06-01 : It's been two weeks since livejournal blocked this post. it's been over a week since I opened a dispute, and they haven't replied back to me with anything. I gave up and am re-hosting this post here, since I am tired of getting messages and e-mail about it. I apologize for not getting all of your comments back up - I have them saved, and will have to get them added on somehow.

In the meanwhile, I've converted another batch of FC drives to NetApp spec. If you format the drive with 520 byte blocks, but do not update the firmware, NetApp will detect the drive, notice that it has non-netapp firmware, but will not go into "I have a 512 byte/block disk, oh my god, I will shut down in 24 hours" mode. Now, I did this with a drive that wasn't NetApp formatted, so filer complained about lack of proper disk labels. Here is the console log of me pulling a netapp branded/labeled spare disk on a filer, and putting in one of Seagate disks that has been formatted to 520 bytes/sector but unlabeled:

Tue Jun  1 12:00:00 MSD [kern.uptime.filer:info]:  12:00pm up 19 days, 22:00 18170097 NFS ops, 0 CIFS ops, 22 HTTP ops, 0 FCP ops, 0 iSCSI ops

Tue Jun  1 12:51:49 MSD [openssh.dispatch.protocol:warning]: SSH dispatch failed from address 10.100.0.69, type 80, packet 7.

Tue Jun  1 12:52:56 MSD [ispfc_init_0:error]: Failed to build FCAL map on Fibre Channel adapter 0b -- resetting adapter.

Tue Jun  1 12:53:02 MSD [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0b.

Tue Jun  1 12:53:23 MSD last message repeated 2 times

Tue Jun  1 12:53:29 MSD [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.

Tue Jun  1 12:53:29 MSD [raid.rg.spares.low.unowned:warning]: /aggr0/plex0/rg0 Assign unowned disks.

Tue Jun  1 12:53:34 MSD [asup.smtp.sent:notice]: System Notification mail sent: System Notification from trotsky (SPARES_LOW) WARNING

Tue Jun  1 12:53:36 MSD [diskown.changingOwner:info]: changing ownership for disk 0b.27 (S/N 3HY6S6B900007435CABN) from unowned (ID -1) to trotsky (ID 84170977)

Tue Jun  1 12:53:36 MSD [raid.assim.disk.nolabels:error]: Disk 0b.27 Shelf 1 Bay 11 [SEAGATE  ST3146807FC      0006] S/N [3HY6S6B900007435CABN] has no valid labels. It will be taken out of service to prevent possible data loss.

Tue Jun  1 12:53:36 MSD [raid.config.disk.bad.label:error]: Disk 0b.27 Shelf 1 Bay 11 [SEAGATE  ST3146807FC      0006] S/N [3HY6S6B900007435CABN] has bad label.

Tue Jun  1 12:53:37 MSD [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.

Tue Jun  1 12:53:39 MSD [asup.smtp.sent:notice]: System Notification mail sent: System Notification from trotsky (DISK BAD LABEL) ERROR

Tue Jun  1 12:54:00 MSD [monitor.globalStatus.nonCritical:warning]: Disk on adapter 0b, shelf 1, bay 11, failed.  There are not enough spare disks. 

Tue Jun  1 12:55:19 MSD [dfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk drives

Tue Jun  1 12:56:04 MSD [ispfc_init_0:error]: Failed to build FCAL map on Fibre Channel adapter 0b -- resetting adapter.

Tue Jun  1 12:56:09 MSD [disk.readReservationFailed:error]: Disk read reservation failed on 0b.27 with return code 2.

Tue Jun  1 12:56:09 MSD [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0b.27 (S/N 3HY6S6B900007435CABN) while reading reservation state

Tue Jun  1 12:56:09 MSD [disk.reserveFailed:error]: Disk reservation failed on 0b.27 (return code 2).

Tue Jun  1 12:56:10 MSD [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0b.

Tue Jun  1 12:56:10 MSD [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0b.27 (S/N 3HY6S6B900007435CABN) while setting disk reservation

Tue Jun  1 12:56:10 MSD [raid.disk.missing:info]: Disk 0b.27 Shelf 1 Bay 11 [SEAGATE  ST3146807FC      0006] S/N [3HY6S6B900007435CABN] is missing from the system

Tue Jun  1 12:56:19 MSD [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0b.

Tue Jun  1 12:56:26 MSD [raid.disk.inserted:info]: Disk 0b.27 Shelf 1 Bay 11 [NETAPP   X274_SCHT6146F10 NA16] S/N [3HY9XFVC000075326QXN] has been inserted into the system

Tue Jun  1 12:56:27 MSD [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.

Tue Jun  1 12:57:00 MSD [monitor.globalStatus.ok:info]: The system's global status is normal. 

At this point I've attempted to use dd under Linux to obtain a label/formatting of a valid NetApp labeled disk (This way I'd not need to run initial setup/format on a filer to get a disk properly labeled), but Debian dd wasn't able to do that (Bug!).

Update: 2010-06-02: I took a few more generic Seagate FC disks, and formatted them 520 bytes/block without upgrading them to NetApp firmware. Then I formatted them using a NetApp filer. The resulting disks happily work with FAS250 on which I've tested them.

trotsky*> Wed Jun  2 18:44:30 EST [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0b.

Wed Jun  2 18:44:36 EST [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.

Wed Jun  2 18:44:38 EST [diskown.changingOwner:info]: changing ownership for disk 0b.27 (S/N 3HY9PWPF00003204MAKZ) from unowned (ID -1) to trotsky (ID 84281663)

Wed Jun  2 18:44:39 EST [sfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk shelves.

trotsky*> Wed Jun  2 18:45:00 EST [monitor.globalStatus.ok:info]: The system's global status is normal.

This basically means that all you need to do is format the disk, you don't need to do evil magic with the firmware flashing, etc.

Here is the sysconfig -a:

trotsky*> sysconfig -a

	NetApp Release 7.2.5.1: Wed Jun 25 09:03:07 PDT 2008

	System ID: 0084281663 (trotsky)

	System Serial Number: 3078437 (trotsky)

	System Rev: C0

	slot 0: System Board 600 MHz (TSANTSA C0)

                Model Name:         FAS250

                Part Number:        110-00016

                Revision:           C0

                Serial Number:      295361

                Firmware release:   CFE 1.2.0

                Processors:         2

                Processor revision: B2

                Processor type:     1250

                Memory Size:        510 MB

                NVMEM Size:         64 MB of Main Memory Used

	slot 0: FC Host Adapter 0b (Dual-channel, QLogic 2312 rev. 2, 32-bit, L-port, )

		Firmware rev:	3.3.25

		Host Loop Id:	7	FC Node Name:	5:00a:098000:001e80

		Cacheline size:	8	FC Packet size:	2048

		SRAM parity:	Yes	External GBIC:	No

		Link Data Rate:	1 Gbit

                 16  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K8115)

                 17  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K8087)

                 18  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433F3696)

                 19  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K7917)

                 20  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K7941)

                 21  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K8057)

                 22  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (43358015)

                 23  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K3477)

                 24  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K0278)

                 25  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433J6350)

                 26  : NETAPP   X274_SCHT6146F10 NA16 136.0GB 520B/sect (3JA23XLL000073529CE5)

                 27  : SEAGATE  ST3146807FC      0006 136.0GB 520B/sect (3HY9PWPF00003204MAKZ)

                 24  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K0174)

                 25  : NETAPP   X274_HJURE146F10 NA14 136.0GB 520B/sect (433K4383)

		Shelf 1: EFH  (DS14-Mk2-FC Based Filer)

                        I/O base 0xfe00, size 0x100

                        memory mapped I/O base 0x41000000, size 0x1000

	slot 0: FC Host Adapter 0c (Dual-channel, QLogic 2312 rev. 2, 32-bit, L-port, <UP>)

		Firmware rev:	3.3.25

		Host Loop Id:	7	FC Node Name:	5:00a:098000:001e80

		Cacheline size:	8	FC Packet size:	2048

		SRAM parity:	Yes	External GBIC:	No

		Link Data Rate:	1 Gbit

                        I/O base 0xff00, size 0x100

                        memory mapped I/O base 0x41001000, size 0x1000

	slot 0: Dual SB1250-Gigabit Ethernet Controller 

		Device Type:        BCM1250

                e0a MAC Address:    00:a0:98:07:20:ea (auto-1000t-fd-up)

                e0b MAC Address:    00:a0:98:07:20:eb auto-1000t-fd-up)

	slot 0: NetApp ATA/IDE Adapter 0a (0x00000000000001f0)

                0a.0                 SanDisk  SDCFB-128        1.24 122MB 512B/sect (122823I2303D1234)

trotsky*> 

Of course, if you have the means to flash the firmware, by all means do so, but if you do not, you are not SOL on the FC based filers.

Once again, big THANK YOU! to everyone who provided additional information, commented, or even just stopped by.