ACCEPTING CASES · MON–FRI 9:00AM–5:00PM
Certified Data Recovery Professional · Phoenix, AZ ☎ (602) 686-2622

Used Enterprise SAS SSDs Can Fail at a Fixed Hour Count. Check Before You Load Them.

Some used HPE SAS SSDs will stop at 32,768 power-on hours, and a smaller HPE list stops at 40,000, if the firmware on the label was never updated. HPE's bulletins say that after that failure, neither the drive nor the data can be recovered, and drives that were put in service together will likely fail together. Dell published a separate 40,000-hour failure for nine of its own SAS SSD models. Read the model, the firmware revision, and the hours before the drive holds anything you need.

Free evaluation · No data, no fee · Talk directly with a technician.

New drives are scarce. That squeeze, and the used drives people are buying because of it, is the price page. A used SSD that sat unpowered and came back empty is a different failure. A hard drive whose SMART hours were wiped and then sold as fresh is a different report. Several disks that drop in the same minute are, until a disk-level read says otherwise, a controller, backplane, cable, or power path. This page is the firmware defect HPE and Dell already published for specific enterprise SAS SSDs: a fixed power-on-hour count, on old firmware, after which those drives stop.

HPE, 32,768 hours

HPE customer bulletin a00092491 (revision, last updated 2024-02-23) is titled as a critical firmware upgrade for certain SAS SSD models to prevent drive failure at 32,768 hours of operation. HPE says it was notified of a firmware defect in November 2019. The scope is any of the models below running firmware prior to HPD8. HPE's words for the result: the issue "results in SSD failure at 32,768 hours of operation (i.e., 3 years, 270 days 8 hours). After the SSD failure occurs, neither the SSD nor the data can be recovered." The same bulletin says SSDs put into service at the same time "will likely fail nearly simultaneously."

Neglecting the update to HPD8, HPE writes, results in drive failure and data loss at that hour count, and requires restoration from backup. That includes a RAID 0, and it includes a fault-tolerant RAID when more members fail than that RAID level can lose. HPE says the model list is comprehensive. No other drive models are affected. The fix they name is firmware HPD8, and the download line says HPD8 (or later).

HPE modelWhat the bulletin calls itExposed firmware
VO0480JFDGT480 GB 12Gb SAS, read intensivePrior to HPD8
VO0960JFDGU960 GB 12Gb SAS, read intensivePrior to HPD8
VO1920JFDGV1.92 TB 12Gb SAS, read intensivePrior to HPD8
VO3840JFDHA3.84 TB 12Gb SAS, read intensivePrior to HPD8
MO0400JFFCF400 GB 12Gb SAS, mixed usePrior to HPD8
MO0800JFFCH800 GB 12Gb SAS, mixed usePrior to HPD8
MO1600JFFCK1.6 TB 12Gb SAS, mixed usePrior to HPD8
MO3200JFFCL3.2 TB 12Gb SAS, mixed usePrior to HPD8
VO000480JWDAR480 GB SAS, read intensivePrior to HPD8
VO000960JWDAT960 GB SAS, read intensivePrior to HPD8
VO001920JWDAU1.92 TB SAS, read intensivePrior to HPD8
VO003840JWDAV3.84 TB SAS, read intensivePrior to HPD8
VO007680JWCNK7.68 TB SAS 12G, read intensivePrior to HPD8
VO015300JWCNL15.3 TB SAS 12G, read intensivePrior to HPD8
VK000960JWSSQ960 GB SAS, read intensivePrior to HPD8
VK001920JWSSR1.92 TB SAS, read intensivePrior to HPD8
VK003840JWSST3.84 TB SAS, read intensivePrior to HPD8
VK007680JWSSU7.68 TB SAS, read intensivePrior to HPD8
VO015300JWSSV15.3 TB SAS, read intensivePrior to HPD8

HPD7 on a model in this table is still prior to HPD8. The 40,000-hour fix below does not clear this list. HPE's SAS SSD remediation page says the firmware for the two issues is not interchangeable.

HPE, 40,000 hours

HPE customer bulletin a00097382 (revision, last updated 2021-03-08) is a second defect. The bulletin says the failure mode is similar and the issue is unrelated to the 32,768-hour bulletin. It also says the issue is not unique to HPE and potentially affects other manufacturers and customers that purchased these drives. The scope is four models, firmware prior to HPD7. HPE's conversion of the count is 4 years, 206 days, 16 hours. After the failure, "neither the SSD nor the data can be recovered." Drives put into service at the same time "will likely fail nearly simultaneously." The example HPE gives of fault tolerance that is not enough is a RAID 5 logical drive with two failed SSDs.

The fix they name is HPD7 (or later). The model list is comprehensive. No other drive models are affected by this bulletin.

HPE modelWhat the bulletin calls itExposed firmware
EK0800JVYPN800 GB 12G SAS, write intensivePrior to HPD7
EO1600JVYPP1.6 TB 12G SAS, write intensivePrior to HPD7
MK0800JVYPQ800 GB 12G SAS, mixed usePrior to HPD7
MO1600JVYPR1.6 TB 12G SAS, mixed usePrior to HPD7

A loose drive is identified by the model on its label. HPE also says drives in a SAN-attached shelf or other external enclosure have to be checked separately from the drives in the server. The original chassis does not clear a model that is on the list.

Dell, 40,000 hours, a different model list

Dell's article 000177640 covers nine Dell Enterprise SAS SSD models with firmware prior to D417. Dell says that firmware issue results in drive failure at 40,000 power-on hours. A drive that has failed this way reports 0 GB of available storage. It goes offline and becomes unusable after a power cycle, in a format corruption state, and reports 03/31/00. Dell says these drives shipped in PowerEdge servers, PowerVault storage, and DSS 7500, and that no other Dell Enterprise products shipped with them.

The fix Dell names is firmware D417 or later, applied prior to 40,000 power-on hours. If the drive has already failed, the article says to contact Dell Support. It does not say the data can be recovered. It also does not use HPE's sentence that neither the drive nor the data can be recovered. Those are HPE's words for HPE's bulletins. Keep them there.

Dell modelDell part number on the same pageCapacity on that page
LT0200MO2XR0K200 GB, SAS12, MU
LT0400MOC06VX400 GB, SAS12, MU
LT0800MO989R8800 GB, SAS12, MU
LT1600MOJDTGX1.6 TB, SAS12, MU
LT0800ROJ19XM800 GB, SAS12, RI
LT1600RO2M61G1.6 TB, SAS12, RI
LT0200WMM7KYX200 GB, SAS12, WI
LT0400WMT2TPF400 GB, SAS12, WI
LT0800WMFHFNJ800 GB, SAS12, WI

The affected-products list spells the 1.6 TB read-intensive model LT1600RO. The parts table on the same page prints LT1600R0 on that row, next to part number 2M61G. Check the label against both spellings before you decide the article does not apply. The Dell model strings are not the HPE model strings. An HPE HPD7 file is not the Dell D417 update.

The drive still identifies and the hours are under the count on the matching bulletin. Update that model's firmware only after anything you need already exists on other media. If it has already failed, stop power-cycling it and have it evaluated. Free evaluation, no data, no fee.

Read the model, the revision, and the hours

Do this before the filesystem is created and before a single folder is copied on. The hours already on the drive are the previous owner's power-on time. A drive that is new to you is not a drive with a fresh counter.

Model. Read the product string on the label and in the drive's inquiry data. On a Linux HBA that exposes the disk itself, that string is the Product line from:

smartctl -a /dev/sdX

Use the device name the HBA actually assigned. Compare Product to the tables above. A seller's "800 GB SAS SSD, tested" is not a model number.

Firmware. The same information section prints Revision. Compare that revision to the fix named for that model. Prior to HPD8 on the first HPE table. Prior to HPD7 on the second HPE table. Prior to D417 on the Dell table.

Hours. On a SAS drive, smartctl prints Accumulated power on time, hours:minutes followed by hours, a colon, and minutes. The hours are the number before the colon. HPE's figure of 32,768 hours is 3 years, 270 days, 8 hours of powered-on time. 40,000 hours is 4 years, 206 days, 16 hours. A drive that has not been powered on the whole time takes longer on the calendar to get there. The counter is powered-on time.

Four pulls reporting 31,000 hours, on the 32,768-hour list, on firmware prior to HPD8, are 1,768 power-on hours short of the count in that bulletin. A drive that stays powered on covers 1,768 hours in 73 days and 16 hours. HPE says drives put into service together will likely fail nearly simultaneously. That is the set reaching the count in the same week.

If the name you can open is a RAID virtual disk, you are not reading the member. HPE's bulletin points to Smart Storage Administrator for power-on hours on their controllers, and publishes separate detection scripts for the 32,768-hour issue and the 40,000-hour issue. The remediation page says both scripts have to be run. Dell's article expects the model and the firmware to be checked, and the update to be applied through their update package, iDRAC, Repository Manager, or OpenManage Enterprise, before the count.

The percentage-used figure, when smartctl prints one, is wear. The bulletins above are written against power-on hours and firmware version. A low wear number does not clear a model that is on the list with old firmware.

Apply the vendor update while the drive still works

The update those pages describe is prevention. HPE says to apply it immediately, and that skipping it means drive failure and data loss at the published count. Dell says upgrading to D417 or later prior to 40,000 power-on hours should address the issue on the models they list. Do it while the drive still identifies, while the hours are still under that model's count, and while anything you would miss already exists on other media. A used drive you have not loaded yet has an easy version of that backup: do not make it the only copy of anything until the revision on it is the fix.

Get the file from the bulletin for that model. HPE's 32,768-hour package and HPE's 40,000-hour package are not interchangeable. Dell's D417 package is the Dell article's package. A firmware file from a different family, or a consumer-drive toolbox, is a write aimed at the wrong drive.

Dell also lists conditions where you do not run the update. Leave it until these are clear: a RAID set that is partially degraded or degraded, a rebuild, resync, or copyback in progress, a background initialization or consistency check, and any virtual disk that is not optimal. Dell says the update does run on drives that remain in a RAID set that is offline. Read that as their condition, then get the set stable or pull the drive to a bench. Do not "update the firmware" of a degraded array because the button is right there. A degraded pool that you update and reboot is the same class of mistake as installing a DSM update on a degraded Synology pool.

A drive that has already stopped is past the prevention step. HPE's fix for the 32,768-hour and 40,000-hour lists is how you keep the failure from happening. It is not a procedure they published for bringing a failed drive back. Flashing a drive that no longer enumerates, or that already shows 0 GB, is the thing our SSD recovery page tells you to stop doing on a failed SSD. A Samsung 990 Pro that is already at zero spare is a different article, and the advice there is copy first, do not flash first. Same order here once the drive is already failing: the bytes come off before anybody writes firmware to the device.

Warning: If the model is on HPE's 32,768-hour or 40,000-hour list, the firmware is older than that list's fix, and the drive has already failed, HPE says neither the SSD nor the data can be recovered. Stop power-cycling it. Do not flash the prevention file to see if the drive comes back.

Do not build the array from one lot

HPE's sentence is the whole design problem. Drives put into service at the same time will likely fail nearly simultaneously. A used "matched set," one server pull, or one auction lot with hours that agree is that sentence, already most of the way to the count. RAID 5 survives one failure. HPE's own example of the backup you will need is a RAID 5 with two SSDs failed. A mirror of two drives from the same shelf dies as a pair. The second member is not a spare if it reaches the count on the same day.

Update every member first, from the package for its model, or leave those drives out of any set that has to survive two deaths at once. Mix pulls that do not share a firmware revision and an hour count. The hour count they already have is what matters. Hours since you installed them are not a reset.

When several drives do drop together, still read the path. A backplane or a controller can take a whole shelf down in one minute, which is that page. Drives that were imaged into service together and die at one published hour count are this page. The model and the hours tell you which description you are in. Rebuilding the set onto more pulls from the same lot repeats the timer.

70,000 hours is a different HPE advisory

HPE advisory a00111296 (last updated 2021-03-18) says data may become inaccessible at 70,000 hours, which HPE converts as 7.99 years, on a different pair of models with firmware HPD3 or earlier. HPE says this issue is unrelated to the 32,768-hour bulletin and unrelated to the 40,000-hour bulletin. The scope list is VK0800JEABE and VO1600JEABF. HPE says that list is comprehensive.

The outcome they describe is also different. Neglecting HPD4 results in a temporary loss of drive operation at 70,000 power-on hours, followed by a power cycle. HPE says HPD4 has to be applied to restore the drive, and that this firmware both prevents the issue and recovers drives that have already hit it. Read that advisory if the label is one of those two models. Do not paste "neither the SSD nor the data can be recovered" onto it. That sentence is in the 32,768-hour and 40,000-hour bulletins.

Other hour counts exist on other models. If the label is not in this page's tables, this page does not identify the drive. Look up that model. A round number you remember from a different SSD family is not a substitute for the notice that names the label in your hand.

If it has already stopped

0 GB, missing from the controller, or a Dell sense of 03/31/00 after the drive passed the count on unfixed firmware: stop. Each extra power-on is another trip through the failure Dell says makes the drive unusable, and another power cycle of a drive HPE says cannot be recovered after the failure. Do not initialize it. Do not format it. Do not let a controller mark the member failed and rebuild onto a survivor that has the same hours and the same old firmware. Rebuilding writes. On a set that can lose a second member at the same count, the rebuild is how the last copy gets spent. That is the same order of operations as RAID rebuild versus recovery.

HPE's 32,768-hour bulletin and HPE's 40,000-hour bulletin both say that after the failure, neither the SSD nor the data can be recovered. We will say that plainly. A lab cannot promise to reverse it. Chip-off, a donor board, and a firmware tool are not a published exception to that sentence, and we will not imply one. If the drive still identifies and still reads, the copy comes off onto stable media before any other write, and the work happens on the copy. If it does not identify, there may be nothing to image. The evaluation is where the technician on your case tells you which of those is in front of us.

Dell's article, for the nine models, tells you to contact Dell Support if the drive has already failed. It documents 0 GB, an offline drive, and a format-corruption sense after a power cycle. It does not hand a third-party lab a recovery procedure. We will not fill that gap with a promise. Bring the label, the revision if you still have it, and the hour reading if you wrote it down. Leave the drive powered off. Shipping, if it comes to that, is the same packing as any other failed drive.

What the public case log actually has

The public case log has no HPE or Dell SAS SSD that arrived because it hit 32,768 or 40,000 hours. We will not invent a case, a method, or an outcome.

The HPE SAS job in that log is 21607: twelve HPE 1.8 TB 10K SAS hard drives on one SmartArray, a five-drive RAID 5 and a seven-drive RAID 6. SMART showed roughly six years and nine months of runtime before they were put in that server, and the spare-sector pool was already exhausted. Three RAID 6 members needed cleanroom head swaps. Nothing was rebuilt on the original controller. Every member was imaged, and the arrays were reconstructed from the images. Full recovery on that set. The notes on the case say those were decommissioned data-centre drives. The picture is an HPE EG001800JWJNR hard drive. That failure is worn magnetic media and a parity set that was left degraded. It is not a SAS SSD firmware timer. A full recovery there is not a rate, and it is not a prediction for a drive HPE says cannot be recovered after this failure. We don't advertise a success rate.

Used enterprise SAS SSD hour-count FAQ

The model on the label is not in these tables. Does it still die at 32,768 hours?
Not according to HPE's 32,768-hour bulletin. That document says its list of impacted SSD models is comprehensive and that no other drive models are affected. The 40,000-hour HPE bulletin says the same about its own four models. Dell's 40,000-hour article names nine model numbers and says no other Dell Enterprise products shipped with those drives. A model that is absent from the document is outside that document. Look up the model you actually have. Do not borrow the hour count from a different notice.
smartctl shows HPD8, or HPD7, or D417. Is that the fix?
Match the revision to the model's own bulletin. On the 32,768-hour HPE models, firmware prior to HPD8 is the scope, and HPE's fix is HPD8 (or later). HPD7 on one of those models is still prior to HPD8. On the 40,000-hour HPE models, firmware prior to HPD7 is the scope, and the fix is HPD7 (or later). HPE's remediation page says the 40,000-hour firmware and the 32,768-hour firmware are not interchangeable. On Dell's listed models, firmware prior to D417 is the scope, and Dell says D417 or later, applied before 40,000 power-on hours, should address it.
The hours are already past the count and the drive still works. What now?
Copy anything you need off it while it still reads. Leave the only copy somewhere else. HPE ties the 32,768-hour and 40,000-hour failures to firmware older than the fix, at that hour count, and says that after the failure neither the SSD nor the data can be recovered. Dell says a drive that fails the 40,000-hour issue reports 0 GB and becomes unusable after a power cycle. Their update instruction is to upgrade prior to 40,000 hours. A drive that is already past the count on unfixed firmware is past the line those pages describe. Do not load it, and do not power-cycle a Dell drive in that state to see what happens.
It already failed, shows 0 GB, or will not come ready. Can the lab get the files back?
HPE's 32,768-hour bulletin says that after the SSD failure occurs, neither the SSD nor the data can be recovered. The 40,000-hour bulletin says the same. We will not promise a result those sentences rule out. Dell's 40,000-hour article says the drive reports 0 GB, goes offline, and becomes unusable after a power cycle in a format corruption state (it reports 03/31/00). If it has already failed, Dell says to contact Dell Support. That article does not describe a recovery and does not say the bytes are gone. We will not invent either outcome. Stop power-cycling the drive. An evaluation is how the technician on your case sees what that specific drive still does, and we don't advertise a success rate.
Can I build a RAID from four matching pulls if I update the firmware first?
Update every member that is on a list, from the file published for that model, while each drive still works and before it is the only copy of anything. HPE says SSDs put into service at the same time will likely fail nearly simultaneously, and gives a RAID 5 with two failed SSDs as the case where fault tolerance is not enough. Four pulls with the same hours are that situation if the firmware is still the old one. A mirror of two pulls from the same lot fails together for the same reason. Updated firmware is what those bulletins prescribe. Identical hours on unfixed firmware are a bad array.
Is this the used-drive hour rollback, or the SSD that sat in a drawer?
No. A report of hard drives sold with the wear wiped out of SMART is the counterfeit page. An SSD that loses files while it sits unpowered is the unplugged-SSD page. This page is a firmware defect the vendor documented at a fixed power-on-hour count. The hours you read are the hours the drive reports. Power-on hours do not increase while a drive sits on a shelf. The NAND can still fade while it sits, which is the other page.
Is the 70,000-hour HPE notice the same failure?
No. HPE advisory a00111296 says data may become inaccessible at 70,000 hours on VK0800JEABE and VO1600JEABF with firmware HPD3 or earlier, and that this issue is unrelated to the 32,768-hour and 40,000-hour bulletins. HPE says firmware HPD4 prevents that issue and restores a drive that has already hit it. The sentence "neither the SSD nor the data can be recovered" belongs to the 32,768-hour and 40,000-hour bulletins. Do not apply it to the 70,000-hour models.
Is there a case in the public log of a SAS SSD that died at one of these hour counts?
No. The public case log has no intake of an HPE or Dell SAS SSD that failed at 32,768 or 40,000 power-on hours. We will not invent one. 21607 is twelve HPE 1.8 TB 10K SAS hard drives, already carrying roughly six years and nine months of runtime, with the spare-sector pool exhausted. Three members needed head swaps. That set was imaged and reconstructed, and the outcome on that set was a full recovery. Those are spinning disks with worn media. They are not this firmware timer, and that outcome is not a prediction. We don't advertise a success rate.

Read the model, the firmware, and the power-on hours before a used SAS SSD holds data. If the matching bulletin still applies, update it while it works, with a copy elsewhere. If it has already failed the way HPE describes, HPE says the data cannot be recovered, and we will not promise otherwise. Start with a free evaluation.

Request free evaluation →

Free evaluation · No data, no fee · Talk directly with a technician.