During a recent UCS firmware upgrade I had quite a few blades show up with the error “CIMC did not detect storage”. Within UCSM I could see that the blade had a critical alert. It initially started after I upgraded UCS Manager firmware as documents in a previous post I wrote about UCS Firmware Upgrades. I did some searching around to find what may be causing the issue and the best answer I could find was to from the Cisco community forums to disassociate the blade, decommission and reseat within the chassis. I later spoke to a Cisco engineer and he advised of the same steps but that it was also possible to do without reseating the blade. This also looks like its a problem when upgrading from 2.2(1d) to other versions of UCSM but I haven’t been able to validate if it’s only that version or if it also affects others.

The full error I saw was for code F1004 and for Controller 1 on server 2/1 is inoperable. Reason: CIMC did not detect storage

cimc error

Within UCSM I could see there was an issue with the Blade

cimc error server blade

Before proceeding with the upgrade of the FIs, IOMs and Blades themselves I wanted to clear any alerts within UCSM, particularly critical alerts. The steps I followed to bring the blade back online were to go to the blade and select Server Maintenance
cimc server maintenance

I then selected Decomission and clicked Ok

cimc decommission blade

I clicked yes to verify I wanted to decommission the blade.

cimc acknowledge decommission

Next I received a notification to advise that the decommission process has started successfully.

cimc maintenance task started alert

Once the decommission had completed an alert appeared to advise that the blade needed some intervention.

cimc blade slot needs resolution

I right-clicked on the blade and selected Re-acknowledge Slot.

cimc re-acknowledge slot

I selected Yes to verify that it is what I wanted to do

cimc discovery task

A notification then popped up to advise that the server had been re-acknowledged.

cimc successful reacknowledge

A KVM session onto the blade showed that it went through a full detection process again

cimc blade boot cimc blade boot 2

Once the blade had completed the discovery and booted into ESXi again it appeared within UCSM without any issues. The re-discover and reboot took just over 10 minutes. One thing to note is that the rediscovery will strip any labels and naming you’ve added to the blade previously so you’ll need to add them again separately.

cimc alert cleared

Once the CIMC error had been dealt with I was able to continue with the firmware upgrade

2 thoughts on “Cisco UCS – CIMC did not detect storage controller error

Leave a Reply

Your email address will not be published. Required fields are marked *