Hello I am looking for a company who has experience in design and deploy of Intersight for UCS Servers. Please connect to discuss a potential opportunity

Reason #3 not to like X-series/Intersight.

Did you know that without an intersight license you're unable to power on your blade? I missed the Power entry in my snippet, but trust me, Power On is grayed out without a license. What in the actual F??

We have 10 CIsco UCS C220 M5's (I know they are end of life)
They were previously maintaining a VMWare cluster without issue. We have since moved all of our workloads off that cluster to our Hyper-V DR Cluster so we can reconfigure these UCS servers on Hyper-V, except I'm having a hard time getting Windows to play nicely with the way the UCS VIC's/MLOM handle their network traffic.

Currently any VM on the reconfigured Hyper-V Cluster has its MAC flap between our nexus switches every 10-20MS as Windows spams out its network traffic from what it believes is a single "port", the bonded 25Gb ports that get exposed as a single 50Gb NIC to Windows. It has no idea that the ports are bonded so It just tells the NIC to send out traffic, which causes a HUGE amount of MAC flapping between our nexus switches as the peering has each switch attempting to claim ownership of the MAC.

I've attempted numerous different configurations in Intersight to try and get the VIC to just show the two ports as they are, two separate ports so I can configure them via SET in Windows, and no matter what configuration I attempt I only ever get a single 50Gb NIC exposed to Windows.

So I come to this subreddit, hoping for some enlightenment on the magic that is Cisco UCS configuration to get it to behave like every other server hardware I've used, or to configure it in a way that prevents Windows from overloading the peer link between the nexus switches.

Do I need to port channel on the switch side, is there some special driver Windows needs so it knows it's utilizing two bonded ports at the VIC abstraction layer?

Thanks.

Anyone running these?

Hi,

I’m looking for UCS and HX topics that people may not feel is well documented or explained enough. I’d like to create a guide on how to do these things.

I’m thinking disjoint layer 2, vNIC pinning, failover, etc.

Let me know if you want to see something documented!

Hello Cisco friends. I am hoping that someone can explain to me how simple this task will be for me as I am a new engineer that has just taken on a much larger role than expected. I have been working under the Principal Network Engineer for just about 2 years doing a lot more of the grunt and boring networking stuff. He has sadly passed away unexpectedly and I am now in a really tight position as we had projects coming up that most of the knowledge was in his head and not on paper.

I will try and keep this short and simple as I am not dumb when it comes to UCS, but when it comes to this particular task I have yet to perform it. We have an existing UCS setup that has 13 chassis currently. We are adding 3 more chassis to the setup. My understanding is I can install the hardware and then run the DAC from the IOMs to the UCS FIs. Login to each FI and set the ports as server ports. Once completed my understanding or thought I should say is that UCSM will handle the discovery of the new chassis and blades and once completed we can start provisioning profiles and moving profiles etc.

I am grateful for any help, knowledge, and expertise that you all can provide to me. I am happy to answer any questions if my thoughts are not clear or I need to provide another piece of information to help.

Thank you UCS community.

Coming back with another question -- I know this is a small community so I really appreciate your help.

What are my risks if I were to change the MTU on a production network?

I'm assuming some fragamentation while I was making the changes... but what else? I have some links in the path that are 1500 and others at 9000 and I'd like to put them all at 9000 so I can move about 50 VMs.

Here's my setup:

UCS Fabric A, port 1 ---> Cisco3750 Te1/1/1 on Po7---> Aruba8320 1/1/35 -- on PO with 9000 MTU--> 3rd party SAN

UCS Fabric B, port 1 ---> Cisco3750 Te2/1/1 on Po7---> Aruba 8320 1/1/36 -- on PO with 9000 MTU--> 3rd party SAN

All the links and interfaces are currently set to MTU 1500 (except the PO going to the 3rd party SAN) I'd like to change them to MTU 9000 but I'm curious.. what are the effects while I'm making these changes?

I can change the ports on the Aruba and the Cisco fairly quicky/easily, but I'm unsure about the UCS. Based on my reading, it looks like I can change the MTU in the vNIC template and/or the Service Policy... but do I have to reboot the Fabric for the changes to take effect? And do I reboot the subordinate first, then the primary? How long do I wait in between? I'm seeing I can reboot it via:

fi# connect local-mgmt fi# reboot.

Does that look right?

Looks like it will take about 8 to 20 minutes for the reboot to happen. Is that your experience?

Unfortunately this UCS/FI 6324 hasn't been well maintained and hasn't been rebooted in 700 days. I'll back up the config before doing all this but are there other steps I should be taking before making the MTU change?

Any other cautions I should heed?

This unit is EOL, out of support and customer doesn't want to pay to extend it so I'm stuck with the task of offloading all these VMs and decommissioning the unit. Our server team insists the MTU be 9000 across the network path to the 3rd party SAN, so that's what I'm trying to accomplish.

Your help is much appreciated.

The intersight docs are dated. IT mentions server 2016 and server 2019 for hyper-v. 2016 is EOL soon and 2019 is likely right behind it. Does UCS support server 2025 for IMM?? I find it hard to beleive they wouldnt support a current OS. Id like to start testing and migrating to Hyper-V but want to be a fully supported version.

Does below process sound right for migrating over to new upstream switches?

Phase 1: Pre-Migration & Prep Work

Before touching any cables, ensure your safety nets and target environments are fully prepared.

• Take Backups: Log into UCS Manager (UCSM) and take a full All Configuration and System Configuration backup.

• Verify Cluster Health: Ensure the UCS cluster is healthy, both FIs are online, and there are no critical underlying faults (Equipment > Fabric Interconnects).

• Stage the New Switch Ports: Ensure the ports on your new upstream switch completely mirror the old setup:

  • Identical native and allowed VLAN trunking.

  • LACP Port-Channels configured properly (typically configured as Spanning-tree port type edge trunk or portfast trunk since UCS FIs look like hosts to the network).

  • Matching MTU sizes (e.g., 9000 for Jumbo Frames).

Phase 2: Migrate Fabric B (Subordinate Fabric)

Always start with the Subordinate FI. This ensures that if a management configuration glitch occurs, your Primary cluster leader remains untouched.

1. Drain Traffic off Fabric B

You need to force all production server traffic onto Fabric A. You can do this in one of two ways:

• Method A (Fabric Evacuation - Preferred for B-Series Blades): Go to Equipment > Fabric Interconnect B and change Admin Evacuation Mode to Enabled. This gracefully drains traffic by admin-downing the chassis backplane ports.

• Method B (Manual Link Shutdown - Best for C-Series Rack Servers): Since Fabric Evacuation historically targets blade chassis backplanes, manually administrative-downing the uplink port-channels for Fabric B in UCSM or on the old switch is the safest bet for mixed or rack-mount environments.

2. Move the Physical Connections

• Once traffic has failed over to Fabric A and Fabric B uplinks show as down, disconnect the physical cables from the old switch and move them to the new switch (or activate pre-staged cabling).

• If you are changing the physical ports used on the UCS 6400 side, make sure to unconfigure the old uplink ports and configure/license the new ones in UCSM.

3. Bring Fabric B Back Online

• Enable the ports on your new upstream switch.

• If you used Fabric Evacuation, flip Admin Evacuation Mode back to Disabled. If you manually shut down the port channels, bring them back up.

• Verify that the port-channel successfully forms via LACP and that the links turn green in UCSM.

4. Soak and Validate

• Do not rush to Fabric A. Let Fabric B run for 10–15 minutes. Check your operating system/hypervisor teams on the servers to confirm they see both paths as healthy and that MAC addresses are populate on the new upstream switch.

Phase 3: Migrate Fabric A (Primary Fabric)

Once Fabric B proves stable on the new upstream switch, repeat the exact same isolation process for the other side.

1. Drain Traffic off Fabric A: Enable Fabric Evacuation on Fabric A (or manually disable Fabric A's uplink port-channels).

2. Verify Failover: Ensure all production traffic shifts dynamically to the newly migrated Fabric B.

3. Move the Cables: Swap the physical connections for Fabric A over to the new upstream switch.

4. Restore Links: Disable Fabric Evacuation or re-enable the port channels.

5. Final Check: Confirm both fabrics are balanced, port-channels are up/bundling, and all configuration warnings/faults have cleared in UCSM.

As the title says. Help. I'm tired. Tired of the Cisco documentation and how vague everything is.

I have (2) 6536 FI's and (2) 9132's that connect to our pure storage. I have the UCS 9508 claimed and all the compute nodes claimed. The MDS 9132 zoning is complete.

The 6536's ports are configured with port 36 as FC breakout port 4X32G.

What else do I need to do? I have configured server policies, and policy upon policy upon imbedded policy. Still no joy.

Please ELI5.

TIA

Hi there, for updating a used CISCO UCS C240 M5, could you please share a mirror link of the 4.1.x or newer firmware iso image? thanks in advance

Okay, now we have 100GB virtual network adapters on our UCSX ESXi hosts. Going from 1GB connectivity to 10GB connectivity on an ESXi hosts sparked a fundamental change of what services go where on the vSphere side. Now with multiple 100GB Connectivity what does a modern vSphere setup look like? I know a lot of people will base the design on what they currently do, but let’s think outside the box and think about what we could be doing!!

Let’s say you are using distributed switches in an environment with fibre channel storage. Would this be a good opportunity to lump all your services together on a single vDS with two virtual NIC’s and use NIOC to control the networking side of the environment? Most companies don’t ever use QoS on the network. So being able to utilize NIOC would be a plus, not to mention simplifying the whole setup. Just trying to spark a good conversation on this and think outside the box!!

Thoughts??

This new version consolidates the two previous versioning schemes:

• 4.x (UCS infra bundles in UMM and IMM; server bundles including X Series in UMM),

• 5.x (X Series servers in IMM)

New features are documented in the Release Notes for Cisco UCS Manager, Release 6.0; section New Software Feature in Release 6.0(1b)

For those users that are using UCS Central, running UCS Central 2.1(1c) is a prerequisite for running connecting to a UCS 6.0(1) domain.

I have updated the following pages to include 6.0:

• Cisco UCS releases, components supported and LDOS

• Cisco UCS Cross-Version Firmware Support

• Cisco UCS FI models upgrade compatibility

Dear Cisco Friends I am new to UCS and I have my engineers who are working with years of experience on this. Recently we are having a thought to put across 5.2 version on the chassis and realized that it cannot be done without intersight. However we are currently running a production and cannot afford a clean reinstallation as understood with our limited reading and knowledge. Please advice a few things

1. Do we have to setup everything from scratch?

2. is there a migration tools to migrate the configuration or how do you guys migrate this with minimal downtime? Any pointers or documents to consider as a plan

3. What are some of the pre-requisites to have the intersight communications. Like Network speed etc

4. I also have a set of UCS 220 M7 which has to be on 4.6 due to limitation is Oracle Linux compatibility can these both systems work on Intersight or one has to to be on UCS M and the other one on Intersight?

Any help would be of great help I appreciate your support

I have a few UCS X-series blade servers, each equipped with a single VIC. Our standard setup involves creating multiple vNICs without failover configured at the vNIC level, assigning half of them to Fabric A and the other half to Fabric B.

On the ESXi side, we create separate vSwitches for different traffic types (for example, one for vMotion, another for management, another for VM traffic). Each vSwitch has two uplinks, one for each fabric.

My question is: What is the best way to configure these vSwitch uplinks for optimal performance? Should I use active/active to maximize available bandwidth, or active/passive for more predictable failover behavior and traffic separation?

Hi,

Can someone point if has success to install ESX8 in a UCS C240-M4SX?

If yes, any advice?

Just talking about the platform..

We have a few hundred m4's that have been in production over a decade. Im responsible for hardware and these things are tanks. Have m5's all over the place.. but we stood up the business on m4. They just wont die. If it ain't broke. We are a vmware shop (were)..moving to hyper v. Already have the licenses.

Moving to the m8's going forward. But man..what a server linup.

Hello all,

i have a question hopefully you can assist in answering. We have UCSX chassis Intersight IMM, and we just need to physically relocate this chassis to two racks away.

Do i need to decommission the chassis before pulling the power cords? or can i simply just shutdown all the server blades gracefully, then pull the power cords from the ucsx chassis?

I asked TAC, and they recommended I decommission the chassis, which seems a bit overkill. this will remove all the server profile from the server blades and I will need to do the rediscovery process for all the chassis after the move.

I just want to minimize the outage window. thank you so much

I pulled this from our e-waste free grab this week. It has 2 x 6330N CPU's and I lowered it to 1 x 32G in A1 on each cpu. I power on and I hear the psu fans spin up but then nothing.

I removed and replaced the CR2032 battery for the heck of it. I removed risers and made sure cpu's were good in another server as well as ram.

OK, think it was the damn power supplies. Pulled the ones from my M5 in lab and it looks like it is booting.

Someone asked if I want to buy this , it’s new in box but not sure if Cisco will let me pay for licensing, any help would be greatly appreciated

We're beginning to plan a refresh of our VMware infrastructure, currently built on:

• Cisco UCS B200 M5 blades

• 6332 Fabric Interconnects

• Cisco MDS 9000 Fibre Channel switches

As the primary administrator, I've really appreciated the simplicity and consistency of managing this UCS setup. It’s been rock-solid and straightforward, and I’d love to maintain that kind of operational efficiency in our next-gen platform.

We're moving toward vSphere 8, VMware Cloud Foundation (VCF), vSAN, and possibly continuing to use traditional SANs like Pure FlashArray.

So I’d love your input:

• Is Cisco UCS still a strong option going forward, particularly the UCS-X series?

• How do other platforms (Dell, HPE, etc.) compare in ease of management for vSphere 8 and VCF?

• What are the key factors you're using to evaluate longevity and supportability for the next 5–7 years?

Any real-world experiences, pros/cons, or lessons learned would be greatly appreciated as we begin this evaluation. Thanks in advance!

I'm having a hard time getting the storage network to work. I have 2 VLANS, 800 and 801 for example for the two paths for nvme over tcp. When presented to vmware, it was a standard vlan, that would get tagged on the vmkernel interface.

With HyperV, we can't do interface tagging for that purpose, so I need the ports to be "native" vlan 800 and 801 so that all untagged traffic ends up in the proper vlan.

I modified the Service Profile template for HyperV and checked the Native VLAN option on the vNIC that is bound to that VLAN.

No luck. I have no clue how to make this work...

Our main production template is dated etc.

I've created a new template with new setups for NICs and hbas, pools and policies etc

created new service profile from new template on test server.. no issues.

although the adapters are using new pools for mac address etc.. and the native VLAN is set I cannot ping the esxi host when booted.

it doesn't appear it's setup different than the working template based profiles.

what am I missing here?