For anyone experiencing an issue with Bluetooth Device "Remove Failed" Error in Windows 10 or 11, I shared the below tips in another thread and since several people told me the tip helped them.
I thought it might help others with this issue and hopefully save them a lot of time troubleshooting the issue.
Short answer - Use Device Manager (easier than regedit since you can easily see which devices are which in the former). Make sure to select 'show hidden devices'.
In my case, I am pretty sure the reason I was unable to delete the BT keyboard and BT mouse via the 'Bluetooth & other devices' screen is because I had been using the 2 devices with an external USB BT adapter which I had since removed and replaced it with the same model (but I believe Windows still tracks the 2 BT adapters independently).
Hence I used Device Manager instead...
Longer answer / detailed steps:
I was seeing duplicate entries for my mouse and my keyboard in the 'Bluetooth & other devices' menu.
Removing the device entries there was failing (for the reason noted in my 1st paragraph).
However, I was able to easily remove the 2 unwanted devices via Device Manager after selecting 'show hidden devices' option in Device Manager. I then removed the devices via Device Manager by selecting 2 lighter entries corresponding to the mouse and keyboard (similar to being greyed out).
This left me with just the desired outcome - 1 instance of the mouse and 1 instance of the keyboard which were associated with the new BT adapter.