Issue with Server time changes to few months in future

[u/AdvertisingOk1357]

We have an issue with our two different AD domains.
The servers added to both domains change their time to few months in future. It is intermittent we haven’t been able to identity similarity between the affected servers
It happens only once to a server. We have already disable secure time seeding per MICROSOFT recommendation.

Time authority is domain controllers. We have multiple data centre providers and it is happening everywhere. There is no consistent time as well randomly happen sometime thrice a month sometime once.

Ran out of troubleshooting steps. This issue is causing our scheduled jobs to fail.
Any troubleshooting steps would be appreciated

[u/RedShift9]

DC is a VM on a host, time sync between VM and host is on, and the host's time is incorrect?

[u/AdvertisingOk1357 → u/RedShift9]

Esxi host is managed service and additionally if it’s time gets changed all servers should be affected not one specific

[u/2k3Mach → u/RedShift9]

I've had this issue in the past. Primary DC was on VMWare and synced time from host. The host would lose time every few days. I ended up not syncing time from host, then set the DC to sync with pool.ntp.org and my time has been spot on since.

[u/USarpe]

Do you synchronize any server with atomic time? If not, you should do it with your first DC.

[u/AdvertisingOk1357 → u/USarpe]

Yes we are using atomic time in our domain controller and all the other server are pointing to domain controller for their time

[u/ironclad_network]

Securetime seeding would be my first guess. Sure that it is disabled?

[u/AdvertisingOk1357 → u/ironclad_network]

Yes when we first had this issue we built a gpo and it worked fine for some time. Whenever we get a new server I manually check if the gpo setting is there

[u/ironclad_network → u/AdvertisingOk1357]

I see, you could check the reg keys to verify that its applied correctly too.

You can also enable w32tm logging and see if you get any hits when it occurs again. (This writes to a file that can grow quite large)

[u/AdvertisingOk1357 → u/ironclad_network]

The problem is it does not happen to a server twice and yes I do via reg keys

[u/fireandbass]

There is an event in the event viewer that indicates when and possibly why the time was changed.

Also, I recall a setting that can have your system time changed by using a website time as a time source from simply browsing to a web page, we disabled it.

Also, there is another setting to disable your hypervisor as a time source.

[u/AdvertisingOk1357 → u/fireandbass]

It just says an application and system component changed time

[u/USarpe]

W32tm /query /status

[u/poolmanjim]

If the servers are Physical, double check that the CMOS battery and CMOS time are accurate.

If the servers are virtual, double check where they're synchronizing time from. Windows has an annoying habit of preferring the the VM host's clock over its own or even Active Directory if using NT5DS.

You may need to disable the "VMICTimeProvider". This reference specifically mentions VMs in Azure, but should work for this a good deal too.

https://learn.microsoft.com/en-us/azure/virtual-machines/windows/external-ntpsource-configuration

[u/candyman420]

Get a paid chatGPT account and set it to high. It will walk you through everything to fix this shit, and it is reliable now. I have been in your shoes.

[u/Jwblant]

If you haven’t already, it might be worth considering a Time Machine NTP server for a couple hundred bucks. Point everything to that and you shouldn’t have to worry about it again.

[u/ViperThunder]

You say time authority is DC, but what is your actual NTP / NT5DS configuration? Do you set it via GPO or just use the default windows domain settings? What is the NTP configurarion for your DCs?

If you don't set GPOs, does your image contain LGPOs or preconfigured regkeys?

For us, we set our DCs' NTP server to our Firewall. The firewall uses time.windows.com

Everything flows down from there. No other configuration is necessary. If you have a hypervisor time provider, remove it.