So as many Aussies know, we have 2 time shifts each year for daylight savings, being the first Sunday in April (clocks back 1 hour), and first Sunday in October (clocks forward 1 hour)
Sunday evening I received an alert from my Isilon cluster saying that the time had drifted by more than 4 minutes from the AD time and Auth would be affected.
Sure enough, it had, any CIFS/SMB access was problematic, but NFS was unphased.
After a discussion with EMC support it was determined that there were too many time sources on the cluster. Being that both NTP and SMBTime. SMBTime is a service that pulls time from an Active-Directory, and NTP being Network Time Protocol.
In our setup both eventually point to the same time source. But it became apparent the two services don’t have a precedence order between themselves, and just end up in a race condition.
EMC support were excellent in assisting me to diagnose the issue and provide recommendations to remediate the system.
On EMC recommendation we disabled the SMBTime service and forced a re-sync so NTP would reset the clock. Once done it was all back to normal for data access and the alerts were cleared as a result.
It has left me with questions as the system was setup by an Integrator/Partner, and left in a “unrecommended” setup
More to follow….
