The IdleTimeLimitThread uses Object.wait(sleepTime) before polling the client connections in order to try idling them out. However the loop over the connections also adjusts sleepTime, and in the customer's case sleepTime became zero. This appeared to cause the thread to wait forever, and the customer observed that connections stopped getting closed due to inactivity.
Some debug instrumentation showed the issue was caused by a connection with an idleTime=1620000, and an idleTimeLimit=1620000.
Changing the idle test to (idleTime >= idleTimeLimit) might be a simple fix, or alternatively adding an additional check to ensure sleepTime is at least 1.