Our @jenkinsci instance is working hard this weekend
oops, it didn't end very well. We've noticed that queue didn't shrink after a few hours. It turned out Jenkins was hanged when trying to schedule new builds although fronted remained responsive.
It was strange because there was no lack of CPU, memory or IO, so I've taken a look at Java Melody monitoring ( https://plugins.jenkins.io/monitoring/ ) and found that thread named "jenkins.util.Timer [#9]" was taking most of CPU time
Stack trace of this thread contained mostly methods (I guess) for working with nodes, labels and queue
You can follow @The_Evernoob.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: