[Systems] CPU usage spikes caused by aggressive buildbot

[Bernie Innocenti]

Thanks, Daniel. Today we didn't get the usual overloading spikes:

  http://munin.sugarlabs.org/sugarlabs.org/sunjammer.sugarlabs.org/cpu.html


On 04/04/14 07:32, Daniel Narvaez wrote:
> Hi,
> 
> I had a bit more time to look into it today and I decided to move to
> periodic scheduling instead of polling for changes. I was already
> planning to implement build triggers on pr submission, so I'll just go
> in that direction now...
> 
> So I don't expect you will see issues like this anymore, but if anything
> looks wrong please let me know.
> 
> 
> On 3 April 2014 17:04, Bernie Innocenti <bernie at codewiz.org
> <mailto:bernie at codewiz.org>> wrote:
> 
>     Since a couple of weeks ago, we've started seeing daily spikes of
>     abnormal
>     "system time" usage on sunjammer:
> 
>      
>     http://munin.sugarlabs.org/sugarlabs.org/sunjammer.sugarlabs.org/cpu.html
> 
>     Today I finally figured out that buildbot pulls from a dozen git
>     repos *in parallel*,
>     probably causing disk trashing:
> 
>     27135 ?        Sl   138:55
>     /srv/www-sugarlabs/buildbot/sandbox/bin/python -c from
>     twisted.scripts import twistd; twistd.run() --no_save
>     --logfile=twistd.log --python=buildbot.tac
>      3559 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/jukebox/mainline.git
>     +master:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Fjukebox%2Fmainline.git/master
>      3573 ?        R      1:53  |   \_ git-remote-https
>     https://src.sugarlabs.org/jukebox/mainline.git
>     https://src.sugarlabs.org/jukebox/mainline.git
>      3571 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/pippy/mainline.git
>     +gtk3:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Fpippy%2Fmainline.git/gtk3
>      3589 ?        R      1:53  |   \_ git-remote-https
>     https://src.sugarlabs.org/pippy/mainline.git
>     https://src.sugarlabs.org/pippy/mainline.git
>      3657 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/terminal/mainline.git
>     +gtk3:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Fterminal%2Fmainline.git/gtk3
>      3670 ?        R      1:54  |   \_ git-remote-https
>     https://src.sugarlabs.org/terminal/mainline.git
>     https://src.sugarlabs.org/terminal/mainline.git
>      3664 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/log/mainline.git
>     +master:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Flog%2Fmainline.git/master
>      3683 ?        R      1:54  |   \_ git-remote-https
>     https://src.sugarlabs.org/log/mainline.git
>     https://src.sugarlabs.org/log/mainline.git
>      3710 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/sugar-base/mainline.git
>     +master:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Fsugar-base%2Fmainline.git/master
>      3724 ?        R      1:54  |   \_ git-remote-https
>     https://src.sugarlabs.org/sugar-base/mainline.git
>     https://src.sugarlabs.org/sugar-base/mainline.git
>      3728 ?        S      0:00  \_ git fetch
>     https://src.sugarlabs.org/chat/mainline.git
>     +gtk3:refs/buildbot/https%3A%2F%2Fsrc.sugarlabs.org
>     <http://2Fsrc.sugarlabs.org>%2Fchat%2Fmainline.git/gtk3
>      [...and many more...]
> 
> 
>     Is there a way to run these sequentially? Additionally, since this
>     is a batch process,
>     can we run the whole thing with lower priority ("nice ionice -c2 -n6
>     COMMAND").
> 
>     --
>      _ // Bernie Innocenti
>      \X/  http://codewiz.org
> 
> 
> 
> 
> -- 
> Daniel Narvaez
> 
> 
> _______________________________________________
> Systems mailing list
> Systems at lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/systems
> 


-- 
 _ // Bernie Innocenti
 \X/  http://codewiz.org