[Systems] Randomised backup - puppet

[Sascha Silbe]

On Sat, Feb 27, 2010 at 09:20:28AM -0600, David Farning wrote:

[Backup timing]
> We have a number of options for dealing with this issue.  The first 
> and
> easiest to implement is to have puppet pick a time random(X) minutes 
> after
> midnight local time to trigger a job. (The same can apply to build 
> runs)
How long is a single backup run?
For the buildslaves (which took ~1h for all packages when run in 
parallel) I've assigned start times manually, 2h apart -- this gives 
plenty of room to grow and the current trend is less packages, not more 
as our dependencies finally start appearing (in suffciently recent 
versions) in the distros. This (manual schedule) gives fully 
deterministic resource usage without spikes / bursts.

> Non-deterministic things seem like they might be evil to the sysadmin 
> way.
Full ack. :)

> But, one area it might be useful is providing build VMs 'on demand'
Is it worth the effort? Would using some Cloud stuff reduce the overhead 
on bender significantly?
The answer might very well be yes as each KVM instance tends to take a 
few percent of CPU (with Fedora seeming to take about twice the amount 
the Debian ones do) for a reason I haven't investigated yet.
As of now I'm unsure whether

a) investigating the cause of the background CPU consumption and fixing 
it if possible (e.g. NOHZ / dyntick / tickless timer) or
b) instantiating VMs only on demand (how does this work? will cron jobs 
continue to work?)

will provide better bang-for-buck.

CU Sascha

-- 
http://sascha.silbe.org/
http://www.infra-silbe.de/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
Url : http://lists.sugarlabs.org/private/systems/attachments/20100227/043c2d11/attachment.pgp