[Systems] Randomised backup - puppet

[David Farning]

On Sat, Feb 27, 2010 at 9:52 AM, Sascha Silbe <
sascha-ml-ui-sugar-systems at silbe.org> wrote:

> On Sat, Feb 27, 2010 at 09:20:28AM -0600, David Farning wrote:
>
> [Backup timing]
>
>  We have a number of options for dealing with this issue.  The first and
>> easiest to implement is to have puppet pick a time random(X) minutes after
>> midnight local time to trigger a job. (The same can apply to build runs)
>>
> How long is a single backup run?
> For the buildslaves (which took ~1h for all packages when run in parallel)
> I've assigned start times manually, 2h apart -- this gives plenty of room to
> grow and the current trend is less packages, not more as our dependencies
> finally start appearing (in suffciently recent versions) in the distros.
> This (manual schedule) gives fully deterministic resource usage without
> spikes / bursts.
>
>
>  Non-deterministic things seem like they might be evil to the sysadmin way.
>>
> Full ack. :)


I guess that this is an example of the simple, cheap and obvious answer
_not_ being the best. :)  I'll set if puppet can handle a construct such as:
for node in nodes_to _be_backed_up{
  set node backup_start_time  = midnight + delay
 delay = delay + delay_interval
}


>  But, one area it might be useful is providing build VMs 'on demand'
>>
> Is it worth the effort? Would using some Cloud stuff reduce the overhead on
> bender significantly?

The answer might very well be yes as each KVM instance tends to take a few
> percent of CPU (with Fedora seeming to take about twice the amount the
> Debian ones do) for a reason I haven't investigated yet.
> As of now I'm unsure whether
>
> a) investigating the cause of the background CPU consumption and fixing it
> if possible (e.g. NOHZ / dyntick / tickless timer) or
> b) instantiating VMs only on demand (how does this work? will cron jobs
> continue to work?)
>

c) what about memory?  My limited understand is that because the vm never
fully sleeps, much of the vm remains in memory rather than swapping out.


> will provide better bang-for-buck.
>

+1. In my mind, the only reason for for using eucalyptus is if it provide a
sane and easy method of instantiating vms on demand.  It has been rather
difficult to figure this out. One has to wade though a lot of marketing
propaganda to find any useful articles on clouds.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.sugarlabs.org/private/systems/attachments/20100227/47b17223/attachment.htm