[Sugar-devel] Jabber Testing Monday Aug 24

[Caroline Meeks]

 Again we are working with
XS1 - responding to jabber.sugarlabs.org. Installed a few months ago and
currently with >1000 registered users, it tends to have about 20 people on
it these days.
XS2 - Installed a few weeks ago, responding to
schoolserver.xsdev.solutiongrove.com with about 100 registered users

First i connected to XS1 and joined a Turtle Art instance.  CPU usage rose
to 100%.  When we stopped sharing the TA it dropped back down.
We then shared a chat and CPU again rose to 100%.

Next we switched to testing on XS2.  Sharing Turtle Art and Chat did not
spike the CPU.

We felt there were 4 differences between XS1 and XS2


   1. The number of registered users
   2. XS 1 is an older version of the XS 0.5.2. XS2 is 0.6beta
   3. XS 2 has Gadget
   4. XS1 had about 20 users logged in XS2 had 4 or 5

Next we turned off Gadget on XS2 and repeated our test. No CPU spike.  Thus
we think #3 is unlikely to be the important difference.

Next we deleted the registered users on XS1 and switched our testing back to
XS1.  When we shared TA and chat there was still spikes up to 100% CPU.
 Thus we think #1 is unlikely to be the important difference.

We will work on this again tomorrow so let us know your ideas

A separate issue is that it hard to get the computers to connect up the the
XS. They tended to stay on Salut, sometimes they would goto Jabber on
resstart, other times not.  We could really use better control and the
ability to force one or the other.

Here is the log of our conversation.


Josely is Caroline at the GPA School
12:06 < joasely-2260> the jabber server is back up to 100% CPU usage.
12:06 < joasely-2260> 22 people are online. over 1000 are registered
12:06 < joasely-2260> one turtle Art is being shared.
12:06 < joasely-2260> the load did not seem to change when the GPA computers
I am controlling stopped sharing turtle art
12:07 < joasely-2260> current hypothisis - its th 1000 registered users that
is making the difference between decent performance and 100% CPU
12:08 < daveb> jabber is back to 0%cpu
12:09 < joasely-2260> Interesting note. The Red Turtle Art activity seems to
have been started by David who is not online anymore.
12:09 < joasely-2260> ah very very interesting.
12:09 < joasely-2260> let me join the turtle art again.
12:09 < daveb> joasely-2260: ok
12:09 < daveb> we have cpu logs every minute back until august 15
2:09 < joasely-2260> ok I have joined the Turtle Art

12:10 < joasely-2260> lets give it 5 minutes with just one person sharing
it. then I'll add another peron

12:10 < daveb> up to 100%
12:10 < joasely-2260> so note, we can scratch they hypothisis that its the
1000+ registered users.
12:10 < daveb> yes
12:11 < joasely-2260> ok very cool.

12:11 < joasely-2260> so sharing a turtle art activity causes 100% CPU

12:11 < joasely-2260> I'm gong to quit TA
12:11 < tomeu> erikos: yes
12:11 < satellit_> daveb: that TA has been up for several days

12:11 < joasely-2260> let me know when its back down and I'll try a chat.
12:12 < joasely-2260> thanks satelit_

12:12 < satellit_> I think it is an orphan

12:13 < daveb> satellit_: yes but when someone joins te cpu spikes
12:13 < daveb> joasely-2260: should we quit that and try a different
activity?
12:13 < daveb> unfortunatly i tink debug is off on jabber

12:13 < daveb> ok cpu back to 0%

12:14 < satellit_> daveb: is there a way to kill orphan shared items?
12:14 < daveb> satellit_: i have no idea

12:14 < joasely-2260> started chat on one computer

12:14 < daveb> cpu 100%

12:15 < joasely-2260> wow before I even joined it anywhere else
12:15 < daveb> yes
12:15 < joasely-2260> still at 100%?
12:15 < daveb> yes
12:15 < daveb> now
12:15 < daveb> did we ever get this on the 0.6 server?
12:15 < joasely-2260> still at 100%?
12:15 < daveb> 60% now
12:15 < daveb> down to 0
1
12:15 < joasely-2260> i'm not sure I thought both XS1 and XS2 was 0.6

12:15 < joasely-2260> ok now going to join with another person
12:16 < daveb> joasely-2260: no XS1 is 0.5.2
12:16 < joasely-2260> ok second person joined
12:16 < tomeu> erikos: fortunately is not big nor invasive, so won't be such
a big deal
12:16 < daveb> cpu between 0 and 3 %
12:17 < joasely-2260> hmm ok I'll chat with myself

12:17 < joasely-2260> anything?
12:17 < daveb> cpu up to 100

12:17 < joasely-2260> ok so doing anythign with collaboration pops the CPU
to 100%
12:18 -!- iwikiwi-1f30 (n=urk at 61.17.45.12) has quit: Read error: 60
(Operation timed out)

12:18 < joasely-2260> should I try setting these to XS2?
12:18 < joasely-2260> and doing the same test?
12:18 < daveb> sure

12:18 < daveb> let me login over there
.
12:19 < daveb> joasely-2260: ok i am ready
.
12:20 < joasely-2260> schoolserver.xsdev.solutiongrove.com right?
12:20 < silbe> tomeu: not logging.exception("message") ?
12:20 < joasely-2260> I'm back to salut :(
12:20 < phitoo> sdziallas: Remind me where to get the rpm. That's not part
of the distribution, right.
12:20 < satellit_> no it hangs on boot. do I have to do an interactive
command? on boot
12:20 < daveb> joasely-2260: correct

12:21 < joasely-2260> hmm maybe I should reboot

12:22 < daveb> joasely-2260: it should work without restarting but sometimes
that helps to get telepaty working again
1
12:24 -!- CarolineGPA-2260 (n=urk at 216.163.216.2) has joined #sugar
12:24 < CarolineGPA-2260> nope still salut
12:24 < daveb> oh i bet its misconfigured
12:24 < daveb> when you change te domain name it messed up ejabberd
12:25 < CarolineGPA-2260> how?
12:25 < tomeu> erikos: it should conflict

12:25 < daveb> CarolineGPA-2260: ejabberd is bound to the host name
12:25 < daveb> kind of annoying
12:25 < CarolineGPA-2260> ah maybe if I delete the keys it'll work again?
1
12:25 < CarolineGPA-2260> so right now you can't just change your host name
and connect to a new jabber server?

12:26 < CarolineGPA-2260> should I try deleting the owner keys and restart?
12:27 < daveb> no
12:27 < daveb> one sec
12:27 < daveb> its the server side

12:32 < daveb> somethinbg is screy
12:32 < daveb> the ejabberd scripts are WAY too picky you need to restart
the whole server
12:32 < daveb> i thought I fixed that but must have missed something.
12:32 < CarolineGPA-2260> ok
12:32 < CarolineGPA-2260> how long do you think the fix will take?
12:33 < daveb> a minute or two
12:33 < CarolineGPA-2260> oh ok :)

12:37 < daveb> CarolineGPA-2260: well the server did not restart and I can't
login to the host machine either
12:38 < daveb> jabber.sl.o is back to 100%
12:38 < daveb> ls
12:41 -!- teefal (n=teefal at 24.229.118.195) has quit: "The computer fell
asleep"
12:41 < daveb> CarolineGPA-2260: it looks like jabber is stll broken on XS2
12:41 < daveb> it does not wish to behave
12:42 < CIA-73> sugar: sayamindu * rddfc2879a308 / (bin/sugar-session
extensions/cpsection/keyboard/view.py): Using logging.exception
12:44 < daveb> CarolineGPA-2260: nevermind i think its working now :)

12:46 < CarolineGPA-2260> back I had a call
12:46 < CarolineGPA-2260> should I give up for today?
12:46 < daveb> CarolineGPA-2260: it looks like te server is up
12:46 < daveb> 1 user online
12:46 -!- tamiel (n=tamiel at 213.30.183.226) has quit: "leaving"
12:46 < CarolineGPA-2260> are you able to connect?
12:47 < daveb> i did not try yet
12:47 < CarolineGPA-2260> ok I'm restarting gpa2
12:48 -!- kristianpaul (n=kristian at 190.165.132.161) has joined #sugar

12:49 < daveb> i am on xs2
12:49 < daveb> i see caroline1
12:50 < CarolineGPA-2260> great i see you and caroline2
12:50 < CarolineGPA-2260> what is the CPU?
12:50 < daveb> 0 now
12:50 < daveb> i don't see you anymore
12:50 < CarolineGPA-2260> ok starting a chat
1
12:50 < CarolineGPA-2260> chat is starting, one user

12:50 < daveb> CarolineGPA-2260: nevermind i had typed in seach
12:51 < daveb> CarolineGPA-2260: cpu 0
12:51 < daveb> CarolineGPA-2260: 1%
12:51 < CarolineGPA-2260> add a person

12:52 < daveb> it goes to 1% for a second then back to 0
12:52 < daveb> CarolineGPA-2260: want to start turtle art?
12:52 < daveb> CarolineGPA-2260: also not gadget is configured on here

12:57 < CarolineGPA-2260> dabeb hows the cpu doing?

12:57 < daveb> CarolineGPA-2260: 0%
12:57 < daveb> i shared tutleart

12:58 < CarolineGPA-2260> joining TA
12
12:58 < daveb> CarolineGPA-2260: ok. cpu went to 1% for a second

12:59 < CarolineGPA-2260> joining TA with 2

12:59 < daveb> still at 0
12:59 < daveb> goes up when you actually do something then back to 0
12:59 < daveb> of cours eonly 3 users online

12:59 < daveb> need 20 or so to compare better
12:59 < tomeu> silbe: ok, I guess you can recover it from git if you want
12:59 < silbe> tomeu: but good to know it's buried in git somewhere and i
just need to revive it :)
13:00 < silbe> tomeu: similar to the data store test suite :)
1
13:00 < CarolineGPA-2260> ok so we have 3 differences. 1- 3 users vs 22
users.  2 - less then 100 registered users vs 1000+ registered users and 3.
XS .52 vs .60
13:00 < CarolineGPA-2260> correct?

13:00 < daveb> CarolineGPA-2260: gadget is on XS2

13:01 < CarolineGPA-2260> ok does that make difference #4?

13:01 < CarolineGPA-2260> does it make sense to turn off gadget and see if
we see a differnece?
13:02 < daveb> CarolineGPA-2260: sure let me see if I can actually do that

13:02 < CarolineGPA-2260> ok I'm gong to run downstairs, they just annouced
free food ;)

13:03 < CarolineGPA-2260> I need to chat up the teachers too so I maybe a
while if I'm lucky

13:03 < daveb> ok
13:03 < daveb> CarolineGPA-2260: ok i restarted without gadget


13:09 < CarolineGPA-2260> looks like we have 5 connected. me 3 times you
twice
13:09 < CarolineGPA-2260> hows the cpu daveb?

13:18 < CarolineGPA-2260> I'm adding some more computers here. they were all
unplugged
13:18 < daveb> ok
13:18 < daveb> i see tem

13:35 < CarolineGPA-2260> daveb, how is the CPU?

13:35 < daveb> CarolineGPA-2260: <1%
13:36 < daveb> CarolineGPA-2260: i shared a new turtle art with 2 people
13:36 < CarolineGPA-2260> we are now running without gadget?

13:36 < CarolineGPA-2260> i'll join in
13:36 < daveb> CarolineGPA-2260: without gadget, correct


13:38 < CarolineGPA-2260> GPA1 fell back to salut

13:38 < CarolineGPA-2260> GPA2 and 3 are in the TA I think.

13:38 < CarolineGPA-2260> pretty messed up sharing though

13:40 < CarolineGPA-2260> hows the cpu?

13:41 < daveb> CarolineGPA-2260: 0 still

13:42 < daveb> CarolineGPA-2260: its not clear exactly what is different
1
13:42 < CarolineGPA-2260> daveb sent you a video of how it looks on the
computers here.
13:43 < CarolineGPA-2260> lets do a chat too
13:43 < daveb> ok
13:43 < daveb> CarolineGPA-2260: shared chat

13:44 < CarolineGPA-2260> joined
13:44 < CarolineGPA-2260> hows it looking
13:45 < daveb> cpu went to 9.6% then right back to 0
13:45 < CarolineGPA-2260> GPA2 has now lost connection to the jabber server
13:45 < CarolineGPA-2260> ok so the experiement was, was Gadget what was
making the difference
13:45 < daveb> CarolineGPA-2260: weird. still 0%
13:46 < daveb> CarolineGPA-2260: right it did not appeat to affect with a
small number of online users
13:46 < CarolineGPA-2260> and the result was no. It works fine with Gadget
off or on.
13:46 < CarolineGPA-2260> under these conditions

13:46 < CarolineGPA-2260> so we have 3 other difference to look at 1.
registered users, 2. .52 vs 6 3. number of total users logged on
13:46 < CarolineGPA-2260> does that sound right?
13:46 < daveb> CarolineGPA-2260: correct.
13:46 < daveb> so there could be some weird interaction wit
13:47 < daveb> collab wit a large number of registered users
13:47 < daveb> like it has to loop through them or someting weird.

13:47 < CarolineGPA-2260> ok so one option is   to switch the DNS so
jabber.sl.org points to XS2
13:47 < CarolineGPA-2260> the other option is to delete the registered users
on XS1

13:49 < CarolineGPA-2260> so the log has it and incase anyone wants to ee
here is the video of how TA sharing looked:
http://www.youtube.com/watch?v=APt0J4ln28w
13:50 < daveb> i tink that is a TA artifact. I saw that here also. One had a
square and te other te turtle had not moved.
13:50 < CarolineGPA-2260> yeah I think that is a separate issue, which I
will report later.

13:50 < daveb> CarolineGPA-2260: ok right, we need to see if alot of
registered users on XS2 works with collab.
1
13:50 < daveb> i can do hyperactivity and add alot fo random users quickly

13:51 < CarolineGPA-2260> seems like the one thing we can do right now is
delete the users on XS1 and see if it works any better
13:51 < daveb> ok
1
13:51 < daveb> makes sense.
13:51 < CarolineGPA-2260> ok shall I swich the gPA back to
jabber.sugarlabs.org?
13:52 < daveb> wait until i restart it
13:52 < CarolineGPA-2260> ok

13:54 < daveb> CarolineGPA-2260: ok to switch
13:58 < CarolineGPA-2260> ok all switched
13:58 < CarolineGPA-2260> but only GPA 1 seems to be on jabber and I don't
see anyone else
13:58 < CarolineGPA-2260> are you connected?"

14:00 < daveb> CarolineGPA-2260: me too!!
14:00 < daveb> it says
14:00 < daveb> 23 people online
14:00 *** erikos is out for today
14:00 < daveb> and new registered users
14:00 < daveb> wait
14:00 < daveb> duh
14:00 < daveb> one sec
14:00 < daveb> i broke it.
14:00 -!- erikos (n=erikos at g225093143.adsl.alicedsl.de) has quit: "Leaving"
14:00 < daveb> again.
14:00 < daveb> :)
14:00 < daveb> CarolineGPA-2260: need to redo the shared roster
14:01 < CarolineGPA-2260> ok
14:01 < daveb> CarolineGPA-2260: should fix itself in a minute i hope
14:02 -!- blendmaster1024 (n=blendmas at 75-169-206-152.slkc.qwest.net) has
joined #sugar
14:05 < dfarning> sdziallas: good morning!  sorry for the late response I am
still a irc/fedora/pidgin newbie.  I need to figure out how to get the
screen blink.
14:06 < daveb> CarolineGPA-2260: restarting ejabberd
14:06 < daveb> wait!
14:06 < daveb> its working finally.
14:06 < daveb> wait restart will be better
14:06 < CarolineGPA-2260> I see Jack
14:06 < sdziallas> dfarning: hey :) (heh. not sure about pidgin, I'm using
xchat-gnome - which blinks)
14:06 < CarolineGPA-2260> are you restarting again?
14:06 < daveb> yeah you can only see people
14:07 < daveb> who were online AFTER i create the group
14:07 < daveb> restarted
14:07 < daveb> should pop up now
14:07 < daveb> there they go
14:07 < CarolineGPA-2260> yup hows the cpu?
14:08 < daveb> 129%!
14:08 < CarolineGPA-2260> nope, now all my computers are local
14:08 < daveb> i see alot of people now
14:08 < daveb> yours will have to reset telepathy
14:08 < CarolineGPA-2260> ok so I think we can eliminate the number of
registered users as the issue
14:08 < daveb> takes a few mins
14:08 < daveb> wait its down
14:08 < daveb> to 6
14:08 < CarolineGPA-2260> I'll restart everything but 1
14:09 < daveb> so it looks like its a high load to update 20 people's
neighborhood though
14:09 < daveb> i will try turtle art
14:10 < CarolineGPA-2260> ok but if its transient thats not really that bad.
14:10 < daveb> CarolineGPA-2260: correct.
14:10 < CarolineGPA-2260> lets see if we can peg it again sharing TA and
chat with a few people.
14:10 < daveb> CarolineGPA-2260: that is only on restart when everyone
updates at the same tme. it'll be the same
14:10 < daveb> when alot of people
14:10 < CarolineGPA-2260> right
14:10 < daveb> in the same calss for example
14:11 < daveb> CarolineGPA-2260: two people in turtle art
14:11 < daveb> 0%
14:12 < CarolineGPA-2260> gpa 2 joining
14:12 < daveb> i stopped it and started a new one
14:12 < CarolineGPA-2260> lets strat with chat
14:13 < CarolineGPA-2260> ok I'm with you n0w
14:13 < daveb> ok then
14:13 < CarolineGPA-2260> GPA2 joined
14:13 < CarolineGPA-2260> anything on the CPU?
14:13 < daveb> nope
14:13 < CarolineGPA-2260> ok I'll join some others
14:14 -!- blendmaster1024 (n=blendmas at 75-169-206-152.slkc.qwest.net) has
quit: "gonna JB my iphone now"
14:14 < CarolineGPA-2260> hmm only GPA2 joined jabber after restart :(
14:14 < CarolineGPA-2260> i wonder what is going on with that.
14:14 < daveb> yeah it might take a few more mins to get telepathy to reset
14:14 < CarolineGPA-2260> ack I have to leave the buidling they are locking
it
14:15 < CarolineGPA-2260> more tomorrow!
14:15 < CarolineGPA-2260> bye
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.sugarlabs.org/archive/sugar-devel/attachments/20090824/3aade30c/attachment-0001.htm