[Sugar-devel] Assessment in Karma

[Martin Langhoff]

2009/8/19 NoiseEHC <NoiseEHC at freemail.hu>:
>
>>  - Automatic assessment is snake oil, Bryan is well intentioned but
>> deeply wrong. See the earlier email at
>> http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg05584.html
>
> Or you are wrong.

I may well be wrong, but to explore that you will have to talk about
what I am stating :-)

We could rephrase it as

 - Computer-based automatic assessment / grading is only passably
accurate for a tiny *tiny* subset of relevant human skills.

 - However, it's very spectacular, and people are drawn to it... so
much that they are drawn to it even when it *clearly does not work*
for the skill being tested. It's so easy (for the teacher) and so
flashy, that people use it regardless of whether it works.

I say this after 9 years of work in the field -- I have seen
interactive SCORM objects, Moodle quizzes & lessons, HotPotatoes
activities, LAMS assessments, lots of other standalone assessment
tools. Have worked with teachers, watching their use.

What did I see? See the 2 points above.

There is a 3rd part... because these tools are cool, easy to use, they
do a lot of damage. In large part because they replace the "I don't
know how my students are doing" with "hey, I have all these scores are
numbers... nevermind they are inaccurate and only cover about 3% of
what these kids should know".

So an inaccurate view of a tiny slice of the skillset -- but hey, we
have a number representing what this kid knows! Let's use it! The link
that follows is from the Asttle project, which I was briefly involved
in several years ago:

  http://www.tki.org.nz/r/governance/consider/steps/analyse_e.php

Even worse, the Asttle project promotes the idea that you can get a
form of 'dashboard' of what kids know. Looks like an airplane
dashboard, lots of dials, full of inaccurate data _about a tiny
subset_ of what matters.

> What more interesting is that there is some research in Hungary [1] about
> the prerequisites of learning certain skills which are based on each other.

Sure, that is interesting. Now how amenable to computer-grading are
those skills? What automated computer tests can assess them with a
decent accuracy?

> ... Automatizing at
> least some of those tests are probably the biggest thing since sliced bread
> in education in my humble opinion.

Are any of those tests "automatizable"? With what accuracy? If it
turns out some can be computer assessed... _how do we keep the
non-automatizable tests in the map_? Teachers forget them
*immediately*.

cheers,



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff