[Sugar-devel] [PATCH sugar-0.84] journal scan of external media

[Sascha Silbe]

Excerpts from James Cameron's message of Tue Sep 07 05:03:40 +0200 2010:

> The code may as well have just recursed right where it was; with this
> number of idle tasks there's very little point in using idle tasks.

Agreed. I wonder if we could somehow wrap os.walk() in a single,
low-priority idle task. Alternatively call find in a subprocess and use
gobject.io_add_watch(). Especially if we want to continue indexing
symlinked directories this would be a good approach (see below).


[Attached patches]

Please use git-send-email to send the patches in the mail body or set
Content-Disposition to inline (Ctrl+D in mutt) to allow easier replying
to the patches in other mail clients.

> -        for entry in os.listdir(dir_path):
> +        try:
> +            entries = os.listdir(dir_path)
> +        except:
> +            entries = []
> +        for entry in entries:

This has been fixed by alsroot in commit 11af549c ("Progress bar for
reading volumes nevers stops if can't read all volume entries #1412")
already, including logging the error to provide at least some (though
rather hidden) indication why the Journal doesn't everything on the
device. That commit looks more invasive than your patch, though.


> +                    link = os.readlink(full_path)
> +                    if link == '.':
> +                        continue
> +                    if link.startswith('/') and full_path.startswith(link):
> +                        continue
> +                    stat = os.stat(full_path)

This will still break for relative links. 

The easiest solution is to do the same most other tools do by default:
Ignore symlinked directories. The disadvantage is that the search
pattern won't match paths containing symlinked directories, but only
the real path. It could be argued that this is a rare use case and the
user should use some low-level tool instead in that case since symlinks
are low-level stuff as well.

The alternative is to do the same as "find -L" does (check the dir_ids /
dir_curr handling in find/find.c, especially in process_path()). It
keeps a list of directories (identified by device id and inode number)
that have been visited before in the current branch of the directory
tree (i.e. while traversing down to the current path). If it encounters
a directory twice, there has been a loop and it won't descend again
(but print a warning).

Sascha

--
http://sascha.silbe.org/
http://www.infra-silbe.de/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
Url : http://lists.sugarlabs.org/archive/sugar-devel/attachments/20100907/27ccd6e8/attachment-0001.pgp