<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#ffffff" text="#000000"> Bernie; Are you familiar with Opensuse-edu using cliced-hybrid file systems? They allow a dd write of .iso to USB and look for a second persistent partition created by fdisk (or a script) The compressed file system on the .iso is converted to a real file system on first boot. (see cyberorg  on #sugar or #opensuse-edu for details) It works very well I have been using livinst and direct installs of sugar distributions to USB sticks for quite a while with NO PROBLEMS of corruption see: <a class="moz-txt-link-freetext" href="http://wiki.sugarlabs.org/go/Talk:Downloads#A_review_of_installation_methods">http://wiki.sugarlabs.org/go/Talk:Downloads#A_review_of_installation_methods</a> <a class="moz-txt-link-freetext" href="http://wiki.sugarlabs.org/go/Category:Liveusb">http://wiki.sugarlabs.org/go/Category:Liveusb</a> Plus <a class="moz-txt-link-freetext" href="http://en.opensuse.org/Live_USB_stick">http://en.opensuse.org/Live_USB_stick</a> <a class="moz-txt-link-freetext" href="http://wiki.sugarlabs.org/go/VMware#openSUSE">http://wiki.sugarlabs.org/go/VMware#openSUSE</a> - NOTE: I think the major problem with sqfs USB sticks seems to be delayed writes not getting done before pulling them out. This borks the USB! I have to wait 5-20 sec after a PC says it is safe to remove them, waiting for the usb led activity to stop. (If the usb has a full install file system and this occurs, it can correct itself because of the journalling fs.) Tom Gilliard satellit see the wiki page :<a class="moz-txt-link-freetext" href="http://wiki.sugarlabs.org/go/Category:Liveusb">http://wiki.sugarlabs.org/go/Category:Liveusb</a> Plus <a class="moz-txt-link-freetext" href="http://en.opensuse.org/Live_USB_stick">http://en.opensuse.org/Live_USB_stick</a> <a class="moz-txt-link-freetext" href="http://wiki.sugarlabs.org/go/VMware#openSUSE">http://wiki.sugarlabs.org/go/VMware#openSUSE</a> Bernie Innocenti wrote: <blockquote cite="mid:1264514768.10198.75.camel@giskard" type="cite"> <pre wrap="">On Mon, 2010-01-25 at 20:21 +0100, Sebastian Dziallas wrote: </pre> <blockquote type="cite"> <blockquote type="cite"> <pre wrap="">Have you tried writing to the journal until you fill up the overlay space </pre> </blockquote> <pre wrap="">On a reasonably big usb stick (with prices dropping all day long), this problem will probably not be solved, but rather appear less and less. </pre> </blockquote> <pre wrap=""> On larger USB sticks it will take *longer* to appear, but it *will* certainly appear after some time. As the journal fills up with activity data, soon or later the overlay snapshot will run out of free pages. </pre> <blockquote type="cite"> <pre wrap="">Note that I'm not saying that we don't need to investigate here. </pre> <blockquote type="cite"> <pre wrap="">Unless I'm seriously misunderstanding how LVM snapshots work, this should systematically make the flash drive inoperable until reformatted, and all data inaccessible. </pre> </blockquote> <pre wrap="">Walter, Dave and others have been doing tests concerning the reliability of our current layout in the post-Strawberry time, if I recall correctly. It turned out that the compressed read-only squashfs was not at all the root of the issues some people were seeing. </pre> </blockquote> <pre wrap=""> Yes, the problem is definitely not squashfs itself, because all it does is store a read/only ext3 image. </pre> <blockquote type="cite"> <pre wrap="">Rather, those were directly related to a corrupted overlay file, whether it was caused by unplugging the usb stick too early or filling up the overlay file quickly. Resetting the overlay file took away all these issues, though. </pre> </blockquote> <pre wrap=""> Unplugging the USB stick shouldn't be causing any major filesystem corruption when using ext3 directly. It may be problematic with the overlay, though, as it cannot guarantee the write ordering semantics expected by the filesystem. The corruption scenario I've seen with Caroline is much simpler to reproduce: 1. write normally to the filesystem (dd if=/dev/random of=foobar) 2. boom! </pre> <blockquote type="cite"> <pre wrap="">As much as it might be wrong: Upstream has a reason in doing so. And we're going to stick with what upstream is doing. I used to explain a major goal for this release to be increasing the sustainability of the whole process. Diverging from upstream is not going to achieve this. </pre> </blockquote> <pre wrap=""> I tried to make sense of the reasons behind the current layout by asking around. My impression is that it evolved in incremental steps over the years, without rethinking the design for USB storage at all. 1) a read-only squashfs of course made sense for the live CD 2) storing an ext3 image inside squashfs provided a bug-free filesystem and probably better compression 3) transferring this ext3-within-squashfs to a DOS filesystem was the most obvious way to create a bootable USB stick 4) using device-mapper snapshots was a clever way to make the embedded ext3 filesystem also writable See? Every step makes perfect sense if you consider the previous situation. But... wait a moment... all these steps, 1 through 4, are totally superfluous when you have a USB stick!!! </pre> <blockquote type="cite"> <pre wrap="">OLPC went through this. And so did I. The final Blueberry build was created late in the night before I was having another school exam in the morning and directly afterwards rushing to the airport to catch a plane to Toronto. </pre> </blockquote> <pre wrap=""> You did an amazing job, this is not being questioned. </pre> <blockquote type="cite"> <pre wrap="">Anything that diverges from upstream (let it be Fedora, Sugar, or any other project) leaves us with a gap. Everything we hack up ourselves will return to us when it comes to support. </pre> </blockquote> <pre wrap=""> I'm not proposing to hack a new thing. I'm proposing to _remove_ a hack. All we have to do, is install Fedora to the USB stick normally, as it would be installed to a hard drive. We can do this using the standard tools for installing fedora (liveinst, or whatever). At the end of composition, we just drop the additional steps performed by livecd-iso-to-disk. </pre> <blockquote type="cite"> <pre wrap="">And we don't want our users to download either a 4 GB image (or a 700 MB one, which takes them half a day to uncompress), right? </pre> </blockquote> <pre wrap=""> If the image is padded with zeros and compressed with xz (lzma), it may become even smaller than the current iso image. Wanna bet? :-) </pre> <blockquote type="cite"> <blockquote type="cite"> <pre wrap="">The fix for us is easier, because we don't really need any of the fancy things that livecd-iso-to-disk does. All we need to do is stop mentioning it in our wiki and switch to liveinst, as Peter Robinson suggests, or any other tool that does the (rather trivial) job of transferring an ext3 image onto a (flash) drive. </pre> </blockquote> <pre wrap="">I don't think there's an "us" and "them" there. People from Fedora contribute to Sugar on a Stick, in the same way as people from Sugar Labs do. It's somehow what makes this project so cool, too. </pre> </blockquote> <pre wrap=""> I don't want to give the wrong impression that I'm anti-Fedora. I've been using Fedora on multiple machines since when it was still called Red Hat :-) Nevertheless, the Fedora Live CD has a different audience and use-cases than Sugar on a Stick. We don't necessarily have to blindly imitate them (regular Fedora Live) if it doesn't work well for us (SoaS). The Fedora Live CD is mostly for showcasing Fedora without installing it, while SoaS is supposed to be a production environment. </pre> <blockquote type="cite"> <pre wrap="">So. I was talking to Sascha yesterday on IRC and we came up with some ideas how to proceed here. I don't think that the we should stop using squashfs images. Because I don't see a reason to stop doing so. </pre> </blockquote> <pre wrap=""> Let me reverse the viewpoint: do you see a reason to *continue* using squashfs? </pre> <blockquote type="cite"> <pre wrap="">On the overlay, an approach that we came up with introduced a new partition (possibly ext4) for the overlay only. </pre> </blockquote> <pre wrap=""> Where would this new partition be mounted? </pre> <blockquote type="cite"> <pre wrap="">That would mean that the bootloader and the squashfs file would stay on a fat32 partition, allowing users even to exchange data with their windows machines. </pre> </blockquote> <pre wrap=""> This is currently not possible anyway because the journal is stored in the ext3 partition, and windows does not have tools to read the journal anyway. Even if one could import/export files to the DOS partition, I don't think we should (mis)design our partition layout all around this usecase. </pre> <blockquote type="cite"> <pre wrap=""> On the other hand, we'd reference the new overlay partition (instead of the overlay file) in the syslinux.cfg. I don't know whether this is possible. But now is the time to work it out. It would allow us to continue to use the existing infrastructure, while working on the roots of relevant issues directly. </pre> </blockquote> <pre wrap=""> Even if it were possible (but I do not understand how), this would indeed require some additional coding and testing to get there from where we're standing. What useful features does squashfs bring to pay off for all the extra work and complexity it requires? /me is a minimalist. What's not there will not break ;-) </pre> </blockquote> </body> </html>