Failed escape

I've been using Ubuntu since the spring of 2005, that is for over 8 years. I appreciate the Debian underpinnings, predictable releases, general fit and finish. However, now I'm using Gnome Shell instead of Unity and I feel that there's not much value provided to me in over plain Debian Testing. Moreover, I'm unconvinced in the general direction of Ubuntu is taking for the past couple of years, the "Not Invented Here" mentality of developing competitors to other Open Source projects (Unity, indicatorors, Mir). So I decided to switch.

I downloaded a Debian Testing (wheezy) netboot 23 MB mini.iso image, set it up to boot with GRUB from a USB flash drive, made a copy of my system partition, and rebooted. Long story short, the installer failed on the base system install stage, something about unable to authenticate the busybox package. I've spent a couple of hours fumbling around trying to get past the error, downloaded a different version of the wheezy netboot image, to no avail. I also tried the 200 MB mini-CD image, but that failed with "Could not mount the CD-ROM" being booted off a flash drive.

After a while I just restored the backup, updated the new root FS UUID in /etc/fstab and /boot/grub/grub.cfg and went on with life. On Ubuntu. I'm still on Precise, probably worth trying an upgrade to Raring. I'll keep the backup of the Precise system partition, though.

Ubuntu Oneiric FTW :(

I was used to having uptimes in the order of weeks on the laptop. A couple of suspends, resumes every day, hooking up and disconnecting from the external monitor. Everything worked without a hitch on Ubuntu during the last couple of years. I was being conservative and used Gnome 2 without compositing most of that time. But Ubuntu 11.10 Oneiric is different. Gnome 3 used to crash 5 times a day, Unity has a ~50% chance of surviving a day or two. Most often things seem to crash after the screensaver has kicked in. Yay Ubuntu. Might be time to look elsewhere. Linux Mint or good ole Debian.

Updating Samsung Galaxy S SCL GT-I9003 to Gingerbread in Ubuntu

So, I came across the news on the net that Android Gingerbread 2.3.4 update has been made available by Samsung to GT-I9003 phones. Apparently, it was being rolled out on a country or region basis via Samsung's Kies application. First I tried updating the phone with Kies in Virtualbox, but that didn't quite work, I could not get Kies to connect to the device. (Might be the reason was that the phone was not in Kies USB mode). Then I tried booting the OEM Windows Vista on the laptop. Vista had worked after I copied the partition from another hard drive, but now it refused to boot. Apparently, system could come up from hibernation, but depended on the hard drive unique ID in order to boot. Some web browsing, several reboots and trials later, I repaired the Vista boot configuration with the help of a warez Vista DVD. The laptop's recovery partition is useless -- basically it offers restoring from an external backup made earlier or a "factory reset". No Windows recovery, no command prompt.

So, turns out Samsung Kies sucks on real Windows as much as it does on Virtualbox. Granted, it did connect to my phone, but I could get more out of it than a backup of the 7 contacts in my phone's memory (the rest are synch'ed with Google). But backing up the 300 photos I have made Kies show a progress bar for 15 mins and then choke. And worst of all, there was no update of any kind offered! I wasted about 10 minutes staring at progress bars waiting for it to install, and then some 30 more minutes staring at progress bars waiting for the backups to fail.

So then I looked again at flashing the phone myself. I got the firmware from samfirmware.com, found the instructions for flashing with the leaked proprietary Samsung Windows flasher called Odin, got the Open Source flasher Heimdall (Debian packages worked fine), found an encouraging blog post on how to use it.

The firmware from Samfirmware is a RAR archive (password: samfirmware.com) containing a tar:

-rwxrwx--- 1 alga alga 191658005 2011-09-19 01:23 I9003XXKPH_I9003XXKPH_I9003SERKPH.rar
-rw-rw-r-- 1 alga alga 353699840 2011-08-10 09:53 I9003XXKPH_I9003XXKPH_I9003SERKPH_HOME.tar

Inside the tar, there are the firmware images:

$ tar xvvf I9003XXKPH_I9003XXKPH_I9003SERKPH_HOME.tar
-rwxr--r-- jonghoon/na-cdma-sw 262144 2011-08-09 15:19 boot.bin
-rwxr--r-- jonghoon/na-cdma-sw 1310720 2011-08-09 15:19 Sbl.bin
-rwxr--r-- jonghoon/na-cdma-sw 618496 2011-08-09 15:19 param.lfs
-rwxr--r-- jonghoon/na-cdma-sw 5787648 2011-08-09 15:19 normalboot.img
-rwxr--r-- jonghoon/na-cdma-sw 324751360 2011-08-09 15:19 system.rfs
-rwxr--r-- jonghoon/na-cdma-sw 12582912 2011-08-08 06:23 modem.bin
-rwxr--r-- jonghoon/na-cdma-sw 8378368 2011-08-09 15:20 cache.rfs

Apart from these, you need a partition table image (*.pit, latona_20110114.pit in my case), that can be downloaded from the Samfirmware site, too.

The phone needs to be placed in download mode. Everyone on the internet says this can be done by holding down Volume Down, Home, and Power buttons at the same time, but this did not work for me. An alternative is to use adb:

$ adb reset download

Then the flash command is as follows:

$ heimdall detect
Device detected
$ heimdall flash --pit latona_20110114.pit --primary-boot boot.bin --secondary-boot Sbl.bin --normal-boot normalboot.img --cache cache.rfs --modem modem.bin --param param.lfs --system system.rfs

About 2 tense minutes of progress counters uploading these files, a long reboot, and presto! The phone boots up into Android 2.3.4 Gingerbread!

The heimdall flash command can be run with the --repartition flag or without it. Repartitioning is needed if the partition layout is different, or, as one forum post suggests, a filesystem type changed. However, it seems that the partition layout is the same for all GT-I9003 versions (Samfirmware suggests the same pit file), so just flashing the partition images is OK.

Looking at other firmware archives, it looks like the number of images and their names vary. You just need to match up the files you have with the heimdall option flags (in other words, partition names of your phone). The heimdall print-pit can be useful here, it prints out the actual partition layout of the device.

The nice thing about flashing Android updates is that none of the user data was overwritten -- the system lives on separate partitions from installed programs and their data. Although, of course, there is a risk that something goes wrong, so it's wise to back up data before flashing.

Now, speaking of Gingerbread, it seems that most of the goodies added in it Samsung had already forward-ported in their Froyo firmware. I hope that they fixed the stability issues. My phone used to crash often with a black screen on Froyo when sitting idle.

Freeing up some space on Ubuntu

I tried upgrading to Ubuntu Maverick Meerkat, but, as it often happens, the upgrade manager brushed me off saying I need to clear up about 970 MB of disk space on the root filesystem.
Time to buy a new hard drive, but in the meantime I still want to upgrade with what I have.

apt-get autoremove freed up some space, but more than an order of magnitude less than the gig I needed to find.

deborphan has always been a great tool to seek out the unneeded packages that use up disk space. If you run it without any arguments, it reports the libraries that are not required by any application. So you can safely remove every package deborphan reports unless you know you need it (for your own development, proprietary software you installed manually, etc.). When deborphan reports nothing, it's time to lax up the requirements:

   deborphan -anz | sort -n

This lists all packages that are not required by other packages, even if they aren't libraries, and sorts them in a descending order by size. Old kernel images show up in this list near the bottom. There's usually more low-hanging fruit at the end of the list that can help you free up several hundred megs of space. Here you need to be careful not to delete something that you need from the system, as with these flags deborphan lists even packages that are apps rather than libraries, and even includes those that are suggested or recommended by other packages.

We're still several hundred megs short, but there are no obvious large packages to remove. The space is taken up by hundreds of relatively small ones. An effective technique I have found for this situation is start up Synaptic, order packages by origin, then go through restricted, multiverse and universe, sort packages by install status and size, and remove all the small packages you know nothing about, and various utilities you installed and used once. Sometimes it will threaten to remove a program that you need, then you need to back out. After you apply the changes, it's a good idea to close Synaptic and let apt-get autoremove and deborphan have a go again. If you're still not there, start up Synaptic again and prune Universe some more.

Presto! I got my gig (and more!) of free space by removing old kernels (480 MB), sacrificing Flightgear, QCad parts library, and lots of small utilities from Universe (900 MB).

upicasa -- an upload script for PicasaWeb

So, as planned last time, I developed my PicasaWeb script a bit further. After I got home from a niece's first birthday party, I had some photos I wanted to upload, so I added the support for listing and creating albums. Then today I moved the authentication credentials out of module globals into a combination of a config file and interactive email/password query. With my password out of the source code it became ready for the initial import into a version control repository.

I chose Mercurial as a VCS just to try it out. Thus, Bitbucket was a natural place to host the repository. Unfortunately, yesterday evening it was inaccessible. Mercurial feels really fast and convenient.

Bitbucket was a total blast to get going. I registered in with my Launchpad OpenId, created a new repository for my project, uploaded my SSH pubkey, and was able to push my initial version within two or three minutes. Way to go, dudes! Very clear, simple, and good looking. A stark contrast with Launchpad, which leaves me confused whenever I try to do something that should be really simple. Here's the link to the upicasa hg repo on Bitbucket.

So, anyway, I wrote a setup.py script, a README file, polished the script a bit and released it to PyPI: upicasa 0.1. Writing the setup.py and the README in order to package the script took about the same time, if not more, than writing the script in the first place, but that's how it goes with Open Source, doesn't it?

The more I use PicasaWeb the more I like it. It has a nifty semi-automatic face recognition feature. It makes tagging people on photos pass the threshold of convenience where it becomes practial. Finally I'll be able to find a photo of someone I know I have. Full screen album display feature is also very nice.

So, now I have a warm fuzzy feeling of having done a useful weekend programming project. Tried something new. Learned a bit. Now it will be great to see whether other people will find it useful.