The UNIX Ronin

SSL Enabled

Evan Cofsky — Mon, 21 Jun 2010 16:11:24 -0700

Since I'm a huge fan of end-to-end encryption, I've set up certificates for www.theunixronin.com. The certificate is signed by the certificate authority of The UNIX Man Consulting, and will most likely cause your browser to complain without the certificate bundle.

How Articles Are Written

Evan Cofsky — Sat, 19 Jun 2010 05:56:03 -0700

Open Drafts

I'm starting an experiment in editing content directly on the web with Plone's visual editor and versioning. Each time I save something, Plone creates a new version, and the revision history can then be used to see how things changed over time. I'd also like to present the entire writing process open to all readers, both to provide insights into how articles evolve and to solicit feedback from other people on how to improve my writing. As I explore the existing tools in Plone and Quills for this, I'll post additional information on how to observe and contribute.

The UNIX Ronin and the Environment

Evan Cofsky — Sat, 19 Jun 2010 03:56:04 -0700

I don't drive, mostly by choice, and partly by circumstance. My license expired last year, and my driving record has never been exemplary, so I opted not to renew it after my insurance carrier dropped me around the same time. A trifecta, if you will. And so I sold my car, a second Miata to replace the first lost in a war in New Mexico, and began riding my bike again.

This is not the first time I've been without a motor vehicle of my own, but it had beed several years since the last time. I wasn't in particularly great shape, having lost over 40 pounds in a year as a result of many factors, including medications ("drugs" in some readings), stress, working for a string of startups failing craptacularly at the hands of epic asshats, and the same war in which I lost the previous Miata. However, I've always been fairly strong as an endurance and utility cyclist, and within days I was commuting 20 miles from the house I was staying in Ontario to the Corona train station to get to Anaheim Hills.

Several events then transpired, and I found myself living in North Hollywood with no bike. Fortunately, the Los Angeles MTA has a subway station there, and between the subway, Amtrak, and the OCTA, I can get to Anaheim Hills in just over two hours. I recently acquired a bike, too, and this will actually decrease the commute time.

So really, I'm living the carless life right now. I don't actually intend to replace the license or the car any time soon, since I'm actually spending far less money, even after buying transit passes for three agencies each month. There's no insurance expense, no fuel expense, no taxes, and no stress from traffic. On top of that, I also get exercise consistently, walking several miles each day, and soon riding several more once I relocate close enough to cycle without taking transit.

It's a beautiful life in some ways.

US Public Transit Myths

Evan Cofsky — Mon, 07 Jun 2010 19:34:05 -0700

In Response To

This post is in response to a rant about a rant brought up by a friend.

Underwater Plumes

"Oh please. Nobody with a lick of common sense believes what you put forward as a premise. But it makes for good internet-writer's theater-of-the-absurd."

The oil well cap can collect 630,000 gallons per day, less than the well-head is spewing. Currently much of the oil is gathering in underwater plumes, and while it's convenient to state that just because the excess oil is gathering in plumes, it won't continue to gather in plumes, currently we actually know that it is gathering in plumes, and nothing in the situation has changed other than the amount of oil being released into the ocean, and it's much more reasonable to make the claim that since released oil has been gathering in underwater plumes, it will continue to gather in underwater plumes as long as the oil continues to be released.

Mass Transit

"Really? What's particularly pathological about it? You have a form of transportation that, whether you like it or not, happens to be particularly-suited to our large land-mass and relatively low population-density across that landmass. This is in distinct contrast to other nations which are the converse, and where as a consequence mass transportation makes more sense."

Actually, our population density is not much different from other industrialized nations, as long as you ignore the unpopulated areas which don't have large amounts of commuter traffic.

Ad Hominem

"Where do you live Howard? Oh, Saratoga Springs is your PO box. But this back yard doesn't look like the sort of place you claim everyone else should live in, does it?"

and also

"I did you know - for a number of years. While running MCSNet in the 1990s I lived in the Lakeview neighborhood of Chicago. I owned a car but it didn't get driven all that much - I took the subway to work (it was hella-much cheaper than parking alone!), could (and did) walk to the store, there were a few bars within a couple of blocks. Utopia? No, but functional. And in the years before MCSNet, I lived there too and took that same train. Why? Because it was easier and cheaper. Again, I owned a car - but unless I wanted to go out to the suburbs for some reason, I didn't use it much."

No Ad hominem argument is relevant, no matter how cute and pithy.

Lower Marine Riser Management Package

Evan Cofsky — Tue, 01 Jun 2010 10:03:22 -0700

In light of the Top Kill procedure failing, and with data gained from the previous capping attempt, BP now is preparing a smaller capping called the Lower Marine Riser Package (LMRP). The general synopsis is that after the previous attempt BP feels it has enough data to try a modified version of that project. The cause of the previous failure was an unexpected amount of methane clathrate crystals increasing the buoyancy of the cap. These crystals form when methane is dissolved in water at high pressures. Theoretically the LMRP will avoid this by pumping a combination of warm seawater and methanol into the cap as it is lowered and positioned. Another round of trying to plug the well with drilling mud will also help reduce the flow momentarily while the cap is positioned.

This procedure also requires cutting the damaged pipe to get a clean seal. There is concern that this will dramatically increase the rate of the leak, by approximately 20%. Should this attempt fail, there will be no method to reduce the flow to current levels.

Ultimately the only possible solution will be the completion of relief wells. A somewhat odd term for the uninitiated, these relief wells do not relieve pressure in the leaking well by removing fluid, rather they provide access to the leaking well for heavy drilling mud to be deposited to force the oil down and provide access then to seal the leaking opening.

Disappearing Reappearing Slick

Evan Cofsky — Tue, 01 Jun 2010 07:55:25 -0700

In another brilliant work by fishgrease he explains the extent of the underwater spread of oil, how dispersants affect the clean-up process, algal blooms and oil-metabolizing microörganisms, and how the society and culture of the region will be affected in years to come by the lava lamp effect.

In the comments, the futility of the knowledgable "boomers" is mentioned in contrast with the lack of interest on the part of the people leading the efforts. The rate of the leak and the abject desparation of the "Top Kill" is also presented, with a top rate of 8,000 gallons per day mentioned, and later the flow rate calculation methodology questioned. The 8,000 gallons per day figure was calculated using pressure measured below the kink in the pipe, and there is no way to correlate that with the pressure above the pipe without direct measurement, so until that measurement is made other means will have to suffice. In another sub-thread discussing that figure, the hiding effect dispersants have is presented.

Supertankers and 200,000 Gallons/Day

Evan Cofsky — Tue, 01 Jun 2010 07:21:43 -0700

From the Dylan Ratigan show on May 26, 2010, industry experts Matthew R. Simmons and Nicholas Posi discuss the rate of flow, the difficulties of getting supertankers into the gulf, ways to recover oil after its in the water, and the hubris of BP executives. They also specifically indicate that the video footage might be 5,000 gallons per day, but the amount of oil required to create the massive spill seen on the surface is much larger, implying that the actual leak is separate from the riser pipe.

Booming School II

Evan Cofsky — Tue, 01 Jun 2010 07:00:50 -0700

In Booming School II the writer presents the challenge of deploying booming to contain oil traversing a pass between the mainland and a barrier island. He also discusses the implications of building sand barriers to contain the oil using soil from a different environment. It's another interesting article with unusually insightful comments.

Booming School I Video

Evan Cofsky — Tue, 01 Jun 2010 05:50:28 -0700

On YouTube can be found a video with footage and a voiceover reading the Booming I article. It has quite a bit of video supporting the assertions from the article that the booming in the Gulf is being deployed improperly.

Booming School I

Evan Cofsky — Tue, 01 Jun 2010 05:35:22 -0700

DailyKos has an article posted by a person who seems to be knowledgable in the field of oil booming. The article itself is well done, even if a bit caustic. However, the comments on the article are also of much higher quality than usual for similar articles and sites, and I found even briefly reading them quite informative.

The basic synopsis is that while booming can be very effective at recovering oil from the surface of water, protecting the beaches and preserving the environment, it isn't being deployed in a manner consistent with these goals in the gulf. The images in the article, and in particular one of the images in the comments demonstrates this quite clearly. Another side point made is that there is a deep schism between the drilling engineers and the booming engineers, and cross-training is not normally seen as valuable in the industry.

The How Stuff Works blog has another summary of the article.

...science is one cold-hearted bitch with a 14-inch strap-on.

Evan Cofsky — Tue, 01 Jun 2010 05:15:37 -0700

First, some background. I'm a scientist, with a small S. Unlike many Scientists with a big S, my education isn't so formal. I spent several years at two universities widely regarded for their programs despite being in California, the land of the failed educational system. I took mostly courses that interested me, mainly graduate-level courses in music, physics, mathematics, philosophy, languages, computer science, and plenty of others I can't be bothered to remember. I'm one of the most inquisitive people I know by a very large margin.

Having said that, I'm completely enthralled by this massive event in the Gulf. I'm now subscribed to many web site syndication feeds, and every minute end up with 10 or 20 new articles. They range the gamut from technical through human interest to business and a host of other topics. And one thing that seems to be coming out is that the Gulf spill is not a failure of engineering, of our knowledge, of science. Like so many other failures, it's a failure of humanity itself. By refusing knowledge, turning away from reality, certainty, experience, and embracing rash, blind hope and demagogery, we opened a hole in the sea floor that we aren't capable of stopping before destroying the lives of animals, people, and quite possibly even altering the climate even more than we already have.

However, being a scientist, I know that I'm pretty much without power to change the course of events here. Yet the one thing I can do is use this small island on the Internet I've managed to carve out over the past decade with my own blood, sweat, and tears, and report on what I see.

The next several postings here will basically be summaries with links to source material. I'm gathering information much faster than I can analyse it, so I'll save that for another time when things have slowed down.

PARAVIRTUALIZED XEN 4.0 ON X86_32 LUCID 3

Evan Cofsky — Wed, 26 May 2010 19:06:35 -0700

USB Keyboard

The Mini 311 has a USB keyboard. Or at least I thought it did. In retrospect, and in checking the output of lsusb and the /sys tree, I can see now that it's not USB at all. I tried loading the ohci and usb_keyboard drivers in grub, which ultimately failed to boot altogether. Then I had the problem of recovering the system, which from the USB stick has always been a bit dicey.

USB Recovery

I found that by booting from the Lucid 10.04 USB image created with the Startup Disk Creator, then from the Advanced Options selecting Free Software Only and Expert Mode I could reliably boot, go through some extra steps, and have it avoid trying to detect the network devices altogether, which is usually where it would fail. This laptop has a Broadcom wireless chipset using the non-free wl driver which probably confuses something in the beta 10.04 image I have. I'll try booting with only the Free Softwary Only option without Expert Mode at some point now that I've narrowed it down.

GRUB2 and the Mini 311

Getting GRUB2 to show a menu turned out to be much simpler since I discovered the keyboard isn't USB at all. It the /etc/default/grub file I commented out the options GRUB_HIDDEN_TIMEOUT and GRUB_HIDDEN_TIMEOUT_QUIET, ran update-grub, and on the reboot the menu displayed and Grub recognized the keyboard and let me select entries, meaning I can now procede with my Xen experiments since I can easily select an image to boot, and I also know how to get the USB startup disk to function without crashing much more reliably.

Booting Xen 4.0.0

After building Xen 4.0.0 from source, and having to install several additional packages, I ended up with a tree of files in the dist/install directory under the source tree. I made a tarball of these, then unpacked it under the root filesystem to install it. This way I could use the manifest from the tar file to remove the files once I create a Debian package for it if I get that far.

I edited /etc/grub.d/40-custom with a menu entry to boot the new Xen, and the file now has the following contents:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry "Xen 4.0.0 / Ubuntu 10.04 kernel 2.6.32.22-xen" {
        insmod ext2
        set root='(hd0,1)'
        search --no-floppy --fs-uuid --set 87587474-bbf2-4506-a582-bf378591dc01
        multiboot /xen-4.0.0.gz
        module /vmlinuz-2.6.32.22
        module /initrd-2.6.32.22 root=/dev/mapper/vera-rootfs ro   quiet splash

My next task is to try booting it.

Paravirtualized Xen 4.0 on x86_32 Lucid 2

Evan Cofsky — Wed, 26 May 2010 14:31:08 -0700

Xen 4.0 From Source

I decided to forego the first plan, and to build Xen 4.0 from source concurrently with trying out the Xen LiveCD and both QEMU and VirtualBox. The entire purpose of this exercise is to get virtual x86 machines running on the hardware as much as possible, i.e. virtualized and not emulated, so that there is a certain minimal level of performance. I'd also like to be able to do driver development for the platform-agnostic portions of drivers, so being able to load kernel modules, run tests, and reset quickly will be important. And having a debugger wouldn't hurt, either.

Eventually, having the entire virtual environment integrated with a toolchain, target filesystem, and quite possibly some driver emulation will be icing on the cake. Imagine hitting the "Go" button in the IDE of your choice, or running "make virtual" and getting gdb attached to the virtual image for debugging, automatically updated, etc., and you can see where I'm headed. Altogether a neat idea.

I'll post some notes on building Xen 4.0.0 once I have a successful build.

Paravirtualized Xen 4.0 on x86_32 Lucid 1

Evan Cofsky — Tue, 25 May 2010 20:40:14 -0700

The Hardware

The hardware is an HP Mini 311 1000NR that I purchased last year when my larger Compaq NC6400 seemed to lose its display right before a long road trip. It has since come back to life, but only after a year of being stashed in an old cube running a development toolchain for an embedded system.

The output of /proc/cpuinfo:

evan@vera:~/Documents/Coding/ipython$ cat /proc/cpuinfo 
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 28
model name      : Intel(R) Atom(TM) CPU N270   @ 1.60GHz
stepping        : 2
cpu MHz         : 800.000
cache size      : 512 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de tsc msr pae mce cx8 apic mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 xtpr pdcm movbe lahf_lm
bogomips        : 3199.73
clflush size    : 64
cache_alignment : 64
address sizes   : 32 bits physical, 32 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 28
model name      : Intel(R) Atom(TM) CPU N270   @ 1.60GHz
stepping        : 2
cpu MHz         : 1600.000
cache size      : 512 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 1
initial apicid  : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 xtpr pdcm movbe lahf_lm
bogomips        : 3199.90
clflush size    : 64
cache_alignment : 64
address sizes   : 32 bits physical, 32 bits virtual
power management:

It's not really a powerhouse compared to the x86_64 Core 2 in the 6400, but it's a laptop, and in the past I've run Xen on much lesser hardware with some use. Having the ability to run it on this laptop would definitely be useful, especially since I seem to be getting an increase in demand for developing board support packages. Ironically one would never know that I'm a bit of a Python Metaprogrammer by the amount of C macro hacking I've been doing the past year or so.

Hardware Flaws

First of all, it's pure 32-bit Intel. The Atom N series is hyperthreaded, but only a single core. The chipset only handles 3GB of RAM, and the first 1GB is hardwired on the motherboard. The nVidia graphics card is very nice, though, and I've become incredibly fond of having full screen effects, including compositing and other niceness from KDE 4. I'm running Kubuntu on it, which while not given as lavish attention as some of the other Ubuntu versions is still extremely friendly.

Software suspend works a couple of times before the disk controller seems to give up, the filesystem becomes read-only, and a reboot is required. The video camera doesn't survive waking up, and occasionally the HD audio driver for the nVidia chipset freezes. But it's still a decent laptop all around. It's the right size for me, has a sizable hard drive, and the graphics are well-supported. And after spending a decade in front of 80x24, it doesn't take much to please me with graphics.

Xen 3.3

At the moment I'm going to reboot it and try getting Xen 3.3 running with the packages from the Lucid repository. I've made a boot USB image for recovery since Grub doesn't seem to handle the USB keyboard. Next up I'll report back, and hopefully move on to booting from a live Xen 4.0 image.

Bazaar and SVN

Evan Cofsky — Tue, 04 May 2010 17:57:52 -0700

I'm not stable, so much as meta-stable. I just ended a three-year stint with "stability", and it ended. Badly. So I'm consulting again. One of the things I really don't like doing is sitting in an office. The fluorescent light aggravates the autism, and the ADHD, well, ADHD isn't looked upon well by "adults".

One of my larger clients has me working on a software project. It's kind of neat, since it's building and training an organization on how to develop on Linux, and how to handle modern software processes well. It also conveniently fits in well with my penchant for 5150s and sick days.

And of course they use Subversion. I'm used to this though, it's the first mistake every organization makes when choosing a new VCS for their new software model. I come prepared with then Bazaar Subversion Plugin. It's actually decent when it works. However, usually I've managed to not get it working very well. But I trudge on.

And then I found Lorna Jane. She seemed to spend the time actually figuring this thing out, something I never really tried. I'm an ass.

So kudos to Lorna. And maybe my next batch of commits will go much smoother.