Posts

Posted on

XML for documents, not for large data streams

I like XML, and I hate XML. XML is great because robust parsers already exist for nearly every programming language, thus saving work for programmers and reducing bugs. XML stinks because it’s not always the right tool for the job — it’s ugly, and it’s bulky. So when I read Michael E. Driscoll’s [comparison of documents (including XML) to trees and data to streams](http://dataspora.com/blog/the-rise-of-the-data-web/), it struck a chord with me:

> Trees are rooted and finite: you can’t chop up a tree and easily put it back together again. Streams can be split, sampled, and filtered. The divisibility of data streams lends itself to parallelism in a way that document trees do not. The stream paradigm conceives of data as extending infinitely forward in time. The Twitter data stream has no end: it ought have no end tag. Conceiving of data as streams moves us out of the realm of static objects and into the realm of signal processing.

He also [explains why XML shouldn’t be used for large data streams](http://dataspora.com/blog/xml-and-big-data/):

> XML is a poor language for data because it solves the wrong problems — those of documents — while leaving many of data’s unique issues unaddressed. But many promising alternatives exist — microformats like JSON, Thrift, and even SQLite’s file format.

I wouldn’t have thought of using SQLite’s file format — it has become somewhat ubiquitous. I admire Google ProtocolBuffers and Apache Thrift for offering open source, multi-language binary encoding for data. Now programmers won’t be as likely to reinvent the wheel, and they can rely on robust libraries.

Posted on

Vim multi-line search-and-replace for wordpress comments

When I switched web hosting providers, I migrated my wordpress instance by exporting to wordpress XML format (as opposed to doing a SQL export).

I didn’t want the SPAM comments to be imported into the new wordpress instance, so I used vim multi-line search and replace to delete the unwanted comments from the XML.

:%s#\_.\{-}<.wp:comment>##

I gleaned that syntax from from [http://osdir.com/ml/editors.vim/2002-06/msg00468.html](http://osdir.com/ml/editors.vim/2002-06/msg00468.html)

Posted on

Palm T|X Security: Counterproductive

The other day, I was looking through the preferences on my Palm T|X, and I found out that I could enable “Intrusion Protection”. I set it so that it would destroy all data on the TX if I failed to enter my password 25 times. That seemed like enough grace period that I wouldn’t accidentally destroy my data, even if I mis-typed the password several times.

The next day, I let my three-year-old play “Bombel”, and draw on the “Note Pad”. Several minutes later, I noticed that she was pushing buttons willy-nilly at the password screen.

“Oh!”, I thought, “That’s not good.” She was well on her way to exceeding the 25-password attempts and wiping out my data. I knew I could get it back with a hot-sync, but I didn’t want to resort to that.

Palm “intrusion detection” became counterproductive when placed in the hands of a child.

—-

I also tried the Palm TX feature to “Encrypt data when locked”. First, I tried using [AES](http://en.wikipedia.org/wiki/Advanced_Encryption_Standard) encryption, since it would likely be “stronger” than the default of [RC4](http://en.wikipedia.org/wiki/RC4). AES was unusable — it took minutes to encrypt and decrypt my calendar and address databases. RC4 was barely usable, taking ten seconds or so to encrypt and decrypt my calendar. When I whip out my Palm, I want access to my data immediately, so I disabled encryption.

—-

I’ve chosen convenience over confidentiality for the data on my Palm TX, because I felt that the price to pay for confidentiality was too high. I’m not sure that it’s the right decision. I might feel differently if the Palm is lost or stolen. And so might some of the contacts in the address book. I would re-evaluate my decision if I were required to notify those contacts in the case of a lost Palm.

Posted on

Fedora 11 and Virtualization (KVM)

I’ve recently upgraded another computer from Fedora 9 to Fedora 11, and I’ve decided to try the built-in [KVM](http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine) (i.e. Applications -> System Tools -> [Virtual Machine Manager](http://virt-manager.et.redhat.com/)). I wanted a virtual machine that had bridged mode networking, but it wasn’t available by default. To get it as an option, I disabled SELinux (not sure if it was necessary), followed [some special instructions](http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging) to setup a bridged interface, and restarted my network and libvirtd.

Now I’ve got a working guest OS inside of KVM, and I like it. The guest OS feels snappy and responsive.

Update: KVM and the accompanying tools aren’t as mature as VirtualBox or VMWare. E.g. I didn’t see how to get my USB flash drive to be recognized by a KVM guest OS. At one point, I tried to use VirtualBox at the same time as KVM. VirtualBox told me I needed to disable the KVM kernel module before using VirtualBox.

Posted on

Switched from digitalspace to justhost

I’ve been running my website on digitalspace.net hosting for years. Then they sold out to jumpline, and my ability to push changes to my website via ‘[rsync](http://www.samba.org/rsync/)’ disappeared, and was never restored. Although I still had ssh shell access, the account was seriously limited. It was probably a good security decision on their part, but I missed having wget, tar, gunzip, chmod, and other essential utilities that I used when upgrading my blogging software. It became tedious, at best, to maintain my website.

I’ve finally switched to hosting through http://www.justhost.com and the transition has taken more time than I wanted. As a father of four dear children, I feel the time pinch. Migrating wordpress has been more tedious than expected. And then there’s email — that was a pain to switch as well. At one point, I even considered abandoning my website and switching my blog to a site like blogger.com. But I stuck with it.

Jumpline support has been good to work with, and I’m pleased with my ssh shell access. I get the power of a typical linux shell with my favorite utilities: rsync, tar, etc.

Posted on

My impressions of Fedora 11

Here’s my take on installing Fedora 11, which was released June 9, 2009. I chose not to do an upgrade as I often do. Instead, I did backup, followed by a fresh install, preserving my /home partition, but wiping out the other partitions. Then I used [`meld`](http://meld.sourceforge.net/) to restore my configuration files in /etc — such as ssh server keys, printer settings and file system mounts. I found that I had to use the kernel boot option `nomodeset` in order to avoid system lockups. Overall, I’ve been pleased with my Fedora 11 experience, despite the bumps.

Fedora 11 useful resources:

– [Release Notes](http://docs.fedoraproject.org/release-notes/f11/en-US/)
– [Common Bugs](http://fedoraproject.org/wiki/Common_F11_bugs), with workarounds.
– [Fedora Guide](http://www.fedoraguide.info/), explaining how to configure a Fedora system.

Pre-install:

– `cp -a /etc /home/backup/etc`
– `cp -a /root /home/backup/etc`
– backup /home
– booted the LiveCD to make sure it would detect my hardware and run

Install

– I decided to preserve my partition layout, which isn’t the default option upon fresh install
– Didn’t delete my `/home` partition.
– Reformatted all other partitions, with “/” as ext4

Post-install:

– Had to enable eth0 in NetworkManager, and make “enabled” the default.
– `yum install -y meld nautilus-actions nautilus-open-terminal vim-X11 zsh screen mc rdesktop`
– `meld /home/backup/etc /etc`
– Restored /etc/ssh settings
– Restored /etc/cups printer settings
– Checked /etc/fstab differences
– Installed [NX Server](http://www.nomachine.com/)

Pleasurable:

– Bootup is very pleasant, and seems faster. 30 seconds boot. 17 seconds login. 14 second shutdown. This is on an AMD Athlon 2400 Mhz Sempron with an ATI video card.
– Artwork is top notch (backgrounds on login screen and default wallpaper)

Pain points:

– Unavailable extensions for Thunderbird 3.0
— Enigmail
– Unavailable extensions for Firefox 3.5
— Aardvark
— QuickProxy
– Computer locked up every few hours until I added `nomodeset` to my kernel settings in /etc/grub.conf.

Posted on

Gnome Slideshow Screensaver Sanity, Take 2

Last year, I wrote about how to achieve [Gnome Slideshow Screensaver Sanity](http://jaredrobinson.com/blog/?p=106). I’ve recently upgraded to Fedora 11, and I noticed that GLSlideshow isn’t installed by default (maybe it never was), and I wondered if I could alter the settings for gnome slideshow. By default, it uses pictures out of the $HOME/Pictures folder, and there’s no way in the user interface to change that location, which can be frustrating. Here’s how I worked around it. Note the use of the `–location` option, and that I changed my `Name=` setting.

– `cp /usr/share/applications/screensavers/personal-slideshow.desktop ~/.local/share/applications/my-slideshow.desktop`
– `gedit ~/.local/share/applications/my-slideshow.desktop`


[Desktop Entry]
Encoding=UTF-8
Name=Custom Photos
Exec=/usr/libexec/gnome-screensaver/slideshow --location=/home/images/Photos
TryExec=/usr/libexec/gnome-screensaver/slideshow
StartupNotify=false
Terminal=false
Type=Application
Categories=GNOME;Screensaver;
OnlyShowIn=GNOME;

Go into the screensaver preferences (System -> Preferences -> Screensaver), and select “Custom Photos”. There’s no way to customize the duration to display each photo, but at least I don’t have to settle for Gnome’s default location.

Posted on

Backup that laptop!

Recently, a relative called and said her laptop wouldn’t boot. She wondered whether I could help. I asked if she had a backup. “No” was her answer.

I booted into linux (using [Knoppix](http://www.knoppix.net/) from a bootable CD) and attempted to back up her Windows account to an external USB hard drive. As I worked with the laptop, I discovered it was shutting itself off. On my second attempt, I managed to make a successful backup.

I tried running the system restore, but it would fail at random intervals. Next, I booted into the [System Rescue CD](http://www.sysresccd.org/) and ran the memory test. It shut at random intervals during each memory test. I figured it couldn’t be the hard drive that was at fault, but that the hard drive had probably gotten corrupted from the computer powering off suddenly.

My relative took her laptop to Geek Squad to see if they could diagnose the problem. They ran the system restore, and it succeeded. They didn’t do any further troubleshooting. They charged her $50.00 without solving the root problem. It continued to shut off at random intervals.

A colleague of mine looked at the computer, and found that the heat sink on the CPU was clogged with dust. Most likely, the CPU was getting too hot and powering off. He removed the dust buildup, and from then on, the laptop seemed to work well. I was able to restore the files, and my relative was much happier.

And she bought an external USB hard drive to do future backups. Good thinking.

I use [Mozy](http://mozy.com/) for automated, regular backups of my most important files. It’s not a complete solution for my whole hard drive, but it’s far better than nothing, and it only costs $5.00 a month. For linux, I need a similar solution. It turns out that there is one: [spideroak](https://spideroak.com). It runs on Windows, Mac and Linux. And it can synchronize files between several computers.

Posted on

Wind farms — hot air?

The media has a fascination with “alternative” and “renewable” energy, such as wind power. What if wind power is a bunch of hot air?

> …if we wanted to go totally green, we’d have to carpet the [United Kingdom] with more windmills than exist in the whole world.

Undoubtedly, we need more power plants in the future. Utilizing windmills here and there sounds okay, but I don’t want to see an entire countryside covered with them. I believe that nuclear power will be the most cost effective form of mass power, the cleanest and [the safest](http://www.google.com/search?q=nuclear+power+safest) in terms of the cost of human life — compared to coal and hydro electric.

Posted on

Every day physics

This past weekend, our family visited [Hill Aerospace Museum](https://www.hill.af.mil/library/museum). While I was fascinated by the jets and both awed and disturbed by the cluster bombs and nukes, my children were most interested in the educational area where the flight simulators were located. They had building blocks, toy jets, flight suits, and hands-on experiments to demonstrate things like kinetic energy by smashing two [steel balls](http://www.grand-illusions.com/acatalog/Large_Ball_Bearings.html) together with a sheet of paper sandwiched in between to burn holes in it, [resonance](http://www.intuitor.com/resonance/index.php) demonstrated with a [notched stick propellor](http://home.frognet.net/~ejcov/geehaw.html), and the structural strength of honeycomb, which is used in the building of aircraft.

I realized that I’m a novice on every day physics. I’m sure I studied resonance in my electronics and physics classes, but I don’t remember it.

As for aircraft, I found it fascinating that faster is not always better — the [A-10 Thunderbolt II](http://en.wikipedia.org/wiki/A-10_Thunderbolt_II) was built to fly slowly so it can support ground troops effectively, and it can still fly with half a wing and half a tail missing. Technology from 1950 is still in use today in the form of the [B-52 Stratofortress](http://en.wikipedia.org/wiki/B-52_Stratofortress), and will be for the next 30 years or more. So newer isn’t necessarily better. I looked up the [B-2 stealth bomber](http://en.wikipedia.org/wiki/B-2_Spirit) and found out that it’s staggeringly expensive to build and operate. Coworkers mentioned the [F-22 Raptor](http://en.wikipedia.org/wiki/F-22_Raptor), and I was amazed to learn of its high bandwidth communication abilities. Apparently, many military pilots eat breakfast in the morning with their families, and arrive home in time for dinner because they fly remote controlled [drone planes](http://en.wikipedia.org/wiki/Unmanned_aerial_vehicle). When the pilot is removed from the cockpit, the plane can do things that would be impossible otherwise. It’s amazing how much technology, expertise and money goes into building jets.