Ubuntu and .local hostnames in a corporate network

In the past, I’ve had trouble getting my Ubuntu machine to resolve the .local hostnames at work. I didn’t know why Ubuntu had this problem while other machines did not.

When I did a DNS lookup, it failed, and ping of host.something.local failed. Yet ping of the hostname without the .something.local extension worked. Odd. I googled various terms, but nothing useful came up. I tried watching the DNS lookup with tcpdump, but it didn’t capture anything.

Eventually, I thought of using ‘strace ping host.something.local’ to see what was happening, and it turns out that DNS was never being queried — it was talking to something called avahi.

I googled “avahi”, and was reminded that hostname resolution is configured in /etc/nsswitch.conf. In the case of Ubuntu, it’s configured to send *.local requests to Avahi (mdns4_minimal), and no further — i.e. if Avahi doesn’t resolve it, it doesn’t try DNS.

In my case, I want corporate DNS to resolve .local addresses. So I changed my /etc/nsswitch.conf from this:

hosts: files mdns4_minimal [NOTFOUND=return] wins dns mdns4

to this:

hosts: files wins dns mdns4_minimal mdns4

And now my Ubuntu development machine can communicate with our internal .local machines without having to resort to using IP addresses, short names, or having to place the mapping in /etc/hosts.

Grepping archived, rotated log files — in order

Say you’ve got the following log files with the oldest entries in myapi.log.3.gz:


If you want to ‘grep’ them for a string, in order of date, oldest to newest, there’s no need to extract them one at a time, and there’s no need to concatenate the files first. Use sort to put the files in the proper order, and zgrep to search though the compressed files.

Here’s how to order the file list:

ls myapi.log.*.gz | sort -nr -t . -k 3,3

Here’s how to ‘zgrep’ them in the proper order:

ls myapi.log.*.gz | sort -nr -t . -k 3,3 | xargs zgrep “404”

Ubuntu Unity application launchers for IntelliJ IDEA and PyCharm

Here’s how I got Ubuntu Unity to show application launchers for >IntelliJ IDEA and PyCharm

In $HOME/.local/share/applications, add an IDEA.desktop file with these contents:

#!/usr/bin/env xdg-open
[Desktop Entry]
Name=IntelliJ IDEA
GenericName=Text Editor

And a PyCharm.desktop file:

#!/usr/bin/env xdg-open
[Desktop Entry]

I use the default keyboard shortcuts, and on Linux, CTRL-ALT left arrow doesn’t work with PyCharm or IDEA (jumps back to where I was before I followed a symbol with CTRL-B). I’ve found that CTRL-WINDOWS-ALT left arrow does work. Same thing for many other shortcuts that use CTRL-ALT.

vCloud Director CentOS Guest OS Customization Script

I needed to customize a custom Linux VM image with a static IP address in vCloud Director. Here’s the script I came up with. It doesn’t work unless VMWare tools is installed and running in the VM (kernel upgrades tend to break it). The VM should be stopped, then paste the script into the “Guest OS Customization” script area. Boot the VM with the “Power On and Force Recustomization” option.


echo "------------------------------------------------------------------------------"
if [ x$1 == x"precustomization" ]; then
  set -e
  echo "Running Pre-Customization tasks..."
  if [ x$HOSTNAME != x ]; then
    echo "Configuring hostname as $HOSTNAME"
    sed -i "/HOSTNAME/d" /etc/sysconfig/network
    echo "HOSTNAME=$HOSTNAME" >> /etc/sysconfig/network 
    hostname $HOSTNAME
    # Set it so it can be used below
  if [ x$IPADDR != x ]; then
    echo "Configuring network IP=$IPADDR NETMASK=$NETMASK"
    rm /etc/udev/rule.d/70-persistent-net.rules
    sed -i "/IPADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/NETMASK/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/HWADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/BOOTPROTO/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    sed -i "/DHCPHOSTNAME/d" /etc/sysconfig/network-scripts/ifcfg-eth0
    cat >> /etc/sysconfig/network-scripts/ifcfg-eth0 <<ENDCFG
  echo "Done with Pre-Customization"
elif [ x$1 == x"postcustomization" ]; then
  echo "Running Post-Customization tasks..."
) | tee -a /root/customize.log


Update: It turns out that having VMWare Tools functioning properly is enough for vCloud Director to configure the network settings all by itself, without the help of a script.

Tools and trade-offs

Ten or more years ago, I purchased a Gerber multi-tool. It was less expensive than a Leatherman, and it was painted in my favorite color. Since then, I saw my brother’s Leatherman Wave, and wanted one. For Father’s Day this year, Janice gave me a Wave. I like it — it’s a sleek piece of engineering in every way. The screwdriver fits more of the screws that I encounter compared to the bulky phillips head on the Gerber. The saw works far better. It’s easier to access the Leatherman’s knives, saw and file.

Leatherman Wave vs Gerber multi-tools

However, I’ve found that when I’m out and about in the yard, the Gerber is easier to work with. It’s belt-holster is secured with velcro — easy not only to open, but to close. And opening the pliers is an easy one-handed flick-of-the wrist, which means I don’t have to put the tool in my left hand down in order to open the pliers like I do with the Leatherman.

The Gerber’s pliers can be used to pry things apart. The Leatherman folds up when I try the same thing. The Leatherman is a more dangerous tool to open and use — more likely to pinch or draw blood (as it has done on a few occasions).

The tools definitely have trade-offs. I admire the Leatherman most, but for regular use, the Gerber wins.

The same thing is true of operating systems. I’ve used Linux and Windows for years, and have wanted to try out Mac OS X and a MacBook Pro.

My wish came true. For the past two months, I’ve been using a MacBook Pro 17″ with Retina display. Not only is the engineering and design gorgeous, the screen is eye-poppingly clear and crisp. The battery lasts and lasts, and the laptop doesn’t get too warm. The track-pad is the best I’ve ever used, and I love the gesture support to go forward and back in the web browser, and the gestures to switch desktop workspaces, activate Mission Control and Launch Pad.

Yet the keyboard shortcuts have taken some getting used to compared to the ones I know from both Gnome (Linux) and Windows 7. I find that it’s easy to accidentally hit Command-Q when I meant to hit Command-W. Control-Left and Control-Right don’t work — it’s Option-Left and Option-Right to jump forward and backward by a word. Home and End go to the beginning and end of a document instead of the beginning and end of a line. There’s no built-in window-snapping like Windows 7 or Gnome-Shell.

When I switch back and forth between Mac, Linux, and Windows, I hit the wrong keyboard shortcuts. Is it Command-C or Control-C for copy? On-and-on it goes — sharp edges. But I’m getting used to context switching. I love the MacBook Pro.

Last week, a Gallego UltraPro arrived from System76. It’s a compact powerhouse, and best of all, the keyboard shortcuts work like I expect, and I can get work done faster because I’m used to the underlying OS and tools. When programs break, I know where to look and what utilities to use to solve the problems (I love having /proc available). But the UltraPro with Ubuntu has sharp edges as well. The screen is a bit small, and it’s not as beautiful as the Retina display. It doesn’t have a backlit keyboard, and so far, I haven’t been able to get Evolution to talk to our Exchange server (which isn’t a problem on Mac because it has Outlook).

I admire the refinement, the visual beauty of OS X and the design of the MacBook Pro. For regular use in getting my work done, the UltraPro and Ubuntu currently win.

Debugging shell scripts, User Interfaces and compiled programs on Linux

When I want to debug a shell/bash shell script, I add the following to the script:

set -x
export PS4='+(${BASH_SOURCE}:${LINENO}): '

Other times, I export PS4 in my interactive shell, and run the script with “bash -x /path/to/script”. More bash script debugging tips are here: http://wiki.bash-hackers.org/scripting/debuggingtips

When I want to see what commands are being run on a Linux machine by a higher-level user interface, I turn to startmon. It shows me every process created, along with its arguments. This can be useful for figuring out how a CD/DVD-creation program calls wodim/cdrecord, or how it mounts a drive.

When I don’t have source code to a compiled program, and I want to see what system calls it is making, and check into why it might be failing to run, I use strace.

When I want to find out which programs are preventing me from unmounting a DVD or a USB flash drive, I use ‘lsof‘, which is available for nearly every UNIX-like operating system.

When I want to know what programs are consuming the most disk I/O and making things go slow, I use ‘top’ to see the top-memory consumers, and whether they have a high number of page faults. I use ‘iotop’ as well.

Chasing Linux mount point indirection

It can be a challenge to track down what mount point on a Fedora/RHEL Linux box belongs to which physical disk partition when there are several layers of indirection including Logical Volume manager, Encrypted Disks and UUIDs. Fortunately, the computer does it for us most of the time. But when I, as a human being, needed to step in and figure it out, google came to the rescue. Here are my old notes.

cat /etc/fstab (my transcription of the info):
 / is /dev/VolGroup00/LogVol00
 /home is /dev/mapper/luks-625f820f-1aba-45b3-aacd-4d17dcc9240a
 swap is /dev/mapper/luks-a9362b00-c1c6-470f-9b5b-4e062d96ff10

cat /etc/crypttab:
 luks-625f820f-1aba-45b3-aacd-4d17dcc9240a UUID=625f820f-1aba-45b3-aacd-4d17dcc9240a none

sudo blkid
 /dev/mapper/VolGroup00-LogVol02: UUID="625f820f-1aba-45b3-aacd-4d17dcc9240a" TYPE="crypt_LUKS"

sudo lvscan
 ACTIVE '/dev/VolGroup00/LogVol02' [363.09 GB] inherit

sudo pvscan
 PV /dev/sda2 VG VolGroup00 lvm2 [465.66 GB / 0 free]
 Total: 1 [465.66 GB] / in use: 1 [465.66 GB] / in no VG: 0 [0 ]

sudo fdisk -l
Device Boot      Start         End      Blocks   Id  System
/dev/sda2 14 60801 488279610 8e Linux LVM

I don’t like chasing down that many levels of indirection, so I normally try to simplify things on a desktop system by not using LVM.

Ubuntu and how to disable advertising

Post-Fedora life with Ubuntu 12.04 has been good. I haven’t upgraded to 12.10 (I’m going to stick with a long-term-support release for now), but I’ve heard from people that refuse to upgrade due to the new advertising that shows up in search results. Apparently, Canonical 1) is trying to find ways of generating revenue, 2) made it very easy to disable the advertising. LWN.net explains what’s going on, and the Electronic Frontier Foundation explains how to disable the advertising.



The simple command to disable the advertising is:

sudo apt-get remove unity-lens-shopping

If one is concerned about this, it seems to me that one ought to also be concerned about online advertisers and about gmail sorting through one’s email.


Postfix on Ubuntu 12.04 with Comcast

After having had Ubuntu 12.04 installed for months, I finally moved my old Fedora services over to it, including Postfix. Here’s how I configured it.

I configured /etc/aliases so that local email destined for ‘root’ (including regularly scheduled system cron jobs) is delivered to my local account, instead of being sent to root@comcast.net (they don’t appreciate getting my SPAM):

root: myusername@localhost

I added the following to /etc/postfix/main.cf. Notice that I appended a “.NOT” to the relayhost. I did this initially so that I could send test email messages, and make sure message delivery was correct, before allowing email to go to my ISP. More on this later.

alias\_maps = hash:/etc/aliases
alias\_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost.$mydomain, localhost, robinson-lin
relayhost = [smtp.comcast.net.NOT]:587
smtp\_sasl\_auth\_enable = yes
smtp\_sasl\_password\_maps = hash:/etc/postfix/sasl\_passwd
smtp\_sasl\_security\_options =
sender\_canonical\_maps = hash:/etc/postfix/sender\_canonical
mynetworks = [::ffff:]/104 [::1]/128
mailbox\_command = /usr/bin/procmail -a "$EXTENSION"
mailbox\_size\_limit = 0
recipient\_delimiter = +
inet\_interfaces = loopback-only

I added the following to /etc/postfix/sasl\_passwd:

smtp.comcast.net MyUsername:MyPassword

I added the following to /etc/postfix/sender\_canonical:

jaredrob MyUsername@comcast.net

I ran the following commands:

  chmod o-r /etc/postfix/sasl_passwd
  postfix check
  postmap /etc/postfix/sasl\_passwd
  postmap /etc/postfix/sender\_canonical
  postalias /etc/aliases

Next, I tested my email configuration by sending test email messages (using ‘mutt’ or ‘mail’)…

  1. From my local linux account to my local linux account, to make sure local delivery worked and didn’t get sent to my ISP.
  2. From the local ‘root’ account to my local linux account.
  3. From the local ‘root’ account to my local ‘root’ account, to make sure it was forwarded to my local user account (remember /etc/aliases?)
  4. From my local account to an external account (gmail, etc.)
    1. I didn’t expect it to be delivered at this point. Remember the “.NOT” I added to my relay host? This way, I was able to see where outbound email was destined using “mailq”. This saved me grief because I found misconfigurations that would have lost the email.  I used ‘postsuper -d ALL’ to delete all test messages from the queue.
    2. When I knew delivery was destined for the correct machines, I removed the “.NOT” from main.cf, and ran “service postfix reload” (as root).
    3. Next, I sent email to an external email address, and made sure it was delivered

Then I configured and ran ‘fetchmail’ briefly (hitting CTRL-C after a few were downloaded), and made sure messages were delivered to my local account properly. I’ve been pleased using Ubuntu with postfix, fetchmail and procmail and archivemail. I often log in remotely using SSH and use mutt to read my email. I also use Google Mail. I’ve set up rules to forward some email to my ISP account, which get downloaded using fetchmal via cron job.

Why do I bother saving my email to my local machine? I like redundancy and backups, I like the speed of using a command line email reader, and I have ten years of email archived on my local machine. It’s been very useful to be able to search through that email for old software license keys, website passwords, email addresses and such.

Why Linux?

I’ve acquired a used desktop computer with 6GB RAM, and I’ve been trying to figure out whether to put Linux or Windows 7 on it in the long term. I’d like it to be useful to the whole family.

Windows 7 is great for playing DVDs, Youtube movies, doing Netflix streaming, games, printing to our aging Kodak EasyShare 5100 printer, syncing with my iPod Touch, and it has a fantastic photo screensaver. Some of our favorite applications are available for Windows, but not Linux.

Windows has great parental control software available — via it’s built-in controls, or via third party software. We limit the amount of time our children can be on the computer each day, and we limit the kinds of websites they can visit.

Linux, on the other hand, costs less and supports the sound card and the graphics card better than Windows. And much to my surprise, Linux now supports our Kodak printer (but not its scanner).

There are several Linux distributions available, and I’ve tried Fedora 17 and Ubuntu 12.04 on this machine in their 64 bit variants. Both are well supported by a vibrant community and by commercial companies.

Fedora doesn’t play DVDs, has trouble with some Youtube videos, can’t stream Netflix, doesn’t have parental controls, doesn’t sync with my iPod Touch, and doesn’t even include a screensaver — just a screen blanker (thanks to the unfinished state of its shiny new-and-highly-immature desktop, Gnome 3). It’s possible to rectify many of these limitations, but it takes time to find and configure the software. Netflix streaming isn’t an option. Getting a photo screensaver requires the use of an alternate desktop environment, like KDE or XFCE. In short, Fedora isn’t family desktop ready.

Ubuntu has most of the same limitations as Fedora, but its desktop experience is more polished and the online help is phenomenal. The Ubuntu Software Center not only has application ratings, but can also recommend other apps. Ubuntu automatically told me there were proprietary ATI drivers available, and it was painless to switch to them, with enough improvement in accelerated performance to make it worthwhile. Ubuntu has parental controls in the form of Gnome Nanny, but it doesn’t work with 64 bit Linux, although a web browser extension is an option.

So why do I bother with Linux? It’s an exceedingly useful swiss army knife, runs blazingly fast, and doesn’t slow down over time when software is added. I’m heavily invested in it, and it gives me a technical edge in the workplace. There are thousands of interesting tools that are easy to install — Inkscape, Gimp, meld, Chromium and others. Linux has full featured, freely available development tools for C, C++ and Java — not crippled ones. And it’s multi-user enabled via ssh, remote-X, vncserver, or NX server. Mail servers, web servers and other servers work extremely well. With Linux, it’s easy to see what application or service is slowing down my computer with ‘top’ or ‘iotop’, whereas in Windows, it’s obscured because services run as threads in svchost. I love Linux Live CDs for troubleshooting computer problems, testing hardware and internet connections, and for recovering data from broken Windows installs.

So Linux is great for me as a software developer, but I’m not convinced that Ubuntu, Fedora or any other Linux distribution deserves to be a family desktop environment.