Category: Linux

Posted on

Teriffic Linux Books

One of my QA coworkers had The Linux Phrasebook sitting on his desk. I noticed it was by Scott Granneman, and immediately thought, “if it’s by him, it’s got to be good.” As I thumbed through the pages of the book, I was impressed at the practical coverage of command line utilities including yum and rpm, apt and deb, etc. I highly recommend this book. It’s small so it’s easy to keep on hand.

For anyone wishing to write applications that take advantage of all that Linux has to offer, I recommend The Linux Programming Interface (also available from Amazon), aka TLPI. I used to recommend Stevens’ Advanced Programming in the UNIX Environment, but now I recommend TLPI because it is more comprehensive and just as readable. It’s not a small book, so a PDF makes it easier to keep on hand. I purchased my copy through the publisher, NoStarch with the coupon code “Mamaku” that gave me 30% off and a free PDF. The order total came to $80.00, including shipping.

Posted on

Awesome Linux profiling tools: top -H and pstack

When I needed to figure out where the performance bottlenecks were in some Linux software, I found helpful answers at http://stackoverflow.com in the form of two tools: 1. naming threads in combination with “top -Hp <pid>” and 2. “pstack <pid>”. The first was helpful in watching which threads were consuming the most CPU. The second was useful in sampling the application over time to find the hot spots and their stack traces.

  • How to name a thread in Linux (or a process) — use prctl().
  • How to use stack sampling to find performance problems — use pstack.
Posted on

Preventing laptop hard drive from overheating

Six months ago, I replaced the failing hard drive in my Linux laptop, and already, the SMART tools are telling me that I should back up and replace the hard drive — a high number of sectors have gone bad.

Hmmm. What’s this? SMART also reported that the hard drive had reached “overheating” temperature ranges. Why would that be? I did some Google searching, and came up with the following advice:

  1. Don’t close the laptop lid while it is powered up! This is how I had normally run my Linux laptop — it’s a server, and I leave the lid closed. Oops! I’ve changed the power settings so that when the laptop lid is closed, it sleeps.
  2. Edit /etc/grub.conf and add acpi_osi=Linux or try acpi=off to seee if apm (automatic hardware control) will take over. I’ve just started trying the former. UPDATE 8 Feb 2011: Using this prevented my laptop from waking up from sleep, so I stopped using it.
  3. Vacuum the dust off the fan screen (to prevent airflow blockage)
  4. Monitor the temperature with ‘smartclt’

Based on a tip from my father (a long time Linux expert), I ran “smartctl -H /dev/sda”, and it says “SMART overall-health self-assessment test result: PASSED”. I assume it means the hard drive is still okay, but I had better not forget to make regular backups, and monitor the status of the hard drive.

Posted on

Miscellaneous Linux tips and tricks

Gnome Key Bindings and IntelliJ IDEA

IntelliJ IDEA key bindings conflict with Gnome’s window manager. In IDEA, I can type CTRL-B to jump to a symbol definition. Normally, I’d type CTRL-ALT-LEFTARROW to navigate back to where I had come from. Gnome’s MetaCity intercepts that key mapping before IDEA sees it, and tries to move my desktop to the left. There are several other Gnome/Metacity key bindings that conflict with IntelliJ IDEA. Rather than remap the keys in Gnome, I found that on Fedora, I could add the Windows key to the mix, and Gnome would ignore it, and pass it along to IDEA. This means that I can type CTRL-ALT-WINDOWS-LEFTARROW to navigate backward, and so forth.

Unfortunately, this doesn’t work in RHEL 5 and CentOS 5. The solution is to Go to the Gnome menu bar and select System > Preferences > Keyboard (not Keyboard Shortcuts). Then select the “Layout Options” tab, and expand the “Alt/Win key behavior”. Then I select “Super is mapped to the Win-keys”.

Every time I log in after that, Gnome tells me that my X keyboard settings conflict with my Gnome Keyboard settings, and it asks which I want to use. Selecting the Gnome settings is what I want.

Bandwidth limiting downloads with ‘curl’ or ‘wget’

When downloading a large file, it’s nice to be polite to others on the network, so I use the --limit-rate option for curl and wget:

* curl -O --limit-rate 20k http://server.com/linux.iso
* wget --limit-rate=20k http://server.com/linux.iso

GDB TUI (text user interface)

After starting gdb, it’s possible to switch to its text user interface with CTRL-X, CTRL-A. Typing it a second time exits TUI mode.

Vim C++ Auto completion with ctags

I appreciate full blown IDEs in Linux, but I like the quick start up time of vim. Until recently though, I didn’t have C++ auto completion (also known as vim omni completion).

This got me up and running, and was a great resource:
http://vim.wikia.com/wiki/C%2B%2B_code_completion

This would have been useful if I was a new comer to vim and ctags:
http://www.justlinux.com/nhf/Programming/Introduction_to_C_Programming.html

xdg-open, gnome-open, start, cygstart

How to easily open files and URLs from the command line
http://www.dwheeler.com/essays/open-files-urls.html

  • Linux: xdg-open filename_or_URL
  • Linux: gnome-open filename_or_URL
  • Mac: open filename_or_URL
  • Windows: cmd /c start filename_or_URL
  • Cygwin: cygstart filename_or_URL

Nomachine NX and ALT-TAB

I use the Nomachine NX client from time to time to get a remote-desktop like connection to a remote Linux machine. It’s faster than VNC, but it suffers from not forwarding all of my keyboard shortcuts to the remote end of the connection.

Usually, I start the nxlcient from within a Gnome login session. Gnome happily grabs ALT-TAB before the NX client gets to see it. That’s not what I want. To work around this limitation, I log into a virtual terminal, and start X manually as follows:

Type CTRL-ALT-F2
Login
Run: startx -- :1 gnome-terminal

From the gnome-terminal, run: nxclient

And then I connect to the remote machine in full screen mode. There’s no local window manager to interfere with my keyboard shortcuts.

Remote desktop and dual screens

I’ve been using Remote Desktop to connect to Windows XP, Vista and 7 machines. Until Windows 7, there was no way for a local computer having dual monitors to connect and have the remote end display across both monitors.

So I used linux’s ‘rdesktop’ program to do it:

rdesktop -0 -a16 -f -rdisk:CLIENT=/home/jared/Desktop -r sound remote.host.com

I notice that in Windows 7, there are some new options in the Remote Desktop client (mstsc.exe): /multimon and /span. Or run mstsc /? to list all possible options.

Editing windows registry files on Linux

Use Gedit: gedit --encoding=UTF-16LE myfile.reg

Gvim: LANG=UTF-16LE gvim myfile.reg

If already in gvim: :e! ++enc=utf-16le
or :e ++enc=utf-16le myfile.reg

Convert, edit, convert:

iconv -f UTF-16LE -t utf-8 myfile.reg > myfile.reg.utf8

Edit myfile.reg.utf8, then convert it back

iconv -f utf-8 -t UTF-16LE myfile.reg.utf8 > myfile.reg


How Firefox opens files and mime types

I needed to give Firefox some extra help knowing how to open a custom file type with a custom application. Here’s some helpful information.

https://developer.mozilla.org/en/How_Mozilla_determines_MIME_Types

Firefox uses mime.types on Linux, as well as other things. I helped Firefox by the mime type to the link in the generated HTML file. Either one of the following seems to work:

* <a href=”file:subdir/file1.cst” type=”application/octet-stream”> open file </a>
* <a href=”file:subdir/file1.cst” type=”application/x-extension-cst”> open file </a>

Posted on

Worth the money: Automated, online backup

*10 February 2010*

Yesterday, I found out I’d lost over three thousand calendar entries, and I had lost them five months ago. Fortunately, I had been using an
automated, [online backup service](http://mozy.com) and was able to restore the missing data.

I found out about my loss when I searched for a phone number on my Palm TX that should have been in my calendar, but was missing. I wondered what was up, and started going through my calendar a month at a time. I noticed that calendar entries after Sept 7, 2009 were present, but nearly everything before that was missing.

My Palm TX is synchronized frequently with Windows, and infrequently
with Linux. My Linux copy of the calendar wasn’t going to help me,
because it was missing the calendar entries as well. The same was true
for the Windows copy.

The Palm-to-SD-card backup that happened every night wasn’t going to
help, because it deletes any backups older than seven days old to make
room for the new backups. I needed something that stretched back five
months or more.

The backup of my Linux computer wasn’t going to help me, because I _overwrite_ my old
backups with new copies of the same files, using ‘rsync’.

I thought my Mozy backups worked the same way. Fortunately, I was
partially wrong. Mozy keeps point-in-time backups of some files. I don’t
know how they determine which files to do it for, but they did it for my
Palm Pilot calendar database file. I was able to restore my missing calendar entries, which was a huge relief.

I heartily recommend automated online backups. Manual backups aren’t
done by most people and if they are done, they’re sporadic and
incomplete. My intermittent manual, replace-the-old-files style of
backup to USB hard drive wouldn’t have allowed me to restore the
calendar entries. The $5/month that I spend for online backup was very
worthwhile, and easy to justify considering that it’s less than the cost
of eating out for lunch. It’s less expensive than a cell phone or
monthly internet service.

If you aren’t already doing automated backups, I recommend that you sign up with an online backup service today. Here are some recommendations:

1. [Dropbox](https://www.dropbox.com/) is the most popular. Works on Windows, Mac,
Linux, iPhone.
1. [Spideroak](http://spideroak.com/) is the second most popular. Works on
Windows, Mac, Linux.
1. Alternatives to these, including [Mozy](http://mozy.com), which is what I use for Windows: [http://alternativeto.net/desktop/dropbox/](http://alternativeto.net/desktop/dropbox/).

—-

A word of caution: backups can’t work miracles. If a file was
corrupted BEFORE it was backed up, no backup solution is going to be
able to solve the problem. This is why I make two copies of all photos
from my digital camera BEFORE deleting them from the camera. Still, if
the memory card in the camera contained corrupted images, even this
wouldn’t be good enough.

—-

The missing calendar entries were, in fact, not missing. They were
corrupted. I found this out by running `jpilot-dump -D | sort -r` on my linux computer. I had 3462 blank entries listed on 12/31/1969. The first time
I restored my Windows datebook.dat, and hot-synced, all of the restored
records were again “deleted” because my Palm though it had the more
current copy of those records in 1969. I had to purge the records from
my Palm _before_ hot-syncing with the restored datebook.dat file.

—-

Techrepublic has a [Review of 10 outstanding Linux backup utilities](http://blogs.techrepublic.com.com/10things/?p=895), many of which work on
other platforms as well.

Personal solutions (not hosted):

– [Simple Backup Suite](https://help.ubuntu.com/community/BackupYourSystem/SimpleBackupSuite) for Ubuntu and Fedora, which does full and incremental backups, on a schedule or manually. Install it on Fedora by running “`yum install sbackup`”. Configure and run by running “`/usr/bin/simple-backup-config`”
– [fwbackups](http://www.diffingo.com/oss/fwbackups), of which Techrepublic says, “This is, by far, the easiest of all the Linux backup solutions.”
– [Rsnapshot](http://rsnapshot.org/)
– [Duplicity](http://duplicity.nongnu.org/) which is a command line utility, and is recommended by http://rsync.net

Posted on

f-spot and sqlite

I recently tried using Linux [f-spot](http://f-spot.org/), with the intent to make it easier to browse, manipulate, manage and publish my photos. I wanted f-spot to manage my photo screen saver as well. f-spot seems to be good at importing photos, but getting photos removed is a bit more difficult.

I organize my photos by date an a directory structure such as “2010/2010.01.01 New Years Day”. The “2010” directory contains several sub directories. Each sub directory consists of a date and a description. If, for some reason, I import photos into f-spot that I don’t want in its database, I know what directory the photos pertain to. Unfortunately, F-spot doesn’t allow me to remove photos from its catalog by filename or file path. That’s okay though, because it stores its database using sqlite.

I figured this out by running lsof -p pid-of-f-spot, and noticed a file descriptor opened to “/home/jared/.config/f-spot/photos.db”. Then I ran file ~/.config/f-spot/photos.db and it helpfully told me that it is a “[SQLite](http://www.sqlite.org/) 3.x database”.

After a bit of google research, I figured out I could install a SQLite manager on my Fedora system: yum install -y sqliteman, followed by running sqliteman ~/.config/f-spot/photos.db. I was expecting to see a command-line client, but to my surprise, I found a pleasant graphical interface. It was simple to browse the table schema and to run queries to update and morph the f-spot photo database. Note: I’d recommend making a backup copy of the database before altering it.

F-spot may not be everything I want it to be, but I managed to work past its limitations due to the fact that it used a well known, open data storage format.

Posted on

Minimizing tracing/instrumentation overhead, injectso

Reading these articles from lwn.net: [Minimizing instrumentation impacts](http://lwn.net/Articles/365833/) and [Debugging the Kernel using Ftrace](http://lwn.net/Articles/365835/), reminded me of [Microsoft detours](http://research.microsoft.com/en-us/projects/detours/) and [Linux injectso](http://c-skills.blogspot.com/) (updated to work with current glibc, kernels).

Posted on

Using rsync with SELinux

Last week, I needed to move /home from one Fedora computer to another, and I used rsync over ssh move the data.

On the new system, I noticed that procmail didn’t seem to be working, and neither did Dovecot. Nor could apache serve up my files. This had all been working on my previous Fedora system, which was running SELinux, as was my new system. What had happened?

I hadn’t told rsync to bring across the SELinux file contexts, which are stored in extended attributes. Here is the rsync option I should have used:

-X, –xattrs

I could have used ‘tar’ to move my home directory as well. In that case, I would have needed one of the following options: `–selinux` or `–xattrs`

I resolved my SELinux issues using the excellent [SETroubleShoot](https://fedorahosted.org/setroubleshoot/), which explained what commands to run to restore the proper SELinux contexts on various files.

SELinux requires time to tune, and I use it because it enhances the security of my linux system, which serves up content over HTTP (Apache), IMAP (dovecot) and CIFS (Samba).