mwrc: Puppet by Andrew Shafer

Puppet
by Andrew Shafer, works for Reductive Labs

Code, Tribes, Dawn of Time, Clouds, evolution, opportunity, and triumph

It’s difficult to automate the download ruby gems because the URL paths are
not friendly. Isn’t it interesting that RubyForge.org uses PHP to run?

Idempotence: an important property to have

He likes Debian’s setup of apache. It’s easier to work with for setting up
multiple sites than the way most Linux distributions do it.

He showed several examples of using Puppet to install packages from gem
repositories and packages.

How do you test your puppet deployment code? The best he’s come up with is to
use RSpec to make sure a machine is in the correct configuration, and run
continuous integration tests on it with your puppet code.

The Cloud. Anyone can bring up a server for 10 cents per hour, and when you’re
done, you turn it off.

Conversation with the audience:

Things get complicated when you’re using puppet to manage deployment to
different distributions or even platforms. E.g. Debian has two packages for
something that is a single package on another. The only way to make sure your
puppet deployment works is to try it and make sure the machine is in the
correct state afterward.

Does puppet help you to remove packages? Some providers purge unrequested
packages. It’s hard to back out a messed up install, and is easier to start
over with a clean slate.

mwrc: Sequel: The Database Toolkit for Ruby by Jeremy Evans

Sequel: The Database Toolkit for Ruby
by Jeremy Evans

He’s not the original implementer of the Sequel module. He’s improved it quite
a bit. He added Jruby and Ruby 1.9 support, among other things.

Sequel is a domian-spefic-language (DSL) for Ruby for dealing with databases.
It’s not designed with web apps in mind, although it does work with them.

He spoke very quickly, as he was constrained for time. Sounded interesting,
but was a bit hard to follow due to the speed.

sequel_model – an ORM build on Sequel

He uses it successfully with the adaptors for Oracle, MySQL and PostgreSQL adaptors. Has a thread safe connection pool. Insures db connections aren’t leaked.

Sequel::Dataset sets it apart from other Ruby db libraries.

You can write complex SQL data queries without writing SQL, although you can if you want.

– Dataset Graphing
– Pagination Support
– Create table, Schema modification, etc.
– Master/Slave databases — requires no modifications to your code.
– Sharding/Partitioning – simple to support with Sequel module
– Bound variables /might/ be faster, but don’t depend on it.
– Prepared statements are similar to bound variables, but potentially faster.
Only use after profiling/benchmarking.

Has a command line client called ‘sequel’. The -E option causes all SQL to be
echoed to standard output.

The future: Sequel 3.0. Many features will be moving to plugins, and the
majority of the work is already complete. It’s going to be easier to use, and
require less memory.

Q: Does it support sharded database migrations?

A: No, not at this time. You can submit a patch.

mwrc: Rails and Merb Merge by Yehuda Katz

Rails and Merb Merge by Yehuda Katz or The Future of Rails 3.0

Engine Yard has a new offering called [Solo](http://www.engineyard.com/solo/)

ActionORM is a way of specifying an abstraction around an object. It makes any
ORM look like ActiveRecord. He mentioned Rack integration.

Merb provides some middleware. Rails added various middleware. It’s easy to use different kinds of middleware stack pieces.

Orchestra – an idea to build a module that is a common instrumenting framework
for SQL and web frameworks.

Speed of languages: Ruby 1.9 and JRuby are very close to Perl in speed. He showed Python 2 as
faster than Ruby 1.9 and Perl. Ruby 1.8.6 is the slowest of the comparisons he
had posted. I just hope we all remember that speed benchmarks aren’t worth much.

He spoke quite a bit about community agreement and standards. Perhaps that’s a
sign of the growing maturity of Ruby.

mwrc: Rack Middleware by Jon Crosby

Rack Middleware
by Jon Crosby, an employee of Engine Yard

He started by showing an example of monolithic Perl CGI, and moved quicky to Rails. Then he moved to talking about authentication and Rack. I was a bit lost as to what Rack does, or why I would care — probably because I’m not a middleware developer.

Authentication: OpenID + Oauth
Rack is the Web. The Web is Rack
Rack is based on WSGI. It’s a webserver interface. http://rack.rubyforge.org/
“Rack provides an minimal interface between webservers supporting Ruby and Ruby frameworks.”

He showed some examples of rack.

– Rack::Pool:Session
– Rack::Profiler
– Rack::MailExceptions
– Rack:CSSHTTPRequest
– Rack::Cache – highly recommended

Cooperative Middleware
Take a look at URI spaces
CloudKit – and Open Web JSON client http://getcloudkit.com/
– expose :notes, :todos
Has an OAuth plugin “filter”

“Via” HTTP headers. Used by Rack middleware

env[‘cloudkit.auth’] = 1
env[‘cloudkit.via’] << 'cloudkit.filter...' Alternative stacks - Rack::Map could let you create two apps for your same blog - a public and a private one. - Rack::Map + Sinatra 0.9.1 Question: How do you detect colissions in the URI namespace? Answer: There's no facility in Rack to do that. It's a good time to work on it before Rack hits version 1.0.

mwrc: Code Reading by James Edward Gray II

I’m attending the third annual Mountain West Ruby Conference in Salt Lake
City. Since my notes may not represent the speakers accurately, it may be
wise to read the summaries of the presentations over at the
over at the [web site](http://mtnwestrubyconf.org/2009/speakers). I’m taking notes for each of the speakers, and I’ll prefix the title of each blog post with “mwrc”.

Code Reading by James Edward Gray II

Little Big Planet: A cool PS3 game that people have created levels for. It’s interesting to see the code that shows how people create levels.

Why is code reading important?

– Shows common idioms
– Get better at working on other’s code
– Understand how something works
– Understand limitations
– Seeing bad code helps you write better code
– Always need more ideas

rest_client: Networking code done right
You can tell from the structure that it’s clean code. He showed us the ugliest parts of the code, and it looked clean.

Restful-shell: IOD shell
If you create a log file, it’s executable code that you can then edit and enhance.

He created the Towers of Hanoi for LittleBigPlanet. He had to redesign the playing interface serveral times to get it to fit into the game correctly.

### FasterCSV

Sometimes users want to work using indexes of columns. Other times, they want to work with column names. Sometimes, column names repeat.
This library was one of the first users of the new Ruby m17n libraries, and he found a lot of bugs at that time.

BJ, Slave, Terminator: Ruby libraries to build multiprocessing UNIX software. Built to be very robust.

– Terminator is better than Ruby’s timeout handlers. “plot_to_kill pid”
– BJ – robust background priority queue for Rails. Notices changes from outside world via signals.
– Slave – multiprocessing with built in IPC. Heartbeat.

### The Art of Code Reading

– Take a deep breath and relax
– Not all code sucks
– Don’t start with Rails. There’s a ton of great stuff in there, but it’s really complex and hard to follow.
– Have a goal. “How does it do that?”
– gem unpack
– Start with the tests
– Check for “examples”
– Try irb: irb -r

Bypassing the I.T. security fortress

On the back of my mind for the past few years, I’ve been thinking about how I.T. security becomes less meaningful as time goes on. The use of digital cameras isn’t usually allowed, yet a company isn’t (usually) going to boot out an employee for having a cell phone with a digital camera — or even using it to take a snapshot of a diagram that will be placed on a corporate wiki. The use of USB thumb drives for transferring and storing corporate data is perceived as a risk, but often, it’s a practical way of getting one’s job done. Remember network firewalls? They’re still in place, but they’re increasingly meaningless. They certainly don’t keep out viruses and trojan horses. And with the increasing prevalence of wireless networking, there’s even less incentive for people to play by the I.T. security rules. Dan Kaminsky [expresses these thoughts better than I have](http://www.doxpara.com/?p=1245)
:

> … every restriction, every alteration [I.T. makes] in people’s day to day business, carries with it a risk that users will abandon the corporate network entirely, going “off-grid” in search of a more open and more useful operating environment. You might scoff, and think people would get fired for this stuff, but you know what people really get fired for? Missing their numbers.

> Its never been easier to get away with going off-grid. Widespread availability of WiMax and 3G networks mean there’s an alternate, unmonitored high speed network available at every desk.

Kaminsky [goes on](http://www.doxpara.com/?p=1245) to discuss some of the ramifications of these ongoing changes, including “the Cloud” (e.g. Google docs) and the security of corporate data.

jvisualvm: A free Java memory and CPU profiler

I needed to profile a Java application, and since we had a JProfiler floating license, I used it. JProfiler works well, although it’s pricey. I was googling for other Java profiling tools, and [stackoverflow.com](http://stackoverflow.com/search?q=visualvm) made mention of [jvisualvm](https://visualvm.dev.java.net/), which comes bundled with JDK 6 release 7. I noticed that on my Fedora 10 box, the java-1.6.0-openjdk package includes jvisualvm. None of my coworkers had heard of it.

JProfiler introduces a significant performance penalty into the code it profiles, whereas other tools including jvisualvm and YourKit have a much lower impact. I’m going to give jvisualvm a try, once I get the target environment set up properly with the new JDK.

UPDATE: jvisualvm won’t profile remote applications like JProfiler can. jvisualvm is not quite as easy to use, and I haven’t figured out how to get stack traces on the CPU and memory hot spots. Overall, I like the tool.

UPDATE 2: jvisualvm can be configured to give a stack trace of memory hot spots. I’ve learned that performance between the Java 1.5 and 1.6 jvms can be very different. I’ve learned that I can run ‘kill -3 ‘ to print a stack trace of my running java processes. It’s helped me to narrow down bottlenecks in an application when the profiler wasn’t granular enough.

The future of Gnome Apps: JavaScript?

There’s an interesting article called “[Building desktop Linux
applications with JavaScript](http://arstechnica.com/articles/paedia/javascript-gtk-bindings.ars?bub)” By Ryan Paul, January 19, 2009.

I didn’t immediately understand the vision. Don’t we already have
Python, Ruby, Java, C++ and Perl bindings for Gnome? Yes, we do. So why
would we add JavaScript to the mix? Or any other scripting language?

The best way to think about it is Firefox plugins, like Greasemonkey,
that actually modify the web browser to give you a new experience.
Firefox extensions are written in JavaScript. JavaScript has hooks into
the application (Firefox) to manipulate it.

Gnome hackers want to do the same thing for Gnome. Not only could you
write Gnome application in JavaScript, you could extend a Gnome
application using JavaScript, no matter what language it was written in.

Another way to think about it is this: When most people think of Java,
they don’t think of the language. They think of the platform — the
libraries that are shipped with the language (networking, database
connectivity, etc.). The same is true for Python, Perl, and Ruby.

The goal is to us an embeddable language to tweak the Gnome platform,
not to use a platform (like Java, Python or Perl) to tweak Gnome. When
they embed a language into Gnome, application developers will use the
Gnome platform way of doing networking, instead of doing it the Java
library way. They will use the Gnome way of opening file picker, not the
Java library way. They will use the Gnome way of doing HTTP, not the
Python or the Java or the Perl way.

Fedora 10 lacks “wow” appeal; OpenSolaris 11

I upgraded one of my machines to Fedora 10 last month, and for me, this release lacks the “wow” appeal that other releases have had. A minor annoyance is that the keyboard repeat delay [is broken](https://bugzilla.redhat.com/show_bug.cgi?id=475747) for me and so far, there is no fix other than disabling keyboard repeat. On the plus side, Fedora 10 includes OpenOffice.org 3 and other [new](http://fedoraproject.org/wiki/Releases/10/ReleaseSummary) [features](http://www.heise-online.co.uk/open/features/print/112093). Be sure to check out the [Common Issues](http://fedoraproject.org/wiki/Bugs/F10Common) people have experienced with Fedora 10.

The other day, a co-worker handed me an OpenSolaris 11 Live CD. I booted it, expecting to be underwhelmed like I was with the Solaris 10 JavaDesktop. I was pleasantly surprised, however. Sun’s “Nimbus” GNOME theme knocks the socks off of the boring Fedora window manager themes. The experience felt like I was running Linux. It was responsive, supported my newer hardware, and the system was built with GNU utilities on the command line so I get my favourite options to ‘ls’, ‘grep’, etc. It supported my NVidia card out-of-the box, and had Compiz eye-candy as an option. The only thing I missed (in my superficial test) was the familiar ‘yum’ and ‘rpm’ for package management. I suspect that if I used it from day to day, I’d find other things I miss. Does OpenSolaris support encrypted file systems? Does it have as much optional software as I can get with Fedora Extras?

I’ll keep my eye on [OpenSolaris](http://www.opensolaris.com/) a little more closely in the future.

Backup: Immediate, Full and Long-term

Preserving the availability of digital artifacts is a goal worthy of pursuit. First, I’ve got thousands of digital family photos, and I don’t want to lose them to hard drive failure or lock them up so that they’re hard to get to. Second, I’ve got my email stored on my computer for the past several years. The recent email is what’s most valuable to me, but every once in a while, I need to search through email archives to find things, like a license key for [Date Book 5](http://www.pimlicosoftware.com/datebk5.htm). Third, it took weeks to install software and configure our laptop. I don’t want to have to repeat that work if the hard drive happens to stop working — especially if a project I’m working on needs to be done soon.

There are three main types of backup that are important to me: Immediate backup, full backup and long-term archival.

__Immediate Backup__
—————–

What I’m currently working on with the computer is usually more important than what I was doing on the computer a few weeks ago. The auto-save and even the “undo” feature of most word processing programs can help me when one of my children touches the keyboard and accidentally deletes most of the text. Auto-save and undo won’t help if my laptop is stolen or the hard drive fails. That’s why I use [mozy.com](http://mozy.com) for automated, off-site backups of my Windows laptop. It’s well worth $5.00 per month for this service, and it’s easy to pay for: skip eating out for lunch once per month.

For backup to happen regularly, automation is key — especially for immediate backup. I would make full backups more frequently if it were an automated process. I use a monthly repeating reminder so I remember to backup the things that aren’t automated.

__Full Backup__
———–

Admittedly, hard drives don’t fail often, and laptops that usually stay at home aren’t often stolen (at least, not in my neighborhood). But when it does happen, it’s a pain to reinstall the myriad of applications we use on a semi-regular basis. This is why a periodic, full backup is valuable. Doing a full backup with optical media takes too much time. External hard drives are much faster, have more capacity, and are inexpensive. They plug in using standard connectors such as USB, FireWire or eSATA. I store my external USB hard drive in a fireproof box.

__Long Term Archival__
——————

I want the best of my digital memories (e.g. photos) to be preserved for decades or centuries. A CDROM may be readable in ten or twenty years, but not fifty or a hundred. There probably won’t be hardware to read it in fifty years. Will computers in fifty years recognize JPEG format? No idea!

To preserve digital artifacts for that long requires refreshing it periodically into newer formats and storage media. It’s a good idea to use open, standardized formats rather than proprietary formats. For photos, this means to use JPEG and PNG in preference to Photoshop format.

Rather than refresh constantly, there’s the option of _printing_ photos and documents. It’s going to be easier to view a physical photo or a printed document in a hundred years than to unlock the secrets of an old hard drive.

__Trust, but verify__
—————-

I tend to trust my backup solutions, but it’s necessary to verify that they’re working. My brother’s computer periodically downloads my digital photos. I trusted that this was, at least in part, a good off-site backup. I learned recently, however, that his computer deletes old photos when space gets low, which is often.

__Resources__
———-
[Preserving Your Digital Memories: What you can do](http://www.digitalpreservation.gov/you/digitalmemories.html)

A few backup solutions: Mozy, Carbonite, SyncBackSE, and JungleDisk.

Interesting projects to backup using P2P protocols (including featurs such as encryption and
fault tolerance): [Tahoe](http://allmydata.org/trac/tahoe)
with a [writeup from LWN](http://lwn.net/Articles/280483/) and [Flud](http://www.flud.org).