Category: Programming

Daniel Philpott: Herding Tigers — Software Development and the Art of War

He’s been a manager for a long time. He’s built systems for the DOD, worked
for CitiBank, ticketmaster, cisco, etc. Now he’s working for AT&T interactive,
and they’re putting up with him.

Why is software development so difficult? Because people don’t admit the
truth — they’re afraid — people don’t know what the end result will be. Why
not? Change.

A tiger team is one answer to the problem. He manages them very little. They
are a team. QA is part of the team. Members show leadership, and self-improve.
Almost no meetings. QA people and Test automation from the beginning. Shared
psychology and intelligence in the group. Bold. Winning. Excellence — members
push each others. Not afraid of “the dark” — of not knowing where they’re
headed when they get started. They know clients don’t know what they want.

Who’s on a tiger team? All the staff needed to deliver a product. Leader, 4
devs, one AQA, doc, etc.

He doesn’t demand that tiger team members pair program. They can pair, swarm,
or whatever they want — “shared intelligence”.

Business is battle. The battlefield is the marketplace, and is typically
divided into attrition and maneuvering. GM, Home Depot are experiencing
attrition warfare, and it’s not useless. Maneuver warfare — eBay, mySpace —
people didn’t see them coming.

General Sherman lost 1% of his men in the Civil War. He decided he didn’t need
to fight the confederate army to stop slavery. Instead, he started burning
down plantations, and the confederates came running home. He saved lives.
Daniel grew up in the south and feels bad that he grew up hating Sherman.

General Patton decided that instead of fighting the Germans in Normandy, he’d go
around them, up the Rhine, and attack them from behind.

Speed adds job satisfaction, excitement. Time flies when you’re having fun.
When you’re having fun, time flies. Speed builds credibility.

A cow is twice the size of a tiger, so which of the two wins? The one with
speed. Are you using agile terms to describe a non-agile process? It’s the
difference between cows and tigers: A cow in tiger’s clothing.

Leadership makes or breaks a tiger team. Faith in your team is critical. Gotta
love your battle buddies. Need to hope. If you have no hope, you might as well
find a new job.

Success belongs to the team. Fear belongs to the leader. No finger pointing —
the buck stops here.

“I have fear, but I do it anyway”.

Be a listener and a learner. He, as a leader, asks his teams what he can do to
make their jobs better. He protects his team from outside stupid influences.

Discussed U.S. Marines management techniques. Reward failure, which seems
counter intuitive — which means he thanks people for their efforts — not
that he gives gift certificates, but he will give them a drink. No failure
means no progress. Glorify the grunts — your programmers and QA people are
extremely important. Be polite and professional — no poison in the team.

Agile isn’t a methodology, it’s a mindset. And it’s inevitable.

Don’t get too attached to your code, because we’re going to rip it out.

Recommends a book called “warfighting” from the marines. You can buy it on
Amazon.

Jeremy McAnally: Jive Talkin’: DSL Design and Construction

He started by talking about how China has a ton of language dialects. And the
island [Papua New Guinea](http://en.wikipedia.org/wiki/Papua_New_Guinea) has an extremely high language density. North America has a relatively low number of language dialects.

Ruby Modules that have Domain Specific Languages:

* Cucumber
* Ruleby
* Configuration
* Puppet

Internal

* Rails Templates – very concise. some people would like more bubbly language
* RSpec – the model for Ruby DSLs these days
* Permissionable

Machine Learning by David L. Richards

– TenaciousG – a graphs tool with persistence.
– Sirb – statistics and erb. A bit of a hack — it monkey patches other classes. It’s kind of cool
– TeguGears – Tegu is a never finished gem for machine learning, all in one
place. TeguGears is meant to be its core functionality.

statisticus – an immature “R” type stats package for Ruby. Don’t use it — it will break — if you can even get it to run. He’s still cleaning it up, and feels that it has potential.
rbtree

A lot of machine learning uses graphs.

Approaches

– Ruberl
– Rocaml
– Blurred Weasel

Functional style programming has three primary advantages:

– Composable
– Concurrent
– Distributed
– Etc.

Fibers. They’re a “new to Ruby” idea — coroutines. They start “paused” and
they are “resumed”.

– PipelineElement – stolen from Pragmatic Dave (Thomas).
– [NeverBlock](http://espace.com.eg/neverblock) is an I/O oriented library. Works with Fibers.

Brian Marick – Test-driving GUIs (with RubyCocoa). Brian is a good, enjoyable presenter.

Objective-C is more like Ruby than it is like Java.

He tests apps written in Objective-C using Ruby. This works fairly well, with
an occasional necessity to explicitly convert from a Ruby boolean to an Obj-C
boolean type.

The danger of using mock objects is that they don’t behave like the real world
objects that they mock. Sometimes, he has the test verify the expected result,
but runs the interaction with the application manually. He’s the “mechanical
turk”.

FFI (Foreign Function Invocation): Making Cross Engine Extensions
by Jeremy Hinegardner

He Loves Ruby and C. This talk is about Ruby extension development.

FFI is a way for something in one language to call functions in another
language. It’s great because FFI works between JRuby and libsomething.so, or
between Matz Ruby and libsomething.so, etc. FFI reduces the amount of code you
have to write to bridge between Ruby and C. I wonder how it compares with
Java’s Java Native Access. Are they complimentary (for JRuby)?

There’s an FFI generator that takes C header files and generates FFI code for you.

FFI saves developer time. It’s not as fast as a traditional C extension. When
you need to support JRuby or Rubiniess, then it saves even more developer
time.

How many gems are there that use or depend on FFI? Four. ffi-ncurses,
ffi-swig-generator, tidy_ffi, ffi-zlib. There’s another one for Open GL.

How many gems are there with extensions? 346

Puppet
by Andrew Shafer, works for Reductive Labs

Code, Tribes, Dawn of Time, Clouds, evolution, opportunity, and triumph

It’s difficult to automate the download ruby gems because the URL paths are
not friendly. Isn’t it interesting that RubyForge.org uses PHP to run?

Idempotence: an important property to have

He likes Debian’s setup of apache. It’s easier to work with for setting up
multiple sites than the way most Linux distributions do it.

He showed several examples of using Puppet to install packages from gem
repositories and packages.

How do you test your puppet deployment code? The best he’s come up with is to
use RSpec to make sure a machine is in the correct configuration, and run
continuous integration tests on it with your puppet code.

The Cloud. Anyone can bring up a server for 10 cents per hour, and when you’re
done, you turn it off.

Conversation with the audience:

Things get complicated when you’re using puppet to manage deployment to
different distributions or even platforms. E.g. Debian has two packages for
something that is a single package on another. The only way to make sure your
puppet deployment works is to try it and make sure the machine is in the
correct state afterward.

Does puppet help you to remove packages? Some providers purge unrequested
packages. It’s hard to back out a messed up install, and is easier to start
over with a clean slate.

Sequel: The Database Toolkit for Ruby
by Jeremy Evans

He’s not the original implementer of the Sequel module. He’s improved it quite
a bit. He added Jruby and Ruby 1.9 support, among other things.

Sequel is a domian-spefic-language (DSL) for Ruby for dealing with databases.
It’s not designed with web apps in mind, although it does work with them.

He spoke very quickly, as he was constrained for time. Sounded interesting,
but was a bit hard to follow due to the speed.

sequel_model – an ORM build on Sequel

He uses it successfully with the adaptors for Oracle, MySQL and PostgreSQL adaptors. Has a thread safe connection pool. Insures db connections aren’t leaked.

Sequel::Dataset sets it apart from other Ruby db libraries.

You can write complex SQL data queries without writing SQL, although you can if you want.

– Dataset Graphing
– Pagination Support
– Create table, Schema modification, etc.
– Master/Slave databases — requires no modifications to your code.
– Sharding/Partitioning – simple to support with Sequel module
– Bound variables /might/ be faster, but don’t depend on it.
– Prepared statements are similar to bound variables, but potentially faster.
Only use after profiling/benchmarking.

Has a command line client called ‘sequel’. The -E option causes all SQL to be
echoed to standard output.

The future: Sequel 3.0. Many features will be moving to plugins, and the
majority of the work is already complete. It’s going to be easier to use, and
require less memory.

Q: Does it support sharded database migrations?

A: No, not at this time. You can submit a patch.

Rails and Merb Merge by Yehuda Katz or The Future of Rails 3.0

Engine Yard has a new offering called [Solo](http://www.engineyard.com/solo/)

ActionORM is a way of specifying an abstraction around an object. It makes any
ORM look like ActiveRecord. He mentioned Rack integration.

Merb provides some middleware. Rails added various middleware. It’s easy to use different kinds of middleware stack pieces.

Orchestra – an idea to build a module that is a common instrumenting framework
for SQL and web frameworks.

Speed of languages: Ruby 1.9 and JRuby are very close to Perl in speed. He showed Python 2 as
faster than Ruby 1.9 and Perl. Ruby 1.8.6 is the slowest of the comparisons he
had posted. I just hope we all remember that speed benchmarks aren’t worth much.

He spoke quite a bit about community agreement and standards. Perhaps that’s a
sign of the growing maturity of Ruby.

Rack Middleware
by Jon Crosby, an employee of Engine Yard

He started by showing an example of monolithic Perl CGI, and moved quicky to Rails. Then he moved to talking about authentication and Rack. I was a bit lost as to what Rack does, or why I would care — probably because I’m not a middleware developer.

Authentication: OpenID + Oauth
Rack is the Web. The Web is Rack
Rack is based on WSGI. It’s a webserver interface. http://rack.rubyforge.org/
“Rack provides an minimal interface between webservers supporting Ruby and Ruby frameworks.”

He showed some examples of rack.

– Rack::Pool:Session
– Rack::Profiler
– Rack::MailExceptions
– Rack:CSSHTTPRequest
– Rack::Cache – highly recommended

Cooperative Middleware
Take a look at URI spaces
CloudKit – and Open Web JSON client http://getcloudkit.com/
– expose :notes, :todos
Has an OAuth plugin “filter”

“Via” HTTP headers. Used by Rack middleware

env[‘cloudkit.auth’] = 1
env[‘cloudkit.via’] << 'cloudkit.filter...' Alternative stacks - Rack::Map could let you create two apps for your same blog - a public and a private one. - Rack::Map + Sinatra 0.9.1 Question: How do you detect colissions in the URI namespace? Answer: There's no facility in Rack to do that. It's a good time to work on it before Rack hits version 1.0.

I’m attending the third annual Mountain West Ruby Conference in Salt Lake
City. Since my notes may not represent the speakers accurately, it may be
wise to read the summaries of the presentations over at the
over at the [web site](http://mtnwestrubyconf.org/2009/speakers). I’m taking notes for each of the speakers, and I’ll prefix the title of each blog post with “mwrc”.

Code Reading by James Edward Gray II

Little Big Planet: A cool PS3 game that people have created levels for. It’s interesting to see the code that shows how people create levels.

Why is code reading important?

– Shows common idioms
– Get better at working on other’s code
– Understand how something works
– Understand limitations
– Seeing bad code helps you write better code
– Always need more ideas

rest_client: Networking code done right
You can tell from the structure that it’s clean code. He showed us the ugliest parts of the code, and it looked clean.

Restful-shell: IOD shell
If you create a log file, it’s executable code that you can then edit and enhance.

He created the Towers of Hanoi for LittleBigPlanet. He had to redesign the playing interface serveral times to get it to fit into the game correctly.

### FasterCSV

Sometimes users want to work using indexes of columns. Other times, they want to work with column names. Sometimes, column names repeat.
This library was one of the first users of the new Ruby m17n libraries, and he found a lot of bugs at that time.

—

BJ, Slave, Terminator: Ruby libraries to build multiprocessing UNIX software. Built to be very robust.

– Terminator is better than Ruby’s timeout handlers. “plot_to_kill pid”
– BJ – robust background priority queue for Rails. Notices changes from outside world via signals.
– Slave – multiprocessing with built in IPC. Heartbeat.

### The Art of Code Reading

– Take a deep breath and relax
– Not all code sucks
– Don’t start with Rails. There’s a ton of great stuff in there, but it’s really complex and hard to follow.
– Have a goal. “How does it do that?”
– gem unpack
– Start with the tests
– Check for “examples”
– Try irb: irb -r