mwrc: Sequel: The Database Toolkit for Ruby by Jeremy Evans

Sequel: The Database Toolkit for Ruby by Jeremy Evans

He's not the original implementer of the Sequel module. He's improved it quite a bit. He added Jruby and Ruby 1.9 support, among other things.

Sequel is a domian-spefic-language (DSL) for Ruby for dealing with databases. It's not designed with web apps in mind, although it does work with them.

He spoke very quickly, as he was constrained for time. Sounded interesting, but was a bit hard to follow due to the speed.

sequel_model - an ORM build on Sequel

He uses it successfully with the adaptors for Oracle, MySQL and PostgreSQL adaptors. Has a thread safe connection pool. Insures db connections aren't leaked.

Sequel::Dataset sets it apart from other Ruby db libraries.

You can write complex SQL data queries without writing SQL, although you can if you want.

  • Dataset Graphing
  • Pagination Support
  • Create table, Schema modification, etc.
  • Master/Slave databases -- requires no modifications to your code.
  • Sharding/Partitioning - simple to support with Sequel module
  • Bound variables /might/ be faster, but don't depend on it.
  • Prepared statements are similar to bound variables, but potentially faster. Only use after profiling/benchmarking.

Has a command line client called 'sequel'. The -E option causes all SQL to be echoed to standard output.

The future: Sequel 3.0. Many features will be moving to plugins, and the majority of the work is already complete. It's going to be easier to use, and require less memory.

Q: Does it support sharded database migrations?

A: No, not at this time. You can submit a patch.

mwrc: Rails and Merb Merge by Yehuda Katz

Rails and Merb Merge by Yehuda Katz or The Future of Rails 3.0

Engine Yard has a new offering called Solo

ActionORM is a way of specifying an abstraction around an object. It makes any ORM look like ActiveRecord. He mentioned Rack integration.

Merb provides some middleware. Rails added various middleware. It's easy to use different kinds of middleware stack pieces.

Orchestra - an idea to build a module that is a common instrumenting framework for SQL and web frameworks.

Speed of languages: Ruby 1.9 and JRuby are very close to Perl in speed. He showed Python 2 as faster than Ruby 1.9 and Perl. Ruby 1.8.6 is the slowest of the comparisons he had posted. I just hope we all remember that speed benchmarks aren't worth much.

He spoke quite a bit about community agreement and standards. Perhaps that's a sign of the growing maturity of Ruby.

mwrc: Rack Middleware by Jon Crosby

Rack Middleware by Jon Crosby, an employee of Engine Yard

He started by showing an example of monolithic Perl CGI, and moved quicky to Rails. Then he moved to talking about authentication and Rack. I was a bit lost as to what Rack does, or why I would care -- probably because I'm not a middleware developer.

Authentication: OpenID + Oauth Rack is the Web. The Web is Rack Rack is based on WSGI. It's a webserver interface. http://rack.rubyforge.org/ "Rack provides an minimal interface between webservers supporting Ruby and Ruby frameworks."

He showed some examples of rack.

  • Rack::Pool:Session
  • Rack::Profiler
  • Rack::MailExceptions
  • Rack:CSSHTTPRequest
  • Rack::Cache - highly recommended

Cooperative Middleware Take a look at URI spaces CloudKit - and Open Web JSON client http://getcloudkit.com/ - expose :notes, :todos Has an OAuth plugin "filter"

"Via" HTTP headers. Used by Rack middleware

env['cloudkit.auth'] = 1
env['cloudkit.via'] << 'cloudkit.filter...'

Alternative stacks

  • Rack::Map could let you create two apps for your same blog - a public and a private one.
  • Rack::Map + Sinatra 0.9.1

Question: How do you detect colissions in the URI namespace?

Answer: There's no facility in Rack to do that. It's a good time to work on it before Rack hits version 1.0.

mwrc: Code Reading by James Edward Gray II

I'm attending the third annual Mountain West Ruby Conference in Salt Lake City. Since my notes may not represent the speakers accurately, it may be wise to read the summaries of the presentations over at the over at the web site. I'm taking notes for each of the speakers, and I'll prefix the title of each blog post with "mwrc".

Code Reading by James Edward Gray II

Little Big Planet: A cool PS3 game that people have created levels for. It's interesting to see the code that shows how people create levels.

Why is code reading important?

  • Shows common idioms
  • Get better at working on other's code
  • Understand how something works
    • Understand limitations
  • Seeing bad code helps you write better code
  • Always need more ideas

rest_client: Networking code done right You can tell from the structure that it's clean code. He showed us the ugliest parts of the code, and it looked clean.

Restful-shell: IOD shell If you create a log file, it's executable code that you can then edit and enhance.

He created the Towers of Hanoi for LittleBigPlanet. He had to redesign the playing interface serveral times to get it to fit into the game correctly.

FasterCSV

Sometimes users want to work using indexes of columns. Other times, they want to work with column names. Sometimes, column names repeat. This library was one of the first users of the new Ruby m17n libraries, and he found a lot of bugs at that time.


BJ, Slave, Terminator: Ruby libraries to build multiprocessing UNIX software. Built to be very robust.

  • Terminator is better than Ruby's timeout handlers. "plottokill pid"
  • BJ - robust background priority queue for Rails. Notices changes from outside world via signals.
  • Slave - multiprocessing with built in IPC. Heartbeat.

The Art of Code Reading

  • Take a deep breath and relax
  • Not all code sucks
  • Don't start with Rails. There's a ton of great stuff in there, but it's really complex and hard to follow.
  • Have a goal. "How does it do that?"
    • gem unpack
  • Start with the tests
  • Check for "examples"
  • Try irb: irb -r

jvisualvm: A free Java memory and CPU profiler

I needed to profile a Java application, and since we had a JProfiler floating license, I used it. JProfiler works well, although it's pricey. I was googling for other Java profiling tools, and stackoverflow.com made mention of jvisualvm, which comes bundled with JDK 6 release 7. I noticed that on my Fedora 10 box, the java-1.6.0-openjdk package includes jvisualvm. None of my coworkers had heard of it.

JProfiler introduces a significant performance penalty into the code it profiles, whereas other tools including jvisualvm and YourKit have a much lower impact. I'm going to give jvisualvm a try, once I get the target environment set up properly with the new JDK.

UPDATE: jvisualvm won't profile remote applications like JProfiler can. jvisualvm is not quite as easy to use, and I haven't figured out how to get stack traces on the CPU and memory hot spots. Overall, I like the tool.

UPDATE 2: jvisualvm can be configured to give a stack trace of memory hot spots. I've learned that performance between the Java 1.5 and 1.6 jvms can be very different. I've learned that I can run 'kill -3 ' to print a stack trace of my running java processes. It's helped me to narrow down bottlenecks in an application when the profiler wasn't granular enough.

The future of Gnome Apps: JavaScript?

There's an interesting article called "Building desktop Linux applications with JavaScript" By Ryan Paul, January 19, 2009.

I didn't immediately understand the vision. Don't we already have Python, Ruby, Java, C++ and Perl bindings for Gnome? Yes, we do. So why would we add JavaScript to the mix? Or any other scripting language?

The best way to think about it is Firefox plugins, like Greasemonkey, that actually modify the web browser to give you a new experience. Firefox extensions are written in JavaScript. JavaScript has hooks into the application (Firefox) to manipulate it.

Gnome hackers want to do the same thing for Gnome. Not only could you write Gnome application in JavaScript, you could extend a Gnome application using JavaScript, no matter what language it was written in.

Another way to think about it is this: When most people think of Java, they don't think of the language. They think of the platform -- the libraries that are shipped with the language (networking, database connectivity, etc.). The same is true for Python, Perl, and Ruby.

The goal is to us an embeddable language to tweak the Gnome platform, not to use a platform (like Java, Python or Perl) to tweak Gnome. When they embed a language into Gnome, application developers will use the Gnome platform way of doing networking, instead of doing it the Java library way. They will use the Gnome way of opening file picker, not the Java library way. They will use the Gnome way of doing HTTP, not the Python or the Java or the Perl way.

Using the 2.6.26 Linux Kernel Debugger (KGDB) with VMware

Reading the linux kernel documentation on KGDB wasn’t enough for me to be able to use the newly built-in KGDB kernel debugger with version 2.6.26 or 2.6.27. The breakthrough for me was reading part of Jason Wessel’s guide.

I have two machines:

  • developer – where I run gdb
  • target – where the kernel is being debugged, running in VMware

Configure VMware on the developer machine

  • Power down the guest (target)
  • Edit the VM guest settings
  • Add a serial port
    • Use named pipe /tmp/com_1 (it’s really a UNIX domain socket)
    • Configure it to “Yield CPU on poll” (under Advanced)
  • Install ‘socat’, if not already installed

Configure and Compile the kernel on the developer or the target machine

  • Get kernel 2.6.26 or newer
  • make menuconfig # or make gconfig
  • Under Kernel Hacking:
    • enable KGDB
    • enable the Magic SysRq key
    • enable “Compile the kernel with debug info”
  • Build kernel: make

Configure target

  • Enable Magic SysRq key on target:
    • Edit /etc/sysctl.conf and set kernel.sysrq = 1
    • or run sysctl -w kernel.sysrq=1 # this doesn’t survive a reboot
  • Install developer kernel
    • On the developer machine: rsync -av --exclude .git ./ root@target.host.name:/mnt/work/linux-2.6.26
    • On the target, a RedHat based system: make install make modules_install
  • Edit /boot/grub/grub.conf and set timeout=15
  • Boot into the newly installed kernel

Start debugging

  • On target: echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
  • On developer: socat -d -d /tmp/com_1 PTY: # notice what pty is allocated — /dev/pts/1 in my case gdb vmlinux set remotebaud 115200 target remote /dev/pts/1
  • On target, do one of the following:
    • echo "g" > /proc/sysrq-trigger
    • Type ALT-SysRq-G
  • Ready, get set, go! Go back to developer machine and use gdb to set breakpoints, continue, etc.

I set up debugging because I wanted to understand the behavior of the kernel when loading a module. It turns out that loading of the module failed because sitting in a debugger delayed the execution, causing a timeout in module load by the time I stepped through the code. Use of printk turned out to work better.

Web App Security Statistics

Perhaps this is a bit old, but it’s the first time I’ve seen it, and I thought it was interesting enough to share.

http://www.webappsec.org/projects/statistics/

  • more than 7% of analyzed sites can be compromised automatically
  • Detailed manual and automated assessment using white and black box methods shows that probability to detect high severity vulnerability reaches 96.85%.
  • The most prevalent vulnerabilities are Cross-Site Scripting, Information Leakage, SQL Injection and Predictable Resource Location