Best of Breed, or Best of Mediocrity?

Having worked for some time as a software engineer in the enterprise security software world, I know that customers (enterprises) look for "best of breed" software. For a large company customer, this usually means that a software solution distinguishes itself in some way that makes it work well in their environment. Often, this translates to reliability, cross-platform support, person-to-person support and the ability to function beyond what is advertised.

As many are aware, there is "consolidation" going on in the security market. Big fish are swallowing smaller fish, and it's lucrative, in the short term, for everyone except customers. Supposedly, the consolidation means that two separate products can be "integrated", or unified. Never mind the previous competitive relationship that may have existed between the product teams and their management. For some reason, people seem to think that competition evaporates and that the two product teams will happily work together to build the next generation "Best of Breed" software solution.

Not so.

In any big corporation or software company, there are constant power plays being made. You could call this "decision making", and if you have uncommonly good leaders, you might even say good decisions are being made. Unfortunately, it is human nature for most people to misuse and abuse positions of power. Instead of making product decisions that are best for their merged customer base, they make decisions that keep themselves in a position of power.

So, we have two best of breed products: Overdog and Underdog. Underdog is easier to manage, but isn't as complete in its offerings. Overdog is more complete, but is more expensive to deploy and manage. Overdog has the advantage of being used in Fortune 500 companies. Underdog, on the other hand, is trying to break into that market space.

Enter Big Fish -- a.k.a. Consolidator. Consolidator buys Overdog, and a few years later, buys Underdog. We take two products, both "Best of Breed" in different ways, and expect to see them merged together to make something "next generation" -- better, faster, stronger, and easier to use.

Whenever there is a consolidation, talented people get fired, and their creative ideas and abilities are lost. Product integration never happens as easily as anyone would like to believe (if it happens at all). And in the end, customers end up with a product that we can best label as "Best of Mediocrity". Consolidation means that customers lose their "Best of Breed" solutions.

What can you expect from Software Consolidators? Mediocre solutions. Look elsewhere for excellence.

The purpose of security…

A coworker made these assertions about security. I think they're worth repeating:

  • The purpose of security is to establish accountability of an individual.
  • The purpose of auditing is to verify the trust that has been placed in an individual.

Principles of Reputation

One of my past professors, Phil Windley, posted some principles of reputation:

  • Reputation is one of the factors upon which trust is based
  • Reputation is someone else's story about me - this means that I can't control what you say about me although I may be able to affect the factors you based your story on. Also, every person should be able to have their own story about me.
  • Reputation exists in the context of community - this is different than saying "communities have a reputation about someone."
  • Reputation is based on identity - reputation, as someone else's story, isn't part of your identity, but is based on an identity or set of identities.
  • Reputation is a currency - while you can't change it, reputation can be used as a resource. Paul Resnick has a paper showing the value of a positive eBay reputation.
  • Reputation is narrative - you have to apply metaphor to interpret, reputation is dynamic becase the factors that affect it are always changing, reputation may require weaving together of plot lines.
  • Reputation is based on claims (verified or not), transactions, ratings, and endorsements. - this brings up the issue of evidence, recourse for slander or mistakes, etc.
  • Reputation is muti-level - a reputation isn't just based on facts, but is also based on other's beliefs about the target of that reputation. This requires some way of signaling beliefs to others.
  • Mutiple people holding the same opinion increases the weight o that opinion - repeat behavior is also another way of weighting reputation.

Is Data Mining Fools Gold?

Here's a thought provoking article about the problems of large-scale data mining by governments. It's written by a person living in the UK.

"Data-mining is complicated, and the more data you are mining, the more false positives your software will throw up. If you act upon a false positive for a motoring offence, it's an inconvenience for the motorist, but for an alleged case of child abuse, it can rip the family apart and ruin the child's life."

"Furthermore, gathering large amounts of data is inherently dangerous. Whatever information governments find interesting will also draw the attention of criminals. Databases can be hard to keep secure, and it's not necessarily hackers that we should be worried about, but unauthorised access by employees of the agencies that use these databases. Equally, the more data you have, the more difficult it is to maintain accuracy. In 2000, an audit of the Police National Computer found that 86% of records contained errors, 85% of those errors were serious, and some were libellous."

"Technology can be a very powerful tool, but what it can't do is replace real human beings or traditional investigative work. Designed badly or used poorly, databases are the technological equivalent of fools gold."

Article: Crash-only software

Crash-only software: More than meets the eye
by Valerie Henson July 12, 2006:

Properly implemented, crash-only software produces higher quality, more reliable code; poorly understood it results in lazy programming. Probably the most common misconception is the idea that writing crash-only software is that it allows you to take shortcuts when writing and designing your code. Wake up, Sleeping Beauty, there ain't no such thing as a free lunch. But you can get a more reliable, easier to debug system if you rigorously apply the principles of crash-only design.