Into The Weeds

Yes. I want one. Right Now.

From the Chicago Tribune's review:

...a bacon and cheese sandwich where bread is replaced with fried chicken. Re-read that last sentence, and just try to grasp its cultural significance. The toothpaste is out of the proverbial tube.

...Double Down's larger implication is that KFC has broken through a barrier of culinary decency, besmirching the good name of sandwiches and all that is honorable... The absence of bread robs this “sandwich” of dignity.

All told, the ripples from this landmark product launch will be significant. Stand-up comedians will construct 10-minute riffs around this. Taco Bell will follow suit and introduce a taco called the Carne Go-Torta, where the tortilla shell is replaced with a meat sleeve. Freedom-loathing terrorists will hate us for this. I can see it now.


America, just because you can, it doesn't mean you have to.

After seeing black smoke outside the window, I snapped a couple of photos of what's going on in the Back Bay Fens now. Those flames look to be at least 50 feet high. Whatever it is, it seems mostly contained now, with some flareups. The fire department is referring to it as a "large brush fire" over their radios (I'm listening to the scanner).

Sox vs. Yankees tonight at Fenway. Here's the view outside my window right now:

So, I learned about this today. And was giddy like a schoolgirl after watching the trailer (below).

Made of awesome. Full of win. Nothing more needs to be said.

New A-Team Trailer Increases Nostalgia Factor by Roughly 300 percent

What does it look like when the Systems Engineers from around the world gather together in Second Life for our weekly meeting? I took a snapshot today (that's me in the leftmost-foreground):

People have expressed interest in seeing some more details about our use of MySQL at Linden Lab, so I shall indulge in the next couple of posts. Over the past few months, I have been involved in two major projects involving our central MySQL database cluster. The Second Life central database cluster (known affectionately as mysql.agni) actually consists of a single read/write master database server, with a tree of somewhere around 15 read-only slaves hanging off of it that are split into groups of various purposes (some behind load balancers) and serve the bulk of the queries that make Second Life work.

MySQL 5 Upgrade

Up until this year, our mysql.agni master was running version 4.1. Shortly after I started at Linden Lab, there was a project to upgrade the entire tree to version 5.0. Our resources were limited back then, and the operation was a total failure. It turns out that MySQL 5.0 has a much different disk i/o usage profile for our query rate and load type. When the master was upgraded to version 5.0, it promptly collapsed under heavy disk i/o load and unacceptable response times. Worse still, we did not have a proper version 4.1 host slaving off of the new master, so there was extended downtime and some data loss as we failed back to the older version and had to replay queries from binary logs.

Fast forward to the end of 2009, and we were much smarter, and better equipped to upgrade. By this point, all of the slaves in the tree had been upgraded, and it was becoming difficult and confusing to maintain a mixed-version slaving tree, with some on version 4 and some on version 5. We were operating on upgraded hardware with more capacity and headroom -- particularly concerning disk i/o.

My former co-worker, Charity Majors, authored an extremely thorough public blog post concerning this upgrade, and our planning to make it come off without a hitch. We built a robust load-testing and analysis rig, and put the new version and hardware through its paces multiple times, in multiple ways, to make sure that we'd be OK this time. The major difference was our consultation with Percona and our decision to use one of their high-performance tuned builds of MySQL. In addition, we came up with an awesome plan for performing the upgrade, and a fallback mechanism if we needed it.

The trouble with fallback in this case is that normally, it's not possible to hook up a MySQL version 4.1 slave off of a 5.0 master. This means that once we upgraded to the new master and had writes and updates going to it, we'd have no version 4.1 server to fall back to without losing data. Charity was able to rig up a script that rotated the binary logs on the new 5.0 master every 5 minutes or so, and sent the contents of the log over the network to a fallback host, where we would feed the binary log queries (via mysqlbinlog piped into mysql and various filters to compensate for 4.1/5.0 incompatibilities) into the database and get it up to date and ready to fail over to if we had issues with version 5.0. Of course, performance wasn't our only concern, and there were millions of lines of code just waiting to bite us in the butt and potentially require falling back.

On January 5th, 2010, all went as well as could have possibly been imagined. We cut over to the version 5.0 master, and there were no issues. It's so nice to finally be on a relatively recent version of the software, and more importantly, to have our master and slaves all be on the same version. And so our march onwards continues towards ever-improved performance and stability!

Richard Michael Raven

Age 62, of Shorewood, passed away peacefully Friday, March 5, 2010 at University of Chicago Medical Center.

Born August 23, 1947 in Chicago, he enlisted in the U.S. Air Force and served during the Vietnam War. He worked as a Project Manager and Communications Instructor for Ameritech.

His family was his life.

Surviving are his wife, Kathy A. (nee Kupka) Raven; his children, Richard James (Jill) Raven, Cherie (Brian Chapin) Raven, and Kristin Ellen (Ben O'Connor) Raven; four grandchildren, Brook Elizabeth and Richard John Raven, and Audrey Katherine and Edward Michael Chapin; one brother, Daniel Joseph Raven; and several nieces and nephews.

Preceded by his parents, Stanley and Dorothy (nee Coakley) Raven; and three siblings, Raymond, Robert, and Dorothy Raven.

Funeral services for Richard M. Raven will be Wednesday, March 10, 2010 at 11:00 a.m. at the funeral home chapel. Internment will be in Resurrection Cemetery, Romeoville. In lieu of flowers, memorials to University of Chicago Cancer Research Center, 5841 S. Maryland Ave MC 1440, Chicago, IL 60637 would be appreciated. Visitation Tuesday, from 3-8 p.m. at:

Fred C. Dames Funeral Home 3200 Black at Essington Rds., Joliet (815) 741-5500 www.fredcdames.com

Randall Munroe of XKCD hits the nail on the head with this one:

"The weird sense of duty really good sysadmins have can border on the sociopathic, but it's nice to know that it stands between the forces of darkness and your cat blog's servers."

You like peanut butter? You like bacon? Why not put them together! I've been making peanut butter and bacon sandwiches for a while now, so here is my technique -- a special gift for you and whomever you wish to spread the knowledge to.

Ingredients:

• ~3 strips bacon (I use precooked for convenience)
• 2 slices of bread
• Some peanut butter

Lightly toast two slices of bread. Spread peanut butter on the toasted bread. I like to put three half-slices of cold, pre-cooked bacon on each slice and then put it back in the toaster oven for a few minutes. If you're using regular bacon, or don't have a toaster oven, I suppose you could cook up the bacon in a skillet. Put the two pieces of the sandwich together, let it cool down for a bit, and then enjoy. Yum-tastic!