Why your project should have a Getting Started guide.

My new team at work is writing a bunch of Rails applications. This is one of those codebases that one would call "legacy" without much argument. Most of these apps have their own patched, vendorized Rails versions.

Getting up and running was an absolute pain. This project existed before Bundler and the list of gem dependencies are not checked in. I got the output of running gem list on a colleague's box, wrote a Ruby script to generate a shell script that installs all the gems. When I tried running the tests in one of those apps, I got a nice error.

It looked obvious that we were using patched Rails versions. Surprisingly, theses were not checked in to the vendor/ directory. This was proving to be a pain, and today I sat with a team member and wrote a developer guide to get started on the project. We actually had to pull patched Rails tar balls from a remote box and untar them to vendor/ directory. I was really surprised that these patches had not been checked in. We checked in all the patches. Apart from the Rails patches, there was a gem that had to be checked in.

This is why I think every project needs a Getting Started guide:
1. As a developer joining a new team, I want to look at the code as soon as I can. For me, this involves reading and running the tests.
2. Due to various reasons, a project may have patches and dependencies. Being very specific to the project, there is no way a new developer on the team would figure out these hidden dependencies.
3. There needs to be a single place where all the dependencies are specified.
4. While spoon-feeding someone may not be the best way to get them started, a little bit of hand-holding will not do anyone any harm.

One of the biggest problems I see with things like these guides is that as the application and it's dependencies evolve, the guides are not updated to reflect these changes. The only possible time the guide will be updated is when a new developer joins the team, follows these instructions and finds the need to update the guide.

Debugging: C#'s HttpWebRequest, 100-Continue and nginx

Recently I spent some time debugging an issue our team was facing around some C# code making a request on one of our servers. The request was throwing a "The server committed a protocol violation. Section=ResponseStatusLine" error.

Initial investigation suggested that this could happen if we are making HTTP/1.1 requests to a server configured for HTTP/1.0. Our Rails application runs on Mongrel fronted with nginx 0.6.5. We modified the C# code to use HTTP/1.0 and the error went away. The following line does the trick.

request.ProtocolVersion = HttpVersion.Version10;

But wait! This means that somewhere in the chain, a server is configured to use HTTP/1.0. It looked unlikely and further debugging revealed that it was indeed not the case. Further staring at the Rails logs showed that one of the headers that the app expects was not being set, when the request was done using HTTP/1.1 from the code.

After some time, we figured out[1] that the .Net library throws the "server committed ..." error if it is expecting the HTTP 100 (Continue) response in the wrong way. We set the code to not expect the HTTP 100 response from the server using

      request.ServicePoint.Expect100Continue = false;

and voila, it worked. The Rails app received all the headers it expected and things worked fine. The code looked like this:

So what is happening?

The HTTP 100 status is supposed to work like this. When a client has to send some data, instead of sending it upfront, it can send some headers along with the "Expect:100-Continue" header. The server responds with a 100 if it is willing to accept the request or send a final status. The spec is here[2].

We are using nginx as a proxy. The specification says that the proxy should forward the request if it knows that the next-hop server is HTTP/1.1 compliant. The proxy is supposed to ignore the "Expect:100-Continue" header, if the request came from a client using HTTP/1.0.

In our case, the default behavior of the .Net HTTP client library is to set "Expect:100-Continue" header on every request for HTTP/1.1. So the client sends only some headers and waits for the 100 response from nginx. Nginx sees the request, knows that Mongrel supports HTTP/1.1 and just forwards the request. The app sends a 401 because it could not authenticate. The client is expecting a 100 and gets a 401. It thinks the server committed a protocol violation.

When we ask the client to use HTTP/1.0, the .Net library does not use the Expect header, sends all the headers and nginx forwards the request to Mongrel. The authentication goes through.

When we explicitly set the Expect 100 property of the library to false, it sends all the headers at once and the authentication goes through fine.

Looks like there is a way to tell .Net not to expect 100 from the server through configuration, by putting this in .exe.conf

1. http://stackoverflow.com/questions/2482715/the-server-committed-a-protocol-violation-section-responsestatusline-error

2. http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3

Talk on Football and Politics

I recently did a talk on football and the politics behind it, as part of the Banyan Tree talk series at ThoughtWorks. The slides from the talk can be found here on Speakerdeck.

It was great fun for me. I thought I did well. It is a topic I know my way around. The history behind the El Clasico - the Spanish civil war, Franco, the trade unions and Di Stefano was discussed. The Old Firm was dissected. I also talked about Athletic Bilbao's use of football as a form of cultural resistance was looked at. The last parts of the talk were about multiculturalism, united Europe and football as a globalized game.

 About 20 people turned up and while most of them liked it, some had the opinion that they needed to understand more European history to fully appreciate it.

7L7W - Ruby Days 1 and 2

I joined a 7 langauges in 7 weeks study group started by jocranford and others. The group started on 24th October, but I joined only on yesterday. The language for the first week is Ruby. I have done a fair bit of Ruby and consider myself fairly familiar with Ruby.

I have managed to complete days 1 and 2 of Ruby. The book encourages one to Google, read the API and figure out stuff yourself. I have found this approach really nice because following this has enabled me to learn some stuff I did not know about Ruby.

I have put the code for Day 1 and Day 2 on Github.

Stuff I learned from Day 1 and Day2 (Mostly Day 2):

• Hash sorting is funny. Hashes get converted to Arrays before being sorted. There are other ways to do stuff.
• Ruby does not have named params. I realized that I never noticed this because I used hashes.
• A symbol always have the same object id and symbols are not garbage collected.
• Enumerable#inject and Enumerable#reduce are the same. And there is no need to pass an initial value.
• File.owned? - Pretty useful API.

This looks like fun.