After some time off I return to our XP simulation where I attempt to implement a little messaging application. For the iteration ahead there is only one story planned: “A user can send a text message to another user - the console version.” The story is estimated at 5 points. We have a good idea of what this story entails as we have an acceptance test that describes the details.

We begin by planning the iteration by breaking down the story that is up on the white board into tasks for the developers. We estimate those tasks as best we can. When estimating the tasks we don’t use an abstract notion like story points; a simple time estimate will do. None of the tasks should take a lot of time to do, it should be a matter of hours or, at maximum a day.

When we created the user stories we did our best to ignore dependencies. When we plan the tasks for the iteration, we allow ourselves to acknowledge that there is an order to how certain things ought to be implemented. It does make sense to make two applications connect to each other before we attack sending messages between them.

This is the first iteration so we don’t have a good idea of how much we can accomplish in one iteration. My gut feeling tells me that we probably could squeeze more into one iteration, but let’s wait with that until we have marked some tasks as “done” and have a better grip on our velocity. History has proven me to sometimes be too optimistic.

Here’s the project white board after we’ve done the iteration plan:

The next thing to attack: connecting two applications.

While working on the IM application, a pattern seems to be emerging for the tests that I write. I have an object that is supposed to wait for something to happen, and when that something occurs that object is expected to notify another object by making a call to it. For instance, I have a class that waits for something to be sent over a socket, and signal a client when it has received what it needs. Waiting for something to happen on a socket means that the current thread cannot do anything else while waiting - that operation needs to be done on a worker thread.

When I write a test for that type of operation it follows a pattern similar to:

1. Create a test double that we expect to be notified by the class under test.
2. Call the CUT and pass it the test double
3. The CUT starts a worker thread, which waits for the event to occur and calls back when it does
4. The call to the CUT returns to the test code before the callback to the test double has been made
5. The test code does whatever it needs to make the CUT notice that it needs to notify the test double
6. The test code waits for the test double to be called
7. The CUT makes the call to the test double that in turn signals to the test code that the operation has finished
8. The test code makes whatever assertions it need to make sure that the test has passed

I ended up creating a class, SignallingWrapper that wraps all of this behavior. Using it for a test like the one above looks something like this

def test_with_threads
  testDouble = create_test_double

  wrapper = SignallingWrapper.new(testDouble, ["method_on_test_double"])
  wrapper.call_in_context do
    object_under_test.method_to_test(wrapper)
    wrapper.wait_for("method_on_test_double")
  end

  assert(testDouble.didItWork?)
end

In the code above we create a SignallingWrapper around the test double, passing along the name of the method that we expect the CUT to call back. We then call the CUT inside a test context, which makes sure that thread synchronization works properly. After that we wait for the method on the test double to be called properly. Finally, we assert whatever is necessary to complete the test.

You can specify, and wait for, more than one method on the test double.

The SignallingWrapper seems to do the trick for me. It may very well be the case that I really should refactor the tests so that threading is not an issue, but I don’t see a clean way to do that at the moment. In any way I had a blast implementing this. It introduced me to concepts like object-specific classes and dynamically defining methods. Ruby contains a lot of stuff. Perhaps too much…

You can take a look at SignallingWrapper here.

After the first iteration I have learned quite a few things about Ruby basics and the support for what I need to get started developing the application. Looking at the project whiteboard we now see that the next thing to do is to tackle the first user story - “A user can send a text message to another user.”

That little piece of information is not enough to start implementing. We need to know a lot more about what the customer wants and how we are going to support that feature. The most common way of describing how a feature is to be implemented is probably by creating a use case.

A use case describes the flow of the system while it solves something for a user. When I first encountered user stories my immediate reaction was “what is the difference between a user story and a use case?” My take on it is that while a user story is a promissory note, a token that tells us that we are going to get to the details of a feature when it is about time for it in the plan, a use case is a way to actually document those details. In addition user stories are a tool for planning a project. They have a cost attached to them. They can be split and combined. Use cases don’t have these abilities.

We also need a way to verify that a feature works properly. For this we need to specify a test that needs to be passed in order for us to consider the feature implemented. More often than not (at least in my experience) there is great similarity between a test case and a use case. With the test case being a concrete example of the use case. Where the use case specifies an actor “first time customer”, the test case specifies a user “Alice”, whose “user profile does not contain any information about a previous visit to the system under test.”

The test can tell us when we have completed the implementation of a feature. That is information that we want feedback on as early as possible. Even so, my experience is that test cases are often specified after the feature is implemented, when it is time for the dreaded system test. Often the test case is created by someone who is isolated from the rest of the team. In some organizations there seem to be a perceived advantage to have an almost antagonistic relationship between testers and developers in particular. I am not sure that I understand why.

If we detail the user story as a test before we implement it, we might remove some of the waste that it means to document the same thing in both use case format and test case format. We also have an opportunity to get immediate feedback on whether we can consider the feature we are working on finished or not. Developing against a pre-specified test case will give feedback on whether the test case makes sense as well. All of this is feedback we want long before we deploy or go into system test.

Having the possibility to run the test cases automatically by just pressing a button, running a script, or just have them run every time someone checks in files to the version control system makes it really easy to make sure that a feature still works after additional changes have been made to the system. Fit and Fitnesse enables us to specify tests in a way that is readable to non-programmers while they at the same time can be executed automatically. We will use Fitnesse to create test cases in this project.

Discussing the Details

With that in mind, the customer, the developers, and quite possibly a tester sits down and tries to flesh out the details of how our application is to enable users to send messages to each other. We begin with the main scenario. No communication errors or timeouts yet. Let’s start simple. The result is a Fitnesse test that looks like this:

This describes how two clients are started. One client is started by the user Alice, and the other by the user Bob. The users exchange some messages and we expect the message history on both clients to contain the messages along with information about who sent them. The tables in the document are interpreted as test fixtures by Fitnesse.

This test should of course be supported by other tests that take care of other scenarios. We should for instance have tests that verify the behavior when errors occur. But let’s stick with this test in this example.

In order for the test to be able to run we need to create some code that takes care of calling the system under test on behalf of our fixture tables. We implement some stub classes and methods so that we can get our test to execute:

require 'go_fixture'
require 'fit/row_fixture'

class EstablishConnection < GoFixture
  def start_client_with_user(client, user)
    return false
  end
end

class StartupResults < Fit::RowFixture
  def query
    return Array.new
  end
end

class Conversation < GoFixture
  def user_sends_message_to(fromUser, message, toUser)
    return false
  end
end

class MessageHistory < Fit::RowFixture
  def query
    return Array.new
  end
end

Running the test gives the following result

Our assignment for this iteration is basically to get these tests to pass. Ideally, these tests should execute a complete system with no fake data or classes. At the moment I haven’t decided on how to approach this though. I can either start two instances of the application and have them communicate with each other, or I can instantiate and configure the internal classes for both client Alice and client Bob in the test code and have them communicate with each other via a fake communication channel. The former approach is the most realistic and tests the actual system, while the latter is easier to test. Right now I think I am going to start with the latter and then see if I can transform it to the former, all in the name of moving forward. Any input on this is highly appreciated.

Now it’s time to start planning the iteration in more detail…

I have not been able to find a port of FitLibrary for RubyFit. I especially miss the functionality of DoFixture where you can, amongst other things, write fixture tables that contain rows like this

which I find really easy to read.

DoFixture interpret the first and every second cell after that as a name of a method to call and the rest of the cells are interpreted as arguments to that method. So the table row above would be translated into a call to the method similar to

public bool userLogsOnToServer(String user, String Server) {...}

where “Alice” would be passed as the first parameter and “AliceServer” would be passed as the second parameter.

While waiting for this to hit Ruby (if it already has, please let me know), I wrote GoFixture, which looks kind of like an ActionFixture that can call methods DoFixture style.

A table like this

can be backed up by the following class

require 'go_fixture'

class MyGoFixture < GoFixture
  def user_logs_on_to_server(user, server)
    ...
  end
end

It seems to work with RubyFit 1.1. If you want to run its unit tests you need FlexMock 0.3.2

You can download GoFixture here.

 I spent the first iteration investigating basic Ruby and the tools available for creating automated tests. I am nowhere near feeling confident in all aspects of the language or on all the intrinsics of the tools but I hopefully I have enough to get the story planned for the next iteration going.

Ruby has a unit test functionality as part of its standard library, which is nice. It is a dynamically typed language so there should be a few new ways to create test doubles other than the ways its done in languages like Java or C#. There are a few mocking frameworks (Test::Unit::Mock and Flex Mock seems to be the candidates to choose from) available but I haven’t looked at those in any detail yet.

I was glad to see that RubyFIT is available for creating acceptance tests in FIT or Fitnesse. I haven’t been able to find FitLibrary and especially DoFixture implemented for Ruby. There seem to be some work going on with FitLibrary for Ruby though. If anyone has any info on this, please let me know. I will try to get on without it for now.

So now the second iteration starts. It is time to tackle the first user story!

Wojtek Biela is documenting his progress as he helps implementing agile practices in a project that is in pretty bad shape. Will be really interesting to follow.

Last time I did a rough estimation of all user stories supplied so far. In addition the iteration length was set to one week. This time I am going to plan the work for the near future. I may even come up with a rough estimate for the final release.

Ruby

To throw some risk and uncertainty into the mix, I have decided to implement the application in Ruby. I have never worked with Ruby before, and I will address this by adding time-boxed investigations to the plan where needed. Normally, I would probably spend a lot of time up front learning a new technology or language, trying to find all the really cool features of the language so I would be able to take advantage of them from the beginning, but this time I am going to add targeted investigations throughout the development plan. These investigations will be targeted at solving a specific problem or giving enough detail to implement a user story.

The rationale for doing it this way is to see how it works to take a big unknown, or risk, and instead of addressing it as much as possible upfront, address it when needed at different times during the project. It will force me to investigate just enough for the problem at hand, and make me implement features the simplest possible way I can. So be prepared to see a lot of Ruby code with rookie mistakes!

The First Iterations

At this stage I think it is time for the customer to make some decisions about in what order the user stories should be implemented. So, putting my customer hat on, I try to think about what I want to see from the application in its first incarnations. The most important aspect of this application will be the messaging interface; how the user interacts with the system while writing and receiving messages. So as the customer I determine that I want to see the user story “A user can send messages to another user” implemented first. I want to see the GUI for exchanging messages with another user up and running so I can give some feedback on that as soon as possible.

As the developer, I suggest that we add some investigations into Ruby in the first iteration, and that it may be too much to both get the hang of basic language features and GUI support at once. I suggest that we compromise and implement the story with a simple console front end at first, and then tackle the GUI.
As the customer, I am not overly excited about not getting the first crack at the GUI at once, but I trust the developers when they say that the cost for adding a richer GUI to the mix now is too high. But I do want that GUI sooner rather than later though.

So the team decides that the first version of the application will be console based. As we won’t be implementing any central operator service with support for looking up other users in the system, the installed client will be configured to connect directly to another client. Using the metaphor, the team calls this version for the “Direct Line” version. You just lift up the receiver and have another subscriber on the line.

So it is decided that the first iteration will be spent investigating the basics of Ruby and what the options are for implementing simple network connectivity. During that iteration I’ll also look into unit testing support and acceptance testing support.

The second iteration will be spent implementing the “A user can send messages to another user” story. Looking at the estimate for that story we see that it has 5 points. This is relative to the other stories. As all of the stories involve GUI one way or the other, that relative estimate still holds. But we will need to add a user story which adds GUI. As we don’t have enough knowledge about GUI in Ruby, we cannot estimate the new story properly at the moment. We need to add an investigation to get enough info to estimate it properly. We plan to do that investigation in the iteration that follows the implementation of the “Direct Line” console version. In the meantime we give that story an estimate of 3, and expect that estimate to be revised after the investigation.

So the first iterations are planned as follows:

1.Initial Investigations
2.Direct Line – Console Version
3.Ruby GUI Investigations
4.Direct Line – GUI Version

The rest of the stories are deferred to later iterations. I think one could do a more high level release plan that spans further into the future. A plan of that sort would contain a few intermediate releases prior to the final release. I could have done that in this project as well, but for now I’ll just consider this a one release project.

First Shot at a Release Date

The team estimates its velocity to about 5 points per iteration. This project has to share its resources (me) with my family, the swedish summer, some house renovations, and the world cup. So we’re far from the 40-hour work week. The total number of points at the moment are about 45. That would suggest that we need about 9 iterations to complete all stories. This is a rather naive way of doing estimation. We’ve already used up two iterations for investigations, for instance, and there will be more. A really rough, gut feeling, estimate lands at between 10 to 15 iterations.

I expect this estimate to be constantly refined, and I will be in a much better position to estimate after a few iterations. I have deliberately not spent too much time on the overall estimation at this point. Perhaps I should have done it in more detail, but I want to see if, and how, the estimates are self regulated.

The Plan So Far

When I started out I intended to use XPlanner to keep track of my progress. Although I think this may be a great tool, especially for multi-site teams, it doesn’t seem to give me any added value in this particular project. Instead I have switched to plain old pens, notepads, and post-it notes.

This is the project “white board” as it looks at the moment:

At the top are the stories that are in the current iteration. At the bottom to the right are the stories that are scheduled for the next iteration. At the bottom in the middle are the stories in the iteration after that. At the bottom to the left are all stories that are scheduled for later iterations. Pink post-its are investigations. Yellow post-its are user stories.

Time to start looking at Ruby…

Twice a month, Kevin Rutherford puts together a digest of what’s being discussed in the agile blogosphere. This site got mentioned in the latest edition. There’s a lot of other interesting stuff there as well. Check it out!

As an excursion in XP I am currently implementing a simple instant messaging application so that I can try out the mechanics of it’s practices. As I am alone on this I will not be able to take full advantage of some of the practices (it will be hard to pair-programming for one). The system metaphor is one of the practices that I really don’t see how it will affect development for me personally, for a team I can definitely see the uses. Nonetheless I will attempt to come up with a metaphor to see if it does me any good.

A system metaphor is an analogy that describes the system in a way that it can be discussed by all parties that have an interest in the system to be developed. This is by far the concept in XP that I have most trouble grasping, and from browsing through mailing lists and talking to people, I know I am not the only one. From what I understand the metaphor is not as explicit in the second edition of Beck’s book as it was in the first, but I’m going to try it out anyway.

The first thing that comes to mind is that of a telephone network. Especially the old school types, where you just lifted the receiver and got connected to an human operator. You’d then ask the operator to connect you to whomever you wanted to talk to.

So the act of sending a message to another user may be described as follows:

“A subscriber contacts an operator and asks to be connected to another subscriber. The operator connects the call between the subscribers, who then continues their conversation directly; without involving the operator.”

After getting some feedback on this from the extreme programming group I hesitated whether I should use this metaphor at all. Does this really add anything that thinking about “a client connects to a server and…” I mean, I think of this project as a simulation of an XP project, but I am still the only project member. I don’t think I can simulate the informal communication going on in an agile team at all. So, do I need a metaphor to describe something to myself in other words than clients, servers, and user databases?

I’m going to stick with the metaphor some more and see. One thing I can see already is that the system metaphor makes the application design come to life. When I think of it in terms of clients and servers, the image that pops into my head is that of a UML-like diagram. Something like a sketch on a white board. With the metaphor I get images of people doing things like making calls, working as an operator and connecting phone calls. It’s a more vivid image.

I’ll try to be aware of when this metaphor helps me, or hinders me.

Last time the customer brought the first approach user stories to the rest of the team. By just looking at them, a few things about the design could be inferred, but the team has not created any architecture or design for the application yet. That doesn’t mean that it is not there. It’s simply not yet documented anywhere yet.

It is now time for the developers to begin estimating the user stories that are presented so far. There are several ways to do estimation. For this project we elect to assign points to each story. The number of points assigned to a user story is an indication of it’s size. A user story with 2 points is estimated to be twice as large as a user story with 1 point assigned to it.

Using the abstract notion of points has a few advantages over simply estimating the time it will to take implement a feature:

• How long something will take is dependent on who is doing it. A team’s ability to perform may change several times during the life of a project. Members come and go. Assignments outside of the project is competing for it’s resources. The size of the stories is supposedly not as susceptible to change however.
• It’s easier to estimate the relative size of one thing compared to another than to estimates its absolute size.
• Estimates, especially at the beginning of a project are just that - estimates. However, over time estimates have a way of being transformed into promises. By using the more abstract notion of points, it can be easier to distinguish between estimates of tasks and estimates of release dates.

Points have some disadvantages as well, the main one being that it is often a hard concept to describe to someone not used to them. “Traditional” estimation includes time rather directly, which is kind of intuitive. With points we only estimate the size of things and expect another concept, velocity, to help us get a time estimate. This may take some getting used to.

For a great exploration into the details of planning agile projects, read Agile Estimating and Planning by Mike Cohn.

Iteration Length

The team should be able to implement each user story within an iteration. This will ensure that the system contains a set of completed features that are of value to the customer at the end of each iteration. To be able to estimate the user stories that fit in an iteration, we need to determine our iteration length. For this project we choose an iteration length of one week. I’m doing this on my spare time, so I don’t expect my velocity to be very high. In a real world scenario the selection of an iteration length should be given more careful thought.

The Stories

Let’s review the user stories so far:

1. A user can send text messages to another user
2. A user can block another user
3. A user can create a chat room and invite other users
4. A user can use a display picture
5. A user can send a file to another user
6. A user can store conversations in a local file
7. An administrator can ban a user
8. An administrator can broadcast messages to everyone

Starting with the first story, let’s consider roughly what that would entail. We need a simple GUI where a user can enter text. We need to be able to discover other users in order to select to whom a message should be sent. We need to log on to a server. We need to be able to store information about users on a server. We need a protocol for discovery and message sending. Etc.

This story seems too big to fit inside an iteration, so we need to split it into smaller stories that can fit inside an iteration. Let’s imagine the team sat down and did just that. This is the result:

• A user can log on to the messaging server
• An administrator can add a user to the server
• A user can list the other users on a server
• A user can send a text message to another user

Now these seem a bit more manageable than the original story. Going through the rest of the stories, we estimate that they all have a decent size. In a real world scenario, we would probably find a couple of more stories that we have problems estimating. In addition to being difficult to estimate due to hiding too much functionality, as with the story earlier, stories may also be difficult to estimate because we don’t know enough about how to solve them. We might, for instance, not know enough about a certain technology. In these cases we might schedule a spike, a time-boxed investigation aimed to give us enough answers to give a good enough estimate.

After going through our stories so far, we end up with the following estimates:

Remember that these estimates only says that we think that story X is approximately twice as large as story Y, and that we believe roughly that we will fit at least one story in each iteration. The story with size 8 might prove to be too large. Or any of the stories for that matter. We may have to revise this later.

With this we should have enough information to start planning our iterations.