Here is the talk outline:

Programming Java applications is often viewed as less than pleasurable. This perception is exacerbated when enterprise Java frameworks are stacked up next to all the wonderful new web frameworks and methodologies available to developers today. Grails, a Java-based, agile web development MVC framework, may change all that.

In this talk, learn from a one-time skeptic how your team can use Grails to speed up project delivery, streamline maintenance and support, increase functionality and performance, allowing you to spend more time focused on business objectives in your next web project.

If this sounds like something that might interest you, register here!

I first became interested in the open data dialogue back in early 2009 at ChangeCamp in Toronto, which was an event aimed at re-imagining government and citizenship in the age of participation. ChangeCamp spawned ChangeEngine, a project where a group of citizens (including myself), imagined a geo-aware issue tracking system, aimed at bringing communities together to solve problems and to provoke systemic change.

As the launch of Toronto’s open data site approached, some of my colleagues at Architech Solutions and I, met with Mark Kuznicki (organizer of ChangeCamp) to discuss the possibility of building a community companion site to the city’s site. We all felt there was a need, and so DataTO was born.

The Monday following our Friday lunch marked one week to go until the city’s launch date and Mark and I met for our first planning session at the Centre for Social Innovation in Toronto. At this meeting we focused on nailing down our user roles, scoped out the feature set for the initial release and roughed out some layout concepts.

Having had a full-day meeting crop up for later in the week, I was now down to 3 days of build time. Time to get going. Given the time constraint, I decided that building the app with PHP/CodeIgniter simply wasn’t feasible, so I decided to go with Grails, a Java-powered web application framework, which I had been tinkering with for a couple weeks at the time. This decision was not taken lightly, as I would not classify myself as a Java developer, but the fact that Grails utilizes Groovy, an agile dynamic programming language for the Java platform, and that Grails is so well suited for agile development, I felt that it was the way to go.

For those of you who are not familiar with Grails, think of it as the Java equivalent to Ruby on Rails. I won’t get into Grails vs. Rails in this post, so suffice it to say they are comparable, each with pros and cons. While Grails is relatively unknown at the moment, I suspect it will gain in popularity very quickly due to it’s power, flexibility and ease of use.

Having decided on a framework, I quickly spec’ed out the app, and started the build. Since Grails makes it so easy to build object models and generate scaffolding, I had more or less completed a working wireframe by the end of the first build day. The biggest benefit to having such an easy build process is that more time is left for UI, which is clearly very important to me.

In fact, the second build day was spent largely on UI. It was clear to me at this point that some of the features we had planned on being in the initial release were not going to be ready in time. So I decided to pare back, and focus on making a select few features as solid as possible, rather than diluting my efforts, and producing a greater number of mediocre components.

Day 3 of the build came and went very quickly, and was focused on skinning the application and debugging. The week had come and gone, and one daunting task remained — deployment.

Not being a Java developer, I wasn’t familiar with the various containers, but decided to go with Tomcat based on reputation and available documentation. I installed Tomcat 6 on a MediaTemple Dedicated Virtual server behind Apache and deployed the application the night before launch. This is somewhat of a “budget” setup, but it has held up well for this proof of concept. Ideally, I’d like to migrate to Amazon AWS for the infrastructure, but that will require funding (hint, hint).

Since launch, the site has received thousands of visitors from all over the world. It currently has over 200 members, and is approaching 100 requests for datasets. It has been mentioned in numerous major publications, and was profiled on The Discovery Channel’s Daily Planet.

I was extremely reluctant to put the site out there in it’s incomplete state, as I’m somewhat of a perfectionist, and I was worried that it might be a poor reflection of what I do, but I really couldn’t be more pleased. I’ve really enjoyed this as a learning opportunity, and a chance to do something good for the community. The site still has a long way to go, but I am continuing to work on it and can’t wait to see where it goes.

To start validating zip/postal code fields, include this code somewhere after you’ve included the plugin:

jQuery.validator.addMethod("postalcode", function(postalcode, element) {
	return this.optional(element) || postalcode.match(/(^\d{5}(-\d{4})?$)|(^[ABCEGHJKLMNPRSTVXYabceghjklmnpstvxy]{1}\d{1}[A-Za-z]{1} ?\d{1}[A-Za-z]{1}\d{1})$/);
}, "Please specify a valid postal/zip code");

Now you can add postalcode to your validation rules like so:

$("#myform").validate({
	rules: {
		postalcode: {
			postalcode: true
		}
	}
});

Note: this will accept both Canadian postal codes (with or without a space) and U.S. zip codes. Enjoy!

To download and get more information, check out the user library project page.

Let me know what you think about Facebook Connect, and its many appications.

The life of a developer is fraught with repetition. Minimizing said repetition makes me very happy. The other day I noticed that whenever I start working on a project I start with the following routine:

1. Open Terminal
2. cd to the working folder for the project so I can pull the latest files from the Git repository
3. Open the working files in TextMate.

After these steps, the process diverges, but those first three steps are constant. I keep my project files well organized so I have a hope in hell of remembering where things are, so even changing directories to my working files follows a pattern (cd projects/projectname.com/working).

Having followed these steps half a billion times, I decided it was time to streamline, so I wrote this very basic little script:

#!/bin/sh

mate ~/projects/$1/working

osascript<<END
try
    tell application "Terminal"
        do script with command "cd ~/projects/$1/working;echo \"You're now working on $1\""
    end tell
end try
END

By typing that code into blank text file, saving it as proj, chmodding it to 755, and moving it to /usr/bin I got a little shortcut for starting projects. Now I just open terminal, and type “proj projectname.com”, and I get a terminal window cd’ed to the working files, and TextMate opened up to my project files.

Of course, this is a very simplistic implementation, I may add additional steps, such as automatically pulling the latest files from the repository, or opening up the log of most recent changes. Anything is possible. While this particular script may not be useful to you, I urge you to examine the tasks you perform regularly and experiment with what can be automated to save you time. Simple scripts like these, or those generated by Apple’s Automator are easy to make and save tons of time in the long run.

ASCII Characters to HTML Entities Translator

Each and every one of us estimates poorly in our own unique way. Some of us over-estimate, some of us under-estimate, and some of us couldn’t hit the broad side of a barn. However, we usually follow a pattern of some kind. The degree to which our estimates are off generally conforms to roughly the same proportions. If we were to examine a particular worker’s estimated task completion time to the actual completion time over the course of many tasks, we can determine the approximate factor to apply to his estimates to find the actual completion time.

The root cause of this predictability is the simple fact that humans are better at being precise than we are at being accurate when it comes to estimates. In other words, we’re quite good at approximating the relative size of a task, but we fail at determining the appropriate scale.

Take the chart below. Here we see an intentionally simplistic view of a worker’s estimated completion time versus their actual completion time.

At a glance, our worker appears to be a poor estimator, as he has underestimated each task. However, if we look at the data a little closer, we can see that on average, he was of by a factor of approximately 1.8. If we were to apply this factor to his estimates in advance we’d see a slightly different picture.

As you can see, by applying his average estimation scale factor to the figures we can calculate a much more accurate set of estimates. Of course, in practice, five data points will not provide a very accurate estimation scale factor, but by collecting this data over the course of several projects we can get a pretty good sense of this worker’s estimating characteristics.

Tracking this data manually would be cumbersome to say the least, but it’s something that a computer does very well. Given the proper tool, this data could be collected behind the scenes with little to no impact on workflow.

It just so happens that I am in the midst of building such a tool. For now, let me say that this will be a web-based, hosted, project management tool. I will be applying my many rants on the subject to creating not only a better tool, but a better process for project management. This will not be another to-do list manager, I promise you that.

Now that the cat is out of the bag, I will be posting regularly throughout the development of this tool and I will be craving your feedback. Many, many more details to come. Stay tuned.

At some point in human history, estimates evolved into something more than what they’re intended to be. Perhaps we have shady contractors, auto garages, and wedding planners to thank for the modern perception that billing a client more than the estimated value means someone has been mislead, or taken advantage of. Or, perhaps it’s the fact that generating an accurate estimate is genuinely, extremely difficult — especially given that many estimates are expected to be delivered while there are still many unsolved variables on the slate. Somewhere along the way, we’ve lost sight of the fact that an estimate is really just your best guess, based on a combination of the information available at the time and your experience.

Setting expectations is a crucial aspect of delivering your estimate. Your client must be reminded that an estimate is your best guess and not a draft copy of the final invoice.

Now, setting expectations makes the discrepencies between the estimate and the final invoice easier to swallow, but it doesn’t make you a better estimator. Traditional estimates often focus on deliverables. Deliverable A will be X dollars, and Deliverable B will be Y dollars. This is a format is easily understood by the client, but makes the poor soul creating the estimate sweat. Deciding on a time or dollar value based on a mental image of a loosely defined deliverable is asking for trouble.

To become a better estimator you need to keep in mind that even the most complex projects can be broken down into a series of small tasks. It’s practically impossible to accurately estimate how long it will take to complete a complex project. It’s far more reasonable to guess how quickly you can perform individual tasks.

My process of task generation follows a few basic guidelines:

1. Tasks should be small. The smaller the task, the more accurate the estimate will be. Additionally, smaller tasks give a psychological edge as they’re are inherently better defined, and less daunting than large tasks.
2. Tasks should all be approximately the same size. It’s human nature to put off doing larger, less defined tasks. This is rarely a good thing and can easily be circumvented by making all tasks the same size.
3. Tasks should be detailed. Within reason, tasks should include as much detail as possible.  If you’re working on a team, you may not be the person executing the task, so provide detail that others would understand so the task need not be outlined twice. If you find yourself writing paragraphs here, you probably need to split the task into a series of smaller ones.

Now, we have a big list, of managable, equally-sized tasks. Since the tasks are small, it’s easy to take three or four of them and make fairly accurate estimates on them. Since the tasks are all approximately the same size, the average estimate for those three or four tasks multiplied by the number of tasks will leave you with a reasonably accurate overall estimate, saving you having to estimate each step of the project directly. We can also group tasks back into our deliverables and provide more accurate deliverable by deliverable estimates for easy client consumption.

Aside from generating more accurate estimates, we’re also left with a valuable outline of the project, which can be used to execute the development of the project. Additionally, going through this process will often bring issues and questions to the surface, which will improve your perception of the project, ultimately making your estimates all the more precise.

In a future post, I will go into how this method of generating estimates fits into the new project management framework I’ve been hinting at.

Let’s first assume we’re dealing with a client who is open to stepping outside their comfort zone and adopting a new engagement framework with you. Let’s assume they’re willing to make themselves available for regular planning sessions and demo/review periods. Let us also assume that they’re willing to be held accountable for their role in the project’s completion. Let’s assume our client meets all these requirements, what do you do when the cost of educating the client on the methodologies, the processes, and the language of Agile is greater than the reward of putting those tools to work?

To make matters worse, I have found that the majority of the organizations I’ve dealt with in my career were not willing, or in some cases even capable, of meeting the afore mentioned requirements, nor did their projects posses what I deem to be the characteristics of a project, which would benefit from Agile.

I am by no means an expert on Agile, in fact my working experience with Agile is minimal — largely due to these substaintial stumbling blocks. Agile gurus, here’s your cue to jump in and tell me why I’m wrong. Please demonstrate how Agile addresses the issues I’ve raised. I’d love to be way off base on this, it’d sure make my life a lot easier.

In the meantime, enough about why Agile doesn’t work in these scenarios. Let’s get to the fun part: What does work?

Setting out to design a better way of managing small to mid-sized projects, I believe there are several criteria which must be met, on top of several of Agile’s parameters:

1. It must be easily, and quickly adoptable. When I say quickly, I mean pick up and go with no prior knowledge.
2. It must utilize familiar language. Part of easy adoptability means we must avoid new terms wherever possible. Terms, processes and tools must totally self-explanatory because no one wants to go back to school for the pleasure of working with you.
3. It must cater to different levels of client involvement. No company is the same and trying to force a client to do something that is dramatically different than what they do routinely does not work in the course of short term engagements. We can’t always have dream clients who can provide continuous, and timely feedback so we need a system that minimized the expense of less than timely interactions.
4. It must come with a tool. I truly admire the spirit of Agile’s “individuals and interactions over tools” notion, but let’s face it, even the most skilled carpenter isn’t very useful without his tools. The tool must be flexible and capable of adapting to meet the needs of variable of usage, but the tool must exist.

In my next post, I will begin to isolate the bottlenecks I see in the Agile process, discuss how they adversely affect small projects and start to explore alternatives. Until then, aside from the four criteria I mentioned here today, which characteristics must a process posses to work for your business?