DataTO.org is a community website, built as a venue for the users of Toronto’s open data to request and discuss open datasets. On top of enabling conversation, the site also helps users garner support for their request so as to establish a democratic prioritization of requests.
I first became interested in the open data dialogue back in early 2009 at ChangeCamp in Toronto, which was an event aimed at re-imagining government and citizenship in the age of participation. ChangeCamp spawned ChangeEngine, a project where a group of citizens (including myself), imagined a geo-aware issue tracking system, aimed at bringing communities together to solve problems and to provoke systemic change.
As the launch of Toronto’s open data site approached, some of my colleagues at Architech Solutions and I, met with Mark Kuznicki (organizer of ChangeCamp) to discuss the possibility of building a community companion site to the city’s site. We all felt there was a need, and so DataTO was born.
The Monday following our Friday lunch marked one week to go until the city’s launch date and Mark and I met for our first planning session at the Centre for Social Innovation in Toronto. At this meeting we focused on nailing down our user roles, scoped out the feature set for the initial release and roughed out some layout concepts.
Having had a full-day meeting crop up for later in the week, I was now down to 3 days of build time. Time to get going. Given the time constraint, I decided that building the app with PHP/CodeIgniter simply wasn’t feasible, so I decided to go with Grails, a Java-powered web application framework, which I had been tinkering with for a couple weeks at the time. This decision was not taken lightly, as I would not classify myself as a Java developer, but the fact that Grails utilizes Groovy, an agile dynamic programming language for the Java platform, and that Grails is so well suited for agile development, I felt that it was the way to go.
For those of you who are not familiar with Grails, think of it as the Java equivalent to Ruby on Rails. I won’t get into Grails vs. Rails in this post, so suffice it to say they are comparable, each with pros and cons. While Grails is relatively unknown at the moment, I suspect it will gain in popularity very quickly due to it’s power, flexibility and ease of use.
Having decided on a framework, I quickly spec’ed out the app, and started the build. Since Grails makes it so easy to build object models and generate scaffolding, I had more or less completed a working wireframe by the end of the first build day. The biggest benefit to having such an easy build process is that more time is left for UI, which is clearly very important to me.
In fact, the second build day was spent largely on UI. It was clear to me at this point that some of the features we had planned on being in the initial release were not going to be ready in time. So I decided to pare back, and focus on making a select few features as solid as possible, rather than diluting my efforts, and producing a greater number of mediocre components.
Day 3 of the build came and went very quickly, and was focused on skinning the application and debugging. The week had come and gone, and one daunting task remained — deployment.
Not being a Java developer, I wasn’t familiar with the various containers, but decided to go with Tomcat based on reputation and available documentation. I installed Tomcat 6 on a MediaTemple Dedicated Virtual server behind Apache and deployed the application the night before launch. This is somewhat of a “budget” setup, but it has held up well for this proof of concept. Ideally, I’d like to migrate to Amazon AWS for the infrastructure, but that will require funding (hint, hint).
Since launch, the site has received thousands of visitors from all over the world. It currently has over 200 members, and is approaching 100 requests for datasets. It has been mentioned in numerous major publications, and was profiled on The Discovery Channel’s Daily Planet.
I was extremely reluctant to put the site out there in it’s incomplete state, as I’m somewhat of a perfectionist, and I was worried that it might be a poor reflection of what I do, but I really couldn’t be more pleased. I’ve really enjoyed this as a learning opportunity, and a chance to do something good for the community. The site still has a long way to go, but I am continuing to work on it and can’t wait to see where it goes.
15 Comments to “Building DataTO.org – An Overview”
Adam, thanks so much for writing about your experience building DataTO.org! It was a great experience for me too. In particular, I was really happy with how the concept, user personas and use cases came together so quickly during our 4 hour design session. It just seemed to flow.
I couldn’t be happier having had a chance to work with a talented developer like you and really appreciate all the folks at Architech for all their support of this project.
Now to get more of that data opened up…
You’re one high Q developer Mr. Thody! Good work with the DataTO.org project. Can’t wait to see how it evolves.
Adam-
Thanks for the interesting story on how dataTO.org came to be.
I’m interested in the business aspects of it. Was this done on your own initiative, or a contract, or through Architech? Who funded it? Who owns it? Who will pay for its ongoing hosting and improvements? How long is such an arrangement guaranteed?
Nice; it’s a testament to Grails and to your work that you were able to put the site together particularly with limited Java and Grails experience.
Nicely done, and thanks for the writeup. Are you planning to make the source public?
Stephen & Oshoma: I work for Architech, and they sponsored the project by donating my time for that week, and subsequent maintenance/development has taken place partly on Architech time, and partly on my own time.
I would submit that nobody “owns” it per se. Mark and I are currently managing its operation and ongoing development, but the desire is for this to be a community project. The remaining question is how to best orchestrate such a project.
The hosting has been arranged by Mark.
There has been talk of open sourcing the project. I feel there is much to be gained by sharing this platform with other cities, and having the global developer community contribute to it. Again, this is now more of a question of how best to go about it. In the mean time, I’m committed to moving the development side of things forward both as a representative of Architech, and on a personal level.
Adam-
Thanks for the reply.
Personally, I’m not aware of functional websites that have done well as an open-source platform. For instance, consdier http://www.php.net: even though the site is for an open-source product, the site is not itself opensource.
(AFAIK!)
Stephen, to clarify, we’d like the source to be openly available for others to use and modify to their liking for their own uses.
However, contributed source would be carefully managed, so as to maintain a high level of quality throughout the DataTO application. Trust me, the last thing I want is for it to turn into a mishmash of bad code, and feature bloat, which is the death of so many open source projects.
I will add that the domain is owned by me on behalf of the community and that I see the site as an important community asset. Other places have already asked about adapting it for their cities, which Adam and I would love to support however we can.
Longer-term, a good community sponsor could really help make it sing in a sustainable way. For now, we really appreciate the contributions of Adam and his employer.
Great story, Adam, about working with severely limited constraints to focus on delivering value in quick turnaround. Reminds me of a quote I saw recently on Twitter from LinkedIn founder – If you aren’t embarrassed by your first release, you’ve waited too long.
Gerry, that’s a great quote, and I couldn’t agree more. Thanks!
Congratulations on the execution of this idea. I’d agree with a number of the comments regarding the sharing of this site with other cities. I’m positive that others would benefit from following this concept of a community led space when we go down this path.
Were there any constraints in developing the website? Or will there be as we hopefully see more integration with the city and the site?
Hey Malcolm, constraints in what respect? Thus far the only constraint to speak of has been time and budget.