Saturday, May 10, 2008

Benchmark Suite development plan

It's only draft version, any suggestions are welcomed!

Breaking application into several pieces help to make ease of use and modification each part, so lets break:
1)'Typical' RoR application
An application that can hit performance cases most rails developers runs into. I see such an application as some quick project with most usefull plugins such as will_paginate, etc(any suggestions?). May be we should look into 15 minute projects from rails site(are they need to be updated?)

2)Rake tasks
Automated rake tasks able to pull latest version of rails edge from git and latest jruby(ruby) version build and test. It is suitable for running on server( for example in continiuos building) and locally by developers for introduced regressions =)

3)Baseline
Ruby1.8+latest stable Rails.
Most used for now is MRI 1.8 and JRuby measured against 1.8, so that's the primary metric by which we need to measure.

4) Rating
Provide rates for combinations ruby implementation+rails against baseline. Write all that data into database to make it's usage in rails app.... magic circle =)

5) Visualize results
Write some app that can show pretty statistics for obtained data

UPD: For now we choose typo blog engine as typoical rails webapp. It uses many commoan for modern webapp features:

No comments: