Archive for the ‘ruby’ Category
The Ruby Community Rocks
The last time we checked in, I was delivering the bad news about having to let a bunch of my team go. I received a lot of kind words of encouragement during the hard time, which I want to thank everyone for.
In addition to kind words, a number of people stepped up and indicated they had positions available that my newly unemployed friends would be great fits for.
One such offer was from Daniel Yoder and Charles Hornberger at AT&T Interactive in the R&D department, the makers of the yellowpages.com. I’m extremely happy to announce that Eero Saynatkari (rue on IRC) has recently been hired by them and even given time to continue work on Rubinius!
This development makes me so happy. To see the community pull together in a tough time and even continue to make an external investment in Rubinius.
Thanks again guys! You’re what make me love being a Rubyist.
A Sad Day
There have been some sad developments within the Engine Yard Rubinius team that I’d like to address head on.
Earlier today, I had the unfortunate task of reducing the team size to 2 people, which meant laying off the rest of the team.
I’m sure this comes as a shock to many, as it did to my friends to whom I had to give walking papers. This was certainly never a scenario that I had ever hoped to find myself in when Engine Yard offered me this dream job early in 2007.
The reason for the layoffs is not Engine Yard divesting interest in Rubinius,
but rather a necessary reorganization of budget priorities. That’s a fancy way of saying that EY could no longer afford to sustain the large team we had.
This is a sad day for me, one that I’ve been dreading. It stings not only
because of what it means to Rubinius but also because of what it means to my friends with whom I will no longer be working. They’ve put blood, sweat, and tears into Rubinius and their everyday presence will be sorely missed. I hope that they do not think badly of me or Engine Yard.
When Engine Yard gave me the go ahead to hire a team, they did it with the best
intention: to help Rubinius grow. And we have definitely done that. In the
last year, we’ve achieved amazing goals within the project:
- We went from running very little ruby code to running rails.
- We got rubygems up and running well.
- We got a parser entirely in ruby integrated.
- We wrote a whole new VM to build on.
We’ve had our fair share of setbacks, but the team has always rallied.
Rubinius will continue to move forward, continually bolstered by the awesome group of people who give up their free time to help on the project.
Tom Mornini has posted on the EY blog as well; you should read his take.
Welcome to the Club
I’ve like to formally welcome the maglev development team over at Gemstone to the Ruby environment club.
For those of you that haven’t yet heard of maglev, it’s a brand new Ruby VM being developed by the folks over at Gemstone. Gemstone is the makers of probably the most advanced object-oriented database used today, and have traditionally been a Smalltalk shop till recently.
With the tide rising on Ruby, I’m happy to see another player enter the field. This only means that Ruby is continuing to mature and see that the community is healthy.
I was personally excited to read an interview with Bob Walker and Avi Bryant concerning maglev, because Rubinius is mentioned more than a few times. They’re looking at Rubinius for a couple of reasons. For one, the RubySpec suite we’ve developing and are about to spin off. The more people that we see using the suite and depending on it, the more mature it will become. Not having a spec for Ruby is commonly touted as a reason that it’s a toy, immature language, and anything we can do to dispel that thinking is good for the community.
The other reason that I’m excited about maglev is that they’re taking a very similar approach to the problem of building a Ruby environment. Like Rubinius, the VM is minimal and most of the kernel is implemented in Ruby.
My hope is that the kernel of Rubinius can be refactored and developed to be generic enough for other environments to use. While I know little about maglev’s current environment, they’re a natural build off the work in the Rubinius kernel. I’d hate to see people develop the code functionality of a ruby environment yet again (I count 5 code bases to this effect currently: MRI, JRuby, Ruby.NET, IronRuby, and Rubinius).
Being able to use a generic Ruby kernel is not unique to a smalltalk style VM. With some luck, it could be used by the folks in other environments as well. In my eyes, this is a big win for everyone. For one, this would mean a common code base that consists of the primary Ruby functionality, and thus would mean a vastly reduced worry of fragmentation. Plus it would alleviate the need for this code to be written again, letting future environment developers focus on taking Ruby to the next level in terms of platform integration, performance, etc.
super is your friend
Sitting here in Copenhagen, at RubyFools, I thought I’d share a technique that I’ve known about for some time, but seems to not have gotten into the normal ruby vernacular.
This trick is the use of super in methods contained in a Module. Consider the following code:
module N
def go
puts "N#go"
super
end
end
class B
def go
puts "B#go"
end
end
class A < B
include N
end
A.new.go
Will print:
N#go B#go
This is HIGHLY useful implementing rails plugins, where normally you’d use alias_method_chain to change a method directly inside ActiveRecord::Base. Instead, simply call super in the method that provides the new functionality, and when your module is included in your module class (which is a subclass of ActiveRecord::Base), and the main, ActiveRecord::Base implementation will be called.
NOTE: This trick only works if the method you wish to wrap is located in a superclass of the class you have defined the module in. IE, if N were included in B directly rather than A, N#go would never be called.
Day to day rubinius
The venerable Eric Hodel (drbrain) has whipped the rubinius team into blogging more, so this should be the first of many posts to come.
Development front
We continue to push forward, getting rubinius running everyday ruby code. I continue to use irb under rubinius daily, and it’s proved quite stable.
Though, we’ve begun to hit the standard lib code that is quite tricky. Case in point, mathn.rb. mathn.rb adds some new methods to Fixnum and Bignum, as well as redefining some very core methods, like, Fixnum#/ (divide). We’re currently working through how to support this, because as is, when mathn redefines that method to start returning Floats, most of the rubinius system goes south. Thats because the kernel currently assumes that if it uses Fixnum#/, a Fixnum is returned, and it can pass that to other primitives and such.
We’re still unsure about the long term solution for this, but it does bring up a problem with having a very open, dynamic language, where the kernel of the language uses that same open, dynamic runtime. A user can change the behavior of a core, system method, and effect a lot more than they could in MRI. This is the famous double edged sword.
My hope is that we’re able to code the kernel a little defensively and really stabilize the kernel to the point that these kinds of changes wont cause the whole system to go south.
Conference front
Seems that 2008 is going to be the year of conferences for me.
- On Feb 8th and 9th, Ezra and I will be at acts_as_conference, giving a charity tutorial talk about how to be a better ruby developer.
- Next, I’ve been invited to give at talk about rubinius, as well as the “Party” keynote (not sure what the party part is actually) at RubyFools over in Copenhagen, Denmark. This should be a lot of fun for a few reasons.
- I’ll be giving a non-Rubinius specific talk for once (the keynote), which is something I’ve wanted to do for a while.
- I’ve never been to Denmark, so it’s always fun to visit a new city.
- Abby is coming too! She’s going to sightsee and such while I’m at the conference, and we’ll have a few days before and after to take day trips and such. She loves europe, it should be a great time.
- There a bunch after this, but I’m not yet sure which I’m going to. The total is something like 5 or 6 before June 1st, so it’s going to be busy busy busy no matter what.
More to come!
Performance benchmarks
The project is getting more and more visible, so I feel it’s important that people have an accurate assessment of where we are. Most of us have seen these old benchmarks, so lets see some more recent results.
I’ve only done rubinius versus MRI. This was performed running on my Powerbook G4 1.67Ghz laptop. I’ll run these again soon on a big beefy server so we can all compare and contrast.
Data
Test MRI Rubinius
bm_app_answer.rb 4.727393 0.619218
bm_app_factorial.rb 1.703541 1.268409
bm_app_fib.rb 18.739502 5.309634
bm_app_mandelbrot.rb 11.175115 Error
bm_app_pentomino.rb Timeout Timeout
bm_app_raise.rb 6.825073 5.744597
bm_app_strconcat.rb 4.46694 4.086749
bm_app_tak.rb 24.502033 7.291604
bm_app_tarai.rb 21.071624 11.594766
bm_loop_times.rb 16.028508 Timeout
bm_loop_whileloop.rb 29.844976 4.767554
bm_loop_whileloop2.rb 5.232881 1.103926
bm_so_ackermann.rb Timeout 6.626395
bm_so_array.rb 18.276808 Timeout
bm_so_concatenate.rb 5.597246 Timeout
bm_so_count_words.rb 0.099502 Error
bm_so_exception.rb 9.262875 7.553756
bm_so_lists.rb 3.372947 Timeout
bm_so_matrix.rb 5.605184 23.462452
bm_so_nested_loop.rb 17.162697 54.288111
bm_so_object.rb 19.170284 21.458258
bm_so_random.rb 7.168005 24.018174
bm_so_sieve.rb 4.870225 15.966473
bm_vm1_block.rb Timeout Timeout
bm_vm1_const.rb 47.509964 11.05456
bm_vm1_ensure.rb 45.855905 5.395634
bm_vm1_length.rb 49.408877 24.633311
bm_vm1_rescue.rb 35.682676 6.107035
bm_vm1_simplereturn.rb 47.935916 15.641316
bm_vm1_swap.rb Timeout 8.041324
bm_vm2_array.rb 18.284155 12.393373
bm_vm2_method.rb 34.70641 16.282999
bm_vm2_poly_method.rb 45.63907 23.423347
bm_vm2_poly_method_ov.rb 11.462504 6.403511
bm_vm2_proc.rb 22.432265 27.886953
bm_vm2_regexp.rb 11.580949 Timeout
bm_vm2_send.rb 11.223254 21.448888
bm_vm2_super.rb 12.599854 4.998904
bm_vm2_unif1.rb 10.726272 3.185878
bm_vm2_zsuper.rb 14.236729 4.948068
bm_vm3_thread_create_join.rb 0.086081 Timeout
Analysis
There are a few trends in the data I’d like to point out.
- Of tests that did not error or timeout, rubinius was faster in 24 of 31. Wow first off, thats a huge improvement over the previous run.
- The slowest section for bm_so. Rubinius was only faster in 2 of 11, and actually error or timeout on 4. If you look at those benchmarks, you’ll see that they are basically tests of a few core methods, mainly things like String#<<. So it makes sense that at this stage, we’re slower on those. We haven’t yet tuned those at all.
- One big trend is that tests that only stressed the VM architecture came out WAY faster. 2 examples are bm_vm1_swap and bm_vm1_simplereturn. The first swaps two local variables using
a, b = b, aa few million times. This is a good example where the bytecode VM is much faster than the tree walker in MRI. Next, bm_vm1_simplereturn shows off rubinius’s ability to create a method context quickly and return to the sender quickly. I’m thrilled about this number because even though rubinius MethodContext’s are first class, they’re still 3 times faster with no programming power loss.
All in all, I’m very happy with these results. They show that the project is advancing and is a viable ruby implementation, not just a toy.
Update: More Data
Here’s the data from running on a 64bit xeon. The rations are not the same because I haven’t yet got direct threading working properly on 64bit platforms, so that impacts rubinius performance negatively in this case.
Test MRI Rubinius
bm_app_answer.rb 0.674141 0.357815
bm_app_factorial.rb Error 0.40302
bm_app_fib.rb 6.023813 2.86666
bm_app_mandelbrot.rb 2.305716 Error
bm_app_pentomino.rb Timeout Timeout
bm_app_raise.rb 1.634094 2.681252
bm_app_strconcat.rb 1.541677 1.466644
bm_app_tak.rb 7.749194 4.02251
bm_app_tarai.rb 6.194152 6.621082
bm_loop_times.rb 3.520025 32.848938
bm_loop_whileloop.rb 8.091596 2.464447
bm_loop_whileloop2.rb 1.736418 0.609515
bm_so_ackermann.rb Error 3.579584
bm_so_array.rb 4.89737 Timeout
bm_so_concatenate.rb 1.573779 Timeout
bm_so_count_words.rb 0.145074 Error
bm_so_exception.rb 3.179525 3.461771
bm_so_lists.rb 1.429547 Timeout
bm_so_matrix.rb 1.842544 10.748483
bm_so_nested_loop.rb 5.337045 18.885963
bm_so_object.rb 5.428432 9.856728
bm_so_random.rb 2.612983 11.789056
bm_so_sieve.rb 0.711854 5.268267
bm_vm1_block.rb 26.471025 37.74646
bm_vm1_const.rb 14.004854 5.930651
bm_vm1_ensure.rb 14.199208 2.854205
bm_vm1_length.rb 16.117594 13.692691
bm_vm1_rescue.rb 11.509271 2.859993
bm_vm1_simplereturn.rb 14.78014 8.154143
bm_vm1_swap.rb 22.52124 5.419691
bm_vm2_array.rb 6.238171 4.938015
bm_vm2_method.rb 9.747336 9.017925
bm_vm2_poly_method.rb 12.513751 11.709814
bm_vm2_poly_method_ov.rb 3.961969 2.150468
bm_vm2_proc.rb 6.393898 10.224857
bm_vm2_regexp.rb 3.224304 54.639602
bm_vm2_send.rb 3.375969 12.034005
bm_vm2_super.rb 4.012679 2.795978
bm_vm2_unif1.rb 3.005257 1.716368
bm_vm2_zsuper.rb 4.336752 2.83723
bm_vm3_thread_create_join.rb 0.14954 Error
Back from MWRC
Got back from Mountain West RubyConf (MWRC) Saturday night. It was quite a fun rubyconf. The talks were by and large good, and there was lots of socializing (ie. networking), which is always one of the biggest reasons I go to conferences.
I was on a ruby implementors panel with the JRuby guys (Charlie and Tom), the CLR guy (John), and the Parrot guy (Kevin). It was fun, we just took questions from the audience and discussed them. It was all recorded, so I’ll be sure to post the link once the videos are up.
I got the word out about rubinius quite a bit and I’m really pleased at how interested people are in it. I got everything from “How does your garbage collector work?” to “I’d love to help, but I know crap about VMs”, which tells me that the project is appealing to not just VM geeks, but the normal programmer as well.
Pat Eyler and I talked briefly about setting up a release schedule for rubinius, working toward the 1.0 release in October. I’m pretty sold on it and I’ll be posting details in a bit, once I nail down the dates.
Continuations, additional stability, and clean ups
Some recent commits to rubinius have yield some great results:
- Continuations: They’re under fire right now in the ruby community as to whether they’re a feature that implementations need to support. Rather than wait for the dust to clear, I went through the motions of implementing them in rubinius. It was amazingly simple actually. It required some debugging to get it correct, but all in all there wasn’t actually all that much code, just a primitive VM method to manually activate a method context (aka a stack frame). Having method context’s be first class objects is what made it easy, and it’s again validated my decision to make everything possible first class.
- additional stability: more work to stabilize the core has yield some new error conditions that get nicely translated into exceptions being thrown, the primary one being when the stack is blown (all used up) a StackExploded exception is thrown and can be handled (carefully)
- clean up: I’ve gone through the whole process front a fresh checkout and fixed up the order of things that were missed when I was constantly using a copy that had some cached object files.
You should be able to build and test shotgun now pretty easily, doing as follows:
$ rake shotgun $ rake test_shotgun
It will go through building shotgun and running the current set of core tests against it.
If you encounter errors, please email me with them, and attach a patch if you want!
Soon I’ll be release some code with rubinius that will let people automatically report errors, I’ll post once it’s up and running.
Shotgun (Prototype S) release “sand in your eyes” committed
I’ve just checked in Prototype S, called shotgun, to svn. It’s not even close to complete, but it’s running compiled methods that were generated by the Prototype B compiler. It’s a big first step, and things will hopefully start to go quickly.
