evan.musing << current

I’m happy to announce that I’ve been invited to speak about rubinius at the upcoming SDForum Ruby Conference. I’ll be spicing up my usual rubinius intro talk by diving into some of the internals and showing how people can really customize the entire system.

I may also do my usual “blow your mind, then ask for questions” approach, where I go through everything, then leave about 20 minutes at the end for questions. Leave a comment if you think you’d like to see that.

Big thanks to (hasmanyjosh) Josh Susser for getting me on the speakers list (and also wrangling the conference as of late).

Also, please leave a comment on this post if there is any topic you’d like to see me go over. As usually, I won’t have the talk finalized until an hour before I give it.

Got back from Mountain West RubyConf (MWRC) Saturday night. It was quite a fun rubyconf. The talks were by and large good, and there was lots of socializing (ie. networking), which is always one of the biggest reasons I go to conferences.

I was on a ruby implementors panel with the JRuby guys (Charlie and Tom), the CLR guy (John), and the Parrot guy (Kevin). It was fun, we just took questions from the audience and discussed them. It was all recorded, so I’ll be sure to post the link once the videos are up.

I got the word out about rubinius quite a bit and I’m really pleased at how interested people are in it. I got everything from “How does your garbage collector work?” to “I’d love to help, but I know crap about VMs”, which tells me that the project is appealing to not just VM geeks, but the normal programmer as well.

Pat Eyler and I talked briefly about setting up a release schedule for rubinius, working toward the 1.0 release in October. I’m pretty sold on it and I’ll be posting details in a bit, once I nail down the dates.

I’m very happy to announce I’ve just committed revision 809, which contains subtend, the rubinius C extension interface. It’s modeled after the MRI C interface, to allow most if not all C extensions that run under MRI to compile and run under rubinius.

Techy details: While subtend sports the same C API we’ve all grown to love from MRI, under the hood it is quite different. To properly support the accurate garbage collector, subtend uses a handle abstraction layer for object references the C code holds, which allows the VM to remain flexible with regard to object allocation and movement. Probably the thing I’m most proud of is that it allows C code to call back into ruby code without enlarging the C stack. I’ll talk more about that later. Come celebrate with us in #rubinius on freenode.net!

It’s been too long since my last post, so I’m going to try to keep to a strict schedule of at least one post a week.

For this week, I’d first like to wish everyone a happy holidays.

Now onto rubinius. The pace of the project has increased a lot — we’ve moved from revision 25 to revision 255 in only about a month, which is amazing. There are about 4 of us who have arisen as the core developers, with around 6 or so others that have sent in patches and gotten commit rights. Ohloh has begun tracking our stats.

There has been a big push to get a good testing framework worked out to move further towards rspec. brixen and srbaker have been committing spec files and getting the filesystem laid out, as well as working on a driver to run specs against rubinius easily. I think that soon there will be a rubinius testing committee that arises to drive all the rubinius testing.

As for me, I’m currently focused on about 4 main goals. The first was to get a good framework for the rubinius workflow to seen by everyone and to set down a little bit of process to keep everything moving smoothly. To that end, I’ve been hacking furiously on collaboa, adding features that I want to use. The result is rubini.us. It’s running our custom collaboa instance, to which i’ve added 2 of the nicest features of trac: a wiki and a timeline. The other feature I’ve added is tags, to help organize and search the tickets and wiki.

I’ll be putting down collaboa for a little bit to focus on the rubinius source again. My first task is to get the VM self-hosted, that is to say, getting the compiler to compile itself so that the standalone rubinius VM can actually compile and run programs all by itself. The last push to this goal had me integrating the parser from syd-parser to generate sexps from strings and files full of ruby code. There is still some debugging to work out there, but it is functioning. Now I’ll be focusing on getting the compiler runtime to work properly. I suspect that I’ll have a first cut of this goal committed within the next few days, then I’ll dig in and fix a few bugs that I’m aware of in the compiler process. My hope is that the first cut will work well enough that the testing guys will be able to just ask the VM to run a file full of ruby code and it will do it properly. This will also make running the bfts tests under the VM a lot easier.

The last big goal on my plate is the integration of code to allow for methods to be implemented by C functions. 1.8 sports this ability and it is used constantly, actually forming the core of the 1.8 core library. rubinius will have the exact same C api as 1.8, to make porting C extensions from 1.8 to rubinius that much easier. The idea is to write a shim library that sits between the C extension and the rubinius core. This shim library will protect the core from corruption and still give the C extension all the same power that it currently has. The working name for this framework is RNI (Ruby Native Interface), but people have been pretty vocal about picking a different name. If you have a suggestion, leave it in the comments.

Well, that rounds out this post. See you in 2007!

UPDATE: Changed the link to point to the permlink.
I’ve been doing an email interview with Pat Eyler all about rubinius and it’s now up on his blog.

So, I’ve officially started accepting donations after Geoff Grosenbach became my new hero and donated $1000. So naturally, questions have come up…

• Why should I donate time/money to rubinius?

  The answer is simply because you believe that rubinius benefits the community. Therefore getting it to a state where the community can get it is a goal you want to help with.

  So…

• How does rubinius help the community?

  I love the ruby language. Its grace and elegance are second to none in my opinion. But lurking under that beautiful language is a runtime in distress. Even Matz has admitted this by putting the energy of the future into YARV. So, there are features that the runtime and language needs that the current interpreter just isn’t up to: bytecode and native threads being just 2.

  So why not put your money/time into YARV? I’m not putting my money/time into YARV because I don’t believe it’s the right way to accomplish the goals. (NOTE: I’ve tried hard in the past few months to not be critical of YARV and the current interpreter too much. But people have asked, so here’s my opinion) YARV may or may not see the light of day. It’s been in development for 2+ years and while it’s getting close, matz has not committed to it for at the very least another 2 years. So, if we wait for YARV, the community is still a long ways away from a modern VM. Plus, I believe that YARV has a lot of the same problems the current interpreter has. It’s filled with very complicated, undocumented structure that makes working on the core quite difficult.

  I believe that the future of the ruby core is in a simple core and a simple architecture that is easy to modify. Rubinius goes to vast lengths to keep the core parts as simple as possible. Because in the end, it shouldn’t just be me and a few core developers who can dive into the core and change things. I want it to grow and flourish in the hands of everyone, and I believe the only way to do that is to build in that virtue from the beginning.

• How will rubinius make my life/work easier/better?

  There are a lot of different ways to answer this question. So I’ll say this: rubinius aims to be simple at the core so it can be a lot of things to a lot of people. The longer way of saying that is that firstly, rubinius can be customized to make your life easier. While we know ruby is already highly customizable, rubinius aims to go even beyond that.

  Secondly, rubinius will fit into places the current interpreter won’t. One good example is that rubinius is all driven by bytecode and will even have hooks to further help companies protect their ruby intellectual property. To companies, that means ruby is much more enticing to use. And the more companies that use ruby, the more people, and the community grows. But don’t misread that and think that I’ve only developed rubinius to make companies happy. That’s just an example of where the “simple core, simple architecture” paradigm allows the whole project to remain agile and customizable by everyone that wants to use it.

I hope that clears things up rather than making them more confusing. Please comment on this post and I’ll clear things up and answer any questions people have.

I got a few reports that there was troubles compiling rubinius on linux so I dug in. Had to make some bug fixes since linux seems to use all 32 bits for a memory address. Also did a bunch of minor cleanups so that gcc on linux complains less.

If you’re running on linux and still getting compiler problems, please enter a bug!

Some recent commits to rubinius have yield some great results:

• Continuations: They’re under fire right now in the ruby community as to whether they’re a feature that implementations need to support. Rather than wait for the dust to clear, I went through the motions of implementing them in rubinius. It was amazingly simple actually. It required some debugging to get it correct, but all in all there wasn’t actually all that much code, just a primitive VM method to manually activate a method context (aka a stack frame). Having method context’s be first class objects is what made it easy, and it’s again validated my decision to make everything possible first class.
• additional stability: more work to stabilize the core has yield some new error conditions that get nicely translated into exceptions being thrown, the primary one being when the stack is blown (all used up) a StackExploded exception is thrown and can be handled (carefully)
• clean up: I’ve gone through the whole process front a fresh checkout and fixed up the order of things that were missed when I was constantly using a copy that had some cached object files.

You should be able to build and test shotgun now pretty easily, doing as follows:

$ rake shotgun
$ rake test_shotgun

It will go through building shotgun and running the current set of core tests against it.

If you encounter errors, please email me with them, and attach a patch if you want!

Soon I’ll be release some code with rubinius that will let people automatically report errors, I’ll post once it’s up and running.

I’ve just checked in Prototype S, called shotgun, to svn. It’s not even close to complete, but it’s running compiled methods that were generated by the Prototype B compiler. It’s a big first step, and things will hopefully start to go quickly.