Helpful syntax errors
Syntax errors are a part of life for programmers. The language of the computer, no matter how flexible the language, is very picky.
And thus how the language communicates back to the user about what it didn’t understand is important, because time is spent in this phase, no matter the skill level of the programmer.
In Ruby specifically, MRI’s parser (and by extension the melbourne parser Rubinius uses) use yacc, and thus suffer from syntax errors which can be particularly difficult to understand. One particular syntax error that commonly occurs is when there is an ‘end’ missing from an expression. This results in the dreaded syntax error, unexpected $end, expecting kEND message.
Here is a quick example:
class Spaghetti
class Sause
def add(plate)
while more?
plate << self
end
end
end
Now, this is a short example and so spotting the error is fairly easy. But this error typically occurs when you’re working on a 600 line file with multiple classes inside classes and with complicated logic, making it quite difficult to find.
This evening, I decided to try and at least help make this easier to find. So now in Rubinius, rather than
syntax error, unexpected $end, expecting kEND
you get
missing ‘end’ for ‘class’ started on line 1.
Thats a big improvement, because now first off, it’s fairly clearly communicated what is wrong, i.e. that you’ve forgotten an ‘end’. In addition, it tells you what element still required a ‘end’, in this case, a ‘class’ on line 1.
Now, this is far from perfect. It’s pointing you to the element that was unclosed, rather than the one that you actually forgot the ‘end’ on. But it at least is now pointing you to the chunk of code that is the offending code. In practice, this can be a big help.
In the future, it might be possible to try and use indentation to try and narrow down where the missing ‘end’ should be. But for now, every little bit helps.
Fixing colors in Terminal.app on 10.6
It’s Snow Leopard day zero, so of course I had to upgrade. All in all, everything is great. (Especially the multiple monitor window migration fix!)
But the #1 thing that annoys me about all OS X releases in the colors in Terminal.app. They’re pretty much unusable on a dark background (especially the blue). For some time, there have been hacks to fix the problem. Well of course these hacks didn’t work on 10.6 anymore.
Never one to shy away from the problem, I dove in. And we have success!
Here’s how to make it work:
- Find Terminal.app in Finder (/Applications/Utilities), right click, “Get Info”
- There is a checkbox “Open in 32-bit mode”, Check it!
- Install SIMBL. Plugsuit was installed on my machine before, it freaks out because of the 10.6 changes. SIMBL silently just works or doesn’t.
- Get My updated TerminalColours SIMBL plugin. See the original post for details on how to install it.
- Restart Terminal.app
- Enjoy your readable colors!
This works because InputManagers still work in 32bit mode, but not 64bit mode. So by forcing Terminal.app to run in 32bit mode, SIMBL can still hook in. I just had to update TerminalColours to swizzle a new method that 10.6 uses to pick colors.
Hope you enjoy!
Update: I’ve changed the tar.gz download link to one that should work better.
Doing a Time Machine full restore even if it doesn’t want to
Got a new harddrive for my wifes MacBook. Couldn’t find her 10.5 upgrade DVD, so I tried to use the install DVD for my new MacBoor Pro. The DVD promptly told me it would not install (Apple cripples the OEM DVDs to only reinstall on the same make of machine), but it did not exit. So I select Restore from Time Machine Backup from the Utilities menu.
Nothing happens.
I try a few times, nothing. I give up for a while. Upon returning and seeing it still doesn’t work, I open the Install Log under Window.
I notice there are number of entries listed: Unable to load XIPanel_RestoreIntroduction nib file. Ok, well something is wrong with the Installer.
Terminal to the rescue. Open it up, do find / -name “XIPanel*” and find a number of them in a Resource folder. Ok, so it’s there, this must be a bug in the installer. Now we get serious. If you’ve gotten this, here’s what you do to get the Time Machine Restore to launch:
- Open Terminal
- Run ps ax and find the listing for Mac OS X Installer.app, note the number of the far left for it (the Pid)
- Run kill pid_of_installer
- You should now get a dark grey background, you’re doing great.
- Run export LANG=en_US.UTF-8. I’m not sure if this matters, but I did it, so you should too
- Run cd “/System/Installation/CDIS/Mac OS X Installer.app/Contents/MacOS/”
- Run ./Mac\ OS\ X\ Installer “/System/Installation/Packages/OSUpgrade.pkg”
- The installer should popup and say Welcome and such (likely with no graphics, thats ok)
- Click Yes/Continue enough for the Utilities Menu to appear at the top and select Restore from Time Machine Backup
- With luck, it will load! You can now do the restore!
Thanks for playing! Booo to Apple for having bugs in their Installer, Yay to Apple for leaving Terminal available in the Installer!
NOTE: Be sure to leave the Installer in the foreground while it restores, otherwise it will stall!
Rumors of our Demise are Greatly Exaggerated
We’ve been pretty quiet with Rubinius developments for a while, so I thought I’d bring people up to speed.
The previous year has seen a lot for the project. We were sad a number of developers were laid off the project, but that has only increased our desire to get the project to a usable state.
Some of the highlights include, but are not limited to:
- Rewriting the VM in C++
- Experimenting and building multiple JIT compilers
- Pushing RubySpec completeness and compliance levels
- Getting large scale libraries like Rails and RubyGems running
All those things are available today in our git repo on github.
Recently, Brian Ford and I published a roadmap, laying out the activities of most importance over the next few months. We’re going to try and be more vigilant about updating blogs and roadmaps in the coming months, to keep people more up-to-date.
Finally, a lot of people ask me “How can I help on Rubinius? I don’t have a lot of time.” The answer is simple:
- Download your favorite library
- Try it under Rubinius:
bin/rbx test/test_whatever.rborbin/rbx gem install rspec; bin/rbx -S spec my_spec_dir
- Report bugs that you find to our github Issue tracker.
The more people start to report bugs, the more coverage we get over the vastness of the ruby landscape. So while we’re hard and work getting the performance up, you can help out getting the compliance up.
Thanks again to the ruby community for all the patience you have shown the team over the years. Rubinius has been a long road, but I really feel like we’re onto something big.
In the coming months, I’m going to try and post more posts about technical aspects of Rubinius, so look for those.
The Ruby Community Rocks
The last time we checked in, I was delivering the bad news about having to let a bunch of my team go. I received a lot of kind words of encouragement during the hard time, which I want to thank everyone for.
In addition to kind words, a number of people stepped up and indicated they had positions available that my newly unemployed friends would be great fits for.
One such offer was from Daniel Yoder and Charles Hornberger at AT&T Interactive in the R&D department, the makers of the yellowpages.com. I’m extremely happy to announce that Eero Saynatkari (rue on IRC) has recently been hired by them and even given time to continue work on Rubinius!
This development makes me so happy. To see the community pull together in a tough time and even continue to make an external investment in Rubinius.
Thanks again guys! You’re what make me love being a Rubyist.
A Sad Day
There have been some sad developments within the Engine Yard Rubinius team that I’d like to address head on.
Earlier today, I had the unfortunate task of reducing the team size to 2 people, which meant laying off the rest of the team.
I’m sure this comes as a shock to many, as it did to my friends to whom I had to give walking papers. This was certainly never a scenario that I had ever hoped to find myself in when Engine Yard offered me this dream job early in 2007.
The reason for the layoffs is not Engine Yard divesting interest in Rubinius,
but rather a necessary reorganization of budget priorities. That’s a fancy way of saying that EY could no longer afford to sustain the large team we had.
This is a sad day for me, one that I’ve been dreading. It stings not only
because of what it means to Rubinius but also because of what it means to my friends with whom I will no longer be working. They’ve put blood, sweat, and tears into Rubinius and their everyday presence will be sorely missed. I hope that they do not think badly of me or Engine Yard.
When Engine Yard gave me the go ahead to hire a team, they did it with the best
intention: to help Rubinius grow. And we have definitely done that. In the
last year, we’ve achieved amazing goals within the project:
- We went from running very little ruby code to running rails.
- We got rubygems up and running well.
- We got a parser entirely in ruby integrated.
- We wrote a whole new VM to build on.
We’ve had our fair share of setbacks, but the team has always rallied.
Rubinius will continue to move forward, continually bolstered by the awesome group of people who give up their free time to help on the project.
Tom Mornini has posted on the EY blog as well; you should read his take.
CPP work branch change
Hi everyone. I’m super happy to announce that we’ve gotten the C++ branch stable enough that we’re making in the default branch. This means that those of you with existing clones are going to likely do a little work to get them sane though.
Here is what was done:
- The old master branch was rename shotgun.
- The cpp branch was copy to the name master.
- The cpp branch was then deleted.
Anyone that has up to now been working on the cpp branch has a couple of options.
- Delete your clone and re-clone. This is the easiest. The default checkout will be code in the cpp branch and you’re off and going.
- Fix up your current repo. I did this by doing the following commands:
git checkout mastergit reset --hard origin/mastergit branch -D cpp
This will get your local master branch repointed and properly checked out. In addition, the old cpp local branch can be deleted.
Hopefully no one experiences much pain due to this change. It’s been a long time coming and I’m really excited.
If you do run into problems, post a comment or stop on by IRC and we’ll work it out for ya.
Thanks!
Rubinius Status
Hey folks, sorry for the quietness here. Thought I’d fill everyone in on the current status of Rubinius.
We (the Rubinius team) have been hard at work on a couple of fronts:
- a new C++ VM: the team has been hard at work getting a new VM up and running back to the level we had the old system at. Our output has slowed to a little bit as the rest of the team has gotten up to speed on this new code base. Now, I’m sure you’re wondering why we’ve begun working on a new VM. Well, there are a few reasons:
- Better organized. We’ve learned a lot in the building of the last VM about how to structure things. For instance, using C++ lets us model Ruby classes as C++ classes, providing the VM with the same familiar structure and execution as their Ruby counterparts. This lowers the barrier for understanding and using the code base.
- Better tested. The old VM, I’m ashamed to say, had no unit tests. From day one of the new VM, we’ve been writing unit and integration tests. This has helped us a lot to keep the code base under control, as everyone who writes unit tests knows.
- More potential. One of the big changes is keep a lot of parts of the system open. For example, we’re actively experimenting with using LLVM to speed up method execution a lot. The old code base, with no tests, was quite tangled and sadly didn’t provide any easy way forward for a lot of experiments we wanted to do
- We’ve been working a lot of Ryan Davis’s ruby_parser project lately. We’re actively looking to use that code base as Rubinius’ internal parser. This is towards our goal of more Ruby code, but as anyone who’s written a parser will tell ya, it can be a real pain.
Ryan has made great progress getting it working and integrating it with Rubinius’ Compiler. - Conferences: Wilson Bilkovich was just in Berlin, talking technical about Rubinius at RailsConf Europe. I’m here in Austin, at Lone Star Ruby Conf, finishing up my keynote that I’ll be giving later today.
The whole team will be at RubyConf in November as well, and a few are likely going to OOPSLA as well.
I know that a lot of people are eagerly anticipating Rubinius, and I want to thank you all for your patience with me and the rest of the team.
This is a dream project, and turns out to be pretty darn hard and a lot of work. I’ve made the mistake in the past about talking about when I think that we’ll release 1.0. Something I don’t think I properly understood a year ago was the level that people are looking for a 1.0 to operate at. If we released a 1.0 that was 10x slower than MRI, we’d probably be in pretty tough shape.
At the some time, I know people want to play with Rubinius. We fell off doing monthly releases a while back, and we’re going to start getting back on that soon. Hopefully that will give people insight into the project’s progress more.
Again, thanks to Engine Yard and everyone else for wonderful support.
The Rubinius Summer
Hi everyone. Been too long since the last update, so wanted to get everyone up to speed.
Rubinius
Things have been a little quiet on the Rubinius front, as I’m sure a lot of you have noticed. We’re still hard at work, currently getting the new C++ VM into shape.
This new C++ VM fixes a lot of fundamental problems the shotgun VM had (type safety, expression ordering, etc), which is a major reason we’re migrating our work to it.
Things have been a little quieter, commit wise, as the rest of the team gets up to speed on the new VM that I’ve been working on. Shotgun has been put into maintenance mode, with updates to the current main coming mainly in the form of bug fixes to the kernel.
I know that the advances we’re making in the new VM everyone will love, from more performance to less crashes to better code organization.
Comic-Con
I’ve just returned from Comic-Con, having spend 4 days in the sun down in San Diego with the rest of nerdom.
It was quite a fun con though quite tiring. We managed to get into some good panels, but didn’t make it into the Hero’s and Lost panels. Seems you had to arrive at 6am to even attempt to get a seat. The line to get in was literally a mile long (no really, I’m not kidding.)
Conferences
Since we’ve been hard at work trying to get Rubinius to 1.0, I haven’t done too many conferences this summer. The next one I’ll be at is Lone Star Ruby Conf down in Austin. Should be fun, I’ve never been to Austin before and people seem to like the city.
Rubinius version 0.9.0 released!
I’m super proud to say that we’ve release version 0.9.0. It’s a snapshot of the work we’ve already been doing, but we’re trying to formalize our releases a bit more.
We’re going to be doing another release, 0.10, next month, as well. We’re working to do more releases, more often.
