GSoC2013 Summary: Right-to-Left Support in VisualEditor

If you’ve read any of my previous posts, you should know by now that I have been participating in the Google Summer of Code internship, working for the Wikimedia Foundation. And now, as the summer has ended, so has this internship, and it’s time for a summary. So here it is in a crunch:

thatwasawesome

Okay, yeah, this can’t be the real summary, so let me try and summarize this tremendous summer adventure with a bit more content.

Wikimedia Foundation’s VisualEditor

This summer, I worked for the Wikimedia Foundation, which is the organization behind the MediaWiki platform that is the system behind the worldwide Wikipedia websites. More specifically, I worked with the VisualEditor team.

VisualEditor is a highly anticipated upgrade to the way Wikipedia articles can be edited. Ever since MediaWiki (and Wikipedia) was born, editing involved using a special syntax called “Wikitext”. The systax is relatively straight forward: Wrap text with ‘===’ to get a heading, or ”’ to get a bolded text. Start a line with a star (*) for a bullet list or a pound (#) for a numbered list. The idea was to help editors write articles simply without having to know the elaborate functionality of HTML tags.

And for the most part, it worked, rather beautifully, even. It’s all about accessibility; Wikipedia has evolved into the biggest encyclopedia, making it possible for anyone to edit and add to its growing body of knowledge without having any technical know-how.

But as Wikipedia grew in size and in articles became more elaborate, so did wikitext. It evolved to include templates and styles. Wikitext evolved with the community to allow for more and more possibilities and complex usage. Writing articles involved learning a new syntax, and while it was easier than the alternative HTML tags, it still has a learning curve, especially for new users to learn, and even users who are familiar with it sometimes have challenges using it properly. But the biggest issue (at least as far as I am concerned) is using it with right-to-left languages.

The issue of Right-to-Left languages

Wikitext is plain-text code that is left-to-right, so writing articles in right-to-left language means mixing directions, and that’s a recipe for trouble. Symbols like pipe “|” colon “:” or parentheses “(” and “[” flip the sentence around when the sentence has mixed directions. You can read about some of those challenges in my blog post about that.

VisualEditor: The Solution

That’s where VisualEditor comes in. Instead of having to type articles using wikitext syntax, the idea is to have a visual interface that allows anyone to add content into articles easily, like a “WYSIWYG” system or a Word processor. This serves the purpose of making it easier for everyone to edit articles, even without knowing wikitext or being intimidated by weird symbols in the text editing area — and also help right-to-left language speakers to edit articles without bidirectionality frustration. And, also, it’s a lot more convenient to edit articles the way they are supposed to look than their source code.

Project Summary: Right-to-Left Language Support in Visual Editor

So, my project involves adding in language and right-to-left support, both in terms of fixing up bugs that prevented VisualEditor from being used on right-to-left Wikis, and also developing tools to help editors handle languages when editing wiki articles.

Language Inspector

The Language Inspector adds a “Language” button to the toolbar that allows users to mark certain text as a particular language. The code uses UniversalLanguageSelector to provide visual GUI for language selection and relate the language to its directionality.

A screenshot of a draft version of the VisualEditor language inspector. Credit: Amir Aharoni

TemplateData Generator

A MediaWiki extension that provides visual GUI for editing json-based <templatedata> tags.

TemplateData is a json string that is used by VisualEditor to recognize the elements and parameters of MediaWiki templates. Since templatedata is json string, it is not easy for users to manually edit it — especially so in right-to-left environment, where the mix of english-based json with rtl-parameters can make the editing process incredibly challenging.

TemplateData Generator allows users to use the GUI to either create the templatedata json string from scratch, by importing parameters, or to edit an existing one, without having to edit the text itself. The extension is also planned to be merged into an existing TemplateData extension that is deployed on the international Wikipedia.

TemplateData Generator with Hebrew parameters

Bug fixes

On top of those two projects, I’ve also spent some time fixing up right-to-left related bugs, some of which were blocking VisualEditor from being used in right-to-left wikis:

It. Was. Awesome.

So that was the actual work, the “deliverables” that resulted from this great internship. But there were quite a lot of other beneficial results. Here’s a couple of things I’ve learned.

Programming Collaboratively

Since MediaWiki is a collaborative project, I learned a lot about how to write code with other people. Up until this project, my programming was mostly by myself. Sure, I’ve cooperated on code before, but it was mostly involving a tiny group of people, or it was me getting into someone else’s code and sending in a patch request. I knew about the idea that code can actually be worked on — and reviewed! — collaboratively, but it’s been completely different actually doing it.

I’ve learned that the industry relies on communication and cooperation to produce working programs, and while there are teams that do work on different aspects of these products, cooperation over making them work means there is “fresh eyes” going over the pieces, improving the system in general. Cooperating on code not only made the code I worked on be better, it made me be a better programmer.

Communication

I learned to communicate the problems I encounter and the solutions I come up with clearly. Sometimes, while thinking about how to explain the problem I got stuck with, I realized I might have a potential solution. Explaining to others clearly meant understanding things better for myself.

Since I speak Hebrew and my work involves Right-to-Left issues, I also tried to be in constant communication with the user base – especially the Hebrew Wikipedia – to hear about the bugs and problems that frustrate them. In that aspect, I had to learn how to translate the user complaint into a technical requirement and vise versa – when technical features came out, it was helpful to communicate them back to the users for their comments.

Programming Properly

I started this internship feeling rather insecure about my programming ability.

I’ve been programming various languages since I was rather young, but it was always more a hobby than an actual profession. I could make programs work, and for the most part, they work just fine, but this project involved getting into a really big, complex codebase and starting to make considerations that I’ve never had before. This project made me start considering not just working code solutions, but solutions that are efficient, that work with long-term product strategy, that work consistently, that survive millions of users and provide proper usability in wide variety of systems.

I wasn’t sure I can do that, but thanks to extremely talented and helpful team and mentors, and passionate users who pointed out the challenges they wanted fixed, I delved in, and I managed.

I think I did more than just managed, but the best thing of all of this, as far as I’m concerned, is that I’ve learned — a lot — about how to do things right. Really right. And I tell ya, I had a blast learning it.

Advice for future Google Summer of Code interns

If you’re going for a Google Summer of Code project, you know you’re going to work with Open Source companies, and if you really want to contribute and also get the most out of the internship, here is my two cents’ worth:

Don’t just Do it — Own it

If you want to really learn and get the best out of Google Summer of Code, you should own your project.

Find bugs that are related to the system you’re working on and fix them. Get in touch with the user base and encourage them to talk to you, be bold, ask questions, bug people on IRC and the mailing lists, try to see if there are parts of your projects that touch other related projects and go bug them, too.

Utilize the Community

I think that’s the best thing I’ve learned from this internship and the best advice I can give to future interns: Open source projects are a community.

You don’t just collaborate on code, you work together, plan together, and help one another.

Don’t be shy, don’t be mute, and don’t work in a bubble. Communicate, cooperate, discuss, do NOT let yourself be frustrated over problems for too long — the community is right there, waiting to be challenged with questions.

Do things right

Don’t just program – program with the open source code in mind. Document your code, learn the organization’s code style, ask for reviews and try to write code that other people can understand and tweak.

Don’t be intimidated. It’s worth it, really, because before you know it, people start sending others your way for answers on the things you’re working on.

And that’s the Best. Feeling. Ever.

So, yeah. It. Was. Awesome.

Thanks and Appreciation

So, as I sum up this incredible summer, I want to finish up with a thank-you list. This list is not complete, not even by a long shot; MediaWiki is a huge project with lots of volunteers, and many of them affected me and the progress of this project. But I will point out a few of the more notable groups of people who helped me this summer, and to which I am incredibly thankful.

You guys rock, y’all. You made this summer awesome, left to right and right to left!

So, Thank You to —

My mentors, Amir Aharoni and Inez Korczyński, for being incredibly accessible, supportive and helpful. Thanks, Amir, for encouraging me to take this project head-on despite my doubts about my abilities, for helping me see the big picture and what I need to look for, and, most of all, for making me feel like you’re actually enjoying the fact I constantly bug you. Thanks Inez for helping me get into a rather complex code and understand it enough to work with it, and for not letting me panic when I thought I broke things. Also, for not letting me break things too badly.
Timo Tijhof and Roan Kattuow from the VisualEditor team, for not only being experts, but allowing me to squeeze their expertise over and over again, and for being incredibly patient with my “wait wait wait, what?” baffled moments. Timo, thanks for showing me (by example and by code comments) what to look for and how to consider strategizing code, and for helping me strive to produce actually good code that others can use and work with. Roan, thank you for your endless patience (and yes, I mean e-n-d-l-e-s-s), for helping me get into the codebase, and for helping me climb out of the pits of panic when I managed to screw things up, and thank you both for making me feel like I kinda know what I’m doing, even when I didn’t.
James Forrester, VisualEditor product manager, for alleviating my worries and insecurities by being extremely welcoming, for making me feel like part of the team by assigning me bugs, by being there when I needed help (which was quite often, really,) by inviting me over to San Francisco to meet the team, and for offering me a job. That was awesome.
The VisualEditor superteam – Trevor Parscal, Ed Sanders, Rob Moen, and David Chan, who were there for my questions even when they didn’t realize it. Also, for an amazing San Francisco visit!
To the Wikimedia Language Team, and their hard work, for making me realize that speaking and writing RTL does not an RTL-programmer make, and for the absolutely rawkin’ UniversalLanguageSelector.
Sumana Harihareswara and Quim Gil without which I would not have submitted a proposal on time, or at all, for being totally welcoming for newbies like me, and for the great support throughout!
The Hebrew Wikipedia editors and system managers and users, for complaining and complementing and adding bugs to the list, so we can deal with them and make RTL support in VisualEditor awesomer and awesomer.
… And anyone and everyone who I interacted with this summer, from Parsoid to Vagrant to the various office hours and late-night-a-la-early-morning banter in IRC.
And to my parents, who brought me this far (this works better in Hebrew)