Archibald

Almost five years ago, at the traditional KDE PIM meeting at Osnabrück, I drew the first version of the Akonadi architecture on a whiteboard. It felt appropriate to put the server into the center and arrange the different layers of the system around it in circles. The result was a beautiful diagram.

Trouble stroke, when I tried to create a digital version of the diagram. Putting texts on rounded shapes and arranging partial circle sections in a nice way was too much for all vector and diagram drawing applications I tried. So as a hacker, how do you solve problems? Right, you do it in software. So I wrote an editor for the diagram in Qt.

The mission was accomplished, the diagram up on our web pages. So the code for the diagram tool got buried under a pile of other stuff on my harddisk. But last winter, again at the traditional KDE PIM meeting at Onsbrück Steve reminded me of the tool by showing me an updated version with the then current state of the diagram, and a bit later at FOSDEM Paul complained that Kolab didn't show up in the first version. So there clearly was a need for an updated version of my tool.

After a little bit of hacking the tool is now able to handle multiple versions of the diagram, and here is a view of Kolab in the Akonadi architecture.

While hacking on the tool I also moved it into a git repository and published it on github. So if you want to have look, or if you feel inclined to play around with the Akonadi architecture, go ahead and clone Archibald. Let me know, if you do something interesting with it. I'm curious.

ownCloud in a box

I had some free minutes today at Akademy, so I decided to build an ownCloud appliance. I did a few iterations in SUSE Studio and there you go: ownCloud in a box.

The screenshot shows the ownCloud web interface running in a testdrive on SUSE Studio. The appliances is available as live CD, USB image, virtual image for VMware, VirtualBox or KVM, or as disk image. This should make it really easy to run your own ownCloud server for just having a look or for quickly getting an instance up on some computer.

It's a first version, so don't be too surprised, if there are still some issues. Feedback is welcome as always.

Akademy 2010

I'm on my way to Tampere for Akademy 2010. It's my seventh Akademy, not counting Kastle in 2003, the first KDE conference, which became the predecessor for all the Akademy events following afterwards. I'm especially looking forward to Akademy this year, because there are so many interesting topics to discuss, which could make a big difference for KDE in the future. So here are a couple of the things, which I'm particularly interested in this year:

Conference: The conference program is excellent as always. It's great that there is a strong mobile track. This is a still uderused potential for KDE, and I would love to see KDE expand much more in this area. Many years ago I wrote KOrganizer/Embedded which ran on a PDA, but I'm still waiting for the moment where I have a device running KDE software, which I actually want to have with me all the time.

BoFs: The BoF sessions are always a very important part of the Akademy experience. This is where actual work gets done, and where plans are forged. I'm involved in running three BoFs this year about KDE documentation, one about the supporting membership program, and a session about KDE e.V., which the board would like to use to answer questions about KDE e.V. and discuss what people expect from KDE e.V., and how we can improve and excel. They all will be interesting and I'm of course looking forward to attend some other BoFs as well.

Meeting old and new friends: KDE is an amazing community. Akademy and other meetings always feel like meeting friends, even if you haven't ever seen the other people before, and maybe just exchanged a couple of emails. So I'm looking forward to meet older gentlemen as well as the young and wild new members of the community.

Thinking future: Akademy always is a place to think about the future of KDE, of our community, our software. This year this is particularly interesting. There are a lot of interesting developments, be it on the hardware side, where devices which could run KDE software are becoming much more diverse, or be it on the software side, where web applications and the associated technologies, or new platforms like MeeGo or Android are having lots of momentum. So I hope we can have some serious and creative discussions about how and where KDE can find its place in this universe today, and in the coming years. Our potential is huge, bit we also have to make use of it.

There certainly is lots more exciting stuff happening and going on at Akademy. Let's see what it brings. And now...

Hackweek V: Graphical SUSE Studio Client

Hackweek V is over and I can happily report success. My graphical SUSE Studio client is functional and has the first cool features implemented. The focus of the client is not to duplicate the functionality of the SUSE Studio web interface, but to provide those features, which are hard or not possible to do on the server. This is stuff like managing downloads, running and deploying appliances locally, or native access to testdrive. From this list I got the native testdrive done. On the click of a button, the client starts a testdrive on the SUSE Studio servers and then connects with an embedded native client. This gives great performance and a very smooth and integrated experience.


The client uses the HTTP API of SUSE Studio to talk to the server. This way it gets the list of appliances and builds of a user and can operate on this data. It does some local caching of results of HTTP request in order to provide a very responsive user interface. The client is implemented using the KDE development platform, which provides all the bits and pieces to do the UI, to perform the HTTP communication, and also the embedded VNC viewer. It's pretty much standard technology, but I'm still amazed of how well all these pieces fit together and how easy it is to write such a client, which allows to move into areas, which aren't covered by the web application.


One detail of the implementation is particularly interesting, that's the XML parsing part. The REST-like API, which uses XML over HTTP, obviously involves quite a bit of XML parsing. Usually this is tedious code to write. This always annoyed me, so I wrote a tool to generate the parsing code, KXML Compiler. It's one of my hobby projects I work on from time to time since a couple of years now. It takes an example XML file and generates C++ classes from that, which represent the data, including parsers and writers from and to XML. So the application developer gets a nice native API to the data represented in XML without having to write any code. The tool works well for my use cases, but of course there are quite some situations, where it could produce better results or where it doesn't cover everything, so it sometimes needs fixes and improvements when taking on new data. It's an ongoing project and quite some fun as it involves some mind-twisting metaprogramming. The code is hosted as part of the Kode repository on github.

The code for the client is hosted in the Studiosus repository on gitorious.org. Some more information about the development during this hackweek can be found in my Hackweek blog.


Hackweek was fun. I'm happy about the progress I made. It also was great to see some of the other exciting projects which were done during this week. This is a fantastic way to stimulate innovation. I'm really looking forward to see more of what came out of this week and what future hackweeks will bring.

The quest for the perfect permalink

For SUSE Studio we are looking into adding nice permalinks to appliances. This turns out to be an amazingly difficult problem. The implementation is not too hard, but getting the scheme of the links right poses quite some interesting challenges.

So what do I actually mean by permalink? A permalink is a nice and convenient way to point to objects on a web site from outside of the web site itself. In our case this would be links which point to appliances on SUSE Studio. To make this nice and convenient the link needs to have a couple of attributes:

• Permanent. The link should not change or depend on the state of the site or attributes of the user session. If you publish the link on another web site, e.g. in your blog, it should not break after a while or for other users.
• Pretty. As the permalink is meant to be suitable for publication, it should have a pretty format, so that you can integrate it into text without completely destroying formatting and flow.
• Expressive. When you see a permalink, it should be recognizable where it points to, so you don't have to click it to find out what it actually is about.
• Short. For sharing the link it's nice, if the link is short. This is especially important when there are limitations for the length of the link, for example when sharing it via Twitter.

Meeting all these requirements is not easy, but there are also some additional challenges:

• Handling change. The objects permalinks point to are being worked on, so they change in various ways. For example the name of an object could change. Permalinks have to handle this conflict between permanence and change in some way.
• Namespacing. A site might handle different types of objects, so they need to be addressed in a way which doesn't cause conflicts. As we are talking about user-provided content here, there also is a cause for conflict by different users trying to use the same names. So we need to do some namespacing to handle these conflicts.
• Potential abuse. The permalinks are pointing to user-provided content. So depending on how much influence the user has on the link, there might be some potential for abuse by users who try to create links which misrepresent the site.
• Non-ASCII characters. If you base permalinks on names, you have to deal with characters which are natural in names, but not in URLs. This can make it hard to create permalinks.

Let's use a fictive example to illustrate the requirements and challenges:

http://www.flickr.com/photos/kimberlykv/ / CC BY 2.0

John Doe likes baking cakes. He also likes to share, so he publishes his recipes on example.com. His favorite recipe is the chocolate cake of his aunt Tilly. So he publishes it and the site creates the permalink example.com/chocolate_cake. John tweets the link. His friends get it, bake the cake, and everybody is happy. This permalink is short and pretty. It conflicts with all other recipes named chocolate cake, though. So the first to publish a recipe wins. This is good for John, but bad for other users, so not a perfect solution.

One way to avoid the problem of conflicting names would be to add a namespace for the user, so the permalink would become example.com/jdoe/chocolcate_cake. It makes it longer, though, and there still is the potential for conflicts in the user name. So when John's sister Jane Doe joins example.com, she'll not be able to use her favorite user name, which also is jdoe, but has to choose something else. Still not perfect.

Now aunt Tilly is a modern lady. She reads the tweet and sends an email to John: "Hi John, I gave you this recipe. Be a good boy and mention this on your web site. All the best, Tilly". John is a good boy and changes the name of the recipe to "Aunt Tilly's Chocolate cake". The web site creates the permalink example.com/aunt_tillys_chocolate_cake. This makes aunt Tilly happy, it's still expressive, pretty and relatively short, but it breaks the link in John's tweet. So the site has to redirect the old link to the new one. It at least has to prevent that the old URL is used for something different. It makes the nicer URL example.com/chocolate_cake unavailable for other recipes in any case. This is good for permanence, but bad for pretty and short links.

Another problem which is illustrated by the name change is handling of special characters. The apostrophe is hard to handle in an URL, so the site just removes it for the link. You can come up with all kind of rules to handle these special characters, but they will eventually fail to generate pretty URLs, e.g. when somebody uses a Japanese name. This means that either the user edits the link, which introduces lots of opportunity to change and the problems associated with it, you give up on pretty URLs for at least some cases, or you let users deal with the problems of encoding special characters in URLs and the issues you can run into, e.g. when using tools which don't properly handle all of this. Another stumbling block on our quest for the perfect permalink.

An easy solution to avoid most of these problems is to generate random permalinks. This also removes all complexity with user-editable content and changes, as the the link is independent of the content. So aunt Tilly's yummy chocolate cake would be referenced by example.com/3hd63lbdxz. This is short and permanent, but not pretty or expressive.

You can think of various variations and combinations of these schemes, but meeting all requirements really is very hard. Seems like there is no perfect permalink. But let's look at some real-world examples.

Real-world examples

Wikipedia provides short and pretty permalinks. They have the advantage that the number of terms represented in links is limited, pretty well-defined and not completely up to users. They still have to deal with conflicts and do that with their disambiguation pages. They take on the challenge of encoding special characters, which is nice.

Gitorious lets users choose the permalink (or slug as they call it). They forbid special characters. The permalink becomes a top-level path, which is nice. You can't name your project login, though. You can change your slug, but this breaks old URLs.

Github goes a slightly different path. They prefix all projects with the user name. This is nice as it avoids conflicts and it also stresses the social aspect of the site. Some URLs become a bit ugly, e.g. github.com/rails/rails. They seem to cleanup their users and projects from time to time, as a nasty URL, which used to exist, I wasn't able to find anymore today.

s.opensu.se is a site to provide links to various openSUSE resources. You can for example reference repositories by short links like s.opensu.se/r?network:utilities. This is nice and short and reasonably expressive. If it's pretty is a bit a matter of taste. It doesn't address changing links.

Markmail is an example for random permalinks. This is probably the only way to cope with the number of objects they manage as a mailing list archive and the links are still short and relatively pretty: markmail.org/message/vyjutm3jkecxprzj. Change is not an issue for them as objects are static.

There are tons of other examples out there. If you know of one which solves the problems of permalinks in a particular good or interesting way, please let me know.

What do you think?

My preliminary conclusion is that it's probably impossible to come up with a perfect scheme for permalinks, and we need to do some compromise. I like including user names in the URLs as it solves some of the conflict issues, is actually useful information, and makes for expressive URLs. But of course there are other solutions as well.

What do you think? How would you like permalinks to SUSE Studio appliances to look like?

Are you up for a new challenge in the SUSE Studio team?

Last time I blogged about open positions in the SUSE Studio team we were just preparing the first public alpha of SUSE Studio. We were excited about our application, but we didn't know what users would say. Now we are running SUSE Studio Online with more than 50.000 registered users. We have released an onsite version as part of the SUSE Appliance Toolkit, have won awards, and we get a lot of fantastic feedback. We have achieved a lot. To sustain this growth and success we are looking for some smart people to join our team. This could be you.

We love to learn, we like challenges, and we are passionate about great user experience. We are a tightly knit team of a diverse set of people spanning the whole range of development, from user interface design over Rails and system programming to QA. We have people who just started their career and others who have decades of experience in the business. Technology, innovation, open source, team work, and meeting the goals of our users are important to us.

If you are interested, look at the job descriptions. We are looking for user interface designers, web developers, Rails experts, and all-rounders with a hang for backend programming. You will learn, you will grow, you will be working on exciting software. Apply now.

Anatomy of a developer sprint

Two weeks ago I was at the annual KDE PIM meeting at Osnabrück. It was the eighth time that this meeting took place, and it was a blast, once again. This is amazing, so I'm taking the opportunity to reflect a bit on what makes this meeting so successful, how it evolved over time, and what we all can learn for running great developer sprints.

It all started in 2003. KDE PIM as a group didn't really exist yet, the kdepim module in CVS contained only a small collection of apps, mainly KAddressbook and KOrganizer, and some prototypes for what would become Kontact. The work on the Kolab server and clients had just started, and Intevation who was doing the project management for the Kolab project invited key people of the community and those who were working on Kolab to Osnabrück for some serious discussions, hacking, and community building. It was the first weekend in January and this date should stick as the traditional date for the KDE PIM meeting for coming years.

For many of us it was the first time we met each other in person. It's always amazing how much of a difference this makes. Of course we knew each other via mailing lists, IRC discussions or working together on code, but meeting face to face added another dimension. The astonishing thing was that it felt like meeting old friends. This is an experience I made again and again when meeting KDE people in person for the first time, and I know that many others have made the same experience. What a motivation!

One result of the first meeting was that KMail was moved from kdenetwork to kdepim. This was the birth of the KDE PIM community as we know it today. Even if it were mainly technical details which motivated this move, it had a big effect on how we felt as a group. The second big topic was the integration of the work which had been done on the Kolab client and how to combine that with the efforts going into a brand new application called Kaplan, which would later be known as Kontact as the central application of KDE PIM. So bringing all these people and different code streams together formed a new part of the KDE community, which had an unprecedented focus on creating great PIM solutions.

You see that already one meeting of the right people in the right atmosphere can make a big difference. But for KDE PIM it should quickly evolve into a tradition and constant source of energy to have these developer sprints. Over the next years we kept the annual meeting in January at Osnabrück and Intevation continued to kindly act as a host. This made organization of the meetings very easy and thanks to the support of KDE e.V., which provided us with financial support for travel and accommodation, we were able to get together most of the central people each year. In addition to the annual meeting some spin-offs started to happen, meetings at Akademy and the wonderful KDE PIM meeting in the Netherlands, and as we consciously always invited new people to the meetings they also became a source of fresh talent and energy.

While we developed some routine in having these meeting, in 2006 something special happened. Till came up with the idea for a new backend for storing PIM data. As we all felt the pain of the limitations of current solutions, this sparked a lot of interest, and over the time of the meeting we put together our combined experience in laying the foundation for this new framework.

We intend to design an extensible cross-desktop storage service for PIM data and meta data providing concurrent read, write, and query access. It will provide unique desktop wide object identification and retrieval.

Akonadi was born.

With Akonadi a whole new world was opened and the nature of the meetings changed a bit. They had grown in size and scope and while the early meetings were dominated by ferocious hacking, other activities like coordination, learning from each other, and strategical discussions became more important and took a bigger share of the meetings. So as a complement to the annual Osnabrück meetings we introduced a new set of Akonadi sprints as very focused small events dedicated to writing as much code as possible, taking advantage of having the central developers in the same room for an intensive weekend.

So where are we today? We just had the eighth Osnabrück meeting, the Akonadi port of all of the KDE PIM apps is landing in trunk for the KDE 4.5 release. Kontact as the central product of the KDE PIM community is running on an impressive bunch of different platforms, including of course Linux, but also Windows, MacOS and in the future also mobile platforms. The Kolab consortium is running commercial development based on KDE PIM for as long as the KDE PIM meetings exist. We have come a long way. This wouldn't have been possible without all the developer meetings.

Finally I would like to reveal one of the secret ingredients which makes our developer sprints so successful. It is having fun. Be it the snow ball fight before the group photo, the game of go after dinner, or meeting friends, having inspiring discussions, or the occasional grand hack. When you bring together passionate people with a common vision, who enjoy working on outstanding free software, great things happen. That's what we have our developer sprints for.