Wednesday, July 9, 2014

The one donation you will want to make today

In the first week of August 45 KDE people will meet at Randa in the Swiss mountains. They will spend a week of their free time and an uncountable amount of passion and dedication to work on free software. It needs money to bring them together and make the best out of their energy. You can help. We are running a campaign to make this happen. It ends today. Please donate now.

I had the opportunity to be at Randa in 2011. I have been at lots of KDE sprints over the years. Randa is one of the very special ones. There is an amazing level of energy, the buzzing atmosphere of getting things done, a deep sense of purpose. Randa is a good place to create free software.


Part of that is the environment. In the middle of the mountains with not much around than the impressive nature of the Swiss Alps, you feel physically focused on what's important. Everybody is living in the same house for a week, eating, sleeping, and hacking. There are no distractions, there is a quietness which is inspiring.

Another part is the deep commitment of Mario, the organizer of the meeting. He puts in a lot of personal energy. He even dragged in his family and friends. He equipped the house with WLAN. During the meeting he tries hard to create the best possible environment for all the volunteers who come to Randa, so they can focus on creating free software and all what is around that.

With this in place, magic happens. KDE Frameworks 5 was started at Randa. The famous tier diagram was created there. One of the projects I have been working on, Inqlude, originated there. Sebas came up with the name, the idea was discussed and prototyped, and on the train home I wrote the first version of the web site, inspired and motivated by the energy from the meeting. Lots of other good stuff originated from Randa.


All this is only possible with the help of all of you. Many people put in their passion, energy, vacation, free time. But it also needs money to bring people together who otherwise couldn't afford it. You can help with a donation.

Are you a KDE user? Do you use KDE software for work or leisure? You can help the community to sustain the development of the software you use. You can give something back with a donation.

Are you a KDE contributor or have been one in the past? You have experienced what a difference meetings such as the one at Randa can make. Maybe you have met your employer or your employees at a KDE sprint. Maybe you started as a student in the KDE community and now have found your dream job as a software developer. You know what it means to learn and grow in the KDE community. You can help, you can give back, you can contribute with a donation.

Do you care about freedom? You want to be in control of your software and your privacy. You want to be able to study your software to see what it does, be able to change it and help others by giving them the changes as well. KDE is committed to this for more than 17 years now, to protect the freedom of users and contributors, and give access to great technology to everybody. Eva Galperin said in her Akademy keynote last year: "You are the developers. You are our last and only hope. Save us". That's what meetings such as Randa help to do. You can support it with a donation.

Please donate now.

Tuesday, June 3, 2014

How to write unit tests dealing with files

I have written quite some Ruby code recently which worked with files in a file system somehow. Reading, writing, manipulating data stored in a file, or in a set of files in a directory, or in a set of directories, or one of the countless variations of this. There is a lot of code like this, and one question always is: How do I write unit tests for this code?

You need test files as input for tests. You need space to write files, which is cleaned up after the test. You often have series of tests which use similar but not quite identical sets of files. Doing all this in unit tests is inconvenient, as you usually can't just do it in the test code, but have to interface with some actual file system.

One approach is to stub all the interfaces dealing with the file system and fake them with test code, which mirrors the desired environment without ever hitting a real file system. FakeFS is one low-level way of doing this. You can also often do something similar on a higher level stubbing functions of your own code, when interaction with the file system is well encapsulated.

But this approach often is too limited. When you are using external libraries or calling external tools, you don't have access to the code interacting with the file system, so you can't stub it. You also limit your test coverage, as the actual interaction with the file system usually is an essential part. Finally it can be quite some work to isolate, separate, and fake the code, which interacts with files.

So we do want to have a better solution, right?

My solution is GivenFilesystem. It is a set of helpers for use in RSpec-based tests. It also provides a basic class, if you prefer more direct access or want to integrate it with some other test framework.


GivenFilesystem provides a simple and convenient way to create test files and directories. These can be structures as complex as needed. They are built up from test data you store as part of your tests. But you only need to store the data once, and create different scenarios using this data in your unit test code. GivenFilesystem also provides an easy way to create temporary locations in the file system for writing files. These are automatically cleaned up after the tests.

Here is an example of some code creating test files (see the details in the docs):
path = given_directory "mydir" do
  given_directory "one" do
    given_file "myfile"
    given_file "myotherfile"
  end
  given_directory "two" do
    given_file "myfile2", :from => "myfile"
  end
end
The only assumption is that you have a way to specify the path your code uses to read and write files. There is no magic behind the scenes, which replaces internal code accessing the file system, but your code has to be structured in a way that you can tell it where files are. Usually this is something you want to have anyway, and which gives nicely structured code.

So if you come to the point, where you need to set up files for unit tests, or worry about cleaning up files written by your tests, consider giving GivenFilesystem a try. It has made my life a little bit more simple and convenient, maybe it can do the same for you.

I put some effort into polishing GivenFilesystem. It comes with tests, has full test coverage, and clean code. This turned out quite nicely, also thanks to the magic of code review.

Get the gem, read the docs, get started.

Feedback and other contributions are always welcome. Or if you just want to talk about it, simply drop me an email.

Thursday, May 22, 2014

My first patch

It was 1999, when I submitted my first patch. I had started to work with Qt at my job at the university, and was intrigued by it. Programming with Qt was fun and exciting. It was C++ as it was meant to be, and you could do graphical user interfaces in the best way possible at that time. I had heard about the ideas of free software, and of course used a lot of free software already. So I was interested to try writing some myself.

Not really being content with the software I had on the HP-UX workstation I used for most of my work, I thought it would be a good idea to get organized a bit and write a calendar application. A colleague of mine had told me about KDE, which was the shiny new thing back then. So I decided to write "KOrganizer".

Looking a bit closer I realized that KOrganizer was already there. That was a nice surprise. I checked out the source code from CVS, compiled it, ran it, and it crashed immediately. So I began debugging and found a solution. I ran it again, and it crashed later. This was progress, so I submitted my first patch:


As you can tell, I didn't really have a clue. I told the people on the developer mailing list how to change the one line, which caused the crash. I didn't know about diff and patch. But the patch was accepted and I was hooked.

I continued to work on KOrganizer and was blown away by the community. The people were helpful, passionate, and excellent in what they were doing. It felt like meeting old friends, although we didn't really know each other, and mostly only communicated via the Internet. Personal meetings came later, and the feeling of meeting friends has never gone away. It's part of the magic of free software.

Over the years I wrote a lot of code, maintained frameworks and applications. I learned a lot. I grew into the board of KDE e.V. and am serving as its president now. I met a lot of people in KDE and in many other communities. I got a job working on and with free software, and I'm still doing it. It has been an incredible ride.

This is were I am, precisely fifteen years after I sent my first patch. I wouldn't be here if I wouldn't have sent this first patch. It was the beginning of a journey, which shaped my career and my life. I'm grateful to all the people which I met on this journey and for all what free software gave me.

When do you send your first patch?

Thursday, December 12, 2013

ownCloud 6 on openSUSE 13.1


ownCloud and openSUSE are two of my favorite projects. After they both have released major new versions I have updated my ownCloud in a box appliance now, so that it's easy to get the latest and greatest what ownCloud and openSUSE have to offer. The appliance is built with SUSE Studio as usual and comes in a variety of formats to allow installation on physical and virtual systems, running it from live media, or deploying it to the cloud. Check it out on SUSE Studio's gallery.


One of the big new features of ownCloud 6 is ownCloud Documents, a collaborative editor based on ODF. This is a pretty cool feature as it allows groups of people to work together on a document with rich text. The best is that it's all free and open and you can run it on your own server so you have full control about your data.


There is much more. Read on about what's new in ownCloud 6 and what's new in openSUSE 13.1. Congratulations to both projects for two fabulous releases. I'm looking forward to more to come.

Sunday, November 10, 2013

One place to collect all Qt-based libraries

A few weeks ago, during SUSE Hack Week 10 and the Berlin Qt Dev Days 2013, I started to look for Qt-based libraries, set myself the goal of creating one place to collect all Qt-based libraries, and made some good progress. We had come up with this idea when a couple of KDE people came together in the Swiss mountains for some intensive hacking, and where the idea of Inqlude, the Qt library archive was born. We were thinking of something like CPAN for Qt back then. Since then there was a little bit of progress here and there, but my goal for the Hack Week was to complete the data to cover all relevant Qt-based libraries out there.


The mission is accomplished so far. Thanks to the help of lots of people who contributed pointers, meta data, feedback, and help, we have a pretty comprehensive list of Qt libraries now. Some nuts and bolts are still missing in the infrastructure, which are required to put everything on the web site, and I'm sure we'll discover some hidden gems of Qt libraries later, but what is there is useful and up to date. If some pieces are not yet, contributions are more than welcome.

Many thanks as well to the people at the Qt Dev Days, who gave me the opportunity to present the project to the awesome audience of the Qt user and developer community.

Format


The first key component of the project is the format for describing a Qt-based library. It's a JSON format, which is quite straightforward. That makes it easy to be handled programmatically by tools and other software, but is also still quite friendly to the human eye and a text editor.


The schema describes the meta data of a library and its releases, like name, description, release date and version, links to documentation and packages, etc. The data for Inqlude is centrally collected in a git repository using this schema, and the tools and the web site make use of it to provide nice and easy access to users.

Tools


The second key component is the tooling around the format. The big advantage of having a structured format to describe the data is that it makes it easy to write tools to deal with the data. We have a command line client, which currently is mostly used to validate and process the data, for example for generation of the web site, but is also meant to help users with installing and downloading libraries. It's not meant to replace a native package manager, but integrate with whatever your platform provides. This area needs some more work, though.


In the future it would be nice to have some more tools. I would like to see a graphical client for managing libraries, and integration with IDEs, such as Qt Creator or KDevelop would also be awesome.

Web site


The third key component is the web site. This is the central place for users to find and browse libraries, to read about details, and to have all links to what you need to use them in one place.


The web site currently is a simple static site with all its HTML and CSS generated from the meta data by the inqlude command line tool. Contributing data is still quite easy by providing patches to the data in the git repository. With GitHub's web interface you can even do that just using your web browser.

Notes


There are a few things worth pointing out explicitly as I got similar questions about these from various people.

The first thing is that Inqlude is meant to be a collection of pointers to releases, web sites, documentation, packages. It's not meant to host the actual code, tar balls, or any other content belonging to the libraries. There are plenty of ways how to do that in a better way, and all the projects out there already have something. Inqlude is just meant to be the central hub, where to find them all.

Another thing which came up from time to time is the question of dependencies. We don't want to implement yet another package management system, or another dependency resolver. So there we rely on integration with the native tools and mechanisms of the platforms, we run on. Still it would be nice to express dependencies in the meta data somehow, so that you have an easy way to judge, what you will need to run a given library. We will need to find a way how to do that in the best way, maybe a tier concept, like KDE Frameworks 5 is using it, would do the trick.

Finally I would like to stress that Inqlude is open to proprietary libraries as well. The infrastructure and the tooling is all free software, but Inqlude is meant as an open project to collect all libraries which are valuable for users on the same terms. The license is part of the meta data, so it's transparent to users, under which terms the library can be used, and this also allows to categorize libraries on the web site according to these terms. There still is a little bit of work missing to do that in a nice way, but that will be done soon. Free software libraries of course do have the advantage, that all information, code, and packages, is directly available, and can be accessed immediately.

Future


There are a couple of short term goals I have for Inqlude, mostly to clean up loose ends from the work which happened during the last couple of weeks:

  • Collect and accurately present generic information about libraries, which is not tied to a release. This is particularly relevant for providing a place for libraries, which are under development and haven't seen a formal release yet.
  • As said above, the listing of proprietary libraries needs some work to categorize the data according to the license. Then we can display libraries of all licenses nicely on the web site.
  • Currently we have one big entry for the Qt base libraries. It would be nice to split this up and list the main modules of Qt separately, so it's easier to get an overview of their functionality, and use them in a modular way.

There also are a number of longer term goals. Some of them include:

  • Integration with Qt Designer, so that available libraries can be listed from within the IDE, and being used in your own development without having to deal with external tools, separate downloads or stuff like that.
  • Build packages in the Open Build Service, so that ready-to-use binary packages are available for all the major Linux distributions. This possibly could be automated, so that ideally putting up the meta data on Inqlude would be all what it takes to generate native packages for openSUSE, Fedora, Ubuntu, etc.
  • Integration with distributions, so that libraries can be installed from inqlude via the native package management systems. This already works for openSUSE provided the meta data is there, but it would be nice to expand this support to other systems as well.
  • Upstream the meta data, so that it can be maintained where it's most natural. To keep the data up to date it  would be best, if the upstream developers maintain it at the same place and in the same way as they also maintain the library itself. This needs a little bit of thought and tooling to make it convenient and practical, and it's probably something we only want to do when the format has settled down and is stable enough to not change frequently anymore.

There might be more things you would like to see to happen in Inqlude. I'm always happy about feedback, so let me know.

This was and is a fun side project for me. It's amazing what you can achieve with the help of the community and by putting together mostly existing bits and pieces.

Thursday, October 10, 2013

Inqlude progress

My Hack Week project is progressing. Qt Dev Days have finished. I presented Inqlude there and got lots of good feedback. I even won a prize for my lightning talk. Thanks a lot for that.



I got a lot of input on which libraries were still missing and have a pretty long list to process now. The patches I got on GitHub I have already merged. Tomorrow I intend to go through the rest of the list and add the missing data. This should get us a lot closer to the goal of having all Qt-based libraries listed in one place.

One question came up a few times. Do we also want to list proprietary libraries, which are not available under a free software license? The answer is: Yes, we do want to also list proprietary libraries. We already are collecting the license information for all libraries, so this would be just another entry in the license field.

Inqlude is meant to be an open system. The goal is to have all libraries listed in one place, which are part of the Qt ecosystem and can be useful for application developers. So it's consequent to list all libraries there, independent of if their source code is available, or if there is a commercial model behind them. We will show the license data and add a separate section on the web site, so that people can easily find what they are looking for according to their preferences in terms of licenses.

I'm looking forward to the next two days of Hack Week. This is a fun project, and it looks like we can reach a state where it actually will be useful for quite a number of people.

Tuesday, October 8, 2013

An audacious goal for SUSE Hack Week 10

I have set myself an audacious goal for the tenth SUSE Hack Week: Create a complete collection of all Qt-based libraries which exist.


There are many places where you can find Qt-based libraries. Qt itself already comes with a number of modules. KDE has created a rich set of libraries with additional functionality on top of that, and you can find a lot of other third party libraries on sites like Gitorious, GitHub, Google Code, and more.

Now getting an overview and access to all the Qt library goodness, which is out there, is not particularly easy. To solve that we started Inqlude as a project to create an archive of all available Qt libraries back then at one of the Randa meetings. I worked on it some more at Hack Week 7, and continued to spend a little bit of time here and there. We have reached a state now, where it starts to become useful, and so I thought Hack Week 10 is a great opportunity to fill in the missing bits and pieces, and make it ready for prime time.


We have the web site and a documented format for library meta data. We also have tooling to process the meta data, e.g. for updating the data on new releases or to retrieve packages of libraries for installation. The web site also gets generated with these tools.

One thing which is lacking a bit, is the packaging of libraries, and the integration of the tools with the native package management systems on various platforms. With the help of the Open Build Service the packaging part should be solvable. For the integration on different platforms help of users of these platforms would be greatly appreciated. There is some support for openSUSE, but for other Linux flavors, or non-Linux systems, there is still some work to do.

The other thing is the coverage of the archive. It already has quite a list of libraries, but there are more out there, and I really would like to see it to be as complete as possible. All libraries, which reasonably can be considered useful to developers using Qt, should be on Inqlude. So I'm looking for third party Qt libraries now.

I'll be on the Qt Dev Days in Berlin the next two days, and hope that I can use this opportunity to get some more input and pointers to libraries, we haven't covered yet. If you are there as well and have hints, please talk to me. I'll also give a lightning talk about the current state of Inqlude, so you can learn about where we are first hand.

So this is my project for Hack Week 10. You'll find some more details and updates on the progress on the Hack Week project page. If you want to join the project, don't hesitate to add yourself there and contact me]. We'll use the inqlude mailing list to coordinate work as needed.