Tuesday, July 5, 2016

Is my OpenStack ready for Cloud Foundry?

This year's first Cloud Foundry Summit took place in Santa Clara at the end of May. Beyhan Veli from SAP and I gave a presentation about the Cloud Foundry OpenStack Validator, a new tool we developed as one of the results of our collaboration with SAP on the BOSH OpenStack Cloud Provider Interface project.




The validator gives operators of OpenStack systems a simple tool at hand to check if their OpenStack installation is ready to run Cloud Foundry. OpenStack comes with a lot of options and flexibility, and not all configurations are equally suitable to run Cloud Foundry. The requirements are documented and there are instructions how to manually check them, but it requires quite some expert knowledge to get everything right.

The validator is supposed to encapsulate the expert knowledge and instructions. It checks all requirements automatically and gives operators a report with the information if requirements are met or what they need to change. Cloud Foundry makes it simple for developers to deploy their applications. We try to capture the same spirit of simplicity for operators of Cloud Foundry on OpenStack.

We are running a test pipeline for the continuous integration of the BOSH Cloud Provider Interface for OpenStack which is used in Cloud Foundry. This runs on SUSE OpenStack Cloud. It works smoothly now, but while getting there it would have been great to have the validator to have a simple check of the setup without having to go through manual deployment and testing and finding issues with the setup late in the process. We captured some of this experience in code, and it is much easier with the validator now.

The architecture of the validator is quite simple. It uses the same interfaces as BOSH and Cloud Foundry to access the underlying cloud infrastructure, so it closely resembles what is happening in production. Most of this goes through the Cloud Provider Interface. Some lower level checks directly use the OpenStack APIs.

Tests are written in RSpec, which gives a nice concise way to describe and run the checks which are run by the validator. The validator is provided as a standalone tool, which can simply be run on the command line and will print out a report with what checks it has run, if they have succeeded, and what needs to be done if something is missing. In the end it will answer the question, if my OpenStack is ready to run Cloud Foundry, and this hopefully will be a big green yes.

There is ongoing work to improve the validator, to add more checks, and make it more complete in checking the requirements of the full stack, from OpenStack, through the Cloud Provider Interface, to Cloud Foundry. This will be a continuous improvement process with new checks being added as they are found to be relevant when setting up OpenStack based Cloud Foundry installations. To make this easier and also allow to add custom checks, there likely will be some kind of plugin interface to add new checks to the validator.

As it is using the abstract Cloud Provider Interface it might also be interesting to use the validator with other cloud infrastructures, and we need to evaluate how it plays with other approaches of validating cloud infrastructure. The project is open, the code is published as open source, and we are welcoming feedback and discussions about its future direction.

It was the first time I attended a Cloud Foundry Summit, and I really enjoyed the event. It covered a broad range of topics, inspiration, user experience, deep dive into the technical foundation, science fiction, and much more. Good information, great inspiration, and meeting lots of awesome people, it was an excellent event.

The next summit will be in Frankfurt, Germany, in September. I'm looking forward to see you there.

This blog entry was first published on the SUSE blog.

Saturday, March 19, 2016

Students, join the Summer of Code!

If you are a student, you have a unique opportunity right now. Join the Summer of Code, submit a project proposal until March 25th, and work with the people from an open source community during the summer. You will get paid for three months, you will become part of a community, and you will have the chance to make an important step in your career as a software developer. Google is organizing and sponsoring the program and hundreds of mentors from all kind of open source projects all over the world are ready to help you to do your next step in open source. I really would have loved to have such a program when I was a student. You are living in good times.

I have been a mentor for KDE and openSUSE for many years and have worked with many amazing students. This year we made an extra effort in openSUSE and set up a new web site to help mentors and mentees to find together. It collects a lot of information about how you can succeed as a mentor as well as a student, and provides a portal for all the openSUSE project ideas people from the community have come up with. KDE also has a list of exciting KDE project ideas. Pick one or, even better, come up with your own.


There are so many interesting projects and lots of good students looking for working on them. While I try to help with the overall organization a bit, I limit myself to mentoring maximally one student during the summer, so that I can do a decent job with that and don't get stretched to thin. I have put up a couple of ideas in both projects, KDE and openSUSE. If you would like to help with mentoring one, you are more than welcome. This way we could be able to provide mentoring for more projects. If you are willing to help please get in touch with me. It is a great and rewarding experience to be an open source mentor.

The KDE project, where I'm looking for a student, is Inqlude, the Qt library archive. It's a nice project mixing an interesting set of technology, some Ruby, a bit of HTML and CSS, plus some C++ and Qt, and last but not least the opportunity to do some user interaction design. You will certainly learn something, and as it's a pretty small project within the universe of KDE, you will be able to make a significant impact during the summer and beyond.

In openSUSE we have a couple of projects around Machinery, our pretty new systems management tool for inspecting, inventorying, validating, and migrating Linux systems. It comes from openSUSE, but it also runs on and supports other flavors of Linux. It's Ruby, and one of the interesting aspects is that we are doing serious test-driven development, code review, automatic tests on all levels, and more what constitutes the craft of software development. You will learn a lot there. Mauro and Andre are available as mentors there as well.

There is one more project, I find really interesting. That's reproducible builds for openSUSE. The reproducible builds project coming from Debian aims at making it possible to prove that a binary is actually built from the sources it claims to be built from. This can help to make build systems more efficient, but more importantly it helps to protects against malicious modification of binaries during the build process. This can have a huge impact. Bernhard has started to do some work on it and by joining this you could make a real difference for free software by making sure users can trust in the software they run.

I'm repeating myself, but this is a wonderful opportunity for mentors and students alike. Don't hesitate, get your proposals together, and join the summer of code.

Monday, December 7, 2015

How to run Rails with PostgreSQL on openSUSE Leap 42.1

I wrote about how to run Rails with PostgreSQL on openSUSE 12.3 before. Things have changed since then. While Rails, PostgreSQL, and openSUSE are still excellent choices, new versions have been released. This warrants an update. So here is how to get a development environment of Rails with PostgreSQL running on the latest and greatest openSUSE Leap 42.1.

Install PostgreSQL:

    sudo zypper install postgresql-server postgresql-devel

Start the database server:

    sudo systemctl start postgresql

Enable server to be started on boot:

    sudo systemctl enable postgresql

Switch to the postgres user to set up the database:

    sudo su -l postgres

Create database user:

   createuser -d USERNAME

Return to your normal user, exchange the database driver in the Gemfile from sqlite3 to pg and run

   bundle install

Change the configuration of the database driver to something like:
  default: &default
    adapter: postgresql
    username: cs

  development:
    <<: *default
    database: APPNAME_development

  test:
    <<: *default
    database: APPNAME_test

  production:
    <<: *default
    database: APPNAME_production
Create the databases:

    rake db:create
    rake db:migrate
    rake db:migrate RAILS_ENV=test

That's it.

Monday, September 14, 2015

184 Qt Libraries

We have collected 184 third party Qt libraries on Inqlude now. This is a pretty complete map of the Qt ecosystem, quite an impressive number, and lots of useful libraries extending Qt for many purposes.


Inqlude is based on a collection of manifests. If you like to add or update a library, simply submit a pull request there. The inqlude tool is used to manage the manifests, it generates the web site, but you can also use it to validate manifests, or download libraries. There also is inqlude-client, which is a C++ client for retrieving sources of libraries via the data on the Inqlude web site. It's pretty handy, if you want to integrate some library into your project.

If you want to get a brief introduction into Inqlude, you might want to watch my award winning lightning talk from Qt Dev Days 2013: "News from Inqlude, the Qt library archive". It still provides a pretty accurate explanation of what Inqlude is about and how it works.


A big part of the libraries which are collected on Inqlude are coming from KDE as part of KDE Frameworks. We just released KDE Frameworks 5.14. It's 60 Qt addon libraries which represent the state of the art of Linux desktop development and more.

Inqlude as well as KDE Frameworks are a community effort. Incidentally they both started at a developer sprint at Randa. Getting community people together for intense hacking and discussions is a tremendously powerful catalyst in the free software world. Randa exemplifies how this is done. The initial ideas for Inqlude were created there and last year it enabled me to release the first alpha version of Inqlude. These events are important for the free software world. You can help to make them happen by donating. Do this now. It's very much appreciated.


One more recent change was the addition of a manifest for all libraries part of the Inqlude archive. This is a JSON file aggregating all latest individual manifests. It makes it very easy for tools who don't need to deal with the history of releases to get everything in one go. The inqlude client uses it, and it's a straight-forward choice for integration with other tools which would like to benefit from the data available through Inqlude.

At the last Qt contributors summit we had some very good discussions about more integration. Integration with the Qt installer would allow to get third party library the same way you get Qt itself, or integration with Qt Creator would allow to find and use third party libraries for specific purposes natively in the environment you use to develop your application. One topic which came up was a classification of libraries to provide some information about stability, active development, and support. We will need to look into that, if there are some automatic indications we can offer for activity, or what else we can do to help people to find suitable libraries for their projects.

It's quite intriguing to follow what is going on in the Qt world. As an application developer there is a lot of good stuff to choose from. Inqlude intends to help with that. The web site is there and will continue to be updated and there also are a number of ideas and plans how to improve Inqlude to serve this purpose. Stay tuned. Or get involved. You are very welcome.

Monday, April 13, 2015

Gone Hacking

This week we have Hack Week at SUSE. The whole engineering team works on projects of their choice during this week. Everybody is free to innovate, to learn, and to collaborate with others.


We are doing it for the twelfth time. Sometimes magic happens, sometimes people learn a new skill, sometimes we become smarter because we know one more way how not to do things. We always have lots of interesting projects. Hack Week is an amazing experience. You actually can join.

I'm part of the organization team again, providing the environment where our engineers can be creative, productive, have fun, and learn. If I find some time to work on my own project I will tackle the one I worked on last Hack Week already, Project MySelf.

I'm gone hacking now. See you next week.

Friday, April 10, 2015

55.555 downloads of ownCloud in a box

Recently I passed a magic number with my ownCloud in a box appliance. It hit the five fives, 55.555 total downloads, on SUSE Studio. It is the most popular download there. I would never have imagined that so many people would be interested in it and use it when I started with this. Amazing.


When Frank started to discuss the idea of ownCloud and announced it at Camp KDE five years ago (another five, hooray) I was immediately intrigued. The idea of giving people control about their data in the cloud was powerful. Over the years it actually has gained even more power and relevance, as not only the cloud has become ubiquitous but also the threats to abuse it.

With SUSE Studio it became so easy to build easily deployable images for a broad variety of targets, such as a live CD, a VMware image, or an installation disk for a physical machine, that I just had to do it. It follows along the philosophy of ownCloud to make it as easy as possible to run it, so many people can do it on their own. ownCloud in a box is an easy way to try it and get started.

I'm looking forward to the next fives, whatever they will be. ownCloud has great success, and its development is coming along nicely, but I would still like to see some more things.

From a technical point of view, client-side encryption would allow to use a hosted ownCloud without having to trust who is hosting the server. That would extend the benefit of controlling your data in the cloud to a whole new group of people, who can't or don't want to run their own server. There is some work going on in this area. Let's see where this goes.

From a community point of view it would be great, if the contributor agreement would be a bit more fair towards contributors. It's quite asymmetric. You broadly give all rights on contributions to ownCloud Inc. but only get back what you would get anyway by publishing your code under the AGPL. I understand that this enables the ownCloud Inc. business model, and that a successful company is good for the community. I also have no doubt that the company is operating with the best intentions for the community. But if there needs to be a formal agreement, it would be nice if it would better take into account the interests of both sides. I'm sure there would be ways how to do that without jeopardizing the business model.

In general I'm very impressed with how far ownCloud has come, and I'm happy that I could contribute my little part in it.

Saturday, April 4, 2015

The Space-Time Continuum of KDE's Activities

This is not about KDE e.V.'s new time travel program. This is about Plasma and its concept of Activities. They have been a topic of hot debates. Some people love them, some don't care. Björn called for finding a new metaphor which better fits the mental model of the user, so that Plasma's activities can appeal to a larger group of people. Here are my thoughts.

Orthogonal or hierarchical?

The premise of Björn's quest for a new metaphor is that the current one doesn't fit the mental model of the people using activities, especially how activities are related to virtual desktops. He sees activities not as an orthogonal concept to virtual desktops, but as an hierarchical one.

Matching the mental model of the user is crucial, because this defines how natural and easy it is to use them. You can learn abstract concepts and be trained in creating new mental models or in using rules to work with concepts without having a good deeper understanding. But this needs time and effort, which especially casual users are rightfully not willing to spend. And it's not fun.

Orthogonal concepts are independent of each other, they live in different dimensions, you can use one without affecting the other. Hierarchical concepts use the same dimension, but there is an order, one concept is above the other, such as one providing a grouping for another.

You could see activities as independent of virtual desktops, and that's how they are implemented. But there is one issue with that. The presentation we use addresses the same mental model, that of areas arranged in space.


That makes it hard for the user to know where things are. Does the screen present an activity or a desktop? Do I have to change the activity to go to my email client or do I have to switch desktops? When I go to the left or right, do I change activities or desktops? You can learn the concept of activities to be able to answer these questions, but if you use the same mental model for activities and desktops it can appear confusing and inconsistent.

I think there are two ways to address this issue. One is to clearly express that the concepts are orthogonal in the user interface and the naming. The other is to make activities and virtual desktops fit into the same mental model by clearly separating their responsibilities, and eliminate the metaphors which use the same mental model for different parts of functionality.

Time

One powerful way of expressing the orthogonality of activities and virtual desktops would be to represent virtual desktops in space and activities in time. These are two different dimensions, and it actually would go quite naturally with what the two concepts represent.

Virtual desktops are different segments of the space of your desktop. They are usually arranged in a plane and there is a relation of left/right and above/below. Desktop effects support this model by providing spacial transitions when switching desktops.

Activities are less about where you do something but about when you do it. They represent projects or different times of your usage of a computer. You program as part of your job in your IDE in the morning, browse the web and watch YouTube movies in the evening, and once everybody else is sleeping you play a game. You use different applications at different times, need different configurations of your desktop, and there is a relation of before/after between these.

To express this in the UI the activity switcher and configuration should not use concepts of space which overlap with how virtual desktops are presented. The closest idea I have seen is the activity switcher in Plasma Active. This uses a different and specific mechanism to switch activities. It even vaguely resembles a clock with its circular movement of activities when browsing through available activities, expressing the concept of time.

Screenshot from contourproject
When going with this mental model and the corresponding metaphors, suitable names for activities and virtual desktops could be "times" and "spaces".

Space

Following the idea of a hierarchical relationship of activities and virtual desktops we could represent them using the same dimension, but express activities in groups of virtual desktops.

Here is a mockup illustrating this idea:


Activities would control how groups of virtual desktops would behave. They would be used to define what wallpaper and which desktop widgets to show, what applications to start or stop, what settings for notifications or power saving to use, etc. All this would be tied to on what virtual desktop the user is. But there wouldn't be two different ways how to switch between desktops and activities, but there would only be one. Activities would implicitly be switched by switching between virtual desktops.

So when you are at work you would be on the left side of your desktop with your corporate wallpaper, running IDE, web browser, what you need for work. The right side of your desktop with the games would be shut off. For a presentation you would go to the desktop which switches off notifications and power saving. Email would be there as a shared resource all the time. In your free time you would be on the right of your desktop, with nude pictures of your cat as background, running inappropriate videos in your web browser, and switching off everything else than your games for maximum frame rates. Or something like that...

In some way activities would be a subset of virtual desktops and in some other way they would be a configuration for a virtual desktop. This follows along the lines of the "Different widgets for each desktop" configuration option, where each virtual desktop more or less gets its own activity.

The hierarchy would not strictly be activities above and desktops below, but there would be two concepts, the group above the desktop, and the configuration below the desktop.

This would need a few changes in representation of activities and virtual desktops, but it could better match the mental models of users, because concepts are clearly separated and existing understanding of controls can be used.

When going with this mental model and the corresponding metaphors, suitable names for activities could be "desktop groups" and "desktop configuration", and virtual desktops would just remain "desktops".

Conclusion

Naming is important, but even more important is expressing concepts clearly and unambiguously. In the user interface behavior and visual representation trumps names. That said, for talking about the concepts, especially in code and documentation, good names are essential.

To me seeing activities as configuration of groups of desktops is most natural. That matches my mental model.