Drupal Dev Days 2015 wrap up

Last week, I was at Drupal Dev Days in Montpellier with a few other Liipers. It was, as often is with such conferences in the Drupal community, greatly encouraging to see the passion and effort of the many Drupal developers there.

Throughout the week, hundreds of people were working on different projects in the “sprints” (as such code-marathons are called in the Drupal community). Drupal 8 core was (unsurprisingly) the biggest group, but there were many other efforts related to Drupal 8. I was primary involved in the Rules for Drupal 8 effort, which was a fairly large group. We managed to get quite a few issues solved, and the road to a user interface for Rules in Drupal 8 was begun, which is one of the biggest outstanding issues.

It was challenging work, at times, but there was a good mood in the group, and the D8rules team were really great at helping beginners getting started, so many thanks to them for that.

I was unfortunately forced to forgo the conference part of the event due to illness, but I hear it was great. In total, I think the organisers of DDD did a great job – especially with the food. So thanks to them as well. See you in the issue queues.

Rails Girls Summer of Code

The Rails Girls Summer of Code is a global initiative that provides 3-month stipends to women to work full-time on an Open Source project of their choice. The students work in teams of two from July 1st to September 30th, aided by local coaches; applications are still open until April 14th, 23:59 UTC (that’s tomorrow!) and anyone can apply if they fulfil the requirements.

The program is now in its third year and has successfully collected enough money to fund 10 student teams for 2015, thanks to wonderful sponsors (one of which is Liip!) and individuals, through a crowd funding campaign  For the first time in RGSoC’s history, we are attempting to fund an 11th team. The crowd funding campaign is still open, and we’d love to have your support  Follow us on twitter or check out our blog to stay up to date.

Help us foster diversity in Open Source!


Swisscom open-sources CleanerVersion

An interesting Django package was recently open-sourced by Swisscom. Manuel Jeckelmann wrote an interesting article about how the process of open-sourcing this worked at Swisscom. In the context of work on a security data warehouse at Swisscom, Liip has also worked on developing this package. I’ll briefly describe here what’s special about this package and how it can be used.

CleanerVersion lets you maintain a history of model instances and their relations. There are a number of packages that can be used for versioning in Django. What makes this one different?

Continue reading about Swisscom open-sources CleanerVersion


Improving the User Experience of the Symfony CMF Backend

Contrary to frontend experience, CMS backend experience is rarely adapted to the needs and tasks of its real users: the few people who publish and administrate the site. Standard backend themes often remain untouched after installation. The most common adaptations are the addition of menus and shortcuts, often due to addition of functionalities to the CMS.

We (David Buchmann and Benoit Pointet) spent some time to look into the backend we have for the liip.ch website, built with the Symfony CMF. We ran user testing sessions to collect our website’s editors painpoints, a community survey to collect the pain points of other CMF users, then did a lot of small tweaks both to the public bundles and in liip.ch specific code, and identified and documented improvements that needed more development work.

Continue reading about Improving the User Experience of the Symfony CMF Backend

Tags: ,

Webnesday St. Gallen

Before I started working for Liip I did my apprenticeship in St. Gallen. It was not until I started working in Zürich, that I discovered, companies here liked sharing knowledge with each other. This concept was new to me and I thought it was a great idea. There are plenty of events in Zürich to learn from like-minded people. There’s Webtuesday, JSZurich and Coders Only, just to name a few.

Once I started working in St. Gallen again, I was eager to bring this spirit with me and start such an event in St. Gallen.

People from St. Gallen seemed to like the idea. Peter joined the Webnesday Team and Roman offered us the Namics St. Gallen office for the first Webnesday.

What is Webnesday?

Webnesday is an open forum, for web interested people from and around St. Gallen. It’s scheduled every 2 months. Every odd Webnesday is a full talk between 30 and 45 minutes. Every even Webnesday consists of a few Lightning talks between 10 and 15 minutes.

On the first Webnesday, I held a talk about AngularJS. It was a success. About 21 people showed up and asked interesting questions. The slides can be found on Speakerdeck.

When is the next Webnesday?

The next Webnesday will be held on the 5. November. It will take place at Haufe-Umantis’ office in St. Gallen. Since it’s an even Webnesday, we will do Lightning talks.

You can sign up here for the next Webnesday.

We are still looking for additional speakers. If you discovered something cool that you would like to share with the crowd, please contact us.

Tags: , , , , ,

getting to know Cassandra

We did a small two person hackday to get to know Cassandra a bit. Lukas already learned a bit about Cassandra at Berlin Buzzwords but it was time to get some hands on experience to better understand how Cassandra would work out in a real client project. Our agenda for the day was however essentially: get it to run, see how to get it connected with PHP and then import some data and do some queries.

Brief summary

But first a very brief summary of what Cassandra is about:

Getting started

To get up and running we tried different approaches. One was just to download one of the VMs provided by DataStax (the main company driving Cassandra). That sort of works but the VM operating system is not really nicely configured. But it was at least enough to run a first few queries we found in the Cassandra wiki.

Now we wanted to install the PDO driver for Cassandra. So we setup a new VM and installed Cassandra. Compiling the PDO driver proofed to be a bit more complicated. We messed around for a while until Henrik saved us by uploading debian packages for us compatible for Ubuntu 14.04 64bit:

Playing around

We then started playing a bit. We looked for some interesting data sources and found this blog post listing a bunch of options for us, we ended up going with a dump of reddit posts. Not a particularly challenging data set for Cassandra, but enough to get a feel for things. Cassandra provides a PostgreSQL insipired variant of COPY to import CSVs. Using the following table definition, we started writing some queries against the dataset.

We quickly realized that for index supported queries, one needs to define composite key indexes. With a bit of time we got it to work.

We then played with a counter, but realized that counters cannot be in a composite index. Finally we briefly looked at their “transactions”, which however require every query essentially specify a condition for concurrency.

Pros and cons

  • Write speed
  • High availability due to decentralisation, no master node
  • Linearly scalable without downtime
  • Consistency highly controllable, use when needed
  • Ad-hoc queries possible via SQL-ish language (*)
  • Index supported queries need to be planned in advance, may require denormalization to support different index supported queries in the same data-set
  • Not all queries possible (very limited aggregation capabilities), so in practice other systems might need to be attached for searching

Tags: ,

Integrate Varnish and Nginx into PHP applications with FOSHttpCache

Earlier this week, I released version 1.0 of the caching proxy library FOSHttpCache and the Symfony2 FOSHttpCacheBundle. The library implements talking to caching proxies to invalidate cache, the bundle integrates the library into Symfony2 and adds other caching related features to Symfony2.

The library is all about talking to caching proxies to do active cache invalidation. It also provides the foundations for the User Context caching scheme that allows to share caches that do not depend on individual credentials but common roles or permissions of users.

The Symfony2 bundle FOSHttpCacheBundle integrates those features into Symfony2, using the configuration, providing annotations for invalidation, invalidating by Symfony2 routes and a default user context implementation based on the user roles. On top of this, the bundle allows to configure caching headers on request patterns, similar to the Symfony2 security component, and adds support for cache tagging.

Both library and bundle are well documented and thoroughly tested. The testing setup includes integration tests running a Varnish or Nginx instance. The test cases can also be reused for functional testing in your applications.

The cache header rules concept has been ported over from the LiipCacheControlBundle which is now deprecated. A migration guide is provided to update your projects.

Development of this started when I met David de Boer from driebit at last years SymfonyCon in Warsaw. He presented DriebitHttpCacheBundle which started from the cache invalidation, while the LiipCacheControlBundle started at caching headers. But both bundles had started to overlap already. We decided to build one common codebase that does both tasks. After more than half a year of writing better tests, refactoring and properly documenting, the code is now in a very good state. Besides David, a shoutout has to go to Joel Wurtz for contributing the user context implementation, Simone Fumagalli for the Nginx integration, Christophe Coevoet (stof) for his valuable feedback and reviews, and all the other contributors of the library and the bundle (some of those coming from the LiipCacheControlBundle git history).

Tags: ,

A report from DrupalCon Austin

DrupalCon group photo

DrupalCon Austin was last week, and I was the Liip delegation. Being my first American DrupalCon (6th overall), it was very interesting to experience the biggest DrupalCon yet.

The Drupal community was as vibrant and welcoming as ever, and the amount of energy being expended by thousands of volunteers to get Drupal 8 done is astounding.

Whither Drupal 8?

Having been through quite a few Drupal release cycles by now (4.6.x, 4.7.x, 5.x, 6.x, 7.x, and now 8.x), I know better than to ask (or attempt to answer) the “When is it coming out?” question, because the answer is the same as always: “When it’s done”.

That being said, we are getting close. The most important thing at the moment is the beta blockers list, which currently consists of 11 issues. Once all beta blockers are resolved, Drupal 8.0-beta1 will be released, and if this release will be anything like the previous, we will see a lot of new momentum when that happens.

Once the beta lands, contrib module authors will have a stable API to develop against, which means that they can start the mad rush to get their modules upgraded to Drupal 8.

It also means that using Drupal 8 for new projects becomes a reasonable option, and given how much more functionality comes out of the box in Drupal 8, I think we will see a lot of people doing this, since unlike previously (especially with Drupal 7), they will not have to wait for Views and other indispensible modules to be upgraded.

So, at least to a developer like me, it doesn’t matter much when the official, final 8.0 stable release will come. It matters a lot more when I can start using it (I have a few Drupal 8 hobby projects already, but it’s not being used at Liip yet), and that is getting very close.

How was the conference

Seasoned DrupalCon visitors will know that some the most important work and interaction happens outside of the official prorgam. Hallway chats, code sprints and social events are where long-time community members spend most of their time.

And this year was no different. Every night had one or more different parties (Viking Fest, woo), and hundreds of people were hanging out in the coder’s lounge, before, during, and after the main event. Meeting new people, and collaborating is really the name of the game, and in that regard, this year was a great sucess for me.

I spent quite a bit of time working on Twig-related issues (one of the new exciting changes in Drupal 8 is the replacement of the old and tired PHPTemplate engine with Twig), and a general clean up of the markup output by Drupal.

The #DrupalTwig effort is just one of the many new things in Drupal 8, but it was rewarding to see several long-standing issues get resolved during the week, and it looks like we will have a great theme system in Drupal 8. Thanks to Scott, Joël, Morten and the others for spearheading this effort and helping me and others get up to speed so we could help move the project along.

Looking forward to next time :)


How I use Docker on OS X

We at Liip started using Docker for some of our projects. For now mainly on the CI server, where it already helps us a lot in regards of reliability and performance. But we’d like to use it locally for development as well (and hopefully one day on production servers)

As we have quite a heterogenous environment on the dev laptops (everything from linux to windows to OS X), we use Vagrant to have a consistent setup locally. This makes it very easy to have the same versions of the needed software everywhere and we don’t loose time setting up things on the different OS (a great time waster for new people before vagrant)

The same easy setup can be provided with Docker and for Linux users it’s very straightforward and you can ditch your VM entirely. But Docker only runs on Linux, so OS X and Windows users still need a VM.

Currently we just added Docker to the project specific vagrant setup, which works fine. But as for some projects there’s more than one vagrant setup needed, it’s a waste of resources and disk space, as eg. you will download the same Docker images in each box.

There’s boot2docker and vagrant-docker, which both try to solve that problem. But that didn’t really work for me. boot2docker can’t mount filesystems from your host (at least not without many workarounds) and vagrant-docker seems to be not flexible enough for my taste and needs. Or at least I couldn’t figure it out.

So I started a new little project, called “dockerbox”, available at github.com/liip/dockerbox. It does exactly what I need (NIH syndrome anyone?). I took the Phusion Ubuntu Vagrant Box and adjusted it a little to my needs.

The basic idea was to start only that box and mount all project directories in there. So I just need this one box and not one for every project and I can share the base docker images.

Usually I install my projects into /opt/git, but when I already have vagrant boxes running somewhere there, I can’t mount the parent again in another box. So I created /opt/docker-git/ and put all projects I want to use with docker in there and put the following into VagrantfileExtra.rb:

With this I have the exactly same directory path in the box and on the host system (see below for why)

Now I can ssh into the box, go to the project directory I want and start Docker for using it.

But I wanted it more comfortable. As I switch often between the box and the host system, I wanted an easy and fast way to switch between them. So I created a script called ssh2docker.sh. With this I can just type ssh2docker.sh into my command line and I’m immediatly logged in in the vagrant box, and if the directory exists where I was on the host system, it automatically takes me there. (btw, it needs “vagrant global-status to be able to login from everywhere and IIRC this is only available since Vagrant 1.6)

vagrant ssh is usually not the fastest thing for logging in, therefore ssh2docker.sh caches the essential info in a file (/tmp/dockerboxid). After the first run, it should log in really fast (it also automatically does vagrant up on the docker box, if it’s not running yet).

It also takes commands to be run in the box. You can do for example ssh2docker.sh ./run/my/script.sh.

That was still too much for me. To start docker images and the test runs, we have some scripts which do the heavy work. To be able to just start them on the host system without having to think about if I’m in the correct environment, I added the following to the scripts:

I just set now SSH2DOCKER on my OS X machine to the path where the script is and I can run ./scripts/docker/build.sh on OS X as if I’d be in the vagrant box. I’d hardly ever have to login into the docker box anymore.

Additionally I installed boot2docker and set on my host

and then I can do eg. “docker pull ubuntu:14.04” or whatever docker command right from OS X and all happens in that docker box.

I don’t use this system since a very long time, maybe it has still some caveats compared to a “traditional” vagrant setup, but it saves me resources on my laptop, in terms of RAM and disk space (which is cheap, but not really on an 11″ air ;)). I don’t have to run multiple vagrant machines anymore and provisioning of single projects is usually also much faster (at least, when you downloaded the base docker images in the VM)

And of course you have all the advantages of Docker. Currently we’re switching from ElasticSearch 0.9 to 1.x, which is not backwards compatible. With docker, we just created a new image for ElasticSearch 1.1 and use that, when we need to in the new branches. With the vagrant setup we have, it’s quite a pain and time-consuming to have either two versions of ES running or to have 2 vagrant boxes running with different versions. It’s all possible and once you have done it, not too difficult, but once you have a docker setup for your projects, it’s definitively much easier this way.

And by the way, we will do a docker hack day on the 26th june at our open innovation day in Zurich with a big office opening party afterwards, everyone is welcome to attend (or in any of the other tracks).

Tags: , ,

Hasta la vista, Mallorca

The Liip delegation at #phpeceu, signing off :)

The Liip delegation at #phpeceu, signing off :)

The past weekend held the third European PHP Unconference in Palma de Mallorca.
Four Liipers attended: Adrian Philipp, Fabian Schweizer, Dorian Villet and Mikkel Høgh.

We had a great weekend in the company of other PHP developers and enthusiasts from all across Europe. As an unconference, the focus was not as much on presenting already known solutions, but rather on encouraging discussion and growth of new ideas and common understandings. The topics were many and diverse, and the surroundings were great for socialising and after-hours discussions, so among the other things we learned was the location of one of the best tapas bars imaginable.

As it has already been announced, that the event will also be on Mallorca next year, we can say: we’ll be back.