Symfony: A look back and what it all means

As we were preparing the news about becoming a Sensiolabs Silver Partner, I brought back a bit to the history of Symfony here at Liip. We did do a few symfony v1 projects at Liip but things only really took off with Symfony2. Back in 2009 Fabien came to Zurich to discuss some of the Symfony2 components (still PHP 5.2 compatible at the time) he had just released as well as a few he hadn’t yet released. Jordi, who was working at Liip at the time, and I integrated all of them into our company internal framework over the following months which we later presented at the Symfony Live. This means Liip in fact build the first Symfony2 framework, even before there was the official Symfony framework.

Continue reading about Symfony: A look back and what it all means

Tags: , ,

DPC – Dutch PHP Conference in Amsterdam

amsterdam

Last week, I was at the Dutch PHP Conference in Amsterdam. I did a workshop on content management with Symfony with the CMF and a workshop as well as a talk on HTTP caching and the Varnish reverse proxy together with David de Boer. I will give a similar tutorial at PHP Summer Camp in Rovinj, Croatia and at PHP Conference Bulgaria in Sofia later this year. Let me know if you are interested in having me give this tutorial at your company.

A nice perk of talking at a conference is of course that I also get to attend the conference and see other talks. There was a couple of interesting talks I managed to attend in Amsterdam. Most notably was Implement Single Sign On easily with Symfony by Sarah Khalil from Sensiolabs, who explained the authentication process of symfony so good that I finally feel like I really understand what is going on. The talk on HTTP/2 was mainly interesting for its detailed analysis of what was cleaned up in HTTP/1.1 since RFC 2616. Another good talk was by Arne Blankerts on the Content Security Policy (CSP). And I really enjoyed Daan van Berkels talk on Ada Lovelace and the Analytical Engine, he made maths and assembler sound like fun! There was also a code night in which David de Boer and I hosted a FOSHttpCache session with the main outcome of providing reusable varnish configuration files for the features of the library, instead of copy-paste documentation.

Besides the conference, there was of course also time to visit the beautiful city of Amsterdam. I had not visited the Netherlands since I was a small child. I even found the time for a trip to Friesland, visiting Schiermonnikoog in the Wadden Sea and taking lots of pictures of sand and birds. On the way back to Amsterdam, I stayed at Leeuwarden to do a Symfony CMF introduction at PHP Friesland Usergroup.

Improving the User Experience of the Symfony CMF Backend

Contrary to frontend experience, CMS backend experience is rarely adapted to the needs and tasks of its real users: the few people who publish and administrate the site. Standard backend themes often remain untouched after installation. The most common adaptations are the addition of menus and shortcuts, often due to addition of functionalities to the CMS.

We (David Buchmann and Benoit Pointet) spent some time to look into the backend we have for the liip.ch website, built with the Symfony CMF. We ran user testing sessions to collect our website’s editors painpoints, a community survey to collect the pain points of other CMF users, then did a lot of small tweaks both to the public bundles and in liip.ch specific code, and identified and documented improvements that needed more development work.

Continue reading about Improving the User Experience of the Symfony CMF Backend

Tags: ,

Symfony Live in Berlin: Lots of Symfony in Germany

Last week, I went to Berlin to present the Symfony Content Management Framework to the German PHP crowd. I did a presentation almost exclusively based on code. I showed how to develop an application with the Symfony CMF step by step. But rather than trying to live code, I prepared a series of pull requests that I switched through and showed the interesting code changes. This is a rather unusual format and I lost a few people, but I also got enthusiastic feedback from the audience. The tutorial is also online at github for those that want to have a look at it.

Fellow Liiper Lukas Kahwe Smith did two presentations. RESTing with Symfony brings together REST theory and Symfony implementation tricks for REST APIs. There is a Bundle for that explains how to find and evaluate third party bundles for Symfony, and is an appeal to not reinvent the wheel. Rather than spending time to do yet another mediocre bundle for the same purpose, invest that time makeing an existing bundle better – which is harder but more sustainable in the long term.

I also went to some other talks. One of my highlights was Dennis Benkert with his presentation on Dockerizing Symfony Applications. The talk contains many interesting recepies how to handle a symfony application including console support in docker. It was also clear about the current limits and challenges with docker (persistent filesystem for example). The other highlight was Johann-Peter Hartmann talking about Leadership in der Softwareentwicklung. This talk is about the management perspective of agile companies and held a lot of interesting and sometimes amusing background that developers who care about their work environment should think about.

Sensio Germany did a lot to ensure the conference was a nice experience. There was a social evening with free drinks in (and outside the) rainmaker loft club and colab space. I got to talk with many people about the CMF, explaining ideas and concepts and getting valuable feedback. One important bit is that I should write a blog post about the CMF routing component and how its usable stand alone even outside a full stack Symfony project. At the hackday on Saturday, a couple of people got their hands into the CMF and investigated the frontend editing, looked at the UX of the sonata admin and fixed warnings from Sensiolab insight.

Tags: , ,

Discussions and Pizza at PHPDay Italy

PHPDay Logo Last week, I was invited to do a Symfony Content Management Framework tutorial at the PHPDay in Verona. I saw great talks, most notably a demo of Docker. I had a good crowd of people for the two hour walkthrough of building a website with the CMF. In this post, i share some experiences from the conference.

Having arrived on wednesday, I went to see the you have ruined javascript rant by Rob Ashton. It was definitely an entertaining and passionate talk. I feel not qualified to decide how much Rob is exaggerating, but he definitely brought forth valid critics and reminded the general rule of not always using a large framework even for small tasks. In the evening I had dinner in downtown Verona.

Thursday started of with an introduction to AngularJS in combination with Symfony2. I liked the examples how even when doing a frontend application, Antonio still manages to use the power of Symfony Forms and other components without doing everything twice. When using Symfony2 and AngularJS you definitely want to look at these slides. In the afternoon I did my Symfony CMF tutorial. I walked the audience through building an application with the CMF. I showed a lot of code and explained both the “how?” and the “why?”. I had decided not to do hands on, as two hours is too short for that. I did hands-on CMF at a workshop at PHP Benelux earlier this year, and the 3 hour slot was just enough. The walkthrough was well received and i had interesting discussions with some participants after the workshop.

The day was wrapped up by an unconference talk about the joys and frustrations of working on composer by Composer himself (yeah Jordi loves it when you call him that :-P ). Jordi showed some really funny and some sad examples of reactions he got. After that, it was off to the speakers dinner, which was awesome italien cuisine (as in, not Pizza but delicious food with good wine).

Saturday started with meeting Simone Fumagalli who is contributing Nginx support to the FOSHttpCache library. We met for the first time in real live, and wrapped up his pull request. Meeting people is just as important to me about conferences as the actual talks. I made it in time to see a mind-blowing presentation on Docker by Alexander Mols. I definitely need to look into this tool. On a modern Linux (which is of course what I work on), its just so insanely faster and easier than running Vagrant to normalize the development setup. And even with other operating systems, it will work at least as good as vagrant. Thanks to dokku it is even possible to use Docker in a Heroku-style deployment process by simply pushing to a git remote. Alexander cautions that both projects do not yet claim to be production ready, however. In the afternoon, I ended up sitting in the sun and talking to a bunch of interesting people most of the afternoon. Discussions ranged from PHPBridge, a project meant to introduce total non-programmers to PHP with a strong social twist, over community building to more personal discussions with people from all over Europa.

It was my second time in Verona and this edition of PHPDay was just as great as I remembered the first time. I saw good presentations and got new inputs, and had many interesting discussions with interesting people. It was great to meet friends from the PHP community and make new friends. I am thankful that Liip supported me going to the conference, and also sponsored the conference.

Tags: , , , ,

prismic.io content repository as a service

Just before Easter I tweeted out an idea that I already threw out there a few weeks ago: Creating a PHP Content Repository (PHPCR) implementation on top of the prismic.io content repository as a service. Well I had a lot of time on the train over Easter, so I decided to make it happen. Obviously coding against a remote service on the train isn’t that much fun, so in fact I did a lot of coding, waited for the next train station, hit refresh a few times and so on and on. I briefly tried to setup php-vcr to log all web requests and replay them, but unfortunately the library isn’t quite ready to do this yet .. but mit might be soon. Also as prismic.io’s content pages are actually all versioned we will soon hopefully get caching provided inside their PHP SDK. But I digress. So without further ado, I present to you an initial working version of PHPCR integration with prismic.io. There are obviously still a ton of todo’s left .. but fundamentally it works: I can read prismic documents as PHPCR nodes with properties!

Why bother?

At this point you may either be excited or wondering why I am bothering with this. If you are in the former group, you can maybe skip to the next section, if you are in the later group, let me expand. One reason to bother with this is to simply show its possible, that PHPCR indeed fullfils this promise of being able to provide a unified content repository API. We obviously have 3 implementations already but taking such a service seems to push the boundaries a little harder. The next reason is more practical: prismic.io is simply a very useful service with an awesome user experience, especially for people focused on managing content rather than managing pages on a website. And they are doing this with an awesome UI, argueable the main week spot of the CMF (though I am quite excited about the work by the sulu.io team).

The sky is the limit?

Now prismic.io has some major differences in how content can be structured compared to plain PHPCR. Specifically they do not really structure their content into a tree per se. Instead they just have a pool of documents with UUIDs. Now it is possible to reference those UUIDs inside documents which kind of creates a tree structure. There is also native support for tagging and finally a way to bookmark document UUIDs to names. In a first step all I did was create a virtual root node that has the entire document pool as children where the names match the bookmark name (if one exists) or the UUID. All properties, including nested documents, are then mapped to properties on those children. Eventually nested documents should however maybe be exposed as child structures. Another limitation is that prismic.io currently only provides a read API. But given their awesome UI, why would one want a write API? ;)

The sky is the limit!

But there is a lot possible yet. For example they provide a predicate based query API that I am very confident we will be able to map to the PHPCR query languages SQL2/QOM, similar to how we map them to SQL with Doctrine DBAL. They also support versioning, which can probably be mapped to the PHPCR simple versioning concept. They also support node types which I have already mapped but there is still work todo to map all the contraints. They also support custom namespaces which PHPCR also supports. The way they support images (and binaries) also lends itself very nicely to the stream based approach in PHPCR. So with a few more days of work, I think we can provide a pretty powerful PHPCR API for prismic.io. What will make the collaboration easier is that it turns out that Thibault, founding member of the Symfony CMF initative gone Scala-lover, works for zengularity.com who created prismic.io. But Rudy was actually the first one to jump on my tweets and he has already signed up to the CMF mailinglist and given important insights.

Where to go from here?

I am not sure how much time I will have on this in the coming days. The CMF 1.1 release is still the top spot on my open source todo list. I think with what I have setup here, it should be possible for people even without prior knowledge of Jackalope to help out implementing some of the open items in the issue tracker. So I hope that the community chips in as well. Judging from the feedback on twitter, there are quite a few people interested!

PS: As part of my initial playing time with prismic.io, I also refactored their Symfony2 starter project a bit. Might be interesting to take that for a whirl as well before diving into the PHPCR world. They also have a few open tickets on their PHP SDK.

PPS: Once all the PRs I send the prismic.io folks have been merged and we have figured out where the PrismicBundle is supposed to live, I will likely also publish a fork of the php-symfony-starter where I use the Jackalope Prismic transport instead.

Tags: , ,

Content storage done right

Jackalope and PHPCR have been a reoccuring topic on this blog. Back in 2009 we here at Liip began exploring the possibility of integrating Jackarabbit, the reference implementation of the Java Content Repository specification, with PHP. The vision was two fold: First up we wanted to make it possible to directly interact with content stored in AdobeCQ (called Day Communiqué at the time) or Magnolia. Additionally we also felt it would be a great asset to the PHP CMS world to be able to leverage all the power of JCR from PHP, hence PHPCR. The initial attempts made use of the Zend Java Bridge to communicate directly from PHP to Java. However eventually we realized that it would be more feasible to use the native HTTP API provided by Jackrabbit. But things only really took off when the Symfony CMF initiative decided to adopt our work. Now four years later we finally have the first stable releases of PHPCR, Jackalope and the hibernate inspired object mapper PHPCR ODM.

It is PHP

The gut feeling of many PHP developers when faced with Java is usually one of worries of factory factories, endless XML configuration and deep class structures. This sentiment might best be illustrated by this famous saying: Java is a DSL for taking large XML files and converting them to stack traces. At the same time perceptions have changed. Lucene is the basis for Solr and ElasticSearch, the go to full text search engine for most PHP developers. In fact pretty much all PHP CMS defer to either of these solutions when dealing with larger data sets when it comes to search. By the way, Lucene is integrated into Jackrabbit right out of the box. Furthermore, many PHP developers have realized that there is in fact value in decoupled architectures and design patterns which indeed tend to result in more complex class structures. The benefits are however reuseability, testability and the fact that each unit of code is much more approachable on its own. But of course this does not mean that there is no value in the PHP platform aside from its ubiquity resulting in the fact that in October 2013 PHP was used on 80% of all domains. The conclusion is that there is no need to jump ship from PHP but there is value in bringing outside ideas to the platform. Obviously porting JCR to PHPCR cannot be done as a one to one mapping. So PHPCR leverages the fact that PHP provides associatives arrays where JCR has to rely on more unwiedly object structures. And most importantly for making PHPCR relevant for the real world, there is also an implementation that is written purely in PHP connecting to a RDBMS using Doctrine DBAL, ie. its possible to use PHPCR without running any Java at all.

It is community

Through out this entire effort Liip has done a significant portion of the work. However there has always been people from the community involved as well and this was the goal from the very start. At this point one can therefore with confidence state this this is a community effort and that development is no longer driven entirely by Liip. This includes reporting bugs but also fixing bugs, improving or adding features, tests and of course also documentation. I want to briefly highlight some key contributors:

  • Karsten for his initial work on the PHPCR interfaces which we adopted
  • Uwe and Johannes for becoming the first non Liipers to make significant improvements to Jackalope and PHCR ODM
  • Benjamin for laying the groundwork for Jackalope Doctrine DBAL
  • Dan for his work on the PHPCR ODM Query Builder

What is next?

This release of PHPCR provides compatibility with JCR 2.1. Jackalope, the reference implementation of PHPCR, integrates with Jackrabbit and Doctrine DBAL. The next steps consist of completing some of the optional features and further performance improvements. For example it would be interesting to make it possible to be able to use Solr/ElasticSearch in combination with the Doctrine DBAL implementation. Another feature we are looking forward to is improved logging and caching capabilities. We are also looking forward to work picking up again on the MongoDB implementation. We are also keeping an eye on the next major version of Jackrabbit, code name Oak. In fact we have already tested compatibility with the current releases together with the Adobe engineers. But generally we are most looking forward to people to add PHPCR to their applications where ever they feel they can benefit from a storage solution that provides unstructured content in a tree structure with support for node types, binaries, versioning and full text search. Case in point we are looking forward to the imminent release of Symfony CMF.

Tags: , ,

Updating old Symfony2 CMF projects

We released RC1 of the Symfony CMF last week. Now it is time now to upgrade older installations to the latest and greatest. I decided to keep a record of what i had to do and write it down for others to follow the steps. There are notes in the CHANGELOG.md files of each CMF bundle, but a common blogpost is more convenient. The whole update took me a bit less than a day of work. Now that we are in release candidate state with the project, further upgrades should need no more changes, or only small ones.

Continue reading about Updating old Symfony2 CMF projects

Tags: ,

PHP family meeting at FrOSCon

FrOSCon is one of those conference close by that I have missed attending for many years, mostly because its on the weekend and especially during the summer my weekends are reserved for ultimate frisbee. Well this year I set my priorities straight to finally be able to attend this conference. As it turns out this is one well organized conference even with free cake for speakers :)

Hanging out with the PHP family

Of course with this title I am selling the event short, since it actually brings together the entire open source community. More importantly its not just software but there are also lots of hardware hackers with booths. So I did stroll past all the booths peeking a bit at what people are up to. However I must admit I ended up mostly hanging out with the PHP folks. Of course it was mostly developers (many of which brought their significant other with them) from Germany but I was also thrilled to once again meet Derick and finally personally met Mathias in person. I have been very intrigued by his posts about domain driven design (DDD) and his presentation was quite inspiring.

Symfony2 REST

On Saturday I was asked if I could cover for Gordon who has submitted a talk on his Symfony2 REST Edition. At first I thought I would just show my normal REST examples but about 20 minutes before the talk I decided, why not just install Gordon’s examples instead? Its kind of amazing how github, composer and Symfony2 make it possible for me to just pull down this code that I had previously only looked over briefly a few months before and give a talk on it. But I think its worked quite ok. Of course I spiced things up with a few of my examples to show the different tools provided by FOSRestBundle, NelmioApiBundle, JMSSerializerBundle and FSCHateosBundle (soon to be replaced by the BazinghaHateoasBundle).

Symfony2 CMF

I gave a presentation on the CMF initiative. I had hoped to already be speaking about a stable release but we are still putting the finishing touches on a few details there (namely image uploading and a rewrite of the PHPCR ODM QueryBuilder). However there is of course a lot of other things to show and concepts to explain. And as always there is quite a few people interested but two or three people that really feel the pain of todays CMS solutions and are totally excited.

Tags: ,

Checking out Sylius: Symfony2 e-commerce

Last Friday we did a one day hackday with the goal of checking out the Symfony2 based e-commerce solution Sylius. Among the attendees were even two guests with Fabian and Stefano joining the Liipers David, Matteo, Patrick, Tobias and myself.

Getting started

We did not have any specific agenda and decided to just install and start exploring. For the most part this was a quite painless experience it just takes a while when 7 people clog up a wifi connection. Pawel had just completed the update for Symfony 2.3 the day before, so we found some minor bugs that we fixed. Stefano also submitted a PR to fix PDF generation on travis. Matteo spend some time updating the german translations.

Behat tests

Fabian decided to dig into the Behat tests which were failing on travis-ci.org due to memory issues. Sylius comes with a very extensive set of Behat tests. As a matter of fact maybe this is the best source of documentation of Sylius features. In the end his solution was to simply split the test execution by tag.

Adding some PHPCR love

One thing we noticed quickly while reviewing the database schema is the table used to store the ProductProperty class instances. It uses the Entity-Attribute-Value pattern, one of the reasons why its not so trivial to scale Magento. Essentially this is one commonly used approach to deal with unstructured data in an RDBMS while retaining some ability to run search queries. The big issues with the approach is that reading data out is quite hard but worse it basically requires one column for any kind of data being stored. In the case of Sylius its using a VARCHAR(255). So we figured we would simply use Doctrine listeners to modify the persisting to use PHPCR instead. The advantage being that with PHPCR it would allow to support all sorts of data types without length limitations. So this would for example allow to associated additional images, dates and longer descriptions with products. Stefano and I struggled a bit with the ORM listeners. We learned that persisted entities are not stored in the identity map until the RDBMS has actually generated an ID. Until then one has to use getScheduledEntityInsertions() to get access to the relevant entities. Once we figured that out the actual tasks of integrating PHPCR was comparatively trivial. Now it might however make sense to even go a step further and store the Product class itself in PHPCR as well using PHPCR ODM.

Dynamic routing

Sylius uses two “catchall” routes for products and categories. This is a quite common pattern found in CMS’s build on top of Symfony2 or many other general purpose web development frameworks. Basically one defines a route that simply catches anything below a specific pattern without knowing if anything even exists in the database that would match that. Then inside the controller a DB lookup is made and in case the data is missing a 404 is returned. In the Symfony2 CMF initiative we solved this need by introducing a chain router and dynamic router which has been adopted by ezPublish 5 and Drupal 8. It allows one to chain the standard Symfony2 router and any number of dynamic routers which means that static routes (like f.e. the homepage) do not need to be moved to the DB. The job of the dynamic router is to do a DB lookup during the routing stage. This means the controller is no longer responsible for this task. Stefano and I managed to implement this surprisingly quickly. One draw back of this approach is that now a DB lookup is also required to generate a route. Enabling the ORM entity cache however solved this problem quite easily.

Taxonomies for the CMF

So while Stefano and I worked in integration code form the CMF into Sylius, David attempted to do the opposite. He tried to integrate the SyliusTaxonomiesBundle into the CMF. However unfortunately he didn’t yet manage to get it to work.

Admintool

Matteo and Tobias looked a bit more deeply at the admin tool and unfortunately found several missing features. More over they quickly realized that the different tools allowed bringing the data to an inconsistent state. They then spend some time making the order status actually dynamic, at the time its was all hardcoded. They were still a bit unsure about the actual concept of how the order status relates to the relevant potentially several different shipping status states.

Sylius in the real world

Patrick is actually already running a Sylius web shop in a private project. So he spend some time investigating how to upgrade and what has changed. He found that indeed a lot of has changed but everything that has changed, changed for the better. However there are still things missing. For example in his project they needed additional notifications when an order is created.

Take away

Sylius looks really promising and we were positively surprised with the provided feature set. The code is clear and well structured and very easy to get started for a Symfony2 developer. However there are still important missing features making it very hard to plan the amount of time needed to do an actual real world project. The added fact that a lot of code is still very much in flux makes things even harder. It seems however that one could take the code at any point in time and complete a project with fairly well working code. In this regard the situation is quite similar to the CMF which has also already been used by several companies to build production applications. No doubt once Sylius has a stable release it will be a very viable alternative to Magento for projects that need very heavy customizations. In this sense it will also be interesting to see how OroCRM and Akeneo will turn out as these could provide some advanced CRM and product management capabilities that should be easily integrable since both of these applications are written in Symfony2 as well.