Playing well with others

I was invited to speak at the T3DD15, the TYPO3 Developer Days 2015, on the topic of community collaboration. Indeed my presence there was also made happen because there was a big interest both from the TYPO3 CMS as well as from Neos CMS to explore adopting Symfony. Another topic was adopting of PHPCR, again both TYPO3 and Neos CMS expressed an interest in this topic, though not as urgently as adopting Symfony. In fact the Symfony adoption is already under way with TYPO3 CMS developers having already integrated the Console component and looking towards the Routing component next. Neos CMS has long ago adopted the Yaml component and are looking towards the Process component next. Indeed especially Neos CMS might be beginning to adopt quite a few more Symfony, and other components. They are even willing to explore going to Symfony full stack, though it would need to be made compatible with their unique Proxy based AOP solution that enables them to effectively overload the “new” operator.

Anyway, back to the title of this blog post. I did in fact choose the same title for my talk in which I tried to give some considerations about legal but more importantly community topics related to working with third party projects. I tried to make it clear that there valid points for choosing to not adopt third party solutions in every case. I also tried to address a bit how to get people from other communities more involved into the TYPO3 and Neos CMS communities. Overall I must say I am still not quite as fluent with soft topic talks compared to hard topic, ie. code related, talks. I still do not know how much content to put on the slides and how much I can free style. I also still have a hard time estimating how long my talk will last. I guess I will get better with practice.

It was also an interesting time to give this talk, as TYPO3 and Neos CMS only recently decided to part ways. It is not clear if this will be the last joint conference. During my talk I wanted to make things a bit more concrete, especially when discussing points about what might prevent people from collaborating with TYPO3 and Neos CMS. It felt a bit strange as I was mentioning issues with each of the projects. I was worried that I might set of a flamewar, but everything seemed to stay quite civil :)

At any rate, I spend two great days with lots of interesting discussions. The location was awesome, essentially right in the Nürnburg castle, with lots of rooms for people to gather to discuss and hack. Aside from the Symfony and PHPCR related discussions I also entered into a session on diversity as well as on CQRS. Overall it was a very productive environment and a fun exchange with lots of old and new faces!

Tags: , , ,

DPC – Dutch PHP Conference in Amsterdam


Last week, I was at the Dutch PHP Conference in Amsterdam. I did a workshop on content management with Symfony with the CMF and a workshop as well as a talk on HTTP caching and the Varnish reverse proxy together with David de Boer. I will give a similar tutorial at PHP Summer Camp in Rovinj, Croatia and at PHP Conference Bulgaria in Sofia later this year. Let me know if you are interested in having me give this tutorial at your company.

A nice perk of talking at a conference is of course that I also get to attend the conference and see other talks. There was a couple of interesting talks I managed to attend in Amsterdam. Most notably was Implement Single Sign On easily with Symfony by Sarah Khalil from Sensiolabs, who explained the authentication process of symfony so good that I finally feel like I really understand what is going on. The talk on HTTP/2 was mainly interesting for its detailed analysis of what was cleaned up in HTTP/1.1 since RFC 2616. Another good talk was by Arne Blankerts on the Content Security Policy (CSP). And I really enjoyed Daan van Berkels talk on Ada Lovelace and the Analytical Engine, he made maths and assembler sound like fun! There was also a code night in which David de Boer and I hosted a FOSHttpCache session with the main outcome of providing reusable varnish configuration files for the features of the library, instead of copy-paste documentation.

Besides the conference, there was of course also time to visit the beautiful city of Amsterdam. I had not visited the Netherlands since I was a small child. I even found the time for a trip to Friesland, visiting Schiermonnikoog in the Wadden Sea and taking lots of pictures of sand and birds. On the way back to Amsterdam, I stayed at Leeuwarden to do a Symfony CMF introduction at PHP Friesland Usergroup.

Playtime with OroCRM

We continued our exploration of e-commerce applications in the Symfony2 ecosystem at this months hackday with some playtime with OroCRM. OroCrm is currently in alpha phase and is unique in that its primary focus is to provide CRM tools for B2C rather than B2B of traditional CRM solutions. They plan to integrate with different webshop solutions to for example automatically open a lead if a customer leaves the site without checking out but had filled the shopping cart with a certain number of products. Since the venture is started by Yoav, one of the founders of Magento, their first target is of course Magento which we also leverage at Liip as our primary e-commerce platform. What is also interesting is that the OroCRM created a business application platform on top of Symfony2. The goal is to facilitate the creation of other business applications which can then be easily integrated together. One concrete example of this is AkeneoPim. Note that the platform uses the liberal MIT license while the CRM uses the OSL 3.0 license, same as Magento. Its quite similar to the LGPL with additional provisions that terminate the license for anyone initiating patent litigation against other users of the application.

Getting setup

In general OroCRM is quite trivial to install for anyone familiar with Symfony2. It was nice to see that they use the popular LiipImagineBundle among many other community provided Bundles. We made some small PRs to address some tiny details. We were a bit sad to see that the OroCRM team currently does not leverage travis-ci to make it easier for contributors to ensure that their contributions do not break the tests. But at least it was good to see that there is an extensive test suite.

First look at the UI

The first impressions of the UI were quite positive. There are some very useful navigational tools that are provided as part of the application platform that provide shortcuts, links to recently used tools and a search that can search across all data using a custom inverted index stored in the RDBMS. Note that the search index looks to be pluggable, so expect Solr support to be provided eventually as an option. That being said, the current set of features do not cover things like leads yet. There was unfortunately also no code yet to handle Magento integration. Its clear that at this point the focus is still more on providing infrastructure rather than trying to provide a finished feature set. Its an alpha after all.

Digging in a bit

At first we ran the application via the PHP 5.4 build in server that Symfony2 integrates conviniently via app/console server:run but the application behaved a bit flaky then. Its basically a single page app where all content is loaded as JSON and using the build in webserver we frequently ended up with JSON replacing the actual UI. OroCRM seems to use anchors to update the URL bar rather than push state. Using apache we didn’t notice those issues but were forced to create a virtual host since the Javascript code had several URLs pointing to /api/rest without taking care of the subdirectory the application was running in. Speaking of the UI we were dissapointed to notice that it is currently not responsive and no provisions seems to have been made for touch devices. We feel like it would be wise to address this topic early to ensure that all the infrastructure pieces will work for mobile users too.

Its all REST

The backend seems to support a REST API via WSSE authentication. SOAP also seems to be there or at least planned but we didn’t try it out. To make it easier to generate the required WSSE headers we send a PR with a little command. However many of the controllers use json_encode explicitly rather than leveraging the FOSRestBundle view layer. This in turn means that XML is not supported and furthermore its not so easy to configure on the model what fields should be returned. Its not clear why this is the case because in some controllers FOSRestBundle is used. It might be that this will eventually be refactored.

Low level infrastructure

In the system tab we found several very useful pieces that will no doubt be useful for many other business applications too. First there is a notification system which hooks into the ORM. Administrators can for example automatically send an email when a new contact is created. There is also an audit log that can be enabled on a per entity basis. Entities can be configured in the UI which generates the code and automatically updates the database schema. A new entity will by default map to a new table. There also seem to be some provisions for EAV to handle more unstructured content but its not yet possible to configure this via the UI. I would urge to look into PHPCR here instead or just serialize to JSON/XML.

Looking at the current CRM features

The key thing that is currently provided is the ability to create accounts and contacts. We noticed some issues in the validation. Sometimes there was not enough information given or validation was limited to server side validation. There were also some issues that due to the fact that validation was server side, red boarders denoting errors caught in the server side validation dissapear. There were also some usuability issues like there was no way to copy the address of the account over into the shipping address fields and there was no way to quickly create a new contact for an account. But hey, its an alpha so no worries there. We also looked briefly at security. Symfony2 out of the box provides quite a few tools to help write secure applications. So we didn’t find any significant things. We just noticed that while in most places html tags were simply escaped, for tags we found that html tags were stripped on display but persisted in the database.

The take away

OroCRM certainly looks promising though at this point the main thing that can be judged is the platform underneath. There are not that many actual CRM features, let alone webshop integration. Note that OroCRM alpha6 was released today featuring among many things a “Flexible Workflow Engine” and “Leads and Opportunities Workflow”. So it looks like they made a big step towards becoming an actual CRM.

Tags: , ,

Jackalope and Magnolia CMS: Recording online, questions and answers

Last thursday, i did the webinar about PHPCR and Magnolia CMS. You can download the slides or watch the recorded presentation (you need to register to see it). Thanks to all the attendees, I hope you enjoyed it.
There where some questions that i want to answer here on the blog to have the answers available to everybody.

Question: How reliable is Jackalope, can i trust my data to it?

Answer: Jackalope is still on the young side, but running successfully in a couple of real life applications. We also built a PHPCR API test suite that checks on many features. The PHPCR test suite is by no means as complete as the JCR technology compatibilty kit (TCK) testing suite. With jackalope-jackrabbit, the data is stored into Jackrabbit however. Jackrabbit is really mature and validates the data coming in over DavEx. In the very worst case when Jackalope totally fails, you could still write Java code or use Java tools to interact with Jackrabbit directly. In my experience I never needed to resort to that.

Question: You mentioned that same name siblings are not supported by Jackalope. Is there any particular reason for Jackalope for not supporting this? Is support coming, or is this a design choice?

Answer: Same name siblings is one of the optional chapters of JCR. Jackrabbit supports them, so it would be doable to implement them in Jackalope as well. It can be rather tricky to do it, having unique child names made a lot of the code easier. But if somebody wants to take the effort to implement it, I would be happy to provide support and help understanding implementation details.
The main reason to have them, by the way, is that JCR / PHPCR can import arbitrary XML documents, and such documents tend to have same name siblings, just think of a XHTML document for example.

Question: What if I need a particular PHPCR feature not yet implemented in Jackalope?

Answer: Jackalope is an open source project. Contributions are always welcome, and the community will help. Liip is also open to consulting work for such tasks. Note that there are a few chapters that will be impossible to implement with jackalope-jackrabbit until the jackrabbit side implements exposing them over DavEx. This is not impossible, but considerably more effort. The next major version of Jackrabbit, codename “Oak” will focus more on feature completeness over the remoting protocol.

Question: Where are queries executed? Is there any way to measure the impact on the Magnolia repository? (I wouldn’t want my main sites performance to degrade dramatically) Is there some sort of cache or index on the PHPCR side?

Answer: Normal requests for nodes are cached by Jackalope for the PHP session. SQL2 or QOM queries however are not cached. The impact will not be higher than when implementing the functionality in Java in a custom module inside Magnolia CMS. The load should be slightly lower even, as the DavEx remoting is more low-level than the full JCR API. And the PHP application can life on its own separate server. Nontheless it makes of course sense to cache rendered pages or page-fragments on the PHP side. But that is an application level task.

Question: DavEx provides to the Jackrabbit used in the Magnolia CMS. Is there a way to connect to different repositories?

Answer: DavEx is an implementation specific protocol invented for jackrabbit to do remoting. Jackalope thus can talk to any Jackrabbit instance, and it has no problem opening several sessions with different servers. But connecting to any non-jackrabbit JCR implementation is not supported. JCR is an API, not a protocol. Note that there are ideas to generalize the jackrabbit-protocol into another standard called JSOP.

Question: Something is unclear to me: Are calls from Jackalope to the Jackrabbit repository bypassing Magnolia?

Answer: Yes, exactly. The Magnolia CMS DavEx module just provides a servlet that receives the requests and pipes them through to the jackrabbit DavEx layer. This is needed because Magnolia CMS runs Jackrabbit in its own process and does not use the DavEx remoting.
The following diagram shows the interaction between Jackalope, Jackrabbit and Magnolia CMS. The PHP application is talking to the PHPCR interfaces, implemented by Jackalope. Jackalope uses the HTTP based DavEx protocol to talk to the Jackrabbit DavEx handler which is exposed by the servlet Magnolia plugin. Magnolia CMS is accessed through web requests and uses its in-process connection to Jackrabbit to read data. If Magnolia CMS needs to know about data changes done by the PHP application, it would have to use the observation feature of Jackrabbit to listen for changes on the content.

When not using Magnolia CMS, you could also run jackrabbit standalone or use the .war archive in any servlet container like Tomcat.

Question: Is there also support for the other “direction”? A case where Magnolia would use a PHPCR-backed repo/app instead of being the master?

Answer: This is partially answered above – JCR and PHPCR are APIs for code, not a protocol. A PHPCR server implementation would need to expose its content over DavEx for jackrabbit to connect to it. Besides the effort to build that, the scenario does not make too much sense. PHP is typically used in request-response setup where all objects are lost after a request. Such a server would be extremly slow as it would spend most of the time bootstrapping. While there are solutions for building PHP application servers like react, i don’t see a real benefit.
What can make sense is to configure Magnolia CMS to talk to a remote Jackrabbit rather than use the in-process Jackrabbit bindings. Then Magnolia CMS and the PHP client would both connect to the same repository outside of Magnolia CMS.

Question: Will the Magnolia DavEx Module be available as a Maven dependency?

Answer: It is available in the maven repository of Magnolia CMS. Here’s the dependency snippet:


You can use the repository, which contains all public artifacts from Magnolia, CE and Forge, releases and snapshots.

Or – for just the Magnolia Forge projects, either of these, depending on what you’re after:


Tags: , ,

Announcement: PHPCR and Magnolia CMS: Bridging the PHP and Java Worlds

Liip is a PHP company but we are not agnostic to what happens in other fields. And sometimes we need to integrate with other systems like a Java based CMS. Rather than using something radical like Quercus, a Java implementation of the PHP language or the rather fiddly PHP/Java Bridge, we wanted something less intrusive and more general purpose.

Taking inspriation from the proven Java Content Repository (JCR) standard, we created PHPCR by simplifying the Java interfaces. The content repository combines the best of document-oriented databases and XML databases, providing developers with a well-defined API to access and manage content. Ebi, Chregu and I started implementing the PHPCR interfaces in jackalope which can talk to Jackrabbit, the JCR standard implementation. The neat thing of this is that you can reuse Java content from a normal and simple PHP environment, through a clean and powerful API.

Last year we had a hackday where we integrated PHP Jackalope with the Java MagnoliaCMS. On 28 Feb 2013, I will present the topic as a MagnoliaCMS webinar entitled “Connect PHP Applications with Magnolia CMS through PHPCR”.

If you’re a PHP developer interested in getting your feet wet in the Java world, or a Java developer interesting in seeing PHP code talking to your favorite Java CMS, this webinar is a must-attend for you. You’ll find more information and a registration form for the webinar on the Magnolia website. Please subscribe and attend the presentation online!

Tags: , ,

PHP talking to Magnolia CMS

Since we started working on Jackalope, we always claimed it would also provide an integration point with other enterprise systems. Last week, we set out to proof this idea. Grégory Joseph of the Java based Magnolia CMS came to help us on the Magnolia side of things. Magnolia is using the JCR reference implementation Jackrabbit for storing its content. After an interesting exchange on their design decisions and our ideas, we started to hack.

Trying to import an XML export from Magnolia in the JCR standard format showed that the Jackalope importer is not yet perfect. In the end Greg was faster in building a Magnolia module that exposes the Jackrabbit Davex binding that Jackalope uses to communicate with the server.

After that, things really started flowing. We managed to not only read data, but also write using the frontend editing feature of the CMF. Additionally, we managed to configure PHPCR-ODM to determine document classes based on the MetaData child every Magnolia page node has. And all of that using several parallel connections to the repositories, so that the normal pages and navigation can still be loaded from our standalone jackrabbit backend (you could just as well use another PHPCR implementation like jackalope-doctrine).

All in all a very successful hackday. We integrated the demos into the cmf-sandbox (branch magnolia_integration). See the MagnoliaController in src/Sandbox/MagnoliaBundle/Controller.

Screencapture showing the edit functionality