All posts by David Buchmann

Going Crazy with Caching – HTTP Caching and Logged in Users

HTTP caching is an efficient way to make your application scalable and achieve great response times under heavy load. The basic assumption of HTTP caching is that, at least for some time, the same web request will lead to an identical response. As long as “same” means simply the same domain name and path, you will get many cache hits. When users are logged in, we have the opposite situation, where potentially everybody will see different content. Lets take a closer look to see where we can still find safe uses for HTTP caching even with logged in users.

Controlling the HTTP Cache Behaviour

A HTTP request is not only the URL, but also the headers. Some are only for statistics or not relevant for your application. But for some web applications, there are relevant headers. The Accept-Language header can be used to decide on the content language, or when building an API, the Accept header can be used to choose whether to encode the answer in JSON or XML.

HTTP responses can use the header Vary to declare what headers lead to distinct responses on the same URL. A HTTP cache uses the Vary to keep the variants of the same URL apart. This works well when there are few variants – you will still get frequent cache hits. However, if every request comes with a different header, caching on the server side makes no sense anymore. There is no benefit in storing results in the cache that will rarely be reused. Even worse, this is a waste of resources that should be used for caching relevant data.

For this reason, caching proxies like Varnish will by default not attempt any caching as soon as there is a Authorization or Cookie header present in the request. Cookies are commonly used to track a session in the application, meaning the user might see a personalized page that can not be shared with any other user. If you force caching with cookies and have your application send a Vary: Cookie header, you will have the situation described above, where you get no value out of your cache.

The rest of this article will dig into various aspects of what you can do to still do some HTTP caching:

  • Avoid Session Cookie, remove when no longer needed
  • Delegate to the frontend: “Cheating” with Javascript
  • Different cache rules for different parts
  • User Context: Cache by permission group

Continue reading about Going Crazy with Caching – HTTP Caching and Logged in Users

Tags: ,

Fresh inputs from Confoo Vancouver 2016

I was invited to present at Confoo in Vancouver, Canada. I gave my talk “HTTP caching with Varnish” and a Symfony introduction. After having been invited to Confoo Montreal early this year, it was great to see how Confoo got an even broader range of technologies and languages covered. Many talks where on concepts first, rather than specific languages: HTTP, in-application caching, databases and so on. Other talks actively invited to learn new languages, like “Python for non-Python developers”.

Continue reading about Fresh inputs from Confoo Vancouver 2016

IPC: International PHP Conference 2016 in Munich

I have been invited to Munich to do two talks at the IPC. I gave my introduction to HTTP caching with Varnish and a talk on practical tools to build REST APIs. The IPC wanted some talks in German, so the slides are in german. You can find older versions of the slides in english for HTTP caching and REST APIs. I always enjoy presenting on a topic I care about, and the discussions after the talk. I am glad to help people, and more often than not, questions lead to me having to reflect why I have that opinion or outright learning something new. The organization of the conference has been flawless and the venue in the center of Munich was very convenient.

I could not stay for very long unfortunately, but managed to sit in to a few talks. Most notable was the talk on content strategy by Neos CMS core developer Robert Lemke with a lot of valuable information. I found the slides of his talk. The other talk I managed to see was by Michael Haeuslmann on dependencies in large projects. Michael is the developer of dephpend (pronounced “defend”), a tool to analyse dependencies of your PHP code and detecting architecture violations. He advocate such tools to identify the most important places to start improving a large code base.

Web Summercamp in Croatia – PHP track

David Buchmann introducing HTTPlug

HTTP client workshop in the PHP track of Web Summer Camp Rovinj

I was invited to do a workshop at Web Summer Camp in Rovinj (yes, last year it was called PHP Summer Camp – it got bigger this year). This year, I did a workshop on building HTTP clients in PHP. I did the workshop together with Márk Sági-Kazár, the lead author of HTTPlug, the HTTP client abstraction on top of PSR-7. We used PSR-7 and HTTPlug to build a client against an example API to manage TODO items. Then we showed how to do a Symfony integration for such a client, and how the HttplugBundle can be used to debug requests that are done in Symfony. The slides are online on slides.com – though note that this was a hands-on workshop, important parts of the workshop are not visible on these slides. The workshop was well received and I hope to teach it again in the future.

Web Summer Camp is unique in being a conference that only consists of 3 hour hands-on workshops and no only talks – aside from a lightning talks session. There are workshops in the morning and in the afternoon, so during the 3 days, you get to do 6 in depth-workshops. This year, there was the PHP track with 2 parallel workshops, and an eZ publish track as well as a UX / agile track. The Summer Camp is a community organized event with a great atmosphere, nice evening activities – and keeping the prices impressively low for what you get. The included boat trip on saturday is another unique and awesome thing. In short, you should strongly consider visiting the Summer Camp next year!

Testing in the Cloud – Using Bamboo with Amazon AWS

Bamboo is the continous integration service by Atlassian, the company owning the code management service Bitbucket (as well as the Jira issue tracker and Confluence wiki). Bamboo can run test suites and build any kind of artefact like generated documentation or installable packages. It integrates with Amazon Web Services, allowing to spin up EC2 instances as needed. So far, I mostly worked with travis-ci, because of open source projects I maintain on github.com. What Bamboo does really better than travis-ci – besides supporting other code repository SaaS than github.com – is the dynamic allocation of your own EC2 instances. Bamboo is just the manager, and can be configured to spin up EC2 instances when the number of tests to run increases. This keeps the costs at Amazon to a minimum while offering large capabilities to run tests when needed.

Besides licensing a Bamboo CI server to run yourself, you can also use it as a cloud service. I recently helped set up tests with this. Unfortunately, the documentation is really worse than expected, and the integration hampered by really silly mistakes that we had to dig up in discussion boards and on stackoverflow. This blog post contains a few notes if you want to do the same that hopefully will help others facing the same challenges. A word of caution: we did most of this in March and April 2016 – things might get fixed or change over time…

Continue reading about Testing in the Cloud – Using Bamboo with Amazon AWS

Tags: , , ,

Symfony: A Tool to Convert NelmioApiDocBundle to Swagger PHP

We have an API built with Symfony that outputs its specification in the Swagger format. We needed to upgrade from version 1 to 2. As we switched the library to generate the specification while upgrading, we had to convert the configuration. In our case that configuration was so extensive that we decided to build a script to convert the configuration.

Swagger is a standard to document REST APIs. Using a JSON file, an application can document its API. Swagger specifies the path for each resource and allowed HTTP methods, as well as input parameters and the returned data. On top of this specification, tools like Swagger UI can automatically provide an API client in a browser. This is an excellent way to explore the documentation and also very helpful when investigating data issues.

We have been using NelmioApiDocBundle with our application for a while now. This bundle reads annotations on the controllers and combines them with the Symfony routing informations to produce an API documentation in the Swagger 1 format. Support for Swagger version 2 however was not available in NelmioApiDocBundle at the time of this blog post. We would have stayed with NelmioApiDocBundle, as it worked well for us, but we did not want to invest the time to refactor that bundle to Swagger 2.

Continue reading about Symfony: A Tool to Convert NelmioApiDocBundle to Swagger PHP

Tags: ,

Report from the HTTP caching tutorial tour

Tutorial at BGPHP In the last month, i was invited to do my HTTP caching with Varnish tutorial twice. I was at the PHP summer camp in Rovinj, Croatia and at the first edition of the Bulgarian PHP conference BGPHP. Both where great conference experiences. Prior this year, I did a similar tutorial at PHP Benelux in Antwerp and DPC in Amsterdam. I would be open to give the tutorial for companies too – if you are interested please contact me.

Continue reading about Report from the HTTP caching tutorial tour

DPC – Dutch PHP Conference in Amsterdam

amsterdam

Last week, I was at the Dutch PHP Conference in Amsterdam. I did a workshop on content management with Symfony with the CMF and a workshop as well as a talk on HTTP caching and the Varnish reverse proxy together with David de Boer. I will give a similar tutorial at PHP Summer Camp in Rovinj, Croatia and at PHP Conference Bulgaria in Sofia later this year. Let me know if you are interested in having me give this tutorial at your company.

A nice perk of talking at a conference is of course that I also get to attend the conference and see other talks. There was a couple of interesting talks I managed to attend in Amsterdam. Most notably was Implement Single Sign On easily with Symfony by Sarah Khalil from Sensiolabs, who explained the authentication process of symfony so good that I finally feel like I really understand what is going on. The talk on HTTP/2 was mainly interesting for its detailed analysis of what was cleaned up in HTTP/1.1 since RFC 2616. Another good talk was by Arne Blankerts on the Content Security Policy (CSP). And I really enjoyed Daan van Berkels talk on Ada Lovelace and the Analytical Engine, he made maths and assembler sound like fun! There was also a code night in which David de Boer and I hosted a FOSHttpCache session with the main outcome of providing reusable varnish configuration files for the features of the library, instead of copy-paste documentation.

Besides the conference, there was of course also time to visit the beautiful city of Amsterdam. I had not visited the Netherlands since I was a small child. I even found the time for a trip to Friesland, visiting Schiermonnikoog in the Wadden Sea and taking lots of pictures of sand and birds. On the way back to Amsterdam, I stayed at Leeuwarden to do a Symfony CMF introduction at PHP Friesland Usergroup.

8 years at Liip

David LinkedIn 8 years

A couple of congratulations on linkedin made me notice that I now have been working for Liip since 8 years.
This is a very long time indeed. But I am not tired of working at Liip at all.

Of course, i occasionally thought whether I should move on and see other companies. But there was always some cool project to do or some interesting new challenge to tackle. When looking back at what I did in those past 8 years, it actually feels like I got a new job every couple of years.

Continue reading about 8 years at Liip

Improving the User Experience of the Symfony CMF Backend

Contrary to frontend experience, CMS backend experience is rarely adapted to the needs and tasks of its real users: the few people who publish and administrate the site. Standard backend themes often remain untouched after installation. The most common adaptations are the addition of menus and shortcuts, often due to addition of functionalities to the CMS.

We (David Buchmann and Benoit Pointet) spent some time to look into the backend we have for the liip.ch website, built with the Symfony CMF. We ran user testing sessions to collect our website’s editors painpoints, a community survey to collect the pain points of other CMF users, then did a lot of small tweaks both to the public bundles and in liip.ch specific code, and identified and documented improvements that needed more development work.

Continue reading about Improving the User Experience of the Symfony CMF Backend

Tags: ,