All posts by Lukas Kahwe Smith

Headless B2B marketplace?

The eCommerce world is undergoing a major new evolution step, moving away from tightly coupled monoliths towards modular monoliths. I know the word monolith has a lot of bad connotations in the IT world but I think this image is a bit of a knee jerk reaction. In most cases a monoliths is the right starting point for a project. The key thing is choosing a monolith that allows for evolution as the project scope or needs expand or change. This is something that eCommerce solutions of the past tended to fail at, hence the bad image for monoliths. Magento v1 is an awesome eCommerce platform as long as the needs for customization fit within the bounds hardcoded into the code base. However simply replacing Magento v1 by a micro service architecture would maybe prevent any hardcoded walls but would bring productivity to a halt. Martin Fowler states “don’t even consider microservices unless you have a system that’s too complex to manage as a monolith”. Or the corollary to this: Microservices must help to simply your system.

With this precursor out of the way, lets look at some of the emerging players in the eCommerce space and since I am currently evaluating options for a concrete project lets look at them with the requirement working as a headless B2B marketplace. But let me briefly explain those requirements

Headless

The customer is aiming to provide multiple touch points. As such there will be a web interface for desktop class monitors which will get the initial focus. But there will be potentially additional desktop optimized UIs but more importantly various mobile apps (maybe one focusing on purchasing, another focused on logistics etc) down the line. As such the eCommerce platform needs to make it possible to easily get to the data and business logic via some sort of API.

B2B

Business to business generally changes requirements quite a bit from changing the focus from presentation to efficiency. On a B2B website keyboard short cuts will have higher relevance than on a B2C site for example. Searching by SKUs or ability to scan barcodes are also more relevant. So are topics like permissions management to define who can fill shopping carts and who can complete purchases. Multiple shopping carts, shopping lists, repeat orders etc are also classic B2B requirements. Topic like up-selling are also relevant but might need a different spin due to the separation of people choosing products and people completing purchases. Also pricing is very different from B2C. Starting from using non VAT prices to customer and quantity specific rules.

Marketplace

On a marketplace there are more than one merchant selling products. Usually any product can be sold by multiple merchants each with their own price and stock. Meta data however tends to be normalized across all merchants creating additional challenges in how to import that data and ensuring high consistent quality. Shipping costs and even more importantly handling of returns also become considerably more complex.

So lets look at some of the contenders ..

Magento v2

I should rather say v2.2, since previous versions were plagued by bugs and releases contained a lot of backward compatibility issues. The new release promises to finally stabilize the APIs and a good sign here is that finally Magento is more actively integrating the feedback of the community into the release and development process. My fellow Liiper Maxim is quite happy with how things are going there. I am not entirely sure why they build so much infrastructure code rather than just using ZF2+ or Symfony but over all they implemented similar patterns which enable easier refactoring and extensibility. Magento is also taking B2B more and more seriously but out of the box the feature set still needs a bit work. Overall Magento v2 is a big step forward from v1 however its also clear that the code was refactored not entirely reinvented. The big advantage of Magento is of course still its huge community. While the code is open source, for bigger projects we would recommend using their Enterprise offering. Unfortunately they do not seem to provide a good overview of the differences.

Spryker

Spryker is kind of a hype at the moment. I say hype not in the negative sense here, they simply managed to get a lot of attention very quickly. Quite impressive! One of the things that makes Spryker unique is that they denormalize relational data into Redis, which enables high performance without a reverse proxy. That being said, reverse proxies and invalidation are challenges but they are well understood. Also when working with international customers, a reverse proxy in different regions around the world might be necessary anyway. Keeping the data in Redis sufficiently in sync provides a challenge that is unique to Spryker. On the siroop.ch project, who are using an older version of Spryker, I saw this was a constant source for issues. They are rewriting this part right now so hopefully this will be less painful in the future. They also told me that they will improve the management of the data contract between the system writing to Redis (Zed) and reading from Redis (Yves) more explict. Another aspect which sets Spryker apart is that their backend is designed to enable separating f.e. user management and ordered management onto different servers if necessary. This architecture could lend itself to flowing more naturally into a micro service architecture. They are planning to add B2B features and even marketplace functionality to the core, but these features will only start appearing in the coming months. In the same way while Zed and Yves talk via HTTP, Yves itself isn’t made for headless setups out of the box. Here again they are planning to soon over a solution. They are hopeful that the first such features will start landing this year already. Their licensing is a bit unique in that one pays for developer seats.

My main criticism of Spryker is that due to being based on Silex, rather than on Symfony full stack, they are lacking a lot of developer productivity tools. For example to solve performance issues, they have multiple independent Pimple, ie. DI containers, for each module. This in turn means its impossible to get a global overview of available services. For the routing they could in theory provide something like this fairly easily but have not emphasized this.

Sylius

I followed this project from its very beginning when Pawel started appearing on the #symfony IRC channel. His attention to quality impressed me right away and really shows in Sylius starting with the use of BDD which is also a great way to document behavior via test code. While initially this perfectionism prevented him building a community, the project has long overcome this and is now quite an active community. On top of this there is now a company dedicated to Sylius support and training. The code is entirely open source, no enterprise versions or other licensing fees. They are now very close to that very first 1.0 release. However releases for a while have come with quite detailed upgrade instructions whenever backwards compatibility breaks had to be done. In terms of modularity I think they are on par with Spryker. Given that they are build on Symfony fullstack framework, its also easy to leverage and integrate with the countless Bundles out there. Their large set of re-useable components are also ready to be used even outside of Sylius itself. I am also quite excited about their attention to single page apps (SPA).

OroCommerce

The Oro team, given that they are essentially a lof of ex-Magento core developers, is certainly quite experienced in eCommerce. Their first product was OroPlatform and OroCRM, which are both interested topics for customers that want to highly customize their CRM or that simply want to build desktop-style custom applications. OroCommerce is a bit younger but they emphasized B2B very early on. I suspect because they sensed that Magento v2 didn’t initially put much of a focus on B2B. Similar to Sylius they are based on Symfony fullstack. In terms of licensing they offer a free open source basic version but the Enterprise version seems necessary for larger projects. I feel like they have not yet really managed to build a large community.

DrupalCommerce v2

While Drupal 8 certainly improved its core towards easier and cleaner extensibility by adopting interfaces, I would still argue that form the lists in this post, its still the one that is most like the monoliths of the old days. However DrupalCommerce still manages to be remarkably extensible. We build Freitag.ch on Drupal 7 and Commerce v1 which is very unique in that most products on the store are unique. Their next version, which builds on Drupal 8, looks to further improve here. They basically started by first building re-useable components which they then integrated into Drupal to build version 2 which just reached RC1 status. Where DrupalCommerce has always shined, which is the reason why we picked it for Freitag.ch, was the fact that that through Drupal it provides a full featured CMS where the commerce parts are first class citizens. This means that advanced story telling is possible, entirely driven by content authors, rather than requiring a development sprint for each bigger change. As such, story telling does not tend to be a key requirement for B2B for now but I think it will become more relevant. Those companies that have been holding out with adoption digital B2B mostly did so exactly because they felt that with the approach most companies are taking, the inspiration part of the commerce experience was lacking. The code is fully open source and consulting offerings are available

What it all means ..?

Quite honestly I am very impressed with this new generation of eCommerce solutions. All of them have different strengths and weaknesses.

Spryker has a unique architecture that sets it apart from the rest, which however also means in terms of minimal setup it has the most moving parts. Sylius, OroCommerce and DrupalCommerce all are based on a larger ecosystem which will allow integration of other features and even applications quite seamlessly.

Sylius and DrupalCommerce are fully open source, while Magento and OroCommerce provide Enterprise offerings which are really sort of required for serious work. Spryker again goes a different direction by requiring developer licenses. They all have companies behind them that offer commercial support when needed. In terms of community Magento seems to have the lead and OroCommerce seems to be struggling here a bit.

Sylius is the only option that from what I am aware have stated to have headless/SPA as a focus. All of them currently provide the key APIs necessary with Spryker lagging behind a bit since Yves out of the box does not provide any API yet.

What all of the projects on this list have in common: They are still in a fairly young state of development either only gone stable in the past year or two .. or even just approaching the first stable release. The good news is of course this means that there will still many years of support for those versions and the ecosystems will continue to grow. However in terms of getting things done quickly for more standard eCommerce requirements, Magento v1 probably still beats them all.

Overview of open source API gateways

HTTP based APIs have long established themselves as a successful pattern for organizations. Increasingly these APIs are made available to the public or at least are leveraged more and more by disconnected development teams within organizations. Where the first APIs just drove the live search on a website, APIs these days provide extensive functionality for internal and external development. As such there is a need to centralize access to documentation, authentication and permissions so that users can easily discover and leverage those APIs in a way that prevents negative effects for other users.

As often when new patterns emerge, a new type of software solution emerges, in this case API gateways. Given our affinity towards open source here at Liip, we have studied the market a bit and want to present a very high level overview. We would very much appreciate additional first hand experience feedback in the comments below!

3scale

3scale was bought by RedHat in 2016 and subsequently open sourced at github.com/3scale. We have not tried to set it up ourselves but from past experience, previously proprietary software can tend to be tricky to get running. The product covers all the key pieces: API management, rate limiting, access control, analytics. There is a hosted option starting at $750 per month with 500k API calls per day and some other limitations.

WSO2

Originally created at IBM, Wso2 has a close affinity to the Apache community. It can be self-hosted but the company behind this project also offers a hosted cloud solution. Setup for a quick proof of concept was simple and we had a proxy running within 10mins. We found the UI a bit complicated and limiting and ran into some errors when we tried to save our definitions.

Kong

Mashape build Kong on top of Nginx, which is the web server of choice for most of our projects these days. They originally required Cassandra for config management but since version 0.8 they also support PostgreSQL. The fact that they are not yet 1.0 makes me a bit nervous but we didn’t find much complaints about backwards compatibility issues from a web search. Anyone have some practical experience to share here? Mashape of course also offer a hosted enterprise version but no word on pricing on their website. They do not seem to offer an admin GUI as part of their open source offering but there are quite a number of open source options available. There are quite a lot of available plugins and writing your own in Lua isn’t too hard.

Tyk

Another option that makes it easy to run locally or in the cloud is Tyk. The dashboard requires a commercial license but for on-premise its free for a one node setup. The hybrid setup is an interesting option as it allows you to keep the API calls in your datacenter while leaving the dashboard and management to the cloud.  The current version assumes that the backend API is secured by IP whitelisting but they are looking to improve here. Setup was very easy and we were up and running within minutes. Tyk seems to focus on simplicity which is both good and bad.

We gained in depth first hand experience with Tyk by setting up the opentransportdata.swiss API gateway last year.

Update: We originally incorrectly claimed that the on-premise did not provide a dashboard. For on-premise a dashboard license can be bought, its free for 1 node setups. I remove pricing information since there are simply too many options to choose from given that they provide on-premise, cloud and hybrid. The good news is that for cloud there is a free tier to start with for upto 50k calls per day.

Take away thoughts

In general there seems to be quite a lot of solid choices for open source API gateways. They all check most of the boxes. They also all provide some sort of commercial/hosted option. So in the end it seems that the devil is in the details and as such from the point of view of an agency, it makes sense to standardize as much as possible. Given that we are running a large project already on Tyk, it makes sense for Liip and our customers to lean towards Tyk.

APIs for the public sector

I recently gave a presentation (in German) at the Beschaffungskonferenz. This is a conference for the public sector to exchange round procurement of IT. There were several tracks some focusing more on legal aspects, different procurement processes and agile development while I presented in the tech track. In my talk I presented some of the more established new development paradigms of the past years. But the key message was that APIs need to become a key aspect of how IT projects are planned for the public sector. Specifically I named transport.opendata.ch as a shining example of how providing existing data via a public API can lead to an entirely new economy of use cases on top of it. The idea is really: “Built it and they will come”.

Update: Another good example of a well documented API in the public sector is api3.geo.admin.ch which we have used for various projects here at Liip in the past already.

Update 2: An article which provides an additional perspective: API First at data.gov.uk

Continue reading about APIs for the public sector

Startup tool inspiration – what we use

Liip has a fair number of startup customers who often struggle with finding the right set of tools, so I will share here a bit what we are using on a daily basis. We traditionally use a lot of open source tools in our projects. For our infrastructure tooling we also use a fair bit of open source but also an ever increasing amount of SaaS products. Additionally we build some tools internally, some of which we have made open source. One of my holacracy roles is called “Platform Gardener” with the purpose “Provide corporate-wide streamlined digital services and tools”. This role gives me a pretty good overview of the tools we use, which I would like to share below.

Continue reading about Startup tool inspiration – what we use

Symfony: A look back and what it all means

As we were preparing the news about becoming a Sensiolabs Silver Partner, I brought back a bit to the history of Symfony here at Liip. We did do a few symfony v1 projects at Liip but things only really took off with Symfony2. Back in 2009 Fabien came to Zurich to discuss some of the Symfony2 components (still PHP 5.2 compatible at the time) he had just released as well as a few he hadn’t yet released. Jordi, who was working at Liip at the time, and I integrated all of them into our company internal framework over the following months which we later presented at the Symfony Live. This means Liip in fact build the first Symfony2 framework, even before there was the official Symfony framework.

Continue reading about Symfony: A look back and what it all means

Tags: , ,

Playing well with others

I was invited to speak at the T3DD15, the TYPO3 Developer Days 2015, on the topic of community collaboration. Indeed my presence there was also made happen because there was a big interest both from the TYPO3 CMS as well as from Neos CMS to explore adopting Symfony. Another topic was adopting of PHPCR, again both TYPO3 and Neos CMS expressed an interest in this topic, though not as urgently as adopting Symfony. In fact the Symfony adoption is already under way with TYPO3 CMS developers having already integrated the Console component and looking towards the Routing component next. Neos CMS has long ago adopted the Yaml component and are looking towards the Process component next. Indeed especially Neos CMS might be beginning to adopt quite a few more Symfony, and other components. They are even willing to explore going to Symfony full stack, though it would need to be made compatible with their unique Proxy based AOP solution that enables them to effectively overload the “new” operator.

Anyway, back to the title of this blog post. I did in fact choose the same title for my talk in which I tried to give some considerations about legal but more importantly community topics related to working with third party projects. I tried to make it clear that there valid points for choosing to not adopt third party solutions in every case. I also tried to address a bit how to get people from other communities more involved into the TYPO3 and Neos CMS communities. Overall I must say I am still not quite as fluent with soft topic talks compared to hard topic, ie. code related, talks. I still do not know how much content to put on the slides and how much I can free style. I also still have a hard time estimating how long my talk will last. I guess I will get better with practice.

It was also an interesting time to give this talk, as TYPO3 and Neos CMS only recently decided to part ways. It is not clear if this will be the last joint conference. During my talk I wanted to make things a bit more concrete, especially when discussing points about what might prevent people from collaborating with TYPO3 and Neos CMS. It felt a bit strange as I was mentioning issues with each of the projects. I was worried that I might set of a flamewar, but everything seemed to stay quite civil :)

At any rate, I spend two great days with lots of interesting discussions. The location was awesome, essentially right in the Nürnburg castle, with lots of rooms for people to gather to discuss and hack. Aside from the Symfony and PHPCR related discussions I also entered into a session on diversity as well as on CQRS. Overall it was a very productive environment and a fun exchange with lots of old and new faces!

Tags: , , ,