Why did I change my mind about open data?

Knowledge against fear and suspicion – open data is beneficial

Generally disagreeing about any kind of data sharing, I realized my behavior was mostly based on fear. Fear is a major impediment to anything innovative and to any kind of change. Why did I change my mind about open data? It is about differentiating between public and private data, and about the fact that data made public are first of all edited.

New work – new ideas

In November 2015, I started working at Liip and I had a lot of new projects and inputs. The core of my work is the same, I completely changed field though. I stand now in the middle of a flow of innovative ideas and energy, which is very motivating and helps me be constantly open-minded.
One of my projects, last spring, was the coordination of Liip’s involvement at the annual opendata.ch conference. No, I cannot communicate about anything if I don’t understand it! Otherwise I would write complete bulls**t, people would notice it and Liip would lose all credibility on the subject. In other words, I had to know what I was talking about in order to be able to talk about it.

Fear & suspicion, people will be stalking me and CFF tickets will get more expensive

I used to completely disagree on any kind of open data. I mean, why, on Earth, would I be okay to share my personal data with the rest of the world?
Seriously, I was sure that, if I collaborate with the CFF surveys, tickets will get more expensive between Geneva and Lausanne, because they will know my commuting habits. I was convinced that Swisscom will soon be selling my personal data to private polls. If I use a MBudget Card, some people will be stalking me as they know that I always go grocery shopping at the same Migros.

Fear brings fear. It is a major impediment to anything innovative and to any kind of change. The first step to any change of perspective was me realizing that my reaction was based on some purely irrational feeling instead of rational information.

Let’s get started with some reading!

Knowledge the best enemy of fear: What is actually ‘open data’ ?

My education to open data started with learning about the projects that Liip developed (like the open data catalogue of the city of Zurich or the project with the Swiss Federal Archive). I discussed with my colleagues, but the concept of open data remained difficult to grasp.
What is concretely open data?

Step 1: A definition:

The definition of open data as given by the opendefinition.org

“Open data and content can be freely used, modified, and shared by anyone for any purpose”

As it is very general, the full open definition gives a detailed list of attributes. It is a bit of an unfriendly reading, the open data handbook provides a summary of the most important aspects:

“Availability and Access: the data must be available as a whole and at no more than a reasonable reproduction cost, preferably by downloading over the internet. The data must also be available in a convenient and modifiable form.
Re-use and Redistribution: the data must be provided under terms that permit re-use and redistribution including the intermixing with other datasets.
Universal Participation: everyone must be able to use, re-use and redistribute – there should be no discrimination against fields of endeavor or against persons or groups. For example, ‘non-commercial’ restrictions that would prevent ‘commercial’ use, or restrictions of use for certain purposes (e.g. only in education), are not allowed.”

The important thing to remember is that it is a type of content completely available to anyone – public or private sector or even an anonymous quidam – and for any kind of purpose.

Step 2: Private data is not public data

I find important to differentiate between private and public data. When I heard about open data, I felt my privacy instantly threatened. However, opening public data poses no threat to my privacy.
Take the example of cadastral plans, they are available on request. A formal petition to the right administrative service would provide you the info. Opening cadastral plans merely mean saving people working with it the time necessary for the administrative hurdle.

In other words public data is often already available through an administrative process and in an inconvenient format. Opening public data mostly signifies processing and editing them in a convenient format and leaving them in a digital library. Opening public data is about creating a digital library, allowing people to find them without the administrative process.

Private data is one’s individual data, for example, the data-gathered by your period application, your MBudget card, your mobile phone’s GPS tracking. This data is usually under confidentiality.

Opening private data, is not only about making it accessible, but also about editing it, to make sure that it cannot be linked to anyone.

What if open data was beneficial to the community?

Commuting to Geneva, I used to be angry with the CFF, because my Intercity Train was poorly connected to my inter-regio train which lead to me losing 15 to 20 minutes twice a day. Spending 2 hours, 2 hours 30 minutes or 3 hours to commute makes a huge difference. What if the CFF knew that a significant amount of commuters take these same trains as I did? They could delay the train 5 minutes and I would be happier everyday because my travel would be shorter! What if a mobile company shared its data about the mobility of people and made these data available to the CFF? What if I answered the poll? Or better, what if, instead of spending money on a poll, the CFF could access data of commuters? The data quality might be higher and more relevant!

Urbanism: Pully as a case study

The city of Pully is pioneering in the domain with its urban project in collaboration with Swisscom. The project team is analysing the traffic – car, public transportation, bike or pedestrian – based on data provided by Swisscom’s mobile network. (More info in French about this project).
The city of Pully saved the investment necessary for a poll and had reliable data available. Processed to be anonymized, this kind of data is not a threat to my privacy. In this case, opening the data is beneficial to the whole community.
The objective of this project is to develop the urbanisation of the city according to people’s real needs and not to any lobby. It could result in the creation of bike lanes or an improvement of traffic flows.

Pully – Photo Credit l’Avenue Digital Media

Conclusion : open data supports innovation

Sometimes I hear people disagreeing strongly against the concept of open data. I hear fear and threat. The discussion very often goes towards an emotional level rather than staying pragmatic. I find it difficult to rationally fight against emotional argument. First of all, I wish people against open data could make the difference between public and private data. Opening public data means the digitalisation of an already existing public data, ultimately saving public institution’s time.
Secondly, I wish they could realise that sharing information is beneficial to the community. The potential benefit of open data is greater than its threat. Opening data means, processing data, editing and anonymising it to make it available. Open data is the necessary basis of innovation and of a general increase of our quality of life.

At the moment, data can be made available, mostly by investing money (for a poll or negotiating with a company belonging data). In other words, universities, start-ups or associations are restricted to the data they can find or buy. Do we want to live in a world where innovation is driven by companies having the budget to pay for data?

Further Information:

Listen to this podcast, welcoming Pia Waugh, an open data expert and advocate who explains all to Genevieve Jacobs on ABC Radio Canberra.

Tags: , , , , , ,

Adobe After Effects: how to get started?

Adobe After Effect is an option to build animations. In this blog post, I share some of the tutorials and blogpost I have read in order to get me started with After Effect.

Animation with After Effects

Animations greatly improve the overall user experience of an app. They also bring different benefits to the user. In my last blog post about the SwiftAlps Conference, I wrote about the 12 basic principles of animations in design (see the slides in the blogpost). During the conference, we used CAEmitterLayer from Core Animation to insert a snow background effect.

Adobe After Effect is another option to build animations. After Effects is a powerful tool that is used to create digital visual effects and motion graphics. This tool is mainly used in film and television production although is also popular to create UI/UX animation among designers.

Adobe After Effects Logo

The main difference between After Effects and Swift

When it comes to design complex animations, After Effects is the way to go. Being able to craft an animation in real-time with visual feedback is easier than coding an animation in Swift and having to compile each time to see the results.

For custom animations such as animating the characters of a logo or animating the path of an icon then I would recommend doing it first with After Effects and then importing the animation int your app. If the animations are simple such as moving a layer of changing its alpha value then I’d just user your languages animation properties.

The most important advantage is visual feedback. Having visual feedback you can easily tweak and iterate an animation to achieve the expected desire. Then as we will explain in a future blog post, this animation can now easily be imported to be used in your app projects.

Furthermore, another important advantage is that After Effects is very popular among designers and animators, that said there are many online tutorial and youtube videos with examples of probably what you are trying to achieve.

Playing with After Effects

To get me started I watched the following. They are two video-tutorial that explain how to animate a company logo step-by-step. They have some good basic knowledge explanations that help me understand the basics of how to start.

Continue reading about Adobe After Effects: how to get started?

Tags: , , ,

The DrupalDay 2017 in Rome

This year was the 6th edition of the DrupalDay Italy, the main event to attend for Italian-speaking drupalists.

Previous editions took place in other main Italian cities like Milan, Bologna and Naples.
This time Rome had the privilege to host such a challenging event, ideally located in the Sapienza University Campus.

The non-profit event, was free of charge.

Continue reading about The DrupalDay 2017 in Rome

5 industrial challenges mobile applications can solve

How can mobile applications support industries to undertake a digital transformation? In supply chain, risk management, or information distribution, mobile applications will make tasks easier for your employees, thus increasing efficiency. Read about 5 industrial challenges that mobile apps can solve.

First of all, it is important to carefully chose between a web and a mobile application. Mobile apps are useful when they use one of the device’s native capabilities (e.g. Bluetooth, GPS, camera, etc.), and when they enhance the experience provided to the user (compared to what a web application could do).

Industrial challenges

The challenges industries face are often very good candidates for mobile apps as they leverage all their potential. Therefore, I have identified five recurring issues that industries are facing nowadays. These pain points can be easily relieved by mobile applications.

Challenge 1: Productivity issues due to technical limitations

Continue reading about 5 industrial challenges mobile applications can solve

Tags: , , , , , , , , , , , , ,

Going Crazy with Caching – HTTP Caching and Logged in Users

HTTP caching is an efficient way to make your application scalable and achieve great response times under heavy load. The basic assumption of HTTP caching is that, at least for some time, the same web request will lead to an identical response. As long as “same” means simply the same domain name and path, you will get many cache hits. When users are logged in, we have the opposite situation, where potentially everybody will see different content. Lets take a closer look to see where we can still find safe uses for HTTP caching even with logged in users.

Controlling the HTTP Cache Behaviour

A HTTP request is not only the URL, but also the headers. Some are only for statistics or not relevant for your application. But for some web applications, there are relevant headers. The Accept-Language header can be used to decide on the content language, or when building an API, the Accept header can be used to choose whether to encode the answer in JSON or XML.

HTTP responses can use the header Vary to declare what headers lead to distinct responses on the same URL. A HTTP cache uses the Vary to keep the variants of the same URL apart. This works well when there are few variants – you will still get frequent cache hits. However, if every request comes with a different header, caching on the server side makes no sense anymore. There is no benefit in storing results in the cache that will rarely be reused. Even worse, this is a waste of resources that should be used for caching relevant data.

For this reason, caching proxies like Varnish will by default not attempt any caching as soon as there is a Authorization or Cookie header present in the request. Cookies are commonly used to track a session in the application, meaning the user might see a personalized page that can not be shared with any other user. If you force caching with cookies and have your application send a Vary: Cookie header, you will have the situation described above, where you get no value out of your cache.

The rest of this article will dig into various aspects of what you can do to still do some HTTP caching:

  • Avoid Session Cookie, remove when no longer needed
  • Delegate to the frontend: “Cheating” with Javascript
  • Different cache rules for different parts
  • User Context: Cache by permission group

Continue reading about Going Crazy with Caching – HTTP Caching and Logged in Users

Tags: ,

Time for Coffee available on Android

Do you have time to take a coffee before your next public transportation connexion? Time for Coffee is a project initially started by François Terrier among friends in 2015.  We continued the work to make it available on further devices. 
Coffee cup

Do you have time for coffee before your next public transportation? Get the app!

When the Apple Watch came out, a few Liipers had the idea to make an app for it because having the next departures on the wrist was a perfect use case for this kind of device. The app received quite a lot of attentions in Swiss newspapers and received a Silver in the best of swiss apps in the category “Wearables & New Devices”. Since the Android world deserved also our attention, we made the app available for Android and Android Wear watches. The app is downloadable on the Play Store.

Continue reading about Time for Coffee available on Android

Tags: , , , , , , , ,

Great User Experience in E-Commerce starts with understanding customers!

How can we make an online shop that stands out better than overflowing competitors? Indeed online experience is offering useful information such as a comparison among several products with reviews. Researching became a typical behaviour for customers before making a decision. And there are even more interactions than only selling products on online shops.

In this article, I will describe the key practices how to capture customers needs in E-commerce by understanding them.

Continue reading about Great User Experience in E-Commerce starts with understanding customers!

Tags: , ,

The Liip.ch accessibility certification: a quick look back

Liip entered 2017 with a big success: liip.ch [1], our main website, has been certified as WCAG 2.0 AA compliant by the Access for All foundation.

Where it all began

Historically, Liip has always put great care into web content accessibility. We consider it as an important feature of any web project, for all kind of users, regardless any disability.
Of course, most of the time, customers are simply not aware of the topic at all and this is where we try to first make them know about it and then actively participate.
Depending on the sensibility of the customer, this may take long. But in the past, we have already helped a few of them to achieve the certification.

Ironically, though, our own website was not certified so we decided to start the official certification process in 2014. Without starting from scratch but using our freshly revamped website as a base.

Continue reading about The Liip.ch accessibility certification: a quick look back

Tags: ,

The Data Stack – Download the most complete overview of the data centric landscape.

This blog post offers an overview and PDF download of the data stack, thus all tools that might be needed for data collection, processing, storage, analysis and finally integrated business intelligence solutions.

(Web)-Developers are used to stacks, most prominent among them probably the LAMP Stack or more current the MEAN stack. On the other hand, I have not heard too many data scientists talking about so much about data stacks – may it because we think, that in a lot of cases all you need is some python a CSV, pandas, and scikit-learn to do the job.

But when we sat down recently with our team, I realized that we indeed use a myriad of different tools, frameworks, and SaaS solutions. I thought it would be useful to organize them in a meaningful data stack. I have not only included the tools we are using, but I sat down and started researching. It turned out into an extensive list aka. the data stack PDF. This poster will:

  • provide an overview of solutions available in the 5 layers (Sources, Processing, Storage, Analysis, Visualization)
  • offer you a way to discover new tools and
  • offer orientation in a very densely populated area

So without further ado, here is my data stack overview (Click to open PDF). Feel free to share it with your friends too.

Liip Data stack version 1.0

Liip data stack version 1.0

Click here to get notified by email when I release version 2.0 of the data stack.

Let me lay out some of the questions that guided me in researching each area and throw in my 5 cents while researching each one of them:

Continue reading about The Data Stack – Download the most complete overview of the data centric landscape.

Tags: , , , , , ,

Swift Alps Conference – experimentation and collaboration

The Swift Alps Conference has a special format focused on experimentation and collaboration. This is my report of the conference and the workshop Kilian and I held. The slides are available below.

Last month my colleague Kilian and I were pleased to attend the Swift Alps Conference, an experimental conference about Swift taking place in the Swiss Alps. This conference had a different format from what one can expect from a typical software development conference. In this case the format was more focused in experimenting and collaborating with other attendees with the goal of learning something new.

Experimenting with strangers

Continue reading about Swift Alps Conference – experimentation and collaboration

Tags: , , , , , ,