A recommender system for Slack with Pandas & Flask

Recommender systems have been a pet peeve of me for a long time, and recently I thought why not use these things to make my life easier at liip. We have a great community within the company, where most of our communication takes place on Slack. To the people born before 1990: Slack is something like irc channels only that you use it for your company and try to replace Email communication with it. (It is a quite debated topic if it is a good idea to replace Email with Slack)

So at liip we have a slack channel for everything, for #machine-learning (for topics related to machine learning), for #zh-staff (where Zürich staff announcments are made), for #lambda (my team slack channel) and so on. Everybody can create a Slack channel, invite people, and discuss interactively there. What I always found a little bit hard was «How do I know which channels to join?», since we have over 700 of those nowadays.

Bildschirmfoto 2016-06-16 um 11.34.12

Wouldn’t it be cool if I had a tool that tells me, well if you like machine-learning why don’t you join our #bi (Business Intelligence) channel? Since Slack does not have this built in, I thought lets build it and show you guys how to integrate the Slack-API, Pandas (a multipurpose data scientist tool), Flask (a tiny python web server) and Heroku (a place to host your apps).

Continue reading about A recommender system for Slack with Pandas & Flask

Tags: , , , , ,

Get started exploring Google Analytics data with Python Pandas

The latest release of Pandas (v0.17.1) has brought the deprecation of the Google Analytics data reader submodule (pandas.io.ga). This deprecation decision is actually good news since this submodule had dependencies on packages that are not currently python 3 compatible and was, even under python 2.7, hard to get up and running.

After updating my system to the newest versions of pandas, I had to find a new connector to fetch Google Analytics data, and found an advantageous replacement in the google2pandas module from Panalysis.

This blogpost walks you through the setup of Pandas and google2pandas, and breifly introduces you to fetching and getting Google Analytics data into Pandas dataframes, for further exploration with Pandas.

Continue reading about Get started exploring Google Analytics data with Python Pandas

Tags: , ,

Good vibes at DjangoCon Europe 2015

18510465600_d9ebaecc2f_k

Last year’s DjangoCon Europe took place on its own island, and this time we were welcomed by the beautiful Cardiff University. One of the goals the organizers immediately set was to promote diversity and embrace minorities, encouraging them to participate to this event. I remember when I bought my ticket and I read “all dietary requirements can be met” and I thought “they’re probably trying to set the bar too high”. This is a not so unusual statement, but when confronted to reality you usually end up with dishes not adapted to your dietary requirements, and, as a minority, having to ask for something special can really make you feel excluded. This time was different, as the organizers meticulously took care of that.

I talked about dietary requirements because it’s an important topic for me, but they also made sure to have a code of conduct, a financial assistance for people who want but can’t afford to attend, a wellbeing service, a subscription system for meals to avoid wasting food, reusable aluminum bottles instead of plastic water bottles, live transcription of talks (for hard of hearing or non-english people), and even a crèche.

All of this was not just buzzwords but was carefully handled by the organizers. In the end, it made everyone feel welcome and put a very warm atmosphere to the event. Talking with people was very interesting and rewarding because of the incredible diversity of backgrounds. A few days after the conference the Django Software Foundation even released a diversity statement to make sure everyone feels welcome in the Django community.

The choice of talks was about 50/50 between technical talks and community talks. I found the community talks to be very inspiring such as Ola Sendecka’s “Into the rabbit hole” which reminded me the benefits of the Pomodoro technique and pair programming. Also Adrienne Lowe’s talk “Coding with knives” and Russell Keith-Magee’s touching talk about burnout debunked the myth of the rock-star programmer, and how important it is to help people to get started on things they’re not comfortable with.

I can’t list all the talks I loved because there are way too many but among those I could find the slides for were Ana Balica’s “Demystifying mixins with Django“, Erik Romijn’s “A sincere tale of Django, developers and security“, Aaron Bassett’s “Effortless real time apps in Django“, James Bennett’s “The net is dark and full of terrors“, and of course Rae Knowler’s “Randomised testing for Django with Hypothesis“.

The 3 days of talks were followed by 2 days of sprints and this was a very good opportunity to dive into Django’s code and start fixing things. The core developers did a good job at helping people setup a development environment and start hacking on the code. This was a very rewarding part of the conference as this allowed me to do my first pull requests on Django and Django CMS.

Thanks to the DjangoCon Europe team who did an amazing job at organizing this event. See you next year in Budapest!

Tags: , ,

EdX Open Source Platform

Why edX?

Recently we decided to explore the edX platform. It’s an open source Learning Management System (LMS) and authoring tool. EdX is one of the most popular MOOC platforms alongside Coursera and Udacity, but unlike the two others their code is open source and you can host your own MOOC platform. EdX has been created by MIT and Harvard and is listing other well known universities as contributors. Also, in September 2013, Google, who was previously experimenting its own LMS (Google Course Builder), decided to join the open edX platform as a contributor too. The platform is implemented mostly in Python and they created their own component architecture called XBlock (https://xblock.readthedocs.org/en/latest/). On paper edX sounds great, but how does it compare to Moodle or Totara? Does it fulfill the criteria our clients are targeting?

First steps

Installing a local instance is fairly easy since they provide a vagrant box and the documentation is good. But on the other hand, you will need to run three different servers with three different port forwardings. One for the LMS, one for the CMS (authoring tool) and the last one for the forums (this one is written in Ruby). Moodle or Totara use the same tool for standard users and administrative users which can lead to a bad user experience. So having different tools can have advantages and disadvantages.

Managing users

One key feature Moodle and Totara are offering is the user management. This is also one of the first requests we usually receive when dealing with offers. With Moodle and Totara there are lots of ways to authenticate a user and also to enroll them to a course out of the box. You can also easily manage roles and create groups (even dynamic groups with Totara).
In edX a user can authenticate if he creates first an account on the platform but edX is built on Django so any authentication method from this framework should be available on edX too (e.g. LDAP authentication).
It was not so easy to find how you can restrict the access to a course, this seems to be feasible depending on different settings (they were not explained in the user documentation, but understandable from the code documentation). You can create groups but it’s not clear yet if you can have groups platform-wide too.
Inside a course it’s easy to manage your staff team, define your teaching assistants or other admin users. You can even add beta testers so that some students can test your course before its official public release. You can also have discussion moderators for your courses.

Multilanguage

Having the platform in several languages is possible, and courses in different languages too. But you cannot have the same course in different languages (which is the same for Moodle and Totara, you simply need to create a course for each language). A good and bad thing with the current state of internationalization is that it’s a work in progress. It’s bad because if you want a german edX platform, the translations are not finished yet. But it’s good because there are working on it and plan to finish it for the end of second quarter 2014 and they have a translation tool where anyone can contribute (https://www.transifex.com/organization/open-edx). By default, it would use the browser language, but if you’re logged in you can also have a user language (at least it’s configured that way on www.edx.org).
Hosting
Configuring manually your server to host an edX platform is really difficult, there are a lot of requirements and dependencies to install. For Debian some of them don’t even have a package. But they provide a way to install it using Ansible requirements which seems quite easy to do. They also provide some script to execute update which is nice.

Interaction between users

Like on Moodle and Totara, you can have discussion forums for a course. There is a nice WYSIWYG editor with the possibility to preview your post first.
A nice feature of edX is the ability to comment any exercise or theory page and allowing others to upvote/report those. It’s a pity though that the comments aren’t ordered by upvotes by default.
Students can find study buddies using Meetup communities. EdX can also integrate Google Hangouts.

Reporting

EdX comes with basic reporting tools. One can generate for example the list of enrolled students and the grade reports. Compared with Totara’s custom reports feature, edX has still plenty of room for improvement.

User Interface

Generally the interface looks very nice. The UX-UI concept is very strong and has a clear consistent interaction level. You can also implement custom themes as Stanford did for its own instance. Without going into details, it seems edX has a nice base theme and state-of-the-art Interaction Patterns but they don’t have a responsive theme – only the home page is responsive.

Learning experience

The learning experience is appealing and has the same feel as new online platforms (team-treehouse, codeacadamy, …). Courses offer a lot of options out of the box: video integration (via Youtube), multiple question forms (radio buttons, checkboxes, dropdowns, text, maths inputs, …), Google Hangouts, discussions, text, announcements, integrating an Iframe, full-screen images, zoom images, imaged-mapped input, drag and drop answers, electrical schemas, LTI integration, … but there are bad things too. For example, the grading system is not suitable for the usual Swiss grading system (grades from 1 to 6), it’s apparently made for the American grading system (from F to A).

Conclusion

EdX does less things than Moodle but does them nicely. It’s also better suited for open education which can be achieved in Moodle and Totara too but the user and learning experience seems to be better on edX. Sever configuration, user management, access restriction, internationalization and reporting, all those things are easier to set and ready to use out of the box with Moodle (reporting especially on Totara) and cannot be neglected. Some might say that Moodle does too many things but from a business point of view it’s easier to adapt client’s requests that way. Also the community around Moodle is bigger which is a great thing from a developer point of view. But to end on a positive note, edX provides a roadmap on what they are working or will work on, thus we will keep an eye on this innovative platform and check if their improvements could fit future client requests.

Tags: , , ,