jodal.dev - python

10 years of Mopidy

2019-12-23T00:00:00+00:00

Ten years ago today, on December 23, 2009, Mopidy was born. While chatting with my friend and then-colleague Johannes Knutsen, we came up with the idea of building an MPD server that could play music from Spotify instead of local files.

This is the story of the first decade of Mopidy.

pathlib and paths with arbitrary bytes

2019-12-10T00:00:00+00:00

The pathlib module was added to the standard library in Python 3.4 and is one of the many nice improvements that Python 3 has gained over the past decade. In three weeks, Python 3.5 will be the oldest version of Python that still receives security patches. This means that the presence of pathlib can soon be taken for granted on all Python installations, and the quest towards replacing os.path can begin for real.

In this post, I’ll have a look at how pathlib can be used to handle file paths containing arbitrary bytes, as this is valid on most file systems.

Pykka 2.0 released with better ergonomics and performance improvements

2019-05-07T00:00:00+00:00

I’ve finally released Pykka 2.0, the first major update to Pykka in almost six years.

Pykka is a Python implementation of the actor model. The actor model introduces some simple rules to control the sharing of state and cooperation between execution units, which makes it easier to build concurrent applications.

Pykka 2.0 is a major release because it is backwards incompatible in several minor ways. However, the backwards incompatible changes should only affect quite narrow use cases. Mopidy and its extensions, which is the largest open source ecosystem I know of that uses Pykka, runs unmodified on Pykka 2.0.

In this blog post I’ll go through some of the more important improvements in 2.0.

Mopidy-MPRIS 2.0 released

2018-12-07T00:00:00+00:00

I’ve released Mopidy-MPRIS 2.0, the first major update to Mopidy-MPRIS in about 3.5 years.

Mopidy-MPRIS is a Mopidy extension that makes Mopidy controllable from other programs on the same machine through D-Bus. This makes it possible to control Mopidy from various widgets in GNOME/KDE/etc, as well as with keyboard media keys.

Bringing the Mopidy music server to the browser

2017-08-26T00:00:00+00:00

Note: This is a blog post draft that was originally written in January 2013, left unattended for 1689 days, and rediscovered and published unedited in August 2017.

During the five years since Mopidy entered the browser with its HTTP JSON-RPC API and the Mopidy.js JavaScript library, many successful Mopidy web clients have been built on top of this foundation. The APIs themselves have survived the test of time and have required minimal maintenance, just as I hoped when implementing the APIs back in November 2012.

Mopidy is a music server written in Python. It plays music from various sources, including local disk and Spotify. Mopidy can be remote-controlled by, among others, MPD clients.

In Mopidy 0.10, released in the middle of December, we added an HTTP frontend. The HTTP frontend takes Mopidy’s full core API and makes it available from JavaScript in the browser. This means that you now can make your own web clients for Mopidy in JavaScript, and Wouter van Wijk has already started on his client (updated link).

I’d like to write a bit about how we made the HTTP client.

netsgiro: a parser and builder for AvtaleGiro and OCR Giro files

2017-05-21T00:00:00+00:00

Today I released the first stable release of netsgiro to PyPI. netsgiro is a Python 3.4+ library for parsing and building Nets “OCR” files.

AvtaleGiro is a direct debit solution that is in widespread use in Norway, with more than 15 000 companies offering it to their customers. OCR Giro is used by Nets and Norwegian banks to update payees on recent deposits to their bank accounts. In combination, AvtaleGiro and OCR Giro allows for a high level of automation of invoicing and payment processing.

The “OCR” file format and file format name originates in days when giro payments were delivered on paper to your bank and then processed either manually or using optical character recognition, OCR. I’m not sure how old the format is, but some of the examples in the OCR Giro specification use dates in 1993, and the specification changelog starts in 1999 and ends in 2003. A couple of decades later, the file format is still in daily use by Nets’ AvtaleGiro and OCR Giro services. In other words, I have high hopes that this will be a very stable open source project requiring minimal maintenance efforts.

March and April contributions

2016-04-30T00:00:00+00:00

The following is a short summary of my open source work in March and April, almost like in previous months, except that I haven’t spent as much time as previously on Open Source the last two months.

February contributions

2016-03-01T00:00:00+00:00

The following is a short summary of my open source work in February, just like in previous months.

January contributions

2016-02-01T00:00:00+00:00

The following is a short summary of my open source work in January, just like I did back in November and December.

December contributions

2016-01-01T00:00:00+00:00

The following is a short summary of my open source work in December, following up on my first report in November.

November contributions

2015-11-30T00:00:00+00:00

The following is a short summary of my open source work in November. My hope is that keeping better track of what I’m doing will help me reflect on how I spend my time, and help me to focus my efforts better.

Debian packaging of Mopidy

2014-10-28T00:00:00+00:00

My first upload to Debian as a Debian Maintainer, Mopidy 0.19.4-3, landed in Debian testing today, well in time for the upcoming freeze.

The new version adjusts the LSB facilities the sysvinit script depends on, to make sure that DNS lookups, DBus (used by Mopidy-MPRIS), and Avahi (used for Zeroconf service publishing) are available before Mopidy starts.

pyspotify v2.0.0b1 released, with new event loop and audio sinks

2014-04-24T00:00:00+00:00

pyspotify 2.x is a full rewrite of pyspotify. While pyspotify 1.x is a CPython C extension, pyspotify 2.x uses CFFI to make 100% of the libspotify C library available from Python. It works on CPython 2.7 and 3.2+, as well as PyPy 2.1+.

Comics v2.3.0 released with better mobile support

2014-04-07T00:00:00+00:00

Version 2.3.0 of my comics aggregator is now released. As always, dependencies have been updated, including the routine Django upgrade from Django 1.5 to 1.6, just in time for the upcoming 1.7 release. The largest change this time around is the move from Bootstrap 2 to 3, which includes a refreshed, flatter design and lots of tweaking to make Comics both look good and work nicely on mobile devices, something it didn’t use to do.

New Python wrapper for libspotify

2014-02-14T00:00:00+00:00

Today, after about nine months of working on it now and then, I uploaded the first alpha release of pyspotify 2.0 to PyPI.

pyspotify provides a Python interface to Spotify’s online music streaming service. With it you can access music metadata, search in Spotify’s library of 20+ million tracks, manage your Spotify playlists, and play music from Spotify. All from your own Python applications.

Mopidy packaged in Debian and Ubuntu

2013-12-28T00:00:00+00:00

Looking back to my last post from September, my goal was to have the music server Mopidy packaged in Debian by the end of the year and in Ubuntu in time for the 14.04 LTS release in April. As of yesterday, Mopidy 0.17.0 is a part of Debian unstable and Ubuntu trusty, just a few days after Mopidy’s fourth anniversary on December 23.

Pykka packaged in Debian and Ubuntu

2013-09-03T00:00:00+00:00

The Python actor library Pykka is now packaged as python-pykka and python3-pykka in Debian unstable and Ubuntu saucy, soon to be Ubuntu 13.10. In a few more days, the packages will be probably be available in Debian testing as well.

Mopidy hosted by Rackspace

2013-07-22T00:00:00+00:00

A week ago Jesse Noller tweeted that Rackspace would provide free Rackspace cloud accounts for open source projects. It only took a couple of hours from I first mailed Jesse to we had two servers up and running.

Pykka v1.2.0 with eventlet support and future transformations

2013-07-18T00:00:00+00:00

Pykka v1.2.0 is now available from PyPI, GitHub, and APT. It’s about half a year since the previous release, during which no serious bugs have been found.

Speeding up a Django web site without touching the code

2011-10-19T00:00:00+00:00

I’ve recently been tweaking my server setup for a Django 1.3 web site with the goal of making it a bit faster. Of course, there is a lot of speed to gain by improving e.g. the number of database queries needed to render a web page, but the server setup also has an effect on the web site performance. This is a log of my findings.

Traversable attributes in Pykka

2011-10-06T00:00:00+00:00

In Pykka 0.13–which was released almost two weeks ago–traversing the attributes of an actor is about 8.3 times faster than it used to be. To paraphrase Apple: “8.3X faster. That’s amazing!” (Update: This was written a couple of hours before the news of Jobs’ passing arrived. May he continue to inspire us.)

So, what is “traversable attributes”? Let’s take a few steps back.

pyspotify 1.2 released

2011-06-08T00:00:00+00:00

pyspotify is a Python wrapper for libspotify, which give developers access to the Spotify music streaming service.

Today, I tagged pyspotify 1.2 at GitHub and pushed the new release to PyPI. I’ve also made deb packages of libspotify and pyspotify available at apt.mopidy.com.

Log from the debugging of a segfault

2011-05-24T00:00:00+00:00

The following is a cleaned up log I wrote for myself while debugging a bug. Writing a log while working helps me keep track of the debugging effort in case I’m interrupted (life, sleep, work, etc.). It also requires me to explain all findings to myself in fully spelled out sentences, making my thoughts considerably easier to follow.

In addition to serving as an example of a personal debug log, I hope it can be useful as an introduction to debugging segfaults or other low-level bugs.

Pykka, and porting Pykka to Python 3

2011-03-29T00:00:00+00:00

This is somewhat of an introduction to Pykka, a Python library implementing the actor model, which I’ve been working on now and then since November. And it’s still just 300 lines of code!

The goal of the actor model is to make it easier to develop concurrent programs by removing all shared state and use messaging for all communication. Removing shared state and doing lots of messaging doesn’t make anything easier by itself, but it avoids common problems in concurrent programs like proper lock usage and data corruption (due to the absence of proper lock usage). Also, I believe the actor model makes it easier to reason about concurrent programs, which is a nice property since software development often feels like 10% development and 91% debugging.

multiprocessing.Connection is not free

2011-03-20T00:00:00+00:00

I’ve previously written about how to wrap multiprocessing.Connection objects in a class that makes Connection objects picklable, and thus transferable over another Connection. That approach is still valid, but don’t use it–or Connection objects at all–for lots of use-once-and-throw-away connections.

Traits in Python using multiple inheritance

2011-03-15T00:00:00+00:00

As you probably know, if a class needs to handle two mostly unrelated concerns, the class should probably be split into two. This way, your will achieve higher cohesion in your code, which is generally considered a good thing. Though, there are times where you need to address two mostly unrelated concerns in a single class.

The Scala programming language has a concept they call traits. Scala traits can be compared to Java interfaces, except that Scala traits may be partially or fully implemented, like abstract classes. As with Java interfaces, you can mixin multiple traits in a class. If any of the traits has colliding method signatures, the last trait mixed in overrides the earlier ones. Just as with regular inheritance and interfaces, the class must implement any methods that has not been implemented by the traits, and may override any methods that already has been implemented.

Pickling multiprocessing Connection objects

2011-03-06T00:00:00+00:00

For safe message-based communication between threads and processes in Python, I tend to use multiprocessing’s Queue and Pipe. A pattern often seen is using a queue for sending messages from multiple producers to a single consumer.

When a producer wants a response to its message, I create a Pipe and piggy-back one end of the Pipe (a Connection object) to the message. I use Python dicts as messages, and use the string “reply_to” as the dictionary key for the connection objects.

When the queue consumer processes a message, it doesn’t know who the sender is or how to reach him. Though, if the message has an attached Connection object, the consumer can–almost magically–respond to the sender, across thread and process boundaries.

All good? Nope.