jodal.dev - pykka

Pykka 2.0 released with better ergonomics and performance improvements

2019-05-07T00:00:00+00:00

I’ve finally released Pykka 2.0, the first major update to Pykka in almost six years.

Pykka is a Python implementation of the actor model. The actor model introduces some simple rules to control the sharing of state and cooperation between execution units, which makes it easier to build concurrent applications.

Pykka 2.0 is a major release because it is backwards incompatible in several minor ways. However, the backwards incompatible changes should only affect quite narrow use cases. Mopidy and its extensions, which is the largest open source ecosystem I know of that uses Pykka, runs unmodified on Pykka 2.0.

In this blog post I’ll go through some of the more important improvements in 2.0.

Pykka packaged in Debian and Ubuntu

2013-09-03T00:00:00+00:00

The Python actor library Pykka is now packaged as python-pykka and python3-pykka in Debian unstable and Ubuntu saucy, soon to be Ubuntu 13.10. In a few more days, the packages will be probably be available in Debian testing as well.

Pykka v1.2.0 with eventlet support and future transformations

2013-07-18T00:00:00+00:00

Pykka v1.2.0 is now available from PyPI, GitHub, and APT. It’s about half a year since the previous release, during which no serious bugs have been found.

Traversable attributes in Pykka

2011-10-06T00:00:00+00:00

In Pykka 0.13–which was released almost two weeks ago–traversing the attributes of an actor is about 8.3 times faster than it used to be. To paraphrase Apple: “8.3X faster. That’s amazing!” (Update: This was written a couple of hours before the news of Jobs’ passing arrived. May he continue to inspire us.)

So, what is “traversable attributes”? Let’s take a few steps back.

Pykka, and porting Pykka to Python 3

2011-03-29T00:00:00+00:00

This is somewhat of an introduction to Pykka, a Python library implementing the actor model, which I’ve been working on now and then since November. And it’s still just 300 lines of code!

The goal of the actor model is to make it easier to develop concurrent programs by removing all shared state and use messaging for all communication. Removing shared state and doing lots of messaging doesn’t make anything easier by itself, but it avoids common problems in concurrent programs like proper lock usage and data corruption (due to the absence of proper lock usage). Also, I believe the actor model makes it easier to reason about concurrent programs, which is a nice property since software development often feels like 10% development and 91% debugging.

multiprocessing.Connection is not free

2011-03-20T00:00:00+00:00

I’ve previously written about how to wrap multiprocessing.Connection objects in a class that makes Connection objects picklable, and thus transferable over another Connection. That approach is still valid, but don’t use it–or Connection objects at all–for lots of use-once-and-throw-away connections.

Traits in Python using multiple inheritance

2011-03-15T00:00:00+00:00

As you probably know, if a class needs to handle two mostly unrelated concerns, the class should probably be split into two. This way, your will achieve higher cohesion in your code, which is generally considered a good thing. Though, there are times where you need to address two mostly unrelated concerns in a single class.

The Scala programming language has a concept they call traits. Scala traits can be compared to Java interfaces, except that Scala traits may be partially or fully implemented, like abstract classes. As with Java interfaces, you can mixin multiple traits in a class. If any of the traits has colliding method signatures, the last trait mixed in overrides the earlier ones. Just as with regular inheritance and interfaces, the class must implement any methods that has not been implemented by the traits, and may override any methods that already has been implemented.

Pickling multiprocessing Connection objects

2011-03-06T00:00:00+00:00

For safe message-based communication between threads and processes in Python, I tend to use multiprocessing’s Queue and Pipe. A pattern often seen is using a queue for sending messages from multiple producers to a single consumer.

When a producer wants a response to its message, I create a Pipe and piggy-back one end of the Pipe (a Connection object) to the message. I use Python dicts as messages, and use the string “reply_to” as the dictionary key for the connection objects.

When the queue consumer processes a message, it doesn’t know who the sender is or how to reach him. Though, if the message has an attached Connection object, the consumer can–almost magically–respond to the sender, across thread and process boundaries.

All good? Nope.