August'24: Kamaelia is in maintenance mode and will recieve periodic updates, about twice a year, primarily targeted around Python 3 and ecosystem compatibility. PRs are always welcome. Latest Release: 1.14.32 (2024/3/24)

Kamaelia: Pragmatic Concurrency, A tutorial

This was a tutorial held at Europython '09 in Birmingham on the morning of Sunday 28 June 2009

Concurrency is viewed as an advanced topic by many developers, meaning how to handle concurrency pragmatically is often overlooked. However, many real world systems, including transportation, companies, electronics and Unix systems are highly concurrent and accessible by the majority of people. So, one motivation can be "many hands make light work".

With software this maxim often appears to be false - in no small part due to the tools we use to create concurrent systems. Despite this, the need for concurrency often creeps into many systems - even something as basic as "attaching a debugger".

Kamaelia is a toolset and mindset aimed at assisting in structuring your code such that you can focus on the problem you want to solve, but in a way that results in naturally reusable code that happens to be concurrent.

This tutorial aims to introduce this, and the core aspects of Kamaelia systems. (Accompanying notes)

Kamaelia Europython Tutorial from kamaelian

These core aspects are:

Axon

All Kamaelia systems are dependent on this core library - it provides you with tools for making systems which are naturally concurrent. Its primary metaphor is components with inboxes and outboxes, which get linked by parent components.

Kamaelia Components

The bulk of Kamaelia is actually a large collection of components. By themselves each component is useful, but their real power comes from being linked to each other, like programs in /usr/bin get linked together, forming pipelines.

Applications

This is the point of Kamaelia - to build useful systems. Kamaelia was originally designed for naturally concurrent problems, so there was a desire to make this simple(r) to work with. Using Axon based components means applications generate more reusable components, and also have a naturally concurrent structure. This can simplify many applications, allowing their reuse in unexpected ways.

Testing & Debugging

Testing & debugging concurrent systems is considered hard, we'll cover some approaches we can use in Kamaelia for debugging systems. Some of these are surprisingly familiar. Indeed, some can be used in non-Kamaelia based systems.

Kamaelia was designed originally to make maintenance of highly concurrent network systems simpler, but has general application in a wider variety of problem domains, including desktop applications, web backend systems (eg video transcode & SMS services), through to tools for teaching a child to read and write.

More Information

The rest of this website obviously contains more information, and based on lots of feedback, it does need work, but please do tell us how you think it needs work.

Also, a number of presentations have been made in the past. You can find these Kamaelia presentations on slideshare.

Notes

The notes are available in 3 forms:

One is PDF suitable for printing on A4 paper.

The other 2 ways are via lulu.com, which is a paid for print on demand service.

The other is as a printed book from lulu.com - this is basically at production cost (£1.89), meaning it's cheap, but beware of two points:

The slides used in the tutorial are available on slideshare. As you might expect, different media have different benefits, and so each way of explaining things covers the same things in slightly different ways, but following a common theme.

Videos will go up on blip.tv, and are currently being transcoded.

Two relevant downloads are also available: