August'24: Kamaelia is in maintenance mode and will recieve periodic updates, about twice a year, primarily targeted around Python 3 and ecosystem compatibility. PRs are always welcome. Latest Release: 1.14.32 (2024/3/24)

Kamaelia Project Development

How do you work, how can I join in?

See also, how to contribute to Kamaelia

NOTE: Needs updating to deal with creation of Kamaelia.Community. Consider turning into a tabbed document instead.

Since Kamaelia is exploring a new way of building software in many respects, much of our development work goes into experimenting with ideas, as well as the core codebase. We've established a simple process to help us do this. More detail below, but here are the headlines:

Version control using SVN, courtesy of sourceforge

Experiment in/trunk/Sketches/YOUR_INITIALS

Main code base in /trunk/Code

Use Branches used for substantial changes

Merges should be done by someone else (unless the project admins give you permission)

This page is largely about doing stuff, so we have a separate page about meetings, and stuff like that.

Why all this? We want to be able to explore ideas in a shared collaborative environment without imposing too many risks (bad builds, bad code, bad design decisions, etc). Hopefully you'll think this a sensible way to get things done (please tell us if you disagree) and you'll do something similar. If it works for you, please do consider asking for access to use the the /Sketches space. In practice this should all be quite light touch - despite the length of this document!

Version control

All development for Kamaelia is done under version control using the sourceforge SVN service. See our info on how to access the repository.

All file paths below are relative to the trunk which is where the bulk of development happens.

For reference, the trunk currently contains the following top level entries:

/Sketches (dir)

/Code (dir)

/Tests (dir)

/AUTHORS, /COPYING, /CVSROOT (dir), /Other (dir), /Website (dir)

Any re-development of code in /Code (such as large scale refactoring) should be done on a branch (see below). To make life simple for everyone, the branch should be of the entire development tree.

What do I do then?!

So what do you do?

Experiment in /trunk/Sketches/INITIALS

Move working things into /trunk/Code

Solidify into something we can trust

Add test suites

Preferably re-implement test first

Make large swathe changes in branches called private_INITIALS_AnythingYouWant

Be prepared to merge other people's branches

If all else fails, become an agent of CHAOS :-)

Read on...

Exploratory Development in/Sketches

Why sketches?

Development can be an engineering discipline - if you know your requirements you can start with a spec and work forwards. This applies to both "Big design up front" and to a lesser extent many agile development processes. Alternatively, development can be like writing or painting, where the artist will create many sketches before taking the best and integrating them into a single coherant whole. Long term developers will often experiment, writing small throwaway stubs of code to see how something might work - especially if there are no clear requirements to start off with. You can then often understand the requirements better and then work on an engineered approach.

Kamaelia's development process is designed to support the creative "we don't know what we want, so let's sketch out a few ideas" development through to the fully engineered test-driven development approach. The /Sketches area is a place to do this experimentation and prototyping.

Both viewpoints have merit, and one group may term itself hackers (not to be confused with crackers) or even artists, the other may term themselves engineers or even computer scientists. In reality of development is often a mixture of both. We therefore hope to support both approaches in a reasonable fashion.

Creating working space in/Sketches

Suppose your name is Mr Warren W. King and you want to create QT based components. You probably need to experiment first, after all you don't know exactly what you want, and you may need to change other existing components, or maybe even Axon.

First, create yourslef some working space. You have the right to create a subdirectory named your initials: /Sketches/WWK/ . So do that:

This directory is yours to do with as you please, without clashing with other people. If WWK is gone (say Wendy Wescot-Krinchbottom took it already) just use your full name instead or something else differentiating!

In this particular case, it would make sense to create the following directories for development at that stage:

You can then experiement with various implementations. You might do one implementation, or you might do a variety. They may, or may not, become full Kamaelia components or systems. The point is though, you can develop any way you like. The advantage of doing this in the shared repository is that if you abandon your work, someone else can always pick it up later on. They'll know it might not perfect or complete, and might not have been meant seriously - because it's in sketches. But by having a shared sketch book you may save someone else time later on. Obviously you also benefit from the shared code sketch book.

Suppose you're now happy that your code works and is useful, and now you want to see how it might work in the repository.

Migrating from/Sketchesinto/Code

This is the beginning of changing from an experimental mode to a mode where we can say "this is something we find useful". As a result we take some care in thinking about where code should go and the naming of component before they enter the repository. The guidelines are really as follows:

Please note: you might do a sketch, and put it forward to going into /Code, or equally someone else may do so. Don't be offended if the latter happens!

What's the structure of/Code?

In order to understand the where something might fit, and what it might be named it's useful to know the current structure. This discussion will focus on the Python code tree.

Currently /Code is subdivided by language, then major projects & bindings as follows:

/Code/Python is subdivided into the following key directories:

In the general case, most of the time the result of work on actual components will end up in /Code/Python/Kamaelia. For example Kamaelia.Internet.UDP.SimplePeer resides inside the file /Code/Python/Kamaelia/Kamaelia/Internet/UDP.py (at present).

Before making changes to /Code (on a shared branch - such as a default checkout), please discuss them first on either #kamaelia on freenode's IRC or on the kamaelia mailing list. You will find we are generally accommodating towards requests, but ask you to have some patience, since we might have nefarious plans you're not aware of :-)

Criterion for something going in/Code

Code that goes into /Code is expected to be a candidate for adding to the RELEASE branch at some point, but is not automatically added. Due to this expected next step, it is expected that people committing to the SVN trunk in /Code/Python/Kamaelia will discuss what they're doing either on IRC or on the mailing list, or similar. Should a process be needed, it will be invented as it becomes apparent :-) It will probably take a form similar to that used on python-dev. Project admins act as arbiters if there's a loggerhead.

The basic criterion for something going into /Code is simply "this is something we've found useful for , and would now like to try using it in other things and seeing how it fits".

However, recently we added a simple rule for things that go in /Code :

must have asimpleworking example to accompany it:

should have some docstring documentation:

should declare a global tuple referencing all classes in the file:

__kamaelia_components__ = ( BasicPeer, SimplePeer, TargettedPeer, PostboxPeer, )

Note: Not all components in /Code currently have an example. This is due to the development process evolving over time, and becoming more concrete about what's useful. Also, depending on how useful we percieve your component to be, we may waive this requirement in favour of seeing what can be done.

If you do NOT feel you have time to hang around and wait for us (perhaps only a few hours), please see discussion of the CHAOS branch below.

Use of SVN Branches in Kamaelia

The bulk of day to day development with a view to release occurs on the SVN trunk. It is the natural working point that new developers - especially those new to version control - start from. We prefer to have a situation where you can just get on with development.

If you want to make experimental sweeping changes to the main /Code codebase you need to use a branch - either personal one, or the CHAOS branch.

RELEASEbranch

No edittingat allhappens on the release branch. This tracks the actual releases. It's function is to simplify the automated release build. Files are manually added to the release branch, but automatically mirrored and updated from trunk at release point.

Criterion (as of April 2006) for adding to the release branch:

Desirable when adding to the release branch:

Expected development regarding code on the RELEASE branch:

Adding something to the release branch should only be done with the knowledge of a project admin. (Hey, that's part ofthe idea of project admin after all. Whilst this may seem restrictive, it's simply to ensure a level of consistency, and in practice will often be a very short discussion.

Personal Branches

Much like having a personal namespace in /Sketches is useful, having the ability to change anything and everything in the entire code tree is extremely useful. It's also often by far the best way of demonstrating why your point is right - why your way is better than the current method or a proposed method. It's not always the best way, but sufficiently often to be worth supporting.

As was hinted above in "Version Control", if you do make a branch it should be across the whole code tree, and we have a naming mechanism to simplify things. Suppose once again your name is Warren W King. Suppose also you were the first person with those initials to start developing things in Kamaelia, so your subdirectory name in /Sketches is /Sketches/WWK/ . ie your personal "tag" is WWK.

For sake of argument, lets say you're doing integrations that require a variety of changes in the code tree in various locations, based on some changes in perspective the implementation of the QT module we described above. You might consider calling that INTEGRATION_BOOSTS . However this precludes someone else using the same name.

As a result the suggested approach is to name your personal branches as:

So in this case, the branch name would be:

Again, personally I'd suggested something more descriptive/precise like private_WWK_QT_INTEGRATION_BOOSTS, but any branch with a name starting private_YOURTAG is yours to do as you will. This allows others to see what's happening and share ideas, and see your ideas as they develop.

There is a special rule here though: you are NOT allowed to merge a branch you create (unless all project admins agree). The reason for this is this means someone else has to merge it. This automatically gains 2 things - peer review of code, and also it means that at least 2 people understand the changes embodied in the code. Where possible, pleaseinclude documentation since it's a pain to add later on!

If you don't know how to create branches, please just say it's relatively simple, but remember the rule - the branch must be across the entire tree.

Agent ofCHAOSbranch

If you feel the need to try out radical new ideas, AND also want to have shared development of them, please use the CHAOS branch. Co-ordinate amongst yourselves what you want to do and what the ground rules are. If you're making someone upset here, you're doing something wrong.

If the main CVS head ends up having too much control, the CHAOS branch is design to allow you to bypass that control and produce your own version that potentially has different goals from the project admin. If this idea sounds familiar, it's because it's the approach that governs the development of content on wikis, and we'd like to run the CHAOS branch in that way - you get to add things and work on things your way. If you break things, your code may get reverted by someone else, or changed by someone else.

In theory this sounds like complete anarchy and like it wouldn't work. Experience with another project has shown that if it's taken on board in the manner as described above (never ask permission on CHAOS) then you can get great creative drives and experience from trying out a variety of ideas quickly and effectively.

Note:This branch does not exist at time of writing, but this document allows for its existance. If you feel the need to use the CHAOS branch, please just create and use it.

Finally ... Some General Guiding Principles!

Descriptions of things optimised for the 80% of programmers are preferable to those with only the top 20% will understand. If this seems "dumbing down", consider that it is often harder to describe something complex simply, than to describe something simple in a complex fashion. Remember that a key aim is to make life easier for the bulk of programmers, with a recognition that its worth optimising for maintenance.

Fun descriptions, ideas and code tend to win arguments :-)

You're going to get it wrong. Whether you like it or not, something you do will go wrong and be very wrong. We'd like to aim for a very high hit rate of "good" decisions, but often the only way to find out what the "right" answer is to actually try something and find out. Please don't be afraid of pointing out the emperor has new clothes. We might be embarrassed (possibly even upset) for a while, but we'd rather be told and know when we've got something wrong. We're human too after all :-) For a good discourse on why it's worth taking this risk, read Paul Arden's excellent (and brief!) read "Whatever you think, think opposite".

Implementations change, but interfaces stay the same. If the interfaces change, you've written something new. ie you may add new inboxes and outboxes to components, but if you change behaviours of existing in-/out-boxes you should consider whether you're actually writing something new. If in doubt, talk to the team.

Quick, simple, clear, justified decisions (But not hasty).

All rules have exceptions.

import this :-)

Oh, and if it's not blatantly obvious, yes we'd like to allow you to check in. We have cvs_acls we could use to protect our code if we think you're acting badly, so we're not afraid to share our workspace, we'd love for you to find Kamaelia as useful as we do.