Reverse Chronicles

• Reading Confessions. Enjoying it.


• Bought a Macbook. Running Feisty. More info later.


• Missing iCal terribly.


• Was ill over the Eight Hours Day long weekend.


• Tasmania's bus system doesn't cater to weekend tourism.


• Just watched all of Season 1 of 24. I love how every second sentence ends with "now!".


• Looking to pay people to host mail for myself and Crossroads (several mailing lists and addresses).


• Finished every single level on Super Mario World except the final battle with Bowser.


• Everyone at Crossroads loves WarioWare Smooth Moves. Unfortunately, someone filmed me doing the dance routine from the Tiny Wario section.


• Not contributing much to Twisted at the moment.


• There is a testing-in-python mailing list. There is some interesting discussion going on there. I just hope that the people there who are writing things learn from xUnit, Trial, nose and bzr.tests and don't just make the assumption that not having to write a 'class FooTest(TestCase):' statement makes something cool.


• On testing, glyph has promised to finally tell me what he means about xUnit.

1. Lack of a keyboard-driven way to launch applications or switch between open windows by name. That is, lack of Quicksilver.


2. No convenient way to cycle between open windows of an application. The equivalent OS X operation is Cmd-`. I suspect this is why many linux apps have sprouted tabs.


3. New windows steal focus. I've talked about this a little on IRC. If you launch an application from the menu, then switch to IRC to type something witty, the new application's window might appear and steal your focus mid-witticism. Very irritating.

Even though Australians don't celebrate Thanksgiving, I thought I'd get into the spirit of the season by giving my thanks to the various free and open source software projects that I use every day.

Firefox

I know you guys get a lot of flack and you need to sort out your crazy, crazy licensing, but I still love you anyway. Thank you for DOM Inspector.

Firebug

Thank you. Thank you. Thank you.

Emacs

We've been together for years. Even though I've used other editors, you've always been there for me, patiently waiting and improving.

X-Chat Aqua

Even though I'm stranded in OS X, it still doesn't mean I can't pretend.

Adium X

I can't tell you how much I appreciate being able to have exactly one IM client. Merging contacts is such a good idea.

Trac

Definitely not perfect, but who is? When you are actually up and running, you are reasonably effective at tracking bugs, and I'm thankful for that. Wiki markup in commit messages and ticket comments is an excellent idea.

Python

Namespaces are one honking great idea -- let's do more of those!

Twisted

I've learnt so much from working on Twisted. Once you get past the weirdness and the brash exterior, the community has a heart of gold. I'm glad to be a member.

The rest

Those are the applications and libraries I actually, regularly use. However, most of them owe their existence and vigour to a host of libraries and tools. Thanks to gcc, libgaim, openssl, pyopenssl, gtk, the python standard library, expat, ssh, screen, vim and on and on and on.

The foundations are so vast, it is hard to imagine so many programmers giving so much time. Spending hundreds of hours chasing down bugs, reading specs, writing tests, composing arguments for mailing lists, packaging, implementing UI edge cases, triaging bugs and occasionally actually writing code that does stuff.

Thanks to the men and women of the FOSS community.

Thanks to idnar for this.

• Item is a subclass of Empowered, which has powerUp() and powerDown()
• You pass in another item (the "powerup") and an interface
• You can call item.powerupsFor(interface) and item.interfacesFor(powerup)
• Once you've done foo.powerUp(bar, IFoo), IFoo(foo) will give you bar
• If foo has many powerups implementing IFoo, IFoo(foo) gives you the first one returned (ordered by priority)
• However, you generally want either all IFoo powerups, or you don't want to have more than one IFoo powerup installed in the first place.

For more information, see the Axiom website.

Trial is about to become a fair bit stricter on tests that don't clean up after themselves. In particular, any test that doesn't properly close all of its connections is going to fail ugly. Lots of Twisted's tests fail very ugly.

If you are already glazing over with apathy towards unit tests, unglaze now. Disconnecting is a little more involved than you might expect, and there's every chance you aren't doing it correctly.

Thanks to moshez and teratorn for reviewing this document. Any errors are still my fault.

How to Disconnect in Twisted, Really

You are writing a test for some network code. You have already written many unit tests for the protocol and the protocol factories, and you assume that the lower level socket calls all work. You have deep confidence in your code, but you need more.

You yearn to write an end-to-end test. To set up the server, have it really listen, have the client really connect. You want data to zoom back and forth over the wire like luxury sportscars on the Autobahn.

Fine.

You start with a server and a client.

You want the client to connect to the server. You want things to happen. When things have happened, you want to assert stuff.

After the assertions, you want to disconnect the client and you want the server to go away.

Good.

I'm not going to help you set things up, I'm not going to help you make things happen. I am going to help you make all your connections just go away.

Let me lay down the law:

1. The server must stop listening.
2. The client connection must disconnect.
3. The server connection must disconnect.

You have to make each of these things happen. That's easy. Here's the tricky part: you have to wait for each of these things to happen.

    port = reactor.listenTCP(portNumber, factory)
    ...
    port.stopListening()

    protocol.transport.loseConnection()

OR

    connector = reactor.connectTCP(host, port, factory)
    ...
    connector.disconnect()

Now, just running these commands is not enough. Twisted is asynchronous: you have to wait for it to call you.

The way to wait for something in a Trial test is to return a Deferred from the test method.

stopListening is easy. It returns a Deferred if it is going to take a while. The others are harder. There is only one way to know when the connection has truly been lost: override connectionLost on a Protocol instance. Pass a Deferred to your Protocol and have connectionLost call callback on that Deferred. You will need to do this for both the client and the server protocols. Once you have your three Deferreds, return them in a DeferredList.

Simple.

UPDATE: exarkun tells me that there are two ways to know when the connection has truly been lost on the client side. ClientFactory.clientConnectionLost is the other one.

Summary

In order to clean up a test that connects a client to a server, you need to wait on three Deferreds: one for the listening port, one for the server protocol and one for the client protocol.

Below is an example demonstrating how to disconnect. As you can see, it is quite verbose. Trial does not provide any shortcuts, even though it should.

from twisted.internet import defer, protocol
from twisted.trial import unittest

class ServerProtocol(protocol.Protocol):
    def connectionLost(self, *a):
        self.factory.onConnectionLost.callback(self)

class ClientProtocol(protocol.Protocol):
    def connectionMade(self):
        self.factory.onConnectionMade.callback(self)

    def connectionLost(self, *a):
        self.factory.onConnectionLost.callback(self)


class TestDisconnect(unittest.TestCase):
    def setUp(self):
        self.serverDisconnected = defer.Deferred()
        self.serverPort = self._listenServer(self.serverDisconnected)
        connected = defer.Deferred()
        self.clientDisconnected = defer.Deferred()
        self.clientConnection = self._connectClient(connected,
                                                    self.clientDisconnected)
        return connected

    def _listenServer(self, d):
        from twisted.internet import reactor
        f = protocol.Factory()
        f.onConnectionLost = d
        f.protocol = ServerProtocol
        return reactor.listenTCP(0, f)

    def _connectClient(self, d1, d2):
        from twisted.internet import reactor
        factory = protocol.ClientFactory()
        factory.protocol = ClientProtocol
        factory.onConnectionMade = d1
        factory.onConnectionLost = d2
        return reactor.connectTCP('localhost',
                                  self.serverPort.getHost().port,
                                  factory)

    def tearDown(self):
        d = defer.maybeDeferred(self.serverPort.stopListening)
        self.clientConnection.disconnect()
        return defer.gatherResults([d,
                                    self.clientDisconnected,
                                    self.serverDisconnected])

    def test_disconnect(self):
        pass

1. Get Firefox
2. Get PuTTY
3. Shell into a unix machine, launch screen, launch irssi
4. Configure a Twisted development environment

• bzr
• Make Caps Lock an additional Control
• A decent terminal. (Maybe Terminator. Has anyone tried it?)

Update: I have already been asked to reboot my computer. Ahh, the memories.

1 Set up files and directories

• foo/__init__.py
• foo/test/__init__.py
• foo/test/livetest_foo.js
• foo/test/livetest_foo.py
• foo/static/js/foo.js
• nevow/plugins/foopackage.py

2 Set up namespaces

from twisted.python import util
from nevow import athena
import foo

def _f(*sib):
    return util.sibpath(foo.__file__, '/'.join(sib))

fooPkg = athena.JSPackage({
    'Foo': _f('static', 'js', 'foo.js'),
    'Foo.Tests': _f('test', 'livetest_foo.js')})

3 Write A Single, Failing Test

from nevow import loaders, tags
from nevow.livetrial import testcase

class OurFirstTest(testcase.TestCase):
    jsClass = u'Foo.Tests.OurFirstTest'
    docFactory = loaders.stan(tags.div(render=tags.directive('liveTest'))['OurFirstTest'])

// import Nevow.Athena.Test

Foo.Tests.OurFirstTest = Nevow.Athena.Test.TestCase.subclass('OurFirstTest');
Foo.Tests.OurFirstTest.methods(
    function run(self) {
        self.assertEquals('apple', 'orange');
    });

4 Write A Single, Failing Widget Test

from nevow import loaders, tags
from nevow.livetrial import testcase
from foo import widgets

class OurFirstTest(testcase.TestCase):
    jsClass = u'Foo.Tests.OurFirstTest'
    docFactory = loaders.stan(
        tags.div(render=tags.directive('liveTest'))[
            tags.invisible(render=tags.directive('barwidget'))])

    def render_barwidget(self, ctx, data):
        widget = widgets.Bar()
        widget.setFragmentParent(self)
        return widget

// import Foo
// import Nevow.Athena.Test

Foo.Tests.OurFirstTest = Nevow.Athena.Test.TestCase.subclass('First');
Foo.Tests.OurFirstTest.methods(
    function run(self) {
        var bar = self.childWidgets[0];
        self.assertEquals(bar.frotz(), 42);
    });

from nevow import loaders, tags, athena
 
 class Bar(athena.LiveFragment):
     jsClass = u'Foo.Bar'
     docFactory = loaders.stan(
         tags.div(render=tags.directive('liveFragment'))['Bar'])

// import Nevow.Athena

Foo.Bar = Nevow.Athena.Widget.subclass('Foo.Bar');
Foo.Bar.methods(
    function frotz(self) {
    });

• The code for Step 3
• The code for Step 4