Note: I take a lot of what’s written here from these papers – be warned though, not everything written there would still be valid, they’re over a decade old. I’d also suggest you read Glyph’s The Twisted Way (he speaks Zen) or the krondo tutorial or Jessica’s book.
A couple of assumptions I make about you – that you are familiar with basic socket programming, and know a little bit about I/O multiplexing (select etc.)
In this first post, we’d see what a reactor is, and how it works. In the second part, we’ll talk about how you should and should not use it. The third part will talk about what’s wrong with the way Twisted’s reactor works now, and how it should be instead.
I wish I was only doing an imitation of a dumb user rather than really being one.
My first interaction with the reactor was when I was writing tests for an endpoint. I’d only heard about The Reactor – no one ever talked about how it worked, but everyone knew that it worked. And that it worked pretty well. To understand what a reactor was really doing, and what all the fuss was about, I looked for it hoping to unveil some magic. Alas, not much was found there; it again led to me looking through other some places, which, again, had little besides docstrings in a skeleton.
Since I did not find much in the twisted.internet.reactor module, I looked at the next best alternative, the MemoryReactor, which is a test double that one could use to test code that uses a reactor (e.g. TCP connection attempts). If you’ve been paying attention so far, you’d know that my goal was to test an endpoint. And a fake reactor was sufficiently serving my purpose. Also, MemoryReactor had enough code to tell me what it does when an endpoint is used to establish a connection. Case closed. Until one day, when I had to write tests for another endpoint, which required me to write my own fake reactor. One thing led to another and I ended up understanding the Twisted reactor architecture. And here’s an attempt to help you understand it too.