From Perl to Django: how we developed an open source book platform
This is a first post in a series of articles about Booktype. Before I start getting all technical ("...and that is how you install a new Django app"), I would like to say a couple of words about the history of Booktype.
Booktype is an open source platform used to write, publish and print digital books. It was initially created to run the FLOSS Manuals platform and back then it was just called Booki. FLOSS Manuals is a community of people who love writing good quality documentation for FLOSS software. At first the FLOSS Manuals platform was built on the TWiki (Wiki in Perl) platform.
The early TWiki days
At the time, TWiki was a good choice to prototype this project. We extended its use with our own plugins and some external tools. Because FLOSS Manuals is a collaborative community we needed to modify TWiki to support this type of work.
We had an IRC-Web chat gateway called FM Transmission [post in Croatian], a live table of contents to see what other users were doing, a content remixer, localization plugins to manage book translation, a web-based tool for translating content and many more options that eased the collaborative creative process. It was a good platform but sometimes this system felt like a big hack. After observing some of the problems in the TWiki community [post in Croatian] we decided to write the whole thing again in a new platform - from scratch.
Extending the platform
In the second instance we had resolved that TWiki and Perl were not the best platforms to use. Mainly this was because many users were people who are interested in books and not Perl code.
We decided to use Python and Django as our new platform instead. They were both reasonably easy to master and allowed newbies to extend Booktype's applications. For instance if you want to have a blog on the front page, just install the existing Django blog app, change one line in the settings and put a couple of tags in the template. Anyone could do it!
The role of Objavi
When we talk about Booktype we are really talking about Booktype and Objavi. Booktype is used to edit and manage book content while Objavi is used to output book formats (as pdf, epub, mobi...). However, in this post I will only talk about Booktype in the first instance, as a book editor.
Booktype is really great for collaborative editing and not always the right tool for personal publication projects. It is about building a community around your book (although that does not mean you can not write your personal diary with it!). However, one of the central questions from the development standpoint is how the architecture of the software achieves this.
Django as the new framework
Like I said, we decided to use Django as a web framework mainly because we wanted people to plug Booktype into preexisting websites easily and extend its application to Django apps. I think we made a good choice to go this route. As a database backend we are using PostgreSQL. Sqlite3 is also an option here, but it really should be used for testing and development purposes. We are using Redis in-memory database to store all kinds of web client information exchanges on the Booktype server using Sputnik.
Sputnik is a Django application for two-way communication with web clients. When we started there was no other Django project that could help us so I decided to do what a developer would normally do. I wrote an abstract API (with channels, subscriptions etc...) for communication, and implemented a dummy network layer. This means when the right moment comes, the network layer is replaced with something better without changing the rest of the code.
There are many good frameworks these days that could be used, Socket.IO, for instance. But we use it mainly for the collaborative aspect of Booktype. When you have ten people working on one book at the same time you need a way to notify everyone about the participants actions.
For instance, we use it to make notifications about locking chapters, chatting, making changes to the table of contents, creating new chapters, attachment uploads and other functions. And yes, this means the table of contents changes as soon as someone adds a new chapter. This really improves the user experience and controls the ability overwrite someone's work.
But enough about the history of Booktype's development. In my next article I want to talk about the exciting world of new Booktype projects and how you can easily install and extend Booktype applications!
In the meantime, here are some Booktype links you can check out, or of course you can find me in the Booktype forum!