Moving from Drupal to Django Wagtail

By andre

Close to 15 years I have been using Drupal for my personal web sites. Drupal is a flexible and configurable CMS (Content Management System) web framework, similar to Wordpress. It has been a relatively dependable working horse for my hosting needs - simple and not too flexible, but relatively robust and reliable. It was quite straightforward to bring up a new web site via it:

  1. Create a new MySQL database
  2. Create a config directory for the site
  3. Update the settings file
  4. Go through the setup wizard
  5. Configure more stuff in the admin interface
  6. Voila!

Unfortunately, there are a number of drawbacks to Drupal. I was sort of ignoring those for years, mainly because it “just kinda worked” for most of what I wanted it to do. I didn’t require any advanced configuration that would necessitate code changes, and the rest was relatively simple via the admin interface. However, in the recent year or two I got frustrated with the shortcomings of the framework on one hand, and on the other I decided I did want some more advanced customisation and control over the web sites.

I was considering which route to take in order to reimplement my web sites. I was open to almost anything in theory. I was happy to try and learn a new technology - and there are so many cool frameworks to play with nowadays! One could even say there were too many :)

For example, I considered Rust, arguing that the rigid type system and borrow checker would make my code so my safer as opposed to almost anything else. Unfortunately, I reached the conclusion that Rust and available web frameworks for the language were not mature enough for my goals. Of course, I could have gone ahead and used it to program the sites, but it would have required A LOT of effort, more than I was prepared to invest.

Of course, I rejected outright any node.js (or anything JavaScript) based solution. People behind node.js are single-handedly responsible for the global warming, since, without any doubt, the increase in electricity consumption in the data centres of the world is purely their fault. But their “technology” does serve as an example of how not to build software, and indirectly helps my job security. After all, someone needs to save all those failed node.js projects from collapse!

I looked at other options. Various .NET Core web frameworks, especially F#-based ones looked quite intriguing. Giraffe / SAFE are very promising in my opinion. On the JVM side, my long term favourite still is Play! However, with all these and some other frameworks there was the major drawback - they too had a smaller number of third party modules, as opposed to the one platform I already knew and used quite a bit - Django. I mentioned in another post how Django was a viable option for web development, and I believe it still being the case. Even more so when the target product is not very large and the available time is relatively short, which is most certainly the situation with my web sites.

And so I decided to ditch Drupal in favour of Django. In a bit I’ll explain the migration process, but first I would like to put a few last nails into the coffin of the old framework by listing some of its drawback that irked me the most in order to emphasise the need for the migration.

PHP hammer

What is wrong with Drupal

Drupal is written in PHP

PHP has been around for more than 20 years. It was one of the first web-oriented technologies, simple to deploy and to use, and as such, it gained a lot of popularity. Even Facebook, as well as plenty of other, large and small websites, are based on it. Unfortunately, it is also a deeply flawed technology. I won’t go into details here, but if you are not familiar with those yourself, feel free to google “php sucks” and treat yourself to some entertaining reading.

Granted, as long as I didn’t need to customise my sites beyond what was available via the admin interface, I couldn’t care less. That was the reason I could happily use Drupal, despite PHP, for a decade and a half. After all, to grow sweet apples one often has to use manure. However, once I decided I wanted such flexibility, which only code writing could provide, I simply couldn’t bring myself to do it inside Drupal. Granted, the Facebook employees code away in PHP for years. But these poor sods are compensated handsomely for their intellectual suffering with high salaries and very decent bonuses. Also, they have no choice - the codebase in Facebook is staggeringly large! I, on one hand, am doing this stuff in my free time, and on the other hand don’t have that much legacy, so can afford to start from “scratch”.

Drupal 7 to 8 migration

A good thing about Drupal is the fact that it is under active development. I think I started using it when version 5 was the newest available. Soon I switched to 6 and, a few years later to 7. While each time there was some pain in upgrading, I managed to sort out the issues relatively quickly. However, the move from version 7 to 8 has proven to be much more difficult. I don’t know exactly what changed between the two versions, but despite my repeated efforts, I couldn’t do the upgrade in a clean and reliable fashion. I’m sure, that someone more experienced with Drupal could have solved the issues I was facing. I didn’t want to invest the effort into debugging the problems and simply left my old sites on version 7. This is OK for now, but not sustainable in the long term, since they will stop supporting that version at some point in the future (November 28, 2022 to be exact).

Drupal management applications

Originally there was no special tool to manage a Drupal installation. Then drush was introduced. It can take care of managing modules and any number of other administrative tasks. There is also something called composer, which, if I understand correctly, is a more PHP-generic tool for package management. Overall, for an irregular user this is a bit confusing. Granted, for a full time Drupal admin/developer all this would be trivial, but for me these tools seem to complicate life.

Customisation

As mentioned, customisation is done either via the admin web GUI, o through PHP code changes. There is one peculiar feature of PHP I didn’t mention explicitly, which makes customisation more difficult. The language was originally designed to be freely mixed with HTML. At the time it was considered a good idea, and a few other technologies followed suit, such as JSP and ASP. However, over time people came to their senses and realised, that mixing code and HTML wasn’t such a brilliant idea after all, since it made customising look and feel more difficult. As a result, even such a small thing as modifying Drupal templates or layouts might necessitate some PHP coding:

<html>
 <head>
  <title>PHP Test</title>
 </head>
 <body>
 <?php echo '<p>Hello World</p>'; ?> 
 </body>
</html>

Deployment and maintenance

This has actually nothing to do with Drupal itself, but my old site management sucks. I deployed everything manually by hand, so there are any number of config changes in various places, which I did over the years and most of which I, naturally, only vaguely remember. This is a recipe for a disaster in the long run. Deployment of Drupal can be automated as well as anything else, but if I’m to invest the effort, I might as well do it with the framework I like.

Moving to Django and Wagtail

And thus I made the decision to move to Django. It is a robust framework with multiple third party modules supplying additional functionality. Among those is Wagtail - an CMS module. Since most of my web sites are mainly collections of articles or blog posts, this is exactly what I needed. Naturally, I used other modules as was necessary in order to build the functionality I wanted. Overall, most of the heavy lifting is done by Wagtail and some additional external code. The bulk of what I did was to customise the representation the way I need. I also made some changes to the logical architecture of the setup, so long term maintenance will be so much easier for me.

Data migration

Obviously, I didn’t want to lose any of the data I already had in my old web sites. I extracted it with the help of my Python Drupal download module, which is available as open source. I then wrote a bit of extra code that imported the data into Wagtail. Some points addressed for this job:

  • I made sure the internal links between old pages were preserved
  • I converted code samples to give them a fresher typesetting
  • Comments were also brought over, since some of them were quite helpful
  • The whole process was scriptable and included validation

DevOps - deployment and maintenance

I didn’t want to repeat the mistakes of the past. Today I have a script, which can deploy all the web sites on a blank AWS (or any other) server. Obviously, the code for everything is in a VCS and it is possible to roll deployments back if necessary.

Conclusion

Drupal has been with me for a long time, but I’m not sad to see it gone. In Django / Wagtail pair I have a robust and long term solution which will serve me well for the years to come. I will be able to add enhancements if I want to, or to leave it alone with minimum level of maintenance if I so choose. In the end, freedom of choice is the important thing, both in life and in software development.