Lessons Learned From The Stall of Drupal

Drupal is a web platform, for lack of a better term, that I previously used a lot. I was a paid professional, like so many others. Drupal enabled people to build semi-complex websites quickly. This has lead me to keep an eye on it over the years.

When I was reading the 2019 editon of the Stack Overflow survey I noticed that Drupal is now the least loved and most dreaded of the "web frameworks". Yikes!

Drupal sites can, also, report their version usage back to the Drupal project. The image below is a snapshot in time from the usage.

Drupal usage up to January 2020

Drupal 7, which was superseded by Drupal 8 more than 4 years ago, still has 2.5 times the number of sites reporting in. And, Drupal usage has hit a plateau. Double Yikes!

As someone who works on other open source projects these days (ones that are currently fairly popular), I wanted to take some time to see what had changed with Drupal over time. What lead so many people to dislike it.

The Massive Rewrite

Drupal 8 was a massive re-write. Some projects do this, from time to time, and this is one of those cases. The programming language didn't change and many of the terms, especially user facing ones were the same. The patterns used within the codebase changed massively.

This is important for a few reasons:

  1. Those who developed modules (Drupal extensions) had to learn a whole new way to do things. This was different from the past where the patterns were usually the same but there were new features, new APIs, and some API changes.
  2. Instead of upgrading modules for the new version of Drupal they needed to be re-written to fit within the new system.
  3. The way Drupal sites were styled was changed. "Themers" was a group of people who styled Drupal sites. Someone could learn the basics in a weekend and there are professionals who do this full time. This group had to learn a whole new way to style Drupal sites.

This is a lot of change for the groups that dealt with Drupal sites day in and day out.

On the flip side there were benefits. For example:

  • Drupal 8 follows modern PHP and software conventions. Drupal had long used many of its own design patterns. While some were still there, more typical computer science patterns are now used.
  • Instead of re-inventing the wheel with Drupal modules, people could more easily use existing PHP libraries. There are a lot of open source PHP libraries.

The re-write highlights more than the style of PHP code. It gets into who is writing the code. Is it those who are used to low-code setups or is it professional programmers? What was it for Drupal 7 and earlier compared to Drupal 8 and beyond?

Despite the hurdles, I was left wondering why people didn't transition to Drupal 8. In Badass: Making Users Awesome by Kathy Sierra there's a whole chapter on removing blocks. It comes from the idea that people are motivated, even a little bit. And, there are benefits to Drupal sites and Drupal professionals to switch. Kathy noted that,

Working on what stops people matters more than working on what entices them.

After working a little with Drupal 8 and talking with others who tried to make the leap I found that Kathy made a great general observation that applies to people making the Drupal 8 jump,

A gap between what they wanted and what's actually happening

People making the jump were having a hard time. For example, Drupal is known for doing a fair amount of magic. This is usually documented well. It's designed to help people. When I worked on a Drupal 8 module I found it difficult to figure out the new magic which wasn't documented. In my case I used a debugged to walk through Drupal to figure it out. Not something everyone is going to do.

Badass highlights two big derailers:

  • The Gap of Stuck - where users are motivated but get stuck learning the new thing
  • The Gap of Disconnect - where the focus before you pickup the thing is context but after you pick it up it's tool

Both of these are things, I think, affected Drupal. For example, people who try to learn Drupal 8 can easily get stuck as I did. I have talked to numerous people who fell into this bucket. It was difficult to learn. Or, there is the context of Drupal. Drupal was useful to quickly build websites. Even for those who wanted a low-code environment. This is the context. But, Drupal 8 was all about the tooling before the context of achieving the "quickly build websites" goal or people.

I don't mean to pick on the decision making here. Only in retrospect did I see some of these problems or understand them. The lessons on users that can be seen in Drupal are ones that can be applied to any project.

VC Money

In 2007 when Dries (the founder and BDFL of Drupal) graduated with his PhD he co-founded Acquia around Drupal. Acquia was a Venture Capital funded startup. This provided a shift that many of us didn't see coming at the time.

Prior to Acquia Drupal was lead by the needs of the people using it and contributing to it. This could have been small non-profits, hobbyists, and processionals building sites for the Fortune 500. Dries appeared to be focusing on the growth and usefulness of Drupal.

Once Acquia was founded the goals of venture capitalists were also in the mix. That included Drupal enabling Acquia to have a great return on its investment. To get a return on the investment, Drupal needed to be the kind of platform high paying customers would want to use and pay Acquia for some service.

What do those users want and how does it compare with the existing users of Drupal? For one, many of the developers at enterprise companies have computer science degrees. This is different from hobbyists who might right a small local web shop. The types of users are, in many ways, different.

Now, I don't mean to say that VC funding is bad. Just that it had an influence on direction. One that had a massive, even cultural, directional impact.

To counter this, we can take a look at projects run by a foundation. For example, we can take a look at Linux. Linus, the founder and BDFL, of Linux doesn't work for a single company with a single interest. Nor do the people who work closely with him. They work for a foundation and many companies - including large enterprises, bootstrapped small companies, and VC funded ones - belong to the foundation. No single company has an outsized influence which brings a certain amount of stability and knowledge of expectations to everyone.

This can lead to an interesting thought exercise and choice for our projects? What would Drupal have looked like if all the little companies had invested in a foundation where Dries was employed and he was looking out for all their interests? And, should your next big open source project be coupled to a single company or to some vendor-neutral home?

This is one of the reasons I appreciate the CNCF being around for the cloud native projects I use and why I'm more interested in those with a truly vendor-neutral home than one of a company whose whims can change.

For me, when I evaluate my use of projects and what I do with my own projects I'm going to keep these lessons in mind.