Q: Creating all objects in a DI Container up front seems expensive, what about performance?
Yes and no. Even if most DI containers do support lady loading in various forms, an instance injected into another object still requires to be created first. And sure, there’s a risk that objects get created even if it wasn’t strictly necessary. On the other hand, if you constructors so that object creation is as cheap as possible, this problem might not even be a problem. As a general rule, creating an object should have no for few side effects. In the Hello Testable sample code, an obvious enhancement would be to let the Lyrics object defer loading the text file until it’s strictly needed.
Q: In the original post you kind of describe Interfaces but you never actually use or even mention them. Why?
I agree, the relationship between the main plugin class and the Lyrics class can be made stronger and more formal by defining an Interface and requiring that the object passed into the main plugin class constructor implements that Interface. The only real reason I didn’t do that the original post already tried to stuff too many concepts into a single piece as it was.
Q: If I implement my plugin like you suggest, another developer wont be able to use remove_action to unhook my code. That’s a big problem
Agree, this is an issue. My solution to this is to provide a static method to the needed classes (just the main plugin class in the Hello Testable sample) so that an external piece of code can get a hold of the correct instance for this purpose. I will add a separate post about this when time permits.
I’ve had this post in my head for quite some time now, but I’ve never really found the time to write it. But then a few days ago, Tom McFarlin wrote about how we should avoid calling add_action and add_filter in a class constructor, something I obviously agree with. From some of the comments on Toms post, I realized that some concepts of OOP and writing testable code deserves some additional explanation. I tried to write a comment reply to cover some of the basics, but quickly realized that it’s better to actually get this post out the door.
Not that Tom didn’t do an excellent job with his part, just that I think that we could zoom out a little further so that we more pieces of the puzzle. To get that complete perspective, let’s introduce the star of this show…..
Introducing testable Hello Dolly
With the risk of choosing a too simple example, I’ve created an object oriented version of the classic Hello Dolly plugin, originally written by Matt Mullenweg.
My version is simply called Hello Testable and demonstrates a few of the things I tend to emphasize while writing high quality code for WordPress. The main objective is to show how WordPress plugin code can be made testable and more maintainable by following a few simple principles. Hopefully, I also successfully explain why “testable code” is not just something you fix towards the end of a project, it’s something you should have in the back of your head all along the way.
All the code for this post is available at GitHub. Most interesting parts of the code will be repeated in this post, but for reference and completion you might still want to have a look.
Excuses in advance
Before working with WordPress, I’ve spent a lot of time working with other PHP projects. As a result, I tend to write PRS2 style code rather than follow the WordPress code style guidelines. I’m terribly sorry about any confusion this may cause. Some day I will get in line with the rest of you.
Before going into details and trying to make the case for my approach, let’s look at what happens when the Hello Testable plugin runs.
In the first part, the plugin bootstrap file gets included (1) by WordPress core as you’d normally expect. But instead of directly creating our main plugin class (this is an OOP version, right?) an instance of a Dependency Injection container is created (2). Once we have a container, the bootstrap file asks the container to give it an instance of the helloTestable class (3).
But there is a little caveat. The main plugin class constructor expects that an instance of a Lyrics object is passed in as a parameter. The container object knows this, so it goes about and creates a Lyrics object first and then uses it to create the main plugin class instance (4). This knowledge about how to create other objects is represented in the RuntimeProvider class.
Once the bootstrap function get’s the main plugin class instance back (5), it tells WordPress core that the init function of the class should be hooked up with the WordPress ‘init’ action (6). A little later, when all other plugins had their chance to load, WordPress core will come back and actually call the init function on the main plugin class (7). The init method will in turn hook up two other functions, echoLyric and echoCss to be called by WordPress in response to the ‘admin_notices’ and ‘admin_css’ events, this is not shown in the image above.
And even later when WordPress is ready to render the page, core is going to come back and actually fire the ‘admin_notices’ and ‘admin_css’ events. This is what that parts looks like:
As you can see, at this phase the bootstrap file and the container class has nothing to do with the flow. This is just the normal WordPress flow as you’d expect. A piece of css is echo’ed out and a little bit later, a piece of html follows.
The original Hello Dolly plugin uses the simplest imaginable plugin design. A single PHP file that does everything. My plugin follows a rather different approach:
The point of introducing the dependency injection container is that we get a distinction between creating an object and using an object. For anyone asking why, the easy explanation is that this is what you get if you want your classes to have zero dependencies, for a more in depth answer: read the rest of this post.
In this design, all dependencies are concentrated to the bootstrap module (a simple php file) and the Runtime provider class. The Main plugin class and the Lyrics class doesn’t have any dependencies on any other named class. And by concentrating the amount of dependencies to a single place (ok, two places) we end up with a class design that lends itself well to unit testing. More on that later.
Some formal background
So I’m suggesting an overly complex approach to solving a problem that Matt Mullenweg solved in roughly 80 lines of code years ago. Why? Well, obviously I’m doing this to show you something important and to begin explaining what that is, I first need to mention three (and a half) quite important topics in software design, namely:
Reducing tight coupling and dependencies
When we write code in a module or class that has strong dependencies on other classes and modules, we end up with tightly coupled code, or code with a lot of internal or external dependencies. The way we have written software in the WordPress community has over the years led to a lot of very tightly coupled code.
The problem with tightly coupled code is that we make a simple change in one place, we often get ripple effects that ends up affecting code in lots of different places. It also makes code reuse a lot more difficult. When the code in one class makes explicit assumptions about the mere existence of another class, our first class has a dependency of that other class. As you can imagine, this is quite common in WordPress, we have lots of dependencies everywhere!
As a rule of thumb, anytime you use the new keyword or call a static member of another class, your current class has a dependency on that other class and it’s worth examining if it’s avoidable. The same thing is also true if you ever call a non internal php function in the global namespace, that’s also a kind of dependency. Avoiding global functions is obviously going to be tricky in a WordPress environment and as we shall see later, globals do make our job a bit harder once we get to writing tests. In the WordPress world, we’re going to have to live with global functions for a long time, but there’s no need for us to add more global functions into our own plugins and themes.
Dependency inversion principle and dependency injection
Dependency inversion principle offers a solution to the problem of tight coupling. At the practical level, it simply means that instead of letting code in a class explicitly refer to other classes and create new objects that it needs, we should get those objects passed in to it before they are needed. A class could accept the needed objects via its constructor or via getters and setters, the how is not as important as the why.
As most people figure out, if we pass in all objects to our class via some mechanism, we still need to instantiate those objects somewhere. In a simple application, we could create all needed objects as part of a simple bootstrap function. When the class structure gets a little bit more complex, we could use a dependency injection container as a practical way to manage the object creation in one single place. By adding a dependency injection container to our solution we get an excellent tool for avoiding tightly coupled code.
Separation of concerns
The third principle is about separation of concerns. We should strive for code that keeps each module and class to be as lean as possible. This has many advantages. First, we avoid ending up with a ‘god class’ or ‘god object’, a large do-it-all piece of code that keeps track of everyone and everything. A god class (or worse, function) is a known anti pattern and deals with application control flow, it reads and write files, it encapsulates configuration and usually a 100 or so other things. God classes are often a mess to maintain and debug or even understand just a days after implementation. It’s better to divide the code into smaller, single purpose classes where each class has a distinct responsibility.
The second advantage with separation of concerns is that it promotes reusable code. Chances are that once we’ve put everything that has to do with i.e. reading and writing settings into a separate class, it’s a whole lot easier to reuse that code in other classes.
Avoid side effects
The last topic is picked up from the PHP Framework Interop Group standard PSR1. Section 2.3. It states that loading a module should either declare things (classes, constants, functions etc.) or it should execute logic. It should never do both. I’m sure there’s a non-PHP specific computer science name for this principle, but I’m going to be honest and tell you that my main reason for thinking this is important is because it makes my life as a PHP coder easier.
Hello Testable implementation
Having these concepts in mind, let’s look at the implementation of Hello Testable and how hopefully some of the design decisions I’ve made starts to make sense. First, let’s look at the bootstrap function in hello-testable.php:
The role of the bootstrap function here is to kick of the process of setting up our plugin. It’s called as a side effect when WordPress includes our plugin file.
First we’re trying to exit early from ajax requests since we’re not dealing with ajax.
Next, we’re checking that we are on a new enough version of PHP, this deserves a comment. To make everything work as we need, we’re going to use auto loading and namespaces through the magic of Composer and 5.3.9 is the earliest version Composer will run on. While it’s theoretically possible to get our job done without it and make the code PHP 5.2 compliant, I happen to think that it’s reasonable to skip 5.2 support. Someone else has already built a well supported and fully functioning package manager (Composer) with auto loading and namespace support that the rest of the PHP community adopted years ago. It makes sense to use it even if it’s at the expense of those poor souls who are still maintaining WordPress on PHP 5.2.
I’m sure all 5.2 users are happy that WordPress still runs on their 11 year old web servers that hasn’t been supported for the past 6 years, but I think it would be unreasonable for them to expect that all new code written for WordPress will.
Line 21 is where the fun stuff starts, we’re creating an instance of a Container and on the next row we initialize the container using our own RuntimeProvider class.
Some of you might notice that I’m creating an instance of Pimple (a well known 3rd party dependency injection container) but it appears that Pimple is part of our own, plugin specific, namespace. Isn’t that weird since we’re using Composer and all? Well, yes and yes. I have made a copy of Pimple and made it part of our own code base. And yes, in an ideal world this is exactly the kind of thing that Composer could help us avoid. The problem here is WordPress.
As described in this post, it’s not really safe to include 3rd party libraries via Composer into a plugin or theme for several reasons. In order to safely take full advantage of Composers package management, WordPress would need to have a centralized vendor folder where all 3rd party libraries reside. As things stands right now, another plugin in my WordPress installation could already be using an incompatible version of Pimple and if that’s the case, my plugin would break. So for now, we should be very careful with including 3rd party libraries via Composer into WordPress themes and plugins, especially if we intend to distribute them. Making a local copy of Pimple and renaming it’s namespace is a pragmatic solution to this problem and Pimple has a fairly small implementation so it’s not that expensive anyway.
Next, our bootstrap function asks the dependency injection container for an instance of the main plugin class and then lastly hook the init function up to the WordPress init event. Note that we’re explicit here, we first create the instance of the main plugin class, then we’re explicitly calling add_action. There’s a reason and we’ll get back to it when we look at unit tests for this class.
Also note that this module actually violates the side effects principle, it obviously defines a constant as well as two functions and it certainly executes some logic. My honest opinion is that it’s OK to violate that principle (and a few others) in the bootstrap module, but that’s about the only place it’s OK. I’m sure a lot of pragmatic people will agree, if you don’t agree, please let me know in the comments below.
RuntimeProvider.php – initialize the container
The role of the RuntimeProvider class is simply to initialize the Pimple container:
This is standard Pimple setup where we tell the container how to provide objects of the different classes we’re going to use. When the bootstrap class initializes the Pimple container, the code in the RuntimeProvider::Register is executed. Pimple promotes lazy loading, so it’s not until anyone actually wants a Lyrics object that the closure function will be executed and a Lyrics object is created. In our little sample, we have little use of lazy loading since all objects will be created at plugin load time. But it’s worth mentioning that this approach doesn’t always mean that all objects are always created at plugin load time since it’s a common misconception. It will simply depend a little bit on your code at large.
The Lyrics class constructor takes a file name as parameter and the main plugin class, helloTestable, wants a Lyrics object as parameter to the class constructor. Pimple will resolve the dependency between helloTestable and Lyrics so that when we ask for helloTestable, a Lyrics object is created on the fly.
Lyrics.php – Fetching lyrics
Consistent with the principle of separate concerns, the main plugin class doesn’t concern itself with reading lyrics, we’ve put all the lyrics related code into a separate class. Also note that the Lyrics class doesn’t even know anything about the song Hello Dolly, it just happily accepts the path to an external file to parse.
(note, this has some security implications since anyone with write access to the server could replace the content of the file with something dangerous, it can be mitigated but that’s outside the scope of this post)
If you have seen the source of the original Hello Dolly song, this should look somewhat familiar.
helloTestable.php – The main plugin class
The last class we’re going to look at is the main plugin class. We’ll do this function by function, first, the constructor:
Real life dependency injection!
This is a typical constructor in a class that uses dependency injection, the only real thing that happens is that we store the injected dependency into a private member variable. Apart from that, nothing and that’s exactly the intention. We don’t want the class constructor to fiddle with anything except it’s own internal state, creating an object should have no side effects on state anywhere else in our application.
Next, we know from looking at the bootstrap code earlier that after creation, the bootstrap function will hook up the WordPress init event to the init method of this class, this is where the class gets to do things that affects the state of anything external:
This should be very familiar WordPress code. Note that we hook up the events to instance functions rather than static functions that you also often see in WordPress code.
The two last functions should be fairly straight forward:
Both these functions are almost but not quite completely uninteresting. You want to note that the echoLyric function is where we use the Lyrics object that was passed in via the constructor earlier. It’s actually the only place in the main plugin class where we make any assumptions at all regarding the Lyrics object, the assumption that there is a function named ‘getLyric’.
This is a fine example of separation of concerns, the main class needs to know absolutely nothing about how lyric strings are created. In a later version of this plugin, we might want to change the implementation of Lyrics so that it reads 10 files with 10 songs and displays half of them backwards. Those changes would only affect the Lyrics class, the main plugin class would remain untouched. The other benefit from following the principle of separate concerns is that the Lyrics class is easy reusable. At some point we might want to get a lyrics string as part of an ajax call to WordPress, and if so, the Lyrics class is easy to reuse.
It’s also a fine example of minimizing dependencies. Back in the RuntimeProvider class, we could create any kind of PHP object we like and pass it as a dependency to the main plugin class, as long as that class has an function called getLyric(), our code will work just fine. We could implement a new class called HttpLyrics that fetches the actual strings from a http server somewhere, or a class called DBLyrics that uses a database table to get it’s lyric strings. The main plugin class will never know the difference.
Code complete – let’s test
With that, we’ve finished talking about the implementation and had a look at all the interesting pieces of code from an implementation perspective. We’ve already seen that while real life usage of some of these principles gives some additional overhead, they start to pay off quite quickly in that our code is nicely modularized and easy to extend.
When we now turn to writing unit tests, we’re going to see even more advantages pan out in real life. I fully understand that even if some of the design decisions I’ve made starts to make sense, I think you’ll see that some of them are more or less mandatory as we strive for fully testable PHP code.
First, a word about unit testing.
Unit testing means that we should test our code in the smallest possible executable units. In PHP the smallest executable unit is typically a function so what we want is to test each function individually. The challenge is that most function often calls other functions, sometimes in other classes and sometimes functions in the global scope. It can quickly become quite messy to isolate each function so that test only concerns that specific function.
This is where the idea of avoiding dependencies in classes, having code modules that loads without side effects and writing code according to the principle of separate concerns should start to sound a little bit more appealing.
We know from experience that WordPress uses a lot of functions in the global scope so it doesn’t really matter how good our own OOP design is, if we’re dealing with WordPress we’re going to have to deal with global functions. Also, some of those functions will interact with the database or make assumptions that other parts of WordPress is initialized. It becomes messy real quick.
In unit testing, the solution to this kind of mess is to work with mock objects and functions. If our code calls a global function like add_action, we can mock that function instead of trying to load the relevant parts of WordPress. Mock function and objects can if used correctly, enable us to fully test our WordPress plugin without ever having to load WordPress. Besides that good feeling you get when following some computer science dogmatic principle that also means that your unit tests can be executed super fast. And lets face it, if the unit test suite takes 2 seconds to complete, youre going to run them a whole lot more often than if it takes 10 minutes.
There are lots of mocking libraries for PHP and PHPUnit but the gold standard of mocking WordPress code is a framework from 10up called (surprise!!) WP_Mock. WP_Mock can be a little bit tricky to get started with so I hope that this post also serves to give a few pointers on how that can be done. But this is not a WP_Mock tutorial so let me know in the comments if you think that’s something you’d like to see as a separate post later.
Let’s dive in and look at testing.
First test class – LyricsTest.php
This test class is for testing the Lyrics object. It really only have one interesting method, the constructor will get test coverage as a side effect from running this test:
We’re creating an instance of our Lyrics class. But instead of giving it the standard lyrics for Hello Dolly, we’re passing in a file from the fixtures sub folder. This file just contains 3 rows, all beginning with the word line and a space. This makes it easier for us to write predicable assertions for this function.
Here we also need to use WP_Mock to emulate the WordPress function wptexturize. Remember that our unit tests should run isolated, so including the WordPress implementation of this function is not an option (if we did, we’d be writing an integration test). Our mock implementation of wptexturize just returns the same string but with a space and the word wptexturize added. This makes it easier to verify that our code ran as we expected, a neat little testing trick.
This is what a unit test for WordPress plugin code will often look like.
We create our object as normal, we only need to pass in our known dependencies.
We mock a couple of WordPress functions from the global namespace and then run our function.
Then to round up, we verify that we got the expected result by running a couple of assertions.
Next up – helloTestableTest.php
When testing the main plugin class, we have to use a few slightly different approaches. First of all, the main plugin class init function deals with WordPress internals, it doesn’t really return any data that we can test with assertions. But we can test that the expected add_action functions are called. Here, we talk about setting up expectations for our code and then WP_Mock will verify that our expectations are met:
For this particular test, we don’t even need to bother with passing in a functional Lyrics object since we know that the init function won’t use it and the constructor will only store it, so we can pass in a dummy. Next, we tell WP_Mock that we expect to get two calls to add_action with different parameters. When we run the init method from our test case, WP_Mock will thrown an error if the expectations weren’t met. Since we don’t have any return data from add_action, that’s about the only quality assurance we can do on this code.
Now stop for a minute and ponder the original statement made in Tom McFarlins post about calling add_action in the class constructor. If our main plugin class had violated that principle, we couldn’t have written this test.
In order to set up the expectations for WP_Mock, we’d first need to have an instance of the class. But creating an instance of the class would have already have called the add_action function and we’d miss our opportunity to check for it. So if you’ve read this whole post just to find an additional argument for Tom’s statement, this is it!
Next step is to test the echoLyric function. When doing this, we’re again getting an advantage of the fact that our code so far has very few dependencies. All that function cares about is that it can call a function called getLyric() on a private member variable. We could pass in our already quite easily testable Lyric class and just feed it a predictable text file, but that wouldn’t be testing in isolation so we want to avoid it. Another option would be to use WP_Mock och PHPUnits support for mock objects, they both have good support. But I’m going to show you another testing trick that I use a lot, creating your own mock classes:
First the mock class (located in tests/MockObjects.php):
Then the actual test case code:
Two things to notice. With our mock test class, it’s easy enough for it to just return whatever we initialize it with, it just needs to implement the expected getLyric() function. Creating simple mock objects like this is often very helpful.
Second, the echoLyric function in our main plugin class echoes out directly to standard out so we don’t have any return value to test against. Instead we’re using PHPUnits expectOutputRegex function to ensure that the outputted text contains the string that we initialized the mock object with.
Let’s stop again and think about what this would look like if we had implemented the Lyrics object with static functions and we wanted to test echoLyric(). In that case, the class name ‘Lyrics’ would already be hard coded into the code of helloTestable.php and we’d have a hard dependency to work with. It wouldn’t have been possible to mock the Lyrics object in any sensible way and we would have to accept a test that wasn’t as isolated. We’d be stuck with our test case indirectly using the standard implementation of the Lyrics class and we’d also be stuck with any dependencies that in turn would have. For this simple plugin it wouldn’t be a big deal, but imagine if the Lyrics class fetched lyrics data over http from a very slow or unreliable host. Our test would then depend on that unreliable host.
The last function in the main class is the echoCss function. This function uses the WordPress global function is_rtl() that we need to mock. For good measure, we test both possible return values from this function. This is easy to do in WP_Mock by defining a ‘return_in_order’ element that will give different return values for the first and second time it’s called:
Last but not least – RuntimeProviderTest.php
The runtime provider only have one function and we’re just going to write one test case for it and it’s really straight forward:
Asserting that we get the right kind of object when asking the container is really the only thing that makes sense to test.
With the above tests in place, we should be able to run phpunit from the command line and end up with a coverage report like this:
Some 4500 words after I decided to add a few comments and insights to a post by Tom McFarlin, here we are. The point originally made was that we should avoid calling add_action and add_filter in class constructors which is still very true. If you made it this far into the post, you hopefully also agree that it a good idea.
Hopefully you also agree that it’s a good idea to consider what happens when follow a handful rather simple and obvious rules of how to write code. Rules by the way that have been known in computer science community for decades by now. Follow them and you’ll end up with testable and maintainable code. And as always, once you know the rules and how they help you write better code, you may break them as much as you want because you understand the consequences.
While writing this I think I’ve left out more examples than I left in, there are tons of other points to be made, samples to be shown and arguments to be had. Please let me know what you got out of this. Questions or opinions alike, I’d love to hear from you in the comments section.