How do you keep documentation code examples up to date?

Every time there is a new product update, as documentarians, we need to retrace our steps fixing what has changed instead of spending our time covering new features. However, detecting which documentation needs to be edited after an update is not always as straightforward as testing the output of a deterministic function: someone should evaluate which concepts or guides have become outdated.

Still, there are some documentation areas where we can apply more conveniently testing, automation, and other common development practices—like code examples.

In this article, we are going to see how to make the pieces of code that we can find in almost every documentation for developers maintenance more enjoyable. Firstly, we hope to reduce the number of complaints received with the subject “The code examples are not working!”. Secondly, by spending some time architecting our docs, we will reduce significantly the time spent fixing outdated snippets.

The article is based on the Sphinx documentation system, but the same principles apply to any other documentation project with some custom development.

1. Splitting responsibilities

Amy, who works as a technical writer at BestTechnicalWritingConsulting, is documenting an SDK to order delivery pizza. The documentation page she is working on, written in restructuredText or Markdown, looks similar to the following example:

Example
=======

To order a pizza using the Python SDK, call the 
``order`` method passing as the parameters the 
pizzas you want, your customer id, and the delivery
service. 

The following examples show you how to order a 
Hawaiian pizza to take away.

.. code-block:: python

    api = pizza.Api()
    
    api.order(
        pizzas=[
            {'code':pizza.pizza_codes.HAWAIIAN, 
            'quantity':1}
        ], 
        customer_id='dgarcia360', 
        serviceType=pizza.service_types.TAKE_AWAY)

One might think that the problem with the previous code snippet is that it is a pineapple pizza… but adding pineapple on pizza 🍍🍕 is acceptable! There is a worse problem with the previous doc: the documentation page is mixing code an text in the same file.

Let’s review in-depth the mixed responsibility issue with an example. This week, the Pizza SDK development team launches a new release. Which are the steps Amy should follow to update the documentation?

1. Detect every code example affected.
2. Copy and paste every code example in a new file.
3. Execute the code.
4. Submit the changes.

Copying and pasting every code example (2) and testing them manually (3) are repetitive chores that can be automated. Additionally, following the approach mentioned above, Amy cannot ensure that the code tested in (3) is the same code displayed in the docs.

To start automating the process, Amy decides to move the piece of code into a separate file. For instance, she creates a new folder next to the documentation project to keep all the code examples organized.

The new folder will contain not only the code examples but also all the necessary dependencies to run the code. If the SDK were available in different languages, she could create a primary examples folder with subfolders for each language.

documentation-project/
├── index.rst
├── order.rst
├── examples/
│   ├── javascript
│   └── python
│       ├── order.py
│       ├── tests/
│       └── requirements.txt

What is Amy achieving by separating the code from the text? Now we can:

• execute the code examples and add tests.
• use the code linter, to ensure the code is well-formatted and every bracket closes.
• avoid copying and pasting code to verify that the code examples are working.

2. Don’t repeat yourself

Now that we have the text and code separated, BestTechnicalWritingConsulting wants to render both text and code together. Regardless of the documentation system you use, the idea is to import the code from the files using a directive that allows you to do so.

With the Sphinx documentation system, we could use the directive literalinclude as follows:

.. literalinclude:: examples/python/order.py
   :language: python

This directive renders the content of the file order.py inside our documentation page. With this addition, all our code will also be reusable, as we could reference the same piece of code in multiple documentation pages. Moreover, we are ensuring that similar examples will always behave in the same way.

Do you know any other similar directives for another documentation system? Share in the comments how you embedded code examples in your documentation projects!

3. Highlighting subsets of code

Now that we have testable and reusable code examples, we can go a step further. In some situations, we only need to highlight a subset of the code instead of the complete file. We also want to skip the non-relevant code lines such as the license header, or the list of imports.

To achieve this, we can separate the code using opening and closing tags’ comments. In our case, Amy decided to use the tags # block <number> to open and identify a code block and # endblock <number> to close it.

# block 01
api = pizza.Api()
# endblock 01

# block 02
api.order(
    pizzas=[
        {'code':pizza.pizza_codes.HAWAIIAN, 
        'quantity':1}
    ], 
    customer_id='dgarcia360', 
    serviceType=pizza.service_types.TAKE_AWAY)
# endblock 02

… and then, detect those tags with the help of the directive.

.. literalinclude:: /pizza/order.py
   :language: python
   :start-after: # block 02
   :end-before: # endblock 02

Avoid the trap of importing subsets of code by declaring the number of lines directly! If you define for example “render lines 1-10”, you would have to check that every code line stills correct after every update.

Putting all together

The development team launches a new version of the Pizza SDK. How can we check now that the code examples still working?

1. Update the latest SDK dependency into the code example folder.

2. Run the tests and check which failed.

3. Fix the code examples—get the green light! ✅

4. Push the code changes.

Following these practices, we have reduced the time spent maintaining our code examples in exchange to spend some time architecting a docs as code solution. But, it is not only about working less: the documentation will become more usable for your end-users since there are fewer chances to have code that does not work.

Do you want to automate your docs to increase your team’s productivity? Shoot me a message, and let’s figure out how to enhance your documentation process together.