Publishing Python Documentation to Read The Docs With Jupyter Book
I published the documentation of Suite8080, a suite of Intel 8080 Assembly cross-development tools I’m writing in Python with Replit.
 |
| The Suite8080 documentation website built with Jupyter Book and Sphinx and hosted at Read The Docs. |
Like many Python projects, the documentation comprises a website hosted at Read The Docs. But, unlike most projects, I didn’t use Sphinx directly to generate the website. Instead, I created the documentation with Jupyter Book by The Executable Book Project, well known for Jupyter Notebook. Jupyter Book is a new system for producing publication-quality books and documents for software and research projects.
Jupyter Book is significant and promising because, by building on top of Sphinx as a backend and offering a Markdown-based formatting language, it hides the complexity of Sphinx and reduces friction when writing documentation.
Before this project I checked out Sphinx directly, but its arcane formatting language and complex setup didn’t make me go far. Sphinx is an amazing tool though, so I promised myself I’d try it again. Discovering Jupyter Book changed my plans.
The integration between Jupyter Book and Sphinx is at an early stage. Indeed, I had to overcome a few minor issues before making the systems work with my setup. Therefore, I’m posting these notes to document and share my experience. How meta.
Motivation and goals
As a hobby project, Suite8080 is a small and simple system. Still, I wanted to document it for the usual reasons this is a best practice, such as allowing others to use and understand the system.
But I had additional motivations.
First off, I’m one of those who needs to understand and use Suite8080. I’m working on and extending the system. But sometimes I put it aside and come back to it later, thus needing to re-immerse in its internals. The necessity to understand my own code is no joke.
The Python community has a tradition of producing good documentation, which is a key resource for helping me learn the language and leverage its ecosystem. I’m a text guy and love reading books and software documentation, to the point I often read manuals cover to cover (don’t judge).
Publishing the Suite8080 documentation is a way of giving back to the community and practicing technical writing, hoping to improve my ability to write manuals like the ones I read and love. Additionally, it’s an opportunity to practice the Python publishing toolchain.
Text formatting and publishing toolchain
Producing documentation with my setup relies on two text formatting tools, Jupyter Book and Sphinx, and the Read The Docs publishing platform. My development environment Replit runs and controls the other tools, letting them access the manuscript to process.
Although Jupyter Book, Sphinx, and Read The Docs can work together, there’s no single information source on how to do that. I had to experiment a bit to figure out the steps.
Let’s go over the tools and how to use them with my setup.
Jupyter Book
Jupyter Book provides a formatting language and tools for producing books and software documentation containing text, media, and executable content such as Jupyter notebooks. For text, Jupyter Book supports its own Markdown dialect, MyST, but accepts also reStructuredText files for Sphinx. This is no coincidence as Jupyter Book runs Shpinx as a backend.
The documentation Jupyter Book formats is ready for browsing on a local machine, or for publication as a static website to Read The Docs and other platforms. I love the clean and modern design of the sites Jupyter Book produces.
Jupyter Book is a Python system installable with pip. The Suite8080 REPL on Replit, the cloud workspace I develop Suite8080 with, allows a few ways of installing Jupyter Book.
The first is to run pip from the REPL’s Linux shell, which makes Jupyter Book available only for the current session. Running pip on Replit to install Jupyter Book issues a missing dependency error but a simple fix is to run pip a second time, when the error goes away.
A disadvantage of this option is the need to reinstall Jupyter Book at each new session. This is less of an annoyance than it seems as building or editing the documentation, and hence keeping Jupyter Book ready, is less frequent than working on the Python code.
There are two more ways: installing Jupyter Book with the Replit package manager and making the REPL always-on to leave the tool loaded in the workspace. Both have drawbacks, i.e. the lengthy download of Jupyter Book and its many dependencies (e.g. Sphinx) with the Replit package manager, and using up a precious always-on REPL slot with enabling always-on for a REPL.
I work on the Suite8080 documentation less frequently than on the Python code, so I go with pip-installing Jupyter Book on-demand.
Sphinx
Jupyter Book produces documentation from source files written in a variety of text formatting languages, including its own Markdown dialect MyST and reStructuredText for Sphinx, the well-known text formatting language and static site generator. Despite the ample choice, I use only MyST mostly because of the simplicity and minimalism of Markdown.
Under the hood, Jupyter Book translates the sources to the Sphinx format and runs the latter to build the documentation. So it’s possible to work almost entirely from Jupyter Book.
The build process still requires writing a few Sphinx directives or editing the Sphinx configuration file, for example, to produce API reference documentation with sphinx.ext.autodoc. I converted the Suite8080 Python docstrings to the Google style and configured sphinx.ext.autodoc to include the sphinx.ext.napoleon extension.
Read The Docs
Although I can build the Suite8080 documentation with Jupiter Book and browse it locally on my development system, the final, public result is a website anyone interested in Suite8080 may visit.
I upload that website to Read The Docs, a popular platform for publishing Python documentation formatted with Sphinx. The process is made possible by the integration between Jupyter Book and Read The Docs. To get started, I only had to configure Read The Docs via an online form to tell the platform where to find the Suite8080 GitHub repository.
Once configured, Read The Docs automatically discovers the Jupyter Book documentation sources, builds them, and publishes the resulting website. Then, each time I commit to the GitHub repository, Read The Docs detects any changes to the documentation sources and rebuilds them if necessary. All that is left to do is to check the build logs for any issues.
Replit
I write and document Suite8080 in the cloud with the Replit development and computing environment. Replit is the best fit for my Chrome OS fully cloud lifestyle.
Three Replit features help build and publish documentation with Jupyter Book.
The first is the Linux shell of the Python REPL where I develop Suite8080. Configuring and building the documentation involves running a couple of command line scripts.
The second feature is the version control client built into the REPL, which synchronizes with the Suite8080 GitHub project repository. Assuming Read The Docs is properly configured, as soon as I commit source changes in the REPL, Read The Docs automatically rebuilds the documentation and publishes it. When committing changes, the .gitignore file takes care of skipping the Sphinx build directories.
 |
| The Suite8080 documentation previewed in the project’s Replit workspace (REPL), with the Python http.server module serving the locally generated website. |
Finally, Replit lets me preview the formatted documentation in the REPL, both in the Markdown pane as I type and as HTML built by Sphinx.
Although Replit runs fully in the browser, it can’t render HTML files in a REPL unless an HTTP server is running in the REPL. In Python REPLs, there’s no need to install external servers as the native Python http.server module does a good job of displaying the static sites Sphinx generates. All I have to do is navigate to the directory with the HTML files and run http.server from the shell:
$ cd docs/_build/html
$ python3 -m http.server
This opens in the REPL an HTML pane showing the website. For convenience, I pop out the HTML pane into a full browser tab.
Structuring the documentation
In the early stages of the Suite8080 development, I began documenting the project in a series of Markdown files in the source tree: a README.md file as well as usage and design notes files usage.md and design.md. Anticipating I’d produce more files, I gathered the Markdown sources into a docs directory.
This turned out to be a good arrangement as the structure of the documentation was ready for Jupyter Book. A Jupyter Book book usually comprises a directory holding the manuscript files in Markdown and other formats, as well as some configuration files.
To adapt the Markdown files in the docs directory to Jupyter Book, I repurposed part of README.md as an introductory chapter, lightly edited the usage and design notes as separate chapters, created a chapter with the API reference processed by sphinx.ext.autodoc, and created the configuration files, _toc.yml describing the table of contents and _config.yml for the configuration.
There’s another important configuration file, docs/requirements.txt, needed to fix a Read The Docs build issue a missing dependency causes.
Read The Docs can automatically discover the directory holding the documentation sources to build from. Here is what the Suite8080 directory tree looks like in the Replit REPL from the shell’s home directory:
.
├── asm
├── docs
│ └── _build
│ └── html
│ ├── _images
│ ├── _panels_static
│ ├── _sources
│ └── _static
│ ├── css
│ ├── images
│ ├── js
│ ├── __pycache__
│ └── vendor
│ └── fontawesome
│ └── 5.13.0
│ ├── css
│ └── webfonts
├── suite8080
│ └── __pycache__
└── tests
└── __pycache__
docs is the documentation directory, the other top-level ones hold the Python and Assembly files. The root of the Sphinx build tree is in directory docs/_build. The source tree in the GitHub repository is the same, except for the missing documentation build and compilation cache directories.
Experimenting with a sandbox
Aside from some experimentation with Sphinx, I never used Jupyter Book, Sphinx, and Read The Docs before. Suite8080 is the right opportunity for checking out and learning the tools. Given the different tools and the ways they interact, I wanted to be sure not to mess with the Replit REPL and the GitHub repository that make up my Suite8080 development environment.
Therefore, I practiced publishing the documentation within a sandbox, a dummy project with a new REPL and a separate GitHub repository.
Forking the Suite8080 REPL wasn’t convenient as the forked REPL would have accessed the same GitHub repository, so I ended up copying the Suite8080 files I needed, the Markdown and Python sources as well as the directory structure.
Working in the sandbox gave me the confidence to learn without breaking my main development environment. Once confident I mastered the tools and process, I copied the changed files back to the main Suite8080 project and continued from there.
I could have created a new version control branch from the main REPL, but learning branching models will be a project for another time.
Workflow
The integration between Jupyter Book and Read The Docs is new and a work in progress.
After some experimentation, I figured out what steps to take and which ones aren’t documented. I put together a workflow comprising the initial Read The Docs configuration, and a series of steps to go through in the Suite8080 REPL each time I update the documentation sources.
Initial configuration
I kicked off the work with a couple of onetime actions to configure Read The Docs using only the browser.
First, with the documentation files ready in the source tree and committed to GitHub, I created a Read The Docs account. The platform supports signing up with a GitHub account, as I did. This helps a lot as Read The Docs needs access to the Suite8080 GitHub repository to build the documentation from source.
Next, I created a Read The Docs project for Suite8080. The defaults are fine, but there are two key fields to fill in the creation form, the URL of the GitHub repository and the Read The Docs project slug. I chose suite8080 as the slug, which is associated with the suite8080.readthedocs.io documentation website.
As soon as I entered these settings, Read The Docs cloned the GitHub repository and ran the first build, which failed with a missing dependency error. To satisfy the dependencies and fix the build error, I added to the source tree the file docs/requirements.txt with a reference to the Jupyter Book master branch:
https://github.com/executablebooks/jupyter-book/archive/refs/heads/master.zipTo tell Sphinx about the dependency I added another file, .readthedocs.yaml:
python:
install:
- requirements: docs/requirements.txt
formats:
- pdf
- epub
With the initial configuration complete, Read The Docs was ready to automatically rebuild the documentation and publish the website each time I would commit to the Suite8080 GitHub repository.
Rebuilding the documentation
Whenever I change the Markdown sources, I go through a few steps for rebuilding and publishing the documentation to Read The Docs. First, in the REPL shell I change to the project root directory ~/suite8080 and run Jupyter Book to generate the conf.py file the Sphinx build needs:
~/suite8080$ jupyter-book config sphinx docsNext comes the generation of the API reference. A current limitation of the toolchain is there’s no way of having Jupiter Book specify to Sphinx where sphinx.ext.autodoc can find the Python modules. A workaround is to edit docs/conf.py to add this code:
import os
import sys
sys.path.insert(0, os.path.abspath(".."))
The code adds to the Python path the parent directory of the docs directory, which is where the Python modules can be imported from from.
Having to manually edit conf.py for every documentation build is an error-prone kludge that interferes with automation. But at this time there’s no better sphinx.ext.autodoc integration between Jupyter Book and Sphinx. Ideally, Jupyter Book should allow configuring the module path via, say, _config.yml or directly take care of this.
Finally, everything is ready to run Sphinx to build the HTML site:
~/suite8080$ sphinx-build docs docs/_build/html -b htmlThis leaves the HTML files in the build directory docs/_build/html. Most of the times, I change to that directory and run the Python HTTP server in the REPL to check the resulting website for any issues:
~/suite8080$ cd docs docs/_build/html
~/.../_build/html$ python3 -m http.server
I pop out the small HTML preview pane from the REPL into a full-screen tab and browse the website on the desktop from Chrome OS, as well as from my Android devices.
When I’m satisfied everything looks good on the desktop and on mobile, I commit the changes to version control from the REPL. The commit triggers the rebuild of the website on Read The Docs, after which I do a final check there to make sure there are no issues in the build logs and the site still looks good.
The documentation is now ready for user consumption.
Publishing the Suite8080 documentation is another step in my journey to use Python with Replit in the cloud. I’m continuing to experiment with Replit, write new Python code, try libraries and tools, and share my experience in the Python with Replit blog series. Follow along to not miss a post.
