Last week, the latest version of Cinder (v0.9.0) was released. This release is kind of a big deal. At least it is for the creative coding C++ developer community. Over the last year and a half or so, Cinder has been going through a rewrite to enable the use of Modern OpenGL and to pave the way for for supporting future platforms ports, like Android and Linux. This was a massive undertaking and it’s been amazing to witness the steps taken by the Cinder development team in real time. Other features this update includes are the inclusion of GLM for all math functions, the Logging API, EXR support and much more. You can read the release notes here. This release also came with a new Cinder website and a new docs website.
The new documentation system is a chunk of this update that I had the privilege to be involved in. Historically, Cinder’s documentation hasn’t been the focus. It was always pretty barebones and included an atlas of the source code and a bunch of great guides that never really got the focus that they deserved. The old system used Doxygen, a powerful documentation generation tool, to parse the source and piece the documentation together with their antiquated html system. To customize the output was very tedious and not very flexible. I had tried in the past to help upgrade the docs using just what Doxygen provided, but quickly ran into dead ends. It was apparent that we needed a better system.
Instead of using Doxygen for the actual HTML output, we decided that we would use it for what it was good for, which is combing through source code and organizing it in a way for us humans to understand. Instead of organizing everything as HTML markup, we opted for XML file output along a master cinder.tag file, which is an XML formatted representation of all of the entities found in the source code. This set of XML files provides everything you need to generate all of the documentation that you would ever need. The changes we needed to make for the desired output were available as Doxygen settings.
The Python Script
In order to piece together documentation on your own, you need to parse all of that XML data, pick out what you want to show, mark it up and save it out. Enter Python. Previously, my experience with Python went as far as messing around with Flask apps that other people had built. With that taste of Python, I was eager to pick it up and run with it. Included in the Cinder docs directory is the result of my first real python project.
The larger steps that the docs generation script takes are the following:
- Pulls in the cinder.tag file to determine the namespaces and classes that exist as part of the Cinder source and create a map of sorts to be referenced later
- Parses every xml file, one for each namespace, class, struct and interface
- Parses every HTML source file, which includes the index page, reference pages, and guides
- Organizes the data we deem relevant and organizes it into a Python dictionary object for each page
- Loads a template for each page and passes in the dictionary object, which determines the HTML page content
- Save the HTML file
In addition to the source code documentation generation, the Python script also allows for external supplemental documentation. What this means is that now there’s a way to write guides in HTML and have it deeply integrated into the docs. The guides use the same master HTML template as the source code page markup, and we have a system to link to specific source code entities as well as a way to link from class pages directly to the guides. This is done using what we call <ci> tags. Basically, <ci> tags are markup that the python script is aware of that triggers the linkage of different special functionality within the docs system. You can read more about its uses in the Documenting Cinder guide.
I’m now a big fan of Python and is now my go-to high level scripting language. It’s amazing how almost everything you need is included right in the language. The only external packages used were BeautifulSoup, for generating and parsing HTML, and Pystache, which is a Python implementation of the Mustache templating library.
The other part of building the documentation was the web front-end. For this we are using the following:
- Bootstrap for basic CSS layout and utilities
- Stylus as the CSS Preprocessor along with gulp
- Prism.js for in-page code syntax highlighting
- Lunr.js for the local search engine
search(term) on the search index returns ranked results. Easy Peasy.
The last thing I want to point out is the thing that got this whole project started in the first place. Late last year, I wrote a guide to using Cinder’s powerful, yet underused, Path2D API. After writing it, we realized that if we put it with the rest of the guides, it would likely just get lost. This is when I set upon updating the guides section of the docs, then finding a way to link to the Path2D class itself, and so on… and so on… until 10 months later when I had a new documentation system made.
This project was a long path that I didn’t even plan on taking. Thanks to Cinder’s mastermind, Andrew Bell for his help and encouragement, Keith Butters for his design work on the docs site and icons, Justin Martin for getting the site working on the server and for dealing with my stupid Python questions, and the members of the Cinder community that helped test the new system and were on board and working with it as soon as it was available.