Extending Asciidoctor

I am still in the mid of polishing my post about Gitlab and Podman to conclude my mini-series, but had to dive into extending Asciidoctor recently to extend our Confluence documentation. This required a bit of good 'ol trial & error and reading code on Github, so it probably is worth my time writing about it and maybe yours as well reading it?

What is AsciiDoc?

If you’ve never heard about AsciiDoc before there are plenty of good resources for a primer, like this article from the FreeBSD docs or even full talks like this one Docs-as-Code @ DevConf if you want to something to watch.

My own rationale to favor it over like Markdown is the support of admonitions, callouts, easy source code linking and the nice extensibility we are going to look into now. Obviously since this blog uses AsciiDoc under the hood, many of the named features are also used to bring this to you.

Tooling and support

Originally written in Python, the new reference implementation is its Ruby rewrite under the same name, which is ultimately used at various prominent places like Github or even at O’Reilly.

I think everything has a fork that runs on the JVM and probably nobody is surprised this also applies to Ruby (hello JRruby), so we ultimately are going to use AsciidoctorJ. One advantage of this language mix is we are able to write extensions either Java or in Ruby.

For the ease of use and good sports we are picking the latter.

Writing an extension

I haven’t found a good documentation of the Ruby API besides the actual code on Github, but since it contains loads of commentary it is quite sufficient. There are a few different types of extensions that are possible, the one we need is an inline macro.

The bare minimum that is required for any extension to work is this:

require 'asciidoctor/extensions' unless RUBY_ENGINE == 'opal'

include Asciidoctor

class HelloInlineMacro < Asciidoctor::Extensions::InlineMacroProcessor
    use_dsl

    named :hello (1)
    name_positional_attributes 'name'

    def process parent, target, attrs (2)
        create_inline_pass parent, 'Hello, %s!' % attrs['name']
    end
end

Asciidoctor::Extensions.register do (3)
    if @document.basebackend? 'html'
        inline_macro HelloInlineMacro
    end
end
1 The DSL hopefully is quite easy to read, so this defines the actual name of the macro
2 We aren’t using many features like attributes yet, but this is just to get us started
3 And lastly we need to register our new extension in the registry

Inside a document this macro can be used like any other internals:

$ cat hello.adoc
hello::[name="World"]
$ asciidoctor -r ./hello_inline_macro.rb hello.adoc (1)
$ htmlq -f hello.html div.paragraph p (2)
<p>Hello, World!</p>
1 This uses the asciidoctor gem
2 Never heard of htmlq before? You should have!

This was quite easy, but the original goal is to update docs on Confluence so let us moving forward.

Adding a J

Achieving the same with AsciidoctorJ requires a bit of more boilerplate, but still this whole endeavor is quite doable.

Registration at the registry is a bit more exhaustive and uses a classpath loader and later on SPI-magic as we are going to see.

Registry loader ahead:

package dev.unexist.asciidoctor;

import org.asciidoctor.Asciidoctor;
import org.asciidoctor.jruby.extension.spi.ExtensionRegistry;

public class HelperRegistry implements ExtensionRegistry {

    @Override
    public void register(Asciidoctor asciidoctor) {
        asciidoctor.rubyExtensionRegistry()
            .loadClass(HelperRegistry.class.getResourceAsStream("/extensions/hello-inline-macro.rb"))
            .inlineMacro("hello", "HelloInlineMacro"); /(1)
    }
}
1 See how nice callouts are? Fortunately here is nothing worrying enough for a real comment.

Followed by the magical part:

$ cat src/main/resources/META-INF/services/org.asciidoctor.jruby.extension.spi.ExtensionRegistry (1)
dev.unexist.asciidoctor.HelperRegistry
1 This facilitates the Java service loader to provide access and a way to extend applications

And easily the messiest part to throw everything into a pom.xml:

<build>
   <plugins>
       <plugin>
           <groupId>org.asciidoctor</groupId>
           <artifactId>asciidoctor-maven-plugin</artifactId>
           <version>${asciidoctor.maven-plugin.version}</version>
           <dependencies>
               <dependency> (1)
                   <groupId>dev.unexist.showcase</groupId>
                   <artifactId>asciidoctor-helper-macros</artifactId>
                   <version>0.1</version>
               </dependency>
           </dependencies>
       </plugin>
   </plugins>
</build>
1 Everything has to be bottled up into a jar - a complete example can be found here: https://github.com/unexist/showcase-asciidoc-extensions/blob/master/asciidoctor-helper-macros/pom.xml

Why, but why is all of this necessary? We are getting closer!

Getting this to Confluence

Pushing stuff to Confluence can be done via their REST-API, but fortunately for us there is an easier way that nicely integrates in all the moving parts we have assembled now.

The Confluence-Publisher plugin comes bundled with Asciidoc support and can nicely talk to the REST API. The only drawback here is not all of the features of Asciidoc are currently supported yet (like tables and having a look at the last change probably never will) yet.

Among the unsupported options is the support to pass Ruby extensions easily, but alas we can add Java dependencies and follow the SPI approach from before.

I’d like to shorten the mess, but all of this is somehow required:

<build>
   <plugins>
        <plugin>
            <groupId>org.sahli.asciidoc.confluence.publisher</groupId>
            <artifactId>asciidoc-confluence-publisher-maven-plugin</artifactId>
            <version>${confluence.publisher.version}</version>
            <configuration>
                <asciidocRootFolder>${asciidocDirectory}</asciidocRootFolder> (1)
                <sourceEncoding>UTF-8</sourceEncoding>
                <rootConfluenceUrl>${confluence.url}</rootConfluenceUrl>
                <spaceKey>${confluence.spaceKey}</spaceKey> (1)
                <ancestorId>${confluence.ancestorId}</ancestorId>
                <username>${confluence.publisherUserName}</username>
                <password>${confluence.publisherPassword}</password>
                <pageTitlePrefix xml:space="preserve"/>
                <publishingStrategy>${confluence.publishingStrategy}
                </publishingStrategy>
                <orphanRemovalStrategy>KEEP_ORPHANS</orphanRemovalStrategy>
                <pageTitleSuffix
                        xml:space="preserve"> [${project.version}]</pageTitleSuffix>
                <versionMessage>Version ${project.version}</versionMessage>
                <attributes>
                    <version>${project.version}</version>
                </attributes>
            </configuration>
            <executions>
                <execution>
                    <id>publish-documentation</id>
                    <phase>generate-resources</phase> (2)
                    <goals>
                        <goal>publish</goal>
                    </goals>
                </execution>
            </executions>
           <dependencies>
               <dependency> (3)
                   <groupId>dev.unexist.showcase</groupId>
                   <artifactId>asciidoctor-helper-macros</artifactId>
                   <version>0.1</version>
               </dependency>
           </dependencies>
        </plugin>
   </plugins>
</build>
1 The list of supported attributes and flags can be found here: https://confluence-publisher.atlassian.net/wiki/spaces/CPD/overview?mode=global
2 Hook into the lifecycle: Render and deploy our asciidoc in the generate-resources phase
3 Remember this from before?

Real world example: Collect versions

The next example aggregates versions from two different types of endpoints and can be used to create an overview e.g. via CICD. If you have read so far it should be just a flick of your fingers to get this extension working:

class CheckversionInlineMacro < Asciidoctor::Extensions::InlineMacroProcessor
    use_dsl

    named :checkversion
    name_positional_attributes 'component', 'stage' (1)

    def process parent, target, attrs
        case target (2)
        when 'apps'
            create_inline_pass parent, handle_apps(attrs)
        when 'backends'
            create_inline_pass parent, handle_backends(attrs)
        end
    end

    private

    def handle_apps(attrs)
        case attrs['component']
        when 'maps'
            case attrs['stage']
            when 'appstore'
                case attrs['os']
                when 'ios'
                    load_from_appstore ENV['URL_APPSTORE_IOS']
                end
            when 'playstore'
                when 'android'
                    load_from_playstore ENV['URL_APPSTORE_ANDROID']
            end
        end
    end

    def handle_backends(attrs)
        case attrs['component']
        when 'blog'
            load_from_backend ENV['URL_BLOG_%s' % attrs['stage'].upcase], ENV['API-KEY'] (3)
        end
    end

    def fetch_data uri, headers = {}
        retVal = ''

        begin
            request = Net::HTTP::Get.new uri (4)

            headers.each do |key, value|
                request[key] = value
            end unless headers.nil?

            response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: 'https' == uri.scheme) { |http|
                http.request request
            }

            unless response.nil? and 200 != response.code.to_i
                retVal = response.body
            end
        rescue => err
            p err
        end

        retVal
    end

    def load_from_appstore url
        data = fetch_data URI.parse(url), {
            'accept' => 'application/json'
        }

        JSON.parse(data)['results'].first['version'].gsub(/[^0-9\.]/, '') rescue 'x.x' (5)
    end

    def load_from_playstore url
        retVal = ''
        data = fetch_data URI.parse(url)

        data.scan(/<script nonce=\"\S+\">AF_initDataCallback\((.*?)\);/) do |match| (6)
            begin
                matches = match.first.scan(/(\d+\.\d+\.\d+)/)

                retVal = matches.first.first unless matches.nil? or matches.empty?
            rescue => err
                p err
                retVal = 'x.x'
            end unless match.nil?
        end unless data.nil?

        retVal
    end

    def load_from_backend url, apiKey = nil
        data = fetch_data URI.parse(url), {
            'accept' => 'application/json',
            'API-Key' => apiKey,
        }

        JSON.parse(data)['version'].gsub(/[^0-9\.]/, '') rescue 'x.x' (7)
    end
end
1 Here we use some more positional attributes to pass information
2 Targets are another way to parameterize macro calls, this makes following call possible: checkversion:apps[component=..]
3 Don’t even think about other ways of passing credentials like this from the outside!
4 There are lots of options available, but we stick to the standard tools
5 I never imagined Apple would offer a more sane way to actually fetch app versions
6 Here be dragons: Google hides the actual version behind a dynamically loaded layer, but data has to go somewhere and fortunately versions are easy to distinguish. (Might change any minute..)
7 Be creative what kind of reply you receive here..

And finally above can be used to render a simple page like this:

[IMPORTANT]
====
This page is automatically updated, so please *do not* update manually.
====

|===
| Component | DEV | Test | Staging | Prod | iOS | Android

| Maps
4+h|
a| checkversion:apps[component="maps" stage="appstore"]
a| checkversion:apps[component="maps" stage="playtore"]

| Blog
a| checkversion:backends[component="blog" stage="dev"]
a| checkversion:backends[component="blog" stage="test"]
a| checkversion:backends[component="blog" stage="staging"]
a| checkversion:backends[component="blog" stage="prod"]
2+h|
|===

Conclusion

AsciiDoc and the toolchain around it allow to create optically appealing documentation from an easy to grasp syntax. Supported by a wide array of output formats like pdf or even manpages it fits perfectly well into any documentation-as-code approach.

Run either manually or in a pipeline the Confluence plugin updates wiki pages on changes and allows access to all kind of interesting parties without the hurdle to have a look at any repository.

Additionally the good extensibility allows customization for any domain requirement or just to ease up writing and/or structuring.

All examples can be found here: