:PROPERTIES:
:ID:       47B63DB4-3322-4422-9595-65BB4A438B04
:END:
#+title: ox-html-stable-ids: Stable IDs for ox-html.el
#+author: Jeff Kreeftmeijer
#+date: 2023-05-03
#+options: toc:nil

Ox-html-stable-ids is an Org export extension package that generates HTML with stable ID attributes instead of the random IDs Org's exporter uses by default.

* Introduction

#+headers: :exports none
#+begin_src org :tangle test/fixtures/hello-world.org
  ,* Hello, world!
#+end_src

#+headers: :exports none
#+begin_src emacs-lisp
  (org-publish-file "test/fixtures/hello-world.org"
                    '("ox-html-stable-ids"
                      :publishing-function org-html-publish-to-html
                      :base-directory "."
                      :publishing-directory "."
                      :section-numbers nil
                      :with-toc nil))
#+end_src

When publishing HTML with Org mode's exporters, the headlines in the resulting documents get assigned ID attributes.
These are used as anchors, amongst other things.
By default, these are random, so a headline might get assigned ~org81963c6~ as its ID:

#+headers: :cache yes
#+headers: :exports results
#+headers: :results scalar
#+headers: :wrap src html
#+begin_src shell
  cat test/fixtures/hello-world.html | grep "<h2 id="
#+end_src

#+RESULTS[33e999dd2bafcb1a0e7b04304bc8b645a7c7baf7]:
#+begin_src html
<h2 id="org81963c6">Hello, world!</h2>
#+end_src

Because subsequent exports of the same Org file produce different IDs, there's no way to link to a headline from an external page.
Ox-html-stable-ids provides stable IDs based on the titles of the headlines they're attached to.
In the example above, the headline's ID would be "hello-world".

* Implementation

Ox-html-stable-ids is disabled by default, even after requiring and enabling the library.
It only replaces IDs in exported HTML documents when ~org-html-stable-ids~ is non-nil:

#+name: options
#+begin_src emacs-lisp
(defgroup org-export-html-stable-ids nil
  "Options for org-html-stable-ids."
  :tag "Org Markdown Title"
  :group 'org-export
  :version "24.4"
  :package-version '(Org . "8.0"))

(defcustom org-html-stable-ids nil
  "Non-nil means to use stable IDs in the exported document."
  :group 'org-export-html-stable-ids
  :version "24.4"
  :package-version '(Org . "8.0")
  :type 'boolean)
#+end_src

The function that generates headlines in Org's HTML exporer (~org-html-headline~) calls a function called ~org-export-get-reference~ to generate a unique reference for the headline.
Ox-html-stable-ids adds an advice to overrides that function to return stable IDs, based on the headline's contents, instead.[fn:adam-porter]

First, the ~org-html-stable-ids--extract-id~ helper function takes a headline and returns a stable ID:

#+name: extract-id
#+begin_src emacs-lisp
  (defun org-html-stable-ids--extract-id (datum)
    "Extract a reference from a DATUM.

  Return DATUM's `:CUSTOM_ID` if set, or generate a reference from its
  `:raw-value` property.  If the DATUM does not have either, return
  nil."
    (or
     (org-element-property :CUSTOM_ID datum)
     (let ((value (org-element-property :raw-value datum)))
       (when value
	 (org-html-stable-ids--to-kebab-case value)))))
#+end_src

If the headline has a ~:CUSTOM_ID~ property, that's immediately returned.
If not, the ID is created by taking the headline's contents and converting them to "kebab case".

#+begin_aside

The illustratively named /kebab case/ is a case style (like =snake_case= or =camelCase=) where all characters are lower case, and all whitespace is replaced by dashes resembling a kebab.
It's used in Lisp-style languages, and URL fragments.

An implementation in Emacs Lisp uses a regular expression to replace everything but letters and numbers to a dash, and then downcases the result:

#+name: kebab-case
#+begin_src emacs-lisp
  (defun org-html-stable-ids--to-kebab-case (string)
    "Convert STRING to kebab-case."
    (string-trim
     (replace-regexp-in-string
      "[^a-z0-9]+" "-"
      (downcase string))
     "-" "-"))
#+end_src

#+end_aside

The ~org-export-get-reference~ is overridden by a function named ~org-html-stable-ids--get-reference~, which calls the ~org-html-stable-ids--extract-id~ to extract IDs for headlines.
It uses an internal propetry list named ~:internal-references~ as a cache to store generated IDs in.
If a generated ID matches one that's already in the cache, an error is returned, and the export is aborted:

#+name: org-reference
#+begin_src emacs-lisp
  (defun org-html-stable-ids--get-reference (orig-fun datum info)
    "Return a reference for DATUM with INFO.

      Raise an error if the ID was used in the document before."
    (if org-html-stable-ids
	(let ((cache (plist-get info :internal-references))
	      (id (org-html-stable-ids--extract-id datum)))
	  (or (car (rassq datum cache))
	      (if (assoc id cache)
		  (user-error "Duplicate ID: %s" id)
		(when id
		  (push (cons id datum) cache)
		  (plist-put info :internal-references cache)
		  id))))
      (funcall orig-fun datum info)))
#+end_src

Org's HTML exporter doesn't call the ~org-export-get-reference~ function directly, but has an internal function named ~org-html--reference~ that's called whenever a reference is needed.
To ensure all ids are checked against the internal references list, this package overrides =org-html--reference= to always call ~org-export-get-reference~ directly:[fn:override]

#+name: org-html-reference
#+begin_src emacs-lisp
  (defun org-html-stable-ids--reference (datum info &optional named-only)
    "Call `org-export-get-reference` to get a reference for DATUM with INFO.

  If `NAMED-ONLY` is non-nil, return nil."
    (unless named-only
      (org-export-get-reference datum info)))
#+end_src

Finally, the advise is added (and possibly removed) through the ~org-html-stable-ids-add~ and ~org-html-stable-ids-remove~ functions:

#+name: add-remove
#+begin_src emacs-lisp
  (defun org-html-stable-ids-add ()
    "Enable org-html-stable-ids."
    (interactive)
    (advice-add #'org-export-get-reference :around #'org-html-stable-ids--get-reference)
    (advice-add #'org-html--reference :override #'org-html-stable-ids--reference))

  (defun org-html-stable-ids-remove ()
    "Disable org-html-stable-ids."
    (interactive)
    (advice-remove #'org-export-get-reference #'org-html-stable-ids--get-reference)
    (advice-remove #'org-html--reference #'org-html-stable-ids--reference))
#+end_src

[fn:adam-porter] This is based on [[https://github.com/alphapapa/unpackaged.el#export-to-html-with-useful-anchors][Adam Porter's useful anchors example]], which differs in a couple of ways:

Adam's example uses URL encoded IDs, instead of stripping all non-alphabetic and non-numeric characters and converting it to kebab-case.
For non-unique IDs, it prepends the ancestors' IDs and appends numbers until each ID is unique instead of raising an error and forcing the user to use custom IDs.
It's the better choice if you need stable IDs that sort themselves out without ever breaking your publishing.

[fn:override] The =org-html--reference= function has added logic to check the /html-prefer-user-labels/ attribute.
By calling out to =org-export-get-reference= directly, that functionality is lost, meaning this library implies the /html-prefer-user-labels/ setting.

* Results

Now, all headlines in the file get assigned IDs that match their contents:

#+headers: :exports none
#+begin_src org :tangle test/fixtures/multiple-headlines.org
  ,* Hello, world!
  ,* Another headline!
#+end_src

#+headers: :exports none
#+begin_src emacs-lisp
  (org-html-stable-ids-add)
  (org-publish-file "test/fixtures/multiple-headlines.org"
                    '("ox-html-stable-ids"
                      :publishing-function org-html-publish-to-html
                      :base-directory "."
                      :publishing-directory "."
                      :section-numbers nil
                      :with-toc nil))
  (org-html-stable-ids-remove)
#+end_src

#+headers: :cache yes
#+headers: :exports results
#+headers: :results scalar
#+headers: :wrap src html
#+begin_src shell
  cat test/fixtures/multiple-headlines.html | grep "<h2 id="
#+end_src

#+RESULTS[d77a3ded5df0512226d22272ca65398c78af419a]:
#+begin_src html
<h2 id="hello-world">Hello, world!</h2>
<h2 id="another-headline">Another headline!</h2>
#+end_src

If a headline has a ~CUSTOM_ID~, that's used instead of the generated one:

#+begin_src org :tangle test/fixtures/cutom-id.org
  ,* Hello, world!
  ,* Another headline!
  :PROPERTIES:
  :CUSTOM_ID: custom-id
  :END:
#+end_src

#+headers: :exports none
#+begin_src emacs-lisp
  (org-html-stable-ids-add)
  (org-publish-file "test/fixtures/cutom-id.org"
                    '("ox-html-stable-ids"
                      :publishing-function org-html-publish-to-html
                      :base-directory "."
                      :publishing-directory "."
                      :section-numbers nil
                      :with-toc nil))
  (org-html-stable-ids-remove)
#+end_src

#+headers: :cache yes
#+headers: :exports results
#+headers: :results scalar
#+headers: :wrap src html
#+begin_src shell
  cat test/fixtures/cutom-id.html | grep "<h2 id="
#+end_src

#+RESULTS[1d4aada74dd82755e5d3f517f79ab943301db01f]:
#+begin_src html
<h2 id="hello-world">Hello, world!</h2>
<h2 id="custom-id">Another headline!</h2>
#+end_src

Now, the function raises an error when two headlines resolve to the same ID:

#+begin_src org :tangle test/fixtures/duplicate-headlines.org
  ,* Hello, world!
  ,* Hello, world!
#+end_src

#+headers: :exports none
#+begin_src emacs-lisp
  (org-html-stable-ids-add)
  (org-publish-file "test/fixtures/duplicate-headlines.org"
                    '("ox-html-stable-ids"
                      :publishing-function org-html-publish-to-html
                      :base-directory "."
                      :publishing-directory "."
                      :section-numbers nil
                      :with-toc nil))
  (org-html-stable-ids-remove)
#+end_src

#+begin_example
Duplicate ID: hello-world
#+end_example

As expected, the error is also raised when a custom ID is duplicated:

#+begin_src org :tangle test/fixtures/duplicate-headlines-with-custom-id.org
  ,* Hello, world!
  ,* Another headline!
  :PROPERTIES:
  :CUSTOM_ID: hello-world
  :END:
#+end_src

#+headers: :exports none
#+begin_src emacs-lisp
  (org-html-stable-ids-add)
  (org-publish-file "test/fixtures/duplicate-headlines-with-custom-id.org"
                    '("ox-html-stable-ids"
                      :publishing-function org-html-publish-to-html
                      :base-directory "."
                      :publishing-directory "."
                      :section-numbers nil
                      :with-toc nil))
  (org-html-stable-ids-remove)
#+end_src

#+begin_example
Duplicate ID: hello-world
#+end_example

#+headers: :noweb yes
#+headers: :tangle ox-html-stable-ids.el
#+headers: :exports none
#+begin_src emacs-lisp
  ;;; ox-html-stable-ids.el -- Stable IDs for ox-html.el

  ;; Author: Jeff Kreeftmeijer <jeff@kreeft.me>
  ;; Version: 0.1.1
  ;; URL: https://jeffkreeftmeijer.com/ox-html-stable-ids/

  ;;; Commentary:

  ;; ox-html-stable-ids.el replaces the default, unstable IDs with
  ;; stable ones based on headline contents.
  ;;
  ;; Unstable ID:
  ;;    <h2 id="org81963c6">Hello, world!</h2>
  ;;
  ;; ID generated by ox-html-stable-ids:
  ;;    <h2 id="hello-world">Hello, world!</h2>

  ;;; Code:

  (require 'ox)

  <<options>>

  <<extract-id>>

  <<kebab-case>>

  <<org-reference>>

  <<org-html-reference>>

  <<add-remove>>

  (provide 'ox-html-stable-ids)

  ;;; ox-html-stable-ids.el ends here
#+end_src

* Installation and usage

Ox-html-stable-ids is currently not available through any of the package registries.
Instead, install it from the repository direcly.
Install the package with [[https://github.com/jwiegley/use-package][use-package]] and [[https://github.com/radian-software/straight.el][straight.el]], and enable it by calling ~org-html-stable-ids-add~:

#+begin_src emacs-lisp
  (use-package ox-html-stable-ids
    :straight
    (ox-html-stable-ids :type git :host github :repo "jeffkreeftmeijer/ox-html-stable-ids.el")
    :config
    (org-html-stable-ids-add))
#+end_src

After calling ~org-html-stable-ids-add~, set the ~org-html-stable-ids~ variable to to enable the package while exporting:

#+begin_src emacs-lisp
  (let ((org-html-stable-ids t))
    (org-html-publish-to-html))
#+end_src

Get stable IDs:

#+headers: :cache yes
#+headers: :exports results
#+headers: :results scalar
#+headers: :wrap src html
#+begin_src shell
  cat test/fixtures/hello-world.html | grep "<h2 id="
#+end_src

#+RESULTS[33e999dd2bafcb1a0e7b04304bc8b645a7c7baf7]:
#+begin_src html
<h2 id="hello-world">Hello, world!</h2>
#+end_src

* Contributing

The git repository for ox-html-stable-ids.el is hosted on [[https://codeberg.org/jkreeftmeijer/ox-html-stable-ids.el][Codeberg]], and mirrored on [[https://github.com/jeffkreeftmeijer/ox-html-stable-ids.el][GitHub]].
Contributions are welcome via either platform.

** Set up git remotes                                              :noexport:

Since ox-html-stable-ids.el is mirrored to GitHub, it uses multiple push remotes.
Commits pushed to the main branch should be mirrored.
A script is included in =scripts/remotes= to automate setting up these remotes after pulling:

#+headers: :shebang #!/bin/sh
#+headers: :tangle scripts/remotes
#+headers: :eval no
#+begin_src shell
  for remote in \
    "https://codeberg.org/jkreeftmeijer/ox-html-stable-ids.el.git" \
    "https://github.com/jeffkreeftmeijer/ox-html-stable-ids.el.git"
  do
    git remote set-url --delete --push origin $remote 2> /dev/null
    git remote set-url --add --push origin $remote
  done

  git remote show origin
#+end_src

To set up the git remotes, run the script:

#+headers: :exports both
#+headers: :results scalar
#+begin_src shell
  ./scripts/remotes
#+end_src

#+RESULTS:
: * remote origin
:   Fetch URL: https://codeberg.org/jkreeftmeijer/ox-html-stable-ids.el.git
:   Push  URL: https://codeberg.org/jkreeftmeijer/ox-html-stable-ids.el.git
:   Push  URL: https://github.com/jeffkreeftmeijer/ox-html-stable-ids.el.git
:   HEAD branch: main
:   Remote branch:
:     main new (next fetch will store in remotes/origin)
:   Local ref configured for 'git push':
:     main pushes to main (fast-forwardable)

** Generate README                                                 :noexport:

The =README.md= file is generated from =ox-html-stable-ids.org= with an included script named =scripts/readme=:

#+headers: :shebang #!/bin/sh
#+headers: :tangle scripts/readme
#+headers: :eval no
#+begin_src shell
  emacs --batch ox-html-stable-ids.org --load scripts/readme.el/readme.el --eval "(readme/to-markdown \"README.md\")"
#+end_src

To generate the README, run the script:

#+headers: :exports both
#+headers: :results scalar
#+begin_src shell
  ./scripts/readme
#+end_src

#+RESULTS:

** Tests

Regression tests are written with [[https://www.gnu.org/software/emacs/manual/html_mono/ert.html][ERT]] and included in =test.el=.
To run the tests in batch mode:

#+begin_src shell
  emacs -batch -l ert -l test.el -f ert-run-tests-batch-and-exit
#+end_src

#+RESULTS:
