Motivation

Around 04/16/2024, after hearing again somewhere on the Internet that roam is dead, I decided to try some alternatives, and this time around, I focused on Obsidian.

I would prefer an asciidoc-based system to a markdown-based one, but such does not exist…

I am not paranoid about my notes being stored in the cloud, and actually prefer them to be, since it gets me the ability to synchronize them across my devices and also guarantees that I am not going to lose them. Still, I thought that if I set up backup to something like Google Drive, Obsidian could work for me - although I won’t be able to use it from my phone unless I pay for Obsidian Sync.

One of the advantages of the move: I want to be able to publish my notes easily, in a way that preserves internal links between them (Digital Garden?); roam doesn’t let me do this, and Obsidian does.

More: I can publish my notes and my blog (currently generated by Jekyll and hosted by GitHub pages) in the same place.

More still: I can host the notes themselves in the same repository as my website and blog! This way, I do not need any separate backup to Google Drive, and everything is kept together, modified together and published together (notes that I do not want published need to be moved out of the repository).

Also, there is something to be said for the ability to edit my notes using tools other than Obsidian - e.g., IntelliJ Idea ;)

Import

To move my notes from roam, I installed Importer plugin in Obsidian and followed the export/import instructions; results of the import required manual clean-up:

  • outline bullets from Roam got translated into stars at wrong indent level;
  • tags like #jewish-calendar that I had at the start of some Roam notes moved into the YAML frontmatter’s tags array;
  • tags like #buy and #learn were converted to page references like [​[buy]] and [​[learn]] so that thy are recognized as links by Obsidian and back-links are available at the appropriate pages;
  • similarly, TODOs were prefixed by [​[TODO]], to facilitate seeing them in the list of back-links; once done, [​[TODO]] becomes a [​[DONE]];
    • todo look into Obsidian plugins that support TODOs
  • code blocks needed touch-up;
  • I used Find Orphaned Files and Broken Links Obsidian plugin to clean up broken links (it found two block references :)).

Setup

In this unified setup:

  • blog posts are, as usual, under _posts;
  • blog post drafts - under _drafts;
  • notes are under notes; in Obsidian Settings / Files and Links:
    • Default location for new notes: In the folder specified below
    • Folder to create new notes in: “notes”
  • daily notes are under days; in Obsidian Settings / Core plugins:
    • Daily notes / New file location: “days”
  • Outliner Obsidian plugin could allegedly help with the Roam-style outlining.

Publishing

obsidian sells its own Obsidian Publish, but - although Obsidian Publish and Obsidian Sync together cost less than roam - I want to explore other publishing options: I would like to publish my notes using a custom domain, but Obsidian Publish supports custom domains only at CloudFlare or some such; also, my goal is to keep my notes in the same GitHub repository as my blog and publish everything from there on GitHub Pages.

I looked (briefly) at Obsidian digital garden, Quartz and PublishKit, but decided to see first if I can add publish my notes using Jekyll, which I use currently to publish my blog.

GitHub Publisher is an Obsidian plugin which converts obsidian document into a form suitable for publication and pushes them into a GitHub repository. Unfortunately, it can not handle the situation where the source documents are kept in the same GitHub repository: “If you use your vault directly in a repository, the upload will corrupt your files! This module is not intended for this type of workflow.”

Even if there was a way to keep both the sources of the notes and the result of their conversion to the publishable form in the same repository, just as I rely on Jekyll running on GitHub pages to convert my markdown to HTML, and do not check in the results of running jekyll locally, I prefer not to check in the notes processed for publication by some Obsidian plugins. For now I am pursuing the approach where all the processing needed for publication is handled by Jekyll (with some plugins and other customization), without relying on any Obsidian plugins. Of course, if it turns out to be impossible to make Jekyll produce the results I want, I may have to reconsider this choice of approach.

It used to be the case that GitHub Pages restricted Jekyll plugins that could be used; since I recently switched to running Jekyll using GitHub Actions workflow (and soon everyone will switch), this is no longer a problem: all Jekyll trickery is available now ;) The question is: how to make Jekyll do what needs to be done?

File Names and Titles

Unlike Obsidian, which uses document’s file name as its title, many Jekyll plugins and code snippets that I need to make Jekyll do what I want require the pages they process to have titles; my notes, as imported from roam, do not, so I need to do something about it.

Jekyll Collections

One possibility is to make notes and days into Jekyll Collections, where the member documents have their title, if it is not set explicitly, automatically populated based on the file name - but:

  • Jekyll collection directories have their names start with _, so the notes will have to reside in the directory _notes both in the GitHub repository and on the published website;
  • sub-directories of Jekyll collections get ignored, but I want the ability to have sub-directories in my notes;
  • when Jekyll populates the title of the document from its file name, it capitalizes the first letter of each dash-separated segment, and I am not sure that this is what I want.

Liquid Code

Another possibility is to adjust the code that requires titles to use filename instead (when the title is not set explicitly) - but:

  • this works for Liquid and Ruby code snippets, but not for Jekyll plugins - and I probably need those;
  • without the title being set explicitly, my notes as they are rendered by Jekyll do not have titles - and I want the titles!

Linter Obsidian Plugin

So, I need to make sure that my notes always have titles, just like the blog posts. Obviously, adding the title manually to every little note created by just referencing it from another note is tedious and error-prone; I need to automate setting the title for the notes.

Linter Obsidian plugin can set the title property in each note to its file name; in Obsidian Settings / Community plugins / Linter / Settings:

  • General / Lint on save: enabled
  • YAML / YAML Title / Insert the title of the file into the YAML frontmatter: enabled

Sometimes I need the title to be different from the file name, for instance, for the manually created index.md files that I want to add to Jekyll’s listheader_pages in the _config.yml; in such cases, appropriate Linter rule can be disabled by adding a property to the file’s YAML frontmatter:

---
disabled rules: [yaml-title]
---

Jekyll

Jekyll needs to be enhanced (using Jekyll plugins, layouts and includes and Liquid code snippets) to publish everything the way I want it.

MathJax

I freshened up my Jekyll MathJax setup by upgrading to MathJax 3; I followed the instructions of Bodun Hu - a rare find that addresses MathJax 3, as opposed to the more wide-spread instructions for MathJax 2 integration.

To enable MathJax on a page, add math: true to its front matter:

---  
...  
math: true  
---
...

In _includes/head.html:

<head>
  ...
  {%- if page.math == true -%}  
    {% include mathjax.html -%}  
  {%- endif -%}
</head>

In _includes/mathjax.html:

<script>  
  MathJax = {  
    tex: {inlineMath: [ ['$', '$'], ['\\(', '\\)'] ]},  
    svg: {fontCache: 'global'}  
  };  
</script>  
<script  
  type="text/javascript" id="MathJax-script" async  
  src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg.js"  
>  
</script>

Directory Listing

With the notes published on my website, I want a list of them to be published too - otherwise, nobody can find the individual notes. I found a solution; thank you, Michael Currin!

I did not make sub-folders appear in the lists automatically yet, not just files, so I add them manually, as content of the index files.

  • todo add sub-folders to the folder listing automatically;
  • todo generate the index files themselves automatically.
  • todo paginate the page list.

In _layouts/page.html:

<div class="post-content">
  {{ content }}
  {% if page.name == 'index.md' %}  
    {% include page-list.html %}  
  {% endif %}  
</div>

In _includes/page-list.html:

<ul class="page-list">  
  {% assign pages = site.pages | sort_natural: 'title' %}  
  {%- for item in pages %}  
  {%- if item.dir == page.dir and item.path != page.path %}  
  <li>
    <a href="{{ item.url | relative_url }}">
      <span>{{ item.title }}</span>
    </a>
  </li>  
  {% endif -%}  
  {% endfor -%}  
</ul>

In _sass/custom.scss:

.page-list {  
  font-size: 18px;  
  
  & i {  
    font-size: 14px;  
    // Based on blockquote.  
    color: #828282;  
  }  
}

Unlike obsidian, Jekyll does not understand wiki-links. There is a Jekyll plugin for that: Jekyll Wikirefs, but its 0.0.14 release did not work with current Jekyll; I asked for an update - and release 0.0.15 relaxed the dependency requirements! Thank you, manunamz!

In the interim, I used (after tweaking it a bit) a piece of Liquid code that does what needs to be done with basic wiki-links: brackettest (listed on the plugin’s website :)).

Plugin does not respect the boundaries of the code blocks (but brackettest does), so to stop it from processing wiki-links inside such blocks (for instance, in this post :)), I insert a zero-width space between the two opening brackets: [​[. This is sub-optimal, since the invisible zero-width space ends up in the rendered code, which, as a result, can not be re-used by copying it: it is not what it looks like (in addition, the use of ZWSP throws off the code colorizer and requires an editor that shows invisible characters).

Another way - back-slash-escaping the opening bracket (\[) does not work within the code block. I asked for the plugin to not process links within the code blocks, but this seems to be unfeasible in the “legacy system” Jekyll; awaiting manunamz’s suggestion for a non-legacy system.

Plugin down-cases page titles; I asked for this down-casing to be configurable.

Plugin gathers and puts into the front-matter of each page list of back-links to it. To show a list of back-links on a page, in _layouts/default.html add (note the use of the concat filter to make sure that links from both pages and posts are resolved):

<main ...>  
  <div class="wrapper">  
    {{ content }}  
  
    <div class="backlinks">  
    {% if page.backlinks != blank %}
    <hr/>  
    <h4>Backlinks</h4>  
    {% for backlink in page.backlinks %}  
    {% assign linked_doc = site.pages | concat: site.posts | where: "url", backlink.url | first %}  
    • <a class="backlink" href="{{ linked_doc.url }}">{{ linked_doc.title }}</a>  
    {% endfor %}  
    {% endif %}  
    </div>  
  </div></main>

Links can be styled in _sass/custom.scss:

.wiki-link::before {  content: "[[";  }  
.wiki-link::after {  content: "]]";  }  
.invalid-wiki-link {  background: red;  }  
  
.backlink {  font-style: italic; }
  • todo look into showing the graph with https://github.com/wikibonsai/jekyll-graph
  • todo look deeper into wikibonsai

Pages by Tag

Now that I have my blog and my notes in the same repository and in the same Obsidian vault, I can cross-link blog posts and notes; both can have tags in their YAML frontmatter, and any overall list of tags, tag cloud or whatever I end up publishing on the site needs to include both posts and notes.

There are many code snippets floating around that add listing of tags and documents tagged with them to a site generated by jekyll, for example: “an easy way to support tags in a jekyll blog”, “listing jekyll posts by tag”, “https://www.maggie98choy.com/Add Tags in Jekyll/”, jekyll-tagging jekyll plugin.

All of the samples I saw use site.tags, which maps tags to lists of posts that have the tags, but only posts are included, not pages. To gather tags and associated files across the posts and pages, I copied a chunk of Jekyll code, tweaked it to include all posts and pages, tweaked the sorting, packaged it as a Liquid filter and put it into _plugins/all-tags.rb:

module Jekyll  
  module AllTagsFilter  
    def all_tags(site)  
      @all_tags ||= begin  
       hash = Hash.new { |h, key| h[key] = [] }  
       (site.documents + site.pages).each do |p|  
         p.data["tags"]&.each { |t| hash[t] << p }  
       end  
       hash.each_value { |pages| pages.sort_by! { |page| page.data["title"].downcase } }  
       hash.sort_by { |word| word[0].downcase }
     end  
    end  endend  
  
Liquid::Template.register_filter(Jekyll::AllTagsFilter)

List of tags and pages for each is generated using the all_tags filter in the file tags.html (which I list under header_pages in _config.yml):

---  
layout: page  
title: "Tags"  
disabled rules: [yaml-title]  
description: "Pages by tags"  
permalink: /tags/  
---  
  
<div id="tags">  
  {% assign tags = site | all_tags %}  
  <h2>All tags</h2>  
  <p>
    {% for tag in tags %}  
    <a class="page-tag" href="{{ site.baseurl }}/tags/#{{ tag[0] | slugify }}">{{ tag[0] }}</a>  
    {% endfor %}  
  </p>  
  <h2>Pages by tags</h2>  
  {% for tag in tags %}  
  <div id="{{ tag[0] | slugify }}">  
    <h3>{{ tag[0] }}</h3>  
    {% for post in tag[1] %}  
    <ul>  
      <p>
        <a class="post-link" href="{{ post.url | relative_url }}">{{ post.title | escape }}</a>  
      </p>
    </ul>
    {% endfor %}  
  </div>  
  {% endfor %}  
</div>
  • todo Look into tag pages aliased to tags (with the help of the Tag Wrangler Obsidian plugin?) to bring tags like #computer closer to classifier pages like [​[buy]] and [​[learn]] (while in roam they were equivalent) - or!, just link the tag header in the by-tag list to a page with the same name if it exists :) (possibly ignoring case).

Page Tags

Tags for posts and for pages like notes are listed on the page, linking back to the appropriate place by-tag list.

In _includes/tags.html:

{% if page.tags %} |  
{% for tag in page.tags %}  
<a class="post-tag" href="{{ site.baseurl }}/tags/#{{ tag | slugify }}">{{ tag }}</a>  
{% endfor %}  
{% endif %}

In _layouts/post.html, between the date and the author:

{%- include tags.html -%}

And in _layouts/page.html, after the title:

<p class="post-meta">  
{%- include tags.html -%}  
</p>

Tags are styled in _sass/custom.scss:

.page-tag {  
  display: inline-block;  
  background: $grey-color-light;  
  padding: 0 .5rem;  
  margin-right: .5rem;  
  border-radius: 4px;  
  color: $text-color;  
  font-size: 90%;  
  &:before {  
    content: "\f02b";  
    font-family: FontAwesome;  
    padding-right: .5em;  
  }  
  &:hover {  
    text-decoration: none;  
    background: $grey-color;  
    color: $background-color;  
  }  
}

To use the fancy tag character, make fancy font available in _includes/head.html:

<head>
  <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
</head>
  • todo update the font - or get rid of it if possible

Zotero Integration