Parenting (Notes Hierarchy)

[VKondakoff]

This is my first pull request, so I'm a little bit nervous... ;)
In addition English is not my native language, so, please, excuse me for all the possible typos and mistakes.

[jevakallio]

I feel like that multiple parentage would be better described as [[links]] or as #tags. We know from programming languages that multiple inheritance gets messy quickly, and that trees are simpler than graphs.

For simplicity, I'd propose that parenting allows every note to have only one parent. That will make visualisation significantly easier, too.

[VKondakoff]

Hmm... I fully understand that implementing multiple parents is much more difficult. But having multiple parents seems essential to me: this is one of the main benefits of parenting comparing to a plain file/folder hierarchy, where one note can reside in one and only physical location.

[jevakallio]

Thanks @VKondakoff! I've given this is a quick read. I don't yet have a strong opinion one way or another.

I am absolutely in support of adding parent: other-note-id metadata support into Foam notes, so that other extensions that want to use this information can use Foam's underlying metadata graph to build visualisations, new explorer features etc.

I'm also not fundamentally opposed to building this into the Foam ecosystem itself. We don't yet have a clear-cut decision on what belongs where -- whether we should be creating a monolithic Foam extension that bundles tons of features, or many smaller features that share a core component such as the underlying note graph (perhaps via a Language Server Protocol Service -- as a note mainly to other maintainers, it's a reference to a possible future architectural approach for Foam, see #foam-lsp on Discord).

I like it. If folks want this, I think it should exist. But who should implement it is a bit more unclear. You're always free to implement this as a separate extension, and I'm happy to collaborate to make it Foam compatible.

I'll wait for more comments from others.

[thomaskoppelaar]

Great suggestion @VKondakoff ! I think it's a wonderful suggestion, and something that's quite handy to have. Giving certain links between notes a different type of relation (Parent <-> Child) allows for more clear visualisation of hierarchy. I think this also has the benefit of not being intrusive to other systems, and it's something users can choose to ignore if they wish to do so.

Another idea could be to mark specific files as parents, and have every file that links to it a "child".

[thomaskoppelaar]

Actually - you could combine those ideas to allow hierarchies bigger than one level: Marking a file such as "Object Oriented Programming" as a parent, and adding a parent: university to it as well

[ShaunaGordon]

Could this be tied to the idea about backlinks actually being inserted into pages (particularly "index"/TOC pages)? I feel like that's more or less what the index pages are trying to achieve.

I do agree that having some type of hierarchical organization capability would be useful. I, for one, do sometimes traverse my notes that way (had to do that multiple times over the past few days, in fact), and would love a little bit of categorization as a first-class thing.

That said, as a programmer, I totally get the desire to avoid multiple parentage and the whole "construction over inheritance." But...how much does that actually apply to note organization, especially in a tag-based ecosystem like Foam? I genuinely don't know.

I think this would certainly have an interesting effect on how the graphing works. From what I'm imagining, I can see it resulting in graphs that are a bit more like how I, at least, think. I saw mentioned somewhere that we don't recall things via "remember everything about X topic," and yet...that's sometimes exactly what I do and need to do, and it's something that I need to do more often with my "second brain" or whatever note system, largely because my bio-brain can't do that as well. The resulting graphs would, then, be a combination of "constellations" (or "clouds") and "trees," communicating different types of relationships between the different items.

It might help to consider what information the tags and links, themselves, convey. In a generic tagging system, they simply confer a generic relationship. There isn't really any other information in the links themselves other than that they are linked. But don't we sometimes want to know how they're linked? I think that's where this "parent" link can come in (and opens up other link types, perhaps).

Items can be linked arbitrarily, but sometimes we want them grouped into collections, too. And sometimes, we want them in multiple collections. And sometimes we want specific paths through those collections, because of the nature of the relationship between the items (ie - one is dependent on the context created by another). Having the means to convey that sort of "meta" information would be immensely useful, I think.

If we do it as part of the text-based tagging system, I think it could easily be worked into the core system without being intrusive. I think that would depend almost entirely on what the driving goal/purpose of the Foam core is.

tl;dr - I think this is a fantastic idea and opens up a whole set of ideas for more powerful linking paradigms. :)

[nixsee]

This is an idea I've floated a few times in Discord, notably referring to The Brain which handles this very nicely. Not only are there Parents and Children, but other Children of the same parent implicitly become Siblings. And you can also create a Jump-link that is related but not hierarchically. Following on from the travel analogy, a Jump link would be a link to your personal finance/budget notes or to notes on airplanes or famous travel memoir books. (This is just putting a label on what Shauna described above with some links being arbitrary/ambiguous).

The key to all of this, as others have suggested, is not that ideas aren't hierarchical (so replace hierarchical folder structures with tags/links), but that they belong to multiple hierarchies at the same time (so keep hierarchies and allow for notes to fit in and link between any of them).

The image below shows how The Brain displays these different relations in their graph. It seems pretty ideal to me rather than just having a central node and a bunch of ambiguous links to other files. Its particularly nice how they arrange them in alphabetical, aligned lists (but that's probably another discussion for graphing implementations).

You can learn more about The Brain at their User Guide or explore their tutorials and other Learning resources.

This excerpt from "Jerry's Brain", which is the largest brain with something like 600,000 notes, is a particularly good example of how these link contexts add tremendous value (live view here):

There's gotta be about 75 links emanating from the central "Words I love" (actually, could be something like 150 given the scrollable list of Siblings in the top right corner - each showing which of 6 different parents they are linked to) and it all makes perfect sense because of the contexts (and aligned, alphabetical lists). If it was just 150 links in no particular context, arranged by "the force", it would be a nearly unusable, information-less mess. Moreover, in addition to making it better for each of us to use personally, I think this makes it FAR more usable for OTHER people to look at because they don't know the context that might be embedded in your actual brain.

After singing so many praises of The Brain (its the only graphical implementation for notes/ideas I've ever found that makes any sense at all), I should say that the reason that I don't use it is because I prefer a text-first environment, and its quite clearly graph/visual-first. I'm hopeful that Foam can strike a nice balance between the two - the text is already, more or less, done thanks to VS Code and the various extensions, and (along with improved functions like block-level links/embeds and the associated infrastructure), just need to polish off the graphing function with a node-centered view (already being worked on) and integrate with a publishing process. Adding child/parent/sibling/jump context to that would not just take it to another level, but I think make it actually usable with a Foam bathub of any appreciable size.

[VKondakoff]

@thomaskoppelaar

Actually - you could combine those ideas to allow hierarchies bigger than one level: Marking a file such as "Object Oriented Programming" as a parent, and adding a parent: university to it as well

Exactly! I made a one-level example just for a simplicity. Of course, every parent note may be a child of another parent. Should I mention this explicitly in the RFC?

[thomaskoppelaar]

@VKondakoff sure, I think it's worth mentioning.

[ShaunaGordon]

Oooh! Perhaps some type of link identification system?

So like... the parent: university example becomes [[parent] university] or something? Or [[category: university]]?

Could also add a config capability to define your own types of relationships. Basically, you'd set the name of the relationship (which would take the place of parent or category in the example snippets) and tell the linking system what kind it is (hierarchical/dependent, sibling, etc).

[Whil-]

@ShaunaGordon what you describe looks similar to what's discussed here:
https://org-roam.discourse.group/t/add-link-tags-feature/171

A different context and tool, sure, but the idea seems similar. Food for thought at least.

[nixsee]

Thanks @Whil-, that's a great article.

I actually started writing a response that was critical of it, suggesting that a simple Triumvirate of Parent, Child, and Jump link-tags would cover all use cases without letting the user fall into the common trap of tags: that they become too numerous, forgotten and and senseless to be useful at all.

But, at the risk of allowing that to be done by careless users, I think user-customizable link-tags does have a tremendous amount of power. Putting aside some of the suggested organizational/logistical/structural uses (Index, Daily Note, etc...), as well as some more frivolous use-cases (Person B being an "Enemy" of A but "Friend" of C), the one use-case (beyond The Brain's Triumvirate) that would be exceptionally useful and powerful is being able to discern Compatibility of notes/ideas - probably most relevant to Jump Links than Parent/Child ones.

The article talks about this with regards to striving for coherence rather than having a "collectors mentality" that is all too common in the Knowledge Management world, and suggests tags like "‘hypothesis_of’, ‘conclusion_of’, ‘refuted_by’, ‘derived_from’, etc.". It is this that I'd like to call extra attention to, since it might be lost in that very long post.

Using wikilinks and (hopefully eventually) an interactive graph (where we can create notes and links graphically, like The Brain), it is very easy to create a network of links between related ideas. But, when we link two ideas together (especially in the initial stages of populating our database/graph), I suspect there isn't a great deal of deep learning going on - its just cataloguing/collecting things we already "know". But even when there is a flash of insight or creativity for connecting two ideas, I think this pales in comparison to a different sort of learning.

Real, deep, meaningful learning comes when we are confronted with the reality that two ideas that we hold to be true are not actually compatible with each other. If we dare to reckon with this (generally we don’t), we are forced to either learn more and discard one (or both!) of them, or perhaps even find a way to synthesize the two through a more nuanced concept/understanding . That's not just creation of knowledge, but insight and, maybe even, Wisdom - the holy grail.

But doing this isn't really all that possible in the existing frameworks, because a link sort of just implicitly says "these are compatible". An index of "Influential people of the 20th Century" might have MLK, Hitler, Pol Pot, Ghandi, Che Guevara etc... They are not compatible, but graphically you wouldn't see that. You could write context into the notes or even create an index - headings for Nice Guys and Mean Guys - but, you still don't see that visually, and its also a laborious process to do for every note/idea.

You'd want to evaluate the ideas and principles that underlie these people to get a better evaluation of them all. Perhaps Che Guevara and MLK are both considered to be champions of life and freedom, but MLK was, more than anything, preaching for nonviolent resistance, and Che, obviously, was not. If you're trying to get a conception of how to act in the world, it might be worth either discarding one of them or find a nuanced synthesis between the two.

A very tangible example of what I'm getting at is with a blog/website that I am very passionate, yet ambivalent, about: Brain Pickings. It is a tremendous, heavily-interlinked resource of great ideas and people, and is my most recommended website. But there is tremendous incoherence on that site - frequently writing articles akin to calling Che the greatest, truest, wisest, most prescient (she loves using superlatives) leader and then the next day saying MLK was the saintliest man to ever walk the face of the earth, and then the following day writing about how some morally ambivalent artist's words were the most timely and soul-calming of the 20th century. If you made a network map of that site, you'd have a tremendous amount of nodes and interlinks, but it wouldn't be at all apparent - other than the existence of separate clusters - that a lot of it just doesn't fit together. I'd love to be able to examine which clusters are actually True, not only on Brain Pickings, but especially my own network of ideas, which is surely full of its own incoherences and misconceptions.

Most conceptually/abstractly, if we're really looking for growth (and isn't that what we're all, presumably, after?), its necessary to really examine our notions of Good, Bad, Community, Love, Friendship, Purpose, Meaning, Happiness etc... They're bound to be incoherent, especially if we haven't actively considered, let alone anguished over, them.

So, in addition to allowing for multiple Parent/Child/Jump links per note (which is simply necessary for a purely informative classification of ideas), if we could tag links as Compatible and Incompatible we could then generate lists or even queries to regularly review. Better yet, we could display links on the graph in Green or Red so that you would be confronted with your own incoherence every time you visit the graph. Ideally, you'd grow more fond of mapping out incompatibilities (a stern, but wise, teacher) than compatibilities (Yes Men) and work towards the refinement of your graph/network, rather than ever-expanding towards nothing.

In short, I'd like Foam to help me degrease, rinse off and polish my thoughts, rather than just blowing infinite bubbles in a bucket of soap - fun to do, but not really helping me refine myself.

Of course, you just might want to catalog the documentation for your widget... But I would imagine that link-tags could be useful there too.

p.s. The Brain, of course, allows for all sorts of Thoughts and Link properties - adding labels, colors, types, tags, etc... And even being able to run reports (queries) off of it all. Page 110 to 141 of the user manual goes into more detail.

[ShaunaGordon]

I think that graphic you include, @nixsee, speaks volumes to the goal of this more complex tagging paradigm.

Page 110 to 141 of the user manual

😮

[nixsee]

@ShaunaGordon They say a picture speaks 1000 words - my last post was 995... ;) More comics, less preaching = a better world?

[nixsee]

So I did a bunch of reading and learning today about Ted Nelson - one of the visionaries of this movement of connected notetaking tools. Because of his coinage of block-level "transclusions", one thing it led me to was Roam, which seems very much like an actually successful manifestation of his 50+ year boondoggle with his Xanadu software.

It seems that a block-level data structure (especially with a nested bullet outlining feature) inherently handles Parent/Child/Siblings by virtue of the indent levels. Roam does, in fact, seem to do this, as can be seen in their [page-level graph https://roamresearch.com/#/app/help/page/wJ02xcgZ-) - open it by clicking the graph button in the top right.

Its not quite clear to me how they are arranged (note alphabetical, doesn't even seem to match the order of links in the body. And I can't even exactly discern which ones go above the central node and which below.

Regardless, this Dagre-style graph (which we talked about on Discord) seems to appropriately handle parents/children, I assume especially if you can pass it data with such structure. Here's an example of another view (can't seem to configure it in the public mode) from a blog:

Anyway, I'd still love to be able to customize link metadata for the various reason I mentioned in previous comments. But this notion - block-level linking - might give some assistance to thinking about how to make this all work!

[brentjanderson]

I would suggest that a way to annotate a link between two pages solves note hierarchy and also affords more complex linking between notes (e.g. multiple links, each one with different semantic meaning).

The markup could look something like this:

[[some-other-page?Some context about why this link matters]]

I don't think that's a great way to do the markup, but it would work. An alternative would be:

![[some-link]] Metadata goes here adjacent to the link

The ! preceding the link on a newline would clue that this is an outbound link to the some-link page, and metadata goes adjacent to the link describing an edge.

[nixsee]

I prefer the first suggestion since ![[ ]] is already use for embedding an image in markdown and even the text of the linked note (if not in Foam - not sure - then certainly in Obsidian, Markdown Memo, and others). Also, I don't think having the metadata outside of the link would work since links are often (generally) in-line.

Whatever the specific implementation, I think it will be crucial to have it be mapped to an actual field, rather than just be descriptive text - a field could be filtered and otherwise operated upon. Perhaps even multiple tag-like fields could be included within the metadata section

e.g. [[some-link?$relationship:parent;$compatible:true;$comment:some context about the link]]

Autocompletions within this would be even better!

[brentjanderson]

I saw the conflict with ![[]] by chance after posting this.

Finding the right balance between technical rigor (multiple tag-like fields linking two notes) and ensuring the markdown is human friendly should be a first-order concern.

Dendron has an interesting approach to references which would be more readable for complex/multi-property links than trying to inline it all in one link.

[nixsee]

Agreed - simplicity/human-friendly is key. Unorthodox file naming is why I was turned off of Dendron to begin with, but looking at it again, it is pretty impressive tool... I may have to explore some more.

[nixsee]

Checked it out. Some very cool stuff, but I don't like the highly rigid hierarchies. I've submitted an issue to ask if it is possible to ditch the "."-delimited hierarchies and use the front matter for parent/child relationships, but I suspect the answer will be no.

That's another option here: don't put the link metadata in the actual links, integrate it with the front matter/link references/however Foam ends up handling links. You could just manually add

title: xyz
date: 2020/08/26
parents: abc, def
children: 123, 789

---

Content

Since it would hopefully/need to be integrated with Foam's internal link management, perhaps when you enter an in-line link to [[note123]], there could be a popup with text fields for each attribute that allows for multiple auto-completed or comma separated entries.

Here's a similar example from TickTick, my task manager.

After you've added some, it shows them all as checkboxes when you go to edit again:

Something similar could be done in Foam, except instead of Tags, it would be (or could also be) Parents, Children, Compatible True/False, or any other defined attribute. The GUI would put the relevant data into the backend database and perhaps as well into the YAML front matter. Of course, you could edit directly in the YAML, but data integrity (e.g. spelling) would be an issue, whereas a popup tool could have autocomplete/validation built-in.

[ShaunaGordon]

I want to highlight this bit:

That's another option here: don't put the link metadata in the actual links, integrate it with the front matter/link references/however Foam ends up handling links. You could just manually add

title: xyz
date: 2020/08/26
parents: abc, def
children: 123, 789
 
---

Content
Since it would hopefully/need to be integrated with Foam's internal link management, perhaps when you enter an in-line link to [[note123]], there could be a popup with text fields for each attribute that allows for multiple auto-completed or comma separated entries.

As I was reading through the comments on the inline notations, my thoughts started going to "why not the front matter?"

It seems @nixsee and I are on many of the same wavelengths. 😁

I think this opens up a vast world of possibilities for complex tagging. I think we might want to turn that front matter into an object notation of some sort, something like:

relations
  - parents: abc, def
  - children: 123, 789

This would then allow any arbitrary relationships, such as see-also or related and perhaps related is the default auto-generated relationship when creating a new note using the cmd/ctrl+click method. This opens up the ability to do things like automatically add connected note entries whenever you cmd/ctrl+click from another note, and include/create tools that automatically fill out this relations block as you add links in your note (akin to the automatic imports addons for programming languages such as Python and PHP).

Additionally, specific hierarchical pairs (ie - parent/child) could have similar auto-backlinking style generation. So if I declare that note abc has a parent of def, then Foam "just knows" that def has among its children abc and updates its children list accordingly.

We can then expand on this with configuration hooks that allow for custom hierarchy designations if we want. So the configuration could have key-value pairs where the key is the "parent" and the value is the "child" or whatever.

[nixsee]

Thanks @ShaunaGordon for adding some technical rigor to my conceptual dreaming!

[nixsee]

Here's a discussion in Markdown Links that might be starting to be relevant to this discussion on using YAML tags for link context.
tchayen/markdown-links#60

The issue started over whether to show links to uncreated files, and I was initially against it. But I now see how that could be very useful IF it is possible to indicate which files/links are uncreated. I don't think it can be done with a literally uncreated file, but if there could be a tag to indicate that a file was "unpopulated", then the "uncreated" files could be created, link autocompletion could be used to populate it, and the links to it could be styled with dots or dashes to show that it needs attention.

The same sort of graphical representation of link context could apply to parenting, link/note "coherence", or whatever other relational context might be desired.

[nixsee]

Semi-related: a sneak peak at Obsidian's upcoming Graph Filtering function. Green = Tags, Red = Uncreated Notes, Blue = Attachments. I'd love to be able to do something similar with whatever YAML tag criteria we might come up with.

I wonder if this RFC should be renamed, since parenting seems to have become a subset of a more general note/link YAML tagging tool/infrastructure?