[Suggestion]: Separate Macros and Attributes

[vhyrro]

This change is very large in scope, I encourage reading carefully.

In the existing spec, we have two carryover tag types. They serve two different purposes - for instance, #color would apply the colour to the existing object and to all children (e.g. subheadings), whereas +color would only apply a colour to the existing object and not to children.

This syntax is unfortunately very confusing and inconsistent across various objects. Does "children" include all paragraphs inside a heading, or does that only include subheadings? How do you apply a style only to a title? Or only to the content, but not the title?

This proposal alleviates all these issues with minimal breakages. In doing so, it refactors the following elements:

• Carryover Tags (minor change, big implications)
• Task syntax (minor change, big implications)
• Inline Link Targets (removal of the syntax, minor implications)

Attributes vs Macros

There has been some debate about the core internals of Norg's macro system and some concerns have been raised. Specifically, this concern revolves around abstract objects (AOs). An AO is an unrepresentable state in the Norg syntax. An example is the code block macro - it can't evaluate to any physical Norg markup, so it evaluates to an AO instead.

This behaviour is fine, but how do you then evaluate an AO properly? Take for instance #color - it can't return any Norg markup that physically colours the text, but it can return an AO which abstractly says "this will be coloured". To alleviate this, every macro can have an export function that allows it to implement custom export behaviour per language. Even then, how do you represent such a colour? If someone exports their Norg document to HTML, how do you colour the heading? That's the job of CSS, not HTML.

For this reason, we propose to break the existing implementation into two groups. On the one hand you have macros, which do not change in any way as part of this proposal. On the other, we would like to add a new group of objects called attributes as part of this proposal.

Attributes

An attribute is an abstract property of an object - its name, its colour, its styles. The +this syntax is used to apply an attribute. Attributes don't do anything, they just specify some data about an object.

Example:

+color.title red
* Hello

Macros

Contrary to attributes, macros are functions. Macros are invoked using #this syntax. That means they consume the object and potentially produce completely new (and optionally unrelated) markup, or they produce an AO.

This is fundamentally different from an attribute, which acts as a marker on some object.

Generalization of Attributes

#37 has one fatal flaw - it's not possible to apply a macro (and now by extension an attribute) on an item in a list (only on its content, which may be undesirable if one would like to change the colour of the list marker (-)). To alleviate this, the task syntax can be generalized to not just tasks, but any attribute. This also makes tasks themselves attributes, which provides an unprecedented level of generalization throughout the format.

Therefore, to apply an attribute to a list item, one would use the following syntax:

- List item 1
- (color red) Hello, world!
-- Since this is a child of the list item, this is also red.

Contrary to this example, which would only colour the paragraph Hello, world!

- List item 1
-  +color red
   Hello, world!
-- This is no longer red.

Tasks are now also attributes, but this doesn't change the way we use them. Attribute names also permit various punctuation characters, so examples like - (=) On hold still work just fine.

Undone Task

For compatibility's sake, special meaning is applied to undone items. An empty attribute list means an "undone item": ( ).
Additionally, extraneous delimiting markers can also implicitly describe undone tasks: ( |+ Tue 5th March) or (+ Tue 5th March|) (space is optional).

Generalization of Macros

We now know that ( ) applies attributes. How does one run a macro on an individual list item, then? This is where the last part of the proposal comes into play: remove inline link targets (and find a different way of implementing them) and make < > the counterpart of ( ).

Thus, the following:

- <my_macro> Hello, world!

Would execute my_macro and pass the whole list item (not just the paragraph) as parameters to the macro.
Similarly, attached modifier extensions can be renamed to attached modifier attributes and a < > counterpart can be provided: {? heading title}<if task>. On top of all that, a standalone <> marker would supercede the &variable& syntax (now used for super verbatim, see #33) to invoke a macro in-place. This gives the added benefit of being able to provide parameters to the macro, for instance inline images! Example: Here's a photo: <image my-image.png>.

Conclusion

All in all, splitting macros into macros + attributes is a great combination that permits many positive changes to the specification.

[vhyrro]

Potential issues and workarounds:

• We may want to add a custom char for undone other than whitespace. For example: *. That way - ( ) and - (*) are equivalent, but the latter may be used for disambiguation purposes.
• Current syntax doesn't allow placing an inline image directly in a list: - <image my_image.png> will consume the list item entirely. This isn't much of an issue though, because an LSP will be able to recognize that more parameters are being passed to image than expected, and will be able to issue a warning. Besides, the new extended list syntax proposal (not written yet) will allow for this use case anyway.

[boltlessengineer]

I like the new inline macro syntax.

But I don’t get the point of passing whole list item to the macro. Why would one want to pass <li>…</li> to a macro instead of just …? If they are going to change that list item to a completely different object, they are just breaking the list so they can use other macro types instead (e.g. carryover tags or standard ranged tags.)

- list item
- <macro> this macro is consuming whole 2nd list item
- thus it is unclear if this is still in same list

If they want to pass some ranged content to a macro in mid of the list, I think allowing ranged tags as list item content will solve that use-case.

- list item
- @macro
  content
  @end
- list continues

I think not making different behavior for detached modifiers will also resolve the 2nd issue. If we have that “detached modifiers macros” conflicting with inline macros, what was the point of choosing () instead of [] for detached modifier extensions?

[vhyrro]

For the same reason you may want to alter the attributes of a list item (not just the paragraph), you may also want to run a macro on a whole list item. This argument could extend to anything: why run a macro on a subheading, since that breaks the structure? Any macro invocation can break the structure, so embracing it in a consistent manner makes sense here :)

[boltlessengineer]

For most cases, it is true, but we are talking about lists, and there is imaginary object “list” wrapping all those detached modifiers.
If macro consumes the whole list item and not its content, we would end up creating something like this:

- list item
- <macro> content
- list continues

<ul>
  <li>list item</li>
  <macro>
    <li>content</li>
  </macro>
  <li>list continues</li>
</ul>

Now unordered_list doesn’t only have unordered_list_item but it also contains macro in root level.
So passing list item to a macro will only end up breaking the list. But in syntax, it is unclear if the list is broken or not.

If they actually want to break the list, they can use strong carryover tag,
If they don’t want to break the list, they should use attributes instead.

I think there will be some use-cases where user do want to modify the individual list item while not breaking the list (e.g. when they want to shorten/automate the long attributes.) If so, wouldn’t it better to allow using macro inside the attribute list (( ) part) by adding some prefix instead of passing whole list item? Because there won’t be any situation where they want to alter individual list item to different object.

- list item
- (:macro) content
- list continues

[mrossinek]

I think I agree with everything in the description above 👍

Just some comments below.

---

Mostly a clarification: you speak about tasks in the text but I assume you meant all detached modifier attributes.
Please correct me if I'm wrong.

---

You had the following example:

+color.title red
* Hello

Why is it color.title? Is that to indicate that you only apply color to the title and not also the modifier?
If so, that would make it equivalent to:

* (color red) Hello

Correct?

---

We may want to add a custom char for undone other than whitespace. For example: . That way - ( ) and - () are equivalent, but the latter may be used for disambiguation purposes.

I see the intent but so far we have always tried to avoid duplicate means to achieve the same thing. And I would really not want to require a character for an undone task.

---

Replying to boltless:

But I don’t get the point of passing whole list item to the macro.

It's exactly the example given in the description:

it's not possible to apply a macro (and now by extension an attribute) on an item in a list (only on its content, which may be undesirable if one would like to change the colour of the list marker (-))

Coloring the list item marker to the same color as the content is often done in WYSIWYG editors and we even do that in our syntax highlighting for headings. I think having the ability to do so is quite useful.
And that is just one simple example.

---

EDIT: one more: in your examples you still used the | separator for attributes. With #39 that will change accordingly, correct?

[mrossinek]

I just wrote #37 (comment).
That made me realize:

How exactly do you classify + and # now?
To my current understanding we have:

• + is an attribute and () is an inline attribute
  • where can () be placed? Does it have to be "attached" to an attached modifier (think attached modifier attributes) or a detached modifier (like seen here) or can it be arbitrary (attached to a word?)
• # is a macro invocation and <> is its inline version
  • <> has no placement restrictions outside of titles, right? (just like we do not allow links inside titles)
• = is a macro definition
• | and @ remain unchanged and are stilled referred to as tags, right? (with @ being verbatim)

[boltlessengineer]

can’t remember if I’ve said this idea before, but can we replace the + to #(…) syntax?
As we find a good way to write multi-line attributes with parentheses, I think it would be better to make less prefix token by giving #(…) special meaning.

+color red
+foo bar
paragraph

will be

#(color red; foo bar)
paragraph

or

#(color red;
  foo bar)
paragraph

This allows us to more clearly distinguish between attribute syntax and carryover tag (macro) syntax, which now have different functions and purposes.

before:

• two different ways to attach attributes by context
• tags are generally a macro, but weak carryover tag isn’t. It is an attribute and not a macro.

after:

• two similar ways to attach attributes
• we don’t have “weak carryover tag”. Tag is always a macro, and attribute is completely different syntax by its own.

---

btw null-list-item idea already has this feature without creating any new special grammar:

/ (color red; foo bar) paragraph

/ (color red
  ;foo bar
  ) @table
    @end

is a paragraph with attributes. ;)

[boltlessengineer]

It's exactly the example given in the description:

it's not possible to apply a macro (and now by extension an attribute) on an item in a list (only on its content, which may be undesirable if one would like to change the colour of the list marker (-))

@mrossinek That still doesn't count as a use-case of applying macro to a list item.
Think about how you would do that in HTML. It will be <li style="..."> and not <style-that><li>....

Styling is job of attributes, not macros. Macros are used to change the given object completely. It will most likely change the type of original object.