Tags backend raises error "node_text_content(document) should be unreachable"

[james-ball-qualcomm]

Somehow in RISC-V-Certification-Steering-Committee/riscv-isa-manual#11 the tags added to the ISA manual are causing node_text_context to be called with a "document" node name and Tim's tags.rb isn't expecting this. So, it raises an exception. I'm going to change it to a warning for now to make progress but Tim, can you help?

  # Return the text content of a node. Adapted from `text-converter.rb`
  # in the docs: https://docs.asciidoctor.org/asciidoctor/latest/convert/custom/
  #
  # node: AbstractNode
  # returns: String
  def node_text_content(node)
    content_or_nil = case node.node_name
    when "document"
      warn "node_text_content(document) should be unreachable"
    when "section"
      "\n" + [node.title, node.content].join("\n").rstrip()
    when "paragraph"
      "\n" + normalize_space(node.content)

[james-ball-qualcomm]

It is happening once in Priv ISA manual and a bunch of times in Unpriv ISA manual:

Priv ISA:
node_text_content(document) should be unreachable

Unpriv ISA:
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable
node_text_content(document) should be unreachable

[james-ball-qualcomm]

@Timmmm, is there anything we can print out to isolate the problem?

[james-ball-qualcomm]

I don't know if there's a clue here but after the make build-norm-rules creates the tag JSON file using the tags backend, my script to create normative rules has several issues:

create_normative_rules.rb: Passed command-line: -j -t /build/riscv-privileged-norm-tags.json -t /build/riscv-unprivileged-norm-tags.json -d normative_rule_defs/c-st-ext.yaml -d normative_rule_defs/hypervisor.yaml -d normative_rule_defs/m-st-ext.yaml -d normative_rule_defs/machine.yaml -d normative_rule_defs/priv-csrs.yaml -d normative_rule_defs/rv32.yaml -d normative_rule_defs/rv64.yaml -d normative_rule_defs/zc.yaml -d normative_rule_defs/zilsd.yaml build/norm-rules.json
create_normative_rules.rb: Normative rule definition filenames = ["normative_rule_defs/c-st-ext.yaml", "normative_rule_defs/hypervisor.yaml", "normative_rule_defs/m-st-ext.yaml", "normative_rule_defs/machine.yaml", "normative_rule_defs/priv-csrs.yaml", "normative_rule_defs/rv32.yaml", "normative_rule_defs/rv64.yaml", "normative_rule_defs/zc.yaml", "normative_rule_defs/zilsd.yaml"]
create_normative_rules.rb: Normative tag filenames = ["/build/riscv-privileged-norm-tags.json", "/build/riscv-unprivileged-norm-tags.json"]
create_normative_rules.rb: Output filename = build/norm-rules.json
create_normative_rules.rb: Output format = json
create_normative_rules.rb: Loading definition file normative_rule_defs/c-st-ext.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/hypervisor.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/m-st-ext.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/machine.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/priv-csrs.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/rv32.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/rv64.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/zc.yaml
create_normative_rules.rb: Loading definition file normative_rule_defs/zilsd.yaml
create_normative_rules.rb: Loading tag file /build/riscv-privileged-norm-tags.json
create_normative_rules.rb: Loading tag file /build/riscv-unprivileged-norm-tags.json
create_normative_rules.rb: Normative rule hsxl_param defined in file normative_rule_defs/hypervisor.yaml references non-existent tag norm:hsxl_param
create_normative_rules.rb: Normative rule vsxl_param defined in file normative_rule_defs/hypervisor.yaml references non-existent tag norm:vsxl_param
create_normative_rules.rb: Tag norm:hsxlen_param not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:vsxlen_param not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vs_nonhighctr_hcounteren0_mcounteren1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vs32_highctr_hcounteren0_mcounteren1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_nonhighctr_hcounteren0_scounteren0_mcounteren1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu32_highctr_hcounteren0_scounteren0_mcounteren1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_vs_hinst not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_vs_nonhigh_allowedhs_tvm0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_vs32_high_allowedhs_tvm0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_wfi_tw0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_sret_sfence not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu_nonhigh_supervisor_allowedhs_tvm0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vu32_high_supervisor_allowedhs_tvm0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_wfi_vtw1_tw0 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vs_sret_vtsr1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:H_virtinst_vs_sfence_sinval_satp_vtvm1 not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lui_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:auipc_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:jal_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:jalr_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:beq_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:bne_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:blt_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:bge_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:bltu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:bgeu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lb_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lh_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lbu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lhu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sb_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sh_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:addi_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:slti_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sltiu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:xori_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:ori_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:andi_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:slli_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:srli_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:srai_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:add_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sub_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sll_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:slt_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sltu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:xor_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:srl_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sra_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:or_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:and_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:fence_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:fence-tso_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:pause_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:ecall_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:ebreak_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:lwu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:ld_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sd_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:addiw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:slliw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:srliw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sraiw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:addw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:subw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sllw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:srlw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:sraw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:mul_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:mulh_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:mulhsu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:mulhu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:div_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:divu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:rem_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:remu_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:mulw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:divw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:divuw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:remw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: Tag norm:remuw_enc not referenced by any normative rule. Did you forget to define a normative rule?
create_normative_rules.rb: ERROR: 2 references to non-existing tags

[Timmmm]

Hmm it seems to be because table cells are actually documents (or they can be). Should be easy to fix.

I'm not sure it's anything to do with that h branch - seems like it is failing on main for me. The first offending cell in the privileged spec is Identifies the type of control flow transfer in smcntr.adoc...

So maybe it never worked? Is it tested in CI?

I'll make a fix in the morning anyway. Basically instead of that error you can just return node.content.strip.

[james-ball-qualcomm]

Thanks Tim. My intention was that the test.adoc would be used for CI but haven't had the bandwidth to figure out how to turn it on (probably as a pre-commit check). If you are up-to-speed on that and could create an issue and PR to fix it that would be great.

Anyways, I believe I have tags in tables in my test.adoc in this repo but that passes so I must not be triggering the failure.

[james-ball-qualcomm]

Tim, when you say "main", which repo are you talking about? I can't get any failure in the CSC fork of the ISA manual "main". Also, I don't see any norm: tags in the smcntr.adoc file you are talking about. Is the failure just when your tags backend runs on this file? I tried to add a test case to test.adoc that has the same table cell contents from smcntr.adoc that you identified ("Identifies the type of control flow transfer") but don't see any failure.

[james-ball-qualcomm]

Just figured it out. I need to put "a|" for my table cell to make it a whole document. Now I have a reproducible test case in test.adoc and will try your fix (need it fixed ASAP to not hold up progress).