Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix source-location handling of user comments to be language independent #309

Merged
merged 19 commits into from
Sep 10, 2024

Conversation

achidlow
Copy link
Contributor

@achidlow achidlow commented Sep 10, 2024

SourceLocation now has a comment_lines field to indicate how many lines to fetch as user comments when annotating output source files.

This provides a language independent way of handling source locations that can optionally include comments, whereas the current approach was to strip them out when necessary, assuming # to be the comment delimiter.

Also line numbers for code that has no source data are no longer output - this mostly means the embedded lib functions, for which the line numbers were not particularly useful to begin with.

Merging of source locations when operations are combined in optimisation has also been improved, to only merge locations when they are overlapping or adjacent, thus avoiding including unrelated code in annotations.

Specifically source locations with self. members has also been improved to use the location of the usage, rather than declaration, this previously resulted in extremely large and unrelated annotations in certain situations, see for examples the changes in this PR to VotingRoundApp.approval.teal.

Non-user facing changes:

  • Improved SourceLocation validation, including making end_line non-optional (the existing behaviour of defaulting to line is retained, but is made explicit on the object).
  • Fixed a bug with MIR output alignment caused by virtual ops, which made the column heading misleading.
  • Fixed a bug with column locations when merging source locations.
  • Fixed a bug where column locations were retained when including source comments.
  • Fixed multiple places using user-comment locations when they should have been pointing at code only (e.g. IR labels, log output, ...)
  • Validate rather than resolve SourceLocation paths, to prevent inconsistencies on Windows CI/CD and also to guard against accidental relative path inclusion especially when deserializing AWST

Copy link

github-actions bot commented Sep 10, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/puya_lib
   arc4.py1261260%1–332
   bytes.py990%1–15
   util.py16160%1–33
src/puya
   __main__.py30300%1–42
   arc32.py68297%77, 103
   artifact_sorter.py53198%84
   compile.py147895%89–90, 132–135, 155–158, 288, 309
   context.py31197%35
   errors.py35877%40–42, 44–46, 50–51
   log.py1994179%32–35, 69, 89, 108, 123–124, 166–168, 171–173, 175, 188–197, 219, 288–289, 298, 320–322, 335–349
   main.py35350%1–47
   parse.py87594%31, 40, 50, 56, 87
   template.py44686%18–19, 41–42, 58, 60
   utils.py1991891%50, 67–68, 77–78, 86–87, 171, 177, 195–197, 204, 226, 249, 251, 274, 309
src/puya/awst
   function_traverser.py285399%76, 367, 373
   nodes.py9924496%99, 103–106, 146, 150–153, 351, 537, 553, 587, 628, 657–658, 708, 736–737, 886, 908, 940, 968, 976, 981, 1136, 1221, 1237, 1286, 1359, 1430, 1484, 1488, 1734, 1745, 1747, 1752, 1760, 1765, 1770, 1779, 1784, 1789
   serialize.py57198%20
   to_code_visitor.py439798%131, 315, 365, 621, 646, 674, 678
   txn_fields.py98199%48
   wtypes.py2941794%174–178, 188–191, 208, 226, 250, 305, 319, 353, 424, 469
src/puya/awst/validation
   base_invoker.py47491%55, 62, 69–73
   inner_transactions.py181199%160
   labels.py30873%25–27, 32, 36–41
   scratch_slots.py39490%19, 33, 49, 51
src/puya/ir
   arc4_router.py3122094%110, 173, 229, 307, 355, 424, 433, 491–492, 504, 509, 514, 519, 524, 529, 549–553, 737, 762, 799
   avm_ops.py315199%46
   avm_ops_models.py48394%21, 30, 38
   context.py107893%78, 85, 88, 90, 128–129, 141, 147
   main.py2991296%74–80, 90–96, 122, 163, 221, 565
   models.py5292296%74, 182, 189, 338, 408–409, 414, 420–424, 437, 481, 510, 566, 612, 692, 708, 749, 752, 759, 762, 851–852
   ssa.py130398%51–52, 150
   to_text_visitor.py152895%123, 128, 212, 219–224
   types_.py1011090%50, 57, 91–95, 116, 152, 157–159
   visitor.py1291787%145, 193, 205, 220, 223, 236, 239, 245, 248, 259, 262, 265, 268, 271, 274, 277, 280
   visitor_mutator.py105298%174–175
   vla.py72199%87
src/puya/ir/builder
   _utils.py58395%175–177
   arc4.py5042595%93, 129–134, 375, 400, 403, 461–464, 719–720, 966, 990, 1071, 1078, 1114, 1162, 1172, 1227, 1272, 1291, 1311, 1372–1379
   assignment.py83693%51, 98, 112, 188, 206, 222
   blocks.py140795%55, 92–96, 158, 166, 231
   bytes.py641478%13–45, 129
   callsub.py76692%30–31, 51, 101–102, 123
   flow_control.py95199%56
   iteration.py198597%91–92, 107, 144, 206
   itxn.py2761993%124–125, 127, 141, 189, 210, 235–236, 565, 589–590, 603–612
   main.py5456588%106, 247, 251, 256–274, 279–297, 352, 376, 400–401, 439, 627, 649, 662–663, 719, 744, 781–783, 794, 859, 937, 950, 983, 1038, 1041, 1049, 1052, 1060, 1089, 1100, 1109, 1188, 1205, 1248–1258
   storage.py83298%101, 154
src/puya/ir/destructure
   coalesce_locals.py1011981%119, 128–129, 132–135, 138–147, 163–166
   parcopy.py84298%47, 83
src/puya/ir/optimize
   _utils.py14286%17–19
   collapse_blocks.py92595%65–69
   compiled_reference.py79594%53, 87, 158–163
   control_op_simplification.py1041586%47–48, 92–121, 171, 245–252
   inner_txn.py36197%38
   intrinsic_simplification.py4833393%76, 166–168, 173, 255, 267, 302, 313–314, 338–339, 363, 430, 568, 596, 617, 619, 634, 674, 697, 727, 733, 735, 737, 742, 744, 746, 748, 750, 802–803, 810
   main.py87298%116–117
src/puya/ir/validation
   _base.py29197%24
   compile_reference_validator.py20385%24, 30, 37
   min_avm_version_validator.py11191%15
   op_run_mode_validator.py19574%19–29
src/puya/mir
   annotaters.py159299%62, 212
   builder.py1401093%139, 248–249, 302, 305, 308, 311, 314, 317, 320
   models.py3281695%40, 52, 68, 77, 88, 99, 110, 150, 260, 291, 328, 340, 365–371
   output.py63297%29, 33
   stack.py2501992%72, 119, 135, 146, 157, 167, 184, 201, 211, 230, 250, 256, 258, 297, 310, 334, 366, 371, 409
src/puya/mir/stack_allocation
   baileys.py205399%31, 330–334
   frame_allocation.py80298%21, 68
   koopmans.py61198%55
   peephole.py100694%75, 89, 99, 101, 103, 145
src/puya/teal/optimize
   peephole.py112298%139, 145
   repeated_rotations.py42198%13
   repeated_rotations_search.py88693%34, 40–41, 57, 67–68
src/puya/ussemble
   build.py731086%37, 52–53, 58–61, 65–68, 85–86, 115
   op_spec_models.py22195%20
   optimize.py76199%77
   output.py115199%71
   validate.py17194%19
src/puyapy
   __main__.py37197%153
   client_gen.py1151587%55–56, 65–69, 73, 165–166, 174–175, 190, 194–195, 204
   compile.py1512285%59–68, 81, 162–163, 174, 181–182, 199–209, 219–221, 226, 242
   parse.py1791393%49–50, 86, 154–159, 163, 261, 265–266, 313, 325, 350
   utils.py21576%16–17, 25–28
src/puyapy/awst_build
   arc4_client.py701973%45, 54, 60, 63, 66, 72, 75, 78–83, 86, 89, 92, 95, 98, 101, 104, 107, 110
   arc4_utils.py3244885%45, 48–49, 51, 92, 104–106, 133, 144–148, 164, 166, 169–170, 193, 234–236, 245, 250, 255–256, 280, 284, 296, 303, 305, 319–320, 323–327, 332–333, 339–342, 350, 368, 380, 388–389, 401, 409, 510, 534
   arc32_client_gen.py68297%34, 130
   base_mypy_visitor.py1273969%70–76, 94, 102–115, 129, 131, 133, 145, 150, 154, 157, 160, 166, 188, 191, 194, 201, 205, 208, 211, 215, 233, 237, 241, 245, 249, 253, 257, 261, 265, 269, 273, 277, 281
   context.py2364780%66, 96, 99, 109–110, 128–129, 169, 229, 234, 240–244, 249, 258, 260, 263–265, 267, 274, 276, 287–288, 293–295, 298, 318, 342–343, 355, 369, 372–384
   contract.py2382789%182, 224–225, 227–230, 242, 254, 345, 349, 371, 374, 386, 394, 397, 400, 403, 406, 409, 412, 415, 418, 421, 435, 458–464
   contract_data.py58198%37
   intrinsic_factory.py32391%47, 56, 65
   intrinsic_models.py40198%49
   main.py52198%34
   module.py4025786%115, 129–131, 146–147, 154, 163–164, 172–176, 194–198, 207, 245–246, 257, 279–282, 292–294, 300, 317–320, 333, 367, 374, 396–397, 420–425, 479–480, 508, 519, 522, 528, 534, 544, 550, 553, 565, 568, 590, 610, 615, 619, 623–626, 651, 694, 702, 704
   pytypes.py4885289%85–87, 101–102, 142, 158–164, 187, 207, 282, 291, 310, 329, 333, 349–350, 382, 475–477, 491–492, 561–562, 692, 703–704, 749–750, 755, 804–805, 826–827, 956–957, 981, 1009, 1043–1045, 1071, 1098, 1108–1109, 1145–1147
   subroutine.py6505592%139, 266, 272, 330–333, 339, 391, 398, 401–407, 472, 633, 635–636, 654, 656, 666–667, 676–677, 681, 702, 775, 782, 802–803, 885, 916–917, 934, 956, 962, 1050, 1058, 1069, 1094, 1199–1200, 1221, 1231, 1238, 1247, 1263, 1273–1274, 1282, 1301, 1304, 1307, 1310, 1313, 1333–1335
   utils.py1852487%35, 52–56, 74, 109–110, 112, 156–157, 208, 216, 221, 234–238, 243–246, 288, 296, 339
src/puyapy/awst_build/eb
   _base.py1281985%52, 57–59, 64, 71, 76, 81–83, 142, 153, 177, 182, 187, 192, 203, 218, 223–225
   _bytes_backed.py48296%30–31
   _expect.py1251886%26, 37, 78–81, 93–96, 99, 158–159, 220, 233–236
   _literals.py1382979%43, 72, 91, 120, 128, 142, 146, 150–156, 166–180, 185
   utils.py44198%96
   array.py26965%23, 28–33, 43, 49
   biguint.py96694%56, 98, 135, 150–151, 153
   bool.py50884%37–41, 57, 68, 83
   bytes.py1691889%102–103, 130–131, 136–137, 143–144, 147, 155, 198, 233, 265, 269, 286–287, 302–303
   compiled.py1121388%56, 61, 67–68, 165–171, 206–210, 235
   conditional_literal.py1263572%94, 98, 152, 156–159, 168–170, 179–182, 193–196, 205, 209, 213–216, 231–243, 252–253
   contracts.py83693%58, 64, 76, 101, 122, 126
   dict.py27581%25, 33–35, 39
   ensure_budget.py31197%47
   interface.py84396%302–304, 308
   intrinsics.py97694%43, 62, 69, 82, 89, 160
   log.py42490%45–46, 51, 60
   logicsig.py15193%26
   none.py27196%38
   string.py1451391%72, 115–116, 135, 139, 182, 189, 193, 205, 279–281, 301
   struct.py16569%14–16, 25, 31
   subroutine.py811779%46, 50–53, 68, 71–78, 93, 101–105, 107–110, 115
   template_variables.py37295%30, 58
   tuple.py2851395%76, 148, 154, 189–190, 196, 273–274, 393, 421, 432–433, 496
   uint64.py110595%57, 118–119, 167–168
   uint64_enums.py40295%41, 46
   unsigned_builtins.py1522186%73, 80, 104, 128, 132, 136, 140, 148, 152, 156, 160, 164, 174, 178, 184, 195, 201, 240, 272, 284, 296
src/puyapy/awst_build/eb/arc4
   _base.py91397%186–189, 200
   _utils.py1311092%78–79, 106, 111, 124, 164–167, 207, 211, 225
   abi_call.py3212094%122, 128, 132, 156, 226, 243–244, 325, 342, 400, 428, 458, 481–482, 540, 577, 649, 734–735, 752
   address.py77396%57, 117–118
   bool.py57395%44, 86–87
   dynamic_array.py1271092%57, 126–127, 147, 149, 154, 228, 249, 255–258
   dynamic_bytes.py68396%97–99
   emit.py36197%38
   static_array.py66198%45
   string.py100793%54–55, 103, 126, 131–134
   struct.py49198%49
   tuple.py941584%50–52, 92–95, 98–99, 135–138, 143, 147–148, 158, 168
   ufixed.py70297%43, 102
src/puyapy/awst_build/eb/reference_types
   account.py81298%65, 176
   application.py45198%40
   asset.py65198%48
src/puyapy/awst_build/eb/storage
   _common.py69396%107, 122–123
   _storage.py1001783%76, 84, 88, 92, 96, 100, 104, 108, 112, 122, 126, 130, 134, 140, 151, 157, 169
   _value_proxy.py55787%38, 42, 50, 54, 91, 99, 103
   box_map.py143199%186
   global_state.py138596%108–109, 115, 171–172
   local_state.py1481193%103–104, 108, 157, 161, 165, 175, 179, 203, 290, 314
src/puyapy/awst_build/eb/transaction
   base.py39295%23, 43
   group.py58198%51
   inner.py48296%88–89
   inner_params.py81594%69, 79, 83, 143, 145
   itxn_args.py60198%72
TOTAL21464166792% 

Tests Skipped Failures Errors Time
1263 3 💤 0 ❌ 0 🔥 6m 54s ⏱️

@achidlow achidlow merged commit 1d7b06f into main Sep 10, 2024
4 checks passed
@achidlow achidlow deleted the comment-locations branch September 10, 2024 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants