Roadmap

[zhangtao25]

Canyon Production Coverage Collection

Current situation

During the construction of the test environment pipeline, Istanbul instrumentation is used. Manual testing and automated testing are performed, and coverage reports are collected.

Several core issues of production collection

1. Does instrumentation have a significant impact on the production environment?
2. Is the data collection range all users or a controllable range?
3. Data reporting timing and cost.
4. Is data collection legal?

Some ideas

1. Data can be pruned to only retain hit data. Map data can be obtained during the pipeline, which can reduce 90% of the reported data volume.
2. Progressive instrumentation. First, instrument some file paths (newly added files), observe the impact, and then gradually expand the scope.
3. For data reporting timing, it can be reported when the user closes the page to reduce the impact on the user. The reported js sdk is introduced through the script tag, which does not affect page rendering and can also be updated at any time.

Benefits brought

1. Production coverage data can help us better understand the code running situation, such as which code blocks have not been executed and which code blocks are frequently executed, which is very helpful for optimizing code.
2. It can help us better understand user behavior, such as which functions are frequently used by users and which functions are ignored by users, which is very helpful for product optimization.
3. Compared with burying points, coverage data is more comprehensive, more accurate, more intuitive, and more convenient for analysis.