Skip to content

Latest commit

 

History

History
97 lines (74 loc) · 3.25 KB

README.md

File metadata and controls

97 lines (74 loc) · 3.25 KB

Injection Evaluation

The 2 csv files in this repository contain the evaluation of an automated injection with and without a specification. The following columns can be found:

  1. ErrorType

    Can be one of the following: Typo Error, Domain Error, Structure Error, Semantic Error, Limit Error, Resource Error

  2. InjectionType

    Metadata Text
    inject/structure/section/duplicate Section Duplication Error
    inject/structure/section/reallocate Section Reallocation Error
    inject/structure/section/remove Section Removal Error
    inject/semantic Semantic Error
    inject/resource Resource Error
    inject/typo/transposition Typo Char Transition Error
    inject/typo/insertion Typo Char Insertion Error
    inject/typo/deletion Typo Char Deletion Error
    inject/typo/change/char Typo Char Changing Error
    inject/typo/space Typo Space Insertion Error
    inject/typo/case/toggle Typo Case Toggle Error
    inject/domain Domain Error
    inject/limit/max Max Off-by-one Error
    inject/limit/min Min Off-by-one Error

  3. Key

    The Key which is affected by the injection

  4. Old Value

    The previous value or (null) if it was empty

  5. New Value

    The new value

  6. Error Message

    A filtered message which tries to detect errors (e.g. [ERROR] for java logs) for the application

  7. SpecMessage

    The reason metadata which was emitted by Elektra

  8. SpecCaught

    x if Elektra caught it, otherwise empty

  9. SpecPlugin

    Which specification plugin caught the error

  10. Log Message

    The full log information from the application. In some cases it was manually deleted because of checks done manually

  11. Reaction App

    One of the following reactions can be possible for the application:

    1. GOOD: The exception points to the injected error in some way
    2. NONE: There is no exception thrown despite it should be
    3. BAD: An exception is thrown but is misleading
    4. DEFAULT: No exception is thrown but also not necessary because the application handles the case correctly

  12. Reaction Spec

    The same as Reaction App but for Elektra

  13. Pinpoint Spec

    1. GOOD: Key, Value and Reason(Context is given)
    2. NONE: Only if the reaction itself is NONE or BAD
    3. BAD: The exception does not yield relevant information for the location
    4. DEFAULT: If DEFAULT was the reaction
    5. PARTLY: If either Key or Value was given for the wrong configuration setting

  14. Vocab Spec

    1. GOOD: If no advanced vocabulary is used
    2. BAD: If too advanced words are being used or confusing text appears (e.g., stat file, normalize value, classnames unrelated to the exception in java, etc.)

  15. Pinpoint App

    Same as Pinpoint Spec just for the application

  16. Vocab App

    Same as Vocab Spec just for the application

  17. Personify App

    If the error message is apologetic and personified (e.g., "Sorry, I could not find the XY)

  18. LeakInt App

    If internas of the application are leaked such as method names, POJO structures, etc.

  19. Comment

    Sometimes comments are done such as bug indication or additional text to interesting cases