Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirement redundancies: Approximation of explanation via invariants #681

Draft
wants to merge 11 commits into
base: dev
Choose a base branch
from
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,29 @@ public static void createInvariantResults(final String pluginName, final IIcfg<I
}
new WitnessInvariant(invResult.getInvariant()).annotate(locNode);
}

// Temporary hack: Get the invariant of the pre-error location
// There exists only one in our use case, needs to be generalized if obtaining
// invariants for all pre-error locations is desired
final var errorLocs = IcfgUtils.getErrorLocations(icfg).iterator();
if (errorLocs.hasNext())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The easiest and cleanest solution to me seems to be to write a small method (here, or perhaps even in IcfgUtils) which collects all predecessors of error locations in a Set<IcfgLocation>, and then just add them to locsForLoopLocations (just like getPotentialCycleProgramPoints above).

At that point, we could then rename the set to sth like locsForInvariants ;-)

Copy link
Author

@Paswalt Paswalt Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, this seems like the best solution. In our setting, we only have a single unique predecessor for all error locations whose invariant we utilize. That means for each requirement check, we do something with the resulting invariant of said location. There's not really any combining involved as it is obscuring the results. However, I don't think that changing the code to handle all predecessors of error locations in general would change anything for our setting, so a cleaner implementation like that can easily be done while still being in agreement to our use case, at least if I understand correctly.

Copy link
Author

@Paswalt Paswalt Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have implemented a function in IcfgUtils that returns the predecessors of all error locations in b4ff0a6

After looking more into the code, the issue is more complicated than I initially thought, however. I try to explain in a short manner:

  • In our redundancy analysis, assuming we look at N requirements, we have N error locations. While conceptually it is true that there is only one predecessor, in truth there is a predecessor for each of the N error locations. However, all of the predecessors will always have the same location invariant in our setting (we run one CEGAR per error location + the structure of our problem leads to this observation). Hence, we only need a single invariant result from any of these predecessor locations.

  • The way it is coded now, we create an invariant result for all predecessors. What happens in Pea2Boogie is the following: we have a method that takes trace abstraction results and transforms them into our desired results. (Now, if we check the redundancy of let's say a requirement called r1 with a total of N = 1000 requirements, and it turns out that r1 is indeed redundant, trace abstraction will throw 1000 location invariant results for the predecessors of the error locations into this method. However, all of them will have the same location invariant, which we only want to receive once to output it.

    public IResult convertTraceAbstractionResult(final IResult result) {
    final UnaryOperator<IResult> resultTransformer = reporter::convertTraceAbstractionResult;

  • To circumvent the issue, I added a temporary set into the VerificationResultTransformer until I can think of a better way to handle this. This way the same location invariant is not output N times but only once per requirement that is redundant. Though, the issue remains as all of the 1000 invariant results still enter the method per redundant requirement, and each time 999 are then rejected which is just a waste and not acceptable if the possibility exists to just create one for our use case. In order to think of a better solution I have the following questions:

  1. Would it be possible to have settings for the invariants one wants to create?
  2. Is it possible to check the settings of a project in a different project?
  3. Do you maybe have an idea how we could fit the need of the use case described?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I'm planning to add something like this. We should discuss your use case in more detail (preferably in person) so we can figure out how to best design such settings.
  2. We should not read settings of other projects.
  3. If I understand correctly, you start one CEGAR loop per error location, and you always want the invariant at the predecessor of that particular error location that the CEGAR loop was started for? I think this is possible as follows:
    • In FloydHoareUtils::createInvariantResults you have access to the checked specification as spec = annotation.getSpecification()
    • From this specification, you should be able to identify which error location's unreachability the CEGAR loop has proven. Just filter the error locations by spec::isFinalState. Have a look at FloydHoareUtils::getCheckedSpecifications for an example.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that worked, see 8aa2980

I left the new function in IcfgUtils still there in case it is useful in the future. But if the argument that it's non-usage at the moment makes it superfluous, I can also still remove it. This current version now perfectly matches what I wanted.

In summary, I think in the future a setting for the predecessors of the locations obtained via the spec through isFinalState would be helpful for our purpose. Thank you both so much for your help!

{
final var preLocs = errorLocs.next().getIncomingNodes();
// Hack: For reqcheck there only exists one pre-error location
final var preLoc = preLocs.isEmpty() ? null : preLocs.get(0);
if (preLoc != null)
{
final IPredicate hoare = annotation.getAnnotation(preLoc);
if (hoare == null)
{
return;
}
final Term formula = hoare.getFormula();
final var invResult =
new InvariantResult<IIcfgElement>(pluginName, preLoc, backTranslatorService, formula, checks);
reporter.accept(invResult);
}
}
Paswalt marked this conversation as resolved.
Show resolved Hide resolved
}

public static void createProcedureContractResults(final IUltimateServiceProvider services, final String pluginName,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,8 @@ public String getLongDescription() {
public Set<Check> getChecks() {
return mChecks;
}

public boolean isLoopLocation() {
Paswalt marked this conversation as resolved.
Show resolved Hide resolved
return mIsLoopLocation;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
import de.uni_freiburg.informatik.ultimate.core.lib.results.AllSpecificationsHoldResult;
import de.uni_freiburg.informatik.ultimate.core.lib.results.CounterExampleResult;
import de.uni_freiburg.informatik.ultimate.core.lib.results.IResultWithCheck;
import de.uni_freiburg.informatik.ultimate.core.lib.results.InvariantResult;
import de.uni_freiburg.informatik.ultimate.core.lib.results.PositiveResult;
import de.uni_freiburg.informatik.ultimate.core.model.models.IElement;
import de.uni_freiburg.informatik.ultimate.core.model.models.annotation.Spec;
Expand Down Expand Up @@ -93,6 +94,7 @@
import de.uni_freiburg.informatik.ultimate.logic.TermVariable;
import de.uni_freiburg.informatik.ultimate.pea2boogie.results.ReqCheck;
import de.uni_freiburg.informatik.ultimate.pea2boogie.results.ReqCheckFailResult;
import de.uni_freiburg.informatik.ultimate.pea2boogie.results.ReqCheckRedundancyResult;
import de.uni_freiburg.informatik.ultimate.pea2boogie.results.ReqCheckRtInconsistentResult;
import de.uni_freiburg.informatik.ultimate.pea2boogie.results.ReqCheckSuccessResult;
import de.uni_freiburg.informatik.ultimate.pea2boogie.translator.Req2BoogieTranslator;
Expand Down Expand Up @@ -126,6 +128,7 @@ public VerificationResultTransformer(final ILogger logger, final IUltimateServic
public IResult convertTraceAbstractionResult(final IResult result) {
final AbstractResultAtElement<?> oldRes;
final ReqCheck reqCheck;
InvariantResult<?> invResult = null;
boolean isPositive;
if (result instanceof CounterExampleResult<?, ?, ?>) {
oldRes = (AbstractResultAtElement<?>) result;
Expand All @@ -138,6 +141,25 @@ public IResult convertTraceAbstractionResult(final IResult result) {
} else if (result instanceof AllSpecificationsHoldResult) {
// makes no sense in our context, suppress it
return null;
} else if (result instanceof InvariantResult) {
invResult = (InvariantResult<?>) result;
// Only want to process location invariants
if (invResult.isLoopLocation()) {
return result;
}
oldRes = (AbstractResultAtElement<?>) result;
final var check = invResult.getChecks().stream().filter(c -> c instanceof ReqCheck).findFirst();
if (check.isEmpty()) {
return null;
}
reqCheck = (ReqCheck) check.get();
// Important if other specs receive invariants too,
// in order to prevent the results to slip through to
// the last return unhandled
if (!reqCheck.getSpec().contains(Spec.REDUNDANCY)) {
return result;
}
isPositive = true;
} else {
return result;
}
Expand Down Expand Up @@ -187,6 +209,21 @@ public IResult convertTraceAbstractionResult(final IResult result) {
final String failurePath = formatTimeSequenceMap(delta2var2value);
return new ReqCheckRtInconsistentResult<>(element, plugin, failurePath);
}

// If a loop invariant result is present, transform it into the
// new output, rather than a generic ReqCheckFailResult
if (spec == Spec.REDUNDANCY && invResult != null) {
// Annotation needed for the result to know the respective Check
reqCheck.annotate(element);
StringBuilder resultString = new StringBuilder("");
final String invariant = invResult.getInvariant();
final Set<String> redundancySet = ReqCheckRedundancyResult.extractRedundancySet(invariant);
// TODO: Apply post-processing to the redundancy set here
resultString.append(redundancySet.toString());
resultString.append(" derived by location invariant: ");
resultString.append(invariant);
return new ReqCheckRedundancyResult<>(element, plugin, reqCheck.getReqIds().iterator().next(), resultString.toString());
}
return new ReqCheckFailResult<>(element, plugin);
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package de.uni_freiburg.informatik.ultimate.pea2boogie.results;

import java.util.HashSet;
import java.util.Set;
import java.util.regex.Pattern;
import java.util.regex.Matcher;

import de.uni_freiburg.informatik.ultimate.core.model.models.IElement;

public class ReqCheckRedundancyResult<LOC extends IElement> extends ReqCheckFailResult<LOC> {

private String mReqName;
private String mRedundancySet;

public ReqCheckRedundancyResult(LOC element, String plugin, String reqName, String redundancySet) {
super(element, plugin);
mReqName = reqName;
mRedundancySet = redundancySet;
}

@Override
public String getLongDescription() {
return "Extracted redundancy set for " + mReqName + ": " + mRedundancySet;
}

/*
* Function that parses loop invariants represented by strings. In the terms of the regex,
* we extract every name of the automata appearing inside of the loop invariant.
* TODO: One issue that persist is if a user names the observable of a requirement
* such that it matches the regex (for example: "input FakeReq_ct0_total_pc IS bool").
* For now the method works under the assumption that names created during the formalization process are
* disjoint of the variable names used in the program that simulates the intersection of automata
*/
public static Set<String> extractRedundancySet(String invariant) {
String regex = "[a-zA-Z_]+[a-zA-Z0-9_]*_ct(0|[1-9][0-9]*)[a-zA-Z0-9_]*_total";
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(invariant);

Set<String> redundancySet = new HashSet<>();
Paswalt marked this conversation as resolved.
Show resolved Hide resolved

while (matcher.find()) {
redundancySet.add(matcher.group().split("_ct")[0]);
}

return redundancySet;
}

}