Skip to content

Adding Memory Regions to IR #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 73 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
5bd765a
Injecting memory regions into IR
yousifpatti May 1, 2024
e878a76
Loop handling in MRA (using depth for now)
yousifpatti May 17, 2024
d23172d
Initial Works on VSA
yousifpatti Jun 17, 2024
c0be95b
Changes to partial region accesses
yousifpatti Jun 26, 2024
55f9bd8
Adding regions into IR
yousifpatti Aug 13, 2024
834f34e
Merging only region injection files
yousifpatti Aug 13, 2024
f607fdf
Fixed Small Bugs
yousifpatti Aug 13, 2024
5a20e37
Dont ignore regions
yousifpatti Aug 13, 2024
f798374
Changes to consider local variables and initial offsets of a procedure
yousifpatti Aug 19, 2024
65e6ff7
Adding condition to disable stackIdentification
yousifpatti Aug 20, 2024
20f9cf6
Fixed not loaded data regions
yousifpatti Aug 20, 2024
ac906b9
Fixed duplicate addresses causing tests to fail
yousifpatti Aug 21, 2024
89cf1c7
Initial VSA work
yousifpatti Aug 22, 2024
5ec6d7c
Injecting memory regions into IR
yousifpatti May 1, 2024
cee69a2
Loop handling in MRA (using depth for now)
yousifpatti May 17, 2024
aee3d46
Initial Works on VSA
yousifpatti Jun 17, 2024
12df143
Changes to partial region accesses
yousifpatti Jun 26, 2024
5c0619e
Adding regions into IR
yousifpatti Aug 13, 2024
f623faa
Initial VSA work
yousifpatti Aug 22, 2024
6ac45fc
Merge fixes
yousifpatti Aug 23, 2024
0e47d64
Merge branch 'yousif-VSA' of https://github.com/UQ-PAC/bil-to-boogie-…
yousifpatti Aug 23, 2024
b0853bc
Merge branch 'yousif-fixMemoryInjection' into yousif-VSA
yousifpatti Aug 23, 2024
6e1b51d
Fixed MRA loop bug
yousifpatti Aug 23, 2024
90c1b36
Merge branch 'main' into yousif-VSA
yousifpatti Aug 23, 2024
745245d
ReEnabled shared regions
yousifpatti Aug 23, 2024
5a8d5e3
Handling negatives and using BigInts instead
yousifpatti Sep 3, 2024
6a74737
Added sub-accesses to approximate stack region sizes
yousifpatti Sep 5, 2024
0e0a98c
Corrected negative addresses
yousifpatti Sep 5, 2024
250fdee
Added sizes to global regions
yousifpatti Sep 8, 2024
a873a06
A fix for global accesses
yousifpatti Sep 9, 2024
3df153a
faster MMM and heap support
yousifpatti Sep 12, 2024
40074fd
Heap region tracking using points-to
yousifpatti Sep 12, 2024
52e0bb7
Merging regions at start
yousifpatti Sep 16, 2024
fb2aa27
Update UtilMethods.scala
yousifpatti Sep 16, 2024
a791d5f
Using simple constantProp
yousifpatti Sep 17, 2024
c8537c0
Merge branch 'main' into memory-region-boogie
Sep 18, 2024
79b6259
Fixed regression bug for jumptable2
yousifpatti Sep 20, 2024
4b634c3
cleanup analyses + remove unused 'sharedVariable' in Variable
Sep 26, 2024
cd9fe8b
zero out bss sections
Sep 26, 2024
264fca0
replace memory accesses in boogie pre/post conditions with identified…
Sep 26, 2024
846b020
Changes in MRA and made Reaching def interproc
yousifpatti Sep 30, 2024
75d871f
Merge branch 'yousif-fixMemoryInjection' into memory-region-boogie
Oct 1, 2024
98d8b92
handle regions with size < 64 bits correctly
Oct 14, 2024
eadcf67
include memory regions in renaming to avoid boogie keywords
Oct 14, 2024
733cd0b
Added GRA
yousifpatti Oct 17, 2024
19e4d60
Merge branch 'yousif-VSA' into yousif-fixAnalysis
yousifpatti Oct 17, 2024
72c238e
Better Data Region Approximation
yousifpatti Oct 17, 2024
874b5b0
VSA results feedback
yousifpatti Oct 17, 2024
71f1cba
Handling multiple regions
yousifpatti Oct 21, 2024
277a3fc
use visitor to resolve specification variables
Oct 22, 2024
8612486
fix mistake in SystemTests
Oct 22, 2024
b228b7f
add regions to boogie output for specifications and guarantees
Oct 22, 2024
d601a51
add regions to boogie output for rely
Oct 23, 2024
e4e2682
correct regions in Boogie output for secure update and guarantee chec…
Oct 23, 2024
cbbe539
properly collect Boogie global variables
Oct 23, 2024
6aa0f42
have L function in Boogie take in all relevant memories as parameters…
Oct 23, 2024
74e4810
prevent stack being added to rely
Oct 23, 2024
53e5ad9
Cleanup
yousifpatti Oct 25, 2024
106737d
Merge branch 'yousif-fixMemoryInjection' into memory-region-boogie
Oct 24, 2024
d0ca3ac
re-enable procedure summaries
Oct 28, 2024
717b645
Merge branch 'main' into memory-region-boogie
Oct 28, 2024
46b144c
Fixes to pointers-to-pointers
yousifpatti Oct 29, 2024
8cb5e2a
Merge branch 'yousif-fixMemoryInjection' into memory-region-boogie
Oct 29, 2024
937cfc2
general clean up
Oct 29, 2024
dabf9f9
remove mutability in DataRegion case class
Oct 30, 2024
af29cc1
remove mutability in StackRegion case class
Oct 30, 2024
250d9c7
add --memory-regions flag to control whether regions are added to Boogie
Oct 30, 2024
162da67
Merge pull request #262 from UQ-PAC/memory-region-boogie
l-kent Oct 30, 2024
d1c0870
don't do multiple iterations unless needed for regions
Oct 30, 2024
c6e5d8a
fix issue with L function when analysis is off
Oct 30, 2024
dc1668e
fix issue with initialised memory and stack when analysis is enabled
Oct 30, 2024
19dcdb0
update expected
Oct 30, 2024
ba13989
Merge branch 'memory-region-boogie' into yousif-fixMemoryInjection
Oct 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions examples/one_function_multi_call/one_function_multi_call.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#include <stdio.h>

// Function declarations
int addNumbers(int a, int b);

int callAddFromAnotherFunction(int x, int y) {
return addNumbers(x, y);
}

int callFromFun2(int x, int y) {
return addNumbers(x, y);
}

int addNumbers(int a, int b) {
return a + b;
}

int main() {
int resultFromMain = addNumbers(10, 5);
int resultFromOtherFunc = callAddFromAnotherFunction(20, 15);
int resultFromFun2 = callFromFun2(30, 25);
return 0;
}
416 changes: 416 additions & 0 deletions src/main/scala/analysis/ActualVSA.scala
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't commit unnecessary files like this

Large diffs are not rendered by default.

16 changes: 9 additions & 7 deletions src/main/scala/analysis/Analysis.scala
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ trait Analysis[+R]:

/** Base class for value analysis with simple (non-lifted) lattice.
*/
trait ConstantPropagation(val program: Program) {
trait ConstantPropagation(val program: Program, val assumeR31: Boolean) {
/** The lattice of abstract states.
*/

Expand Down Expand Up @@ -76,15 +76,16 @@ trait ConstantPropagation(val program: Program) {
/** Transfer function for state lattice elements.
*/
def localTransfer(n: CFGPosition, s: Map[Variable, FlatElement[BitVecLiteral]]): Map[Variable, FlatElement[BitVecLiteral]] =
var m = s
n match
case r: Command =>
r match
// assignments
case la: Assign =>
s + (la.lhs -> eval(la.rhs, s))
m + (la.lhs -> eval(la.rhs, m))
// all others: like no-ops
case _ => s
case _ => s
case _ => m
case _ => m

/** The analysis lattice.
*/
Expand All @@ -97,11 +98,12 @@ trait ConstantPropagation(val program: Program) {
def transfer(n: CFGPosition, s: Map[Variable, FlatElement[BitVecLiteral]]): Map[Variable, FlatElement[BitVecLiteral]] = localTransfer(n, s)
}

class ConstantPropagationSolver(program: Program) extends ConstantPropagation(program)
class ConstantPropagationSolver(program: Program, assumeR31: Boolean = false) extends ConstantPropagation(program, assumeR31)
with SimplePushDownWorklistFixpointSolver[CFGPosition, Map[Variable, FlatElement[BitVecLiteral]], MapLattice[Variable, FlatElement[BitVecLiteral], ConstantPropagationLattice]]
with IRIntraproceduralForwardDependencies
with IRInterproceduralForwardDependencies
with Analysis[Map[CFGPosition, Map[Variable, FlatElement[BitVecLiteral]]]]


/** Base class for value analysis with simple (non-lifted) lattice.
*/
trait ConstantPropagationWithSSA(val program: Program, val reachingDefs: Map[CFGPosition, (Map[Variable, Set[Assign]], Map[Variable, Set[Assign]])]) {
Expand Down Expand Up @@ -190,5 +192,5 @@ trait ConstantPropagationWithSSA(val program: Program, val reachingDefs: Map[CFG

class ConstantPropagationSolverWithSSA(program: Program, reachingDefs: Map[CFGPosition, (Map[Variable, Set[Assign]], Map[Variable, Set[Assign]])]) extends ConstantPropagationWithSSA(program, reachingDefs)
with SimplePushDownWorklistFixpointSolver[CFGPosition, Map[RegisterWrapperEqualSets, Set[BitVecLiteral]], MapLattice[RegisterWrapperEqualSets, Set[BitVecLiteral], ConstantPropagationLatticeWithSSA]]
with IRIntraproceduralForwardDependencies
with IRInterproceduralForwardDependencies
with Analysis[Map[CFGPosition, Map[RegisterWrapperEqualSets, Set[BitVecLiteral]]]]
28 changes: 27 additions & 1 deletion src/main/scala/analysis/BitVectorEval.scala
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
package analysis
import ir._
import ir.*
import analysis.BitVectorEval.*

import scala.annotation.tailrec
import scala.math.pow

object BitVectorEval {
Expand Down Expand Up @@ -328,4 +329,29 @@ object BitVectorEval {
}
}

def bitVec_min(s: BitVecLiteral, t: BitVecLiteral): BitVecLiteral = {
if (smt_bvslt(s, t) == TrueLiteral) s else t
}

def bitVec_min(s: List[BitVecLiteral]): BitVecLiteral = {
s.reduce(bitVec_min)
}

def bitVec_max(s: BitVecLiteral, t: BitVecLiteral): BitVecLiteral = {
if (smt_bvslt(s, t) == TrueLiteral) t else s
}

def bitVec_max(s: List[BitVecLiteral]): BitVecLiteral = {
s.reduce(bitVec_max)
}

@tailrec
def bitVec_gcd(a: BitVecLiteral, b: BitVecLiteral): BitVecLiteral = {
if (b.value == 0) a else bitVec_gcd(b, smt_bvsmod(a, b))
}

def bitVec_interval(lb: BitVecLiteral, ub: BitVecLiteral, step: BitVecLiteral): Set[BitVecLiteral] = {
require(smt_bvule(lb, ub) == TrueLiteral, "Lower bound must be less than or equal to upper bound")
(lb.value to ub.value by step.value).map(BitVecLiteral(_, lb.size)).toSet
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is any of this needed for anything actually in use? If not, it shouldn't be included in this pull request

}
247 changes: 247 additions & 0 deletions src/main/scala/analysis/GlobalRegionAnalysis.scala
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this analysis actually do & why do we need it to be separate from the MRA?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This analysis only evaluates data regions and MRA evaluates the remaining stack and heap regions. MRA uses the results of GRA to skip statements that are already evaluated to data regions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need separate analyses for that then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was to separate the eval and transfer functions. They can be merged by bringing the functions over. I will work on that

Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
package analysis

import analysis.solvers.SimpleWorklistFixpointSolver
import ir.*

import scala.collection.mutable

trait GlobalRegionAnalysis(val program: Program,
val domain: Set[CFGPosition],
val constantProp: Map[CFGPosition, Map[Variable, FlatElement[BitVecLiteral]]],
val reachingDefs: Map[CFGPosition, (Map[Variable, Set[Assign]], Map[Variable, Set[Assign]])],
val mmm: MemoryModelMap,
val vsaResult: Option[Map[CFGPosition, LiftedElement[Map[Variable | MemoryRegion, Set[Value]]]]]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't really make sense to have a Map inside an Option - you can just use an empty Map as the default instead of None.


var dataCount: Int = 0
private def nextDataCount() = {
dataCount += 1
s"data_$dataCount"
}

val regionLattice: PowersetLattice[DataRegion] = PowersetLattice()

val lattice: MapLattice[CFGPosition, Set[DataRegion], PowersetLattice[DataRegion]] = MapLattice(regionLattice)

val first: Set[CFGPosition] = Set.empty + program.mainProcedure

private val stackPointer = Register("R31", 64)
private val linkRegister = Register("R30", 64)
private val framePointer = Register("R29", 64)
private val mallocVariable = Register("R0", 64)

private val dataMap: mutable.HashMap[BigInt, DataRegion] = mutable.HashMap()

private def dataPoolMaster(offset: BigInt, size: BigInt): Option[DataRegion] = {
assert(size >= 0)
if (dataMap.contains(offset)) {
if (dataMap(offset).size < (size.toDouble / 8).ceil.toInt) {
dataMap(offset) = DataRegion(dataMap(offset).regionIdentifier, offset, (size.toDouble / 8).ceil.toInt)
Some(dataMap(offset))
} else {
Some(dataMap(offset))
}
} else {
dataMap(offset) = DataRegion(nextDataCount(), offset, (size.toDouble / 8).ceil.toInt)
Some(dataMap(offset))
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to return an Option when it never returns None?


def getDataMap: mutable.HashMap[BigInt, DataRegion] = dataMap

/**
* For DataRegions, the actual address used needs to be converted to the relocated address.
* This is because when regions are found, the relocated address is used and as such match
* the correct range.
*
* @param address: The starting DataRegion
* @return DataRegion: The relocated data region if any
*/
def resolveGlobalOffsetSecondLast(address: DataRegion): DataRegion = {
var tableAddress = address
// addresses may be layered as in jumptable2 example for which recursive search is required
var exitLoop = false
while (mmm.relocatedDataRegion(tableAddress.start).isDefined && mmm.relocatedDataRegion(mmm.relocatedDataRegion(tableAddress.start).get.start).isDefined && !exitLoop) {
val newAddress = mmm.relocatedDataRegion(tableAddress.start).getOrElse(tableAddress)
if (newAddress == tableAddress) {
exitLoop = true
} else {
tableAddress = newAddress
}
}
tableAddress
}

def tryCoerceIntoData(exp: Expr, n: Command, subAccess: BigInt, loadOp: Boolean = false): Set[DataRegion] = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand how this method works, can you explain it? I don't really follow why BinaryExprs are such a special case for it?

val eval = evaluateExpression(exp, constantProp(n))
if (eval.isDefined) {
val region = dataPoolMaster(eval.get.value, subAccess)
if (region.isDefined) {
return Set(region.get)
}
}
exp match
case literal: BitVecLiteral => tryCoerceIntoData(literal, n, subAccess)
case Extract(end, start, body) => tryCoerceIntoData(body, n, subAccess)
case Repeat(repeats, body) => tryCoerceIntoData(body, n, subAccess)
case ZeroExtend(extension, body) => tryCoerceIntoData(body, n, subAccess)
case SignExtend(extension, body) => tryCoerceIntoData(body, n, subAccess)
case UnaryExpr(op, arg) => tryCoerceIntoData(arg, n, subAccess)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of these are robust ways of trying to identify which data regions an expression points to. You can't just ignore these operations like this, it will give incorrect results.

case BinaryExpr(op, arg1, arg2) =>
val evalArg2 = evaluateExpression(arg2, constantProp(n))
if (evalArg2.isDefined) {
val firstArg = tryCoerceIntoData(arg1, n, subAccess, true)
var regions = Set.empty[DataRegion]
for (i <- firstArg) {
val newExpr = BinaryExpr(op, BitVecLiteral(i.start, evalArg2.get.size), evalArg2.get)
regions = regions ++ tryCoerceIntoData(newExpr, n, subAccess)
}
return regions
}
Set.empty
case MemoryLoad(mem, index, endian, size) => ???
case UninterpretedFunction(name, params, returnType) => Set.empty
case variable: Variable =>
val ctx = getUse(variable, n, reachingDefs)
var collage = Set.empty[DataRegion]
for (i <- ctx) {
if (i != n) {
var tryVisit = Set.empty[DataRegion]
if (vsaResult.isDefined) {
vsaResult.get.get(i) match
case Some(value) => value match
case Lift(el) => el.get(i.lhs) match
case Some(value) => value.map {
case addressValue: AddressValue =>
// find what the region contains
vsaResult.get.get(i) match
case Some(value) => value match
case Lift(el) => el.get(addressValue.region) match
case Some(value) => value.map {
case addressValue: AddressValue =>
addressValue.region match
case region: DataRegion =>
tryVisit = tryVisit + region
case _ =>
case literalValue: LiteralValue =>
}
case None =>
case LiftedBottom =>
case _ =>
case None =>
case literalValue: LiteralValue =>
}
case None =>
case LiftedBottom =>
case _ =>
case None =>
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems overly convoluted

if (tryVisit.isEmpty) {
tryVisit = localTransfer(i, Set.empty)
}
if (tryVisit.nonEmpty) {
collage = collage ++ tryVisit
}
}
}
collage.map(i =>
val resolved = resolveGlobalOffsetSecondLast(i)
if !loadOp then mmm.relocatedDataRegion(i.start).getOrElse(i) else resolved)
case _ => Set.empty
}

def evalMemLoadToGlobal(index: Expr, size: BigInt, n: Command, loadOp: Boolean = false): Set[DataRegion] = {
val indexValue = evaluateExpression(index, constantProp(n))
if (indexValue.isDefined) {
val indexValueBigInt = indexValue.get.value
val region = dataPoolMaster(indexValueBigInt, size)
if (region.isDefined) {
return Set(region.get)
}
}
tryCoerceIntoData(index, n, size)
}

// def mergeRegions(regions: Set[DataRegion]): DataRegion = {
// if (regions.size == 1) {
// return regions.head
// }
// val start = regions.minBy(_.start).start
// val end = regions.maxBy(_.end).end
// val size = end - start
// val newRegion = DataRegion(nextDataCount(), start, size)
// regions.foreach(i => dataMap(i.start) = newRegion)
// newRegion
// }

/**
* Check if the data region is defined.
* Finds full and partial matches
* Full matches sizes are altered to match the size of the data region
* Partial matches are not altered
* Otherwise the data region is returned
*
* @param dataRegions Set[DataRegion]
* @param n CFGPosition
* @return Set[DataRegion]
*/
def checkIfDefined(dataRegions: Set[DataRegion], n: CFGPosition): Set[DataRegion] = {
var returnSet = Set.empty[DataRegion]
for (i <- dataRegions) {
val (f, p) = mmm.findDataObjectWithSize(i.start, i.size)
val accesses = f.union(p)
if (accesses.isEmpty) {
returnSet = returnSet + i
} else {
if (accesses.size == 1) {
dataMap(i.start) = DataRegion(i.regionIdentifier, i.start, i.size.max(accesses.head.size))
returnSet = returnSet + dataMap(i.start)
} else if (accesses.size > 1) {
val highestRegion = accesses.maxBy(_.start)
dataMap(i.start) = DataRegion(i.regionIdentifier, i.start, i.size.max(highestRegion.end - i.start))
returnSet = returnSet + dataMap(i.start)
}
}
}
if (returnSet.size > 1) {
mmm.addMergeRegions(returnSet.asInstanceOf[Set[MemoryRegion]], nextDataCount())
}
returnSet
}

/** Transfer function for state lattice elements.
*/
def localTransfer(n: CFGPosition, s: Set[DataRegion]): Set[DataRegion] = {
n match {
case cmd: Command =>
cmd match {
case memAssign: MemoryAssign =>
return checkIfDefined(evalMemLoadToGlobal(memAssign.index, memAssign.size, cmd), n)
case assign: Assign =>
val unwrapped = unwrapExpr(assign.rhs)
if (unwrapped.isDefined) {
return checkIfDefined(evalMemLoadToGlobal(unwrapped.get.index, unwrapped.get.size, cmd, loadOp = true), n)
} else {
// this is a constant but we need to check if it is a data region
return checkIfDefined(evalMemLoadToGlobal(assign.rhs, 1, cmd), n)
}
case _ =>
}
case _ =>
}
Set.empty
}

def transfer(n: CFGPosition, s: Set[DataRegion]): Set[DataRegion] = localTransfer(n, s)
}

class GlobalRegionAnalysisSolver(
program: Program,
domain: Set[CFGPosition],
constantProp: Map[CFGPosition, Map[Variable, FlatElement[BitVecLiteral]]],
reachingDefs: Map[CFGPosition, (Map[Variable, Set[Assign]], Map[Variable, Set[Assign]])],
mmm: MemoryModelMap,
vsaResult: Option[Map[CFGPosition, LiftedElement[Map[Variable | MemoryRegion, Set[Value]]]]]
) extends GlobalRegionAnalysis(program, domain, constantProp, reachingDefs, mmm, vsaResult)
with IRIntraproceduralForwardDependencies
with Analysis[Map[CFGPosition, Set[DataRegion]]]
with SimpleWorklistFixpointSolver[CFGPosition, Set[DataRegion], PowersetLattice[DataRegion]]
Loading