added example parameters + proper readme + minor adjustments

README.md

@@ -1 +1,49 @@
-Requries Python 3.5 and Gurobi 7.0.
+# B-JointSP 
+
+B-JointSP is an optimization problem focusing on the joint scaling and placement (called embedding) of NFV network services, consisting of interconnected virtual network functions (VNFs). The exceptional about B-JointSP is its consideration of realistic, bidirectional network services, in which flows return to their sources. It even supports stateful VNFs, that need to be traversed by the same flows in both upstream and downstream direction. Furthermore, B-JointSP allows the reuse of VNFs across different network services and supports physical network functions.
+
+This repository provides the source code for an optimization approach formulated as MIP that can be linearized as MILP and solved with Gurobi. We also provide a fast heuristic algorithm.
+
+## Installation requirements
+
+* [Python 3.5](https://www.python.org/)
+* [Gurobi 7.0](http://www.gurobi.com/) and [gurobipy](http://www.gurobi.com/documentation/6.5/quickstart_mac/the_gurobi_python_interfac.html) for the optimization approach
+
+## Usage/Execution
+
+### Parameters
+
+To describe an embedding scenario, the following parameters are required:
+
+* A substrate network, with node and link capacities as well as link delays
+* At least one network service (template), specifying the different kinds of VNFs in the service and their interconnection
+* Sources corresponding to the source components of the specified services and located at certain network nodes. Each source specifies at least one outgoing flow (and its flow strength).
+
+Optional parameters:
+
+* Fixed locations of physical network functions
+* A previously existing embedding to be optimized
+
+Each of these parameters is described by a separate csv file. A scenario description (also csv) references these individual parameters. See the `parameters` folder for examples.
+
+When running the MIP or the heuristic, the detailed embedding results are stored in `parameters/results` and logs (for debugging or checking the progress) are in `parameters/logs`.
+
+### MIP
+
+To run the MIP and obtain optimal results (with Gurobi), use the following command: `python main.py mip <scenario> (<repetition>)`, where `<scenario>` is the path to a scenario file and `(<repetition>)` is an optional repetition number just for distinguishing different runs with the same parameters.
+
+For example, `python main.py mip parameters/scenarios/simple.csv` solves a simple embedding example, where a bidirectional network service with a firewall and a vCDN is embedded into a four-node network.
+
+### Heuristic
+
+To run the heuristic algorithm, use the command `python main.py heuristic <scenario> (<seed>)`. Again, `<seed>` specifies the scenario file and `(<seed>)` is an optional seed for randomization.
+
+The same example embedding can also be performed with the heuristic: `python main.py heuristic parameters/scenarios/simple.csv`.
+
+## Contact
+
+This source code belongs to the paper "**Scaling and Placing Bidirectional Services with Stateful Virtual and Physical Network Functions**" submitted to IEEE NetSoft 2018 by Sevil Dräxler, Stefan Schneider, and Holger Karl.
+
+Lead developer: Stefan Schneider
+
+For questions or support, please use GitHub's issue system.

main.py

@@ -17,9 +17,9 @@
 
 # read scenario input
 if len(sys.argv) < 3:
-	print("MIP usage: python3 main.py mip <scenario> (<repetition>)")
-	print("Heuristic usage: python3 main.py heuristic <scenario> (<seed>)")
-	print("Pareto usage: python3 main.py pareto <scenario> <objective> <bound1> <bound2> <bound3>")
+	print("MIP usage: python main.py mip <scenario> (<repetition>)")
+	print("Heuristic usage: python main.py heuristic <scenario> (<seed>)")
+	print("Pareto usage: python main.py pareto <scenario> <objective> <bound1> <bound2> <bound3>")
 	# print("Pareto usage: python3 main.py pareto <scenario> <objective> <bound1> <bound2>")
 	exit(1)
 method = sys.argv[1]
@@ -68,8 +68,8 @@
 	# set up logging into file Data/logs/heuristic/scenario_timestamp_seed.log
 	# logging.disable(logging.CRITICAL)		# disable logging
 	timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
-	os.makedirs("Data/logs/heuristic/obj{}".format(obj), exist_ok=True)
-	logging.basicConfig(filename="Data/logs/heuristic/obj{}/{}_{}_{}.log".format(obj, os.path.basename(scenario)[:-4], timestamp, seed),
+	os.makedirs("parameters/logs/heuristic/obj{}".format(obj), exist_ok=True)
+	logging.basicConfig(filename="parameters/logs/heuristic/obj{}/{}_{}_{}.log".format(obj, os.path.basename(scenario)[:-4], timestamp, seed),
 						level=logging.DEBUG, format="%(asctime)s(%(levelname)s):\t%(message)s", datefmt="%H:%M:%S")
 
 	logging.info("Starting initial embedding at {}".format(timestamp))

parameters/networks/square_network.csv

@@ -0,0 +1,17 @@
+# substrate network: square of four nodes
+# only use spaces as delimiter (no tabs)
+# Nodes: id cpu mem
+0 5 5
+1 5 5
+2 5 5
+3 5 5
+
+# DIRECTED Links: src_id sink_id cap delay
+0 1 10 1
+1 0 10 1
+1 2 10 1
+2 1 10 1
+2 3 10 1
+3 2 10 1
+3 0 10 1
+0 3 10 1

parameters/optional/events.csv

@@ -0,0 +1,11 @@
+# sequence of new scenario inputs to model events
+# allows arbitrary changes (even multiple at once), e.g., adjusted/new/deleted templates, sources, fixed instances, but not changing the substrate network
+# need to add corresponding templates before adding new sources such that the source components can be recognized
+
+# syntax:
+# templates: file_paths (separated by white spaces)
+# sources: file_path
+# fixed: file_path
+
+sources: ../sources/sources2.csv
+sources: ../sources/sources3.csv

parameters/optional/fixed.csv

@@ -0,0 +1,2 @@
+# fixed instances (e.g., legacy NFs): node_id component
+1 CDN1

parameters/optional/prev_embedding1.csv

@@ -0,0 +1,4 @@
+# previous embedding (only instance locations): node component
+0 S1
+1 FW
+1 CDN1

parameters/optional/prev_embedding2.csv

@@ -0,0 +1,6 @@
+# previous embedding (only instance locations): node component
+0 S1
+0 S2
+1 FW
+1 CDN1
+2 vCDN2

parameters/scenarios/heuristic-scenario.csv

@@ -0,0 +1,10 @@
+# scenario: collection of all inputs to the optimization problem (relative file path)
+# order and identifiers (e.g., "network:") are fixed and must not be changed; additional whitespaces (eg., in names or at the end of a line) or inline comments are not allowed
+
+# substrate network, 1+ templates (separated by whitespace), previous embedding, data sources, fixed instances
+# the heuristic ignores prev_embedding and the mip ignores events
+network: ../networks/square_network.csv
+templates: ../templates/cdn1.csv ../templates/cdn2.csv
+sources: ../sources/sources1.csv
+fixed: ../optional/fixed.csv
+events: ../optional/events.csv

parameters/scenarios/mip1-scenario.csv

@@ -0,0 +1,10 @@
+# scenario: collection of all inputs to the optimization problem (relative file path)
+# order and identifiers (e.g., "network:") are fixed and must not be changed; additional whitespaces (eg., in names or at the end of a line) or inline comments are not allowed
+
+# substrate network, 1+ templates (separated by whitespace), previous embedding, data sources, fixed instances
+network: ../networks/square_network.csv
+templates: ../templates/cdn1.csv ../templates/cdn2.csv
+sources: ../sources/sources1.csv
+fixed: ../optional/fixed.csv
+prev_embedding:
+events:

parameters/scenarios/mip2-scenario.csv

@@ -0,0 +1,10 @@
+# scenario: collection of all inputs to the optimization problem (relative file path)
+# order and identifiers (e.g., "network:") are fixed and must not be changed; additional whitespaces (eg., in names or at the end of a line) or inline comments are not allowed
+
+# substrate network, 1+ templates (separated by whitespace), previous embedding, data sources, fixed instances
+network: ../networks/square_network.csv
+templates: ../templates/cdn1.csv ../templates/cdn2.csv
+sources: ../sources/sources2.csv
+fixed: ../optional/fixed.csv
+prev_embedding: ../optional/prev_embedding1.csv
+events:

parameters/scenarios/mip3-scenario.csv

@@ -0,0 +1,10 @@
+# scenario: collection of all inputs to the optimization problem (relative file path)
+# order and identifiers (e.g., "network:") are fixed and must not be changed; additional whitespaces (eg., in names or at the end of a line) or inline comments are not allowed
+
+# substrate network, 1+ templates (separated by whitespace), previous embedding, data sources, fixed instances
+network: ../networks/square_network.csv
+templates: ../templates/cdn1.csv ../templates/cdn2.csv
+sources: ../sources/sources3.csv
+fixed: ../optional/fixed.csv
+prev_embedding: ../optional/prev_embedding2.csv
+events:

parameters/scenarios/simple.csv

@@ -0,0 +1,10 @@
+# scenario: collection of all inputs to the optimization problem (relative file path)
+# order and identifiers (e.g., "network:") are fixed and must not be changed; additional whitespaces (eg., in names or at the end of a line) or inline comments are not allowed
+
+# substrate network, 1+ templates (separated by whitespace), previous embedding, data sources, fixed instances
+network: ../networks/square_network.csv
+templates: ../templates/cdn1.csv
+sources: ../sources/sources1.csv
+fixed:
+prev_embedding:
+events:

parameters/sources/sources1.csv

@@ -0,0 +1,3 @@
+# data sources: node_id source_component data_rate
+# flows: (flow_id data_rate)+
+0 S1 1 1

parameters/sources/sources2.csv

@@ -0,0 +1,4 @@
+# data sources: node_id source_component data_rate
+# flows: (flow_id data_rate)+
+0 S1 1 1
+0 S2 2 1

parameters/sources/sources3.csv

@@ -0,0 +1,5 @@
+# data sources: node_id source_component data_rate
+# flows: (flow_id data_rate)+
+0 S1 1 1
+0 S2 2 1
+3 S2 3 1

parameters/templates/cdn1.csv

@@ -0,0 +1,15 @@
+# template: S1<->FW<->CDN1
+# name
+cdn1
+
+# Components: name type stateful inputs in_back outputs out_back [cpu_coeff] [mem_coeff] [[outgoing_coeff]] [[outgoing_back]]
+# type={source, normal, end}; values separated by tabs; coeff separated by comma - first fw_in, then bw_in, then +constant; outgoing for diff outputs separated by ;
+S1		source	True	0	1	1	0	[0,0]	[0,0]	[]		[]
+FW		normal	True	1	1	1	1	[1,1,0]	[1,1,0]	[[1,0]]	[[1,0]]
+CDN1	end		False 	1	0	0	1	[1,0]	[1,0]	[]		[[1,0]]
+
+# Arcs: direction src_name src_output dest_name dest_input max_delay
+forward		S1		0	FW		0	20
+forward		FW		0	CDN1	0	20
+backward	CDN1	0	FW		0	20
+backward	FW		0	S1		0	20

parameters/templates/cdn2.csv

@@ -0,0 +1,15 @@
+# template: S2<->FW<->vCDN2
+# name
+cdn2
+
+# Components: name type stateful inputs in_back outputs out_back [cpu_coeff] [mem_coeff] [[outgoing_coeff]] [[outgoing_back]]
+# type={source, normal, end}; values separated by tabs; coeff separated by comma - first fw_in, then bw_in, then +constant; outgoing for diff outputs separated by ;
+S2		source	True	0	1	1	0	[0,0]	[0,0]	[]		[]
+FW		normal	True	1	1	1	1	[1,1,0]	[1,1,0]	[[1,0]]	[[1,0]]
+vCDN2	end		False 	1	0	0	1	[1,0]	[1,0]	[]		[[1,0]]
+
+# Arcs: direction src_name src_output dest_name dest_input max_delay
+forward		S2		0	FW		0	20
+forward		FW		0	vCDN2	0	20
+backward	vCDN2	0	FW		0	20
+backward	FW		0	S2		0	20

parameters/templates/unidir_chain.csv

@@ -0,0 +1,13 @@
+# simple chain template: S1->A->B
+# name
+chain
+
+# Components: name type stateful inputs in_back outputs out_back [cpu_coeff] [mem_coeff] [[outgoing_coeff]] [[outgoing_back]]
+# type={source, normal, end}; values separated by tabs; coeff separated by comma - first fw_in, then bw_in, then +constant; outgoing for diff outputs separated by ;
+S1	source	True	0	0	1	0	[0]		[0]		[]	[]
+A	normal	False	1	0	1	0	[1,0]	[1,0]	[[1,0]]	[]
+B	normal	False	1	0	0	0	[1,0]	[1,0]	[]	[]
+
+# Arcs: direction src_name src_output dest_name dest_input max_delay
+forward		S1	0	A	0	20
+forward		A	0	B	0	20

parameters/templates/video.csv

@@ -0,0 +1,23 @@
+# video streaming template with memory idle consumption (cache and server)
+# forward: streaming requests with low dr to cache and server
+# backward: high dr video stream via the cache that is possibly optimized for mobile devices when loaded from the server (distinguished with DPI)
+
+# name
+video
+
+# Components: name type stateful inputs in_back outputs out_back [cpu_coeff] [mem_coeff] [[outgoing_coeff]] [[outgoing_back]]
+# type={source, normal, end}; values separated by tabs; coeff separated by comma - first fw_in, then bw_in, then +constant; outgoing for diff outputs separated by ;
+S2		source	True	0	1	1	0	[0,0]		[0,0]		[]		[]
+Cache2	normal	False	1	2	1	1	[1,1,1,0]	[2,2,2,0.5]	[[1,0]]	[[2,2,0]]
+Server2	end		False 	1	0	0	1	[1,0]		[2,0.5]		[]		[[1.5,0]]
+DPI		normal	False	0	1	0	2	[1,0]		[1,0]		[]		[[0.9,0];[0.1,0]]
+VidOpt	normal	False	0	1	0	1	[2,0]		[1,0]		[]		[[0.5,0]]
+
+# Arcs: direction src_name src_output dest_name dest_input max_delay
+forward		S2		0	Cache2	0	50
+forward		Cache2	0	Server2	0	50
+backward	Server2	0	DPI		0	50
+backward	DPI		0	Cache2	0	50
+backward	DPI		1	VidOpt	0	50
+backward	VidOpt	0	Cache2	1	50
+backward	Cache2	0	S2		0	50

read_write/reader.py

@@ -191,11 +191,11 @@ def read_prev_embedding(file, components):
 			if len(row) == 2:
 				try:
 					# get the component with the specified name: first (and only) element with component name
-					component = list(filter(lambda x: x.name == row[0], components))[0]
+					component = list(filter(lambda x: x.name == row[1], components))[0]
 				except IndexError:
 					raise ValueError("Component {} of prev overlay unknown (not used in any template).".format(row[1]))
 
-				prev_embedding[component].append(row[1])
+				prev_embedding[component].append(row[0])
 	return prev_embedding