Skip to content

Commit

Permalink
Merge branch 'release-1.114.0'
Browse files Browse the repository at this point in the history
  • Loading branch information
dpoltavets committed Jan 31, 2022
2 parents 6def1bd + 44d5488 commit 2ef830c
Show file tree
Hide file tree
Showing 99 changed files with 1,999 additions and 742 deletions.
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# Changelog

## v1.114.0 (31/01/2022)

### Features:
- [#4675](https://github.com/telstra/open-kilda/pull/4675) Y Flow Ping: Periodic pings (Issue: [#4589](https://github.com/telstra/open-kilda/issues/4589)) [**storm-topologies**]
- [#4676](https://github.com/telstra/open-kilda/pull/4676) [test] use yFlowPing in func-tests [**tests**]
- [#4616](https://github.com/telstra/open-kilda/pull/4616) [yflow] tests for a subFlow [**tests**]
- [#4655](https://github.com/telstra/open-kilda/pull/4655) Add diversity to YFlow [**docs**][**northbound**][**storm-topologies**]

### Bug Fixes:
- [#4674](https://github.com/telstra/open-kilda/pull/4674) Fix Y-flow update - handle allocateProtectedPath changes
- [#4677](https://github.com/telstra/open-kilda/pull/4677) Fix PCE for protected YFlows.
- [#4668](https://github.com/telstra/open-kilda/pull/4668) Fix switch connections transaction retry policy [**storm-topologies**]
- [#4671](https://github.com/telstra/open-kilda/pull/4671) Fix Y-flow API - add allocateProtectedPath to returned entities [**northbound**]

### Improvements:
- [#4654](https://github.com/telstra/open-kilda/pull/4654) Reimplement LAG logical port number allocation [**storm-topologies**]

For the complete list of changes, check out [the commit log](https://github.com/telstra/open-kilda/compare/v1.113.0...v1.114.0).

### Affected Components:
ping, nbworker, network, nb, swmanager, flow-hs

---

## v1.113.0 (25/01/2022)

### Features:
Expand Down
3 changes: 3 additions & 0 deletions confd/templates/base-storm-topology/topology.properties.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ port.up.down.throttling.delay.seconds.cool.down = {{ getv "/kilda_port_up_down_t
port.antiflap.stats.dumping.interval.seconds = 60

lag.port.offset = {{ getv "/kilda_lag_port_offset" }}
lag.port.max.number = {{ getv "/kilda_lag_port_max_number" }}
lag.port.pool.chunks.count = 10
lag.port.pool.cache.size = 128
bfd.port.offset = {{ getv "/kilda_bfd_port_offset" }}
bfd.port.max.number = {{ getv "/kilda_bfd_port_max_number" }}

Expand Down
1 change: 1 addition & 0 deletions confd/vars/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ kilda_port_up_down_throttling_delay_seconds_warm_up: 3
kilda_port_up_down_throttling_delay_seconds_cool_down: 7

kilda_lag_port_offset: 2000
kilda_lag_port_max_number: 2999
kilda_bfd_port_offset: 1000
kilda_bfd_port_max_number: 1999
kilda_bfd_interval_ms: 350
Expand Down
8 changes: 7 additions & 1 deletion docs/design/LAG-for-ports/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,13 @@ Response example:
## Details
All logical port related commands are sent to the switches using gRPC speaker.

Open-kilda calculate logical port number using the following rule: 2000 + min of the physical ports number in the LAG. It is not allowed to have one physical port in two LAGs so this rule will provide unique logical port number for any correct port configuration. LAG logical port configuration should be validated before any create operation to avoid inconsistency.
Kilda configuration defines logical port numbers range and amount of chunks in this range. During LAG create operation
random chunk number will be selected and first unassigned number from this chunk will be used as logical port number.
Port number allocation done on per switch basis, so different switches can have LAG logical ports with same numbers.

It is not allowed to have one physical port in two LAGs so this rule will provide unique logical port number for any
correct port configuration. LAG logical port configuration should be validated before any create operation to avoid
inconsistency.

Currently, open-kilda doesn't have any port related information representation in database. We need to save LAG logical port configuration into database to have ability to restore configuration on the switch. Information about LAGs stored as a separate models in order to provide minimal impact on already existing data structures.

Expand Down
17 changes: 15 additions & 2 deletions docs/design/y-flow/y-flow-nb-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ request payload:
"pinned": false,
"priority": 0,
"strict_bandwidth": true,
"description": "description"
"description": "description",
"diverse_flow_id": "diverse_flow_id",
"sub_flows": [
{
"endpoint": {
Expand Down Expand Up @@ -61,6 +62,12 @@ response payload (copy of request payload plus autogenerated fields):
"shared_endpoint": ...
... : ...,
"y_point": null,
"diverse_with_flows": [
"diverse_flow"
],
"diverse_with_y_flows": [
"diverse_y_flow"
],
"sub_flows": [
{
"flow_id": "fAAAAAAAAAAAAAAA",
Expand Down Expand Up @@ -98,7 +105,7 @@ response payload (copy of request payload plus autogenerated fields):
}
],
"time_create": "YYYY-MM-DDThh:mm:ss[.SSS]",
"time_update": "YYYY-MM-DDThh:mm:ss[.SSS]",
"time_update": "YYYY-MM-DDThh:mm:ss[.SSS]"
}
```

Expand Down Expand Up @@ -128,6 +135,12 @@ response payload:
"strict_bandwidth": true,
"description": "description",
"y_point": "00:00:00:00:00:00:01:01",
"diverse_with_flows": [
"diverse_flow"
],
"diverse_with_y_flows": [
"diverse_y_flow"
],
"sub_flows": [
{
"flow_id": "fAAAAAAAAAAAAAAA",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,9 @@ public class FlowDto implements Serializable {
@JsonProperty("diverse_with")
private Set<String> diverseWith;

@JsonProperty("diverse_with_y_flows")
private Set<String> diverseWithYFlows;

@JsonProperty("affinity_with")
private String affinityWith;

Expand Down Expand Up @@ -290,6 +293,7 @@ public FlowDto(@JsonProperty(Utils.FLOW_ID) final String flowId,
@JsonProperty("target_path_computation_strategy")
PathComputationStrategy targetPathComputationStrategy,
@JsonProperty("diverse_with") Set<String> diverseWith,
@JsonProperty("diverse_with_y_flows") Set<String> diverseWithYFlows,
@JsonProperty("affinity_with") String affinityWith,
@JsonProperty("loop_switch_id") SwitchId loopSwitchId,
@JsonProperty("mirror_point_statuses") List<MirrorPointStatusDto> mirrorPointStatuses,
Expand Down Expand Up @@ -329,6 +333,7 @@ public FlowDto(@JsonProperty(Utils.FLOW_ID) final String flowId,
this.pathComputationStrategy = pathComputationStrategy;
this.targetPathComputationStrategy = targetPathComputationStrategy;
this.diverseWith = diverseWith;
this.diverseWithYFlows = diverseWithYFlows;
this.affinityWith = affinityWith;
this.loopSwitchId = loopSwitchId;
this.mirrorPointStatuses = mirrorPointStatuses;
Expand Down Expand Up @@ -377,7 +382,7 @@ public FlowDto(String flowId,
sourceVlan,
destinationVlan, 0, 0,
null, 0, null, null, null, null, null, null, pinned, null, detectConnectedDevices, null, null, null,
null, null, null, null, null, null, null);
null, null, null, null, null, null, null, null);
}

public FlowDto(FlowPayload input) {
Expand Down Expand Up @@ -412,7 +417,7 @@ public FlowDto(FlowPayload input) {
input.getDestination().getDetectConnectedDevices().isArp()),
input.getPathComputationStrategy() != null ? PathComputationStrategy.valueOf(
input.getPathComputationStrategy().toUpperCase()) : null, null, null,
null, null, null, null, null, null, null);
null, null, null, null, null, null, null, null);
}

@JsonIgnore
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,10 @@
* limitations under the License.
*/

package org.openkilda.model;
package org.openkilda.wfm.error;

import com.google.common.collect.Lists;
import org.junit.Assert;
import org.junit.Test;

public class LagLogicalPortTest {
public static final int OFFSET = 2000;
public static final int POST_1 = 1;
public static final int POST_2 = 2;
public static final int POST_3 = 3;

@Test
public void generateLogicalPortNumberTest() {
Assert.assertEquals(POST_1 + OFFSET,
LagLogicalPort.generateLogicalPortNumber(Lists.newArrayList(POST_1, POST_2, POST_3), OFFSET));
public class NoPoolResourcesAvailableException extends Exception {
public NoPoolResourcesAvailableException() {
super("Unable to allocate pool entity - all entity slots are busy");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ public abstract class FlowMapper {
@Mapping(target = "meterId", ignore = true)
@Mapping(target = "transitEncapsulationId", ignore = true)
@Mapping(target = "diverseWith", ignore = true)
@Mapping(target = "diverseWithYFlows", ignore = true)
@Mapping(source = "affinityGroupId", target = "affinityWith")
@Mapping(target = "mirrorPointStatuses", ignore = true)
@Mapping(target = "forwardLatency", ignore = true)
Expand All @@ -82,17 +83,19 @@ public abstract class FlowMapper {
/**
* Convert {@link Flow} to {@link FlowDto} with diverse flow ids and mirror paths.
*/
public FlowDto map(Flow flow, Set<String> diverseWith, List<FlowMirrorPath> flowMirrorPaths) {
return map(flow, diverseWith, flowMirrorPaths, FlowStats.EMPTY);
public FlowDto map(Flow flow, Set<String> diverseWith, Set<String> diverseWithYFlows,
List<FlowMirrorPath> flowMirrorPaths) {
return map(flow, diverseWith, diverseWithYFlows, flowMirrorPaths, FlowStats.EMPTY);
}

/**
* Convert {@link Flow} to {@link FlowDto} with diverse flow ids, mirror paths and flow properties.
*/
public FlowDto map(Flow flow, Set<String> diverseWith, List<FlowMirrorPath> flowMirrorPaths,
FlowStats flowStats) {
public FlowDto map(Flow flow, Set<String> diverseWith, Set<String> diverseWithYFlows,
List<FlowMirrorPath> flowMirrorPaths, FlowStats flowStats) {
FlowDto flowDto = map(flow);
flowDto.setDiverseWith(diverseWith);
flowDto.setDiverseWithYFlows(diverseWithYFlows);
flowDto.setMirrorPointStatuses(map(flowMirrorPaths));
flowDto.setForwardLatency(flowStats.getForwardLatency());
flowDto.setReverseLatency(flowStats.getReverseLatency());
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/* Copyright 2021 Telstra Open Source
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.openkilda.wfm.share.utils;

import java.util.Optional;

public interface PoolEntityAdapter<T> {
long getNumericSequentialId(T entity);

Optional<T> allocateSpecificId(long entityId);

Optional<T> allocateFirstInRange(long idMinimum, long idMaximum);

String formatResourceNotAvailableMessage();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/* Copyright 2021 Telstra Open Source
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.openkilda.wfm.share.utils;

import org.openkilda.wfm.share.flow.resources.ResourceNotAvailableException;

import com.google.common.base.Preconditions;
import lombok.Value;
import lombok.extern.slf4j.Slf4j;

import java.util.Optional;
import java.util.Random;

@Slf4j
public class PoolManager<T> {
protected final PoolConfig config;
private final PoolEntityAdapter<T> entityAdapter;

private final Random random = new Random();
private long lastId;

public PoolManager(PoolConfig config, PoolEntityAdapter<T> entityAdapter) {
this.config = config;
this.entityAdapter = entityAdapter;

this.lastId = config.idMaximum;
}

/**
* Allocate one entity.
*/
public T allocate() {
Optional<T> entity = allocateNext();
if (! entity.isPresent()) {
entity = allocateInChunk();
}
if (! entity.isPresent()) {
entity = allocateInFullScan();
}
if (! entity.isPresent()) {
throw new ResourceNotAvailableException(entityAdapter.formatResourceNotAvailableMessage());
}

T value = entity.get();
lastId = entityAdapter.getNumericSequentialId(value);
log.trace("Pool entity have been successfully allocated id=={}", lastId);
return value;
}

private Optional<T> allocateNext() {
long nextId = lastId + 1;
if (config.idMaximum <= nextId) {
return Optional.empty();
}
log.trace("Attempt to allocate pool entity by id == {}", nextId);
return entityAdapter.allocateSpecificId(nextId);
}

private Optional<T> allocateInChunk() {
long chunkNumber = selectChunkNumber(config.chunksCount);
long chunkSize = (config.idMaximum - config.idMinimum) / config.chunksCount;
long first = config.idMinimum + chunkNumber * chunkSize;

long last;
if (chunkNumber + 1 < config.chunksCount) {
last = Math.min(first + chunkSize - 1, config.idMaximum);
} else {
last = config.idMaximum;
}

log.trace("Attempt to allocate pool entity in chunk from {} till {}", first, last);
return entityAdapter.allocateFirstInRange(first, last);
}

private Optional<T> allocateInFullScan() {
log.trace(
"Attempt to allocate pool entity using full scan (idMin=={}, idMax=={})",
config.idMinimum, config.idMaximum);
return entityAdapter.allocateFirstInRange(config.idMinimum, config.idMaximum);
}

protected long selectChunkNumber(long chunksCount) {
if (chunksCount <= 1) {
return 0;
}
return Math.abs(random.nextInt() % chunksCount);
}

@Value
public static class PoolConfig {
long idMinimum;
long idMaximum;
long chunksCount;

public PoolConfig(long idMinimum, long idMaximum, long chunksCount) {
long size = idMaximum - idMinimum;
Preconditions.checkArgument(
0 < size, String.format(
"Resources pool must have at least one entry (%d(idMaximum) - %d(idMinimum) == %d)",
idMaximum, idMinimum, size));
Preconditions.checkArgument(
0 < chunksCount && chunksCount <= size, String.format(
"Invalid pool chunks count, the expression must be correct: 0 < %d <= %d",
chunksCount, size));

this.idMinimum = idMinimum;
this.idMaximum = idMaximum;
this.chunksCount = chunksCount;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,12 @@ port.up.down.throttling.delay.seconds.warm.up = 5
port.up.down.throttling.delay.seconds.cool.down = 10
port.antiflap.stats.dumping.interval.seconds = 60

lag.port.offset = 2000
lag.port.max.number = 2999
lag.port.pool.chunks.count = 10
lag.port.pool.cache.size = 128
bfd.port.offset = 1000
bfd.port.max.number = 1999
lag.port.offset = 2000

persistence.implementation.default = orientdb
persistence.implementation.area.history = orientdb
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ public void testStatusDetailsMapping() {
public void testMirrorPointStatusesMapping() {
Flow flow = buildFlow();

FlowDto flowDto = FlowMapper.INSTANCE.map(flow, new HashSet<>(), buildFlowMirrorPathList());
FlowDto flowDto = FlowMapper.INSTANCE.map(flow, new HashSet<>(), new HashSet<>(), buildFlowMirrorPathList());

assertNotNull(flowDto.getMirrorPointStatuses());
assertEquals(2, flowDto.getMirrorPointStatuses().size());
Expand Down
Loading

0 comments on commit 2ef830c

Please sign in to comment.