Fix MBS Broadcast content delivery crashes and Distribution Session bugs - #71
jordijoangimenez wants to merge 17 commits into
Conversation
|
@jordijoangimenez wrote:
I don't agree with this solution as it prevents the use of jumbo frames between MB-UPF and gNodeB (we use jumbo frames in our lab configuration). This is why the function used the interface MTU as fallback. This allows this setting to be handled by configuring the interface MTUs correctly for your setup. |
|
@jordijoangimenez wrote:
Actually if you read TS 26.502 Table 6.1-1: "When the push-based object acquisition method is provisioned, the set of Object acquisition identifiers shall be empty." So if an objAcquisitionIdPush is provided in PUSH+SINGLE mode then that is an error. So the original return code was correct (i.e generate an error if it's set). |
|
@jordijoangimenez wrote:
I've looked through TS 29.581 Clause 5.8.8.2 and can find nowhere where it says the notifications have to be delivered reliably. It only states that the response from the MBSF should be 204 for success, 307 or 308 for redirection and one of the 4XX or 5XX error codes from TS 29.500 for errors. It says nothing about what action should be taken on reported failure, failure to connect or timeout. The 307 and 308 responses should cause the request to be reissued to a new URL. For 308 the new URL should replace the old notification URL. (But we never implemented redirection) The semantics of any 4XX error code are that the client made a mistake and should not repeat the same request, so a 4XX response should prevent the events from being sent again. For 5XX responses, the semantics are that there was a server error that may clear and so the request may be repeated again at some point in the future. The client should delay before trying again to avoid rapid retries. So there is an argument for repeating the notifications when a 5XX is received. This also risks indefinite attempts to resend, there probably should be a resend attempts limit. As far as I can see the current behaviour of trying once and then forgetting it does not contradict TS 29.581. |
@jordijoangimenez: Please note that this bug is fixed more comprehensively by @davidjwbbc on PR #72. |
@jordijoangimenez: This issue was also mopped up by @davidjwbbc in PR #72. |
|
Thank you Richard, as soon as possible I will clean up this branch to avoid duplications and we can cherry pick what's useful rather than adopting the full PR |
…nch needs Problem This branch's FLUTE-layer obligations depend on work that is on rt-libflute's own feature branch and not in any release tag: the TS 26.346 annex L.6 profiled FDT schema, the scheme-specific FEC OTI, suppression of Transfer-Length under the MBMS Download Profile, the split expiry setter, and the RFC 5053 Raptor scheme. The wrap pointed at a revision carrying none of it. [code-derived] Basis No clause governs a dependency pin. code-derived only. Raised by Building this branch against the dependency it actually needs. Change Advances subprojects/rt-libflute.wrap to the commit on 5G-MAG's own feature/raptor-raptorq-fec carrying that work, and the rt-common-shared submodule to its consolidated tip. The wrap comment records why a tag cannot be used yet and what has to happen before it can be. Verification T0: the subproject is fetched and the tree builds against it. The behaviour that depends on these revisions is verified by the commits that use it, not here. Not in this change Moving the rt-libflute pin to a tag, which needs a 5G-MAG release carrying the work the comment names. No source change.
Problem
The Nmbstf_DistSession surface answered a wrong Content-Type with 400 rather
than 415, never checked a client's Accept header, never bounded a request
body, and one PATCH handler parsed its body with no Content-Type check at
all. [code-derived]
Basis
TS 29.500 V18.10.0 table 5.2.7.1-1 marks 415 mandatory for POST and PATCH,
406 mandatory for GET, and 413 mandatory where a body is accepted. Its
table 5.2.7.2-1 defines no named cause for 415, so the numeric status is
constructed directly, as this file already does for 405 and 501.
TS 29.581 (TS29581_Nmbstf_DistSession.yaml) requires
application/json-patch+json on both PATCH operations, not
application/merge-patch+json.
The individual resource's representation is DistSession for both the GET and
the PATCH on /dist-sessions/{distSessionRef}; CreateReqData is the request
body of the collection POST only, so an RFC 6902 pointer addresses the
DistSession.
Raised by
reading the authority during this work, and observation of a live activation
Change
Adds NfServer::acceptsMediaType() and answers 406 when the client's Accept
header cannot take application/json; answers 415 for an unexpected
Content-Type on POST and PATCH; adds request_too_large() and a configurable
maxRequestBodySize, answering 413; checks the Content-Type on the
subscription PATCH. Applies a JSON Patch to the DistSession the stored
CreateReqData holds, rebuilding the CreateReqData around the result, so the
pointer a conformant peer sends resolves.
Verification
T2: the end-to-end demo activates a Distribution Session, which the previous
patch target rejected outright.
Not in this change
Authentication on this surface.
…piry time Problem DistributionSessionSubscription parsed expiryTime into m_expiryTime and never compared it against the clock anywhere, so a subscription carrying one kept receiving notifications until its Distribution Session was deleted or the process exited. [code-derived] Basis TS 29.581 V18.6.0, table 6.1.6.2.5-1, expiryTime row: "When present in the subscription creation request, it shall indicate the time up to which the subscription is desired to be kept active and after which the subscribed events shall stop generating notifications." Raised by reading the authority during this work Change Adds a per-subscription timer that removes the subscription when its expiryTime passes, scheduled wherever m_expiryTime is set and cancelled from the destructor. The callback does not remove the subscription directly: it runs from a timer that subscription owns, so it pushes LocalEvents::SUBSCRIPTION_EXPIRED carrying the two ids as plain strings and DistributionSession::processEvent() performs the removal off that call stack, the deferred dispatch SEND_NOTIFICATION already uses. A copy schedules its own timer, a callback being keyed to one object; a move takes the original's over. Timer-pool exhaustion is logged and leaves the subscription without a timer rather than crashing. Verification T2: the end-to-end demo runs with it in place, MBSTF logging no fatal or assertion lines and no timer-creation failure. No test scaffolding exists for this surface. Not in this change An expiryTime in a subscription response, which this MBSTF does not set.
…by earliest deadline Problem Four related defects in how object metadata is held and ordered. ObjectStore::Metadata hand-writes its copy constructor, move constructor and both assignment operators, and three members were missing from them. The availability window (m_availabilityStartTime, m_availabilityEndTime) was absent from all four paths, so any copy taken out of the store lost it: build/tests/testObjectStore aborted with std::bad_optional_access. [observed: test output] The entity tag (m_entityTag) was absent from all four as well, which is quieter and not caught by any test: PullObjectIngester passes it to the conditional re-fetch, so losing it turns an If-None-Match into an unconditional GET and re-downloads an unchanged segment, and ObjectCarouselPackager copies it into the FLUTE file description, which then carries an empty ETag. [code-derived] PullObjectIngester::IngestItem has the same shape of defect in one direction only: its copy constructor carries the availability window and its move constructor did not. Items are moved on the ordinary queueing paths, so an item reaching the fetch queue by any of those routes lost what the copy path preserved. [code-derived] Nothing ordered the packaging queue by when an object has to arrive. [code-derived] Basis TS 26.517 V18.6.0, clause 6.2.3.5: "The MBSTF shall transmit each object in the object list such that the last packet of the delivered FLUTE transmission object (including any FEC recovery packets, when configured) is available at the MBSTF Client no later than its availability start time." The deadline a PackageItem carries is that availability start time, so ordering the queue by it is what implements the clause. The copy and move defects are code-derived: no clause governs a class's own copy semantics, only that a value the class stores survives being copied. Raised by The availability window by running the test suite, which aborted. The entity tag and the ingest item's move constructor by reading the same four paths afterwards, looking for the same mistake again. Both times it was there. Change Adds the missing members to all four ObjectStore::Metadata copy and move paths, in declaration order so initialisation order matches and -Wreorder stays quiet, and to IngestItem's move constructor. Adds earlierDeadlineFirst(), a named predicate rather than an inline comparator so the ordering can be tested directly, the queue itself being private and fed only through a live packager. An item with no deadline sorts after every item that has one: nothing is known about when it must arrive, so it cannot displace an object that does have a stated time. Verification T1: four suites executed and passing -- testObjectStore 16 cases, testPullObjectIngester 19, testObjectListPackager 7, testSubscriberSubscription. Both metadata defects were confirmed to be caught, not merely covered: testObjectStore aborts without the availability fix, and reports "copy=[] move=[] copy-assign=[] move-assign=[]", 15 pass 1 fail, without the entity tag fix. Not in this change IngestItem's own copy and move constructors have no direct test, which is the gap that let the move-constructor defect survive. ObjectStore::Metadata::operator== still does not compare m_entityTag; the equality semantics of the class are not touched here.
…ransmitter Problem The AL-FEC configuration a Distribution Session provisions reached no transmitter, so every session was sent unprotected however it was provisioned. FecOtiHelper, which converts the session's FECConfig into the Transmitter-level FEC OTI, was present in the tree but never listed in src/mbstf/meson.build, so it was never compiled. Neither ObjectListPackager nor ObjectCarouselPackager passed a FEC OTI or a redundancy level to LibFlute::Transmitter, which has taken both since the pinned revision, and no controller supplied a FECConfig to either packager. [code-derived] Basis TS 26.517 V18.6.0, clause 6.2.1: "If FLUTE [12] is used to realise the Object Distribution Method, the MBS Distribution Session shall conform to the MBMS Download Profile as defined in clause L.4 of TS 26.346 [7] with the additional requirements in clause 6.2 of the present document." TS 26.346 V18.2.0, clause L.4.7 "Other aspects of FLUTE delivery": "Regarding Application Layer FEC support, the two FEC schemes referenced in this specification, the Compact No-Code FEC scheme as specified in RFC 3695 [13], and the Raptor FEC scheme as specified in RFC 5053 [91] are optional to implement by the BM-SC and mandatory to support by the UE." Those two together decide which schemes may be sent: the profile clause 6.2.1 selects admits Compact No-Code and Raptor, and no other. RaptorQ (RFC 6330) is not among them. Raised by Reading the branch while checking the review comments on pull request #71. Change Adds FecOtiHelper to the build. ObjectListPackager and ObjectCarouselPackager convert the session's FECConfig with fecOtiFromFecConfig() and pass the resulting FEC OTI and redundancy level to LibFlute::Transmitter. A scheme the profile does not admit throws and is reported as a packaging failure for that session rather than silently downgraded: a session sent unprotected when it asked for protection is a worse outcome than a visible failure. ObjectCarouselPackager gains the FEC parameter ObjectListPackager now also carries, defaulted so no other call site changes, and all controllers supply distributionSession().getFecInformation(). Verification T1: tests/test_FecOtiHelper.cc, 8 cases, all passing, covering absent configuration, a null shared pointer, Compact No-Code yielding no FEC OTI, Raptor yielding a Raptor OTI carrying the requested overhead, and refusal of RaptorQ, an unknown scheme and a negative overhead. The suite is new: FecOtiHelper had never been compiled, let alone tested, and one case caught a real error while being written, an incorrect Compact No-Code URN. All five suites pass, 50 cases. T2: the demo delivers unchanged with this build. That run does NOT exercise the populated path: the demo provisions no FEC, so it evidences no regression, not that FEC transmission works. Not in this change No end-to-end evidence that a FEC-protected session is transmitted and decoded. Nothing here verifies rt-libflute's own Raptor encoding, and no receiver in this project has been shown to decode a protected session. The conformance record must not claim AL-FEC works on the strength of this commit.
… DASH handler its own manifest URL Problem ObjectCollectionController.cc/.hh, 368 lines serving the COLLECTION object distribution operating mode, were present in the tree but never listed in src/mbstf/meson.build, so they were never compiled. The class self-registers with ControllerFactory at file scope, so leaving it out of the build means the registration never runs and a Distribution Session provisioned with objDistributionOperatingMode COLLECTION finds no controller at all. Once added to the build it did not compile: four errors, all from ObjectController::objectStore() having become std::shared_ptr<ObjectStore> rather than a reference. The same file listed ObjectController.cc twice where the second entry should have been its header, so the header was not tracked as a dependency. [code-derived] Separately, DASHManifestHandler::nextIngestItems() declared manifest_url and never assigned it, so both comparisons against it tested the empty string and m_refreshMpd was never set: a re-fetched MPD was ingested without the handler being told its own manifest had changed. [code-derived] Basis COLLECTION is one of the four values ObjDistributionOperatingMode defines (src/mbstf/openapi/model/ObjDistributionOperatingMode.h: SINGLE, COLLECTION, CAROUSEL, STREAMING), so the API accepts a session the build then has no controller for. That is the operative basis and it is code-derived. TS 26.502 V18.6.0, clause 4.5.10, first paragraph: "An object manifest describes a set of objects to be distributed in an MBS Distribution Session that is provisioned in OBJECT_COLLECTION or OBJECT_CAROUSEL operating mode." confirming the mode is one the specification defines rather than a local invention. The fuller description of the mode is in annex B, clause B.2.1, which is informative. Raised by Reading the branch against its own build file while checking the review comments on pull request #71. Rule 14: purpose established before any change, and the code is built rather than removed, because the operating mode it serves is one the model defines. Change Adds ObjectCollectionController to src/mbstf/meson.build and corrects the duplicated ObjectController.cc entry to name the header. Dereferences the object-store shared pointer at the four sites that still treated it as a reference, matching what ObjectCarouselController already does, and checks each dynamic_pointer_cast result before use rather than letting a std::runtime_error leave the controller, where the surrounding try catches only std::out_of_range and one bad ingest response would end the process and every other Distribution Session with it. Initialises manifest_url from the manifest's own fetched URL, the value addMPDRefreshToExtraPullObjects() keys its refresh entry on, and makes it const. Verification T0 for the new code: both files compile with no new warnings and COLLECTION is present in the linked binary, so the controller registers. T1: all five suites pass, 50 cases. T2: the demo delivers with this build, gNB carrying MRB1 and MRB2 and the UE logging CRC-OK broadcast decodes and MCCH receptions. Not in this change Any verification of COLLECTION behaviour. This builds and registers the controller; it does not establish that the mode works, and the conformance record must not claim it. The demo uses STREAMING and no test covers COLLECTION. Nothing here covers the MPD-refresh recognition path either.
…roject cannot follow
Problem
Comments across this branch narrated how the code came to be rather than what it does, and
pointed at material a reviewer cannot read. Three shapes.
A "BUG FIX:" prefix followed by what the code used to do ("this used to be static", "no DELETE
branch existed at all", "the else branch used to unconditionally call gw->write_pdu_mch()"). That
belongs in a commit message; in a comment it dates immediately and tells a reader nothing about
the code in front of them.
References to this project's own internal process framework by number: "(rule 12)", "(S12)",
"see rule 14". Those numbers name nothing in this repository.
Pointers into a separate, private repository: "see Standards2Deployments/projects/rt-mbs/...",
"see the findings register", "the register's own item 4 finding". A reviewer cannot open any of
them. [code-derived]
Basis
No clause governs a comment.
code-derived only.
Raised by
Reading the branch as a reviewer outside this project would.
Change
Comments now state the behaviour, the requirement or the invariant, keeping every specification
citation and every stated reason. Where a comment's only content was its history, the underlying
rule it was protecting is stated instead: "X was never freed" becomes "this context owns X and
must free it on removal". Internal rule numbers are replaced by the reasoning they stood for,
usually that no clause and no configured value fixes a given bound. Private-repository pointers
are removed, with the substance they referred to summarised in place where it was load-bearing.
Deliberately NOT changed: the numbered rules in src/mbsf/MultipartMime.cc's Q-encoding comment.
Those are RFC 2047 section 4.2's own rules 1, 2 and 3, not this project's, and an automated pass
over the phrase "rule N" would have silently corrupted a correct citation.
Verification
T0 for the shape of the change: every hunk is a comment or documentation line, confirmed by
filtering the diff for added lines that are not comments, which comes to zero in all nine
repositories. T1 where a build exists: open5gs, rt-mbs-function and rt-mbs-transport-function
all build clean afterwards, and the shell scripts and JSON touched pass bash -n and json.load.
Not in this change
No behaviour, in any repository. No commit message is rewritten; the history those comments
described stays in the log, which is where it belongs.
…the configured failure limit Problem ObjectManifestController refetched an object on every ingest failure, for any non-PUSH session, with no bound of any kind: no deadline check, no attempt count, no backoff. One object that could never be fetched was therefore retried forever. Observed on a live run: 59554 ingest failures in a single session, none of which stopped anything. [observed: run/logs/mbstf.log] The session-wide limit that exists could not catch it. ObjectController counts consecutive failures and deactivates the session at consecutiveIngestFailuresBeforeDeactivate, but the count is per Distribution Session and is reset by ObjectStore's ObjectAdded/ObjectUpdated events. The same run recorded 4985 successful object fetches, so the counter was reset continually and never reached 5 while one object failed 59554 times. A bound that a healthy session resets cannot bound a single unfetchable object. [code-derived: ObjectController.cc, the reset at the ObjectAdded/ObjectUpdated branch] Basis TS 26.517 V18.6.0, clause 6.1.2, object manifest parameter latestFetchTime: "The MBSTF shall fetch the object no later than this UTC timestamp." So an object whose latest fetch time has passed must not be fetched again, whatever has happened before. The same parameter's description governs the other case: when latestFetchTime is absent "the object shall be present at its origin ... and the MBSTF may fetch it at a time of its choosing", which sets no bound at all. A limit there therefore rests on a configuration option the operator sets, not on a clause. Raised by Reading the authority for what governs a failed object fetch, after a defensive guard written earlier was measured and found not to bound the loop (it rejected the fetch, which produced the failure event, which triggered the refetch). Change The refetch decision now refuses two cases. An object whose latestFetchTime has passed is not refetched, which is the clause above. An object with no latestFetchTime is refetched until consecutiveIngestFailuresBeforeDeactivate consecutive failures of that object, the operator's own existing option applied per object rather than per session. PullObjectIngester::IngestItem gains the per-object counter, carried through all four of its copy and move paths, since items are copied and moved on every queueing path. Verification T1: tests/test_PullObjectIngester.cc, 27 cases passing, 5 of them new: the counter starts at zero, accumulates, and survives copy and move; a deadline in the past, a deadline in the future and no deadline are each distinguished. Confirmed discriminating, not vacuous: with the copy and move carry deliberately reverted the two survival cases fail and the suite reports 25 pass, 2 fail. All five MBSTF suites pass, 58 cases. T2: live demo. Ingest failures fell from 59554 to 15, and the log shows three refusals, each "5 consecutive fetch failures reached the configured consecutiveIngestFailuresBeforeDeactivate limit of 5": three objects, five attempts each, then stopped. Delivery is unaffected, the UE logging 15711 CRC-OK broadcast decodes on the same run. Not in this change Why the ingest URL is corrupted in the first place. A one-byte corruption at index 0 of IngestItem's own copy of the fetched URL is still unexplained and still open; this bounds the retry loop that made it harmful, it does not fix it. The refusal log line shows an empty object id for the affected items, which is consistent with that corruption and is not diagnosed here. No backoff is introduced: the clause names no interval and no configuration option supplies one.
…a released reference
Problem
ObjectStore::getMetadata() takes the store mutex, returns a reference into the store, and releases
the mutex as it returns. Everything the caller then reads through that reference is unsynchronised.
Metadata holds std::strings and ObjectStore::updateMetadata() move-assigns them, so a caller copying
a string while its data pointer and length are being reassigned builds a string from two different
states of the same object.
This is the cause of a corruption that had been open and unexplained: an ingest URL arriving 55 bytes
long with byte 0 zeroed and bytes 1 onward intact, and an object id arriving empty. The store's own
copy was correct at that moment, which is what made it look impossible.
Confirmed by ThreadSanitizer, both sides named, holding different mutexes:
Write of size 8 by thread T4 (mutexes: write M0)
std::string::_M_data(char*)
ObjectStore::Metadata::operator=(Metadata&&)
ObjectStore::updateMetadata(...)
PullObjectIngester::doObjectIngest()
Previous read of size 8 by thread T6 (mutexes: write M1, write M2)
std::string::_M_data() const
std::string::basic_string(const std::string&)
PullObjectIngester::IngestItem::IngestItem(ObjectStore::Metadata const&, ...)
PullObjectIngester::fetch(...)
ObjectManifestController::workerLoop(...)
The same call site also chained keepAfterSend() and compressedSend() onto that reference, mutating
the live store entry with no lock held. [observed, then code-derived]
Basis
No clause governs a component's internal locking.
code-derived and observed only.
Raised by
Running the component under ThreadSanitizer, after three earlier passes narrowed the corruption to
the interval between the copy and the end of IngestItem's constructor without identifying any
writing instruction. Reading the code had already shown the reference outlives the lock; the
sanitiser is what turned that from a hypothesis into the cause.
Change
Adds ObjectStore::takeMetadataForIngest(), which takes the lock, applies the two marks and returns
the metadata by value before releasing it, so the copy the ingest list stores is made while the
entry cannot be mutated. PullObjectIngester::fetch() uses it instead of copying through
getMetadata()'s reference.
Verification
T2: live demo, MBSTF built with -Db_sanitize=thread, same scenario before and after.
Before: 162 data race reports, 12 of them naming IngestItem's constructor.
After: 64 data race reports, ZERO naming IngestItem's constructor.
Delivery unaffected on the same run: gNB carrying MRB1 and MRB2, 2036 CRC-OK broadcast decodes and
118 MCCH receptions at the UE, and no ingest failures at all.
T1: all five MBSTF suites pass, 58 cases.
Not in this change
The other 64 races ThreadSanitizer still reports. They are real and are recorded, but each needs its
own diagnosis and none is this defect. getMetadata() itself is left in place: its remaining callers
read under conditions this commit has not examined, and changing its contract is a wider refactor
than the defect requires.
…cument
Problem
Running tools/verify-citations.py over this branch's own changed files reported
citations it could not confirm. Each was a comment whose quoted sentence differed
from the document it named. [code-derived]
Basis
No clause governs how a comment is written. The defect is that a quotation did not
match its source, which is checkable without any specification claim.
code-derived only.
Raised by
Running the citation checker across this branch for the first time. [rule 13]
Change
Quotations now reproduce contiguous source text. Where an earlier comment joined
two sentences with an elision, each is quoted separately. Where it inserted an
editorial gloss inside the quotation marks, the gloss moved outside them. Where a
specification writes a value inside its own quotation marks (a status code, a state
name, a file extension), that fragment is named without quotation marks rather than
nested, and the comment says why, so nobody restores them.
Two citations that sat inside runtime strings moved into comments beside them: a
full document identifier in a log or exception message is read as a citation by the
checker, which then matches the next string literal in the file.
No behaviour changes. Comments only.
Verification
T1: tools/verify-citations.py reports no unconfirmed citation across this branch's
changed files.
T0: builds clean.
Not in this change
Citations naming a document the local corpus does not hold. Those remain unchecked
and are listed in the project's own specification index.
84d878a to
88e9fa0
Compare
…aves the host Problem FLUTE encoding symbols are sized from getsockopt(IP_MTU) on a socket to the distribution session's ingress. Where the MBSTF and the ingress point are co-located that destination is one of the host's own addresses, so the kernel routes it over loopback and answers 65535. Symbols were sized at 65441 bytes and every object larger than one of them left as a datagram nothing downstream carries. Measured on the rt-mbs-examples broadcast demo, OBJECT_STREAMING session: the FDT advertised FEC-OTI-Encoding-Symbol-Length="65441"; the client was told about 663 distinct TOIs and received data packets for three of them, TOI 0 (the FDT itself), 2 and 4. Every media segment TOI got its FDT Instance and then no data. [observed, code-derived] Basis RFC 5651 section 6.1: "However, network efficiency considerations recommend that the sender uses an as large as possible packet payload size, but in such a way that packets do not exceed the network's maximum transmission unit size (MTU), or when fragmentation coupled with packet loss might introduce severe inefficiency in the transmission." RFC 5651 is the LCT building block TS 26.346 V18.2.0 lists as reference [119]. No clause fixes a number. mbstf.pathMtu is the operator's setting and is documented in mbstf.yaml with a default, which is what rule 12 requires of a bound. Raised by Counting, per TOI, the data packets a client received against the TOIs its FDTs advertised, while chasing why no media segment reached a player. The shape of the fix comes from review by David Waring on pull request #71, who pointed out that clamping a discovered MTU would prevent the jumbo frames his lab runs between the MB-UPF and the gNodeB. [rule 13] Change A discovered MTU is used as it stands whenever the destination is not one of this host's own addresses. A deployment configuring jumbo frames on its interfaces gets them; nothing here caps what the operator set up. get_path_mtu() reports, through via_loopback, whether the destination is a local address. It tests the destination against getifaddrs() rather than for the 127/8 prefix, because a co-located MB-UPF is commonly reached on the address of a real interface -- in the demo, a veth -- which loops back without looking like a loopback address. Only in that case, where there is no path to measure, is Context::pathMtu used, and the substitution is logged with both numbers. The four controllers sequence the discovery before reading the flag rather than nesting the calls: the order function arguments are evaluated in is unspecified. Verification T1: meson test, the five rt-mbs-transport-function suites pass. The other failures in that run are open5gs and libmpdpp subproject tests, untouched by this change. T2: run scripts/mbs-broadcast-demo/start-all.sh in rt-mbs-examples. mbstf.log records "The route to this session's ingress is loopback, so its 65535 byte MTU is not the path to a receiver; sizing FLUTE symbols for the configured 1500 byte path MTU instead", the FDT then advertises FEC-OTI-Encoding-Symbol-Length="1406", and the client receives the announcement bundle, both initialisation segments, the manifest and media segments. Before this change it received data for three TOIs out of 663. Not in this change GTP_HEADER_SIZE in common.hh, which is 2. Whether that is the right allowance needs TS 29.281, which is not held; halted under rule 2.
Problem
In OBJECT_STREAMING every media segment is transmitted several times, each time under a
different TOI. Counting distinct TOIs against distinct Content-Locations in the FDTs a
receiver was given: 4.0 TOIs per object on average, and up to 15.
15 x chunk-stream0-00047.m4s 11 x manifest.mpd 10 x chunk-stream0-00051.m4s
This is not redundancy. A receiver cannot combine symbols across TOIs, so each copy is a
separate object that is independently incomplete, and the copies consume the bearer the
first copy needed. Measured live on the rt-mbs-examples broadcast demo, the effect is that
no media segment ever completes at the client: symbol IDs arrive with gaps throughout, for
example SBN 1 ID 7, 11, 12, 16, 19, 20, 22, 26, 27, 30, 32, 33, 42, while the radio itself
delivered 33382 of 33455 grants with zero CRC failures. [observed]
DASHManifestHandler::nextIngestItems() takes every segment the MPD currently advertises and
clamps any whose availability start has passed to the present, so a segment stays in the
candidate set for the whole of its availability window. Nothing records that it has already
been sent. The ObjectStore cannot answer for it either: ObjectStreamingController leaves
Metadata::keepAfterSend() at false, so ObjectController deletes the object as soon as it is
sent, findMetadataByURL() then misses, and a second object is created for the same URL,
which the packager sends under a second TOI. [code-derived]
Basis
RFC 3926 clause 3.1: "Note that each object is associated with a unique TOI within the scope
of a session."
Sending one file under several TOIs therefore presents it as several objects, and a receiver
has no basis on which to combine their symbols. An MPD advertising a segment states that a
client may still fetch it, not that it still needs transmitting; no clause requires a segment
to be sent more than once.
Raised by
Counting TOIs per Content-Location while establishing why no object completed at a
receiver, after the path-MTU defect was fixed and the segments started arriving. [rule 13]
Change
DASHManifestHandler remembers the media segment URLs it has already handed to the ingester
and skips them on later passes. The set is pruned against the current manifest on every
pass, so it holds at most one entry per segment the MPD still advertises.
Only segments the MPD itself advertises are suppressed. The MPD refresh and the
initialisation segments, which come from m_extraPullObjects, are meant to repeat and are
deliberately not recorded.
keepAfterSend() is untouched. Removing it from ObjectManifestController was correct: an
OBJECT_STREAMING object genuinely should not be retained once sent, and the carousel sets
it for itself. What was missing is a record of what has been sent, which is added here
rather than by retaining objects that are no longer needed.
Verification
T0: builds clean.
Not in this change
Anything in the carousel path, which retains its objects and repeats them under one TOI.
The gNB-side per-slot MBS scheduling limits, which are a separate matter in
srsRAN_Project_mbs.
88e9fa0 to
2958c1e
Compare
|
@davidjwbbc You were right, and thank you — capping a discovered MTU would break your jumbo-frame lab, and that solution is off the branch. What replaced it keeps your mechanism. A discovered MTU is now used exactly as it stands whenever the destination is not one of the host's own addresses, so interface MTUs remain the way this is configured and jumbo frames between MB-UPF and gNodeB are passed through untouched. The one case it treats differently is a destination that is a local address, where the kernel routes over loopback and answers 65535. There is no path to measure there, so Why it matters: with the loopback answer, symbols were sized at 65441 bytes and a receiver was told about 663 TOIs but got data packets for three of them. RFC 5651 §6.1 asks that the sender not exceed the path MTU, which a loopback figure is not. If your lab has the MB-UPF on the same host as the MBSTF, I would like to know — that is the one configuration where this now diverges from pure discovery. |
|
@davidjwbbc You are right on both, and both changes are off the branch.
StatusNotify retry. You are right that I overstated TS 29.581 — I cited it as obliging reliable delivery and it does not say that. The retry change is off the branch, and the current try-once behaviour stands. Your reading of the status codes is the useful part and I have not tried to implement it here: 4xx means the client should not repeat the same request, 5xx may be retried after a delay, and 307/308 should be reissued to the new URL, with 308 replacing the stored one. That plus a retry limit is a real piece of work and belongs in its own change, raised against a maintainer's judgement rather than smuggled in behind a wrong citation. |
|
@rjb1000 Both duplications are gone. The branch was rebuilt on
Nothing was cherry-picked; taking his versions and rebasing was cleaner than picking through mine, which is what I said I would do. The branch has been rewritten since your review, so the commits you read are no longer the ones on it. It now stands at 12 commits, and the description has been rewritten to match. |
…s run
Problem
meson setup failed on any checkout that had not been built before:
src/mbstf/meson.build:66:29: ERROR: File openapi/model/FECConfig.cc does not exist.
FECConfig.cc is generated, not tracked: src/mbstf/.gitignore excludes the whole
openapi directory and the generator writes its 153 files at configure time.
meson's files() checks existence when it is evaluated, and the test source list
naming those two generated files sat above the generator invocation, so a fresh
checkout failed and only a tree left over from an earlier build succeeded.
[observed: clean clone of this branch; code-derived: meson.build ordering]
Basis
No clause governs this; it is this repository's own build definition.
[code-derived]
Raised by
Building this branch from a clean clone, after a user report that third-party
developers must be able to build it themselves.
Change
test_source_fec_oti_helper is defined after the generator has run and written
.openapi.srcs. Its only consumer is tests/meson.build, which the top-level
meson.build enters after src, so nothing else moves.
Verification
T2: a clean clone of this branch builds through to open5gs-mbstfd.
Not in this change
The generator's own failure reporting, which is fixed in rt-common-shared.
…se branch Problem The submodule pointed at a commit that exists only on a local 3GPP-Rel18, while rt-mbs-function pinned the same submodule to feature/mbs-compliance-fixes. The two network functions were therefore built against different versions of a shared library, and this one missed the MBS work and the generator's failure-reporting fix. Because that commit was never pushed, the pin also resolved only on the machine it was made on. [code-derived: the two gitlinks] Basis No clause governs this; it is this repository's own dependency pin. [code-derived] Raised by User question asking why this repository was not pinned to what it needs. Change The submodule advances to the MBS branch's tip, which carries the ACCESS_TOKEN_CLAIM_MISSING change this repository was pinned to 3GPP-Rel18 for, and adds the generator fix and the HTTP server work. Both network functions now build against the same rt-common-shared. Verification T2: a clean clone of this branch builds through to open5gs-mbstfd against that tip. Not in this change Publishing the rt-common-shared branch this points at.
Problem The README opened with a bare heading and two badges, had no At a glance table, and its clone command named no branch and cloned into a path under $HOME. It did not say that the build fetches the 3GPP 5G APIs over the network, nor what the two 5G-MAG libraries it pulls in supply. [code-derived] Basis No clause governs a repository's own README. The structure is the house baseline used by rt-cmmf-encoder. [code-derived] Raised by User report that each repository must state what to install, clone, build, install and run. Change The baseline's header and sections, including Specification with the versions this is built against, and Dependencies naming what rt-common-shared and rt-libflute supply. A section on the 5G APIs fetch: that it needs network access and Java, that forge.3gpp.org serves an incomplete certificate chain, and how to install the missing intermediate rather than disable verification. Verification T2: a clean clone built through to open5gs-mbstfd following these instructions, including the certificate step, which is what makes the generation succeed. Not in this change The generator and the build definition, fixed separately.
Problem
The submodule pinned a commit that no longer exists on rt-common-shared's MBS
branch. The branch was reauthored before being published, which changed every
commit hash on it, and this pin still named the pre-rewrite one. A clean clone
therefore failed at submodule init:
fatal: remote error: upload-pack: not our ref 0bab3d165085287320ab7742941f6ba1902e5afe
[observed: clean clone of this branch after rt-common-shared was published]
Basis
No clause governs this; it is this repository's own dependency pin.
[code-derived]
Raised by
Rehearsing the documented clone-and-build steps against the published
repositories.
Change
The submodule advances to the published tip of rt-common-shared's MBS branch.
Its content is unchanged from the commit previously pinned; only the hashes
differ.
Verification
T2: a clean clone of this branch now initialises its submodule from the public
remote and builds.
Not in this change
Anything in rt-common-shared, whose content this only follows.
Problem This repository pinned a commit on rt-libflute's Raptor line while rt-mbs-client pinned a release tag, so the two components of the same delivery chain were built against different versions of the same library. Neither pin carried what the other component needed, and the client's did not carry what the client itself needed. [code-derived: the two pins] Basis No clause governs this; it is this repository's own dependency pin. [code-derived] Raised by User request that both components be pinned so a third party can build and run the demo end to end. Change revision = feature/mbs-profile, which both this repository and rt-mbs-client now use. It carries the FLUTE-layer obligations this repository depends on -- the TS 26.346 annex L.6 profiled FDT schema, the scheme-specific FEC OTI, suppression of Transfer-Length under the MBMS Download Profile, the split expiry setter and the RFC 5053 Raptor scheme -- alongside the receiver work the client needs. Every commit on it is already on an open pull request; that branch's own merge commit records which, and what was resolved. The wrap comment is rewritten to say this rather than describing the old pin. Verification T2: a clean clone of this branch, with the new pin, builds through to open5gs-mbstfd. Not in this change The pull requests that branch mirrors, which are unaffected.
MBS Transport Function: object distribution over FLUTE, and the SBI behaviour TS 29.580 and TS 29.500 require.
Type: compliance fix + crash fix
Baseline: TS 26.517 V18.6.0, TS 26.502 V18.6.0, TS 29.580 V18.8.0, TS 29.581 V18.6.0, TS 26.346 V18.2.0, RFC 3926, RFC 5651.
Reproduce: 5G-MAG/rt-mbs-examples
feature/mbs-compliance-fixes,scripts/mbs-broadcast-demo/start-all.sh.Closes #73