Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems in cache for Context Source Registrations #1692

Open
ravaga opened this issue Oct 11, 2024 · 2 comments
Open

Problems in cache for Context Source Registrations #1692

ravaga opened this issue Oct 11, 2024 · 2 comments
Assignees
@kzangeli
Labels
bug Something isn't working

Comments

@ravaga
Copy link

ravaga commented Oct 11, 2024

I'm working in the development of an application which manages (creates, updates and deletes) Context Source Registrations (CSRs) in Orion-LD, so this application sends several request to the /ngsi-ld/v1/csourceRegistrations endpoint of the broker.

While performing some tests, I came across an issue related with the internal cache of the broker. Sometimes, when I create new CSRs and I send a GET request to retrieve all the CSRs present in the broker, the newest registrations aren't returned. However, if I make a query inside MongoDB (inside the registrations collection of the orion DB) the registrations are stored. Therefore, this must be an issue related with the internal Orion-LD cache. Furthermore, if I use the options=fromDb parameter to retrieve the CSR, the broker returns it, but it returns a 404 if the parameter isn't used.

In addition, if I try to recreate the same CSRs (using the same IDs), the broker returns a 409 because it looks for the registration in MongoDB first. Furthermore, the CSR is deleted without any problem.
@kzangeli kzangeli self-assigned this Oct 11, 2024
@kzangeli kzangeli added the bug Something isn't working label Oct 11, 2024
@ravaga
Copy link
Author

ravaga commented Oct 17, 2024

Let me add an specific example (in the context of the aerOS project) to try to shed light on the issue.

We have an environment composed of 3 Orion-LD instances (A,B and C) and we want to create the needed CSR in each broker that point to the other 2 brokers to be able to retrieve all the entities of a certain type by making a single GET /entities request to any of those brokers. We use 3 different types of CSRs, so each broker must have a total of 6 CSRs (3 pointing to each other broker) in each broker. The creation and deletion of these CSRs is managed by the custom Federator component.

First, we deploy the broker A and then we create one entity of type Domain, other of type Organization, another of type LowLevelOrchestrator, some of type InfrastructureElement and some of type Organization. This is the first broker, so we don't have to create CSRs yet.

Then, we deploy the broker B and create a new set of entities (with the same types as in broker A). This is the second deployed broker, so the Federator creates these 3 CSR in the broker A:

{
        "id": "urn:aeros:federation:b:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}

And these 3 CSRs in the broker B:

{
        "id": "urn:aeros:federation:a:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}

Now we have 2 CBs in our environment, but we want to add a third CB (broker C). The Federator creates 3 more CSRs in broker A and broker B pointing to the new broker C:

{
        "id": "urn:aeros:federation:c:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:c:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:c:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainC",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.103:1026",
        "aerosDomain": "DomainC",
        "aerosDomainFederation": true
}

Finally, the Federator creates 6 CSRs in broker C pointing to the existing brokers A and B:

{
        "id": "urn:aeros:federation:a:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:a:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainA",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.100:1026",
        "aerosDomain": "DomainA",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:infrastructure",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Domain"
                    },
                    {
                        "type": "LowLevelOrchestrator"
                    },
                    {
                        "type": "InfrastructureElement"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:organizations",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Organization"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}
{
        "id": "urn:aeros:federation:b:services",
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Service"
                    },
                    {
                        "type": "ServiceComponent"
                    },
                    {
                        "type": "NetworkPort"
                    },
                    {
                        "type": "InfrastructureElementRequirements"
                    }
                ]
            }
        ],
        "contextSourceInfo": [
            {
                "key": "Authorization",
                "value": "urn:ngsi-ld:request"
            }
        ],
        "mode": "inclusive",
        "hostAlias": "DomainB",
        "operations": [
            "retrieveOps",
            "updateOps",
            "deleteEntity",
            "deleteAttrs"
        ],
        "management": {
            "localOnly": true
        },
        "endpoint": "http://192.168.1.101:1026",
        "aerosDomain": "DomainB",
        "aerosDomainFederation": true
}

@ravaga
Copy link
Author

ravaga commented Oct 17, 2024 •
edited
Loading

After all these CSR creations, the environment is ready (unless the bug appears and the registrations aren't updated in the cache). Therefore, we can query all the entities described in the registrations and retrieve all the created registrations for management tasks.

After some time, a new broker can be added to the environment (each broker will have 3 more registrations pointing to it, and the new broker will have a total of 9 registrations).

In the same way, a broker can be removed from the system. When a broker is removed (let's say we want to remove broker B), the federator removes all the CSRs with the attribute "aerosDomainFederation": true (it sends a GET /csourceRegistrations?csf=aerosDomainFederation==true and then a DELETE /csourceRegistrations/{regID} for each CSR). Then, it removes all the CSRs in brokers A and C that point to broker B (urn:aeros:federation:b:infrastructure, urn:aeros:federation:b:organizations and urn:aeros:federation:b:services).

The bug in the cache use to appear after the deletion of a broker from the environment or after the addition of multiple borkers. However and unfortunatelly, I haven't been able to find a pattern for the bug or reproduce it intentionally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kzangeli kzangeli

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kzangeli @ravaga