Current architecture snapshot #123
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Arch structure bullet points
The architecture is now: * components * deployment * flows
…flows Split technical view into three categories
* Bullet points; Defining property description * Describe typical content of the Offer * Checkpoint: Publishing Offers; negotiating (not complete) * Negotiating: describe Agreement conclusion * Example negotiations based on chosen-platform property * Describe Resource allocation and Activity creation/destruction * Add DebitNotes -> Invoice mermaid * Describe termination; potential reasons * Update DebitNotes description: DebitNotes payments triggering * Describe sending Invoice and acceptance semantic * Describe monitoring payments confirmation * Improve bullet points grammar * Simplified sequence diagram for activity from Provider perspective * Sequence diagram for negotiations * Apply simple and obvious review comments * Move diagrams to chapters with more detailed description * Refined simplified negotiations: get rid of yagna daemons * Link topics; Review fixes * Rework payments description (work in progress) * Rework payments descritpion in 5th chapter * Don't mention REST API and market module in chapter about publishing Offer * Remove mentioning of Golem protocol * Gramatically fix chapter about resources * Remove market protocol reference from termination description * Rephrase Activity chapter; mention where are Activity starting parameters comming from * Describe payments confirmation * Replace Golem Node with Core Network :( * APply review fixes
* Add DebitNotes -> Invoice mermaid * Describe termination; potential reasons * Update DebitNotes description: DebitNotes payments triggering * Describe sending Invoice and acceptance semantic * Describe monitoring payments confirmation * Simplified sequence diagram for activity from Provider perspective * Sequence diagram for negotiations * Apply simple and obvious review comments * Move diagrams to chapters with more detailed description * Refined simplified negotiations: get rid of yagna daemons * Link topics; Review fixes * Rework payments description (work in progress) * Rework payments descritpion in 5th chapter * Remove market protocol reference from termination description * Describe payments confirmation * Replace Golem Node with Core Network :( * Offer/Demand specification copy introduction from previous docs * Property naming, types etc * Properties examples; flattened vs nested form * Value-less properties * Add constraints basic description * Describe constraints operators * Demand/Offer matching * Pros and cons of using constraints against postponing decision to negotiations * Add example of constraint expression different operators * Market protocols; Describe subnet (work in progress) * Represent Offer/Demnad as table isntead of mermaid * Decribe negotiable properties * Mention mid-agreement payments and link to specification * Describe capabilities based approach; GPU as an example * ExeUnit progress events as an example of capability based approach * Problem with capabilities based approach * Backward compatiblity of negotiation protocols * Add payment procotol as an example of solving compatiblity issues * Describe potential problems with protocol clashes * Add links; Fixes after first reading * Description: How to determine market protocol specification conformance * Describe: where are specifications * Post-revase fixes * Apply review changes - part 1 * Review fixes: Language requirements * Language introduction * Reverse changes outside of this PR scope * Another round of review fixes
* Introduction chapter to Net module * Mermaid visualalizing net interactions * Describe network addressing and translation * Describe broadcasting * Describe how identities are handled * Improve chapter about network identitfication; add link to identity * Describe different net channels * List channels prefixes; other review fixes * Describe net responsiblities * Rephrase channels to transport types * New round of review fixes * Remove old version * Improved description of transfer transport type * Remove opaque
Add Payment Components to architecture snapshot
* Offers propagation: basic assumptions; Bullet points for further work * Mermaids for Offer propagation * Attempt to fix flowchart arrows * Improve broadcasting mermaids descriptions * Describe requirements for neighborhood functions * Subchapters about hybrid/central net difference in context of market; Description of broadcast triggers * Describe mechanism reducing garbase on market * Apply spliest review fixes * Chnage wording of broadcasts part 1 * Chnage fragment about global and local broadcast * Add paragraph about storing Offers * Remove invalidated neighborhood * Chnage broadcast naming semantic
* Describe network addressing and translation * Describe broadcasting * Describe how identities are handled * Improve chapter about network identitfication; add link to identity * Introduction to hybrid net * Describe relay server * Diagram: connecting to relay server * Describe Challenge and Public IP check * IP check: add alternative: timeout when packet is drpped * Describe p2p connections between Nodes * Describe p2p handshake * Describe Forwarding network traffic and virtual TCP * Describe how different channels are implemented in Hybrid Net * Describe broadcasts to neighborhood * Apply trivial review comments * Raw description of neighborhood * Improve neighborhood chapter * Apply trivial review fixes * Describe Node idenitification; Start describing encryption * Apply distinctions between connection and session; Broadcastign terminology * Describe key dervation and symmetric encryption * Remove triling spaces * Rephrase broadcasting description; add example * Explicite mention that transport types use underying session * Introduction word to neighhood concept * Apply correct broadcast naming semantic * Remove duplicates after rebase * Make transfer transport type description more precise
The description only lists the large ones.
The PR is picked up without review to be amended further afterwards.
Closed
mbenke
reviewed
Nov 25, 2024
| Both provider and requestor should specify property `golem.com.payment.protocol.version` | ||
| ### Why | ||
|
|
||
| Provider is unable to validate transactions made via contract. |
There was a problem hiding this comment.
The meaning of this is unclear to me - is this a justification for moving to version 2? If so, how is version 2 addressing this problem?
Comment on lines
+46
to
+47
| * The range of predefined "services" on Golem is far smaller compared to | ||
| traditional cloud providers. |
There was a problem hiding this comment.
Maybe add something that this range keeps growing, provide examples of recent additions.
Comment on lines
+123
to
+126
| The Requestor needs to have a certain level of technical knowledge to interact | ||
| with Golem Network's APIs (e.g., using the Python or Node.js SDKs). Through | ||
| these APIs, they specify what resources they require and how they intend to pay | ||
| for them. |
There was a problem hiding this comment.
This is not necessarily true; a Requestor (i.e. the entity demanding services and paying for them) may also have someone else write them the software needed.
hculap
reviewed
Dec 2, 2024
| Agreement. While this is not mandatory, it is encouraged as it can provide valuable context for the other party, | ||
| serving as diagnostic information or for other purposes. | ||
|
|
||
| The [Agreement](#agreement) can be terminated when either party chooses to end it. Core Network doesn't enforce any |
hculap
reviewed
Dec 2, 2024
| ``` | ||
|
|
||
|
|
||
| Once the Agreement is terminated, the Provider Agent should send an [Invoice](#invoice) to the Requestor Agent summarizing |
| It's important for the Provider to monitor payments after the Agreement is completed. This is when the Provider Agent | ||
| should adjust its market strategy to ensure profitability. Since the Core Network doesn't guarantee payment delivery, | ||
| the Provider Agent should implement measures to prevent being exploited by Requestors. One example is rejecting | ||
| non-paying Requestors and prioritizing those with a good reputation. Lack of payment isn't the only reason for |
There was a problem hiding this comment.
reputation -> Here, it would be probably good to explain what reputation is. Right now it’s unclear, and the protocol does not support something like Reputation... but from this description someone might assume that reputation is built into the protocol.
| Agent should monitor Payment events. This can be done by listening for status changes to Settled on Invoice and Debit Note | ||
| events, or by tracking payment events to receive notifications for each transaction. | ||
|
|
||
| Payment confirmation is received by the Provider Agent from the Requestor once the transaction is confirmed on the |
|
|
||
| Buying on Golem Network, just like [selling](#selling-on-golem-platform), | ||
| involves the [Requestor](#requestor) specifying what one needs in a formal | ||
| language, letting the platform match Offers and a [Requestor |
There was a problem hiding this comment.
What does it mean that the platform matches offers? From earlier descriptions, it seems there is no automatic offer matching, only propagation. It is the responsibility of the respective agents to match offers, not the platform.
|
|
||
| Before the Requestor begins, they must secure appropriate funds. To do this, | ||
| they should have funds on the wallet address from which they will pay, on one of | ||
| the two supported blockchains: Ethereum or Polygon. It is strongly recommended |
There was a problem hiding this comment.
This is the case now, but shouldn't it be mentioned that these platforms are more generic than just two?
|
|
||
| #### 6. Closure the Agreement | ||
|
|
||
| The Requestor is responsible for terminating the Agreement. This notifies the |
There was a problem hiding this comment.
We previously wrote that both parties can terminate the agreement. I
s it okay to indicate the Requestor as the responsible party here?
|
|
||
| ## Layers | ||
|
|
||
| Golem Network's applications are unknown in advance. Therefore, based on past |
There was a problem hiding this comment.
A very strange introduction for the beginning of a section on architecture layers
mbenke
reviewed
Dec 5, 2024
Comment on lines
+140
to
+153
| Typically, humans are not involved in the process of finding, matching, negotiating, or finalizing agreements. Instead, | ||
| users define their needs programmatically, allowing the [Provider Agent](#provider-agent) and [Requestor Agent](#requester-agent) | ||
| software to handle these tasks automatically. | ||
|
|
||
| The [Provider Agent](#provider-agent) is primarily responsible for implementing the logic needed to sell resources on | ||
| the Golem Network. From high level perspective, Provider Agent application should do following things: | ||
| 1. Describe Resources using property language to create an [Offer](#offer) | ||
| 2. Publish the Offer on the market | ||
| 3. Monitor incoming Proposals and negotiate an [Agreement](#agreement) with the most promising [Requestor](#Requestor) | ||
| 4. Allocate the promised Resources in accordance with the [Agreement](#agreement) | ||
| 5. Monitor resources usage and charge Requestor Agent | ||
| 6. Terminate the Agreement or await the Agreement termination event from the [Requestor](#Requestor) | ||
| 7. Send an [Invoice](#invoice) summarizing the total cost of the [Agreement](#agreement) | ||
| 8. Wait until the payment for the [Invoice](#invoice) is settled and payment confirmed. |
There was a problem hiding this comment.
This description may seem intimidating. It might be worth stating that a Golem user does not necessarily need to create a program to do all this things, but simply use a default Provider Agent with minimal configuration effort.
| #### 1. Describe Resources using property language to create an [Offer](#offer) | ||
|
|
||
| While Golem is currently used for trading computational resources, it was designed to support the exchange of any type of | ||
| [resource](#resource). This means the Marketplace does not enforce strict standards on the goods being traded. |
There was a problem hiding this comment.
Suggested change
| [resource](#resource). This means the Marketplace does not enforce strict standards on the goods being traded. | |
| [resource](#resource). This means the Marketplace does not enforce strict standards on the services being traded. |
| #### 3. Monitor incoming Proposals and negotiate an Agreement with the most promising Requestor | ||
|
|
||
| The [Provider Agent](#provider-agent) plays a passive role in negotiations. Offers are propagated across the network and | ||
| received by [Requestors](#Requestor). The offer is matched locally on the Requestor's node with a [Demand](#demand). |
There was a problem hiding this comment.
This is the first place the notion of Demand occurs. It might be worthwhile to explain what it is (especially that the linked definition is not very enlightening)
|
|
||
| ```mermaid | ||
| --- | ||
| title: Simplified negotiations from Provider Agent's perspective |
| a constraint to their demand. | ||
|
|
||
| Instead, the Requestor Agent collects Proposals from the market and evaluates them based on estimated costs. In later stages | ||
| of Proposal exchange, they choose the platform by setting the relevant [property](#property) according to the Providers' scores, |
There was a problem hiding this comment.
Setting the property where? In the counter-proposal? But arch.md#property states "Property is a key-value pair present in Offers and Demands specifying properties of the Node producing the document.". This is confusing.
Comment on lines
+335
to
+340
| Both Debit Notes and Invoices can be either accepted or rejected by the other party. Acceptance signals that the | ||
| Requestor Agent agrees to pay the specified amount. Rejection, on the other hand, indicates refusal to pay the | ||
| non-accepted amount. However, it’s important to note that a rejection does not absolve the Requestor Agent from paying | ||
| for all previously accepted Debit Notes. The conditions under which rejection is allowed should be defined in the | ||
| Agreement. Currently, no payment scheme permits rejections. | ||
|
|
There was a problem hiding this comment.
How can a Requestor safeguard against a dishonest provider?
Or even honest mistakes, if "no payment scheme permits rejections"?
| information in their Offer or Demand. This is a more lightweight approach compared to going through the full negotiation | ||
| process. | ||
|
|
||
| ### Buying on golem platform. |
There was a problem hiding this comment.
Suggested change
| ### Buying on golem platform. | |
| ### Buying on Golem platform. |
| 1. Create a Demand | ||
| 1. Negotiate and Agreement | ||
| 1. Activate the service and supervise the Agreement | ||
| 1. Closure the Agreement |
There was a problem hiding this comment.
I think "closure" is usually used as a noun rather than a verb.
| The Requestor is responsible for terminating the Agreement. This notifies the | ||
| Provider to terminate all activities associated with that contract. The | ||
| Requestor receives an Invoice summarizing the expenses from all Activities | ||
| active under the agreement. The Requestor confirms that the amount is correct. |
|
|
||
| Golem Network's applications are unknown in advance. Therefore, based on past | ||
| experience, a decision was made to design the architecture flexible enough to | ||
| swap out key components. This resulted in a three-layer architecture. |
There was a problem hiding this comment.
It might be helpful to explain how the layers communicate, since from this descriptions they seem to be mostly independent from each other, so as to represent "dimensions" rather than "layers"
mbenke
reviewed
Dec 6, 2024
| derive the shared secret, allowing it to decrypt the message. No special control packet exchanges are required | ||
| between the sender and receiver. | ||
|
|
||
| #### Central net |
Comment on lines
+1370
to
+1371
| A description of the component responsible for making offers, counter-offers, | ||
| negotiations, etc. |
|
|
||
| #### Offer propagation | ||
|
|
||
| - Link to design [decision](#only-providers-offers-are-propagated) |
|
|
||
| #### 3. Monitor incoming Proposals and negotiate an Agreement with the most promising Requestor | ||
|
|
||
| The [Provider Agent](#provider-agent) plays a passive role in negotiations. Offers are propagated across the network and |
There was a problem hiding this comment.
Yet the diagram below states that the provider "updates Proposals" and sends "Counter Proposals". Is this passive?
| these APIs, they specify what resources they require and how they intend to pay | ||
| for them. | ||
|
|
||
| ## Activities |
There was a problem hiding this comment.
"Activity" has a specific meaning in Golem, so this section title should probably be changed
Suggested change
| ## Activities | |
| ## Using the Golem network |
Comment on lines
+9
to
+10
| intended audience is assumed to be technical but not necessarily have deep | ||
| expertise in the crypto world. The level of detail stops short of describing |
There was a problem hiding this comment.
Suggested change
| intended audience is assumed to be technical but not necessarily have deep | |
| expertise in the crypto world. The level of detail stops short of describing | |
| intended audience is assumed to be technical but not necessarily experts in the crypto world. The level of detail stops short of describing |
|
|
||
| This section outlines the components of the Golem Network, including the actors, | ||
| technical artifacts, and the activities these actors may perform. The goal is to | ||
| provide a high-level overview that ties together all key terms, offering a clear |
There was a problem hiding this comment.
Suggested change
| provide a high-level overview that ties together all key terms, offering a clear | |
| provide a high-level overview tying together all key terms, offering a clear |
Comment on lines
+131
to
+132
| to give the reader an overview of the terms introduced by Golem without any | ||
| details and establish a glossary to ensure consistency within the document. |
There was a problem hiding this comment.
Suggested change
| to give the reader an overview of the terms introduced by Golem without any | |
| details and establish a glossary to ensure consistency within the document. | |
| to give the reader an overview of the terms introduced by Golem and establish a glossary to ensure consistency within the document. |
| these APIs, they specify what resources they require and how they intend to pay | ||
| for them. | ||
|
|
||
| ## Activities |
There was a problem hiding this comment.
One major problem I have with this whole section is that it is provider-centriic, as if we did not care so much for requestors. I wonder if there may be a deeper issue underlying this approach. For example, compare the detailed description of "selling on Golem" vs the rudimentary subsection on buying.
| @@ -0,0 +1,2921 @@ | |||
| # Golem — current architecture | |||
There was a problem hiding this comment.
My main problem with this document is that it is sadly a tedious, uninspiring read.
An architecture document should convey the most interesting and novel ideas as well and strive to inspire. It may be technically challenging but this should be balanced by a convinvcing story and gripping storytelling. Alas, this document fails to achieve either of this goals.
mbenke
reviewed
Dec 11, 2024
Comment on lines
+1977
to
+1978
| #### Market strategies | ||
|
|
mbenke
reviewed
Dec 11, 2024
Comment on lines
+1979
to
+1982
| #### Agreement termination | ||
| - Who is allowed to terminate? In what situation? | ||
| - What is specified by protocol and what is left to future specifications? | ||
| - Termination reason concept |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR summarizes the current Golem's architecture.