From b7dd8ea540f9aaa7b7d6dfc878aed9fd497079a2 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Wed, 6 Sep 2023 17:59:49 -0700 Subject: [PATCH 1/9] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fe4c003..38fe257 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ We've designed a new type of Merkle Tree/DAG. Scionic Merkle Trees have small branches like Classic Merkle trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle trees and Merkle DAGs while addressing several of their limitations. -![Tree Comparison Diagram](https://i.ibb.co/5LGmSqm/Tree-Comparison-Diagram.png) +![Tree Comparison Diagram](https://static.wixstatic.com/media/e9326a_d4a555d0ed9e46da8e751967961710a3~mv2.png/v1/fill/w_1679,h_421,al_c,q_90,usm_0.66_1.00_0.01,enc_auto/Merkle%20DAG%20Tree%20Comparison%20Diagram.png) ## Classic Merkle Trees and Merkle DAGs: A Comparison From 87ecc510aaa188d898e16b5faec6a8f39c2f73a6 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 16:39:52 -0700 Subject: [PATCH 2/9] Update README.md --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 38fe257..3348b73 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,12 @@ If a user wants a specific file from a folder on the tree, they no longer need t This streamlined process significantly improves efficiency, especially with large datasets. Scionic Merkle Trees are a powerful tool for handling folders with numerous files, combining the directory-friendly nature of Merkle DAGs and the compact efficiency of Classic Merkle Trees. +### ***Scionic Merkle DAG:*** ![Scionic Merkle Tree Diagram](https://i.ibb.co/XJjbwmP/Scionic-Merkle-Tree.jpg) +### ***Scionic Merkle DAG Branch:*** +![Scionic Merkle Branch Diagram](https://i.ibb.co/nLcNLw1/Merkle-Branch.png) + ## Scionic Merkle Branch Statistics *Comparing the size of a Scionic Merkle Branch to bloated Merkle DAG Branches:* From b7136609eba7bbf269b8939c4aa2390b89416027 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 16:43:09 -0700 Subject: [PATCH 3/9] Update README.md --- README.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 3348b73..444e69a 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ [![codecov](https://codecov.io/gh/HORNET-Storage/scionic-merkletree/graph/badge.svg?token=1UBLJ1YYFI)](https://codecov.io/gh/HORNET-Storage/scionic-merkletree) -# Scionic Merkle Trees +# Scionic Merkle DAG-Trees ## An Evolution of Merkle DAGs and Merkle Trees -We've designed a new type of Merkle Tree/DAG. Scionic Merkle Trees have small branches like Classic Merkle trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle trees and Merkle DAGs while addressing several of their limitations. +We've designed a new type of Merkle Tree/DAG. Scionic Merkle DAG-Trees have small branches like Classic Merkle trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle trees and Merkle DAGs while addressing several of their limitations. ![Tree Comparison Diagram](https://static.wixstatic.com/media/e9326a_d4a555d0ed9e46da8e751967961710a3~mv2.png/v1/fill/w_1679,h_421,al_c,q_90,usm_0.66_1.00_0.01,enc_auto/Merkle%20DAG%20Tree%20Comparison%20Diagram.png) @@ -22,15 +22,15 @@ The number of hashes required for a Merkle proof in a Classic Merkle Tree grows Merkle DAGs were developed as a solution to incorporate folders and files, addressing a key limitation of Classic Merkle Trees. However, this structure has its own challenge: to securely download a single file, you must download the hash of every other file inside the folder its stored in. This requirement can be slow and costly for users when dealing with folders that contain large amounts of files. -## The Strengths of Scionic Merkle Trees +## The Strengths of Scionic Merkle DAG-Trees ### ***Folders and Files:*** -Like Merkle DAGs, Scionic Merkle Trees can accommodate folders and files. However, they also maintain the efficiency of Classic Merkle trees. +Like Merkle DAGs, Scionic Merkle DAG-Trees can accommodate folders and files. However, they also maintain the efficiency of Classic Merkle trees. ### ***Internal Arrangement:*** -The unique feature of Scionic Merkle Trees is their internal structure. Within each folder (parent leaf) across the tree, its list of files (children) is organized as a Classic Merkle tree rather than a plaintext list. +The unique feature of Scionic Merkle DAG-Trees is their internal structure. Within each folder (parent leaf) across the tree, its list of files (children) is organized as a Classic Merkle tree rather than a plaintext list. ### ***Efficient File Download and Branch Verification:*** @@ -38,7 +38,7 @@ If a user wants a specific file from a folder on the tree, they no longer need t ### ***Improved Scalability for Users with Large Datasets:*** -This streamlined process significantly improves efficiency, especially with large datasets. Scionic Merkle Trees are a powerful tool for handling folders with numerous files, combining the directory-friendly nature of Merkle DAGs and the compact efficiency of Classic Merkle Trees. +This streamlined process significantly improves efficiency, especially with large datasets. Scionic Merkle DAG-Trees are a powerful tool for handling folders with numerous files, combining the directory-friendly nature of Merkle DAGs and the compact efficiency of Classic Merkle Trees. ### ***Scionic Merkle DAG:*** ![Scionic Merkle Tree Diagram](https://i.ibb.co/XJjbwmP/Scionic-Merkle-Tree.jpg) @@ -55,21 +55,21 @@ This streamlined process significantly improves efficiency, especially with larg * In the case of a folder with 10,000 files, a Scionic branch requires 15 leaves, while a Merkle DAG branch needs all 10,000. This means the Scionic branch is roughly **710x smaller**. * If the folder contains 1,000,000 files, a Scionic branch for any file in that folder would require around 21 leaves. This Scionic branch would be **50,000x smaller**. -These statistics underline the substantial efficiency improvements made by Scionic Merkle Trees. +These statistics underline the substantial efficiency improvements made by Scionic Merkle DAG-Trees. ## Understanding Growth Patterns: Logarithmic vs Linear -In the case of Scionic Merkle Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches slow down. This makes Scionic Merkle Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** +In the case of Scionic Merkle DAG-Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches slow down. This makes Scionic Merkle DAG-Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** In stark contrast, the number of hashes required to validate a single folder in a Merkle DAG exhibits linear growth. If there are more children (files) in the folder, you must download the hashes to all of them to retrieve a single file. This constant requirement can lead to overly large merkle branches. The amount of hashes needed to validate a single file increases in direct proportion to the number of files in the folder, making it less efficient for large datasets, as it demands more computational work from users for each new file added to a folder in the DAG. ## Syncing Trees Across Relays by Requesting a Range of Leafs -To further enhance the functionality of Scionic Merkle Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the request for ranges of leafs, much like what Graphsync attempts to accomplish, but without the complexity of converting a tree into a graph. +To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the request for ranges of leafs, much like what Graphsync attempts to accomplish, but without the complexity of converting a tree into a graph. The total number of leafs is recorded at the root of the tree, and each folder also carries information about the total number of leafs it contains. By doing so, users can request a range of leafs from a given folder, simplifying data retrieval, and reducing the bandwidth overhead and computational workload required to access multiple files from the same folder. -This approach not only maintains the structural advantages of Scionic Merkle Trees, such as logarithmic growth of branches and efficient file download and verification, but also provides enhanced support for range queries, contributing to their practicality in large-scale data management scenarios. +This approach not only maintains the structural advantages of Scionic Merkle DAG-Trees, such as logarithmic growth of branches and efficient file download and verification, but also provides enhanced support for range queries, contributing to their practicality in large-scale data management scenarios. ## @@ -113,4 +113,4 @@ if err != nil { } ``` -This repository is a work in progress. We plan to make quite a few improvements as we move forward with integrating the Scionic Merkle Trees into H.O.R.N.E.T. Storage. Although, what we have is currently working -- we do not recommend using the trees in anything production-related yet. +This repository is a work in progress. We plan to make quite a few improvements as we move forward with integrating the Scionic Merkle DAG-Trees into H.O.R.N.E.T. Storage. Although, what we have is currently working -- we do not recommend using the trees in anything production-related yet. From 4bc1bbd994b02ef9e3af96a2209db5beab83d4a4 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 16:49:54 -0700 Subject: [PATCH 4/9] Update README.md --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 444e69a..7428548 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ ## An Evolution of Merkle DAGs and Merkle Trees -We've designed a new type of Merkle Tree/DAG. Scionic Merkle DAG-Trees have small branches like Classic Merkle trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle trees and Merkle DAGs while addressing several of their limitations. +We've designed a new type of Merkle Tree/DAG. Scionic Merkle DAG-Trees have small branches like Classic Merkle Trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle Trees and Merkle DAGs while addressing several of their limitations. ![Tree Comparison Diagram](https://static.wixstatic.com/media/e9326a_d4a555d0ed9e46da8e751967961710a3~mv2.png/v1/fill/w_1679,h_421,al_c,q_90,usm_0.66_1.00_0.01,enc_auto/Merkle%20DAG%20Tree%20Comparison%20Diagram.png) @@ -14,7 +14,7 @@ We've designed a new type of Merkle Tree/DAG. Scionic Merkle DAG-Trees have smal ### ***Classic Merkle Trees*** -Classic Merkle Trees are cryptographic structures used to manage and securely verify large amounts of data. However, they have a significant drawback: they cannot store folders or files. + Trees are cryptographic structures used to manage and securely verify large amounts of data. However, they have a significant drawback: they cannot store folders or files. The number of hashes required for a Merkle proof in a Classic Merkle Tree grows logarithmically with the number of files, meaning the growth rate slows as the input (tree) size increases. This pattern makes them very efficient for large datasets because the branches become exponentially smaller as the number of files in the folder rises. @@ -26,7 +26,7 @@ Merkle DAGs were developed as a solution to incorporate folders and files, addre ### ***Folders and Files:*** -Like Merkle DAGs, Scionic Merkle DAG-Trees can accommodate folders and files. However, they also maintain the efficiency of Classic Merkle trees. +Like Merkle DAGs, Scionic Merkle DAG-Trees can accommodate folders and files. However, they also maintain the efficiency of Classic Merkle Trees. ### ***Internal Arrangement:*** @@ -40,10 +40,10 @@ If a user wants a specific file from a folder on the tree, they no longer need t This streamlined process significantly improves efficiency, especially with large datasets. Scionic Merkle DAG-Trees are a powerful tool for handling folders with numerous files, combining the directory-friendly nature of Merkle DAGs and the compact efficiency of Classic Merkle Trees. -### ***Scionic Merkle DAG:*** +### ***Scionic Merkle DAG-Tree:*** ![Scionic Merkle Tree Diagram](https://i.ibb.co/XJjbwmP/Scionic-Merkle-Tree.jpg) -### ***Scionic Merkle DAG Branch:*** +### ***Scionic Merkle Branch:*** ![Scionic Merkle Branch Diagram](https://i.ibb.co/nLcNLw1/Merkle-Branch.png) ## Scionic Merkle Branch Statistics @@ -65,9 +65,9 @@ In stark contrast, the number of hashes required to validate a single folder in ## Syncing Trees Across Relays by Requesting a Range of Leafs -To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the request for ranges of leafs, much like what Graphsync attempts to accomplish, but without the complexity of converting a tree into a graph. +To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the request for ranges of leaves, much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. -The total number of leafs is recorded at the root of the tree, and each folder also carries information about the total number of leafs it contains. By doing so, users can request a range of leafs from a given folder, simplifying data retrieval, and reducing the bandwidth overhead and computational workload required to access multiple files from the same folder. +The total number of leaves is recorded at the root of the tree, and each folder also carries information about the total number of leaves it contains. By doing so, users can request a range of leaves from a given folder, simplifying data retrieval, and reducing the bandwidth overhead and computational workload required to access multiple files from the same folder. This approach not only maintains the structural advantages of Scionic Merkle DAG-Trees, such as logarithmic growth of branches and efficient file download and verification, but also provides enhanced support for range queries, contributing to their practicality in large-scale data management scenarios. @@ -85,7 +85,7 @@ go get github.com/HORNET-Storage/scionic-merkletree/dag #### Example Usage There are good examples inside the dag/dag_test.go file, but below is a basic example to get you started. This library is intended to be very simple while still allowing for powerful usage... -Turn a folder and its files into a Scionic Merkle Tree/DAG, then convert the Scionic Merkle tree back to the original files in a new directory: +Turn a folder and its files into a Scionic Merkle DAG-Tree, then convert the Scionic Merkle tree back to the original files in a new directory: ```go input := filepath.Join(tmpDir, "input") output := filepath.Join(tmpDir, "output") From 370dc7fac7f3f79e95c8c54275de402d7800a222 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 16:52:32 -0700 Subject: [PATCH 5/9] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7428548..46ca1e9 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ In stark contrast, the number of hashes required to validate a single folder in ## Syncing Trees Across Relays by Requesting a Range of Leafs -To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the request for ranges of leaves, much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. +To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. The total number of leaves is recorded at the root of the tree, and each folder also carries information about the total number of leaves it contains. By doing so, users can request a range of leaves from a given folder, simplifying data retrieval, and reducing the bandwidth overhead and computational workload required to access multiple files from the same folder. From d77bef1f4f6b32e6d9e189e74d8a193b19e2186a Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 16:54:01 -0700 Subject: [PATCH 6/9] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 46ca1e9..1a03ea7 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ ## An Evolution of Merkle DAGs and Merkle Trees -We've designed a new type of Merkle Tree/DAG. Scionic Merkle DAG-Trees have small branches like Classic Merkle Trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle Trees and Merkle DAGs while addressing several of their limitations. +We've designed a [new type of Merkle DAG-Tree hybrid](https://www.hornetstorage.com/). Scionic Merkle DAG-Trees have small branches like Classic Merkle Trees, the file storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers to sync quickly. They are an innovative model that merge the advantages of Classic Merkle Trees and Merkle DAGs while addressing several of their limitations. ![Tree Comparison Diagram](https://static.wixstatic.com/media/e9326a_d4a555d0ed9e46da8e751967961710a3~mv2.png/v1/fill/w_1679,h_421,al_c,q_90,usm_0.66_1.00_0.01,enc_auto/Merkle%20DAG%20Tree%20Comparison%20Diagram.png) From 25d0c7c2835333887dade901564c54c54d7b2bf4 Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 17:14:56 -0700 Subject: [PATCH 7/9] Update README.md --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 1a03ea7..ec65acb 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ The number of hashes required for a Merkle proof in a Classic Merkle Tree grows ### ***Merkle DAGs (Directed Acyclic Graphs)*** -Merkle DAGs were developed as a solution to incorporate folders and files, addressing a key limitation of Classic Merkle Trees. However, this structure has its own challenge: to securely download a single file, you must download the hash of every other file inside the folder its stored in. This requirement can be slow and costly for users when dealing with folders that contain large amounts of files. +Merkle DAGs were developed as a solution to incorporate folders and files, addressing a key limitation of Classic Merkle Trees. However, this structure has its own challenge: to securely download a single file, you must download the hash of every other file inside the folder its stored in. This requirement can be slow and costly for users when dealing with folders that contain a large number of files. ## The Strengths of Scionic Merkle DAG-Trees @@ -30,7 +30,7 @@ Like Merkle DAGs, Scionic Merkle DAG-Trees can accommodate folders and files. Ho ### ***Internal Arrangement:*** -The unique feature of Scionic Merkle DAG-Trees is their internal structure. Within each folder (parent leaf) across the tree, its list of files (children) is organized as a Classic Merkle tree rather than a plaintext list. +The unique feature of Scionic Merkle DAG-Trees is their internal structure. Within each folder (parent leaf) across the tree, its list of files (children) are organized as a Classic Merkle tree rather than a plaintext list. ### ***Efficient File Download and Branch Verification:*** @@ -59,15 +59,15 @@ These statistics underline the substantial efficiency improvements made by Scion ## Understanding Growth Patterns: Logarithmic vs Linear -In the case of Scionic Merkle DAG-Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches slow down. This makes Scionic Merkle DAG-Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** +In the case of Scionic Merkle DAG-Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches decrease. This makes Scionic Merkle DAG-Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** -In stark contrast, the number of hashes required to validate a single folder in a Merkle DAG exhibits linear growth. If there are more children (files) in the folder, you must download the hashes to all of them to retrieve a single file. This constant requirement can lead to overly large merkle branches. The amount of hashes needed to validate a single file increases in direct proportion to the number of files in the folder, making it less efficient for large datasets, as it demands more computational work from users for each new file added to a folder in the DAG. +In stark contrast, the number of hashes required to validate a single folder in a Merkle DAG exhibits linear growth. If there are more children (files) in the folder, you must download the hash of each one to retrieve any individual file from the folder. This constant requirement can lead to overly large Merkle branches. The amount of hashes needed to validate a single file increases in direct proportion to the number of files in the folder, making it less efficient for large datasets. ## Syncing Trees Across Relays by Requesting a Range of Leafs -To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a unique number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. +To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a sequenced number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. -The total number of leaves is recorded at the root of the tree, and each folder also carries information about the total number of leaves it contains. By doing so, users can request a range of leaves from a given folder, simplifying data retrieval, and reducing the bandwidth overhead and computational workload required to access multiple files from the same folder. +The total number of leaves is recorded at the root of the tree. By doing so, users can request a range of leaves from a given folder and receive it as a small Scionic Merkle branch, reducing the bandwidth overhead and computational workload required to access multiple files in the same folder. This approach not only maintains the structural advantages of Scionic Merkle DAG-Trees, such as logarithmic growth of branches and efficient file download and verification, but also provides enhanced support for range queries, contributing to their practicality in large-scale data management scenarios. From 84ec664a3deb8bd725d5f960e207dac77dcaae8f Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 17:15:34 -0700 Subject: [PATCH 8/9] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ec65acb..624505e 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ In stark contrast, the number of hashes required to validate a single folder in ## Syncing Trees Across Relays by Requesting a Range of Leafs -To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labelled with a sequenced number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. +To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labeled with a sequenced number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. The total number of leaves is recorded at the root of the tree. By doing so, users can request a range of leaves from a given folder and receive it as a small Scionic Merkle branch, reducing the bandwidth overhead and computational workload required to access multiple files in the same folder. From 968d18e9a840a98f8c2f9c551773fdeeb5b83e2e Mon Sep 17 00:00:00 2001 From: Colby Serpa <129702033+ColbySerpa@users.noreply.github.com> Date: Sun, 10 Sep 2023 17:19:22 -0700 Subject: [PATCH 9/9] Update README.md --- README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 624505e..0558335 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ We've designed a [new type of Merkle DAG-Tree hybrid](https://www.hornetstorage. ### ***Classic Merkle Trees*** - Trees are cryptographic structures used to manage and securely verify large amounts of data. However, they have a significant drawback: they cannot store folders or files. + Merkle Trees are cryptographic structures used to manage and securely verify large amounts of data. However, they have a significant drawback: they cannot store folders or files. The number of hashes required for a Merkle proof in a Classic Merkle Tree grows logarithmically with the number of files, meaning the growth rate slows as the input (tree) size increases. This pattern makes them very efficient for large datasets because the branches become exponentially smaller as the number of files in the folder rises. @@ -59,17 +59,17 @@ These statistics underline the substantial efficiency improvements made by Scion ## Understanding Growth Patterns: Logarithmic vs Linear -In the case of Scionic Merkle DAG-Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches decrease. This makes Scionic Merkle DAG-Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** +In the case of Scionic Merkle DAG-Trees, which incorporate Classic Merkle Trees within their structure, they exhibit logarithmic growth. This means that as the size of the input (the number of files in a folder) increases, the growth rate of the Classic Merkle Tree branches decreases. This makes Scionic Merkle DAG-Trees an efficient structure for managing large datasets, ***as the branches become exponentially smaller with the increasing number of files in the folder.*** In stark contrast, the number of hashes required to validate a single folder in a Merkle DAG exhibits linear growth. If there are more children (files) in the folder, you must download the hash of each one to retrieve any individual file from the folder. This constant requirement can lead to overly large Merkle branches. The amount of hashes needed to validate a single file increases in direct proportion to the number of files in the folder, making it less efficient for large datasets. -## Syncing Trees Across Relays by Requesting a Range of Leafs +## Syncing Trees Across Relays by Requesting a Range of Leaves To further enhance the functionality of Scionic Merkle DAG-Trees and support efficient data retrieval, each leaf in the tree is labeled with a sequenced number. This method facilitates the [request for a range of Merkle leaves](https://www.hornetstorage.com/forest), much like what GraphSync attempts to accomplish, but without the complexity of using complex graph selectors and large request sizes. The total number of leaves is recorded at the root of the tree. By doing so, users can request a range of leaves from a given folder and receive it as a small Scionic Merkle branch, reducing the bandwidth overhead and computational workload required to access multiple files in the same folder. -This approach not only maintains the structural advantages of Scionic Merkle DAG-Trees, such as logarithmic growth of branches and efficient file download and verification, but also provides enhanced support for range queries, contributing to their practicality in large-scale data management scenarios. +This approach provides the structural advantages of Scionic Merkle DAG-Trees, such as logarithmic growth of branches and efficient file download and verification, and also provides enhanced support for ranged requests, contributing to their practicality in large-scale data management scenarios. ## @@ -114,3 +114,4 @@ if err != nil { ``` This repository is a work in progress. We plan to make quite a few improvements as we move forward with integrating the Scionic Merkle DAG-Trees into H.O.R.N.E.T. Storage. Although, what we have is currently working -- we do not recommend using the trees in anything production-related yet. +#