diff --git a/neofs-blocks.mediawiki b/neofs-blocks.mediawiki new file mode 100644 index 00000000..5454f2eb --- /dev/null +++ b/neofs-blocks.mediawiki @@ -0,0 +1,104 @@ +
+  NEP: TBD
+  Title: NeoFS block storage format
+  Author: Ekaterina Pavlova 
+  Type: Informational
+  Status: Draft
+  Created: 2025-04-07
+
+ +==Abstract== + +This proposal outlines the specification for storing Neo blockchain blocks within +NeoFS container. + +==Motivation== + +Neo node synchronization via P2P requires all headers and blocks (~700 bytes each, over 1 GB per year with +15-second blocks) to remain available, limiting scalability as chains grow. Storing blocks in NeoFS +provides a distributed alternative, reducing local storage, enabling on-demand fetches and allowing to drop +blocks and headers from storage for most nodes. + +==Specification== + +===Block Storage Schema=== + +A single NeoFS container is used to store blocks of a single Neo network. Each container includes a fixed +metadata attribute named `Magic` whose value is the network’s magic number in decimal representation. Each +block is stored in a serialized representation following standard Neo block serialization rules as a +separate NeoFS object with a unique OID and the following metadata attributes: + +{| class="wikitable" +! Attribute name !! Attribute value +|- +| Block || String representation of the block index in base 10. +|- +| Hash || Hexadecimal string of the block’s hash. +|- +| PrevHash || Hexadecimal string of the previous block’s hash. +|- +| BlockTime || Millisecond-precision unix timestamp when the block was created. +|- +| Timestamp || Second-precision unix timestamp when the block was uploaded. +|} + +These attributes are stored as NeoFS object metadata; the list of attributes MAY be extended. +The object MAY include additional data after the block. + +===Synchronization Algorithm=== + +Blocks stored in NeoFS and those synchronized via P2P both use the standard Neo block +serialization format, ensuring compatibility across different synchronization methods. +The NeoFS container serves as verified storage for downloading blocks. +[https://github.com/nspcc-dev/neofs-api/blob/b7548cfb16426256f0a92c6d70a3ec794a6facb0/object/service.proto#L170 NeoFS SearchV2 API] +should be used to fetch ordered chunks of block OIDs by Block attribute. Additional +GE/GT/LE/LT SearchV2 filters may be applied to the Block attribute value to filter +out required range of blocks. +[https://github.com/nspcc-dev/neofs-api/blob/b7548cfb16426256f0a92c6d70a3ec794a6facb0/object/service.proto#L48 NeoFS Get API] +should be used to fetch blocks by retrieved OIDs. + +Downloaded blocks are inserted into the blockchain using the same logic as in the +P2P synchronization protocol. + +==Rationale== + +Storing each block as a self-contained NeoFS object moves historical data off full nodes +while preserving byte-exact compatibility with the P2P format. Standard metadata makes +blocks easily discoverable without custom indices, and NeoFS gives durability and fast +parallel fetches. The proposed scheme is implemented as an experimental extension of NeoGo +node ([https://github.com/nspcc-dev/neo-go/issues/3496 nspcc-dev/neo-go#3496]) and +proved to be efficient. + +== References == + +* [https://github.com/neo-project/neo/issues/3463 Using NeoFS to store blocks and snapshots] + +== Test cases == + +An example of a [https://rest.fs.neo.org/v1/objects/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/by_id/Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1 block object]: +
+ID: Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1
+CID: 3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV
+Owner: NVvY1FF67XJ2GTVhy9FqiZGC4jEQtvjmHt
+CreatedAt: 28202
+Size: 697
+HomoHash: 45c98e627910d9b915d58368493789ce49ca194626f16ea5bf6b57bb2cce462921a1d3faf682d252a804b0555ca48905286222ee9209b3ff1ce4677a082ffd4d
+Checksum: fa6cedddfec3a61157c4a12e25f81e85c0f92aac70317d6df7fe193f983b4917
+Type: REGULAR
+Attributes:
+  Block=1
+  Primary=0
+  Hash=5f3fbb43d1e516fe07771e2e873ebc9e2810662401acf603a775aace486220bd
+  PrevHash=1f4d1defa46faa5e7b9b8d3f79a06bec777d7c26c4aa5f6f5899a291daa87c15
+  BlockTime=1627894840919
+  Timestamp=1734362616 (2024-12-16 18:23:36 +0300 MSK)
+ID signature:
+  public key: 02a4920745d86db224c179c936606dc0e4620edad13d568ef036da279352e45f2b
+  signature: 0443ff20d15952759b5101a7d223d70eb992260fd9ad5aecb404a97448b2ea54bd1ad783b12ccddba0097dd6608b55ccac9215c5715f9589ec6a555542ead6dc00
+[Block Binary Data]
+
+ +== Implementation == + +* Go: [https://github.com/nspcc-dev/neo-go/blob/965ae2746cd1a01aba3b4114e1ea26ca7349f34b/cli/util/upload_bin.go utility tool that uploads blocks to NeoFS container] +* Go: [https://github.com/nspcc-dev/neo-go/blob/965ae2746cd1a01aba3b4114e1ea26ca7349f34b/pkg/services/blockfetcher/blockfetcher.go node service that fetches blocks from NeoFS container as an alternative to P2P synchronization]