Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 0 additions & 52 deletions content/codex/updates/2025-07-21.md

This file was deleted.

60 changes: 33 additions & 27 deletions content/codex/updates/2025-07-28.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,51 @@
title: "2025-07-28 Codex weekly"
tags:
- "codex-updates"
date: 2025-08-01
lastmod: 2025-08-11
date: 2025-07-28
lastmod: 2025-07-28
draft: false
---

## Week 31 July 2025
## Week 30 July 2025
Updates on what the past couple of weeks of discussions and refocusing have brought so far. Directions Codex is taking + still ongoing conversations.

### nim libs
### Documents and Knowledge
Codex has had a long-standing problem of documents and knowledge being locked into ad hoc places and contributor's personal accounts and getting lost. This week, we finally made a step towards addressing that. TL;DR, we'll have:

* ABI encoder / decoder integration in web3 (https://github.com/status-im/nim-web3/pull/216)
* Complete decoding
* Fix dynamic data encoding
* Integration with nim-serialization
* Benchmark (https://github.com/2-towns/nim-abi-benchmark)
* extracted async iterators to a spearate Nim package - initial version, work in progress: https://github.com/codex-storage/nim-asynciterators
* a [team HackMD](https://hackmd.io/team/codex-storage) (as opposed to personal accounts) for quick collaboration and ephemeral documents;
* git(hub) as the eventual place where everything "important" should be.

### Altruistic mode
This is good because:

Use cases have been more or less understood through requirements gathering efforts.
* team HackMD means we don't lose sight of documents anymore: it's full-text-searchable, it's markdown, supports github sync, and is very low friction for collaboration and quick-and-dirty docs;
* people who prefer Obsidian can use it, as the github repo is a just an Obsidian vault;
* github flows (PRs and comments on PRs) are still possible;
* the vault builds to a quartz website so it's also conveniently publicly visible/searchable;
* last, but not least, it sits well with the team's preferences.

#### Waku requirements
### Short-term Roadmap

Final round of reviews went into the [FURPs requirements](https://github.com/waku-org/pm/pull/328), which were merged. After much discussion, the Waku use cases are summarised in https://hackmd.io/@codex-storage/B1DYfESPlg.
As part of the effort in re-steering the project towards Logos and faster releases, we have been intensely dicussing internal use cases to get our short-term roadmap going:

#### Status desktop requirements
- Discussion on Waku's requirements: https://github.com/waku-org/pm/pull/328
- Thoughts on assumptions for quick wins: https://hackmd.io/OQaVT3wVScyNAyEO3UJQbQ

An initial meeting with Jo helped us get a better understanding of what was needed from Status Desktop. Plans to create FURPs the following week (spoiler: https://github.com/status-im/status-desktop/pull/18483)
Shout out to Václav, Franck, and the Waku team for being so welcoming and supporting to our efforts. We have also been talking to the Status team and should have more on that shortly.

#### New altruistic design ideas
### Altruistic Mode

* Rahul -- [using an altrustic version of Codex v1](https://hackmd.io/@codex-storage/H1uDkyLvxx)
* Eric -- [replication network](https://hackmd.io/Z3j1YBZ2SF6XKykRJ1ZF2g?both)
**Can we build a meaningful Codex without Marketplace?** There is a hypothesis that thinking of Codex without a marketplace could make things simpler, maybe even revealing a path towards faster, incremental releases. But will it really? We are trying to find out. So far we don't a lot to share, but here goes some unfiltered brain dumps for you:

#### Research efforts
1. Balazs' brain dump: https://hackmd.io/qgsdVzJ6SFC7ZVpDfWTmKQ
2. Giuliano's brain dump: https://hackmd.io/vuWTutZiSUalVZp5YRcImQ
3. Eric's brain dump: https://hackmd.io/ytUZ7Ik6QwuWbXmfcBe0Qg

* Explored DHT light client architectures and evaluated the JS-IPFS implementation.
* Investigating privacy-preserving approaches for message propagation in P2P overlay networks.
* [DHT scaling formulations](https://hackmd.io/@codex-storage/rkIWizSvee), including bandwidth estimates for content distribution, query traffic, maintenance, and repair.
* Generated traffic matrices across varying network sizes.
* Learning the details of multivariate zk protocols. [Draft writeup on sum-check](https://hackmd.io/_3xKeSMKQZ6P-kDxBEhueA)
* Started researching light-weight (off-chain) approaches to sampling proofs - KZG-based, FRI-based, and Ligero-based (mainly ZODA). Plan is to explore these more in the coming days for the altruistic mode.
* Started work on making the plonky2 proof aggregation tree as a standalone general-use repo that can possibly be used by other projects that requires N-to-1 proof tree aggregation.
### Eco Dev/comms
- Update the Codex community regarding upcoming changes
- Workshop in Zanzalu where we are gonna use:
- Testnet for the work shop and get users in. We use it for feedback. And any changed roadmap will basically motivated by "after collecting feedback from users, we've decided to adjust the roadmap priorities"


### Focus for next week

- Energy will continue in building an aligned/meaningful short-term roadmap, and figuring out the ins and outs of an altruistic mode.
90 changes: 25 additions & 65 deletions content/codex/updates/2025-08-04.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,85 +2,45 @@
title: "2025-08-04 Codex weekly"
tags:
- "codex-updates"
date: 2025-08-08
lastmod: 2025-08-11
date: 2025-08-04
lastmod: 2025-08-04
draft: false
---

## Week 32 August 2025
## Week 31 July 2025

### nim libs

* Complete the ABI encoder / decoder PR (https://github.com/status-im/nim-web3/pull/216)
* Fix offsets encoding issue
* Use `nim-fastream` api advance to avoid loading the whole buffer in memory for dynamic data
* Apply PR's feedback (comments, handling int overflow decoding...)
* Replace the old encode / decode functions by the new decoder in nim-web3
* Test in eth1 (https://github.com/status-im/nimbus-eth1/pull/3554)
* Testing in eth2 and nwaku (in progress)
* ABI encoder / decoder integration in web3 (https://github.com/status-im/nim-web3/pull/216)
* Complete decoding
* Fix dynamic data encoding
* Integration with nim-serialization
* Benchmark (https://github.com/2-towns/nim-abi-benchmark)
* extracted async iterators to a spearate Nim package - initial version, work in progress: https://github.com/codex-storage/nim-asynciterators

### Altruistic mode

#### Status desktop requirements

Status desktop requirements FURPs PR has been created: https://github.com/status-im/status-desktop/pull/18483)

#### New altruistic design ideas

* Eric -- [replication network alternative](https://hackmd.io/@codex-storage/BkN_REgulx)
* Eric -- [replication network alternative -- over simplified assumptions](https://hackmd.io/@codex-storage/S1DWaqW_ex)
* Balazs -- additions to the [altruistic musings](https://hackmd.io/jN6IN1qQQyaVSt9uGhHu9Q)

### Specs

Kicked off specs effort for existing modules.
Template: https://hackmd.io/@codex-storage/rkbytlbOxg
Kanban of components to spec:
https://github.com/orgs/codex-storage/projects/6/views/1
Due 15 August
Specs will be contained in https://github.com/codex-storage/codex-docs-obsidian
Use cases have been more or less understood through requirements gathering efforts.

### Offsite
#### Waku requirements

Have more or less decided on mid-November for the offsite, to come after a
prototype is delivered, as there is not a lot of other convenient times due to
holidays and other availability considerations (eg paternity leave). Location
will be in Europe to keep costs down. Initial idea is Canary Islands but could
move to more central Europe to drop costs further.
Final round of reviews went into the [FURPs requirements](https://github.com/waku-org/pm/pull/328), which were merged. After much discussion, the Waku use cases are summarised in https://hackmd.io/@codex-storage/B1DYfESPlg.

### Roadmap

We are working towards a roadmap that replaces bittorrent in status-go with
a prototype of the new Codex.

```mermaid

gantt
title Codex Prototype Roadmap 2025
excludes weekends
dateFormat YYYY-MM-DD

Bittorent replacement prototype :milestone, m01, 2025-11-15, 1d

section 0.1
Existing component specs :specs, 2025-08-11, 5d
Working groupathon :wg, after specs, 10d
New protocol specs :newspecs, after wg, 10d
Engineering :eng, 2025-09-08, 50d
```
#### Status desktop requirements

1. Write existing component specs, 11-15 August
2. Create a new design validated by the requirements. Deliver roadmap. (working
groupathon), 18-29 August
3. Write specs for new protocol design, 1-12 Sept
4. Start engineering, deliver BT replacement *prototype* in `status-go`, 8 September-14 Nov
An initial meeting with Jo helped us get a better understanding of what was needed from Status Desktop. Plans to create FURPs the following week (spoiler: https://github.com/status-im/status-desktop/pull/18483)

### Solutions engineering
#### New altruistic design ideas

* Updates to constellations prototype: automatic support feature, stability improvements.
* Rahul -- [using an altrustic version of Codex v1](https://hackmd.io/@codex-storage/H1uDkyLvxx)
* Eric -- [replication network](https://hackmd.io/Z3j1YBZ2SF6XKykRJ1ZF2g?both)

### Research
#### Research efforts

* Completed hotspot calculations for Zipf-based query distribution.
* Researched on Dandelion++ (Stem & Fluff) based protocols
* [Altruistic musings](https://hackmd.io/jN6IN1qQQyaVSt9uGhHu9Q)
* Explored DHT light client architectures and evaluated the JS-IPFS implementation.
* Investigating privacy-preserving approaches for message propagation in P2P overlay networks.
* [DHT scaling formulations](https://hackmd.io/@codex-storage/rkIWizSvee), including bandwidth estimates for content distribution, query traffic, maintenance, and repair.
* Generated traffic matrices across varying network sizes.
* Learning the details of multivariate zk protocols. [Draft writeup on sum-check](https://hackmd.io/_3xKeSMKQZ6P-kDxBEhueA)
* Started researching light-weight (off-chain) approaches to sampling proofs - KZG-based, FRI-based, and Ligero-based (mainly ZODA). Plan is to explore these more in the coming days for the altruistic mode.
* Started work on making the plonky2 proof aggregation tree as a standalone general-use repo that can possibly be used by other projects that requires N-to-1 proof tree aggregation.
87 changes: 66 additions & 21 deletions content/codex/updates/2025-08-11.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,40 +2,85 @@
title: "2025-08-11 Codex weekly"
tags:
- "codex-updates"
date: 2025-08-15
lastmod: 2025-08-19
date: 2025-08-11
lastmod: 2025-08-11
draft: false
---

## Week 33 August 2025
## Week 32 August 2025

### nim libs

* Update web3 in nimbus eth 2: https://github.com/status-im/nimbus-eth2/pull/7376
* Complete the ABI encoder / decoder PR (https://github.com/status-im/nim-web3/pull/216)
* Fix offsets encoding issue
* Use `nim-fastream` api advance to avoid loading the whole buffer in memory for dynamic data
* Apply PR's feedback (comments, handling int overflow decoding...)
* Replace the old encode / decode functions by the new decoder in nim-web3
* Test in eth1 (https://github.com/status-im/nimbus-eth1/pull/3554)
* Testing in eth2 and nwaku (in progress)

### Specs
### Altruistic mode

#### Status desktop requirements

The majority of the team's effort this week have been in the production of
component and protocol specifications.
Status desktop requirements FURPs PR has been created: https://github.com/status-im/status-desktop/pull/18483)

Three component specs are ready for formalisation by the RFC team:
#### New altruistic design ideas

1. https://github.com/codex-storage/codex-docs-obsidian/blob/main/10%20Notes/Component%20Specification%20-%20Discovery.md
2. https://github.com/codex-storage/codex-docs-obsidian/blob/main/10%20Notes/Component%20Specification%20-%20Erasure%20Coding.md
3. https://github.com/codex-storage/codex-docs-obsidian/blob/main/10%20Notes/Component%20Specification%20-%20Purchase.md
* Eric -- [replication network alternative](https://hackmd.io/@codex-storage/BkN_REgulx)
* Eric -- [replication network alternative -- over simplified assumptions](https://hackmd.io/@codex-storage/S1DWaqW_ex)
* Balazs -- additions to the [altruistic musings](https://hackmd.io/jN6IN1qQQyaVSt9uGhHu9Q)

The remaining component and protocol specs are in review:
### Specs

1. [Sales module specification](https://github.com/codex-storage/codex-docs-obsidian/pull/11)
1. [Proving Module Specification](https://github.com/codex-storage/codex-docs-obsidian/pull/12)
1. [Codex Manifest Spec](https://github.com/codex-storage/codex-docs-obsidian/pull/13)
1. [Store Module Specifications](https://github.com/codex-storage/codex-docs-obsidian/pull/14)
1. [Slot building specification](https://github.com/codex-storage/codex-docs-obsidian/pull/15)
1. [initial draft - block exchange module](https://github.com/codex-storage/codex-docs-obsidian/pull/16)
1. [Codex protocol](https://github.com/codex-storage/codex-docs-obsidian/pull/17)
Kicked off specs effort for existing modules.
Template: https://hackmd.io/@codex-storage/rkbytlbOxg
Kanban of components to spec:
https://github.com/orgs/codex-storage/projects/6/views/1
Due 15 August
Specs will be contained in https://github.com/codex-storage/codex-docs-obsidian

### Offsite

Dates settled November 17-22.
Location still TBD, but will be in Europe to keep costs down.
Have more or less decided on mid-November for the offsite, to come after a
prototype is delivered, as there is not a lot of other convenient times due to
holidays and other availability considerations (eg paternity leave). Location
will be in Europe to keep costs down. Initial idea is Canary Islands but could
move to more central Europe to drop costs further.

### Roadmap

We are working towards a roadmap that replaces bittorrent in status-go with
a prototype of the new Codex.

```mermaid

gantt
title Codex Prototype Roadmap 2025
excludes weekends
dateFormat YYYY-MM-DD

Bittorent replacement prototype :milestone, m01, 2025-11-15, 1d

section 0.1
Existing component specs :specs, 2025-08-11, 5d
Working groupathon :wg, after specs, 10d
New protocol specs :newspecs, after wg, 10d
Engineering :eng, 2025-09-08, 50d
```

1. Write existing component specs, 11-15 August
2. Create a new design validated by the requirements. Deliver roadmap. (working
groupathon), 18-29 August
3. Write specs for new protocol design, 1-12 Sept
4. Start engineering, deliver BT replacement *prototype* in `status-go`, 8 September-14 Nov

### Solutions engineering

* Updates to constellations prototype: automatic support feature, stability improvements.

### Research

* Completed hotspot calculations for Zipf-based query distribution.
* Researched on Dandelion++ (Stem & Fluff) based protocols
* [Altruistic musings](https://hackmd.io/jN6IN1qQQyaVSt9uGhHu9Q)
Loading