use local codegen for protobuf message types #1624

turbocrime · 2024-08-02T09:59:54Z

fixes #1483 #1503

replaces #1582 #1583

it's simplest to use buf generate to create the definitions

changeset-bot · 2024-08-02T09:59:58Z

🦋 Changeset detected

Latest commit: bf60c3f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages

Name	Type
@penumbra-zone/protobuf	Major
minifront	Patch
node-status	Patch
@penumbra-zone/bech32m	Major
@penumbra-zone/client	Major
@penumbra-zone/getters	Major
@penumbra-zone/perspective	Major
@penumbra-zone/query	Major
@penumbra-zone/services	Major
@penumbra-zone/storage	Major
@penumbra-zone/types	Major
@repo/ui	Major
@penumbra-zone/wasm	Major
@penumbra-zone/crypto-web	Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

turbocrime · 2024-08-02T10:01:43Z

package.json

    "@buf/tendermint_tendermint.bufbuild_es": "1.10.0-20231117195010-33ed361a9051.1",
    "@bufbuild/protobuf": "^1.10.0",
    "@connectrpc/connect": "^1.4.0",
    "@connectrpc/connect-web": "^1.4.0"
  },
  "devDependencies": {
+    "@buf/penumbra-zone_penumbra.bufbuild_es": "1.10.0-20240703080008-312294d02bf9.1",


this one buf dep is still necessary because the registry incorrectly fails to declare a dependency

turbocrime · 2024-08-02T10:04:46Z

packages/protobuf/package.json

+    "gen:cosmos-sdk": "buf generate buf.build/cosmos/cosmos-sdk:e7a85cef453e4b999ad9aff8714ae05f",
+    "gen:ibc": "buf generate buf.build/cosmos/ibc:7ab44ae956a0488ea04e04511efa5f70",
+    "gen:ics23": "buf generate buf.build/cosmos/ics23:55085f7c710a45f58fa09947208eb70b",
+    "gen:penumbra": "buf generate buf.build/penumbra-zone/penumbra:312294d02bf945ffa4c1b1dd7cd91328",


versions used in core

https://github.com/penumbra-zone/penumbra/blob/main/proto/buf.lock

turbocrime · 2024-08-02T10:05:38Z

packages/protobuf/src/gen/.npmignore

empty npmignore will allow this dir to be packaged

VanishMax

Great improvement!

This is ultimately a better way than how it was done in #1582. Was there a specific reason to not continue working or commenting on #1582 ?

packages/protobuf/package.json

VanishMax · 2024-08-02T10:46:35Z

packages/perspective/src/translators/swap-claim-view.ts

@@ -2,7 +2,7 @@ import { Translator } from './types.js';
 import {
  SwapClaimView,
  SwapClaimView_Opaque,
-} from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/core/component/dex/v1/dex_pb.js';
+} from '@penumbra-zone/protobuf/penumbra/core/component/dex/v1/dex_pb';


question: any chance in the future we could simplify the imports? Maybe after applying v2 of the generator we could export types as barrels?

the service definitions have a barrel export defined, but i think trying to collect other types into barrels is likely to be very complex and not very useful.

turbocrime · 2024-08-06T04:54:38Z

going to wait until recent core message type changes are merged for ibc compat before trying to merge this

turbocrime · 2024-08-13T03:03:33Z

apps/minifront/src/state/helpers.ts

-const getTxHash = <T extends Required<PartialMessage<TransactionId>> | PartialMessage<Transaction>>(
-  t: T,
-): T extends Required<PartialMessage<TransactionId>> ? string : Promise<string> =>
-  'inner' in t && t.inner instanceof Uint8Array
-    ? (uint8ArrayToHex(t.inner) as T extends Required<PartialMessage<TransactionId>>
-        ? string
-        : never)
-    : (getTxId(t as PartialMessage<Transaction>).then(({ inner }) =>
-        uint8ArrayToHex(inner),
-      ) as T extends Required<PartialMessage<TransactionId>> ? never : Promise<string>);
-
-const getTxId = (tx: Transaction | PartialMessage<Transaction>) =>
+const txSha256 = (tx: Transaction | PartialMessage<Transaction>) =>
  sha256Hash(tx instanceof Transaction ? tx.toBinary() : new Transaction(tx).toBinary()).then(
    inner => new TransactionId({ inner }),
  );


this caused problems at some point during this work, and the types were technically incorrect, so it's now simplified

turbocrime requested a review from VanishMax August 2, 2024 09:59

turbocrime commented Aug 2, 2024

View reviewed changes

packages/protobuf/src/gen/.npmignore Outdated

Copy link

Contributor Author

turbocrime Aug 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty npmignore will allow this dir to be packaged

turbocrime force-pushed the turbocrime/bundle-proto branch 4 times, most recently from 97af744 to 185a5db Compare August 2, 2024 10:13

VanishMax approved these changes Aug 2, 2024

View reviewed changes

turbocrime force-pushed the turbocrime/bundle-proto branch 2 times, most recently from 3289a2e to 5a61797 Compare August 2, 2024 18:32

turbocrime mentioned this pull request Aug 2, 2024

use compat with local bundled protos #1632

Closed

turbocrime force-pushed the turbocrime/bundle-proto branch 2 times, most recently from 79fbeaf to 7115b4c Compare August 3, 2024 09:10

This was referenced Aug 8, 2024

feat(protobuf): #1483: bundle buf into protobuf #1582

Closed

feat: #1483: apply new protobuf imports #1583

Closed

turbocrime force-pushed the turbocrime/bundle-proto branch from 63839f4 to adafb63 Compare August 12, 2024 21:54

turbocrime added 11 commits August 12, 2024 18:30

buf gen protos

c1f13d5

remaining packages

d178385

update imports

d2d29e4

registry is missing dep

365f9e9

test failure?

e6538f0

random lint

f0dcfe8

changeset

24c187e

sort imports 2

7712f14

turbo dep graph, gen dir, tsconfig, package

a075a7f

correct bech32m dep, turbo deps

221298a

syncpack config

b0ac61a

turbocrime added 3 commits August 12, 2024 18:30

correct imports again

98b66ae

version protos

3b5398d

simplify uitility

86eae85

turbocrime force-pushed the turbocrime/bundle-proto branch from b17e61f to 86eae85 Compare August 13, 2024 01:42

turbocrime added 2 commits August 12, 2024 19:23

tests need proto

5641723

update documentation

bf60c3f

turbocrime force-pushed the turbocrime/bundle-proto branch from fa81126 to bf60c3f Compare August 13, 2024 02:38

turbocrime mentioned this pull request Aug 13, 2024

use bundled protos prax-wallet/prax#154

Merged

turbocrime changed the title ~~bundle protobuf messages~~ use local codegen for protobuf message types Aug 13, 2024

turbocrime commented Aug 13, 2024

View reviewed changes

turbocrime merged commit 49263c6 into main Aug 13, 2024
6 checks passed

turbocrime deleted the turbocrime/bundle-proto branch August 13, 2024 03:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use local codegen for protobuf message types #1624

use local codegen for protobuf message types #1624

turbocrime commented Aug 2, 2024

changeset-bot bot commented Aug 2, 2024 •

edited

Loading

turbocrime Aug 2, 2024 •

edited

Loading

turbocrime Aug 2, 2024

turbocrime Aug 2, 2024

VanishMax left a comment

VanishMax Aug 2, 2024

turbocrime Aug 13, 2024

turbocrime commented Aug 6, 2024

turbocrime Aug 13, 2024

use local codegen for protobuf message types #1624

use local codegen for protobuf message types #1624

Conversation

turbocrime commented Aug 2, 2024

changeset-bot bot commented Aug 2, 2024 • edited Loading

🦋 Changeset detected

turbocrime Aug 2, 2024 • edited Loading

Choose a reason for hiding this comment

turbocrime Aug 2, 2024

Choose a reason for hiding this comment

turbocrime Aug 2, 2024

Choose a reason for hiding this comment

VanishMax left a comment

Choose a reason for hiding this comment

VanishMax Aug 2, 2024

Choose a reason for hiding this comment

turbocrime Aug 13, 2024

Choose a reason for hiding this comment

turbocrime commented Aug 6, 2024

turbocrime Aug 13, 2024

Choose a reason for hiding this comment

changeset-bot bot commented Aug 2, 2024 •

edited

Loading

turbocrime Aug 2, 2024 •

edited

Loading