How to decode cbor from ucanto cbor transport?

[gobengo]

Context

• I ran w3ui example https://github.com/web3-storage/w3ui/tree/main/examples/react/sign-up-in
• When I did, it showed me form, so I filled it out and hit enter
• I see a request go out over the wire in cbor. I want to make sure this CBOR is encoding the request I intend.

Question

• how can I decode the http request contents I see in my browser network tab?

Here's something I tried:

• copy the cbor out of browser network tab
• use xxd to hex encode it: on mac pbcopy | xxd -p
• decode hex using cbor.me
  • gets error Expected one of [0-9], '(', '.', [Ee], [ \t\n\r], "/", ',' at line 1, column 2 (byte 2) after
• use @ipld/dag-cbor npm module to decode (see script below)
  • gets error:

    file:///Users/bengo/test-cbor-ipld/node_modules/cborg/esm/lib/decode.js:126
       throw new Error(`${ decodeErrPrefix } too many terminals, data makes no sense`);
             ^
    
    Error: CBOR decode error: too many terminals, data makes no sense
       at Module.decode (file:///Users/bengo/test-cbor-ipld/node_modules/cborg/esm/lib/decode.js:126:11)
       at decode (file:///Users/bengo/test-cbor-ipld/node_modules/@ipld/dag-cbor/src/index.js:116:39)
       at file:///Users/bengo/test-cbor-ipld/index.js:75:13
       at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
       at async Promise.all (index 0)
       at async ESMLoader.import (node:internal/modules/esm/loader:526:24)
       at async loadESM (node:internal/process/esm_loader:91:5)
       at async handleMainPromise (node:internal/modules/run_main:65:12)
    
    Node.js v18.10.0
    

Example ucanto car+cbor request body

:¢eroots�Ø*X%��q� ��[}U·ÛmOähÅ�×�dìIÑ	úG�+L�[I^8>:gversion�Ð��q� ��[}U·ÛmOähÅ�×�dìIÑ	úG�+L�[I^8>:§asY���¤À���¾®¿¯¬9Â��ßnZu���]gÓ�{jë�é Dÿ,<y�üÂX�ý���ß� ®Îhüg�IåýÏ:�ý,�ê�	þ%qäæÀ���#Þ× �4^�ìÐyoW��lèêwOï`8Ì����sóÍ�" aB��¦âÄ9;@N¤��Ãã�=��T�2é2´f?õþ»«¶òù²n¶`/à×El]u(åÃqèá¿wÒ�\�Ýt�$Ë�ÿ*MmiO츯ì?T«������¥yé¸]`å(ç©�v°5¼�0²�ðâ³ÇGäW»�R�Ô»=×¹�>�U¬ilbia�Èñ�ö¥�O��6Çû*¯��s�5B\ave0.9.1catt�£bnb£gproductlproduct:freegservicetdid:web:web3.storagehidentityx�mailto:[email protected]/claimdwithx8did:key:z6MkiNtM6bV4nEgxZfx8qbSbfEo4GDwQ9P2sFYXuePmW2GujcaudR��web:web3.storagecexp�c´Ë�cissY���$0��
�����ÓjÝ�íB��b7ê7²¢�±�ï¥-ËÈÿ å3V8ãðd
ÊúÙzD¥ßm·HwÜMó��±k£ñTÞUÔGu©ò��ãE��xH~b@ÚE�I�£O&E²�<¿,Òb�¡JgÞs(;�)kܶ¶f5£ÓI^À4M�db\�2Yæ#T*¶=Kéñú�]ó
ÀgpPõõ���RÓ��lC|�Ýʺ-�qÏp¥j�¿<ä�Ä�Ãz�Ae/SeÏN½.Ô¦ã-é\�ÏÕ¥o=���ÇäÝ0�Õ{�ìRå[Û\�9Î�PÝ¥|Æ'8²ÿ�¦¤SÈ�¹5´öØ���Ý'ßR���Éýô*q�U�De·�X÷�����cprf�Ø*X%��q� ·ØÃ`.ãÄ=�Ì�BFª��9
Ïüúî�êP/�Ü9ç����q� ·ØÃ`.ãÄ=�Ì�BFª��9
Ïüúî�êP/�Ü9ç�§asXDí¡�@Ë÷/ÞôÆ=A²�ªnÔhì��¯¡Ù±¶ô�í�÷ï.¿X���a¬H�Èu~ó'Ôo×�°-^Ãɲ¡zå�Þ8_��ave0.9.1catt�¢ccana*dwithx8did:key:z6MkiNtM6bV4nEgxZfx8qbSbfEo4GDwQ9P2sFYXuePmW2GujcaudY���$0��
�����ÓjÝ�íB��b7ê7²¢�±�ï¥-ËÈÿ å3V8ãðd
ÊúÙzD¥ßm·HwÜMó��±k£ñTÞUÔGu©ò��ãE��xH~b@ÚE�I�£O&E²�<¿,Òb�¡JgÞs(;�)kܶ¶f5£ÓI^À4M�db\�2Yæ#T*¶=Kéñú�]ó
ÀgpPõõ���RÓ��lC|�Ýʺ-�qÏp¥j�¿<ä�Ä�Ãz�Ae/SeÏN½.Ô¦ã-é\�ÏÕ¥o=���ÇäÝ0�Õ{�ìRå[Û\�9Î�PÝ¥|Æ'8²ÿ�¦¤SÈ�¹5´öØ���Ý'ßR���Éýô*q�U�De·�X÷�����cexpöcissX"í�:Oãè���i�¡��¤
/�j3]KH��$%�a��C�cprf�

[gobengo]

related: https://talk.fission.codes/t/inspecting-a-cbor-file-from-the-command-line/3086

[gobengo]

update: it's because those bytes aren't CBOR, they're CAR files (with dag-cbor blocks)

[gobengo]

The following script tries to decode using both @ucanto/transport car codec and @ipld/car CarReader, but still gets an error when parsing the underlying CAR header as dag-cbor.

error

> node ./src/decode.js

error decoding using @ucanto/transport/car Error: CBOR decode error: tag not supported (2)
    at tokensToObject (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/[email protected]/node_modules/cborg/esm/lib/decode.js:108:11)
    at Module.decode (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/[email protected]/node_modules/cborg/esm/lib/decode.js:118:19)
    at decode (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/dag-cbor/src/index.js:116:39)
    at readHeader (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/decoder.js:84:17)
    at async file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/decoder.js:222:20
    at async decodeReaderComplete (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/reader-browser.js:195:18)
    at async Module.decode (file:///Users/bengo/protocol.ai/ucanto/packages/transport/src/car/codec.js:95:18)
    at async main (file:///Users/bengo/protocol.ai/ucanto/packages/transport/src/decode.js:86:27)
error decoding using @ipld/car CarReader Error: CBOR decode error: tag not supported (2)
    at tokensToObject (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/[email protected]/node_modules/cborg/esm/lib/decode.js:108:11)
    at Module.decode (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/[email protected]/node_modules/cborg/esm/lib/decode.js:118:19)
    at decode (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/dag-cbor/src/index.js:116:39)
    at readHeader (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/decoder.js:84:17)
    at async file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/decoder.js:222:20
    at async decodeReaderComplete (file:///Users/bengo/protocol.ai/ucanto/node_modules/.pnpm/@[email protected]/node_modules/@ipld/car/src/reader-browser.js:195:18)
    at async main (file:///Users/bengo/protocol.ai/ucanto/packages/transport/src/decode.js:93:20)

script

import * as url from 'node:url';
import { CarReader, CarWriter } from '@ipld/car'
import { CarBlockIterator } from '@ipld/car/iterator'
import { CarCIDIterator } from '@ipld/car/iterator'
import * as ucantoTransportCar from './car/codec.js'

const hexedCar = `
3ac2a265726f6f7473c281c3982a58250001711220c2940f5b7d55c2b7c3
9b6d4fc3a468c385c292c3971d64c3ac49c39109c3ba47c29d2b4cc2915b
495e383e3a6776657273696f6e01c3900601711220c2940f5b7d55c2b7c3
9b6d4fc3a468c385c292c3971d64c3ac49c39109c3ba47c29d2b4cc2915b
495e383e3ac2a76173590106c285c2a4c38006c28002c2bec2aec2bfc2af
c2ac39c3821c7fc39f6e5a7506c29b1e5d67c393c29a7b6ac3abc285c3a9
c2a044c3bf2c3c791cc3bcc38258c292c3bd12c29014c39f1220c2aec38e
68c3bc670349c3a5c3bdc38f3a16c3bd2cc28cc3aac29909c3be2571c3a4
c3a6c380191a0623c39ec397c2a0c294345e02c3acc390796f570c04076c
c3a8c3aa0c774fc3af6038c38cc29317c2980f73c3b3c38d0b22206142c2
91c289c2a6c3a2c384393b404ec2a4c28a14c383c3a3113d08c29654c29c
32c3a932c2b4663fc3b5c3bec2bbc2abc2b6c3b2c3b9c2b26ec2b6602fc3
a0c397456c5d7528c3a5c38371c3a8c3a1c2bf77c392c2985cc287c39d74
c29224c38bc29ec3bf2a4d6d694fc3acc2b8c2afc3ac3f54c2ab101d1bc2
861d0ec2a579c3a9c2b85d60c3a528c3a7c2a9c29276c2b035c2bcc29930
c2b210c3b0c3a2c2b3c38747c3a457c2bbc28952c281c394c2bb3dc397c2
b9c2853ec28255c2ac696c626961c28ac388c3b106c3b6c2a5c2844fc298
c29736c387c3bb2ac2afc29c0073c28e35425c617665302e392e31636174
74c281c2a3626e62c2a36770726f647563746c70726f647563743a667265
656773657276696365746469643a7765623a776562332e73746f72616765
686964656e74697479781b6d61696c746f3a62656e676f6572696e674067
6d61696c2e636f6d6363616e6d766f75636865722f636c61696d64776974
6878386469643a6b65793a7a364d6b694e744d366256346e4567785a6678
387162536266456f3447447751395032734659587565506d573247756a63
61756452c29d1a7765623a776562332e73746f72616765636578701a63c2
b4c38b1063697373590110c2852430c282010a02c282010100c3936ac39d
c29dc3ad4214066237c3aa37c2b2c2a218c2b1c288c3afc2a52dc38bc388
c3bfc2a0c3a5335638c3a3c3b0640ac38ac3bac3997a44c2a5c39f6dc2b7
4877c39c4dc3b3c28fc292c2b16bc2a3c3b154c39e55c3944775c2a9c3b2
0c18c29dc3a345c2921078487e6240c39a451b4914c2a34f2645c2b2103c
c2bf2cc39262c29fc2a14a67c39e73283b14290c6bc39cc2b6c2b66635c2
a3c393495ec380344dc28d64625c173259c3a623542ac2b63d4bc3a9c3b1
c3bac29a5dc3b30ac380677050c3b5c3b5c29214c28d52c393c290c2986c
437c08c39dc38ac2ba2dc28971c38f70c2a56a01c2bf3cc3a4c28fc384c2
95c3837a0641652f5365c38f4ec2bd2ec394c2a6c3a32dc3a95c1fc38fc3
95c2a56f3dc299c29108c387c3a4c39d30c282c3957bc283c3ac52c3a55b
c39b5cc28039c38ec29b50c39dc2a57cc3862738c2b2c3bfc29bc2a6c2a4
53c388c28ec2b935c2b4c3b6c398c28d16c290c39d27c39f5217001cc389
c3bdc3b42a71c28a55c28e4465c2b70358c3b7020301000163707266c281
c3982a58250001711220c2b70cc398c383602ec3a3c3843d12c38c124246
c2aa1dc288390ac38fc3bcc3bac3ae0fc3aa502fc295c39c39c3a71ac289
0401711220c2b70cc398c383602ec3a3c3843d12c38c124246c2aa1dc288
390ac38fc3bcc3bac3ae0fc3aa502fc295c39c39c3a71ac2a761735844c3
adc2a10340c38bc3b72fc39ec3b4c3863d41c2b21cc2aa6ec39468c3ac1b
17c2afc2a1c399c2b1c2b6c3b4c290c3ad08c3b7c3af2ec2bf58c295c29b
c29161c2ac48c28fc388757ec3b327c3946fc397c285c2b02d5ec383c389
c382c2b2c2a10c7ac3a503c39e385fc2980f617665302e392e3163617474
c281c2a26363616e612a647769746878386469643a6b65793a7a364d6b69
4e744d366256346e4567785a6678387162536266456f3447447751395032
734659587565506d573247756a63617564590110c2852430c282010a02c2
82010100c3936ac39dc29dc3ad4214066237c3aa37c2b2c2a218c2b1c288
c3afc2a52dc38bc388c3bfc2a0c3a5335638c3a3c3b0640ac38ac3bac399
7a44c2a5c39f6dc2b74877c39c4dc3b3c28fc292c2b16bc2a3c3b154c39e
55c3944775c2a9c3b20c18c29dc3a345c2921078487e6240c39a451b4914
c2a34f2645c2b2103cc2bf2cc39262c29fc2a14a67c39e73283b14290c6b
c39cc2b6c2b66635c2a3c393495ec380344dc28d64625c173259c3a62354
2ac2b63d4bc3a9c3b1c3bac29a5dc3b30ac380677050c3b5c3b5c29214c2
8d52c393c290c2986c437c08c39dc38ac2ba2dc28971c38f70c2a56a01c2
bf3cc3a4c28fc384c295c3837a0641652f5365c38f4ec2bd2ec394c2a6c3
a32dc3a95c1fc38fc395c2a56f3dc299c29108c387c3a4c39d30c282c395
7bc283c3ac52c3a55bc39b5cc28039c38ec29b50c39dc2a57cc3862738c2
b2c3bfc29bc2a6c2a453c388c28ec2b935c2b4c3b6c398c28d16c290c39d
27c39f5217001cc389c3bdc3b42a71c28a55c28e4465c2b70358c3b70203
01000163657870c3b6636973735822c3ad013a4fc3a3c3a8c2947f106915
c2a1c28ac290c2a40a2fc28e6a335d4b48c297132425c2886100c29e43c3
97c28663707266c280
`.trim()

/**
 * @param {string} hexString
 */
const fromHexString = (hexString) =>
  Uint8Array.from((hexString.match(/.{1,2}/g) ?? []).map((byte) => parseInt(byte, 16)));

async function main() {
  const carBytes = fromHexString(hexedCar)

  try {
    const ucantoDecoded = await ucantoTransportCar.decode(carBytes)
    console.log('ucanto decoded', ucantoDecoded)
  } catch (error) {
    console.error('error decoding using @ucanto/transport/car', error)
  }

  try {
    const reader = await CarReader.fromBytes(carBytes)
    console.log('CarReader roots', await reader.getRoots())
  } catch (error) {
    console.error('error decoding using @ipld/car CarReader', error)
  }
}

// run main() if this file is run directly
if (import.meta.url.startsWith('file:')) {
  const modulePath = url.fileURLToPath(import.meta.url);
  if (process.argv[1] === modulePath) {
    main().catch(error => {
      console.error('error in main', error)
    })
  }
}

[MathPSantos]

@gobengo Did you have success trying to fix it? I am having an issue with @ipld/car package and I am not able to fix it