Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance NIP-01 with IPFS CIDs for Decentralized Media Storage #1285

Open
besoeasy opened this issue Jun 4, 2024 · 25 comments
Open

Enhance NIP-01 with IPFS CIDs for Decentralized Media Storage #1285

besoeasy opened this issue Jun 4, 2024 · 25 comments

Comments

@besoeasy
Copy link

besoeasy commented Jun 4, 2024

Here is the current NIP1 Object

{ "id": "3237fd9a04af4cfcca803cc1825173c44f3ab360439820d30826d2016dfc0783", "pubkey": "f680ddd8c13f1ceab92edd3495f0fce747bb80c270e081f80c7401befb62da7c", "created_at": 1717795867, "kind": 1, "tags": [ [ "r", "https://i.ibb.co/JjYZQVN/graph.png" ] ], "content": "Bitcoin Looking Beautiful 😍 \n\nhttps://i.ibb.co/JjYZQVN/graph.png", "sig": "12b47bdf2db31b2630d219cc450b78a2a17a89f209b5fb16c7032c5264735052a3d8cabe1f60ae97957e2c581e07e6808eaa1b391d941a0915d9759825c52604" }

All we need to do is to add a "cid" field that resolves to IPFS Hash

@besoeasy besoeasy changed the title IPFS as Image Storage - Should we create new NIP or merge it with NIP-35 NIP 1 - Improvement - IPFS as Image Storage Jun 7, 2024
@vitorpamplona
Copy link
Collaborator

I don't understand what you are trying to propose...

No client has an IPFS node client to pull images from the IPFS network. Are you proposing this becomes a requirement for all clients?

@besoeasy besoeasy changed the title NIP 1 - Improvement - IPFS as Image Storage Enhance NIP-01 with IPFS CIDs for Decentralized Media Storage Jun 7, 2024
@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

Description

Currently, NIP-01 supports decentralized storage for text-based content. However, it would be beneficial to extend this functionality to include multimedia files like images and videos. This can be achieved by utilizing InterPlanetary File System (IPFS) Content Identifiers (CIDs) to store and reference these files.

Example

  1. NIP-01 Update: Modify NIP-01 to include a new field for storing IPFS CIDs for multimedia files. This would allow users to upload and reference images and videos directly from their Nostr profiles.
  2. IPFS Integration: Integrate IPFS into the Nostr protocol to enable decentralized storage and retrieval of multimedia files. This would ensure that files are stored and accessed in a decentralized manner, reducing reliance on centralized servers.

Benefits

  • Decentralized Storage: IPFS CIDs would allow for decentralized storage and retrieval of multimedia files, ensuring that no single entity controls the storage or access to these files.
  • Improved Security: By using IPFS CIDs, the risk of data loss or tampering would be significantly reduced, as the files are stored in a decentralized network.
  • Enhanced User Experience: Users would have more flexibility in how they store and share multimedia content, making it easier to collaborate and interact with others.

Steps to Implement

  1. Update NIP-01: Modify NIP-01 to include the new field for storing IPFS CIDs.
  2. Integrate IPFS: Integrate IPFS into the Nostr protocol to enable decentralized storage and retrieval of multimedia files.

@vitorpamplona
Copy link
Collaborator

Why not just place it as a tag?

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

Why not just place it as a tag?

There are several reasons why using IPFS CIDs as a native reference for multimedia content is more beneficial than simply placing it as a tag:

1. Native Support in Browsers and OS

IPFS is natively supported in some mainstream browsers like Brave and many Linux-based operating systems. This means that users can directly access IPFS content without needing additional software or plugins. By using IPFS CIDs, we can leverage this native support to provide a seamless user experience.

2. Decentralized Caching

One of the significant advantages of IPFS is that the entire network caches the content. This means that even if a specific gateway goes down, users can switch to another gateway and still access the content. In contrast, if a centralized server like nostr.build goes down, a significant portion of content images would become unavailable.

3. Flexibility and Portability

IPFS CIDs can be accessed through various gateways if native support is missing. This ensures that users can always access the content, even if their browser or OS does not natively support IPFS. The use of IPFS CIDs provides a flexible and portable solution for decentralized media storage.

4. Standardization

IPFS CIDs follow a standardized format, making it easier for different platforms and applications to integrate and support them. This standardization also enables better interoperability across different systems.

5. Security and Integrity

IPFS CIDs are based on the cryptographic hash of the content, ensuring that any changes to the content result in a different CID. This provides a robust mechanism for verifying the integrity and authenticity of the content.

In summary, using IPFS CIDs as a native reference for multimedia content offers several benefits, including native support in browsers and OS, decentralized caching, flexibility, standardization, and security. These advantages make IPFS CIDs a more robust and reliable solution compared to simply placing them as tags.

I been a experienced IPFS developer, i can show a small benchmark if the community wants and really proove that IPFS as media CDN would solve our dependencies on Centerlised Image Hosters

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

I forgot to add that IPFS, supports, all content types

From Images to direct .torrent files

Everything described in all of our NIPs is supported by IPFS

@vitorpamplona
Copy link
Collaborator

vitorpamplona commented Jun 7, 2024

Why are you posting things from ChatGPT?

We know what IPFS is. That's not the question. The question is how are you going to integrate it. Changing NIP-01 is not needed to support IPFS.

In fact, you can integrate into your Nostr app without having to change any NIP. :)

@alexgleason
Copy link
Member

The way to integrate IPFS with Nostr is easy. Just include a URL like https://<gateway>/ipfs/<cid>

This is backwards and forwards compatible. I've been doing it on Ditto: https://gitlab.com/soapbox-pub/ditto/-/blob/main/src/uploaders/IPFSUploader.ts?ref_type=heads

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

i am not using chatgpt, i am using a translator AI, i do not speak english

We know what IPFS is. That's not the question. The question is how are you going to integrate it. Changing NIP-01 is not needed to support IPFS.

i understand your question, i will post comparison, as soon i get some time, showing problems with NIP 1 Object

i will mention you here when i am done !

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

The way to integrate IPFS with Nostr is easy. Just include a URL like https://<gateway>/ipfs/<cid>

This is backwards and forwards compatible. I've been doing it on Ditto: https://gitlab.com/soapbox-pub/ditto/-/blob/main/src/uploaders/IPFSUploader.ts?ref_type=heads

we need to implement it as uri ipfs:// or "cid" field

to get rid of dependencies on gateways, if ipfs content is hardcoded with gateways, it makes it no difference then nostr.build or imgbb

@alexgleason
Copy link
Member

The IPFS path gateway is standardized: https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-types

This is superior to ipfs:// URIs because it works on the legacy web, AND ipfs enabled tools can resolve it.

It is the same as an ipfs URI except it contains extra information. There is no reason not to use this.

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

The IPFS path gateway is standardized: https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-types

This is superior to ipfs:// URIs because it works on the legacy web, AND ipfs enabled tools can resolve it.

It is the same as an ipfs URI except it contains extra information. There is no reason not to use this.

example :

ipfs://QmRhZLf2ccHag8QLNEf8Jws4KXbtA4bx5K6xLf4WLa4kdn

and

https://bafybeibr5yx2gie53vmsgptybfpxfc6ykf5polikw3azhctygs2vfszcre.ipfs.dweb.link/

This demonstrates the versatility of IPFS in serving content through different methods.

My long-term vision for NOSTR is to integrate IPFS nodes as relay pools, similar to how we currently use Nostr relays. This would allow node runners to specify how much NOSTR and IPFS data they want to cache. While this is not a necessity at the moment, it is an improvement that will benefit NOSTR in the long term.

  • having IPFS integration standardised will help users self host content locally on computer without setting up a always on https:.. server

imo this makes NOSTR publicly more appealing

NOSTR is still new we can implement this, but in 2-3 years from now, this integration will be hard

AS NOSTR Community our priority is to keep protocol simple but at the same time, we are competing with a centralised Multi Billion Gaint, it doesn't look good with US telling our new users that images are not natively integrated thru a decenterlised protocol

@alexgleason
Copy link
Member

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

doesn't sound decentralised, and requires always on server i assume

@vitorpamplona
Copy link
Collaborator

The http url renders great on Amethyst. :)

Screenshot 2024-06-07 at 6 28 18 PM

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

The http url renders great on Amethyst. :)

Screenshot 2024-06-07 at 6 28 18 PM

Exactly, it renders good as long as gateway is up

see this
image

gateway goes up and down, and a good gateway for u, will not be best option for me

So if i am dictator in a country XYZ, all i need to do is to blacklist nost.build (few others CDN) and IPFS gateways and boom all of the country would not be able to access the content without VPN

@vitorpamplona
Copy link
Collaborator

Is there an Android app that serves IPFS images as a full node? If so, then we could just create a setting to load the image from the local app node instead of coding everything in Nostr.

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

Is there an Android app that serves IPFS images as a full node? If so, then we could just create a setting to load the image from the local app node instead of coding everything in Nostr.

thats smart, i really like this approach

I am afraid this can only done for non android for now, but it is possible, IPFS does have pure JS implementation

@besoeasy
Copy link
Author

besoeasy commented Jun 7, 2024

I will start a pilot with ipfs:// uri without changing NIP 01 as suggest by @vitorpamplona

i will talk to few clients devs to see if they can support this and see how it works, but i am afraid in order for this to work, we need to adopt this together

but something tells me, its not going to work well in full decentralised way unless we natively use IPFS and let them handle our troubles with files management & p2p delivery

@vitorpamplona
Copy link
Collaborator

Just develop a full IPFS client for Android first. Until that exists and becomes extremely reliable, it's useless to debate how to integrate with Nostr.

@melvincarvalho
Copy link

No changes needed in NIP-01 for this. If you want to add an IPFS URI such as ifps:// you can shoe-horn it into a tag. If you want to reference a CID but not be tied to IPFS transport, and use some other discovery method, I'd suggest using urn:cid:<CID> which decouples the hash, the transport and the discovery. Of course, you'd then need to figure out how to do discovery, perhaps using relays and lookups, but that's another NIP.

@hzrd149
Copy link
Collaborator

hzrd149 commented Jun 8, 2024

It sounds like what you want is Blossom. https://github.com/hzrd149/blossom

doesn't sound decentralised, and requires always on server i assume

Blossom is not a decentralized protocol, but it is the simplest implementation of content addressable files.
If we want censorship resistant media to fight link rot we don't need decentralized or p2p systems, at least not at the base layer. we need universal file IDs

IPFS CIDs are decent for this but they support multiple hashing algorithms which makes them complicated. Torrent hashes are good because they use a single hashing algorithm but they require you to chunk the file, which is complicated.
Blossom proposes using just the sha256 to address files. since its trivial to compute and is universally supported.

Once you have a single universal ID for files, then it becomes easy to ask multiple servers for the file. in the case of IPFS its the gateways job to "ask multiple servers" using the p2p network to find the file. for Blossom its the clients job.
Both of these approaches have their pros and cons, but I believe blossom approach is simpler since it does not rely on a p2p network or DHT table but instead relies on nostr and user specified server lists see docs

Both Blossom and IPFS store the files hash in the url which is critical for tamper resistant media and finding the file when its been taken down. However, again here I think blossom is simpler since it only has one format for urls https://blossom.example.com/<sha256>.png compared to IPFS

The point of this long reply isn't to say that IPFS wont work or isn't a good system for censorship resistant media. but simply that we don't need a system as complicated or heavy as IPFS to get censorship resistant media, we just need universal file IDs and servers that serve them by those IDs

@besoeasy
Copy link
Author

besoeasy commented Jun 9, 2024

The point of this long reply isn't to say that IPFS wont work or isn't a good system for censorship resistant media. but simply that we don't need a system as complicated or heavy as IPFS to get censorship resistant media, we just need universal file IDs and servers that serve them by those IDs

this is what i am trying to commend, either IPFS or our own in house system

@besoeasy
Copy link
Author

besoeasy commented Jun 9, 2024

Maybe easier would be to introduce a new tag in https://github.com/nostr-protocol/nips?tab=readme-ov-file#standardized-tags

["ipfs", "img", "IPFS CID"]
["ipfs", "vid", "IPFS CID"]
["ipfs", "other", "IPFS CID"]

This way it doesn't modify NIP 1, and the same time have IPFS standard for anyone looking to implement it

i see it as win-win for both sides of discussions

@alexgleason
Copy link
Member

You're overthinking this. Just upload to a public gateway like I said and use a regular URL.

@fiatjaf
Copy link
Member

fiatjaf commented Jun 9, 2024

If IPFS worked as advised we wouldn't need Nostr at all, so we should either close this or close the entirety of Nostr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants