NIIIFTY x Storacha

[edsilv]

Project Name: NIIIFTY x Storacha

Proposal Category: Storage

Individual or Entity Name: Mnemoscene Ltd

Proposer: edsilv

Project Repo(s): https://github.com/NIIIFTY/NIIIFTY, Storacha client library (TBD)

(Optional) Filecoin ecosystem affiliations: Storacha

(Optional) Technical Sponsor: Dietrich Ayala

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT and APACHE2 licenses?: Yes

Project Summary

This project proposal builds upon the success of Mnemoscene's previous Filecoin devgrant project, NIIIFTY.

We are delighted to have received positive feedback from our client Royal Holloway, demonstrating their interest in improving upon and testing the software for the benefit of their faculty and students.

Impact

In 2021, Mnemoscene successfully delivered the ARStoryDecks project for Royal Holloway as part of the StoryFutures Train the Trainer programme. This project introduced an augmented reality storytelling platform, enabling the display of 3D models on mobile devices using markers on cards. The project's success led to Will Shüler, Vice-Dean at Royal Holloway, being shortlisted for the prestigious Times Higher Education "Most Innovative Teacher of the Year" award.

However, a setback occurred several months after the project's launch when Google discontinued their 3D model hosting platform, known as "Poly." This platform played a crucial role in the ARStoryDecks project, and its closure meant that existing stories would no longer function, and new stories could not be created.

Furthermore, Royal Holloway has recently implemented an "Acting 360" course for their students, incorporating the use of 360-degree video hosted on YouTube. However, during a recent visit to Shanghai to build partnerships with other universities, they encountered a challenge. YouTube was inaccessible in China, preventing them from sharing these videos with their counterparts.

Outcomes

Royal Holloway is interested in leveraging the 3D model upload, compression, and retrieval capabilities of NIIIFTY. These features could serve as a viable alternative to the now defunct Google Poly platform, allowing them to revive the ARStoryDecks project. Additionally, they see value in NIIIFTY's utilisation of Filecoin (via storacha.network) as a storage medium. This would ensure that their uploaded data is backed up in multiple locations, safeguarding against potential platform failures.

The implementation of IPFS/Filecoin also presents an attractive solution for storing their 360 video content, granting access to their Chinese partners via censor-proof URLs.

To proceed, we propose setting up an instance of NIIIFTY and Storacha account specifically for Royal Holloway.

Furthermore, we will make necessary modifications to the ARStoryDecks website, enabling it to load 3D models from Royal Holloway's NIIIFTY instance rather than relying on Google Poly. NIIIFTY will also require a new Google Poly style search function in order for ARStoryDecks users to be able to locate assets to use in their stories.

We aim to expand the capabilities of NIIIFTY to support the upload and preview of 360 video content.

We also need to make uploaded content discoverable via search by implementing auto-tagging of 3D model thumbnails and 360 videos using the Label Images with Cloud Vision AI and Label Videos with Cloud Video AI Firebase extensions.

Value

• ARStoryDecks becomes usable again by Royal Holloway academics and students.
• Uploaded content is backed up permanently on the decentralised web and is accessible in countries where censorship is an issue.

Deliverables

• A modified version of ARStoryDecks that can search for and load 3D models (glb) from NIIIFTY/IPFS.
• A NIIIFTY extension to allow the upload of GoPro Max 360 video content with in-browser preview.
• Extended documentation at niiifty.com/docs to include the newly developed features.

Adoption, Reach, and Growth Strategies

The target audience is higher education institutions running creative arts and design courses. There are 338 Universities in the UK alone who offer creative arts and design degrees. Currently we engage with Universities like Royal Holloway through programs like StoryFutures.

Development Roadmap

Milestone 1: 3D model search and loading into ARStoryDecks

• Implement a Search Files by Title feature in the NIIIFTY web UI with ability to limit results by file type and uploader user id.
• Implement the Label Images with Cloud Vision AI Firebase extension to generate labels for 3D model thumbnails in order to provide extra searchable terms and greater discoverability.
• Implement a Google Cloud function to enable ARStoryDecks to search the files stored in NIIIFTY via an API call directly within the Add Card web UI.

January 6th - March 14th 2025
10 weeks by one full stack developer

Milestone 2: 360 video

• Implement a 360 video viewer for mp4 files uploaded to NIIIFTY.

March 17th - April 11th 2025
4 weeks by one full stack developer

Milestone 4: Documentation

• Update niiifty.com docs to reflect changes from v1

April 14th - April 18th 2025
1 week by one full stack developer

Total Budget Requested

• 14 weeks of development @ $600 p/d
• 1 week of documentation @ $600 p/d

Total: $45,000.00

Maintenance and Upgrade Plans

Potential paths forward for maintenance/upgrades will largely depend on uptake from the community and any further funded projects that emerge. It is our intention to promote NIIIFTY to this end.

Team Members
Edward Silverton
[email protected]
Github profile

Team Member LinkedIn Profiles
Edward Silverton’s Linkedin profile
Will Shuler’s Academic Pure Profile

Team Website
Mnemoscene

Relevant Experience

Edward Silverton is a full stack web developer with more than 20 years of experience, 10+ years of which working with galleries, libraries, archives, and museums. He is experienced in IPFS, nodejs, next.js, tailwindcss, and firebase, is the lead developer of NIIIFTY, the Universal Viewer, and Exhibit, and is co-chair of the IIIF 3D Technical Specification Group. He presented at the 2018 Internet Archive Decentralised Web Summit on the subject of P2P IIIF.

Will Shuler is the Vice Dean of Education and Student Experience for the School of Performing and Digital Arts at Royal Holloway, University of London, where he has worked as a Senior Lecturer in Drama and Theatre Studies since 2014. He has experience developing funded pedagogic projects integrating new technologies in performance training, including storytelling for Cinematic Virtual Reality and actor training for Volume LED Virtual Production. His published work ranges from articles in Times Higher Education such as “Augmented Reality in Teaching: Key Challenges and How to Overcome Them” and book chapters such as “The Emancipated Educator: Chance, Will and Intellectual Equality in Higher Education Role-Immersion Pedagogies” in Rancière and Performance (paperback re-release 2023).

[ErinOCon]

Hi @edsilv, thank you for your proposal! This proposal has reached the final round of our review. We will be in touch if our team has any questions before proceeding with a final decision.