[Feature]: Create a custom web framework for console browser apps (like Miiverse)

[jonbarrow]

Checked Existing

• I have checked the repository for duplicate issues.

What feature do you want to see added?

Create a custom, modern feeling, web framework for building web apps on the Wii U and 3DS

Why do you want to have this feature?

No modern frameworks currently on the market work (React, Vue, etc.) on the Wii U and 3DS. This makes it incredibly annoying to build web apps for these platforms at times, since we have to raw dog the browser APIs. The platforms also give us access to custom JavaScript APIs to interact with the host console in a lot of useful ways, which existing frameworks obviously don't know about, so we would have to build those components ourselves anyway. Building our own framework would help streamline development of applications like Miiverse and potentially make it easier for newer devs who are more comfortable with modern frameworks to contribute

Any other details to share? (OPTIONAL)

A small amount of work was done for this already but nothing massive:

• https://github.com/PretendoNetwork/NWFX - NWFX is an HTMX-like library but optimized for the Wii U/3DS. It supports a subset of HTMX's features, but not all of them, and is still largely incomplete. This also isn't really a "framework" like most modern devs would think of
• https://github.com/PretendoNetwork/Yeah - Yeah (named after the Miiverse "yeah" system) was an attempt to create a React-style framework that used JSX for building apps for the Wii U/3DS, but it went absolutely nowhere. All it ended up being was a shitty JSX compiler, hardly a "framework"

I've also begun to REALLY enjoy Vue, and Vue might be what we migrate our other web apps (website, admin panel, etc.) to as well, so maybe we should take some design notes from them for this framework? Just to keep things consistent?

[jonbarrow]

Something to look into: I wonder if these browsers support SSE? I doubt all of them (there were several variations) support things like WebSockets

[jonbarrow]

Relevant to both PretendoNetwork/juxtaposition#14 and PretendoNetwork/juxtaposition#12

I have no real code written for this, but I've been toying with the idea/design of a framework for this. The goals would be:

• Use JSX for templating for better integration
• Rely almost entirely on SSR to keep the client-side payload small (as opposed to something like React, which handles things on the client by default). The Wii U and especially 3DS are very slow clients, so keeping their workload low is a priority
• 1st party support for NWFX
• Use SSE if possible for real-time updates (unsure if the Wii U/3DS Miiverse clients support this)
• Ship a minimal abstraction layer for the console-specific browser APIs. Miiverse ships as a browser app, with a customized JS engine that allows the application to interact with the host system/account directly. However this is implemented differently per console, so a small abstraction layer that makes this console-agnostic and improves the method names and such should improve ergonomics. For example, on the 3DS to get the users drawn image as a TGA you would call cave.memo_getImageRawTga(), and on the Wii U call wiiuMemo.getImage(true) (true meaning "is TGA"). This could be simplified down to a single getDrawingTGA function, which internally checks the host system and makes the correct function call
• Modern feeling application design, by abstracting away some of these layers (IE, you shouldn't need to write the SSE initialization code yourself, that should be generated at build time)

Like I said, no code built for this yet but as a basic design I was thinking something like:

juxtaposition
└── src
    ├── api
    └── pages
        └── index.tsx

Where pages contains the pages, using file-based routing, which export JSX for SSR, and api would be an API for data querying (like as mentioned in PretendoNetwork/juxtaposition#10)

index.tsx would look like:

// At build time, the resulting server will check for the NWFX headers
// and then select either the main route data or the NWFX data to send
// back to the client

export function route() {
    // Export the main JSX used to render this entire page,
    // as if you were viewing it for the first time
}

export function nwfx() {
    // Export only the data needed to update the UI when
    // navigating to this page with NWFX
}

Things like the console/account abstraction library would be automatically shipped and available (similar to how Socket.IO hooks into Express and automatically serves the client library https://socket.io/docs/v4/tutorial/step-3, except we wouldn't be dealing with the HTML ourselves at all)

I'm unsure how best to model the SSE portions of this however. Ideally this would also be something that would just be abstracted away for the most part, so we don't have to deal with the specifics. Let the build step take care of that. But we have to do something to actually handle the live data. Maybe something like:

// At build time, the resulting server will check for the NWFX headers
// and then select either the main route data or the NWFX data to send
// back to the client

export function route() {
    // Export the main JSX used to render this entire page,
    // as if you were viewing it for the first time
}

export function nwfx() {
    // Export only the data needed to update the UI when
    // navigating to this page with NWFX
}

export function sse() {
    // Handler for SSE data? Idk I'm spit-balling here
}

But that's kinda ugly, and not super clear on if that's the server-side part or the client-side part. Speaking of the server side, since SSE is kinda basic it lacks built-in auth. I believe there's a way to get the current service token from the Miiverse apps? I know the Wii U has wiiuNNA.getServiceToken which gets the service token, but idk if wiiuNNA is even available in Miiverse on the Wii U (I think it is though). Also the closest I could find for the 3DS is cave.act_acquireAccountTokenEx(password) but that requires the users PNID password and afaik it requests a new token each time, not gets the cached one. I've checked in Ghidra and can't find anything specifically for this, but maybe there's still something? The 3DS does have several functions referring to a "cached service token" but idk how to access it

I'm also thinking that the https://github.com/PretendoNetwork/Yeah repo should be repurposed for this framework

CC @mrjvs @CaramelKat @DaniElectra for suggestions

[CaramelKat]

I think that having a framework like this would be solid, a lot of Juxt already is broken up by "partials" that check for the pjax flag and only deliver essential data so jumping from that to this wouldn't take much work in the short term (aka just migrating without the redesign being done since the design folks are still working on that). I do also like the idea of a shim library for mirroring the console functions across them, but we need to be very careful with it's implementation. The 3DS in particular has a lot of functions defined, but non functional. For example the function you called out above cave.memo_getImageRawTga() is a stub, but cave.memo_getImageBmp() is not. This leads to a lot of different handling ends up split between the 3DS and Wii U server side.

As for the support for SSE, it's not something I've tried. Same thing with the token caching. Right now Juxt does not deal with expired tokens, it just kicks you out so you re-request on from the account server. Gross but functional.

[jonbarrow]

I think that having a framework like this would be solid, a lot of Juxt already is broken up by "partials" that check for the pjax flag and only deliver essential data so jumping from that to this wouldn't take much work in the short term (aka just migrating without the redesign being done since the design folks are still working on that)

That's great actually, sounds like this won't be too painful of a migration then

I do also like the idea of a shim library for mirroring the console functions across them, but we need to be very careful with it's implementation. The 3DS in particular has a lot of functions defined, but non functional. For example the function you called out above cave.memo_getImageRawTga() is a stub, but cave.memo_getImageBmp() is not

The function I used was just an example, as it was the first one I saw in both type defs that lined up. I know that the 3DS has a lot of unused methods. I agree we will have to be careful about this, but I think it's more than doable with a bit of effort

This leads to a lot of different handling ends up split between the 3DS and Wii U server side

I think this is more of a design issue than anything. Going back to the example before, if the 3DS does not use TGAs but the Wii U doesn't, a single endpoint should still be able to handle both images. There really shouldn't be a need for platform-specific handling in the case of image uploads

As for the support for SSE, it's not something I've tried. Same thing with the token caching. Right now Juxt does not deal with expired tokens, it just kicks you out so you re-request on from the account server. Gross but functional

I am not currently setup for local Juxt dev on my new Mac. Can you run the following server and then connect to it on both your Wii U and 3DS in the Miiverse apps? That should tell us if SSE is even usable or not (this should Just Work, but might need tweaking for the Miiverse apps, idk):

const express = require('express');

const app = express();
const PORT = 3000;

app.get('/', (request, response) => {
    response.send(`
        <!DOCTYPE html>
        <html lang="en">
        <body>
            <script>
                var eventSource = new EventSource('/sse'); // Connect to the SSE endpoint

                // This is only used if no "event" is sent in the message
                eventSource.onmessage = function(event) {
                    alert(event);
                };

                // These are used to capture specific "event" type messages
                eventSource.addEventListener('connected', function(event) {
                    alert('Connected event:', event.data);
                });

                eventSource.addEventListener('timestamp', function(event) {
                    alert('Timestamp event:', event.data);
                });

                eventSource.onerror = function() {
                    alert('Connection lost');
                };
            </script>
        </body>
        </html>
    `);
});

app.get('/sse', (request, response) => {
    response.setHeader('Content-Type', 'text/event-stream');
    response.setHeader('Cache-Control', 'no-cache');
    response.setHeader('Connection', 'keep-alive');
    response.flushHeaders();

    console.log(request.headers);

    function sendEvent(event, data) {
        if (event) {
            response.write(`event: ${event}\n`);
        }

        response.write(`data: ${JSON.stringify(data)}\n\n`);
    };

    sendEvent('connected', { message: 'connected' });

    const interval = setInterval(() => {
        sendEvent('timestamp', { timestamp: Date.now() });
    }, 1000);

    request.on('close', () => {
        clearInterval(interval);
        response.end();
    });
});

app.listen(PORT, () => {
    console.log(`SSE server running on http://localhost:${PORT}`);
});

Additionally, it logs the request headers. Oman said that the Miiverse apps ALWAYS send the service token in requests, which is great. If SSE is supported by the clients, AND it sends the service token in the SSE request, then we're golden when it comes to authentication. However if it DOES NOT send the service token in the SSE request, we're going to HAVE to come up with a different model. The easiest one would be to use query params. SSE requests don't natively support adding custom headers or anything, so query params are really our only bet

If SSE is supported, then I think the cleanest way to handle this from our side is to not define an sse method at all, and instead just delegate that to the library/builder. For example, something like this:

import { sse } from '@pretendonetwork/yeah';

export function route(request: SomeRequestTypeDef) {
	if (someCondition) {
		sse.sendTo(request.pid, 'new_message', dmData); // Alert specific user of a new DM message
	} else {
		sse.sendAll('notification', notificationData); // Alert all connected users of a global notification
	}
}

Behind the scenes, the library/app builder will handle the authentication (we can provide a function for token auth to make it flexible, like we do for the NEX servers) and set the request properties, will inject the data needed to establish the SSE connection on the client (like how SocketIO does), and then just give us a simple API to call. That way we don't have to worry about managing connections ourselves?