Create a SES command line REPL

[erights]

See googlearchive/caja#1977

[warner]

What would this look like? Copying from that issue, at present you can start up node in a shell and do:

const SES = require('ses');
s = SES.makeSESRootRealm()
s.evaluate('1+a', { a: 2 }) // emits 3

What sort of features would we want on top of that?

[katelynsills]

we're looking for a compartment switching REPL. It would be like the Node REPL but be able to label and switch between the compartments

[warner]

This might look like a tool (maybe named ses) which behaves like /usr/bin/node but running it without arguments gives you a repo in a new compartment.

It'd be a compartment rather than a SES Realm because SES Realms are frozen (including the global), so the usual practice of holding temporary variables with e.g. const a = foo() wouldn't work: the global object upon which a is a property is frozen and cannot be changed. Compartments, on the other hand, inherit from those frozen globals but use a new (mutable) global object.

A transcript of this basic ses invocation would look identical to running node, except that certain non-deterministic methods would be rejected:

$ /usr/bin/ses
> let a = 4;
undefined
> Math.cos(Math.PI)
-1
> Math.rand()
> s.evaluate('Math.rand()')
TypeError: Math.rand is not a function

Basically the REPL would start by creating a SES realm with consoleMode: 'allow' and errorStackMode: 'allow', create a compartment from that, and then everything you type in gets evaluated in the SES compartment.

Phase two would be to enable multiple realms and/or compartments, with some sort of top-level command to switch between them. This would be most useful for examining the interaction between separate realms, which mostly means triggering identity-discontinuity errors. I'm not sure how useful it would actually be.

Phase three might be to use ses with arguments as the safe module loader, so just like node foo.js is used to run a program, ses foo.manifest might be a way to run a program composed of confined modules. Not sure yet.

[dckc]

All that sounded great until foo.manifest. Why??!?!? Why not just foo.js with endowments passed to main()? (or a default export of some sort)

[warner]

I hear ya. I think we're going to need some experiments to see what approaches work best. My foo.manifest leanings come from my work on Jetpack (https://github.com/warner/JetpackComponents) back when I was at Mozilla, where I wanted a data structure that was both executable (there was an interpreter that uses it to load specific code modules, confined in some way, and wire them together in a way that enforced authority-sharing properties), and auditable/reviewable (by humans, assisted with tools, to decide whether that distribution of authority would meet the desired security/functionality goals). I wanted that data structure to be declarative, so you could analyze it with other programs.

I think the alternative to a manifest file is to have some non-confined entry-point script (a main.js or whatever) build the endowments at startup, then load some yes-confined module, and pass the endowments into that module's main() function. (I think this is how .e and .emaker files are related in the E world). That's kind of the same model except now you've got Turing-complete code building the endowments and deciding how modules should be wired together, which can't be completely/correctly analyzed by other programs (yay Halting Problem).

The manifest thing might belong in a completely separate project, which uses SES to provide an application execution tool that's controlled by this declarative module-graph description thing.

I guess the question is, if node foo.blah is how you run a javascript program with unlimited authority (since foo.blah gets a require() which gets access to everything, including binary .so modules that can do everything, OS-level file-writing authorities which can modify source code to do anything, and all the usual Node builtins which can do a whole heck of a lot), and if we're thinking of building a ses foo.blah program that "runs foo.blah under SES", what does it mean to run it under SES? SES is all about limiting authority, but without some connection to the un-confined outside world, a program run under SES is entirely useless (by design!). So ses foo.blah has gotta have some sort of bridge to something that isn't SES, and we need some language to describe what the SES-confined code gets to do with that outside world, and that bridge could be a non-SES program which sets up the SES environment with endowments and such, or it could be a declarative manifest that does the same thing but in terms of modules with identifiers that other people could study and reason about and publish decisions about.

[dckc]

To reiterate my Jan 29 comment: ses foo.js should load foo.js and call its default export (or main), passing in as endowments all authority that the process has. Then main can decide how to delegate.

This works well in pony and Monte.

[warner]

Hm, interesting. So, in a Node environment, where all authority comes from require (igoring the various Node-specific globals we'd probably want to remove like vm and v8 and os), would bin/ses be implemented basically like this?:

const s = SES.makeSESRootRealm();
const main = s.evaluate(loadFileAsString(argv[1]));
main(require);

The foo.js you give to this would have the responsibility for wrapping the native require they were given to implement some sort of module system for any other code it might want to use.

I see that as being useful (like you said, the smallest thing that could possibly work), but maybe not useful enough to spend the bin/ses name on. That first foo.js file has an awful lot of work to do (defining all the possible ways to restrict the full-powered module loader), and it doesn't seem to provide an obvious way to factor out that definition work so that it could be used by other modules.

I suppose I'm actually criticizing the emaker "imperative attenuation" approach here, not just this JS-specific flavor of it. I'm imagining a more generalized solution in which a "declarative attenuation" manifest is executed/enforced, because I can imagine how that manifest could form the basis of a package registry -like ecosystem hub where people can collaborate on the ways to attenuate/combine/synthesize authority. If the main place to do that is with code in the top-level foo.js, then the auditor who wants to understand what the program does must study code to learn what gets attenuated, and that sounds more difficult.

I'd like to learn more about how this works for larger-scale development in pony and Monte, in particular if there are any cases where the useful attenuators from one project have been picked up in other projects, and how the definition of those attenuators gets transmitted between them. Do you have any pointers I could look at?

[dckc]

(does this discussion really belong in Agoric/SES#9? Oh well...)

I'm not sure the pony community does this very well. I tried and failed to convince the pony designers that the standard network library should support interposition: ponylang/ponyc#301 (comment)
In the pony net library, access to the network is implemented in primitives that use C FFI to call something like libuv. These primitives require that you pass an object that represents authority to access the network, it conforms to ocap discpline in the sense of "you can't connect to the network unless main() delegates such access to you" but it doesn't let me interpose "you can only connect on Tuesdays" logic for all users of this standard net library. Interfaces that support such interposition can be built on top of the primitive + token API, but that doesn't make them ubiquitous.

A quick browse turns up a pure pony postgres client library. Note the way its connection constructor wants the AmbientAuth "god capability", which is even more than the TCPConnection primitive requires:

  new create(auth: AmbientAuth,
             host: String,
             service: String,
             params: Array[(String, String)] val,
             pool: ConnectionManager,
             out': OutStream
             ) =>
    _conn = TCPConnection(auth, PGNotify(this), host, servic

https://github.com/lisael/pony-postgres/blob/master/pg/connection/tcp.pony#L38-L45

I don't think there is any Monte development at the scale of independent projects. :-/ But at least all of Monte's i/o APIs support interposition.

The fact that the first foo.js has so much work to do is a right royal pain; that is: not only does the node API have ambient authority, but it also has pervasive mutability, object graphs that allow traversal from low-privilege to high-privilege objects, and so on. To delegate access to http.createServer, which in Monte would involve passing one argument from main() to another function, took 20 lines of javascript: https://github.com/dckc/do-your-worst/blob/master/server.js#L14-L34 In Monte, the HTTP library is an emaker; the only powerful reference needed is the one for creating TCP endpoints. In node, loading the http module as an emaker and passing it a TCP connection would take major surgery because it's written using inheritance rather than composition. Even url.parse, which is essentially powerless, took 12 lines to wrap: https://github.com/dckc/do-your-worst/blob/master/server.js#L37-L49

But I can imagine automating such boilerplate based on typescript typings.

All that said: if you give us main(require) I will happily spend all my (available) energy nurturing it and helping it grow, including norms that are subject to static analysis to give much of what a manifest would give. (Some would call these norms "smart contracts" :)

If you give the community a manifest format that encourages them to continue to use ambient authority... meh.

[kriskowal]

Related changes coming #398