Plugin API

[benjaminjkraft]

It seems like one day, maybe soon, we'll find value in a plugin API, so users can hook into codegen in ways that we can't anticipate. This issue is to collect both examples of plugins we might want, and ideas for how to do them.

A few things to think about:

• What hooks do plugins get? a few ideas include rewriting the query in between parsing and codegen, and modifying the types between generating our representations of the Go types and templating them into code.
• How do we specify plugins? Go has a plugin API but actually using it in a nice reproducible way is kind of messy (as we learned at Khan with golangci-lint plugins)
• How do we keep the API as stable as possible? No matter what, this sort of thing is tricky to get right and we'll probably want to set expectations that it won't be super stable early on, but the obvious ways to do things may involve exposing a lot of currently-internal types.

Issues that might be best done via a plugin (please suggest any other use cases you come across):

• Plugin for _entities queries in Apollo federation #28
• Plugin for relay-based APIs #78
• Allow __all to request all fields #27 (doesn't really have to be, but might be better that way)
• Generate custom struct tags #234 (possibly two separate plugins here)
• Handling enums with strings that have variations of character casing #265 (and other casing stuff)
• [Meta] Document and improve support for Hasura & Dgraph schemas #272
• Support for @authenticated directive  #332 (if we wanted to add an "add the federation directives to your schema" plugin, not sure if this is useful in practice as such, but something like it might be)

[benjaminjkraft]

Doing some thinking about the plugin API. (This is pretty rough, thoughts are very welcome though.)

Next thing to do is probably to make a POC implementation of this, and try writing some plugins. As pre-work it may be useful to refactor directive-parsing to happen along with query-parsing, and we can probably get #151 as a part of that.

At some point we'll also need to figure out the config, but that seems comparatively easy (and is safer to do in a crappy way (copy from golangci-lint) and then improve later if needed).

query rewriting

A thing we can definitely expose is query rewriting. Some plugins would want that between parsing and validation, others might want it post-validation. Something like:

type PluginInput struct {
  Config *generate.Config
  Schema *ast.Schema
  QueryDocuments *[]*ast.QueryDocument // modify in place(?)
}

type Plugin interface {
  RewriteBeforeValidation(*PluginInput) error
  RewriteAfterValidation(*PluginInput) error
}

That lets you do __all in the obvious way. It's part of what you need for _entities queries, something like:

# @entityquery(by: "id")
fragment MyQueryUser on User { ... }

# compiles to add
query MyQuery(
  $representations: [_Any!]!,
) {
  _entities(representations: $representations) { ...MyQueryUser }
}

but then you also want to do something about that _Any.

genqlient directives

An obvious thing there is you can add a genqlient directive. The exact API for this is not totally obvious, since those aren't really in the AST, but we can come up with something. Then you can add # @genqlient(bind: "MyQueryUserInput") or something to $representations, which gets you partway more, although you still need to generate the Go type MyQueryUserInput. It also lets us do some of the custom-casing requests, at least if we had something like typename for enum values (which I forget if we added or not).

API here could look something like

type GenqlientDirective struct { /* existing, newly exported */ }

type Plugin interface {
  // or: add the return to the two rewrite methods above
  // or: actually this goes in the input so you can preprocess directives too
  AddDirectives(*PluginInput) ([]*GenqlientDirective, error)
}

It's less clear to me exactly what we want for hasura/dgraph, but "add a bunch of genqlient directives automatically" may be mostly sufficient. (Half of the problem there is just that the defaults don't work well for them, which can be solved via configuration, but perhaps more flexibly by a plugin.)

An implementation trouble is that directives are parsed lazily, but that's surely fixable. (And it might improve the code to do so, the directive-checking code gets pretty messy.)

go types

For adding net-new Go types like MyQueryUserInput, maybe the best way to do that is some kind of thing where you provide a fragment, and we generate the type even though it's unused, so you can bind it as desired? Although here it's an input-style type, so it's not quite right. (Maybe we let you edit schemas too, so you can do input MyQueryUserInput { id: ID! }. That solves for things like #332 as well, and in some ways defining your own type, whether input or output, is right -- in the federation example even if you had fragments on input types, User isn't an input type. Although you have to worry about unused-variable validation, at least if you do this naively. Maybe you generate a dummy query, or there's a special directive to say "generate a type for this schema type that doesn't look used".) And of course for net-new Go types that totally don't map to GraphQL, you can give us a string.

Then some of the requests want changes to genqlient-generated structs. We could expose the stuff in types.go for you to manipulate, but that seems very scary from a compat perspective. Probably we just skip that for now, which rules out the gogm thing probably.

In sum that looks something like:

// plugins may modify any fields in place
type PluginContext struct {
  Config *generate.Config
  Schema *ast.Schema
  QueryDocuments *[]*ast.QueryDocument
  Directives *[]*GenqlientDirective
  ExtraGoTypes map[string]string // (nil, fill in if you need it)
}

type Plugin interface {
  RewriteBeforeValidation(*PluginContext) error
  RewriteAfterValidation(*PluginContext) error
}

and I think at least in principle that can do all the potential examples except gogm (although maybe if we had an "add XYZ extra struct tags" directive it would) and maybe relay (which isn't in my head enough to know about)

[benjaminjkraft]

Another question for a plugin API is: how much can we rewrite from the core into plugins? (If we did this they'd of course be builtin plugins, so you don't have to link them in yourself or anything, it's more a test of the abstraction than to get functionality out of repo.)

Scanning through the "non-core" settings in genqlient.yaml:

• ✅ export_operations maybe/probably? there's some question of where preprocessQueryDocument happens (this plugin needs to be after it, but normally it should probably happen after plugin rewrites? to keep plugins from having to know where to add __typename. 3 hooks maybe?)
• ❌ context_type requires deeper changes to Go types, hard to see how that would work even though ideally it'd be a plugin (not sure if anyone except @Khan uses it)
• ❌ client_getter likewise
• ✅ use_struct_references yeah, it's basically just adding some directives to every object/input
• ❌ use_extensions not obviously, I'm ok with that
• ✅ optional is again kinda just adding some directives to every optional field
• ✅ optional_generic_type likewise but only if we do Support optional: generic inline #320
• ✅ package_bindings we could probably support via Config manipulation, although it's a bit annoying if it has to go re-resolve the packages
• ✅ casing is just a bunch of typename directives (again assuming we have an enum-member equivalent)
  Seems like there's a gap of customizing the signature of genqlient's methods -- and see also the various requests about accepting/returning headers -- but I think that will require more care to get right so maybe it's best done as a plugins v2.