Skip to content

Commit

Permalink
Add option to handle scream snake case enums. (temporalio#1529)
Browse files Browse the repository at this point in the history
Add option to handle scream snake case enums.
  • Loading branch information
Quinn-With-Two-Ns authored Jun 27, 2024
1 parent 5c6a928 commit 5487524
Show file tree
Hide file tree
Showing 5 changed files with 2,567 additions and 11 deletions.
54 changes: 53 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,17 @@ See [contrib/tools/workflowcheck](contrib/tools/workflowcheck) for a tool to det
## Contributing
We'd love your help in making the Temporal Go SDK great. Please review our [contribution guidelines](CONTRIBUTING.md).

## Go build and run tags
## Go SDK upgrading past v1.25.1

Go SDK version v1.26.0 switched from using https://github.com/gogo/protobuf to https://github.com/golang/protobuf. While this migration is mostly internal there are a few user visible changes to be aware of:

### Change in types

* `time.Time` in proto structs will now be [timestamppb.Timestamp](https://pkg.go.dev/google.golang.org/[email protected]/types/known/timestamppb#section-documentation)
* `time.Duration` will now be [durationpb.Duration](https://pkg.go.dev/google.golang.org/protobuf/types/known/durationpb)
* V2-generated structs embed locks, so you cannot dereference them.

### Invalid UTF-8

Prior to SDK version v1.26.0 our protobuf code generator allowed invalid UTF-8 data to be stored as proto strings. This isn't actually allowed by the proto3 spec, so if you're using our SDK and think you may store arbitrary binary data in our strings you should set `-tags protolegacy` when building against our SDK.

Expand All @@ -67,5 +77,47 @@ If you see an error like `grpc: error unmarshalling request: string field contai

If you're unsure then you should specify it anyways as there's no harm in doing so unless you relied on the protobuf compiler to ensure all strings were valid UTF-8.

### Incompatible proto/json encoding

Proto enums will, when formatted to JSON, now be in SCREAMING_SNAKE_CASE rather than PascalCase.
* If trying to deserialize old JSON with PascalCase to proto use [go.temporal.io/api/temporalproto]

If users used Temporal proto types in their Workflows, such as for activity output, users may need to modify the default data converter to handle these payloads.
``` go
converter.NewProtoJSONPayloadConverterWithOptions(converter.ProtoJSONPayloadConverterOptions{
LegacyTemporalProtoCompat: true,
}),
```

While upgrading from Go SDK version `< 1.26.0` to a version `>= 1.26.0` users may want to also bias towards using
proto binary to avoid any potential incompatibilities due to having clients serialize messages with incompatible `proto/json` format.

On clients running Go SDK `< 1.26.0`
``` go
converter.NewCompositeDataConverter(
converter.NewNilPayloadConverter(),
converter.NewByteSlicePayloadConverter(),
converter.NewProtoPayloadConverter(),
converter.NewProtoJSONPayloadConverterWithOptions(),
converter.NewJSONPayloadConverter(),
)
```

On clients running Go SDK `>= 1.26.0`

``` go
converter.NewCompositeDataConverter(
converter.NewNilPayloadConverter(),
converter.NewByteSlicePayloadConverter(),
converter.NewProtoPayloadConverter(),
converter.NewProtoJSONPayloadConverterWithOptions(converter.ProtoJSONPayloadConverterOptions{
LegacyTemporalProtoCompat: true,
}),
converter.NewJSONPayloadConverter(),
)
```

Note: Payloads encoded with `proto/binary` will not be readable in the Temporal web UI.

## License
MIT License, please see [LICENSE](LICENSE) for details.
34 changes: 24 additions & 10 deletions converter/proto_json_payload_converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,19 @@ import (
gogojsonpb "github.com/gogo/protobuf/jsonpb"
gogoproto "github.com/gogo/protobuf/proto"
commonpb "go.temporal.io/api/common/v1"
"go.temporal.io/api/temporalproto"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
)

// ProtoJSONPayloadConverter converts proto objects to/from JSON.
type ProtoJSONPayloadConverter struct {
gogoMarshaler gogojsonpb.Marshaler
gogoUnmarshaler gogojsonpb.Unmarshaler
protoMarshalOptions protojson.MarshalOptions
protoUnmarshalOptions protojson.UnmarshalOptions
options ProtoJSONPayloadConverterOptions
gogoMarshaler gogojsonpb.Marshaler
gogoUnmarshaler gogojsonpb.Unmarshaler
protoMarshalOptions protojson.MarshalOptions
protoUnmarshalOptions protojson.UnmarshalOptions
temporalProtoUnmarshalOptions temporalproto.CustomJSONUnmarshalOptions
options ProtoJSONPayloadConverterOptions
}

// ProtoJSONPayloadConverterOptions represents options for `NewProtoJSONPayloadConverterWithOptions`.
Expand All @@ -64,6 +66,10 @@ type ProtoJSONPayloadConverterOptions struct {

// EmitUnpopulated specifies whether to emit unpopulated fields.
EmitUnpopulated bool

// LegacyTemporalProtoCompat will allow enums serialized as SCREAMING_SNAKE_CASE.
// Useful for backwards compatibility when migrating a proto message from gogoproto to standard protobuf.
LegacyTemporalProtoCompat bool
}

var (
Expand All @@ -73,10 +79,11 @@ var (
// NewProtoJSONPayloadConverter creates new instance of `ProtoJSONPayloadConverter`.
func NewProtoJSONPayloadConverter() *ProtoJSONPayloadConverter {
return &ProtoJSONPayloadConverter{
gogoMarshaler: gogojsonpb.Marshaler{},
gogoUnmarshaler: gogojsonpb.Unmarshaler{},
protoMarshalOptions: protojson.MarshalOptions{},
protoUnmarshalOptions: protojson.UnmarshalOptions{},
gogoMarshaler: gogojsonpb.Marshaler{},
gogoUnmarshaler: gogojsonpb.Unmarshaler{},
protoMarshalOptions: protojson.MarshalOptions{},
protoUnmarshalOptions: protojson.UnmarshalOptions{},
temporalProtoUnmarshalOptions: temporalproto.CustomJSONUnmarshalOptions{},
}
}

Expand All @@ -99,6 +106,9 @@ func NewProtoJSONPayloadConverterWithOptions(options ProtoJSONPayloadConverterOp
protoUnmarshalOptions: protojson.UnmarshalOptions{
DiscardUnknown: options.AllowUnknownFields,
},
temporalProtoUnmarshalOptions: temporalproto.CustomJSONUnmarshalOptions{
DiscardUnknown: options.AllowUnknownFields,
},
options: options,
}
}
Expand Down Expand Up @@ -193,7 +203,11 @@ func (c *ProtoJSONPayloadConverter) FromPayload(payload *commonpb.Payload, value

var err error
if isProtoMessage {
err = c.protoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
if c.options.LegacyTemporalProtoCompat {
err = c.temporalProtoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
} else {
err = c.protoUnmarshalOptions.Unmarshal(payload.GetData(), protoMessage)
}
} else if isGogoProtoMessage {
err = c.gogoUnmarshaler.Unmarshal(bytes.NewReader(payload.GetData()), gogoProtoMessage)
}
Expand Down
Loading

0 comments on commit 5487524

Please sign in to comment.