Modify encoding extension to return a list of Key-Value pairs

[varkey98]

Component(s)

pdata

Is your feature request related to a problem? Please describe.

Currently, the encoding extensions can only return a single byte blob, which is directly populated as value on kafka exporter like this

func (p pdataLogsMarshaler) MarshalLogs(ld plog.Logs) ([]Message, error) {
	bts, err := p.marshaler.MarshalLogs(ld)
	if err != nil {
		return nil, err
	}
	return []Message{{Value: bts}}, nil
}

But I think we should be modifying this interface: https://github.com/open-telemetry/opentelemetry-collector/blob/main/pdata/plog/encoding.go#L13 so that we can populate a slice of Message object with list key-value pairs instead of just one.

Not sure if I should raise it in the contrib repo or here

Describe the solution you'd like

I understand that the current interface is tied to the other components and has a wider use case. But I think we should define a new interface or modify the existing one so that we can do something like

func (p pdataLogsMarshaler) MarshalLogs(ld plog.Logs) ([]Message, error) {
	objs, err := p.marshaler.MarshalLogs(ld)
        ret := make([]Message, 0, len(objs))
        for _, obj := rangs objs {
          ret = append(ret, Message{Key: obj.Key, Value: obj.Value}
        }
	if err != nil {
		return nil, err
	}
	return []Message{{Value: bts}}, nil
}

Describe alternatives you've considered

I am currently using a custom kafka exporter so that I manage the marshaling manually. But as I read that the encoding extensions are the preferred way to enhance kafka exporter, I think we should be supporting this

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• pdata: @bogdandrutu @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.