Ability to selectively allow quoting / no quoting in UTF8JsonGenerator#writeRawUTF8String

[StevenCurran]

Hi all, have a use case where I am pulling json payloads from an external store, and passing them into another built json payload that is created from a json generator. The payload that I want to write, is a byte[] of an existing json string.

Using this method in the UTF8JsonGenerator it ends up surrounding the string with additional quotes.

   @Override
    public void writeRawUTF8String(byte[] text, int offset, int len) throws IOException
    {
        _checkRangeBoundsForByteArray(text, offset, len);
        _verifyValueWrite(WRITE_STRING);
        if (_outputTail >= _outputEnd) {
            _flushBuffer();
        }
        _outputBuffer[_outputTail++] = _quoteChar;
        _writeBytes(text, offset, len);
        if (_outputTail >= _outputEnd) {
            _flushBuffer();
        }
        _outputBuffer[_outputTail++] = _quoteChar;
    }

What I want to know, is there a way for me to selectively disable this quoting (i need it in other places though, so i want to reenable it for other parts of the json payload I am creating with the generator).

I have monkey patched this class locally like this, but obviously its not ideal and would rather be able to do it from the library, so if there are any other approaches to solving this I would love to know!

This is in a pretty memory constrained setting, so trying to avoid all unnecessary string / byte[] allocs as much as possible.

   public void writeRawUTF8StringNoQuotes(byte[] text, int offset, int len) throws IOException
    {
        _checkRangeBoundsForByteArray(text, offset, len);
        _verifyValueWrite(WRITE_STRING);
        if (_outputTail >= _outputEnd) {
            _flushBuffer();
        }
       // _outputBuffer[_outputTail++] = _quoteChar;
        _writeBytes(text, offset, len);
        if (_outputTail >= _outputEnd) {
            _flushBuffer();
        }
	//removing these lines that are quoting	
       // _outputBuffer[_outputTail++] = _quoteChar;
    }

Alternative is a way to access the com.fasterxml.jackson.core.json.UTF8JsonGenerator#_writeBytes(byte[], int, int) method, as this is essentially what I want to call:

    private final void _writeBytes(byte[] bytes, int offset, int len) throws IOException
    {
        if ((_outputTail + len) > _outputEnd) {
            _flushBuffer();
            // still not enough?
            if (len > MAX_BYTES_TO_BUFFER) {
                _outputStream.write(bytes, offset, len);
                return;
            }
        }
        System.arraycopy(bytes, offset, _outputBuffer, _outputTail, len);
        _outputTail += len;
    }

So I want to achieve what this patched version is doing.

Many thanks!

[cowtowncoder]

No, unfortunately there is no way to do that.

There are a few writeRaw() and writeRawValue() methods that would in theory work better (in that they specifically do not add any decoration to value) -- but they unfortunately only take in char-based content.

One final possibility to see might be:

    public void writeRawValue(SerializableString raw) throws IOException

where custom implementation of SerializableString might work with byte-backed output; maybe worth experimenting.

[StevenCurran]

Hi there, thank you for the response. yes I had noticed that class.
But I am a bit confused by its usage. The first thing it seems to do in this methods, is an expensive allocation if an instance field is null, which it seems to always be? It also does not look like I can reuse an instance of SerializableString. Am I misunderstanding something?

    @Override
    public int appendUnquotedUTF8(byte[] buffer, int offset) {
        byte[] result = _unquotedUTF8Ref;
		//_unquotedUTF8Ref is always null, so the encoder will call below will always run. 
        if (result == null) {
            _unquotedUTF8Ref = result = JSON_ENCODER.encodeAsUTF8(_value);
        }
        final int length = result.length;
        if ((offset + length) > buffer.length) {
            return -1;
        }
        System.arraycopy(result, 0, buffer, offset, length);
        return length;
    }

[StevenCurran]

So sorry, misunderstood, yes I can just make a custom impl with that interface and pass that back into the generator. Thank you for the tips, will try this soon! Should solve what I need to do exactly. Many thanks for pointing that out.

[cowtowncoder]

Yeah, SerializableString is bit of a monster: it's original use is to reuse pre-encoded JSON names.
But it can be used to pipe in other kinds of pre-/dynamically-encoded content.
(SerializedString is more static implementation that wouldn't work that well for you)

[StevenCurran]

Yeah, SerializableString is bit of a monster: it's original use is to reuse pre-encoded JSON names.

But it can be used to pipe in other kinds of pre-/dynamically-encoded content.

(SerializedString is more static implementation that wouldn't work that well for you)

While this does work for me, I am thinking if there was an api provided that could handle a Bytebuffer passed in, instead of the byte array I could pass my buffer in directly and avoid copying out to a byte[] which is taking a lot of cpu it seems. Is there any interface I could use that handles the bytebuffers directly? I assume it would act in a very similar way to writrBytes.

Many thanks

[cowtowncoder]

ByteBuffer is not really used natively by Jackson at all (it's very clunky interface, unfortunately); there are adapters to allow reading/writing content, but it is not (nor likely will be) used for API otherwise.
(this matters as all content is buffered into underlying byte[] (or char[]) and rarely directly written to OutputStream / Writer)

So even if ByteBuffer was accepted, Jackson would need to copy contents.