Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for brotli ('br') content-encoding #172

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
5bdbb9e
Added support for brotli ('br') content-encoding
danielgindi Jul 10, 2020
6ef8cef
Update README.md
danielgindi Jul 12, 2020
02c06c2
Update README.md
danielgindi Jul 13, 2020
4df713b
Update README.md
danielgindi Jul 13, 2020
87076af
Apply default value also when params is specified
danielgindi Jul 13, 2020
fffe4c7
Increase coverage for specifying params
danielgindi Jul 14, 2020
9145a55
Updated brotli detection method
danielgindi Aug 25, 2020
0bb402e
Prefer br over gzip and deflate
danielgindi Aug 30, 2020
bbcd9c4
feat: use "koa-compress" logic to determine the preferred encoding
nicksrandall Sep 14, 2020
767c62a
test: adding one more test case br/gzip with quality params
nicksrandall Sep 14, 2020
78ad84a
chore: fix linting errors
nicksrandall Sep 14, 2020
4c359b8
fix: hand write encodings lib to be compatible with node 0.8
nicksrandall Sep 14, 2020
8340cde
Fix: fixing lint errors in new lib
nicksrandall Sep 14, 2020
04ab713
Fix: fixing lint errors in new lib
nicksrandall Sep 14, 2020
b024cce
implemented required encoding negotiator without 3rd party dependency
danielgindi Dec 19, 2020
9af45dd
Merge branch 'master' into feature/brotli
bjohansebas Oct 19, 2024
25b68b8
fix
bjohansebas Oct 19, 2024
3d30ab0
use negotiator
bjohansebas Oct 19, 2024
2cabcc3
Merge branch 'master' into feature/brotli
UlisesGascon Oct 20, 2024
662c09d
improve negotiateEnconding
bjohansebas Oct 24, 2024
0ecf49f
Merge branch 'feature/brotli' of github.com:bjohansebas/compression i…
bjohansebas Oct 24, 2024
395ead9
Merge branch 'master' of github.com:expressjs/compression into featur…
bjohansebas Oct 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ The following compression codings are supported:

- deflate
- gzip
- br (brotli)

**Note** Brotli is supported only since Node.js versions v11.7.0 and v10.16.0.

## Install

Expand Down Expand Up @@ -42,7 +45,8 @@ as compressing will transform the body.

`compression()` accepts these properties in the options object. In addition to
those listed below, [zlib](http://nodejs.org/api/zlib.html) options may be
passed in to the options object.
passed in to the options object or
[brotli](https://nodejs.org/api/zlib.html#zlib_class_brotlioptions) options.

##### chunkSize

Expand Down Expand Up @@ -99,6 +103,20 @@ The default value is `zlib.Z_DEFAULT_MEMLEVEL`, or `8`.
See [Node.js documentation](http://nodejs.org/api/zlib.html#zlib_memory_usage_tuning)
regarding the usage.

##### params *(brotli only)* - [key-value object containing indexed Brotli parameters](https://nodejs.org/api/zlib.html#zlib_brotli_constants)

- `zlib.constants.BROTLI_PARAM_MODE`
- `zlib.constants.BROTLI_MODE_GENERIC` (default)
- `zlib.constants.BROTLI_MODE_TEXT`, adjusted for UTF-8 text
- `zlib.constants.BROTLI_MODE_FONT`, adjusted for WOFF 2.0 fonts
- `zlib.constants.BROTLI_PARAM_QUALITY`
- Ranges from `zlib.constants.BROTLI_MIN_QUALITY` to
`zlib.constants.BROTLI_MAX_QUALITY`, with a default of
`4` (which is not node's default but the most optimal).

Note that here the default is set to compression level 4. This is a balanced setting with a very good speed and a very good
compression ratio.

##### strategy

This is used to tune the compression algorithm. This value only affects the
Expand Down
17 changes: 17 additions & 0 deletions encoding_negotiator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
var zlib = require('zlib')
var Negotiator = require('negotiator')

/**
* @const
* whether current node version has brotli support
*/
var hasBrotliSupport = 'createBrotliCompress' in zlib

function negotiateEncoding (req, encodings) {
var negotiator = new Negotiator(req)

return negotiator.encodings(encodings, hasBrotliSupport ? ['br'] : ['gzip'])[0]
}

module.exports.hasBrotliSupport = hasBrotliSupport
module.exports.negotiateEncoding = negotiateEncoding
19 changes: 18 additions & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@ var Buffer = require('safe-buffer').Buffer
var bytes = require('bytes')
var compressible = require('compressible')
var debug = require('debug')('compression')
var objectAssign = require('object-assign')
var onHeaders = require('on-headers')
var vary = require('vary')
var zlib = require('zlib')
var hasBrotliSupport = require('./encoding_negotiator').hasBrotliSupport

/**
* Module exports.
Expand All @@ -48,6 +50,19 @@ var cacheControlNoTransformRegExp = /(?:^|,)\s*?no-transform\s*?(?:,|$)/
function compression (options) {
var opts = options || {}

if (hasBrotliSupport) {
// set the default level to a reasonable value with balanced speed/ratio
if (opts.params === undefined) {
opts = objectAssign({}, opts)
opts.params = {}
}

if (opts.params[zlib.constants.BROTLI_PARAM_QUALITY] === undefined) {
opts.params = objectAssign({}, opts.params)
opts.params[zlib.constants.BROTLI_PARAM_QUALITY] = 4
}
}

// options
var filter = opts.filter || shouldCompress
var threshold = bytes.parse(opts.threshold)
Expand Down Expand Up @@ -187,7 +202,9 @@ function compression (options) {
debug('%s compression', method)
stream = method === 'gzip'
? zlib.createGzip(opts)
: zlib.createDeflate(opts)
: method === 'br'
? zlib.createBrotliCompress(opts)
: zlib.createDeflate(opts)

// add buffered listeners to stream
addListeners(stream, stream.on, listeners)
Expand Down
152 changes: 152 additions & 0 deletions test/compression.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ var zlib = require('zlib')

var compression = require('..')

/**
* @const
* whether current node version has brotli support
*/
var hasBrotliSupport = 'createBrotliCompress' in zlib
var brotlit = hasBrotliSupport ? it : it.skip

describe('compression()', function () {
it('should skip HEAD', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
Expand Down Expand Up @@ -465,6 +472,37 @@ describe('compression()', function () {
})
})

describe('when "Accept-Encoding: br"', function () {
brotlit('should respond with br', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'br')
.expect('Content-Encoding', 'br', done)
})
})

describe('when "Accept-Encoding: br" and passing compression level', function () {
brotlit('should respond with br', function (done) {
var params = {}
params[zlib.constants.BROTLI_PARAM_QUALITY] = 11

var server = createServer({ threshold: 0, params: params }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'br')
.expect('Content-Encoding', 'br', done)
})
})

describe('when "Accept-Encoding: gzip, deflate"', function () {
it('should respond with gzip', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
Expand Down Expand Up @@ -493,6 +531,91 @@ describe('compression()', function () {
})
})

describe('when "Accept-Encoding: gzip, br"', function () {
var brotlit = hasBrotliSupport ? it : it.skip
brotlit('should respond with br', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip, br')
.expect('Content-Encoding', 'br', done)
})
})

describe('when "Accept-Encoding: deflate, gzip, br"', function () {
brotlit('should respond with br', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'deflate, gzip, br')
.expect('Content-Encoding', 'br', done)
})
})

describe('when "Accept-Encoding: gzip;q=1, br;q=0.3"', function () {
brotlit('should respond with gzip', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip;q=1, br;q=0.3')
.expect('Content-Encoding', 'gzip', done)
})
})

describe('when "Accept-Encoding: gzip, br;q=0.8"', function () {
brotlit('should respond with gzip', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip, br;q=0.8')
.expect('Content-Encoding', 'gzip', done)
})
})

describe('when "Accept-Encoding: gzip;q=0.001"', function () {
brotlit('should respond with gzip', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip;q=0.001')
.expect('Content-Encoding', 'gzip', done)
})
})

describe('when "Accept-Encoding: deflate, br"', function () {
brotlit('should respond with br', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'deflate, br')
.expect('Content-Encoding', 'br', done)
})
})

describe('when "Cache-Control: no-transform" response header', function () {
it('should not compress response', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
Expand Down Expand Up @@ -631,6 +754,32 @@ describe('compression()', function () {
.end()
})

brotlit('should flush small chunks for brotli', function (done) {
var chunks = 0
var next
var server = createServer({ threshold: 0 }, function (req, res) {
next = writeAndFlush(res, 2, Buffer.from('..'))
res.setHeader('Content-Type', 'text/plain')
next()
})

function onchunk (chunk) {
assert.ok(chunks++ < 20)
assert.strictEqual(chunk.toString(), '..')
next()
}

request(server)
.get('/')
.set('Accept-Encoding', 'br')
.request()
.on('response', unchunk('br', onchunk, function (err) {
if (err) return done(err)
server.close(done)
}))
.end()
})

it('should flush small chunks for deflate', function (done) {
var chunks = 0
var next
Expand Down Expand Up @@ -710,6 +859,9 @@ function unchunk (encoding, onchunk, onend) {
case 'gzip':
stream = res.pipe(zlib.createGunzip())
break
case 'br':
stream = res.pipe(zlib.createBrotliDecompress())
break
}

stream.on('data', onchunk)
Expand Down
Loading