Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.0.0-rc.1 #1

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
d500075
v1.0.0-rc.1
May 30, 2015
7416d5a
resolving node <=0.8 issue ("debug" target not found)
May 30, 2015
0b8bbc3
add node 0.12 to travis matrix
May 30, 2015
e9badf7
use node 0.6 safe versions of mocha + istanbul
May 30, 2015
5f36bf9
remove dependency on util._extend (node 0.6 support)
May 30, 2015
3e90938
fix gh markdown rendering bug
May 30, 2015
0962a47
restore copyright notice
May 30, 2015
560dfa6
throw error on invalid schema name
May 30, 2015
34ad6cc
lowecase schema names + test
May 30, 2015
4e7e8f4
debug should be under dependencies, rather than devDependencies
May 30, 2015
2d937df
On a disconnected socket, req.connection.remotePort is undefined
May 30, 2015
1520eae
fix: address ordering in rfc2739 processor, fix: ensure processed 'fo…
May 30, 2015
8694d1c
default to rfc7239 schema
May 30, 2015
5e77858
fix: cloudflare https detection
May 30, 2015
cbab687
fix: fastly ssl header existance counts as HTTPS
May 30, 2015
1761f9d
Merge remote-tracking branch 'upstream/master'
May 30, 2015
7a3cb41
add author to LICENSE
May 31, 2015
f39a24f
update author info
May 31, 2015
3363328
added test with real http server and request
May 31, 2015
553fb6f
node 0.6 lacks a callback function on server.close
May 31, 2015
e8f6794
chore(dependencies): update dependencies + replace debug for lighter …
Mar 26, 2016
6ce460a
chore(rename): better use of header prefixes to reflect schema naming
Mar 26, 2016
3459974
refactor(proto): simplified logic for protocol detection
Mar 26, 2016
e2c1292
refactor(rfc7239): use the excellent forwarded-parse by @lpinca
Mar 26, 2016
9e54f7a
feat(defaults): default schemas should be rfc7239 and x-forwarded
Mar 26, 2016
7aa4ffe
feat(schemas): add weblogic proxy support
Mar 26, 2016
8cf1117
docs(readme): update docs with details on each schema
Mar 26, 2016
0dc53a6
chore(package): update mocha to version 2.5.1
greenkeeperio-bot May 23, 2016
482dcec
Merge pull request #3 from ahmadnassri/greenkeeper-mocha-2.5.1
May 23, 2016
104f0a5
chore(package): update mocha to version 2.5.3
greenkeeperio-bot May 25, 2016
18e0181
Merge pull request #5 from ahmadnassri/greenkeeper-mocha-2.5.3
Jun 9, 2016
9712f4d
chore(package): update istanbul to version 0.4.3
greenkeeperio-bot Jun 9, 2016
462d7e5
Merge pull request #1 from ahmadnassri/greenkeeper-istanbul-0.4.3
Jun 9, 2016
4b97e71
chore(package): update mocha to version 3.0.1
greenkeeperio-bot Aug 4, 2016
cb066f8
Merge pull request #8 from ahmadnassri/greenkeeper-mocha-3.0.1
Aug 10, 2016
8980fbf
chore: drop support for Node.js 0.10
greenkeeperio-bot Oct 31, 2016
cad75c3
Merge pull request #15 from ahmadnassri/greenkeeper/remove-node-0.10
Nov 1, 2016
bbe1a0d
chore(package): update dependencies
greenkeeper[bot] Nov 7, 2016
43b9fae
Merge pull request #16 from ahmadnassri/greenkeeper/update-all
Nov 7, 2016
7f03ddf
chore(package): update mocha to version 3.2.0
greenkeeper[bot] Nov 25, 2016
f7e83c8
Merge pull request #17 from ahmadnassri/greenkeeper/mocha-3.2.0
Dec 4, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
language: node_js
node_js:
- "0.6"
- "0.8"
- "0.10"
- "0.12"
- "1.0"
- "1.8"
- "2.0"
- "2.1"
- '1.0'
- '1.8'
- '2.0'
- '2.1'
sudo: false
before_install:
# Setup Node.js version-specific dependencies
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(The MIT License)

Copyright (c) 2014 Douglas Christopher Wilson
Copyright (c) 2014 Douglas Christopher Wilson, Ahmad Nassri

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
Expand Down
65 changes: 58 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
[![Build Status][travis-image]][travis-url]
[![Test Coverage][coveralls-image]][coveralls-url]

Parse HTTP X-Forwarded-For header
Parse *Forwarded* HTTP headers, using the standard: [RFC 7239](https://tools.ietf.org/html/rfc7239) *(Forwarded HTTP Extension)*, as well as commonly used none-standard headers (e.g. `X-Forwarded-*`, `X-Real-*`, etc ...)

review [`schemas` folder](lib/schemas) for a full list of supported headers schemas.

## Installation

Expand All @@ -20,23 +22,62 @@ $ npm install forwarded
var forwarded = require('forwarded')
```

### forwarded(req)
### forwarded(req[, options])

returns an object who's properties represent [RFC 7239 Parameters (Section 5)](http://tools.ietf.org/html/rfc7239#section-5)

```js
var addresses = forwarded(req)
var result = forwarded(req)
```

Parse the `X-Forwarded-For` header from the request. Returns an array
of the addresses, including the socket address for the `req`. In reverse
order (i.e. index `0` is the socket address and the last index is the
furthest address, typically the end-user).
#### options

| name | type | description | required | default |
| --------- | ------- | ----------------------------------------- | -------- | ---------------------------- |
| `schemas` | `array` | ordered list of header schemas to process | no | `['rfc7239', 'x-forwarded']` |

Parse appropriate headers from the request matching the selected [schemas](#options).

###### available schemas

| name | key | description |
| ------------------- | ------------- | ----------------------------------------------------------------------------------------- |
| RFC 7239 | `rfc7239` | [RFC 7239 Standard][rfc7239] |
| X-Forwarded-* | `x-forwarded` | Headers using the prefix [`X-Forwarded-*`][x-forwarded], a de facto standard |
| X-Real-* | `x-real` | Headers using the prefix [`X-Real-*`][x-real], mostly common in NGINX servers |
| Z-Forwarded-* | `z-forwarded` | less common version of `X-Forwarded-*` used by [Z Scaler][z-forwarded] |
| X-Cluster-Client-IP | `x-cluster` | used by [Rackspace][x-cluster], X-Ray servers |
| Cloudflare | `cloudflare` | Headers used by [Cloudflare][cloudflare] |
| Fastly | `fastly` | Headers used by Fastly, for [IP][fastly-ip], Port, and [SSL][fastly-ssl] info |
| Microsoft | `microsoft` | Non-standard header field used by [Microsoft][microsoft] applications and load-balancers |
| Weblogic | `weblogic` | Forwarded IP by Oracle's [Weblogic][weblogic] Proxy |

### returned object

| name | type | description | default |
| --------- | --------- | ---------------------------------------------------------------------------------------- | -------------------------------------- |
| `for` | `array` | alias of `addrs` |
| `by` | `string` | [RFC 7239 Section 5.1](http://tools.ietf.org/html/rfc7239#section-5.1) compatible result | `null` |
| `addrs` | `array` | [RFC 7239 Section 5.2](http://tools.ietf.org/html/rfc7239#section-5.2) compatible result | `[request.connection.remoteAddress]` |
| `host` | `string` | [RFC 7239 Section 5.3](http://tools.ietf.org/html/rfc7239#section-5.3) compatible result | `request.headers.host` |
| `proto` | `string` | [RFC 7239 Section 5.4](http://tools.ietf.org/html/rfc7239#section-5.4) compatible result | `request.connection.encrypted` |
| `port` | `string` | the last known port used by the client/proxy in chain of proxies | `request.connection.remotePort` |
| `ports` | `array` | ordered list of known ports in the chain of proxies | `[request.connection.remotePort]` |

###### Notes

- `forwarded().addrs` & `forwarded().ports`: return arrays of the addresses & ports respectively, including the socket address/port for the request. In reverse order (i.e. index `0` is the socket address/port and the last index is the furthest address/port, typically the end-user).

## Testing

```sh
$ npm test
```

## TODO

- [ ] extract ports from [`Forwarded`](http://tools.ietf.org/html/rfc7239#section-5.2) header: `Forwarded: for=x.x.x.x:yyyy`

## License

[MIT](LICENSE)
Expand All @@ -51,3 +92,13 @@ $ npm test
[coveralls-url]: https://coveralls.io/r/jshttp/forwarded?branch=master
[downloads-image]: https://img.shields.io/npm/dm/forwarded.svg
[downloads-url]: https://npmjs.org/package/forwarded
[rfc7239]: https://tools.ietf.org/html/rfc7239
[x-forwarded]: https://en.wikipedia.org/wiki/X-Forwarded-For
[z-forwarded]: https://en.wikipedia.org/wiki/X-Forwarded-For#Proxy_servers_and_caching_engines
[x-real]: http://nginx.org/en/docs/http/ngx_http_realip_module.html
[x-cluster]: https://support.rackspace.com/how-to/controlling-access-to-linux-cloud-sites-based-on-the-client-ip-address/
[cloudflare]: https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-CloudFlare-handle-HTTP-Request-headers-
[fastly-ssl]: https://docs.fastly.com/guides/securing-communications/tls-termination
[fastly-ip]: https://docs.fastly.com/guides/basic-configuration/adding-or-modifying-headers-on-http-requests-and-responses
[weblogic]: https://blogs.oracle.com/wlscoherence/entry/obtaining_the_correct_client_ip
[microsoft]: http://technet.microsoft.com/en-us/library/aa997519(v=exchg.65).aspx
83 changes: 69 additions & 14 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
/*!
* forwarded
* Copyright(c) 2014 Douglas Christopher Wilson
* Copyright(c) 2015 Ahmad Nassri
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to keep these headers so we know where in the source the copyrights lives (and so the note is retained when people copy the file but not the license).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, forgot to re-add those! (mind tends to zone those out!) :)

* MIT Licensed
*/
*/

'use strict'

Expand All @@ -11,29 +12,83 @@
* @public
*/

module.exports = forwarded
var processor = require('./lib/processor')
var schemas = require('./lib/schemas')

/**
* Get all addresses in the request, using the `X-Forwarded-For` header.
*
* @param {object} req
* @return {array}
* @param {http.IncomingMessage} req
* @return {object}
* @public
*/

function forwarded(req) {
module.exports = function forwarded (req, options) {
options = options || {}

var opts = {
// default to only standard and x-forwarded for backward compatibility
schemas: options.schemas || [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo it should only default to a single schema, not multiple. The reason is people are mainly going to be set up for one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue that typically in libraries such as this, it should be the standard only (i.e. RFC 7239).

however due to the long history of X-Forwarded-* and the relatively new age of RFC 7239 it would make sense to consider both a standard.

in that light, both are the defacto default here, other scenarios would be:

  • RFC 7239 only: would essentially break every dependant's assumptions
  • X-Forwarded-* only: would not be true to the standard and might even be problematic in the future.

this is precisely why the logic at index.js#L52 overwrites properties in the sorted order of this schemas array, existing implementations of X-Forwarded-* will still operate identical to the past, and when Forwarded header is present, it counts as the authoritative source of information as it is the standard.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it counts as the authoritative source of information as it is the standard.

Yes, but this is the biggest issue I have with this assumption: people literally just are not using that header, nor are they blocking it. Because of this, it is extremely easy to fall in a trap where people rely on this module because customers are paying for IP whitelisting. An attacker can then just include a Forwarded header and even those the company's proxies are making sure X-Forwarded-For is correct, they will not have a breach of contract with the customer, as an attacker will include a Forwarded header and suddenly it becomes the authoritative source.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically I think the standard should be the default and people can always change it (and we can simply add messaging to the README). I just don't like the default being multiple types, because I can just see the confusion/issues coming (I have a lot of experience with the things people report ;) ). I like the idea of the module letting you use multiple at once, though, if the user chooses.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to clarify are you suggesting to keep RFC 7239 as the default option?

also, another possible option is to not offer a default, leaving it to the implementer to decide what to support.

e.g.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to clarify are you suggesting to keep RFC 7239 as the default option?

Sure that works. Specifically, I'm suggesting that the default option should be only one schema; which one it is really doesn't matter as much, because people will change it.

another possible option is to not offer a default

Well, that is a possibility, yes, but I'm undecided on that (though if you did it, you'd need to move the schemas out of the options argument, since it's no longer optional).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for what proxy-addr or express may do, that's undecided for next majors, though of course for the existing major, they would stick to 'xff'.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved in 8694d1c

'rfc7239',
'x-forwarded'
]
}

if (!req) {
throw new TypeError('argument req is required')
}

// simple header parsing
var proxyAddrs = (req.headers['x-forwarded-for'] || '')
.split(/ *, */)
.filter(Boolean)
.reverse()
var socketAddr = req.connection.remoteAddress
var addrs = [socketAddr].concat(proxyAddrs)
// start with default values from socket connection
var forwarded = {
addrs: [req.connection.remoteAddress],
by: null,
host: req.headers && req.headers.host ? req.headers.host : undefined,
port: req.connection.remotePort ? req.connection.remotePort.toString() : undefined,
ports: [],
proto: req.connection.encrypted ? 'https' : 'http'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this actually works in Node.js 0.6 and the others? I only ask because the test is just a mock, so we have to verify manually if we cannot add a real test.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was relying on the travis test to be honest, I managed to manually test all the way down to 0.8, then gave up on getting 0.6 installed on my machine. I'll get that working today and confirm.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are only as honest as your tests are--since you are using mocks, then you are not actually testing anything (you have to test a real req for Travis CI to be of any help to you).

}

// add default port to ports array if present
if (forwarded.port) {
forwarded.ports.push(forwarded.port)
}

// alias "for" to keep with RFC7239 naming
forwarded.for = forwarded.addrs

return opts.schemas
// check if schemas exist
.map(function (name) {
// adjust case
name = name.toLowerCase()

if (!schemas[name]) {
throw new Error('invalid schema')
}

return schemas[name]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unknown schemas need to throw an error, not be silently dropped. A user making a typo in a schema name may not realize for a while.

})

// process schemas
.reduce(function (forwarded, schema) {
var result = processor(req.headers, schema)

// ensure reverse order of addresses
if (typeof result.addrs === 'string') {
result.addrs = result.addrs
.split(/ *, */)
.filter(Boolean)
.reverse()
}

// return all addresses
return addrs
// update forwarded object
return {
addrs: forwarded.addrs.concat(result.addrs).filter(Boolean),
by: result.by ? result.by : forwarded.by,
host: result.host ? result.host : forwarded.host,
port: result.port ? result.port : forwarded.port,
ports: forwarded.ports.concat([result.port]).filter(Boolean),
proto: result.proto ? result.proto : forwarded.proto
}
}, forwarded)
}
30 changes: 30 additions & 0 deletions lib/processor.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
'use strict'

var debug = require('debug-log')('forwarded')

module.exports = function processor (headers, schema) {
if (typeof schema === 'function') {
return schema(headers)
}

var result = {}

var fields = ['addrs', 'host', 'port']

fields.forEach(function (field) {
if (schema[field] && headers[schema[field]]) {
var value = headers[schema[field]]

debug('found header [%s = %s]', schema[field], value)

result[field] = value
}
})

// get protocol
if (typeof schema.proto === 'function') {
result.proto = schema.proto(headers)
}

return result
}
17 changes: 17 additions & 0 deletions lib/schemas/cloudflare.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
'use strict'

var debug = require('debug-log')('forwarded')

module.exports = {
addrs: 'cf-connecting-ip',
proto: function protocol (headers) {
try {
var cf = JSON.parse(headers['cf-visitor'])
if (cf.scheme) {
return cf.scheme
}
} catch (e) {
debug('could not parse "cf-visitor" header: %s', headers['cf-visitor'])
}
}
}
9 changes: 9 additions & 0 deletions lib/schemas/fastly.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
'use strict'

module.exports = {
addrs: 'fastly-client-ip',
port: 'fastly-client-port',
proto: function protocol (headers) {
return headers['fastly-ssl'] ? 'https' : undefined
}
}
13 changes: 13 additions & 0 deletions lib/schemas/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
'use strict'

module.exports = {
'cloudflare': require('./cloudflare'),
'fastly': require('./fastly'),
'microsoft': require('./microsoft'),
'rfc7239': require('./rfc7239'),
'weblogic': require('./weblogic'),
'x-cluster': require('./x-cluster'),
'x-forwarded': require('./x-forwarded'),
'x-real': require('./x-real'),
'z-forwarded': require('./z-forwarded')
}
9 changes: 9 additions & 0 deletions lib/schemas/microsoft.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
'use strict'

module.exports = {
proto: function protocol (headers) {
if (headers['front-end-https']) {
return headers['front-end-https'] === 'on' ? 'https' : 'http'
}
}
}
27 changes: 27 additions & 0 deletions lib/schemas/rfc7239.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
'use strict'

var parse = require('forwarded-parse')
var debug = require('debug-log')('forwarded')

module.exports = function (headers) {
if (!headers.forwarded) {
return {}
}

try {
var result = parse(headers.forwarded)

var forwarded = {
addrs: result.for ? result.for.reverse() : [],
by: result.by ? result.by[0] : undefined,
host: result.host ? result.host[0] : undefined,
port: result.port ? result.port[0] : undefined,
ports: result.port,
proto: result.proto ? result.proto[0] : undefined
}
} catch (e) {
debug(e)
}

return forwarded
}
5 changes: 5 additions & 0 deletions lib/schemas/weblogic.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
'use strict'

module.exports = {
addrs: 'wl-proxy-client-ip'
}
5 changes: 5 additions & 0 deletions lib/schemas/x-cluster.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
'use strict'

module.exports = {
addrs: 'x-cluster-client-ip'
}
20 changes: 20 additions & 0 deletions lib/schemas/x-forwarded.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
'use strict'

module.exports = {
addrs: 'x-forwarded-for',
host: 'x-forwarded-host',
port: 'x-forwarded-port',
proto: function protocol (headers) {
if (headers['x-forwarded-proto']) {
return headers['x-forwarded-proto']
}

if (headers['x-forwarded-protocol']) {
return headers['x-forwarded-protocol']
}

if (headers['x-forwarded-ssl']) {
return headers['x-forwarded-ssl'] === 'on' ? 'https' : 'http'
}
}
}
15 changes: 15 additions & 0 deletions lib/schemas/x-real.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
'use strict'

module.exports = {
addrs: 'x-real-ip',
port: 'x-real-port',
proto: function protocol (headers) {
if (headers['x-real-proto']) {
return headers['x-real-proto']
}

if (headers['x-url-scheme']) {
return headers['x-url-scheme']
}
}
}
Loading