Remove confusing path replacement

[5ebastianMeier]

I want to use following folder structure:

config
+-- user
---- +-- shared.json
---- +-- user.json
---- +-- .....
+-- shared.json

In this structure user.json can reference user/shared.json and user/shared.json can reference config/shared.json. When the base directory changes by referencing another file it causes the behavior to be unpredictable. With a fixed base directory this problem doesn't occur for me.

[bojand]

Hello, thank you for the PR. This code is actually needed for proper file dereferencing. Can you please provide sample schemas and code that reproduces the problem.

[nanuuki]

+1 Having the same issue. Everything works as expected then the options.baseFolder is set and the code in question is commented out.

[bojand]

Hello, thanks for the input. The issue with this PR is that it would break existing functionality and cause failing tests. That would have to be addressed as well as we would need tests for whatever situation this PR is suppose to fix. I am still trying to understand the issue. Can you please explain in detail and provide some sample schemas and code. I've tried to reproduce the issue from original post and everything works fine for me:

structure:

test.js
config
+-- user
---- +-- shared.json
---- +-- user.json
+-- shared.json

config/shared.json:

{
  "description": "Just a basic schema.",
  "title": "Basic Object",
  "type": "object",
  "properties": {
    "id": {
      "description": "unique identifier of a the object",
      "type": "string",
      "minLength": 1
    }
  }
}

config/user/shared.json:

{
  "description": "Just a shared user schema.",
  "title": "Basic Object",
  "type": "object",
  "properties": {
    "email": {
      "description": "email",
      "type": "string",
      "minLength": 1
    },
    "dateOfBirth": {
      "description": "date Of birth",
      "type": "string",
      "format": "date-time"
    }
  }
}

config/user/user.json:

{
  "description": "User schema.",
  "title": "Basic Object",
  "type": "object",
  "properties": {
    "id": {
      "$ref": "../shared.json#/properties/id"
    },
    "email": {
      "$ref": "shared.json#/properties/email"
    },
    "dateOfBirth": {
      "$ref": "shared.json#/properties/dateOfBirth"
    },
    "foo": {
      "description": "foo property",
      "readOnly": true,
      "type": "number"
    }
  }
}

test.js:

var path = require('path');
var deref = require('json-schema-deref');

var input = require('./config/user/user.json');
var baseFolder = path.resolve(path.join(__dirname, 'config/user'));

deref(input, { baseFolder: baseFolder }, function (err, schema) {
  if (err) { console.log(err) }
  console.log(schema);
});

and when I run it it outputs:

{ description: 'User schema.',
  title: 'Basic Object',
  type: 'object',
  properties:
   { id:
      { description: 'unique identifier of a the object',
        type: 'string',
        minLength: 1 },
     email: { description: 'email', type: 'string', minLength: 1 },
     dateOfBirth:
      { description: 'date Of birth',
        type: 'string',
        format: 'date-time' },
     foo: { description: 'foo property', readOnly: true, type: 'number' } } }

as expected.

[5ebastianMeier]

I lost this issue out of focus, but taken from our current state of code (with this fix in place) this is a real life example we're actually using.

config/development/shared.json

{
  "database": {
    "database": "some_db",
    "username": "test",
    "password": "test",
    "host": "localhost",
    "port": 5432
  }
}

config/development/user/user_model.json

{
  "database": {
    "$ref":  "shared.json#database"
  }
}

Now the goal was to be able to reference any json in the whole config folder from any other json in the same folder. As you can see we have the environment the config is targeted at as an extra folder in the config structure (in this case development). This folder serves as our root folder for any schema references, because relative paths would be too hard to maintain.

I might find some time on the weekend to put together a full test for a more complex scenario with a deeper folder structure.

[nanuuki]

Indeed, just removing a block of code isn't a viable solution as @bojand mentioned. Actually the current case where this is happening seems to be caused by having other refs in the file that's originally referenced (animals/axolotl.json => refs/common-refs.json => refs/other-refs.json) - so instead of the expected behaviour of having refs relative to the baseFolder the outcome is that dereferencing fails while trying to read animals/refs/other-refs.json. This is all from memory, I'll post a concrete example.

[nanuuki]

Okay @bojand, got an example for you here :)

node_modules
└ (omitted)
objects
└ example-object.json
shared
├ shared-definitions.json
└ shared-object.json
test.js

objects/example-object.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/object/example-object",
  "properties": {
    "attribute": {
      "$ref": "shared/shared-object.json#/definitions/objectAttribute"
    }
  }
}

shared/shared-definitions.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/shared/shared-definitions",
  "definitions": {
    "problematicDefinition": {
      "type": "number"
    }
  }
}

shared/shared-object.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/shared/shared-object",
  "definitions": {
    "objectAttribute": {
      "type": "string"
    }
  },
  "properties": {
    "problematic": {
      "$ref": "shared/shared-definitions.json#/definitions/problematicDefinition"
    }
  }
}

test.js

'use strict';

const path = require('path');
const deref = require('json-schema-deref');
const schema = require('./objects/example-object.json');

const options = {
  failOnMissing: true,
  baseFolder: path.resolve('./')
}

deref(schema, options, (error, fullSchema) => {
  console.log(error);
  console.log(JSON.stringify(fullSchema, null, 2));
});

output, unmodified json-schema-deref:

$ node test.js
{ [Error: ENOENT: no such file or directory, open '/tmp/json-schema-deref-example/shared/shared/shared-definitions.json']
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/tmp/json-schema-deref-example/shared/shared/shared-definitions.json' }
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/object/example-object",
  "properties": {
    "attribute": {
      "$ref": "shared/shared-object.json#/definitions/objectAttribute"
    }
  }
}

output, json-schema-deref modified as in #14:

$ node test.js
undefined
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/object/example-object",
  "properties": {
    "attribute": {
      "type": "string"
    }
  }
}

So the problem in this case seems to be that shared/shared-object.json has the $ref in properties/problematic.

[bojand]

Hmm the way we do file resolution is simply using path.resolve, but the relative paths within schemas are still suppose to be valid when resolved from whatever is supplied to baseFolder. So in the case of the example provided it should work with objects/example-object.json being:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://example.com/object/example-object",
  "properties": {
    "attribute": {
      "$ref": "../shared/shared-object.json#/definitions/objectAttribute"
    }
  }
}

and then baseFolder: path.resolve('./objects') being passed into the options. Does this not satisfy? Maybe the documentation is unclear, i'll try and improve it.

[nanuuki]

@bojand All the refs have paths relative to the baseFolder which is set to the project root initially in the options, and they should work correctly without any modifications to the schemas.

The problem is that baseFolder is changed while dereferencing (the block that's removed in this PR). So the suggested workaround isn't really a solution since it'll probably break other refs later. I would like to be able to specify "absolute" (always relative to the same baseFolder) refs to files. Atm the baseFolder option spec is misleading.

[5ebastianMeier]

@nanuuki That's exactly what I was looking for. Have one base path and let all references - no matter how deep - use paths relative to that very origin.