JSON Schema Validation in Mongo 3.6

Using additionalProperties to validate all fields in the document

https://unsplash.com/photos/6yiq2HPIW7Q

Mongo 3.6 released a new validation system that uses the semi-standard JSON schema format: https://docs.mongodb.com/manual/core/schema-validation/

Switching to a standard format is nice, but not very impactful. The game-changer, however, is the addition of the additionalProperties flag, which allows you to limit exactly what object properties can be on the document.

For example, you used to do this:

db.createCollection('myCollection', {
  validator: {
    a: { $type: 'number' },
    b: { $type: 'string' },
  },
});

Which worked to validate those fields, but unfortunately let you insert any other fields on the document that aren’t a or b.

Now you can do this:

db.createCollection('myCollection', {
  validator: {
    $jsonSchema: {
      bsonType: 'object',
      additionalProperties: false,
      required: ['a', 'b'],
      properties: {
        _id: {
          bsonType: 'objectId',
        },
        a: {
          bsonType: 'number',
        },
        b: {
          bsonType: 'string',
        },
      },
    },
  },
});

The key being additionalProperties: false, which tells Mongo to reject any documents that have fields that are not in the schema.

Which is pretty great. But, there are some pitfalls that aren’t immediately obvious.

Error reporting for failed validation is a bit of a mess

You will get Document failed validation, but it won’t tell you why it failed. The best I’ve figured out to do is manually change the schema incrementally, and test between each change, until you can figure out what part is causing the failure. Remember to comment out additionalProperties: false as a testing step as well, since otherwise documents will fail validation with any extra fields on them.

Hopefully the Mongo folks improve error reporting for validation in the future.

Update: We wrote a package to help with Mongo’s lack of error messages: https://www.npmjs.com/package/mongo-schemer. It runs all schema validation errors through ajv to provide a detailed explanation of why the document failed validation.

Every validator needs _id

Every document has an _id field, so you must have _id in the root of your schema if you use additionalProperties: false, or every document you try to insert or update will fail validation. Example:

_id: {
  bsonType: 'objectId',
}

This makes sense logically, but the documentation could be a bit clearer, especially with validation errors being so vague.

You need additionalProperties on every object definition

If you have nested objects in your schema, then you need additionalProperties: false on every object in order to truly limit what gets inserted and updated.

Example of what not to do:

db.createCollection('myCollection', {
  validator: {
    $jsonSchema: {
      bsonType: 'object',
      additionalProperties: false,
      properties: {
        _id: {
          bsonType: 'objectId',
        },
        otherObject: {
          bsonType: 'object',
          properties: {
            a: {
              bsonType: 'number',
            },
            b: {
              bsonType: 'string',
            },
          },
        },
      },
    },
  },
});

With that schema, this document will fail validation as desired:

{
  otherObject: {
    a: 1,
    b: 'test',
  },
  someOtherField: 'test',
}

But this document will pass validation, because additional properties are allowed on the otherObject nested object:

{
  otherObject: {
    a: 1,
    b: 'test',
    someOtherField: 'test',
  },
}

To prevent this, add additionalProperties: false to every object definition, like so:

db.createCollection('myCollection', {
  validator: {
    $jsonSchema: {
      bsonType: 'object',
      additionalProperties: false,
      properties: {
        _id: {
          bsonType: 'objectId',
        },
        otherObject: {
          bsonType: 'object',
          additionalProperties: false,
          properties: {
            a: {
              bsonType: 'number',
            },
            b: {
              bsonType: 'string',
            },
          },
        },
      },
    },
  },
});

Conclusion

Schema validation with JSON schemas is a neat feature, and finally being able to limit what fields can be inserted by using additionalProperties is great. That said, everyone will almost certainly still need to do application-level validation to provide reasonable error messaging, so this feature is mostly useful to add an extra layer of data integrity to what you’re already doing.