Drivers and Tools

Language drivers and other tools that want to comply with the Bento framework should observe the latest specification.

bento_mdf is a reference driver that validates and loads MDF into a bento-meta python object model.

Validator

A simple command line validator test-mdf.py is included in bento_mdf. Install it like so:

$ pip install bento-mdf

and run:

$ test-mdf.py [model-file.yaml] [model-props-file.yaml] ...

Scripts

Scripts test-mdf.py and load-mdf.py are included in the distribution. test-mdf is a verbose validator that can be used to find issues in a set of local MDFs using the MDF JSONSchema. load-mdf will load a valid set of MDFs into an existing Neo4j Metamodel Database.

test-mdf Usage

$ test-mdf.py -h
usage: test-mdf.py [-h] [--schema SCHEMA] [--quiet] [--log-file LOG_FILE]
                   mdf-file [mdf-file ...]

Validate MDF against JSONSchema

positional arguments:
  mdf-file             MDF yaml files for validation

optional arguments:
  -h, --help           show this help message and exit
  --schema SCHEMA      MDF JSONschema file
  --quiet              Suppress output; return only exit value
  --log-file LOG_FILE  Log file name

See “Validator Notes” below.

load-mdf Usage

$ ./load-mdf.py -h
usage: load-mdf.py [-h] --commit COMMIT [--handle HANDLE] [--user USER] [--passw PASSW]
                   [--bolt BoltURL] [--put]
                   [MDF-FILE ...]

Load model in MDF into an MDB

positional arguments:
  MDF-FILE         MDF file(s)/url(s)

optional arguments:
  -h, --help       show this help message and exit
  --commit COMMIT  commit SHA1 for MDF instance (if any)
  --handle HANDLE  model handle
  --user USER      MDB username
  --passw PASSW    MDB password
  --bolt BoltURL   MDB Bolt url endpoint (specify as 'bolt://...')
  --put            Load model to database

Validator test-mdf.pyNotes

The --schema argument is optional. test-mdf.py will automatically retrieve the latest mdf-schema.yaml in the master branch of this repo.

The script tests both the syntax of the YAML (for both schema and MDF files), and the validity of the files with respect to the JSONSchema (for both schema and MDF files).

The errors are as emitted from the PyYaml and jsonschema packages, and can be rather obscure.

• Successful test

    $ test-mdf.py samples/ctdc_model_file.yaml samples/ctdc_model_properties_file.yaml 
    Checking schema YAML =====
    Checking as a JSON schema =====
    Checking instance YAML =====
    Checking instance against schema =====
  

• Bad YAML syntax

    $ test-mdf.py samples/ctdc_model_bad.yaml samples/ctdc_model_properties_file.yaml 
    Checking schema YAML =====
    Checking as a JSON schema =====
    Checking instance YAML =====
    YAML error in 'samples/ctdc_model_bad.yaml':
    while parsing a block mapping
      in "samples/ctdc_model_bad.yaml", line 1, column 1
    expected <block end>, but found '<block mapping start>'
      in "samples/ctdc_model_bad.yaml", line 3, column 3
  

• Schema-invalid YAML

    $ test-mdf.py samples/ctdc_model_file_invalid.yaml samples/ctdc_model_properties_file.yaml 
    Checking schema YAML =====
    Checking as a JSON schema =====
    Checking instance YAML =====
    Checking instance against schema =====
    ['show_node', 'specimen_id', 'biopsy_sequence_number', 'specimen_type'] is not of type 'object'
    
    Failed validating 'type' in schema['properties']['Nodes']['additionalProperties']:
        {'$id': '#nodeSpec',
         'properties': {'Category': {'$ref': '#/defs/snake_case_id'},
                        'Props': {'oneOf': [{'items': {'$ref': '#/defs/snake_case_id'},
                                             'type': 'array',
                                             'uniqueItems': True},
                                            {'type': 'null'}]},
                        'Tags': {'$ref': '#/defs/tagsSpec'}},
         'required': ['Props'],
         'type': 'object'}
    
    On instance['Nodes']['specimen']:
        ['show_node', 'specimen_id', 'biopsy_sequence_number', 'specimen_type']
  

Testing the tester

The validator code itself can be tested as follows:

pip install tox
cd bento-mdf/validators/mdf-validate
tox