magni.reproducibility.io module

Module providing input/output functions to databases containing results from reproducible research.

Routine listings

annotate_database(h5file)

Function for annotating an existing HDF5 database.

chase_database(h5file)

Function for chasing an existing HDF5 database.

create_database(h5file)

Function for creating a new annotated and chased HDF5 database.

read_annotations(h5file)

Function for reading annotations in an HDF5 database.

read_chases(h5file)

Function for reading chases in an HDF5 database.

remove_annotations(h5file)

Function for removing annotations in an HDF5 database.

remove_chases(h5file)

Function for removing chases in an HDF5 database.

write_custom_annotation(h5file, annotation_name, annotation_value,

annotations_sub_group=None) Write a custom annotation to an HDF5 database.

See also

magni.reproducibility._annotation.get_conda_info

Conda annotation

magni.reproducibility._annotation.get_git_revision

Git annotation

magni.reproducibility._annotation.get_platform_info

Platform annotation

magni.reproducibility._annotation.get_datetime

Date and time annotation

magni.reproducibility._annotation.get_magni_config

Magni config annotation

magni.reproducibility._annotation.get_magni_info

Magni info annotation

magni.reproducibility._chase.get_main_file_name

Magni main file name chase

magni.reproducibility._chase.get_main_file_source

Magni source code chase

magni.reproducibility._chase.get_main_source

Magni main source code chase

magni.reproducibility._chase.get_stack_trace

Magni stack trace chase

magni.reproducibility.io.annotate_database(h5file)

Annotate an HDF5 database with information about Magni and the platform.

The annotation consists of a group in the root of the h5file having nodes that each provide information about Magni or the platform on which this function is run.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database that should be annotated.

See also

magni.reproducibility._annotation.get_conda_info()

Conda annotation

magni.reproducibility._annotation.get_git_revision()

Git annotation

magni.reproducibility._annotation.get_platform_info()

Platform annotation

magni.reproducibility._annotation.get_datetime()

Date and time annotation

magni.reproducibility._annotation.get_magni_config()

Magni config annotation

magni.reproducibility._annotation.get_magni_info()

Magni info annotation

Notes

The annotations of the database includes the following:

• conda_info - Information about Continuum Anacononda install
• git_revision - Git revision and tag of Magni
• platform_info - Information about the current platform (system)
• datetime - The current date and time
• magni_config - Infomation about the current configuration of Magni
• magni_info - Information from help(magni)

Examples

Annotate the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import annotate_database
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file:
...     annotate_database(h5file)

magni.reproducibility.io.chase_database(h5file)

Chase an HDF5 database to track information about stack and source code.

The chase consist of a group in the root of the h5file having nodes that each profide information about the program execution that led to this chase of the database.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database that should be chased.

See also

magni.reproducibility._chase.get_main_file_name()

Name of main file

magni.reproducibility._chase.get_main_file_source()

Main file source code

magni.reproducibility._chase.get_main_source()

Source code around main

magni.reproducibility._chase.get_stack_trace()

Complete stack trace

Notes

The chase include the following information:

• main_file_name - Name of the main file/script that called this function
• main_file_source - Full source code of the main file/script
• main_source - Extract of main file source code that called this function
• stack_trace - Complete stack trace up until the call to this function

Examples

Chase the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import chase_database
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file:
...     chase_database(h5file)

magni.reproducibility.io.create_database(path, overwrite=True)

Create a new HDF database that is annotated and chased.

A new HDF database is created and it is annotated using magni.reproducibility.io.annotate_database and chased using magni.reproducibility.io.annotate_database. If the overwrite flag is true and existing database at path is overwritten.

Parameters:

• path (str) – The path to the HDF file that is to be created.
• overwrite (bool) – The flag that indicates if an existing database should be overwritten.

See also

magni.reproducibility.io.annotate_database()

Database annotation

magni.reproducibility.io.chase_database()

Database chase

Examples

Create a new database named ‘new_db.hdf5’:

>>> from magni.reproducibility.io import create_database
>>> create_database('new_db.hdf5')

magni.reproducibility.io.read_annotations(h5file)

Read the annotations to an HDF5 database.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database from which the annotations are read. Returns:annotations (dict) – The annotations read from the HDF5 database. Raises:ValueError – If the annotations to the HDF5 database does not conform to the Magni annotation standard.

Notes

The returned dict holds a key for each annotation in the database. The value corresponding to a given key is in itself a dict. See magni.reproducibility.annotate_database for examples of such annotations.

Examples

Read annotations from the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import read_annotations
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='r') as h5file:
...    annotations = read_annotations(h5file)

magni.reproducibility.io.read_chases(h5file)

Read the chases to an HDF5 database.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database from which the chases are read. Returns:chasess (dict) – The chases read from the HDF5 database. Raises:ValueError – If the chases to the HDF5 database does not conform to the Magni chases standard.

Notes

The returned dict holds a key for each chase in the database. The value corresponding to a given key is a string. See magni.reproducibility.chase_database for examples of such chases.

Examples

Read chases from the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import read_chases
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='r') as h5file:
...    chases = read_chases(h5file)

magni.reproducibility.io.remove_annotations(h5file)

Remove the annotations from an HDF5 database.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database from which the annotations are removed.

Examples

Remove annotations from the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import remove_annotations
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file:
...    remove_annotations(h5file)

magni.reproducibility.io.remove_chases(h5file)

Remove the chases from an HDF5 database.

Parameters:h5file (tables.file.File) – The handle to the HDF5 database from which the chases are removed.

Examples

Remove chases from the database named ‘db.hdf5’:

>>> import magni
>>> from magni.reproducibility.io import remove_chases
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file:
...    remove_chases(h5file)

magni.reproducibility.io.write_custom_annotation(h5file, annotation_name, annotation_value, annotations_sub_group=None)

Write a custom annotation to an HDF5 database.

The annotation is written to the h5file under the annotation_name such that it holds the annotation_value.

Parameters:

• h5file (tables.file.File) – The handle to the HDF5 database to which the annotation is written.
• annotation_name (str) – The name of the annotation to write.
• annotation_value (a JSON serialisable object) – The annotation value to write.
• annotations_sub_group (str) – The group node under “/annotations” to which the custom annotation is written (the default is None which implies that the custom annotation is written directly under “/annotations”).

Notes

The annotation_value must be a JSON seriablisable object.

Examples

Write a custom annotation to an HDF5 database.

>>> import magni
>>> from magni.reproducibility.io import write_custom_annotation
>>> annotation_name = 'custom_annotation'
>>> annotation_value = 'the value'
>>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file:
...    write_custom_annotation(h5file, annotation_name, annotation_value)
...    annotations = magni.reproducibility.io.read_annotations(h5file)
>>> str(annotations['custom_annotation'])
'the value'

magni.reproducibility.io._recursive_annotation_read(h5_annotations, out_annotations_dict)

Recursively read annotations from an annotation group

Parameters:

• h5_annotations (tables.group.Group) – The group to read annotations from.
• out_annotations_dict (dict) – The dictionary to store the read annotations in.