magni.reproducibility.io module¶
Module providing input/output functions to databases containing results from reproducible research.
Routine listings¶
- annotate_database(h5file)
- Function for annotating an existing HDF5 database.
- chase_database(h5file)
- Function for chasing an existing HDF5 database.
- create_database(h5file)
- Function for creating a new annotated and chased HDF5 database.
- read_annotations(h5file)
- Function for reading annotations in an HDF5 database.
- read_chases(h5file)
- Function for reading chases in an HDF5 database.
- remove_annotations(h5file)
- Function for removing annotations in an HDF5 database.
- remove_chases(h5file)
- Function for removing chases in an HDF5 database.
- write_custom_annotation(h5file, annotation_name, annotation_value,
- annotations_sub_group=None) Write a custom annotation to an HDF5 database.
See also
magni.reproducibility._annotation.get_conda_info
- Conda annotation
magni.reproducibility._annotation.get_git_revision
- Git annotation
magni.reproducibility._annotation.get_platform_info
- Platform annotation
magni.reproducibility._annotation.get_datetime
- Date and time annotation
magni.reproducibility._annotation.get_magni_config
- Magni config annotation
magni.reproducibility._annotation.get_magni_info
- Magni info annotation
magni.reproducibility._chase.get_main_file_name
- Magni main file name chase
magni.reproducibility._chase.get_main_file_source
- Magni source code chase
magni.reproducibility._chase.get_main_source
- Magni main source code chase
magni.reproducibility._chase.get_stack_trace
- Magni stack trace chase
-
magni.reproducibility.io.
annotate_database
(h5file)[source]¶ Annotate an HDF5 database with information about Magni and the platform.
The annotation consists of a group in the root of the h5file having nodes that each provide information about Magni or the platform on which this function is run.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database that should be annotated. See also
magni.reproducibility._annotation.get_conda_info()
- Conda annotation
magni.reproducibility._annotation.get_git_revision()
- Git annotation
magni.reproducibility._annotation.get_platform_info()
- Platform annotation
magni.reproducibility._annotation.get_datetime()
- Date and time annotation
magni.reproducibility._annotation.get_magni_config()
- Magni config annotation
magni.reproducibility._annotation.get_magni_info()
- Magni info annotation
Notes
The annotations of the database includes the following:
- conda_info - Information about Continuum Anacononda install
- git_revision - Git revision and tag of Magni
- platform_info - Information about the current platform (system)
- datetime - The current date and time
- magni_config - Infomation about the current configuration of Magni
- magni_info - Information from help(magni)
Examples
Annotate the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import annotate_database >>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file: ... annotate_database(h5file)
-
magni.reproducibility.io.
chase_database
(h5file)[source]¶ Chase an HDF5 database to track information about stack and source code.
The chase consist of a group in the root of the h5file having nodes that each profide information about the program execution that led to this chase of the database.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database that should be chased. See also
magni.reproducibility._chase.get_main_file_name()
- Name of main file
magni.reproducibility._chase.get_main_file_source()
- Main file source code
magni.reproducibility._chase.get_main_source()
- Source code around main
magni.reproducibility._chase.get_stack_trace()
- Complete stack trace
Notes
The chase include the following information:
- main_file_name - Name of the main file/script that called this function
- main_file_source - Full source code of the main file/script
- main_source - Extract of main file source code that called this function
- stack_trace - Complete stack trace up until the call to this function
Examples
Chase the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import chase_database >>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file: ... chase_database(h5file)
-
magni.reproducibility.io.
create_database
(path, overwrite=True)[source]¶ Create a new HDF database that is annotated and chased.
A new HDF database is created and it is annotated using
magni.reproducibility.io.annotate_database
and chased usingmagni.reproducibility.io.annotate_database
. If the overwrite flag is true and existing database at path is overwritten.Parameters: - path (str) – The path to the HDF file that is to be created.
- overwrite (bool) – The flag that indicates if an existing database should be overwritten.
See also
magni.reproducibility.io.annotate_database()
- Database annotation
magni.reproducibility.io.chase_database()
- Database chase
Examples
Create a new database named ‘new_db.hdf5’:
>>> from magni.reproducibility.io import create_database >>> create_database('new_db.hdf5')
-
magni.reproducibility.io.
read_annotations
(h5file)[source]¶ Read the annotations to an HDF5 database.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database from which the annotations are read. Returns: annotations (dict) – The annotations read from the HDF5 database. Raises: ValueError
– If the annotations to the HDF5 database does not conform to the Magni annotation standard.Notes
The returned dict holds a key for each annotation in the database. The value corresponding to a given key is in itself a dict. See magni.reproducibility.annotate_database for examples of such annotations.
Examples
Read annotations from the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import read_annotations >>> with magni.utils.multiprocessing.File('db.hdf5', mode='r') as h5file: ... annotations = read_annotations(h5file)
-
magni.reproducibility.io.
read_chases
(h5file)[source]¶ Read the chases to an HDF5 database.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database from which the chases are read. Returns: chasess (dict) – The chases read from the HDF5 database. Raises: ValueError
– If the chases to the HDF5 database does not conform to the Magni chases standard.Notes
The returned dict holds a key for each chase in the database. The value corresponding to a given key is a string. See magni.reproducibility.chase_database for examples of such chases.
Examples
Read chases from the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import read_chases >>> with magni.utils.multiprocessing.File('db.hdf5', mode='r') as h5file: ... chases = read_chases(h5file)
-
magni.reproducibility.io.
remove_annotations
(h5file)[source]¶ Remove the annotations from an HDF5 database.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database from which the annotations are removed. Examples
Remove annotations from the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import remove_annotations >>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file: ... remove_annotations(h5file)
-
magni.reproducibility.io.
remove_chases
(h5file)[source]¶ Remove the chases from an HDF5 database.
Parameters: h5file (tables.file.File) – The handle to the HDF5 database from which the chases are removed. Examples
Remove chases from the database named ‘db.hdf5’:
>>> import magni >>> from magni.reproducibility.io import remove_chases >>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file: ... remove_chases(h5file)
-
magni.reproducibility.io.
write_custom_annotation
(h5file, annotation_name, annotation_value, annotations_sub_group=None)[source]¶ Write a custom annotation to an HDF5 database.
The annotation is written to the h5file under the annotation_name such that it holds the annotation_value.
Parameters: - h5file (tables.file.File) – The handle to the HDF5 database to which the annotation is written.
- annotation_name (str) – The name of the annotation to write.
- annotation_value (a JSON serialisable object) – The annotation value to write.
- annotations_sub_group (str) – The group node under “/annotations” to which the custom annotation is written (the default is None which implies that the custom annotation is written directly under “/annotations”).
Notes
The annotation_value must be a JSON seriablisable object.
Examples
Write a custom annotation to an HDF5 database.
>>> import magni >>> from magni.reproducibility.io import write_custom_annotation >>> annotation_name = 'custom_annotation' >>> annotation_value = 'the value' >>> with magni.utils.multiprocessing.File('db.hdf5', mode='a') as h5file: ... write_custom_annotation(h5file, annotation_name, annotation_value) ... annotations = magni.reproducibility.io.read_annotations(h5file) >>> str(annotations['custom_annotation']) 'the value'
-
magni.reproducibility.io.
_recursive_annotation_read
(h5_annotations, out_annotations_dict)[source]¶ Recursively read annotations from an annotation group
Parameters: - h5_annotations (tables.group.Group) – The group to read annotations from.
- out_annotations_dict (dict) – The dictionary to store the read annotations in.