Magni Documentation

magni is a Python package developed by Christian Schou Oxvig and Patrick Steffen Pedersen in collaboration with Jan Østergaard, Thomas Arildsen, Tobias L. Jensen, and Torben Larsen. The work was supported by 1) the Danish Council for Independent Research | Technology and Production Sciences - via grant DFF-1335-00278 for the project Enabling Fast Image Acquisition for Atomic Force Microscopy using Compressed Sensing and 2) the Danish e-Infrastructure Cooperation - via a grant for a high performance computing system for the project “High Performance Computing SMP Server for Signal Processing”.

This page gives an Introduction to the package, briefly describes How to Read the Documentation, and explains how to actually use The Package.

Introduction

magni [5] is a Python package which provides functionality for increasing the speed of image acquisition using Atomic Force Microscopy (AFM) (see e.g. [1] for an introduction). The image acquisition algorithms of magni are based on the Compressed Sensing (CS) signal acquisition paradigm (see e.g. [2] or [3] for an introduction) and include both sensing and reconstruction. The sensing part of the acquisition generates sensed data from regular images possibly acquired using AFM. This is done by AFM hardware simulation. The reconstruction part of the acquisition reconstructs images from sensed data. This is done by CS reconstruction using well-known CS reconstruction algorithms modified for the purpose. The Python implementation of the above functionality uses the standard library, a number of third-party libraries, and additional utility functionality designed and implemented specifically for magni. The functionality provided by magni can thus be divided into five groups:

  • Atomic Force Microscopy (magni.afm): AFM specific functionality including AFM image acquisition, AFM hardware simulation, and AFM data file handling.
  • Compressed Sensing (magni.cs): General CS functionality including signal reconstruction and phase transition determination.
  • Imaging (magni.imaging): General imaging functionality including measurement matrix and dictionary construction in addition to visualisation and evaluation.
  • Reproducibility (magni.reproducibility): Tools that may aid in increasing the reproducibility of result obtained using magni.
  • Utilities (magni.utils): General Python utilities including multiprocessing, tracing, and validation.

See Other Resources as well as Notation for further documentation related to the project and the Tests and Examples to draw inspiration from.

References

[1]B. Bhushan and O. Marti , “Scanning Probe Microscopy – Principle of Operation, Instrumentation, and Probes”, in Springer Handbook of Nanotechnology, pp. 573-617, 2010.
[2]D.L. Donoho, “Compressed Sensing”, IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
[3]E.J. Candès, J. Romberg, and T. Tao, “Robust Uncertainty Principles: Exact Signal Reconstruction From Highly Incomplete Frequency Information”, IEEE Transactions on Information Theory, vol. 52, no.2, pp. 489-509, Feb. 2010.

Footnotes

[5]In Norse mythology, Magni is son of Thor and the god of strength. However, the word MAGNI could as well be an acronym for almost anything including “Making AFM Grind the Normal Impatience”.

How to Read the Documentation

The included subpackages, modules, classes and functions are documented through Python docstrings using the same format as the third-party library, numpy, i.e. using the numpydoc standard. A description of any entity can thus be found in the source code of magni in the docstring of that entity. For readability, the documentation has been compiled using Sphinx to produce this HTML page which can be found in the magni folder under ‘/doc/build/html/index.html’. The entire documentation is also available as a PDF file in the magni folder under ‘/doc/build/pdf/index.pdf’.

Building the Documentation

The HTML documentation may be built from source using the supplied Makefile in the magni folder under ‘/doc/’. Make sure the required Dependencies for building the documentation are installed. The build process consists of running three commands:

$ make sourceclean
$ make docapi
$ make html

Note

In the make docapi command it is assumed that the python interpreter is available on the PATH under the name python. If the python interpreter is available under another name, the PYTHONINT variable may be set, e.g. “make PYTHONINT=python2 docapi” if the python interpreter is named python2.

Run make clean to remove all builds created by Sphinx under ‘/doc/build’.

The Package

The source code of magni is released under the BSD 2-Clause license, see the License section. To install magni, follow the instructions given under Download and Installation.

magni has been tested with Anaconda (64-bit) on Linux. It may or may not work with other Python distributions and/or operating systems. See also the list of Dependencies for magni.

License

Magni is licensed under the OSI-approved BSD 2-Clause License. See https://opensource.org/licenses/BSD-2-Clause for further information.

Copyright (c) 2014-2016,

Primary developers
Christian Schou Oxvig and Patrick Steffen Pedersen.
Additional developers
Jan Østergaard, Thomas Arildsen, Tobias L. Jensen, and Torben Larsen.
Institution
Aalborg University, Department of Electronic Systems, Signal and Information Processing, Fredrik Bajers Vej 7, DK-9220 Aalborg, Denmark.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Download and Installation

All official releases of magni are available for download at doi:10.5278/VBN/MISC/Magni. The source code is hosted on GitHub at https://github.com/SIP-AAU/Magni.

To use Magni, extract the downloaded archive and include the extracted magni folder in your PYTHONPATH.

Note

The magni package (excluding examples and documentation) is also available on an “as is” basis in source form at PyPi and as a conda package at Anaconda.org.

Dependencies

magni has been designed for use with Python 2 >= 2.7 or Python 3 >= 3.3

Note

The below listed dependency version requirements are the absolute minimum tested versions. Some of these libraries may not work with newer versions of Python unless a newer version of those libraries are used.

Required third party dependencies for magni are:

Optional third party dependencies for magni are:

  • Bottleneck (Tested on version >=1.0.0) (For speed-up of some algorithms)
  • Conda (Tested on version >= 3.7.0) (For automatic metadata capture of a Conda managed Python environment)
  • Coverage (Tested on version >= 3.7) (For running the test suite script)
  • IPython (Tested on version >= 2.1) or Jupyter [4] (Tested on version >= 1.0) (For running the IPython notebook examples)
  • Math Kernel Library (MKL) (Tested on version >= 11.1) (For accelerated vector operations)
  • Nose (Tested on version >= 1.3) (For running unittests and doctests)
  • PEP8 (Tested on version >= 1.5) (For running style check tests)
  • PIL (or Pillow) (Tested on version >= 1.1.7) (For running the IPython notebook examples as tests)
  • Pyflakes (Tested on version >= 0.8) (For running style check tests)
  • Radon (Tested on version >= 1.2) (For running style check tests)
  • scikit-learn (Tested on version >=0.15.1) (For using the scikit-learn logistic regression solver in magni.cs.phase_transition)
  • Sphinx (Tested on version >= 1.3.1) (For building the documentation from source)
[4]The IPython project has evolved into Jupyter. When using Jupyter make sure to install the “full” Jupyter metapackage which includes all Jupyter and IPython components.

Note

When using the magni.utils.multiprocessing subpackage, it is generally a good idea to restrict acceleration libraries like MKL or OpenBLAS to use a single thread. If MKL is installed, this is done automatically at runtime in the magni.utils.multiprocessing subpackage. If other libraries than MKL are used, the user has to manually set an appropriate evironmental variable, e.g. OMP_NUM_THREADS.

You may use the dep_check.py script found in the Magni folder under ‘/magni/tests/’ to check for missing dependencies for Magni. Simply run the script to print a dependency report, e.g.:

$ python dep_check.py

Tests

A test suite consisting of unittests, doctests, the IPython notebook examples, and several style checks is included in magni. The tests are organized in python modules found in the Magni folder under ‘/magni/tests/’. Each module features one or more unittest.TestCase classes containing the tests. Thus, the tests may be invoked using any test runner that supports the unittest.TestCase. E.g. running the wrapper for the doctests using Nose is done by issuing:

$ nosetests magni/tests/wrap_doctests.py

The entire test suite may be run by executing the convenience script run_tests.py:

$ magni/tests/run_tests.py

Note

This convenience script assumes that magni is available on the PYTHONPATH as explained under Download and Installation.

Bug Reports

Found a bug? Bug report may be submitted using the magni GitHub issue tracker. Please include all relevant details in the bug report, e.g. version of Magni, input/output data, stack traces, etc. If the supplied information does not entail reproducibility of the problem, there is no way we can fix it.

Note

Due to limited funds, we are unfortunately unable make any guarantees, whatsoever, that reported bugs will be fixed.

Other Resources

Papers published in relation to the Enabling Fast Image Acquisition for Atomic Force Microscopy using Compressed Sensing project:

  • C. S. Oxvig, T. Arildsen, and T. Larsen, “Structure Assisted Compressed Sensing Reconstruction of Undersampled AFM Images”, Ultramicroscopy, vol. 172, pp. 1-9, Jan. 2017, doi:10.1016/j.ultramic.2016.09.011.
  • C. S. Oxvig, T. Arildsen, and T. Larsen, “Storing Reproducible Results from Computational Experiments using Scientific Python Packages”, in Proceedings of the 15th Python in Science Conference, pp. 45-50, Austin, Texas, USA, July 11 - 17, 2016, http://conference.scipy.org/proceedings/scipy2016/christian_oxvig.html
  • P. S. Pedersen, C. S. Oxvig, J. Østergaard, and T. Larsen, “Validating Function Arguments in Python Signal Processing Applications”, in Proceedings of the 15th Python in Science Conference, pp. 106-113, Austin, Texas, USA, July 11 - 17, 2016, http://conference.scipy.org/proceedings/scipy2016/patrick_pedersen.html
  • P. S. Pedersen, J. Østergaard and T. Larsen, “Modelling reconstruction quality of Lissajous undersampled atomic force microscopy images,” in 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 245-248, Prague, Czech Republic, 2016, doi:10.1109/ISBI.2016.7493255.
  • T. Arildsen, C. S. Oxvig, P. S. Pedersen, J. Østergaard, and T. Larsen, “Reconstruction Algorithms in Undersampled AFM Imaging”, IEEE Journal of Selected Topics in Signal Processing, vol. 10, no. 1, pp. 31-46, Feb. 2016, doi:10.1109/JSTSP.2015.2500363.
  • P. S. Pedersen, J. Østergaard, and T. Larsen, “Predicting reconstruction quality within compressive sensing for atomic force microscopy,” in 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 418-422 Orlando, FL, 2015, doi:10.1109/GlobalSIP.2015.7418229.
  • C. S. Oxvig, P. S. Pedersen, T. Arildsen, J. Østergaard, and T. Larsen, “Magni: A Python Package for Compressive Sampling and Reconstruction of Atomic Force Microscopy Images”, Journal of Open Research Software, vol. 2, no. 1, p. e29, Oct. 2014, doi:10.5334/jors.bk.
  • T. L. Jensen, T. Arildsen, J. Østergaard, and T. Larsen, “Reconstruction of Undersampled Atomic Force Microscopy Images : Interpolation versus Basis Pursuit”, in International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 130-135, Kyoto, Japan, December 2 - 5, 2013, doi:10.1109/SITIS.2013.32.

Notation

To the extent possible, a consistent notation has been used in the documentation and implementation of algorithms that are part of magni. All the details are described in A Note on Notation.

Examples

The magni package includes a large number of examples showing its capabilities. See the dedicated Examples page for all the details.