notebooks/eg.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Note\n",
    "\n",
    "In this notebook, I show how to use eg.py.\n",
    "\n",
    "*eg* module is built on top of [**xarray**](http://xarray.pydata.org/),\n",
    "which brings N-dimensional variants of the core pandas data structures.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing some libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from PyLHD import eg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import xarray as xr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load file from local\n",
    "\n",
    "We prepare example eg files at 'PyLHD/testings/eg_example1d.txt'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load\n",
    "filename = '../testings/eg_example2d.txt'\n",
    "example_data = eg.load(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Basic usage\n",
    "\n",
    "The generated *'example_data'* is xarray.DataSet, this inherites all the methods and members.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Access to coordinate data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, to see the coordinate data,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coordinates:\n",
      "  * TIME     (TIME) float64 0.01 0.02 0.03 0.04 0.05\n",
      "  * R        (R) float64 0.1 0.2 0.3\n"
     ]
    }
   ],
   "source": [
    "print(example_data.coords)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *coords* is an OrderedDict.  \n",
    "The dict like method can be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KeysView(Coordinates:\n",
       "  * TIME     (TIME) float64 0.01 0.02 0.03 0.04 0.05\n",
       "  * R        (R) float64 0.1 0.2 0.3)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to see all the keys.\n",
    "example_data.coords.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The entries of the OrderedDict is xr.DataArray that can be used as np.array,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.DataArray 'TIME' (TIME: 5)>\n",
       "array([ 0.01,  0.02,  0.03,  0.04,  0.05])\n",
       "Coordinates:\n",
       "  * TIME     (TIME) float64 0.01 0.02 0.03 0.04 0.05\n",
       "Attributes:\n",
       "    Unit: s"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to access the values in 'TIME' coords\n",
    "example_data.coords['TIME']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.DataArray 'TIME' ()>\n",
       "array(0.15000000000000002)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This acts as array-like.\n",
    "np.sum(example_data.coords['TIME'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Access to variable data\n",
    "\n",
    "To see what data are contained,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KeysView(Data variables:\n",
       "    a        (TIME, R) float64 0.11 0.12 0.13 0.21 0.22 0.23 0.31 0.32 0.33 ...\n",
       "    b        (TIME, R) float64 1.0 1.0 1.0 2.0 2.0 2.0 3.0 3.0 3.0 4.0 4.0 ...\n",
       "    c        (TIME, R) float64 10.0 10.0 10.0 20.0 20.0 20.0 30.0 30.0 30.0 ...\n",
       "    d        (TIME, R) float64 100.0 100.0 100.0 200.0 200.0 200.0 300.0 ...)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data.data_vars.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where 'a', 'b', 'c', 'd' are the name of the contained variables.  \n",
    "To access a particular coordinate,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<xarray.DataArray 'a' (TIME: 5, R: 3)>\n",
      "array([[ 0.11,  0.12,  0.13],\n",
      "       [ 0.21,  0.22,  0.23],\n",
      "       [ 0.31,  0.32,  0.33],\n",
      "       [ 0.41,  0.42,  0.43],\n",
      "       [ 0.51,  0.52,  0.53]])\n",
      "Coordinates:\n",
      "  * TIME     (TIME) float64 0.01 0.02 0.03 0.04 0.05\n",
      "  * R        (R) float64 0.1 0.2 0.3\n",
      "Attributes:\n",
      "    Unit: s\n"
     ]
    }
   ],
   "source": [
    "print(example_data.data_vars['a'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Access to attributes\n",
    "\n",
    "All the information in the header is stored as attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('NAME', 'example-2d'),\n",
       "             ('ShotNo', 6115),\n",
       "             ('Date', '1/2915:14'),\n",
       "             ('comments',\n",
       "              OrderedDict([('PHI', '3.5'),\n",
       "                           ('PHIunit', \"'portNO'\"),\n",
       "                           ('comments', \"'Be thickness = 7.5 micro m'\")]))])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data.attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other attributes that connects to coordinates or variables are stored as their attributes,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('Unit', 's')])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data['TIME'].attrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('Unit', 'V')])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data['c'].attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save\n",
    "\n",
    "To save any data as *eg* file format, please use eg.dump(data, filename) method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "new_filename = 'tmp_file.txt'\n",
    "eg.dump(example_data, new_filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.Dataset>\n",
       "Dimensions:  (R: 3, TIME: 5)\n",
       "Coordinates:\n",
       "  * TIME     (TIME) float64 0.01 0.02 0.03 0.04 0.05\n",
       "  * R        (R) float64 0.1 0.2 0.3\n",
       "Data variables:\n",
       "    a        (TIME, R) bool True True True True True True True True True ...\n",
       "    b        (TIME, R) bool True True True True True True True True True ...\n",
       "    c        (TIME, R) bool True True True True True True True True True ...\n",
       "    d        (TIME, R) bool True True True True True True True True True ..."
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To check the identity, I will reload this file again.\n",
    "example_data2 = eg.load(new_filename)\n",
    "example_data == example_data2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparation for save\n",
    "\n",
    "To save *eg* file format, we need some information other than just a values.  \n",
    "Simplest way is to make a xarray.DataSet file with an appropriate coordinates and variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_axis = np.linspace(0.0,1.0,10)\n",
    "y_axis = np.linspace(0.0,1.0,11)\n",
    "a = np.random.randn(len(x_axis), len(y_axis))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dump_data = xr.Dataset( data_vars = {'a':(('x','y'), a)}, \n",
    "                        coords={'x':x_axis, 'y': y_axis})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.Dataset>\n",
       "Dimensions:  (x: 10, y: 11)\n",
       "Coordinates:\n",
       "  * x        (x) float64 0.0 0.1111 0.2222 0.3333 0.4444 0.5556 0.6667 ...\n",
       "  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0\n",
       "Data variables:\n",
       "    a        (x, y) float64 -1.563 -0.01093 -1.396 -0.7539 -0.7721 0.03514 ..."
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dump_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# It is usually important to link unit data with coordinates and values.\n",
    "dump_data['a'].attrs['Unit'] = 'V'\n",
    "dump_data['x'].attrs['Unit'] = 'm'\n",
    "dump_data['y'].attrs['Unit'] = 's'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "eg.dump(dump_data, new_filename, NAME='temp_diag', ShotNo=6000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.EGdata>\n",
       "Dimensions:  (x: 10, y: 11)\n",
       "Coordinates:\n",
       "  * x        (x) float64 0.0 0.1111 0.2222 0.3333 0.4444 0.5556 0.6667 ...\n",
       "  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0\n",
       "Data variables:\n",
       "    a        (x, y) float64 -1.563 -0.01093 -1.396 -0.7539 -0.7721 0.03514 ...\n",
       "Attributes:\n",
       "    NAME: temp_diag\n",
       "    ShotNo: 6000\n",
       "    Date: 2016/12/1315:42:32\n",
       "    comments: OrderedDict()"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To check dumped data, I will reload this file again.\n",
    "example_data2 = eg.load(new_filename)\n",
    "example_data2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.DataArray 'a' (x: 10, y: 11)>\n",
       "array([[-1.562766  , -0.01092657, -1.395639  , -0.7539233 , -0.7721255 ,\n",
       "         0.03514394,  0.1257069 , -1.933911  ,  0.4982711 ,  1.35267   ,\n",
       "        -0.3206674 ],\n",
       "       [ 1.724778  ,  0.0549739 ,  0.9507024 , -3.324699  , -0.5052582 ,\n",
       "         0.1192041 ,  0.2828641 , -0.1978293 , -0.2525981 ,  0.6909911 ,\n",
       "        -1.759871  ],\n",
       "       [ 0.8069041 , -0.8631582 ,  0.7576942 , -0.6676276 ,  1.815476  ,\n",
       "        -0.2426941 , -0.6901838 , -0.3289584 ,  0.3290385 ,  0.7334659 ,\n",
       "        -1.906926  ],\n",
       "       [ 0.7620053 , -0.1759518 ,  0.9133975 ,  0.03131901, -0.0080099 ,\n",
       "        -1.308937  , -1.747693  ,  0.3069759 , -1.527917  , -2.203252  ,\n",
       "        -0.6787131 ],\n",
       "       [ 0.3426372 , -1.291217  , -1.04788   , -0.3390357 , -0.307747  ,\n",
       "        -0.1843734 , -0.6645383 ,  2.9428    ,  1.923766  ,  0.8632298 ,\n",
       "         0.9401448 ],\n",
       "       [-0.0060025 , -0.3408198 , -0.3991794 , -1.418897  ,  0.6306461 ,\n",
       "         0.5652755 , -0.8449487 , -0.3955003 ,  2.1751    ,  1.40073   ,\n",
       "        -0.2944968 ],\n",
       "       [-1.780778  , -0.1952267 , -1.785852  ,  0.07407973, -0.08320215,\n",
       "         0.1210752 , -0.265016  ,  1.642619  ,  0.115282  , -0.570825  ,\n",
       "         0.2768444 ],\n",
       "       [-0.4773258 , -1.604726  , -1.664188  , -0.8639125 ,  0.1392316 ,\n",
       "        -0.4860617 , -0.8911671 ,  0.2777801 , -0.4393171 ,  0.03017041,\n",
       "         0.1662167 ],\n",
       "       [ 0.2646961 , -0.8563455 ,  0.788785  , -2.419221  , -1.377161  ,\n",
       "         0.3313357 , -0.179437  ,  0.8109942 ,  0.6282819 ,  0.5650783 ,\n",
       "        -0.1933679 ],\n",
       "       [ 0.3411232 ,  0.2779414 , -0.02563158, -1.306824  ,  1.941216  ,\n",
       "         0.1323318 ,  0.616622  , -0.0562768 , -1.221019  ,  0.2388807 ,\n",
       "        -0.287631  ]])\n",
       "Coordinates:\n",
       "  * x        (x) float64 0.0 0.1111 0.2222 0.3333 0.4444 0.5556 0.6667 ...\n",
       "  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0\n",
       "Attributes:\n",
       "    Unit: V"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data2['a']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.DataArray 'y' (y: 11)>\n",
       "array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ])\n",
       "Coordinates:\n",
       "  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0\n",
       "Attributes:\n",
       "    Unit: s"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_data2['y']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remove files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.remove(new_filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}