Interpolation with NaN's, no overshoot and robustness

[uniomni]

Implement and test 2D interpolation where some numbers are NaN (IEEE Not-a-Number).
Apply this in the context of ASCII grids where NODATA of -9999 gets replaced by NaN.

[uniomni]

This should also be written to guarantee that interpolated values remain within the extrema of the source data.
A test for that, which is currently failing (slightly) is available but currently disabled in test_engine.py.

When this issue is being addressed it'll be a good idea to change the tolerances in test_interpolation_tsunami as indicated in the FIXME.

[uniomni]

Also observed this traceback (for the Kampung Bali data set). It was not reproducible, but hints at the need for more robust interpolation.

Traceback:
File "/home/nielso/dev/riab_env/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response

1.                     response = callback(request, _callback_args, *_callback_kwargs)
   

   File "/home/nielso/dev/riab_env/lib/python2.6/site-packages/django/views/decorators/csrf.py" in wrapped_view

2.     resp = view_func(_args, *_kwargs)
   

   File "/home/nielso/dev/riab_env/lib/python2.6/site-packages/django/views/decorators/csrf.py" in wrapped_view

3.     return view_func(_args, *_kwargs)
   

   File "/home/nielso/dev/riab/impact/views.py" in calculate

4.                                    impact_function=impact_function)
   

   File "/home/nielso/dev/riab/impact/engine/core.py" in calculate_impact

5. F = impact_function.run(layers)
   

   File "/home/nielso/dev/riab/impact/plugins/earthquake/hot_building_impact_model.py" in run

6.     H = H.interpolate(E)
   

   File "/home/nielso/dev/riab/impact/storage/raster.py" in interpolate

7.         return interpolate_raster_vector(self, X, name)
   

   File "/home/nielso/dev/riab/impact/engine/interpolation.py" in interpolate_raster_vector

8. f = raster_spline(longitudes, latitudes, A)
   

   File "/home/nielso/dev/riab/impact/engine/interpolation.py" in raster_spline

9. F = RectBivariateSpline(latitudes, longitudes, A)
   

   File "/usr/lib/python2.6/dist-packages/scipy/interpolate/fitpack2.py" in init

10.                                                 kx,ky,s)
    

Exception Type: error at /api/v1/calculate/
Exception Value: (my>ky) failed for hidden my: regrid_smth:my=2

[uniomni]

Write unit test using HOT data to reproduce this problem.
However, this can't really be done until we can enter new data seamlessly: #104

[uniomni]

Saw the same error today with the full HOT OSM dataset.

[uniomni]

Wrapped underlying interpolation function in try-except to provide a better error message:
f467032

[uniomni]

Updated code everywhere to use NaN internally for missing values:
73f097b

All 84 tests pass.

[uniomni]

I think we can remove the dependency on scipy for now, but not game to just do it without having a fresh installation to test it on.

[uniomni]

Could close this now, except for a few minor outstanding issues:
1: Retirement of scipy
2: Start actively using NaNs for missing values everywhere
3: Should consider using the piecewise constant mode - maybe control this from a Risiko configuration set?

These have been moved to three new tickets: #165, #166, #167