Home

Introduction

IJ-OpenCV provides a set of converter functions between OpenCV and ImageJ objects (Images, ROI...).
This allows to easily apply OpenCV functions on images opened in Fiji, by first converting them to an OpenCV matrix.

Documentation

Most (if not all) OpenCV functions are available for Java (and associated scripting languages).
Refer to the OpenCV API for the details.
IJ-OpenCV also has a API documentation that reference the different OpenCV<->ImageJ conversion classes.

NB: Make sure to use objects, functions from org.bytedeco.javacpp and not org.opencv.core which have different attributes and functions definitions which were not functional with IJ-OpenCV/Fiji.

Similarly there is some apparent redudancy for the function in the javacpp API.

ex : Transform exists in 3 different places :

• org.opencv.core.Core.transform This one takes org.opencv.core.Mat as input. It is currently challenging to have such object in Fiji.

• org.bytedeco.javacpp.opencv_core.cvTransform using CvArr as input, but even if you manage to convert your input as a CvArr it crashes Fiji. Apparently it is a deprecated version.

• org.bytedeco.javacpp.opencv_core.transform That's the one to use ! It takes only org.bytedeco.javacpp.opencv_core.Mat as input, which is the most approriate in Fiji/Jython

Matrices

The first thing to know about OpenCV is that most functions works with images as matrix object, while ImageJ uses ImagePlus or ImageProcessor object.
For this purpose, IJ-OpenCV provides some converters :

# Jython snippet for Image conversion between ImageJ and OpenCV (using Fiji script interpreter)
#@ ImagePlus imp 
from ijopencv.ij      import ImagePlusMatConverter as imp2mat
from ijopencv.opencv  import MatImagePlusConverter as mat2ip
from ij               import ImagePlus

# Convert ImagePlus (actually the contained ImageProcessor) to Matrix object
ImMat = imp2mat.toMat(imp.getProcessor())
print ImMat

# Convert Matrix object to ImageProcessor
NewIP  = mat2ip.toImageProcessor(ImMat)
NewImp = ImagePlus("Matrix converted back to ImagePlus", NewIP)
print NewImP

Such kind of converter is also available for PointRoi to opencv keypoints...

Creation and inspection

There are different ways to create matrices, although most of the time you would just get one by converting from an ImagePlus.
By printing a Mat object, you will get information about the dimensions of the matrix (width, height, channels).

To view the matrices values, the simplest is to convert the Mat object to a CvMat object and print that CvMat.
Be careful with priting large matrices, that can take some time.
For multi-channel matrices, each matrix element is a tuple of the channel values.

from org.bytedeco.javacpp.opencv_core	import Mat, CvMat, vconcat

# Create an identity Matrix of size (3x3) and type 8-bit
id = Mat().eye(3,3,0).asMat()
print id
idCvMat = CvMat(id) # Convert the Mat to a CvMat 
print idCvMat       # View the matrix values - time-consuming with large matrices !!

# Matrix of ones (3x3) 
One = Mat().ones(3,3,0).asMat()

# Matrix of zeros (3x3) 
Zero = Mat().zeros(3,3,0).asMat()

# Custom Matrices
# 1D-Matrix can be initialize from a list/array
# By default, the created matrix is a "row" matrix
# it can be converted to a column using t()
mat1 = Mat([1,2,3,4,5,6]) # 1D matrix as row
col1 = mat1.t().asMat() # as column

Note : Sometimes converting the Mat to a CvMat seems to impact the original Mat : passing the original Mat to an opencv function, would make Fiji crash.
If you observe this, you can make a copy of the Mat before converting it to a CvMat, using the Mat copy constructor (shallow-copy) or using the clone() function on the Mat object (deep copy, duplicating independant values).

# Convert to CvMat a copy of Id, to avoid further issues when passing Id to some opencv functions
# Note the copy is linked to the original matrix, i.e changing the copy would change the original one
# Use clone for a unlinked (so called deep) copy
copied = CvMat(Mat(id)) # Mat(id) will make a (shallow) copy of the id matrix, seems enough to fix the issue

The CvMat object offers also a convenient get(i,j) function to recover individual values.

from org.bytedeco.javacpp.opencv_core	import Mat, CvMat

# Create an identity Matrix of size (3x3) and type 8-bit
id = Mat().eye(3,3,0).asMat()

# Access matrix values via the CvMat object (complicated with the original Mat object)
# Note it uses i,j i.e row/column indexes, so inverted compared to the (x,y) coordinates in ImageJ (i.e i=y, j=x) 
i = 1 # row 1
j = 1 # column 1
idCvMat = CvMat(id)
print idCvMat.get(i, j) # should be 1 on the diagonal

Operations

Matrices can be reshaped, splitted and combine.
Remember though that usually the values are not copied, i.e changing the original matrix will impact the transformed matrix too, unless you used clone().

from org.bytedeco.javacpp.opencv_core	import Mat, CvMat, vconcat

# For 2D matrix one can reshape a 1D-matrix
# see https://www.javadoc.io/static/org.bytedeco.javacpp-presets/opencv/3.4.2-1.4.2/org/bytedeco/javacpp/opencv_core.Mat.html#reshape-int-int-
mat  = Mat([6,7,8,9,10,11]) # create a 1D Mat
nrows = 2
mat2D = mat.reshape(0, nrows) # reshape 1D to 2D Mat with 2 rows, the 1st argument (0) is to state that we keep the same number of channels
print CvMat(mat2D)

# You can also append rows together to a 2D matrix using vconcat
row1 = Mat([1,2,3])
row2 = Mat([4,5,6])
mat2Dbis = Mat()          # initialise output
vconcat(row1, row2, mat2Dbis) # output stored in mat2Dbis
print mat2Dbis
print CvMat(mat2Dbis)

# One can also concatenate columns using hconcat
# For that we need to transpose the row matrix to column using t()
col1 = row1.t().asMat() 
col2 = row2.t().asMat()

mat2Dbis = Mat()
hconcat(col1, col2, mat2Dbis) # overwrite mat2Dbis
print mat2Dbis
print CvMat(mat2Dbis)

# Matrices can have multiple channels
# Use split to get individual matrices for each channel

Again use the org.bytedeco.javacpp.opencv_core.Mat and not the org.opencv.core.Mat.

Scalar

In addition to Matrices, opencv allows to use Scalar objects.
A scalar is a 4 item element (v0, v1, v2, v3).
If v1=v2=v3=0 then the Scalar is real.

from org.bytedeco.javacpp.opencv_core	import Scalar

# Real scalar can be initiated with a float parameters
Number = Scalar(5.0)
Number = Scalar(float(5))
print Number

# Using an integer as parameter has a different meaning
Empty = Scalar(5) # This initiate an empty Scalar object of size 5
print Empty

# Alternatively one can set the other values of the Scalar
Complex = Scalar(1,2,3,4)
print Complex

Operations

It is possible to perform some operations between matrices, or between Scalar and matrices.

from org.bytedeco.javacpp.opencv_core	import Scalar, Mat, subtract

A = Mat([1,2,3,4,5])
B = Mat([1,2,-3,-4,0])

Number = Scalar(10.0)

## Number - B ( B-Number is also possible)
Expr = subtract(Number,B)
print CvMat(Expr.asMat())

## A - B
Out = Mat() 
subtract(A,B,Out)
print CvMat(Out)

Pointers

Sometimes some function signature expect some subclass of Pointer (IntPointer, FloatPointer from org.bytedeco.javacpp), which in the end can contains a single value or multiple values.
Usually you pass an uninitialized Pointer to the function, and the function will take care of allocating the right capacity to the Pointer.
If you know that the Pointer will hold a given number of values you can also specify the capacity.

Note that as indicated in the API doc, you always recover values using the get method, while the put method is used to populate a Pointer.

Let's take an example of a IntPointer that is holding an array of values, with the following dummy jython example (try it in the Fiji script editor).

from org.bytedeco.javacpp 	import IntPointer, DoublePointer
from array import array, zeros

## Creating some Pointers
# There are different constructors for pointers
someValues = IntPointer()  # create a uninitialized pointer, since we dont know how many element the function1 will return
function1(someValues)      # imagine this as an opencv function, which will populate the IntPointer

singleValuePointer = IntPointer(1)       # this is a pointer, initialised with a capacity of 1: it will contain only a single integer
pointerFromArray   = IntPointer([1,2,3]) # initialized with an array

## Recover the values from the IntPointer

# - either by index
value_i = someValues.get(i)

# - or writing the full pointer to an array of the identical size of the Pointer
nValues = someValues.capacity()        # number of elements in the pointer
arrayFromPointer = zeros("i", nValues) # make an empty integer array of the right size, used to dumped the values from the pointer
someValues.get(arrayFromPointer)       # dump the pointer to the array
print arrayFromPointer

## Putting some values in an existing Pointer
myPointer = IntPointer(3) # can hold 3 integers
myPointer.put([1,2,3])    # fill the pointer with the value from the array
myPointer.put(1,2,3)      # this works too like the previous signature, at least in jython
myPointer.put(5)          # this will assign the value 5 to the item at first index

# Assigning a value at a particular index should be possible
# however at least in jython, it does not work as expected
index = 2
value = 18
myPointer.put(index, value) # not working as expected, the values are completely disordered

Small notes about jython here, we usually use python list objects directly as equivalent of java arrays. However, lists are converted to read-only arrays when passed to functions so the function can't modify it.
Besides here, the get method really expects an integer array and will complain if it is passed a list.
Therefore we use the zeros object from the array module to create an empty integer java array of the same capacity than the pointer.