Getting started tutorial

This subpage will walk you through you the basics of usage iris package. From it you will learn how to:

  • Perform an IRISPipeline inference call.

  • Configure IRISPipeline environment to modify error handling and return behaviour.

  • Visualize IRISPipeline intermediate results.

1. Running the IRISPipeline inference

Load IR image with opencv-python package.

import cv2
img_pixels = cv2.imread("./sample_ir_image.png", cv2.IMREAD_GRAYSCALE)

Create IRISPipeline object.

import iris
iris_pipeline = iris.IRISPipeline()

Run IRISPipeline inference. There are several methods that wraps IRISPipeline inference call. Each one of them leads to the same source code being called. Possible options are:

  1. Using __call__ operator

  2. Using run method

  3. Using estimate method

# Options for the `eye_side` argument are: ["left", "right"]
output = iris_pipeline(img_data=img_pixels, eye_side="right")
output = iris_pipeline.run(img_data=img_pixels, eye_side="right")
output = iris_pipeline.estimate(img_data=img_pixels, eye_side="right")

The output of IRISPipeline is a dictionary with following keys: ["error", "iris_template", "metadata"].

The error value contains information about potential exceptions being raised during performing inference. The IRISPipeline implements the concept of a state machine. Therefore, error handling is done through setting an appropriate variable and returning it to user for inference success status verification.

If output["error"] value is None, IRISPipeline finished inference call without any exception being raised. If the IRISPipeline raised some exception when performing an inference, output["error"] value will be a dict, containing three keys: ["error_type", "message", "traceback"]. An example of output["error"] with an error looks like:

{
    'error_type': 'TypeError',
    'message': "run() got an unexpected keyword argument 'segmentation_map2'",
    'traceback': 'Very long exception traceback'
}

The iris_template value contains generated by the IRISPipeline iris code for an iris texture visible in the input image. The output["iris_template"] value is a IrisTemplate object containing two fields: ["iris_codes: List[np.ndarray]", "mask_codes: List[np.ndarray]"].

Each code available in output["iris_template"] dictionary is a numpy.ndarray of shape (16, 256, 2). The length of arrays containing iris codes and mask codes is determined by IRISPipeline filter bank parameters. The iris/mask code shape’s dimensions correspond to the following (iris_code_height, iris_code_width, 2). Values iris_code_height and iris_code_width are determined by ProbeSchema’s definition for ConvFilterBank object and num_filters is determined by number of filters specified for ConvFilterBank object. The last 2 value of the iris/mask code dimension corresponds to the real and imaginary parts of each complex filter response.

NOTE: More about how to specify those parameters and configuring custom IRISPipeline can be found in the Configuring custom pipeline tutorial.

The metadata value contains additional information that may be useful for further processing or quality analysis. Metadata information contain in this dictionary presents as follow.

Configuring pipelines error handling and which intermediate results are returned can be achieved through Environment parameter set when the IRISPipeline is instantiate. To understand more about that subject please follow to the notebook’s next section - 2. Configuring ``IRISPipeline`` environment.

1. Configuring IRISPipeline environment

Before diving deeper into how exactly one can modify error handling or return behaviour let’s first investigate what are IRISPipeline instantiation parameters. The IRISPipeline’s __init__ method presents as follow.

def __init__(
    self,
    config: Union[Dict[str, Any], Optional[str]] = None,
    env: Environment = Environment(
        pipeline_output_builder=build_orb_output,
        error_manager=store_error_manager,
        call_trace_initialiser=PipelineCallTraceStorage.initialise,
    ),
) -> None:

There are two parameters we can specify:

  1. config: Union[Dict[str, Any], Optional[str]] - refers to IRISPipeline configuration that specified what nodes pipeline has and how all of them are orchestrated/connected into pipeline graph. How to configure pipeline graph is a subject of the tutorial Configuring custom pipeline tutorial.

  2. env: Environment - refers to IRISPipeline environment that manages error handling and return behaviour of the IRISPipeline.

From that we can see that in order to modify error handling or return behaviour we have to introduce our own Environment object when creating the IRISPipeline object. The Environment object is defined as follow.

class Environment(ImmutableModel):
    call_trace_initialiser: Callable[[Dict[str, Algorithm], List[PipelineNode]], PipelineCallTraceStorage]
    pipeline_output_builder: Callable[[PipelineCallTraceStorage], Any]
    error_manager: Callable[[PipelineCallTraceStorage, Exception], None]
    disabled_qa: List[type] = []

Parameters of the Environment class are responsible for following:

  • call_trace_initialiser - is responsible for initialising the PipelineCallTraceStorage instance in the pipeline.

  • pipeline_output_builder - is responsible for building the pipeline output from the call_trace, which kept all intermediary results so far.

  • error_manager - is responsible for the pipeline’s behaviour in case of an exception

  • disabled_qa - stores a list of Algorithm and/or Callback types to be disabled when performing an inference call.

Apart from Environment that IRISPipeline has setup by default, it also provides additional Environment that user can set. Environment is defined within IRISPipeline and is called DEBUGGING_ENVIRONMENT. As name suggest, this Environment is useful whenever user wants to debug pipeline by getting more insights on information flowing through the system. The DEBUGGING_ENVIRONMENT is defined as follow.

DEBUGGING_ENVIRONMENT = Environment(
    pipeline_output_builder=build_debugging_output,
    error_manager=store_error_manager,
    disabled_qa=[
        iris.nodes.validators.object_validators.Pupil2IrisPropertyValidator,
        iris.nodes.validators.object_validators.OffgazeValidator,
        iris.nodes.validators.object_validators.OcclusionValidator,
        iris.nodes.validators.object_validators.IsPupilInsideIrisValidator,
        iris.nodes.validators.object_validators.IsMaskTooSmallValidator,
        iris.nodes.validators.cross_object_validators.EyeCentersInsideImageValidator,
        iris.nodes.validators.cross_object_validators.ExtrapolatedPolygonsInsideImageValidator,
    ],
    call_trace_initialiser=PipelineCallTraceStorage.initialise,
)

Let’s test it and see the output of the IRISPipeline with DEBUGGING_ENVIRONMENT set.

iris_pipeline = iris.IRISPipeline(env=iris.IRISPipeline.DEBUGGING_ENVIRONMENT)
output = iris_pipeline(img_data=img_pixels, eye_side="right")

In the same manner, we can investigate what has been returned from DEBUGGING_ENVIRONMENT and we can see that more intermediate result are available for us in the output dictionary.

User can also create and introduce to IRISPipeline their own Environment variables as far as they fulfill Environment class variables typings. For examples, please checkout iris.orchestration.output_builders module.

3. Visualizing intermediate results

The iris package provides also a useful module for plotting intermediate results - iris.visualisation. The main class of the module - IRISVisualizer - provides a bunch of plot functions that given appropriate intermediate result creates a ready to display Canvas. Definition of the Canvas type looks like follow.

Canvas = Tuple[matplotlib.figure.Figure, Union[matplotlib.axes._axes.Axes, np.ndarray]]

In order to utilize iris package visualisation mechanisms, we have start with creating the IRISVisualizer class.

iris_visualizer = iris.visualisation.IRISVisualizer()

Having that done, we can use it’s method by either providing iris package specific dataclasses or their serialized versions.

NOTE: Available by default IRISPipeline’s Environment return serialized version of iris dataclasses objects. That behaviour can be changed by creating and specifying as the IRISPipeline parameter your own custom Environment class object (see Section 2 for more details how to do that).

Below you can find a bunch of exemplary iris_visualizer plotting methods calls.

import matplotlib.pyplot as plt

canvas = iris_visualizer.plot_ir_image(iris.IRImage(img_data=img_pixels, eye_side="right"))
plt.show()

canvas = iris_visualizer.plot_iris_template(output["iris_template"])
plt.show()

List of all available IRISVisualizer methods can be found in the iris package documentation..

Thank you for making it to the end of this tutorial!