Handling exceptions

Data processing pipelines are typically composed of multiple nuts and process a stream of data. If one of the nuts within the pipeline fails the data stream breaks and processing stops.

Sometimes a more graceful handling of errors is needed and nuts-flow provides the Try nut for this purpose.

Try

Try(func, default) can wrap any nut function (but not other types of nuts such as processors) and handle exceptions raised by the wrapped nut. In the following example a nut Div is defined, which computes 10/x and is applied to a sequence of numbers:

>>> from nutsflow import *
>>> Div = nut_function(lambda x : 10/x)
>>> [1, 5, 10] >> Div() >> Collect()
[10, 2, 1]

As it is this pipeline will break and not collect any results if any of the input elements is zero:

>>> [1, 0, 10] >> Div() >> Collect()
Traceback (most recent call last):
...
ZeroDivisionError: integer division or modulo by zero

Wrapping the Div function within a Try allows the pipeline to ignore the input element that causes Div to fail. The problematic element and the error message are printed to standard out but the pipeline does not break and collects all other elements:

>>> [1, 0, 10] >> Try(Div(), 'STDOUT') >> Collect()
ERROR: 0 : integer division or modulo by zero
[10, 1]

Try allows defining a default value to be returned if the wrapped function fails. In this case no error is printed the offending input element is replaced by the provided default value:

>>> [1, 0, 10] >> Try(Div(), -1) >> Collect()
[10, -1, 1]

This kind of exception handling can be performed for nut functions or plain Python functions (user-defined or built-in). Here an example where Python’s logarithm function is wrapped and zero values are ignored:

>>> from math import log
>>> [1, 0, 10] >> Try(log, default='STDOUT') >> Collect()
ERROR: 0 : math domain error
[0.0, 2.302585092994046]

The default parameter can also be a function that takes the offending input element x and the exception e as parameters. This allows to replace offending inputs depending on the input value or the types of exception raised. In the following example negative input elements are replaced by their absolute value and zero is replaced by None.

>>> if_invalid = lambda x, e: -x if x < 0 else None
>>> [1, 0, -1, 10] >> Try(log, if_invalid) >> Collect()
[0.0, None, 1, 2.302585092994046]

As a last example, invalid inputs are replaced by the exception they cause:

>>> if_invalid = lambda x, e: e
>>> [1, -1, 10] >> Try(log, if_invalid) >> Collect()
[0.0, ValueError('math domain error',), 2.302585092994046]

The default value for Try(x,default) is default='STDERR', which ignores all elements that raise exceptions and prints error message to stderr. For default='IGNORE', offending inputs are ignored and no error messages are printed.