nanguardmode
#
Guide#
The NanGuardMode
aims to prevent the model from outputting NaNs or Infs. It has
a number of self-checks, which can help to find out which Apply
node is
generating those incorrect outputs. It provides automatic detection of three types
of abnormal values: NaNs, Infs, and abnormally big values.
NanGuardMode
can be used as follows:
import numpy as np
import pytensor
import pytensor.tensor as at
from pytensor.compile.nanguardmode import NanGuardMode
x = at.matrix()
w = pytensor.shared(np.random.standard_normal((5, 7)).astype(pytensor.config.floatX))
y = at.dot(x, w)
fun = pytensor.function(
[x], y,
mode=NanGuardMode(nan_is_error=True, inf_is_error=True, big_is_error=True)
)
While using the PyTensor function fun
, it will monitor the values of each
input and output variable of each node. When abnormal values are
detected, it raises an error to indicate which node yields the NaNs. For
example, if we pass the following values to fun
:
infa = np.tile((np.asarray(100.) ** 1000000).astype(pytensor.config.floatX), (3, 5))
fun(infa)
It will raise an AssertionError
indicating that Inf value is detected while
executing the function.
You can also set the three parameters in NanGuardMode
to indicate which
kind of abnormal values to monitor. nan_is_error
and inf_is_error
has
no default values, so they need to be set explicitly, but big_is_error
is
set to be True
by default.
Note
NanGuardMode
significantly slows down computations; only
enable as needed.
Reference#
- class pytensor.compile.nanguardmode.NanGuardMode(nan_is_error=None, inf_is_error=None, big_is_error=None, optimizer='default', linker=None, db=None)[source]#
A PyTensor compilation Mode that makes the compiled function automatically detect NaNs and Infs and detect an error if they occur.
- Parameters:
nan_is_error (bool) – If True, raise an error anytime a NaN is encountered.
inf_is_error (bool) – If True, raise an error anytime an Inf is encountered. Note that some pylearn2 modules currently use np.inf as a default value (e.g. mlp.max_pool) and these will cause an error if inf_is_error is True.
big_is_error (bool) – If True, raise an error when a value greater than 1e10 is encountered.
Notes
We ignore the linker parameter