Tutorial

Let us start an interactive session (e.g. with python or ipython) and import PyTensor.

>>> from pytensor import *

Several of the symbols you will need to use are in the tensor subpackage of PyTensor. Let us import that subpackage under a handy name like at (the tutorials will frequently use this convention).

>>> import pytensor.tensor as pt

If that succeeded you are ready for the tutorial, otherwise check your installation (see Installing PyTensor).

Throughout the tutorial, bear in mind that there is a Glossary as well as index and modules links in the upper-right corner of each page to help you out.

Basics

• Baby Steps - Algebra
  • Adding two Scalars
  • Adding two Matrices
  • Exercise
• More Examples
  • Logistic Function
  • Computing More than one Thing at the Same Time
  • Setting a Default Value for an Argument
  • Using Shared Variables
  • Copying functions
  • Using Random Numbers
    • Brief Example
    • Seeding Streams
    • Sharing Streams Between Functions
    • Copying Random State Between PyTensor Graphs
    • Other Random Distributions
  • A Real Example: Logistic Regression
• Derivatives in PyTensor
  • Computing Gradients
  • Computing the Jacobian
  • Computing the Hessian
  • Jacobian times a Vector
    • R-operator
    • L-operator
  • Hessian times a Vector
  • Final Pointers
• Conditions
  • IfElse vs Switch
• Loop
  • Scan
  • Exercise
• How Shape Information is Handled by PyTensor
  • Specifying Exact Shape
  • Problems with Shape inference
• Broadcasting

Advanced

• Sparse
  • Compressed Sparse Format
    • Which format should I use?
  • Handling Sparse in PyTensor
    • To and Fro
    • Properties and Construction
    • Structured Operation
    • Gradient

Advanced configuration and debugging

• Configuration Settings and Compiling Modes
  • Configuration
  • Exercise
  • Default Modes
  • Default Linkers
  • Default Optimizers
  • Using DebugMode
• Printing/Drawing PyTensor graphs
  • Pretty Printing
  • Debug Print
  • Picture Printing of Graphs
  • Interactive Graph Visualization
• Debugging PyTensor: FAQ and Troubleshooting
  • Isolating the Problem/Testing PyTensor Compiler
  • Interpreting Error Messages
  • Using Test Values
  • “How do I print an intermediate value in a function?”
  • “How do I print a graph (before or after compilation)?”
  • “The function I compiled is too slow; what’s up?”
  • “How do I step through a compiled function?”
  • How to Use pdb
  • Dumping a Function to help debug
  • Breakpoint during PyTensor function execution
• Dealing with NaNs
  • Check Superparameters and Weight Initialization
  • Run in NanGuardMode, DebugMode, or MonitorMode
  • Numerical Stability
  • Algorithm Related
  • CUDA Specific Option
  • NaN Introduced by AllocEmpty
• Profiling PyTensor function

Further reading

• Loading and Saving
  • The Basics of Pickling
  • Short-Term Serialization
  • Robust Serialization
  • Long-Term Serialization
• Understanding Memory Aliasing for Speed and Correctness
  • The Memory Model: Two Spaces
  • Borrowing when Creating Shared Variables
  • Borrowing when Accessing Value of Shared Variables
    • Retrieving
    • Assigning
  • Borrowing when Constructing Function Objects
• Multi cores support in PyTensor
  • Convolution and Pooling
  • BLAS operation
  • Parallel element wise ops with OpenMP
• Frequently Asked Questions
  • How to update a subset of weights?