Video Tutorial

Lazy vs Eager

Understand the difference between eager and lazy graph tracing and transpilation.

⚠️ If you are running this notebook in Colab, you will have to install Ivy and some dependencies manually. You can do so by running the cell below ⬇️

If you want to run the notebook locally but don’t have Ivy installed just yet, you can check out the Get Started section of the docs.

!pip install ivy

Requirement already satisfied: ivy in /workspaces/ivy (0.0.4.0)
Requirement already satisfied: numpy in /opt/fw/mxnet (from ivy) (1.26.1)
Requirement already satisfied: einops in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (0.7.0)
Requirement already satisfied: psutil in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (5.9.6)
Requirement already satisfied: termcolor in /opt/fw/tensorflow (from ivy) (2.3.0)
Requirement already satisfied: colorama in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (0.4.6)
Requirement already satisfied: packaging in /opt/fw/tensorflow (from ivy) (23.2)
Requirement already satisfied: nvidia-ml-py in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (12.535.108)
Requirement already satisfied: diskcache in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (5.6.3)
Requirement already satisfied: google-auth in /opt/fw/tensorflow (from ivy) (2.23.3)
Requirement already satisfied: urllib3<2.0 in /root/.local/lib/python3.10/site-packages (from ivy) (1.26.18)
Requirement already satisfied: requests in /opt/fw/mxnet (from ivy) (2.31.0)
Requirement already satisfied: pyvis in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (0.3.2)
Requirement already satisfied: dill in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (0.3.7)
Requirement already satisfied: astunparse in /opt/fw/tensorflow (from ivy) (1.6.3)
Requirement already satisfied: ml-dtypes in /opt/fw/tensorflow (from ivy) (0.2.0)
Requirement already satisfied: cloudpickle in /opt/fw/tensorflow (from ivy) (3.0.0)
Requirement already satisfied: gast in /opt/fw/tensorflow (from ivy) (0.5.4)
Requirement already satisfied: tqdm in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ivy) (4.66.1)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /opt/fw/tensorflow (from astunparse->ivy) (0.41.3)
Requirement already satisfied: six<2.0,>=1.6.1 in /opt/fw/tensorflow (from astunparse->ivy) (1.16.0)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/fw/tensorflow (from google-auth->ivy) (5.3.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/fw/tensorflow (from google-auth->ivy) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/fw/tensorflow (from google-auth->ivy) (4.9)
Requirement already satisfied: ipython>=5.3.0 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from pyvis->ivy) (8.17.1)
Requirement already satisfied: jinja2>=2.9.6 in /opt/fw/torch (from pyvis->ivy) (3.1.2)
Requirement already satisfied: jsonpickle>=1.4.1 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from pyvis->ivy) (3.0.2)
Requirement already satisfied: networkx>=1.11 in /opt/fw/torch (from pyvis->ivy) (3.2.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/fw/mxnet (from requests->ivy) (3.3.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/fw/mxnet (from requests->ivy) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/fw/mxnet (from requests->ivy) (2023.7.22)
Requirement already satisfied: decorator in /opt/fw/tensorflow (from ipython>=5.3.0->pyvis->ivy) (5.1.1)
Requirement already satisfied: jedi>=0.16 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (0.19.1)
Requirement already satisfied: matplotlib-inline in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (0.1.6)
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (3.0.39)
Requirement already satisfied: pygments>=2.4.0 in /opt/fw/jax (from ipython>=5.3.0->pyvis->ivy) (2.16.1)
Requirement already satisfied: stack-data in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (0.6.3)
Requirement already satisfied: traitlets>=5 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (5.13.0)
Requirement already satisfied: exceptiongroup in /opt/fw/paddle (from ipython>=5.3.0->pyvis->ivy) (1.1.3)
Requirement already satisfied: pexpect>4.3 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from ipython>=5.3.0->pyvis->ivy) (4.8.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/fw/tensorflow (from jinja2>=2.9.6->pyvis->ivy) (2.1.3)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /opt/fw/tensorflow (from pyasn1-modules>=0.2.1->google-auth->ivy) (0.5.0)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from jedi>=0.16->ipython>=5.3.0->pyvis->ivy) (0.8.3)
Requirement already satisfied: ptyprocess>=0.5 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from pexpect>4.3->ipython>=5.3.0->pyvis->ivy) (0.7.0)
Requirement already satisfied: wcwidth in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=5.3.0->pyvis->ivy) (0.2.9)
Requirement already satisfied: executing>=1.2.0 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from stack-data->ipython>=5.3.0->pyvis->ivy) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from stack-data->ipython>=5.3.0->pyvis->ivy) (2.4.1)
Requirement already satisfied: pure-eval in /opt/miniconda/envs/multienv/lib/python3.10/site-packages (from stack-data->ipython>=5.3.0->pyvis->ivy) (0.2.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

ivy.unify, ivy.trace_graph and ivy.transpile can all be performed either eagerly or lazily. All previous examples have been performed lazily, which means that the unification, tracing, or transpilation process actually occurs during the first call of the returned function.

This is because all three of these processes depend on function tracing, which requires function arguments to use for the tracing. Alternatively, the arguments can be provided during the ivy.unify, ivy.trace_graph or ivy.transpile call itself, in which case the process is performed eagerly. We show some simple examples for each case below.

Unify

Consider again this simple torch function:

import ivy
import torch

def normalize(x):
    mean = torch.mean(x)
    std = torch.std(x)
    return torch.div(torch.sub(x, mean), std)

And let’s also create the dummy numpy arrays as before:

# import NumPy
import numpy as np
np.random.seed(0)

# create random numpy array for testing
x = np.random.uniform(size=10)

Let’s assume that our target framework is tensorflow:

import tensorflow as tf
ivy.set_backend("tensorflow")

x = tf.constant(x)

2023-11-01 06:53:37.201733: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/workspaces/ivy/ivy/utils/exceptions.py:390: UserWarning: The current backend: 'tensorflow' does not support inplace updates natively. Ivy would quietly create new arrays when using inplace updates with this backend, leading to memory overhead (same applies for views). If you want to control your memory management, consider doing ivy.set_inplace_mode('strict') which should raise an error whenever an inplace update is attempted with this backend.
  warnings.warn(

In the example below, the function is unified lazily, which means the first function call will execute slowly, as this is when the unification process actually occurs.

norm = ivy.unify(normalize, source="torch")
norm(x) # slow, lazy unification
norm(x) # fast, unified on previous call

WARNING:root:To preserve the tracer and transpiler caches across multiple machines, ensure that the relative path of your projects from the .ivy folder is consistent across all machines. You can do this by adding .ivy to your home folder and placing all projects in the same place relative to the home folder on all machines.

ivy.array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])

However, in the following example the unification occurs eagerly, and both function calls will be fast:

ivy.set_backend("tensorflow")
norm = ivy.unify(normalize, source="torch", args=(x,))
norm(x) # fast, unified at ivy.unify
norm(x) # fast, unified at ivy.unify

ivy.array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])

Trace

The same is true for tracing. In the example below, the function is traced lazily, which means the first function call will execute slowly, as this is when the tracing process actually occurs.

norm_trace = ivy.trace_graph(norm)
norm_trace(x) # slow, lazy graph tracing
norm_trace(x) # fast, traced on previous call

<tf.Tensor: shape=(10,), dtype=float64, numpy=
array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])>

However, in the following example the tracing occurs eagerly, and both function calls will be fast:

norm_tracing = ivy.trace_graph(norm, args=(x,))
norm_tracing(x) # fast, traced at ivy.trace_graph
norm_tracing(x) # fast, traced at ivy.trace_graph

<tf.Tensor: shape=(10,), dtype=float64, numpy=
array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])>

Transpile

The same is true for transpiling. In the example below, the function is transpiled lazily, which means the first function call will execute slowly, as this is when the transpilation process actually occurs.

norm_trans = ivy.transpile(normalize, source="torch", to="tensorflow")
norm_trans(x) # slow, lazy transpilation
norm_trans(x) # fast, transpiled on previous call

<tf.Tensor: shape=(10,), dtype=float64, numpy=
array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])>

However, in the following example the transpilation occurs eagerly, and both function calls will be fast:

norm_trans = ivy.transpile(normalize, source="torch", to="tensorflow", args=(x,))
norm_trans(x) # fast, transpiled at ivy.transpile
norm_trans(x) # fast, transpiled at ivy.transpile

<tf.Tensor: shape=(10,), dtype=float64, numpy=
array([-0.34431235,  0.51129461, -0.06686894, -0.36452447, -0.98795534,
        0.15493582, -0.91630631,  1.41939619,  1.78909753, -1.19475674])>

Round Up

That’s it, you now know the difference between lazy vs eager execution for ivy.unify, ivy.trace_graph and ivy.transpile! Next, we’ll be exploring how these three functions can all be called as function decorators!