Custom gradient with complex exponential in tensorflow

rassi

As an exercise I am trying to build a custom operator in Tensorflow, and checking the gradient against Tensorflow's autodiff of the same forward operation composed of Tensorflow API operations. However, the gradient of my custom operator is incorrect. It seems like my complex analysis is not correct and needs some brushing up.

import tensorflow as tf

shape = (1, 16)
dtype = tf.complex64

x = tf.cast(tf.complex(tf.random.normal(shape), tf.random.normal(shape)), dtype=dtype)

def fun(x):
    phi = x * tf.math.conj(x)
    e = tf.exp(1j * phi)
    return e

def d_fun(x):
    d_phi = x + tf.math.conj(x)
    phi = x * tf.math.conj(x)
    d_e = 1j * d_phi * tf.exp(1j * phi)
    return d_e

@tf.custom_gradient
def tf_custom(x):    
    e = fun(x)
    def grad(dy):
        d_e = d_fun(x)
        return dy * d_e
    return e, grad

with tf.GradientTape() as g:
    g.watch(x)
    res = fun(x)
    
dy_dx = g.gradient(res, x)

with tf.GradientTape() as g:
    g.watch(x)
    res2 = tf_custom(x)
    
dy_dx2 = g.gradient(res2, x)

print(tf.reduce_sum(tf.abs(res - res2)).numpy())
print(tf.reduce_sum(tf.abs(dy_dx - dy_dx2)).numpy())
elbe

TensorFlow 2 does not directly computes the derivative of a function of complex variables. It seems that it computes the derivative of a function of a complex variable as the function of the real part and the imaginary part, using Wirtinger calculus. You can also find an explanation here.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

How to provide custom gradient in TensorFlow

Defining custom gradient as a class method in Tensorflow

TensorFlow exponential moving average

Matrix with complex exponential elements in Python

Using tf.custom_gradient in tensorflow r1.8

How to assign custom gradient to TensorFlow op with multiple inputs

When use custom layer in Tensorflow 2.0, the gradient returns None

Complex shape with rainbow gradient

Parse complex numbers with exponential notation Java

How do I calculate the exponential of a complex matrix?

Plotting of complex exponential function using Matlab

Tensorflow gradient with respect to matrix

Gradient computations in Tensorflow 2.0

Determinism in tensorflow gradient updates?

What is gradient repacking in Tensorflow?

stopping gradient optimizer in TensorFlow

Tensorflow exploding gradient

Gradient Descent optimizer TensorFlow

Tensorflow: Custom Layer/Gradient result in OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed

TensorFlow Model takes exponential time on large dataSet

Nonlinear Exponential Regression with Tensorflow.js

Linear gradient with custom size

Gradient Color to Custom View

tensorflow operation for complex numbers

Fastest way to calculate exponential [exp()] function of large complex array in Python

Python vs R/Matlab implementation of exponential of a complex number

How to expand one exponential complex equation to two trigonometric ones in sympy?

Background image and complex gradient on Chrome mobile

Tensorflow gradient returns nan or Inf