c++ - Steps for creating an optimizer on TensorFlow -

i'm trying implement new optimizer consist in big part of gradient descent method (which means want perform few gradient descent steps, different operations on output , again). unfortunately, found 2 pieces of information;

1. you can't perform given amount of steps optimizers. wrong that? because seem logical option add.
2. given 1 true, need code optimizer using c++ kernel , losing powerful possibilities of tensorflow (like computing gradients).

if both of them true 2 makes no sense me, , i'm trying figure out what's correct way build new optimizer (the algorithm , else crystal clear).

thanks lot

1. i not 100% sure that, think right. don't see benefits of adding such option tensorflow. optimizers based on gd know work this:

   for in num_of_epochs:     g = gradient_of_loss()     some_storage = f(previous_storage, func(g))     params = func2(previous_params, some_storage) 

if need perform couple of optimization steps, can in loop:

train_op = optimizer.minimize(loss) in range(10):     sess.run(train_op) 

i don't think parameter multitrain_op = optimizer.minimize(loss, steps) needed in implementation of current optimizers , final user can simulate code before, reason not added.

2. let's take @ tf implementation of example optimizer, adam: python code, c++ code.

the "gradient handling" part processed entirely inheriting optimizer.optimizer in python code. python code define types of storage hold moving window averages, square of gradients, etc, , executes c++ code passing calculated gradient.

the c++ code has 4 lines, updating stored averages , parameters.

so question "how build optimizer": 1 . define need store between calculations of gradient 2. inherit optimizer.optimizer 3. implement updating variables in c++.