Training model in Tensorflow C++ with LoadSavedModel

  c++, python, tensorflow

I try to replace the value of nodes in DQN with C++ by loading python-saved model.

In python tensorflow,
graph definition:

w_initializer, b_initializer = tf.compat.v1.random_normal_initializer(0., 0.3), tf.compat.v1.constant_initializer(0.1)
# ------------------ build evaluate_net ------------------
with tf.compat.v1.variable_scope('eval_net'):
    e1 = tf.compat.v1.layers.dense(s, 20, tf.compat.v1.nn.relu, kernel_initializer=w_initializer,
            bias_initializer=b_initializer, name='e1')
    q_eval = tf.compat.v1.layers.dense(e1, n_actions, kernel_initializer=w_initializer,
            bias_initializer=b_initializer, name='q')
    qq = tf.identity(q_eval, name='qq')

    # ------------------ build target_net ------------------
with tf.compat.v1.variable_scope('target_net'):
    t1 = tf.compat.v1.layers.dense(s_, 20, tf.compat.v1.nn.relu, kernel_initializer=w_initializer,
            bias_initializer=b_initializer, name='t1')
    q_next = tf.compat.v1.layers.dense(t1, n_actions, kernel_initializer=w_initializer,
            bias_initializer=b_initializer, name='t2')

replacing operations defined:

with tf.compat.v1.variable_scope('replace_net'):
    t_params = tf.compat.v1.get_collection(tf.compat.v1.GraphKeys.GLOBAL_VARIABLES, scope='target_net')
    e_params = tf.compat.v1.get_collection(tf.compat.v1.GraphKeys.GLOBAL_VARIABLES, scope='eval_net')
    idx = 0
    for t, e in zip(t_params, e_params):
        op = tf.compat.v1.assign(t,e, name='replace'+ str(idx))
        idx += 1

save model:

signature = tf.compat.v1.saved_model.signature_def_utils.predict_signature_def(
        inputs={'s': s}, outputs={'loss/loss': _loss})

builder = tf.compat.v1.saved_model.builder.SavedModelBuilder('model/mySave')

builder.add_meta_graph_and_variables(
    sess=sess,
    tags=[tf.compat.v1.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.compat.v1.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
        signature
        })

builder.save()

In C++,

    TF_CHECK_OK(tensorflow::LoadSavedModel(session_options,
                                           run_options,
                                           dir,
                                           // Refer to tag_constants. We just want to serve the model.
                                           {tensorflow::kSavedModelTagServe},
                                           &bundle));

    std::unique_ptr<tensorflow::Session>& sess2 = bundle.session;
    TF_CHECK_OK(sess2->Run({},{},{"replace_net/replace0"}, nullptr));
    TF_CHECK_OK(sess2->Run({},{},{"replace_net/replace1"}, nullptr));
    TF_CHECK_OK(sess2->Run({},{},{"replace_net/replace2"}, nullptr));
    TF_CHECK_OK(sess2->Run({},{},{"replace_net/replace3"}, nullptr));

After running c++ binary, there is no error message, but the node values in the target network are not replaced.
Can C++ train a model loaded in this way? or is there anything that I miss here?

Thanks in advance.

Source: Windows Questions C++

LEAVE A COMMENT