Question - What are intermediate tensors? Do sessions have lifetime?
Answer -
The intermediate tensors are tensors that are neither inputs nor outputs of the Session.run() call, but are in the path leading from the inputs to the outputs; they will be freed at or before the end of the call.
Sessions can own resources, few classes like tf.Variable, tf.QueueBase, and tf.ReaderBase, and they use a significant amount of memory. These resources (and the associated memory) are released when the session is closed, by calling tf.Session.close.