Disclaimer: I use the Keras interface from TensorFlow 2.0.0-rc1, so I’m not sure if the content applies to the original Keras library or other versions of TensorFlow. And note that TensorFlow 2.0.0-rc1 uses eager-execution mode by default.

Let’s say we have a metric object m = tf.keras.metrics.Mean(). There are two ways to add a number, for example 10, to the metric m: the first way is to call m.update_state(10), and the other way is to simply call m(10). When we want to know the current mean value, we then use m.result().

I was wondering what the difference is between m(10) and m.update_state(10), because I could not find an explanation in the documentation of tf.keras.metrics.Mean (see here). The special member function of a Python class that allows us to use something like m(10) is __call__(). But in that documentation, there is no mention of __call__().

And why did I care? Because in the official tutorial here, the tutorial uses the second way to add a value to metrics without explanation.

If we use an interactive environment, like an IPython console or a Jupyter Notebook, we can find some information about the two methods through help(tf.keras.metrics.Mean):

• tf.keras.metrics.Mean.update_state:

  Accumulates statistics for computing the reduction metric. For example, if values is [1, 3, 5, 7] and reduction=SUM_OVER_BATCH_SIZE, then the value of result() is 4. If the sample\_weight is specified as [1, 1, 0, 0] then the value of result() would be 2.

• tf.keras.metrics.Mean.__call__: > Accumulates statistics and then computes metric result value.

So apparently, update_state simply updates the sequence of numbers but does not do the calculation, which in this example means averaging the numbers. And __call__ updates the sequence and also does the calculation.

From this point of view, I would guess that calling __call__, or m(10) in our example, is less efficient. After all, we do not really need to know the current mean value every time we add a number to the sequence.

We can actually see this in the source code. As indicated by help(tf.keras.metrics.Mean), the method __call__ is inherited from tf.keras.metrics.Metric. Its source code can be viewed here:

1
 def __call__(self, *args, **kwargs):
2
    """Accumulates statistics and then computes metric result value.
3
    Args:
4
      *args:
5
      **kwargs: A mini-batch of inputs to the Metric,
6
        passed on to `update_state()`.
7
    Returns:
8
      The metric value tensor.
9
    """
10

11
    def replica_local_fn(*args, **kwargs):
12
      """Updates the state of the metric in a replica-local context."""
13
      update_op = self.update_state(*args, **kwargs)  # pylint: disable=not-callable
14
      with ops.control_dependencies([update_op]):
15
        result_t = self.result()  # pylint: disable=not-callable
16

17
        # We are adding the metric object as metadata on the result tensor.
18
        # This is required when we want to use a metric with `add_metric` API on
19
        # a Model/Layer in graph mode. This metric instance will later be used
20
        # to reset variable state after each epoch of training.
21
        # Example:
22
        #   model = Model()
23
        #   mean = Mean()
24
        #   model.add_metric(mean(values), name='mean')
25
        result_t._metric_obj = self  # pylint: disable=protected-access
26
        return result_t
27

28
    from tensorflow.python.keras.distribute import distributed_training_utils  # pylint:disable=g-import-not-at-top
29
    return distributed_training_utils.call_replica_local_fn(
30
        replica_local_fn, *args, **kwargs)

Removing the docstrings, we can see that __call__ is actually calling update_state(...) and then result().