fix(query/control): add all of the statistics from flux statistics instead of only metadata (#23375)

This changes the statistics merging to use the `Add` method on the
entire statistics struct instead of only the metadata. The purpose of
this is that we want to utilize the existing function for merging the
statistics so that additional properties can be added without modifying
the query controller.

At the same time, there are certain properties that are computed in the
controller and we want to ensure they aren't double counted if flux
starts computing these itself. So we blacklist certain attributes that
we compute so that if flux is modified to return these values, we just
ignore them until we change our own code to use those values.

In immediate terms, we're changing to use the `Add` method so that we
can add profiles to the statistics and have those profiles propagate to
the query controller. This property doesn't exist yet so we can't add it
and we don't want to add it after flux is modified because it could
break the operator profile.

But, we also don't want to only use `Add` because we want to move the
properties such as `TotalAllocated` and `TotalDuration` into flux itself
and to remove the controller. But, the controller needs to be compatible
with whatever changes we make to flux so that there's no circumstance
where functionality stops working.
pull/23384/head
Jonathan A. Sternberg 2022-05-26 07:28:18 -07:00 committed by GitHub
parent afd0b99bdb
commit 2e9e174c24
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 20 additions and 2 deletions

View File

@ -714,8 +714,7 @@ func (q *Query) Done() {
q.err = q.exec.Err()
}
// Merge the metadata from the program into the controller stats.
stats := q.exec.Statistics()
q.stats.Metadata = stats.Metadata
q.mergeQueryStats(q.exec.Statistics())
}
// Retrieve the runtime errors that have been accumulated.
@ -749,6 +748,25 @@ func (q *Query) Done() {
<-q.doneCh
}
func (q *Query) mergeQueryStats(other flux.Statistics) {
// Clear out the durations and the statistics that we calculate.
// We don't want to double count things.
other.TotalDuration = 0
other.CompileDuration = 0
other.QueueDuration = 0
other.PlanDuration = 0
other.RequeueDuration = 0
other.ExecuteDuration = 0
other.Concurrency = 0
other.MaxAllocated = 0
other.TotalAllocated = 0
other.RuntimeErrors = nil
// Now use the Add method to combine the two.
// This should pick up any statistics from the other statistics.
q.stats = q.stats.Add(other)
}
// Statistics reports the statistics for the query.
//
// This method must be called after Done. It will block until