You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Comet Native Executors: report a query's metrics in the Spark UI #7373
A query that runs on a Comet Native Executor cluster (#7289) shows in the Spark UI as a single CometNativeExecutorsExec with no metrics: no rows, bytes, spill or time per operator, and no view of the stages it ran as.
Describe the potential solution
Return Ballista's per-stage and per-operator metrics with the query's result, and map each stage's metrics back onto the Spark operators of the native block it ran, so the plan's metrics fill in as they would for Comet on Spark.
Additional context
#7288 lists Spark UI parity as out of scope at first.
Activity