SparkOperator

Physical plan nodes and raw metrics in an action's operators.

dataclass linescope.model.SparkOperator[source]

Represent a physical operator and the metrics Spark makes available.

Attributes

id : str Identifier of the physical plan node. name : str Spark's operator name for this node. description : str Captured plan description for inspecting the operation. metrics : dict[str, Any] Available raw metrics, retaining names, values, and unit metadata. children : list[SparkOperator] Child nodes in the observed physical plan. locations : list[SourceLocation] Project source locations correlated with this operation.

See Also

SourceLocation

Locate a one-based line and zero-based character column in a snapshot.

SparkExecution

Snapshot one observed action and its available executed query plans.

SparkExecutionStats

Keep wall duration separate from cumulative executor work.