Apache Spark Receiver
OpenTelemetry Collector receiver · contrib distribution
The Apache Spark Receiver fetches metrics for an Apache Spark cluster through the Apache Spark REST API - specifically, the /metrics/json, /api/v1/applications/[app-id]/stages, /api/v1/applications/[app-id]/executors, and /api/v1/applications/[app-id]/jobs endpoints.
- Component ID:
contrib-apachesparkreceiver - Name:
apachesparkreceiver - Type: receiver
- Distribution: contrib
- Repository:
opentelemetry-collector-contrib
Stability
| Level | Signals |
|---|---|
| alpha | metrics |
Attributes
| Attribute | Type | Description |
|---|---|---|
direction | string | Whether the metric is in regards to input or output operations. |
executor_task_result | string | The result of the executor tasks for which the metric was recorded. |
gc_type | string | The type of the garbage collection performed for the metric. |
job_result | string | The result of the job stages or tasks for which the metric was recorded. |
location | string | The location of the memory for which the metric was recorded.. |
pool_memory_type | string | The type of pool memory for which the metric was recorded. |
scheduler_status | string | The status of the DAGScheduler stages for which the metric was recorded. |
source | string | The source from which data was fetched for the metric. |
stage_active | bool | Whether the stage for which the metric was recorded is active. |
stage_complete | bool | Whether the stage for which the metric was recorded is complete. |
stage_failed | bool | Whether the stage for which the metric was recorded is failed. |
stage_pending | bool | Whether the stage for which the metric was recorded is pending. |
stage_task_result | string | The result of the stage tasks for which the metric was recorded. |
state | string | The state of the memory for which the metric was recorded. |