Skip to main content

DESCRIBE JOB

Description​

Show the configurations for a specified Job.

Note that the SQL statement does not end with ;

Syntax​

DESCRIBE JOB `<job_name>`

Examples​

DESCRIBE JOB `count_transactions`

Required parameters​

  • <job_name>: Specify the name of the Job to describe.

Status API​

Example Status API response​

The Status API response of the Job run description.

{
    "apiStatus": "API_OPERATION_STATUS_SUCCESS",
    "apiResponse": {
        "describeJobApiResponse": {
            "sparkJob": {
                "name": "<job_name>",
                "clusterName": "<cluster_name>",
                "jobUrl": "<job_url>",
                "mainApplicationFilePath": "<main_application_file_path>",
                "arguments": "<application_arguments>",
                "sparkConfigs": {
                    "<spark_config_key1>": "<spark_config_value1>",
                    "<spark_config_key2>": "<spark_config_value2>",
                },
                "hudiConfigs": {
                    "<hoodie_config_key>": "<hoodie_config_value>"
                },
                "jarConfig": {
                    "mainClass": "<jar_main_class>"
                },
                "pythonConfig": { },
                "latestJobRunId": "<latest_job_run_id>",
                "latestJobStatus": "<latest_job_status>",
                "type": "<jar | python>",
                "latestJobRunSubmittedAt": "YYYY-MM-DDTHH:MM:SS.ssssssZ",
                "sparkLogsBasePath": "<spark_logs_base_path>",
                "maxConcurrentRuns": <max_concurrent_runs>
            }
        }
    }
}

Response fields​

  • jobUrl: A deep-link to the Job's page in the Onehouse UI. Empty string if a URL is not available.
  • hudiConfigs: The Apache Hudi configurations set on the Job with --hudi-conf. Absent when the Job has none. See Set Apache Hudi configurations on a Job.
  • maxConcurrentRuns: Number of runs of this Job that can be active at the same time. Absent for Jobs that were never configured for concurrent runs, which run one at a time.
  • sparkLogsBasePath: The parent S3 directory holding the forwarded logs for every run of this Job. Empty string before the Job's first run, or if Spark Log Forwarding is not enabled.

Each run writes to a run=<job_run_id> directory beneath this path:

s3://onehouse-customer-bucket-ab12cd34/spark-logs/job=daily-ingest/

Onehouse refreshes this path on each run, so it shows where logs land now. To get the exact directory for one run, use DESCRIBE JOB_RUN.