ALTER JOB

Description

Modify an existing Job.

Note that the SQL statement does not end with ;

Usage Notes:

You may specify just one parameter to modify per ALTER JOB API command. To modify multiple parameters, you must use multiple commands.
You cannot alter the Job's Type after creation.

ALTER JOB `<job_name>`
    [ SET CLUSTER = <string> ]
    [ SET PARAMETERS = <array> ]
    [ SET MAX_CONCURRENT_RUNS = <integer> ]

Example commands for modifying a Job.

ALTER JOB `count_transactions`
    SET CLUSTER = 'high_prio_cluster_spark'

ALTER JOB `count_transactions`
    SET PARAMETERS = ( '--class', 'com.example.MySparkApp', '/path/to/my-spark-app.jar', 'arg1', 'arg2' )

ALTER JOB `count_transactions`
    SET MAX_CONCURRENT_RUNS = 3

Use one optional parameter

You must use exactly one of the optional parameters in the query.

CLUSTER: Specify the name of an existing Onehouse Cluster with type Spark to run the Job.
PARAMETERS: Specify an array of Strings to pass as parameters to the Job, which will be used in a spark-submit (see Apache Spark docs). This should include the following:
- For JAR Jobs, you must include the --class parameter.
- Optionally, include any other Spark properties you'd like the Job to use.
- Optionally, include Apache Hudi configurations as --hudi-conf <key>=<value> pairs — see Set Apache Hudi configurations on a Job. An ALTER JOB re-submits the whole parameter list, so keep the pairs you want to retain.
- Include the cloud storage bucket path containing the code for your Job. The Onehouse agent must have access to read this path.
- Optionally, include any arguments you'd like to pass to the Job.
MAX_CONCURRENT_RUNS: Maximum number of runs of this Job that can be active (Queued or Running) at the same time. Set it to 1 to go back to one run at a time. See Concurrent Job runs for the requirements and limits.
- Must be a whole number >= 1.
- Any value greater than 1 requires concurrent runs to be enabled for your project — contact Onehouse support. Without it, the command fails with Concurrent runs are not enabled for this organization; max_concurrent_runs must be 1.
- Lowering the value does not cancel runs that are already active. It only applies to runs triggered after the change, so the Job can temporarily have more active runs than the new value.

API_OPERATION_STATUS_SUCCESS from Status API indicates that the Job has been modified.
API_OPERATION_STATUS_FAILED from Status API does not necessarily mean the Job modification failed. To confirm whether the Job run was modified, send a request to the DESCRIBE JOB API and check the field in sparkJob.

The Status API response of successful modification of a Job.

{
    "apiStatus": "API_OPERATION_STATUS_SUCCESS",
    "apiResponse": {}
}