Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ All notable changes to the `python-domino` library will be documented in this fi
### Added

### Changed
* Clarified `job_start()` docstrings: `commit_id` pins the project's internal DFS
commit and cannot be used to pin a commit on a git-based project's main git
repository (it will fail). Documented that `main_repo_git_ref` already supports
`{"type": "commitId", "value": <sha>}` for that purpose, alongside the previously
documented `"branches"`/`"tags"` types. No behavior change — this type was already
accepted by the API. (DOM-79862)

## [2.2.0]

Expand Down
7 changes: 5 additions & 2 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,8 @@ Start a new job (execution) in the project using the v4 Jobs API.
For example: `domino.job_start(command="main.py arg1 arg2")`
* _commit_id (string):_ (Optional) The commit ID to launch from.
If not provided, the job launches from the latest commit. Mutually exclusive with `branch`.
NOTE: this always pins the project's internal DFS commit, regardless of project type. For git-based projects, that is not the same as a commit on the main git repository — a valid DFS commit SHA still works here, but a commit SHA from the main git repository will fail.
To pin a commit on a git-based project's main repository, use `main_repo_git_ref={"type": "commitId", "value": "<sha>"}` instead.
* _branch (string):_ (Optional) The branch name to launch from.
If not provided, the job launches from the latest commit on the default branch.
Mutually exclusive with `commit_id` and `main_repo_git_ref`.
Expand Down Expand Up @@ -667,8 +669,9 @@ If `on_demand_spark_cluster_properties` and `compute_cluster_properties` are bot
* _external_volume_mounts (List[string]):_ (Optional) External volume mount IDs to mount to execution.
If not provided, the job launches with no external volumes mounted.
* _title (string):_ (Optional) Title for Job.
* _main_repo_git_ref (dict):_ (Optional) Raw git ref dict for advanced use cases.
For example: `{"type": "branches", "value": "my-feature-branch"}` or `{"type": "tags", "value": "v1.2.3"}`.
* _main_repo_git_ref (dict):_ (Optional) For git-based projects, specifies the branch, tag, or commit to run from.
For example: `{"type": "branches", "value": "my-feature-branch"}`, `{"type": "tags", "value": "v1.2.3"}`, or `{"type": "commitId", "value": "960a4c99a4cc38194cbacbcce41caa68ba5369ea"}`.
Use `"commitId"` to pin a specific commit SHA on the main git repository — `commit_id` cannot be used for that purpose.
Mutually exclusive with `branch`.

==== job_stop(job_id, commit_results=True):
Expand Down
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -654,7 +654,13 @@ Start a new job (execution) in the project using the v4 Jobs API.

- *commit_id (string):* (Optional) The commit ID to launch from. If
not provided, the job launches from the latest commit. Mutually
exclusive with `branch`.
exclusive with `branch`. **Note:** this always pins the project's
internal DFS commit, regardless of project type. For git-based
projects, that is not the same as a commit on the main git
repository — a valid DFS commit SHA still works here, but a commit
SHA from the main git repository will fail. To pin a commit on a
git-based project's main repository, use
`main_repo_git_ref={"type": "commitId", "value": "<sha>"}` instead.

- *branch (string):* (Optional) The branch name to launch from. If
not provided, the job launches from the latest commit on the default
Expand Down Expand Up @@ -717,14 +723,17 @@ Start a new job (execution) in the project using the v4 Jobs API.
- *title (string): (Optional) Title for Job.

- *main_repo_git_ref (dict):* (Optional) For git-based projects,
specifies the branch or tag to run from. Must contain
specifies the branch, tag, or commit to run from. Must contain
`"type"` and `"value"` keys. For example:

{"type": "branches", "value": "my-feature-branch"}
{"type": "tags", "value": "v1.2.3"}
{"type": "commitId", "value": "960a4c99a4cc38194cbacbcce41caa68ba5369ea"}

If not provided, the job launches from the latest commit on the
default branch.
Use `"commitId"` to pin a specific commit SHA on the main git
repository — `commit_id` cannot be used for that purpose. If not
provided, the job launches from the latest commit on the default
branch. Mutually exclusive with `branch`.

### job_stop(job_id, commit_results=True):

Expand Down
15 changes: 12 additions & 3 deletions domino/domino.py
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,13 @@ def job_start( # noqa: C901
:param commit_id: string (Optional)
The commit_id to launch from. If not provided, will launch
from latest commit.
NOTE: this always pins the project's internal DFS commit,
regardless of project type. For git-based projects, that is
NOT the same as a commit on the main git repository — a valid
DFS commit SHA still works here, but a commit SHA from the
main git repository will fail. To pin a commit on a git-based
project's main repository, use
main_repo_git_ref={"type": "commitId", "value": <sha>} instead.
:param hardware_tier_id: string (Optional)
The hardware tier ID to launch job in. If not provided
it will use the default hardware tier for the project
Expand Down Expand Up @@ -490,13 +497,15 @@ def job_start( # noqa: C901
:param title string (Optional)
Title for the Job
:param main_repo_git_ref: dict (Optional)
For git-based projects, specifies the branch or tag to run from.
Must be a dict with "type" and "value" keys, e.g.:
For git-based projects, specifies the branch, tag, or commit to
run from. Must be a dict with "type" and "value" keys, e.g.:
{
"type": "branches",
"value": "my-feature-branch"
}
Supported types: "branches", "tags".
Supported types: "branches", "tags", "commitId". Use type
"commitId" to pin a specific commit SHA on the main git
repository (commit_id cannot be used for that purpose).
Cannot be combined with branch.
:param branch: string (Optional)
Convenience parameter. For git-based projects, launch the job
Expand Down
33 changes: 33 additions & 0 deletions tests/test_jobs.py
Original file line number Diff line number Diff line change
Expand Up @@ -389,6 +389,39 @@ def test_job_start_sends_main_repo_git_ref(requests_mock, dummy_hostname):
assert jobs_start_request.json()["mainRepoGitRef"] == git_ref


@pytest.mark.usefixtures("clear_token_file_from_env", "mock_job_start_blocking_setup")
def test_job_start_main_repo_git_ref_supports_commit_id_type(
requests_mock, dummy_hostname
):
"""
Confirm main_repo_git_ref with type "commitId" is passed through as-is, and that
it is independent of the (DFS-only) commit_id field: for git-based projects this
is the correct way to pin a main-repository commit, since commit_id cannot be
used for that purpose.
"""
requests_mock.get(
f"{dummy_hostname}/v4/jobs/{MOCK_JOB_ID}",
json=MOCK_JOB_RESPONSE_COMPLETED,
)

d = Domino(host=dummy_hostname, project="anyuser/anyproject", api_key="whatever")

git_ref = {"type": "commitId", "value": "960a4c99a4cc38194cbacbcce41caa68ba5369ea"}
d.job_start_blocking(
command="foo.py",
main_repo_git_ref=git_ref,
poll_freq=1,
max_poll_time=1,
)

jobs_start_request = next(
req for req in requests_mock.request_history if req.path == "/v4/jobs/start"
)
request_body = jobs_start_request.json()
assert request_body["mainRepoGitRef"] == git_ref
assert request_body["commitId"] is None


@pytest.mark.usefixtures("clear_token_file_from_env", "mock_job_start_blocking_setup")
def test_job_start_branch_sets_main_repo_git_ref(requests_mock, dummy_hostname):
"""
Expand Down
Loading