Skip to content

acc: cover config-remote-sync write-back for split lists - #6116

Open
ilyakuz-db wants to merge 1 commit into
mainfrom
configsync/acc-split-list-baseline
Open

acc: cover config-remote-sync write-back for split lists#6116
ilyakuz-db wants to merge 1 commit into
mainfrom
configsync/acc-split-list-baseline

Conversation

@ilyakuz-db

@ilyakuz-db ilyakuz-db commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Changes

Adds end-to-end coverage for bundle config-remote-sync write-back when a resource's list field is defined in more than one physical YAML region — a top-level resources.<type>.<name>.<list> block plus a targets.<target> override, possibly spread across included files.

Six new directories, all running locally against the in-process test server under both deployment engines: split_keyed_edit, split_keyed_twoblock, split_keyed_remove, split_multifile, split_positional, split_isolation.

The committed goldens record today's behaviour, which is wrong. That is deliberate: the fix lands on top of this and shows up as a golden diff, corruption on one side and correct output on the other. acceptance/bundle/config-remote-sync/README.md describes each directory. What the goldens currently show:

  • split_keyed_edit — editing a task defined only in the target block appends a keyless - timeout_seconds: 111 element to the top-level block; the next sync then reports tasks[task_key='']: remove for the element it just created.
  • split_keyed_remove — removing one task per block fails with remove index key out of bounds (idx 2, len 2), because a merged index is used against a block with fewer elements. Removing a task defined in both blocks deletes only the top-level half, orphaning the other.
  • split_multifile — an edit to a task defined in an included override file is written to the top-level databricks.yml instead.
  • split_positional — a field edit on a target-block pipeline cluster fails with parent path ... does not exist.
  • split_isolation — a rename of a two-block task is applied by guessing, leaving the task duplicated.

Why

Loading merges the separate YAML regions into one list (keyed lists are also sorted by key), but on disk they stay separate regions that have to be edited independently. A remote change is expressed against the merged view, so write-back has to map it back to the right (file, block, position).

config-remote-sync runs unattended — nobody reads its output or exit code — so a silently wrong write is the worst outcome. None of this was covered before: the 21 existing directories are all Cloud = true, so they don't run in normal CI, which makes "fails before the fix, passes after" impossible to demonstrate. validation_errors showed the hermetic path works, and these follow it.

The single-block path is already covered by job_multiple_tasks, which exercises edit, add, remove and rename on a resource defined in one block; it stays untouched by the fix, which is the regression evidence.

Tests

This PR is only tests. Both engines pass locally, in seconds, with no cloud access.

@github-actions

Copy link
Copy Markdown
Contributor

Waiting for approval

Based on git history, these people are best suited to review:

  • @denik -- recent work in acceptance/bundle/config-remote-sync/

Eligible reviewers: @andrewnester, @anton-107, @janniklasrose, @lennartkats-db, @pietern, @shreyas-goenka

Suggestions based on git history. See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 3854cff

Run: 30658589113

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 3 1 4 313 1073 7:16
🟨​ aws windows 3 1 4 315 1071 8:36
🟨​ azure linux 3 1 4 313 1072 7:23
🟨​ azure windows 3 1 4 315 1070 9:06
💚​ gcp linux 1 5 313 1073 4:54
💚​ gcp windows 1 5 315 1071 5:56
8 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestFetchRepositoryInfoAPI_FromRepo 🟨​K 🟨​K 🟨​K 🟨​K 🙈​S 🙈​S
🟨​ TestFetchRepositoryInfoAPI_FromRepo/root 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestFetchRepositoryInfoAPI_FromRepo/subdir 🟨​K 🟨​K 🟨​K 🟨​K
Top 3 slowest tests (at least 2 minutes):
duration env testname
5:05 azure windows TestAccept
4:55 gcp windows TestAccept
4:02 aws windows TestAccept

@ilyakuz-db
ilyakuz-db force-pushed the configsync/acc-split-list-baseline branch 3 times, most recently from 00635ac to 0d5de02 Compare July 31, 2026 17:55
@ilyakuz-db
ilyakuz-db force-pushed the configsync/acc-split-list-baseline branch from 0d5de02 to 3854cff Compare July 31, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants