mirror of
https://github.com/go-gitea/gitea.git
synced 2026-06-14 03:29:55 +00:00
fix(actions): deadlock between PrepareRunAndInsert and UpdateTaskByState (#37692)
Fix #36234 ## Bug Logs show `PrepareRunAndInsert: InsertRun: Error 1213: Deadlock found`, which `handleWorkflows` silently swallows via `log.Error + continue`, so the triggered run is dropped. ## Root cause The path `UpdateRun -> UpdateRepoRunsNumbers` runs the following SQL inside every status-changing transaction: ```sql UPDATE repository SET num_action_runs = (SELECT count(*) FROM action_run WHERE repo_id = N), num_closed_action_runs = (SELECT count(*) FROM action_run WHERE repo_id = N AND status IN (...)) WHERE id = N; ``` On any DB that treats subqueries inside an UPDATE as locking reads, this statement takes locks in two steps: 1. The outer UPDATE acquires an X lock on `repository[id=N]` 2. The embedded SELECT subqueries are evaluated as locking reads, taking S locks on every `action_run` row matching `repo_id = N` Two such concurrent transactions form a cycle via `repository[N]`: | Tx | Holds | Wants | Blocked by | |---|---|---|---| | A: `PrepareRunAndInsert` (push trigger) | X on inserted `action_run` row R_A; X on `repository[N]` (outer UPDATE already through step 1) | S on `action_run` rows for repo N (subquery, step 2) | B's X lock on R_B | | B: `UpdateTaskByState` (runner callback) | X on `action_run` row R_B (from `UpdateRun`) | X on `repository[N]` (outer UPDATE, step 1) | A's X lock on `repository[N]` | | **Cycle** | A waits for R_B; B waits for `repository[N]` | | deadlock error -> `handleWorkflows` swallows -> run lost | PostgreSQL's MVCC reads do not take these locks and SQLite serializes writers, so the symptom only surfaces on MySQL/MSSQL. ## Fix Split `UpdateRepoRunsNumbers` into small SQLs to avoid locking reads and move it out of DB transactions. --------- Signed-off-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Nicolas <bircni@icloud.com>
This commit is contained in:
@@ -250,6 +250,8 @@ func DeleteRun(ctx context.Context, run *actions_model.ActionRun) error {
|
||||
return err
|
||||
}
|
||||
|
||||
actions_model.UpdateRepoRunsNumbers(ctx, repoID)
|
||||
|
||||
// Delete files on storage
|
||||
for _, tas := range tasks {
|
||||
removeTaskLog(ctx, tas)
|
||||
|
||||
@@ -78,7 +78,11 @@ func NotifyWorkflowRunStatusUpdate(ctx context.Context, run *actions_model.Actio
|
||||
}
|
||||
triggerUser = attempt.TriggerUser
|
||||
}
|
||||
|
||||
notify_service.WorkflowRunStatusUpdate(ctx, run.Repo, triggerUser, run)
|
||||
|
||||
// Recomputes the repository's num_action_runs / num_closed_action_runs counters since the run's status changed
|
||||
actions_model.UpdateRepoRunsNumbers(ctx, run.RepoID)
|
||||
}
|
||||
|
||||
// NotifyWorkflowJobsStatusUpdate notifies status updates for jobs without task.
|
||||
|
||||
Reference in New Issue
Block a user