feat(deployment): add concurrent deployment queue system#3744
feat(deployment): add concurrent deployment queue system#3744mhbdev wants to merge 4 commits intoDokploy:canaryfrom
Conversation
…bility - Add deployment concurrency configuration for servers (1-5) - Add local deployment concurrency configuration in web server settings - Implement queue-based deployment system with separate queues per target - Add queue summary display in deployment views showing running/queued jobs - Update UI to show "queued" status messages instead of immediate success - Replace single deployment queue with target-specific queues (local/server) - Add deployment worker refresh when concurrency settings change - Update database schema with deploymentConcurrency fields - Update all deployment enqueue points to use new queue system
| !!id && | ||
| (type === "application" || type === "compose") && | ||
| isCloud === false, | ||
| refetchInterval: 1000, |
There was a problem hiding this comment.
1-second polling may be too aggressive
refetchInterval: 1000 means every open deployment page fires a queueSummaryByType query once per second. This query calls findApplicationById/findComposeById, findServerById, then fetches jobs from Redis. With several users or tabs open, this could create noticeable database and Redis load.
Consider increasing the interval to 3-5 seconds, or switching to a WebSocket/SSE push model that the rest of the deployment UI already uses.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
- Enhance `withRefreshLock` to coalesce concurrent refresh requests, ensuring only the latest concurrency settings are applied - Extend `resolveTargetFromDeploymentJob` to correctly route both application and preview deployments - Ensure `cleanAllDeploymentQueue` cancels active workers before clearing jobs - Increase polling interval for deployment UI to reduce load - Add comprehensive test suite covering queue routing, cleanup, refresh coalescing, and summary calculations
…ub.com/mhbdev/dokploy into feature/deployment-queue-observability
|
That would be amazing, it would be fantastic if this were merged! |
|
This is easily one of the most requested feature on Dokploy. There is a similar PR open: #2127 It might be a good idea to consider both implementations to merge in the best implementation given the current limitations of the stack. |
What is this PR about?
Please describe in a short paragraph what this PR is about.
Checklist
Before submitting this PR, please make sure that:
canarybranch.Greptile Summary
This PR replaces the single-queue deployment system with a per-target (local/server) queue architecture that supports configurable concurrency (1-5) for both the local host and remote servers. It adds new DB columns, UI controls, a queue summary display, and updates all deployment enqueue points to route jobs to the correct target queue.
resolveTargetFromDeploymentJobchecksbuildServerId || serverIdfor regular applications but only checksserverIdforapplication-preview, meaning preview builds will bypass the designated build server.cleanAllDeploymentQueuecalledcancelAllJobs()to stop active deployments; the new implementation only removes waiting/delayed jobs, leaving active deployments running.withRefreshLocksilently drops concurrent refresh requests, so rapid concurrency changes may not take effect.isServiceJobingetQueueSummaryByTypedoesn't matchapplication-preview, potentially underreporting queued jobs for an application.Confidence Score: 2/5
apps/dokploy/server/queues/queueSetup.ts— it contains the preview deployment routing bug, the cleanAllDeploymentQueue regression, and the refresh lock race condition.Last reviewed commit: 41eaa67