Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 44 additions & 32 deletions edi_queue_oca/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,22 +64,22 @@ Per-type job configuration

Each **Exchange Type** gains a *Queue* tab with optional settings:

+---------------------------+------------------------------------------+
| Field | Purpose |
+===========================+==========================================+
| **Job channel** | Route jobs to a specific channel (e.g. |
| | ``root.edi.high``). |
+---------------------------+------------------------------------------+
| **Job priority** | Integer priority passed to the queue job |
| | (lower = higher priority). |
+---------------------------+------------------------------------------+
| **Enable ETA Scheduling** | Toggle to activate daily job |
| | accumulation (see below). |
+---------------------------+------------------------------------------+
| **Execution time** | Hour, minute, and timezone at which |
| | accumulated jobs are released. Visible |
| | only when ETA Scheduling is enabled. |
+---------------------------+------------------------------------------+
+---------------------------+-----------------------------------------+
| Field | Purpose |
+===========================+=========================================+
| **Job channel** | Route jobs to a specific channel (e.g. |
| | ``root.edi.high``). |
+---------------------------+-----------------------------------------+
| **Job priority** | Integer priority passed to the queue |
| | job (lower = higher priority). |
+---------------------------+-----------------------------------------+
| **Enable ETA Scheduling** | Toggle to activate daily job |
| | accumulation (see below). |
+---------------------------+-----------------------------------------+
| **Execution time** | Hour, minute, and timezone at which |
| | accumulated jobs are released. Visible |
| | only when ETA Scheduling is enabled. |
+---------------------------+-----------------------------------------+

Accumulating jobs until a fixed daily time
------------------------------------------
Expand All @@ -91,20 +91,20 @@ during the day accumulate and are released together at that moment.

**Typical use cases:**

- A trading partner's receiving system only processes incoming files at
a specific nightly window (e.g. 22:00).
- Resource-intensive EDI operations (large exports, heavy
transformations) should be deferred to off-peak hours to avoid
competing with daytime workloads.
- Operational preference to send a batch of documents at a predictable
daily time instead of dispatching them one by one in real time.
- A trading partner's receiving system only processes incoming files at
a specific nightly window (e.g. 22:00).
- Resource-intensive EDI operations (large exports, heavy
transformations) should be deferred to off-peak hours to avoid
competing with daytime workloads.
- Operational preference to send a batch of documents at a predictable
daily time instead of dispatching them one by one in real time.

The execution time is configured with three fields:

- **Hour** — hour of the day (00–23).
- **Minute** — minute of the hour (00–59).
- **Timezone** — the timezone in which the hour and minute are
interpreted. Defaults to the current user's timezone.
- **Hour** — hour of the day (00–23).
- **Minute** — minute of the hour (00–59).
- **Timezone** — the timezone in which the hour and minute are
interpreted. Defaults to the current user's timezone.

At runtime the configured time is converted to the next matching UTC
datetime and set as the queue job ETA. If the target time for today has
Expand All @@ -121,6 +121,18 @@ An identity key is attached to every queued job, so re-triggering an
action for a record that already has a pending job does not enqueue a
duplicate.

Garbage collection of stranded jobs
-----------------------------------

``queue_job`` cancels dependent jobs only when a parent is explicitly
cancelled, so a dependent of a *failed* parent stays in *Wait
Dependencies* forever. The cron *EDI exchange garbage collect stale
jobs* cancels those jobs once no parent can still bring them to
execution, and only after a grace period (24 hours by default, set by
the system parameter ``edi_queue_oca.gc_stale_jobs_grace_hours``) that
leaves time to requeue the failed parent by hand. The cron is disabled
by default.

Bug Tracker
===========

Expand All @@ -143,11 +155,11 @@ Authors
Contributors
------------

- Simone Orsi <simahawk@gmail.com>
- Enric Tobella <enric.tobella@dixmit.com>
- Manuel Regidor <manuel.regidor@sygel.es>
- Thien Vo <thienvh@trobz.com>
- Jordi Masvidal <jordi.masvidal@forgeflow.com>
- Simone Orsi <simahawk@gmail.com>
- Enric Tobella <enric.tobella@dixmit.com>
- Manuel Regidor <manuel.regidor@sygel.es>
- Thien Vo <thienvh@trobz.com>
- Jordi Masvidal <jordi.masvidal@forgeflow.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AaronHForgeFlow time to add yourself? :)


Maintainers
-----------
Expand Down
1 change: 1 addition & 0 deletions edi_queue_oca/__manifest__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"security/ir_model_access.xml",
"data/job_channel.xml",
"data/job_function.xml",
"data/cron.xml",
"views/edi_exchange_record.xml",
],
"demo": [],
Expand Down
13 changes: 13 additions & 0 deletions edi_queue_oca/data/cron.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8" ?>
<odoo noupdate="1">
<record id="cron_edi_backend_gc_stale_jobs" model="ir.cron" forcecreate="True">
<field name="name">EDI exchange garbage collect stale jobs</field>
<field name="active" eval="False" />
<field name="user_id" ref="base.user_root" />
<field name="interval_number">1</field>
<field name="interval_type">days</field>
<field name="model_id" ref="edi_core_oca.model_edi_backend" />
<field name="state">code</field>
<field name="code">model.search([])._job_gc_stale()</field>
</record>
</odoo>
81 changes: 80 additions & 1 deletion edi_queue_oca/models/edi_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,25 @@
# Copyright 2025 Dixmit
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).

import logging
from datetime import timedelta

from odoo import models
from odoo import fields, models

from odoo.addons.queue_job.exception import RetryableJobError
from odoo.addons.queue_job.job import (
CANCELLED,
DONE,
ENQUEUED,
PENDING,
STARTED,
WAIT_DEPENDENCIES,
)

_logger = logging.getLogger(__name__)

# States a parent job can still leave on its own to unblock its dependents.
LIVE_JOB_STATES = (PENDING, ENQUEUED, STARTED, WAIT_DEPENDENCIES)


class EDIBackend(models.Model):
Expand All @@ -20,3 +35,67 @@ def _send_retryable_exceptions(self):

def _retryable_exception(self):
return RetryableJobError

def _job_gc_stale_grace_hours(self):
"""Return the delay before a stranded job is collected.

The grace period leaves time to requeue the failed parent by hand,
which brings its dependents back to life on its own.
"""
param = self.env["ir.config_parameter"].sudo()
return int(param.get_param("edi_queue_oca.gc_stale_jobs_grace_hours", 24))

def _job_gc_stale_domain(self, grace_hours):
deadline = fields.Datetime.now() - timedelta(hours=grace_hours)
return [
("model_name", "=", self.exchange_record_model._name),
("state", "=", WAIT_DEPENDENCIES),
("date_created", "<=", deadline),
]

def _job_gc_is_stale(self, job, parent_states):
parent_uuids = (job.dependencies or {}).get("depends_on") or []
if not parent_uuids:
return False
# A parent missing from the mapping has been vacuumed away: nothing
# will ever wake this job up again.
blocking = [
parent_states.get(uuid)
for uuid in parent_uuids
if parent_states.get(uuid) != DONE
]
return bool(blocking) and not any(x in LIVE_JOB_STATES for x in blocking)

def _job_gc_stale_result(self):
return self.env._(
"Cancelled by the EDI garbage collector: "
"no remaining parent job can ever resume it."
)

def _job_gc_stale(self):
"""Cancel exchange jobs that can never be woken up.
Comment thread
AaronHForgeFlow marked this conversation as resolved.

``queue_job`` cancels dependents only when a parent is explicitly
cancelled: when a parent fails they stay in ``wait_dependencies``
forever, as EDI retries spawn a brand new job graph.
"""
job_model = self.env["queue.job"].sudo()
candidates = job_model.search(
self._job_gc_stale_domain(self._job_gc_stale_grace_hours())
)
if not candidates:
return job_model.browse()
parent_uuids = set()
for job in candidates:
parent_uuids.update((job.dependencies or {}).get("depends_on") or [])
parent_states = {
job.uuid: job.state
for job in job_model.search([("uuid", "in", list(parent_uuids))])
}
stale = candidates.filtered(
lambda job: self._job_gc_is_stale(job, parent_states)
)
if stale:
_logger.info("EDI exchange GC: cancelling %d stale jobs.", len(stale))
stale._change_job_state(CANCELLED, result=self._job_gc_stale_result())
return stale
10 changes: 10 additions & 0 deletions edi_queue_oca/readme/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,13 @@ usual.

An identity key is attached to every queued job, so re-triggering an action
for a record that already has a pending job does not enqueue a duplicate.

## Garbage collection of stranded jobs

`queue_job` cancels dependent jobs only when a parent is explicitly cancelled,
so a dependent of a *failed* parent stays in *Wait Dependencies* forever. The
cron *EDI exchange garbage collect stale jobs* cancels those jobs once no
parent can still bring them to execution, and only after a grace period (24
hours by default, set by the system parameter
`edi_queue_oca.gc_stale_jobs_grace_hours`) that leaves time to requeue the
failed parent by hand. The cron is disabled by default.
40 changes: 26 additions & 14 deletions edi_queue_oca/static/description/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -392,13 +392,14 @@ <h1>Edi Queue Oca</h1>
<li><a class="reference internal" href="#per-type-job-configuration" id="toc-entry-3">Per-type job configuration</a></li>
<li><a class="reference internal" href="#accumulating-jobs-until-a-fixed-daily-time" id="toc-entry-4">Accumulating jobs until a fixed daily time</a></li>
<li><a class="reference internal" href="#duplicate-job-prevention" id="toc-entry-5">Duplicate-job prevention</a></li>
<li><a class="reference internal" href="#garbage-collection-of-stranded-jobs" id="toc-entry-6">Garbage collection of stranded jobs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#bug-tracker" id="toc-entry-6">Bug Tracker</a></li>
<li><a class="reference internal" href="#credits" id="toc-entry-7">Credits</a><ul>
<li><a class="reference internal" href="#authors" id="toc-entry-8">Authors</a></li>
<li><a class="reference internal" href="#contributors" id="toc-entry-9">Contributors</a></li>
<li><a class="reference internal" href="#maintainers" id="toc-entry-10">Maintainers</a></li>
<li><a class="reference internal" href="#bug-tracker" id="toc-entry-7">Bug Tracker</a></li>
<li><a class="reference internal" href="#credits" id="toc-entry-8">Credits</a><ul>
<li><a class="reference internal" href="#authors" id="toc-entry-9">Authors</a></li>
<li><a class="reference internal" href="#contributors" id="toc-entry-10">Contributors</a></li>
<li><a class="reference internal" href="#maintainers" id="toc-entry-11">Maintainers</a></li>
</ul>
</li>
</ul>
Expand All @@ -417,8 +418,8 @@ <h3><a class="toc-backref" href="#toc-entry-3">Per-type job configuration</a></h
<p>Each <strong>Exchange Type</strong> gains a <em>Queue</em> tab with optional settings:</p>
<table border="1" class="docutils">
<colgroup>
<col width="39%" />
<col width="61%" />
<col width="40%" />
<col width="60%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Field</th>
Expand All @@ -431,8 +432,8 @@ <h3><a class="toc-backref" href="#toc-entry-3">Per-type job configuration</a></h
<tt class="docutils literal">root.edi.high</tt>).</td>
</tr>
<tr><td><strong>Job priority</strong></td>
<td>Integer priority passed to the queue job
(lower = higher priority).</td>
<td>Integer priority passed to the queue
job (lower = higher priority).</td>
</tr>
<tr><td><strong>Enable ETA Scheduling</strong></td>
<td>Toggle to activate daily job
Expand Down Expand Up @@ -482,26 +483,37 @@ <h3><a class="toc-backref" href="#toc-entry-5">Duplicate-job prevention</a></h3>
action for a record that already has a pending job does not enqueue a
duplicate.</p>
</div>
<div class="section" id="garbage-collection-of-stranded-jobs">
<h3><a class="toc-backref" href="#toc-entry-6">Garbage collection of stranded jobs</a></h3>
<p><tt class="docutils literal">queue_job</tt> cancels dependent jobs only when a parent is explicitly
cancelled, so a dependent of a <em>failed</em> parent stays in <em>Wait
Dependencies</em> forever. The cron <em>EDI exchange garbage collect stale
jobs</em> cancels those jobs once no parent can still bring them to
execution, and only after a grace period (24 hours by default, set by
the system parameter <tt class="docutils literal">edi_queue_oca.gc_stale_jobs_grace_hours</tt>) that
leaves time to requeue the failed parent by hand. The cron is disabled
by default.</p>
</div>
</div>
<div class="section" id="bug-tracker">
<h2><a class="toc-backref" href="#toc-entry-6">Bug Tracker</a></h2>
<h2><a class="toc-backref" href="#toc-entry-7">Bug Tracker</a></h2>
<p>Bugs are tracked on <a class="reference external" href="https://github.com/OCA/edi-framework/issues">GitHub Issues</a>.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us to smash it by providing a detailed and welcomed
<a class="reference external" href="https://github.com/OCA/edi-framework/issues/new?body=module:%20edi_queue_oca%0Aversion:%2019.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**">feedback</a>.</p>
<p>Do not contact contributors directly about support or help with technical issues.</p>
</div>
<div class="section" id="credits">
<h2><a class="toc-backref" href="#toc-entry-7">Credits</a></h2>
<h2><a class="toc-backref" href="#toc-entry-8">Credits</a></h2>
<div class="section" id="authors">
<h3><a class="toc-backref" href="#toc-entry-8">Authors</a></h3>
<h3><a class="toc-backref" href="#toc-entry-9">Authors</a></h3>
<ul class="simple">
<li>Dixmit</li>
<li>Camptocamp</li>
</ul>
</div>
<div class="section" id="contributors">
<h3><a class="toc-backref" href="#toc-entry-9">Contributors</a></h3>
<h3><a class="toc-backref" href="#toc-entry-10">Contributors</a></h3>
<ul class="simple">
<li>Simone Orsi &lt;<a class="reference external" href="mailto:simahawk&#64;gmail.com">simahawk&#64;gmail.com</a>&gt;</li>
<li>Enric Tobella &lt;<a class="reference external" href="mailto:enric.tobella&#64;dixmit.com">enric.tobella&#64;dixmit.com</a>&gt;</li>
Expand All @@ -511,7 +523,7 @@ <h3><a class="toc-backref" href="#toc-entry-9">Contributors</a></h3>
</ul>
</div>
<div class="section" id="maintainers">
<h3><a class="toc-backref" href="#toc-entry-10">Maintainers</a></h3>
<h3><a class="toc-backref" href="#toc-entry-11">Maintainers</a></h3>
<p>This module is maintained by the OCA.</p>
<a class="reference external image-reference" href="https://odoo-community.org">
<img alt="Odoo Community Association" src="https://odoo-community.org/logo.png" />
Expand Down
1 change: 1 addition & 0 deletions edi_queue_oca/tests/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from . import test_exchange_type
from . import test_record
from . import test_backend_jobs
from . import test_backend_gc_jobs
from . import test_backend_input_jobs
from . import test_backend_output_jobs

Expand Down
48 changes: 48 additions & 0 deletions edi_queue_oca/tests/test_backend_gc_jobs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Copyright 2026 ForgeFlow S.L.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).

from datetime import timedelta

from odoo.tests import tagged

from odoo.addons.queue_job.job import Job

from .common import EDIQueueCommonTestCase


@tagged("-at_install", "post_install")
class EDIBackendTestGCJobsCase(EDIQueueCommonTestCase):
def _make_chained_jobs(self, parent_state, age_hours=0):
record = self._make_record()
parent = Job(record.action_exchange_receive)
child = Job(record.action_exchange_process)
child.add_depends({parent})
parent.state = parent_state
child.date_created -= timedelta(hours=age_hours)
parent.store()
child.store()
return parent, child

def _set_grace_hours(self, hours):
self.env["ir.config_parameter"].sudo().set_param(
"edi_queue_oca.gc_stale_jobs_grace_hours", hours
)

def test_gc_stale_exchange_jobs(self):
pending_parent, live_child = self._make_chained_jobs("pending", age_hours=25)
failed_parent, stale_child = self._make_chained_jobs("failed", age_hours=25)
self.assertEqual(live_child.db_record().state, "wait_dependencies")
self.assertEqual(stale_child.db_record().state, "wait_dependencies")
self._set_grace_hours(48)
self.assertFalse(self.backend._job_gc_stale())
self.assertEqual(stale_child.db_record().state, "wait_dependencies")
self._set_grace_hours(24)
collected = self.backend._job_gc_stale()
self.assertEqual(collected, stale_child.db_record())
self.assertEqual(stale_child.db_record().state, "cancelled")
self.assertEqual(
stale_child.db_record().result, self.backend._job_gc_stale_result()
)
self.assertEqual(live_child.db_record().state, "wait_dependencies")
self.assertEqual(failed_parent.db_record().state, "failed")
self.assertEqual(pending_parent.db_record().state, "pending")
Loading