Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 18 additions & 13 deletions docs/codebase/jobs.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
# Jobs System

Ghost's jobs system runs work inline or in a worker thread. Jobs can run once or
on a schedule.
Ghost's jobs system runs work inline, in a worker thread, or in-process through
the class-based jobs service in
`ghost/core/core/server/services/jobs-service/`. Jobs can run once or on a
schedule.

Use inline jobs for short work which does not block the event loop. Inline jobs
cannot be scheduled. Scheduled and offloaded jobs run in worker threads, so
they must initialize their own dependencies and cannot rely on the main Ghost
process's memory.
cannot be scheduled. Scheduled and offloaded jobs registered through the legacy
Bree-based service run in worker threads, so they must initialize their own
dependencies and cannot rely on the main Ghost process's memory. Jobs migrated
to the class-based service (token cleanup, gift cleanup, update checks) run
in-process and share the main process's initialized services.

## Adding a job

Expand All @@ -17,7 +21,7 @@ events.

Existing examples include:

- Update checks, which run in a worker on a schedule.
- Gift reminders, which run in a worker on a schedule.
- Imports, which run as inline jobs.
- Email analytics, which uses scheduled worker jobs.

Expand All @@ -31,13 +35,14 @@ first request.

## Testing

Tests for the jobs wrapper live in
`ghost/core/test/unit/server/services/jobs/`, with update-check integration
coverage in `ghost/core/test/integration/jobs/`. Tests should cover the job's
result and failure behavior.
Tests for the legacy jobs wrapper live in
`ghost/core/test/unit/server/services/jobs/`, and tests for the class-based
service in `ghost/core/test/unit/server/services/jobs-service/`. Tests should
cover the job's result and failure behavior.

## Scheduling

The jobs system uses Bree for scheduled work. Schedules use the server's system
timezone. Offloaded jobs should have unique names, be safe to run more than
once, and receive identifiers rather than large objects where possible.
The legacy jobs service uses Bree for scheduled work; the class-based service
schedules with cron expressions through its backend. Schedules use the server's
system timezone. Jobs should have unique names, be safe to run more than once,
and receive identifiers rather than large objects where possible.
14 changes: 9 additions & 5 deletions ghost/core/core/boot.js
Original file line number Diff line number Diff line change
Expand Up @@ -483,12 +483,14 @@ async function initBackgroundServices({ config }) {
const giftService = require('./server/services/gifts');
giftService.recoverPendingDeliveries();

const jobsService = require('./server/services/jobs-service').getInstance();

// Runs before activitypub.init for the same reason as the send recovery
// above: gifts would otherwise go uncleaned for the life of the process
// if an unrelated background service fails.
try {
const giftJobs = require('./server/services/gifts/jobs');
await giftJobs.scheduleGiftCleanupJob(require('./server/services/jobs-service').getInstance());
await giftJobs.scheduleGiftCleanupJob(jobsService);
} catch (err) {
const logging = require('@tryghost/logging');
logging.error(err);
Expand Down Expand Up @@ -526,10 +528,12 @@ async function initBackgroundServices({ config }) {
]);
}

const updateCheck = require('./server/services/update-check');
updateCheck.scheduleRecurringJobs();
if (config.get('updateCheck:forceUpdate')) {
updateCheck.scheduleBootJob();
try {
const updateCheck = require('./server/services/update-check');
await updateCheck.scheduleJobs(jobsService);
} catch (err) {
const logging = require('@tryghost/logging');
logging.error(err);
}

// Remote feature-flag overrides (config-gated; inert unless explicitly configured).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ import ExternalMediaInliner from '../media-inliner/external-media-inliner';
import ExternalMediaInlinerJob from '../media-inliner/external-media-inliner-job';
import ContentCSVImportJob from '../content-import/jobs/content-csv-import-job';
import * as contentImport from '../content-import';
import UpdateCheckJob from '../update-check/jobs/update-check-job';

const updateCheck = require('../update-check');

interface RegisterJobHandlersDependencies {
jobsService: JobsService;
Expand Down Expand Up @@ -63,4 +66,8 @@ export default function registerJobHandlers({
jobsService.handle(ContentCSVImportJob, async (job) => {
await contentImport.handleJob(job);
});

jobsService.handle(UpdateCheckJob, async () => {
await updateCheck({ rethrowErrors: true });
});
}
26 changes: 13 additions & 13 deletions ghost/core/core/server/services/update-check/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ const api = require('../../api').endpoints;
const config = require('../../../shared/config');
const logging = require('@tryghost/logging');
const urlUtils = require('../../../shared/url-utils').default;
const jobsService = require('../jobs');
const UpdateCheckJob = require('./jobs/update-check-job').default;

const request = require('@tryghost/request');
const ghostVersion = require('@tryghost/version');
Expand Down Expand Up @@ -66,25 +66,25 @@ module.exports = async ({
await updateChecker.check();
};

module.exports.scheduleRecurringJobs = () => {
const scheduleRecurringJob = async (jobsService) => {
// use a random seconds/minutes/hours value to avoid spikes to the update service API
const s = Math.floor(Math.random() * 60); // 0-59
const m = Math.floor(Math.random() * 60); // 0-59
const h = Math.floor(Math.random() * 24); // 0-23

const at = `${s} ${m} ${h} * * *`;
logging.info(`[Background Job] update-check scheduled at ${at}`);
jobsService.addJob({
at, // Every day
job: require('path').resolve(__dirname, 'run-update-check.js'),
name: 'update-check',
});
await jobsService.scheduleRecurring(new UpdateCheckJob(), { cron: at });
};

module.exports.scheduleBootJob = () => {
logging.info('[Background Job] update-check-boot queued');
jobsService.addJob({
job: require('path').resolve(__dirname, 'run-update-check.js'),
name: 'update-check-boot',
});
const scheduleBootJob = async (jobsService) => {
logging.info('[Background Job] update-check boot run queued');
await jobsService.dispatch(new UpdateCheckJob());
};

module.exports.scheduleJobs = async (jobsService) => {
await scheduleRecurringJob(jobsService);
if (config.get('updateCheck:forceUpdate')) {
await scheduleBootJob(jobsService);
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { Job } from '../../jobs-service/job';

export default class UpdateCheckJob extends Job {
static type = 'update-check';
}
63 changes: 0 additions & 63 deletions ghost/core/core/server/services/update-check/run-update-check.js

This file was deleted.

142 changes: 0 additions & 142 deletions ghost/core/test/integration/jobs/update-check.test.js

This file was deleted.

Loading
Loading