Repository navigation
Expand file tree
/
Copy pathconfig.example.yaml
More file actions
304 lines (268 loc) · 14.2 KB
/
Copy pathconfig.example.yaml
File metadata and controls
304 lines (268 loc) · 14.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
# Example configuration file for the Statusengine Go worker (cmd/app).
#
# Usage: go run ./cmd/app -config /path/to/config.yaml
# (or STATUSENGINE_CONFIG=/path/to/config.yaml go run ./cmd/app)
#
# Every key below is optional - omit anything you don't want to override.
# Precedence, for every setting, is:
#
# explicit CLI flag > environment variable > this file > built-in default
#
# So this file is the place for your "boring", rarely-changing settings,
# while flags/environment variables (handy in Docker/CI) can still
# override anything here for a one-off run without editing the file.
#
# All values below are shown with their built-in default.
# Queue backend to consume from: "gearman" or "rabbitmq".
consumer: gearman
# Gearman job server address (host:port). Only used when consumer: gearman.
gearman_addr: 127.0.0.1:4730
# Maximum number of Gearman job handlers running at once, PER QUEUE. This
# is not a throughput knob: the library dispatches every assigned job on
# its own goroutine and grabs the next without waiting, and a handler
# blocks on MySQL once a bulk-insert buffer is full. Without a cap, a
# backlog accumulates inside the worker (one goroutine plus a decoded
# payload per job) instead of staying queued at the job server.
#
# Per queue, because the worker opens one Gearman connection per queue and
# each has a Work loop of its own. A shared budget is a budget that one
# backlogged queue takes all of - and since that loop stops reading its
# socket once the budget is gone, every other queue stops being served
# entirely, not merely slowly. The process-wide worst case is therefore
# this value times the number of queues (12), so 8 here is 96 overall. An
# idle queue costs nothing; a cap only binds when there is traffic.
#
# Must be >= 1. NOTE: this replaces the old `gearman_max_concurrent_jobs`,
# which was a single budget shared by all queues. The worker refuses to
# start if that key is still present rather than reinterpreting its value,
# because carrying a 64 over would mean 768 concurrent handlers here.
gearman_max_concurrent_jobs_per_queue: 1
# RabbitMQ broker URL (amqp://user:pass@host:port/vhost). Only used when
# consumer: rabbitmq.
rabbitmq_url: amqp://statusengine:statusengine@127.0.0.1:5672/
# Maximum unacknowledged deliveries the broker may push, per queue. The
# RabbitMQ counterpart of gearman_max_concurrent_jobs_per_queue: without it AMQP
# delivers as fast as it can and the whole backlog ends up buffered in
# this process. Applies per queue, so the worst case in memory is this
# value times the number of queues (currently 12). Must be >= 1.
rabbitmq_prefetch: 100
# MySQL data source name (go-sql-driver/mysql format).
mysql_dsn: statusengine-dev:statusengine-dev@tcp(127.0.0.1:3306)/statusengine-dev?parseTime=true
# Maximum number of open MySQL connections, also used as the idle-connection
# limit so the pool doesn't tear down and redial connections between the
# bulk-inserters' 250ms flush ticks. Keep this below the server's
# max_connections, and remember it applies per worker process.
mysql_max_open_conns: 25
# Rows buffered per table before a bulk INSERT is flushed ahead of the
# 250ms ticker. The default of 100 is conservative; 700 is the documented
# production value and also the maximum.
#
# The ceiling is not a matter of taste. Every flush is sent as a server-side
# prepared statement, and those are limited to 65535 placeholders. What has
# to fit is not this number of rows but roughly twice it: on a shutdown or
# core-restart flush the buffer is topped up with whatever is still in
# flight, and all of it becomes one statement. At 700 the widest table
# (statusengine_servicestatus, 43 columns) needs 1399 x 43 = 60157 of the
# 65535 - with room for three more columns. At 1000 it would need 85957 and
# fail with "Error 1390: Prepared statement contains too many placeholders",
# which is deterministic, therefore never retried, and would drop every
# batch: the two status tables would silently stop being written.
#
# Raising this only matters under sustained load. With a 250ms ticker a
# batch size of N only binds above roughly 4*N events per second on that one
# table - 100 caps at ~400 rows/s per table, 700 at ~2800. Note also that a
# failed flush drops its batch, so this is equally the number of events one
# bad row can take down with it.
mysql_batch_size: 500
# Metrics buffered before a Graphite Carbon write is flushed ahead of the
# 250ms ticker. Default 100, maximum 1000, which is also the documented
# production value.
#
# Higher than the MySQL ceiling because nothing in the protocol limits it: a
# batch is one Write of newline-delimited plaintext lines. The cap is purely
# about loss - a failed dial or write drops the entire buffered batch and
# leaves re-dialling to the next flush, and Graphite metrics have no
# redelivery behind them.
graphite_batch_size: 100
# Address the WebSocket HTTP server (/ws) listens on. Loopback-only by
# default: the /ws stream carries every monitoring event the worker sees,
# so exposing it on the network is an explicit decision, not something
# that happens by leaving a setting alone. Set ":8080" to listen on all
# interfaces - and set api_keys below to a real secret if you do.
listen_addr: 127.0.0.1:8080
# Address the Prometheus /metrics HTTP server listens on.
metrics_listen_addr: ":9105"
# Graphite Carbon plaintext receiver address (host:port). Only used when
# perfdata_route is "graphite" or "both".
graphite_addr: 127.0.0.1:2003
# Prefix prepended to every Graphite metric path:
# <graphite_prefix>.<hostname>.<service_description>.<metric_label>
graphite_prefix: statusengine
# How long one connection to Carbon is used before it is replaced with a
# fresh one - the Graphite counterpart of MySQL's conn_max_lifetime.
#
# A precaution: Carbon's plaintext protocol acknowledges nothing, so a
# successful write only means the bytes left this process. A peer that
# accepts them without delivering them to Carbon (e.g. a Graphite container
# still starting up when the worker connected after a reboot) looks
# perfectly healthy - no error, no dropped metrics - and was observed to
# swallow everything until the worker was restarted. This bounds that to one
# lifetime. Any Go duration ("15m", "1h"); "0" keeps a connection until a
# write fails.
graphite_conn_max_lifetime: 15m
# Where statusngin_service_perfdata metrics are written: "mysql",
# "graphite" or "both".
perfdata_route: mysql
# node_name value written into statusengine_hoststatus/statusengine_servicestatus rows.
nodename: statusengine
# API keys accepted by the /ws endpoint. Real clients should send one of
# these via the "Authorization: Bearer <key>" or "X-Api-Key" header; the
# browser-based demo client (web/ws-test-client.html) uses the ?api_key=
# query parameter instead, since browser JavaScript can't set custom
# headers on a WebSocket handshake.
#
# Leaving this empty does NOT disable authentication: the worker then
# generates a random key at startup and logs it as a warning. That is a
# safety net so an unconfigured worker is never an open event stream - it
# changes on every restart, so configure a real key here for anything
# that has to reconnect on its own.
api_keys: []
# api_keys:
# - change-me-to-a-real-secret
# - a-second-key-for-another-client
# API keys accepted by the /commands endpoint, which submits Naemon
# external commands. Deliberately a separate list from api_keys above,
# with no overlap in either direction: an api_keys entry grants *reading*
# the event stream, an entry here grants *controlling the monitoring
# core* - passive check results, forced checks, downtimes, acknowledgements
# and, via "raw", most of Naemon's external command set.
#
# Unlike api_keys, leaving this empty really does disable the endpoint.
# There is no generated fallback key: a random write-access key nobody
# asked for is worse than no endpoint, so with nothing configured here the
# worker does not serve /commands at all and says so at startup.
command_api_keys: []
# command_api_keys:
# - a-secret-only-you-knows
# Address the external-command HTTP server (/commands) listens on. Its own
# port rather than sharing listen_addr, so the read stream and the write
# endpoint can be exposed - or not - independently. Loopback-only by
# default.
command_listen_addr: 127.0.0.1:8081
# On a core restart (Naemon/Nagios/Icinga reload), clear out stale
# hoststatus/servicestatus rows how?
# false (default): TRUNCATE both tables outright.
# true: delete only rows for objects openITCOCKPIT's own hosts/services
# tables no longer know about, so status for objects that still
# exist survives the restart instead of needing to be rebuilt.
enable_openitcockpit_tweaks: false
# Which notification method event to store in statusengine_host_notifications
# and statusengine_service_notifications?
# false (default): the END event, as the core brokers it once a notification
# command has run.
# true: the START event. For a core whose notifications a broker module such
# as mod_gearman distributes: it answers the START event with
# NEBERROR_CALLBACKOVERRIDE, the core then never runs the command itself
# and never brokers the END event, so with the default nothing is ever
# stored. end_time is set to start_time, since the end is not known.
store_notification_start: false
# Discard statusngin_hoststatus and statusngin_servicestatus events older
# than this, instead of writing them to MySQL and broadcasting them.
#
# These two queues carry a full snapshot of an object's current state, sent
# again on every check and overwriting its predecessor in a single-row-per-
# object table - so a snapshot from ten minutes ago has no reader left.
# While the worker is down they pile up one job per check interval per
# object, and draining that backlog costs the same MySQL time live traffic
# needs, which the live traffic then waits for. Skipping it lets the worker
# catch up in seconds instead of minutes.
#
# Applies to those two queues ONLY. Every other queue carries history -
# check results, state changes, notifications - where each event is the
# single record of something that happened and age is no reason to drop it.
#
# Any Go duration ("5m", "90s", "1h"). "0" processes every event regardless
# of age. Note the comparison is between the monitoring core's clock and
# this worker's: if the two hosts disagree by more than this, both queues
# stop being written entirely. statusengine_queue_events_discarded_stale_total
# is the metric that shows it.
status_max_age: 5m
# Minimum log level: "debug", "info", "warn" or "error".
#
# At "info" the worker logs lifecycle events and one summary per 30 seconds
# per active table ("db: write stats", "graphite: write stats") - nothing per
# job, per batch or per row. Leave it there for a service: "debug" adds one
# line per bulk-insert flush, which was measured at ~145 MB of journal per
# hour under sustained load. Use "debug" to explain a specific flush, not to
# watch a healthy worker. Errors are logged at every occurrence regardless of
# this setting.
log_level: info
# Structured log output format: "text" or "json".
log_format: text
###############################################################################
# DATA RETENTION (cmd/db_cleanup only)
#
# Everything below is read by the cleanup tool, not by the worker - the
# worker ignores these keys, just as the cleanup tool ignores everything
# above except mysql_dsn, log_level and log_format. Both binaries share
# this one file:
#
# go run ./cmd/db_cleanup -config /path/to/config.yaml
#
# Run it from cron or a systemd timer, once a day is plenty. In a cluster,
# run it on exactly one node - or on several at clearly different times.
#
# Every value is a number of DAYS of history to keep, configured
# separately for hosts and services. The key names are the ones the legacy
# PHP worker uses, so an existing config.yml can be carried over value for
# value - including its convention:
#
# 0 disables cleanup of that table entirely.
#
# A missing key falls back to the built-in default shown here; only an
# explicit 0 switches a table off.
###############################################################################
# Every executed check. By far the largest tables - these two are the
# reason the cleanup exists.
# statusengine_hostchecks.start_time / statusengine_servicechecks.start_time
age_hostchecks: 5
age_servicechecks: 5
# Acknowledgements.
# statusengine_host_acknowledgements.entry_time / statusengine_service_acknowledgements.entry_time
age_host_acknowledgements: 60
age_service_acknowledgements: 60
# Notifications, one row per notified contact.
# statusengine_host_notifications.start_time / statusengine_service_notifications.start_time
age_host_notifications: 60
age_service_notifications: 60
# Notification log, one row per notification event.
# statusengine_host_notifications_log.start_time / statusengine_service_notifications_log.start_time
age_host_notifications_log: 60
age_service_notifications_log: 60
# State changes. Kept far longer than checks because availability reports
# are computed from these.
# statusengine_host_statehistory.state_time / statusengine_service_statehistory.state_time
age_host_statehistory: 365
age_service_statehistory: 365
# Past downtimes. Despite the legacy key name, this cleans the *history*
# tables - currently scheduled downtimes are never touched.
# statusengine_host_downtimehistory.entry_time / statusengine_service_downtimehistory.entry_time
age_host_downtimes: 60
age_service_downtimes: 60
# Naemon/Nagios log entries.
# statusengine_logentries.entry_time
age_logentries: 5
# Performance data. Only relevant when perfdata_route above writes to
# MySQL; with Graphite, its own retention applies instead.
# statusengine_perfdata.timestamp_unix
age_perfdata: 90
# Rows deleted per DELETE statement. Each batch is its own transaction, so
# a smaller value holds locks for shorter, keeps the undo log small and
# produces binlog events replicas can digest - at the cost of more
# round-trips. Must be >= 1.
cleanup_batch_size: 5000
# Pause between two batches of the same table, as a duration string
# ("50ms", "1s"). The default of no pause deletes as fast as the database
# allows, which is usually right for a nightly run on an idle system; set
# a pause if the cleanup competes with live check results.
cleanup_batch_pause: 0s