diff --git a/docs/Collecting Metrics/Chart Template Format.mdx b/docs/Collecting Metrics/Chart Template Format.mdx index 49145ead82..4161786adc 100644 --- a/docs/Collecting Metrics/Chart Template Format.mdx +++ b/docs/Collecting Metrics/Chart Template Format.mdx @@ -1225,7 +1225,11 @@ All rules below produce semantic validation errors unless noted: ## Programmatic API > [!NOTE] -> Most collectors ship a static `charts.yaml` and never touch the Go API. This section is for collectors that **build a chart template at runtime** — for example from discovery results or selected profiles — and return it from `CollectorV2.ChartTemplateYAML()`. +> Most collectors ship a static `charts.yaml` and never touch the Go API. This section is for collectors that **build a chart template at runtime** — for example from discovery results or selected profiles — and return it from `StaticChartTemplateProvider.ChartTemplateYAML()`. + +For changing active membership, use native `chartengine.NewTemplateSet` and +`collectorapi.ChartTemplateSetProvider`; see [named active sets](https://github.com/netdata/netdata/blob/master/src/go/plugin/framework/chartengine/README.md#named-active-template-sets). +Static document composition remains supported through the API below. The package exposes a small Go surface for decoding, cloning, and re-emitting templates: @@ -1233,12 +1237,13 @@ The package exposes a small Go surface for decoding, cloning, and re-emitting te |----------------------------------------------------|-------------------------------------------------------------------------------------------------| | `DecodeYAML([]byte) (*Spec, error)` | Strict parse, apply decode-time defaults, then validate. The canonical read path. | | `DecodeYAMLValidated([]byte) (*Spec, Validation, error)` | Decode plus immutable derived validation artifacts for runtime consumers such as chartengine. | +| `NormalizeGroups([]Group) ([]Group, error)` | Return owned, default-applied, validated native groups without serialization. | | `Group.Clone() Group` | Typed deep copy of a group and everything nested under it. | | `Spec.MarshalTemplate() (string, error)` | Validate (only) and serialize a runtime-built template to YAML. | -### Building a template at runtime +### Building a static document at runtime -`CollectorV2.ChartTemplateYAML()` returns a plain `string`, so build the template where the error can be handled — typically once during `Init` — and cache the result; `ChartTemplateYAML()` then returns the cached string. Assemble a `Spec` from `charttpl` types and serialize it with `MarshalTemplate`: +`StaticChartTemplateProvider.ChartTemplateYAML()` returns a plain `string`, so build the template where the error can be handled — typically once during `Init` — and cache the result; `ChartTemplateYAML()` then returns the cached string. Assemble a `Spec` from `charttpl` types and serialize it with `MarshalTemplate`: ```go func buildChartTemplate(groups []charttpl.Group) (string, error) { diff --git a/docs/Collecting Metrics/Collectors/Databases/Microsoft SQL Server.mdx b/docs/Collecting Metrics/Collectors/Databases/Microsoft SQL Server.mdx index 696b13bd75..5a57be444b 100644 --- a/docs/Collecting Metrics/Collectors/Databases/Microsoft SQL Server.mdx +++ b/docs/Collecting Metrics/Collectors/Databases/Microsoft SQL Server.mdx @@ -5,8 +5,8 @@ learn_status: "Published" toc_max_heading_level: "6" toc_collapsible: "true" learn_rel_path: "Collecting Metrics/Collectors/Databases" -keywords: [db, database, mssql, sql server, microsoft] -description: "This collector monitors the health and performance of Microsoft SQL Server instances." +keywords: [mssql, sql server, microsoft sql server, microsoft, azure sql, azure sql managed instance, always on, sql server agent, query store] +description: "Monitor Microsoft SQL Server performance, databases, SQL Server Agent jobs, replication, and Always On Availability Groups." message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE" sidebar_position: "240" learn_link: "https://learn.netdata.cloud/docs/collecting-metrics/collectors/databases/microsoft-sql-server" @@ -27,73 +27,67 @@ Module: mssql ## Overview -This collector monitors the health and performance of Microsoft SQL Server instances. - -It collects metrics from: -- Performance counters (buffer manager, memory manager, SQL statistics) -- Dynamic management views (DMVs) for wait statistics, locks, and sessions -- Per-database transaction and lock statistics -- SQL Server Agent job status and execution history -- Always On Availability Group health, replica states, and per-database synchronization metrics -- Replication publication status, warnings, latency, and subscription counts - -All SQL Server editions are supported, including Express, Developer, Standard, Enterprise, and Web. -On editions that lack certain features, the collector omits the unavailable metrics and continues -collecting the rest: -- SQL Server Express and other editions without SQL Server Agent: job metrics are not collected. -- Disabled SQL Server Agent jobs retain their administrative enabled/disabled status by default, while their - execution charts are excluded. Set `collect_disabled_jobs: true` to retain their execution charts; stock - execution alerts still apply only while a job is enabled. -- Editions without Always On Availability Groups (e.g., Express): AG metrics are not collected. -- Editions without replication configured: replication metrics are not collected. - -Core metrics — performance counters, wait statistics, locks, per-database transactions, I/O latency, -and memory — are collected on every edition without special configuration. The top-queries function -uses Query Store where it is available and enabled, and otherwise falls back to the plan cache -(`sys.dm_exec_query_stats`), which every edition since SQL Server 2008 provides. - - -It connects to the SQL Server instance via TCP using the go-mssqldb driver and executes queries against: - -- `sys.dm_os_performance_counters` - Performance counter values -- `sys.dm_exec_sessions` - Connection information -- `sys.dm_os_wait_stats` - Wait statistics -- `sys.dm_tran_locks` - Lock information -- `sys.dm_io_virtual_file_stats` - I/O stall (latency) statistics -- `sys.dm_os_process_memory` - SQL Server process memory -- `sys.dm_os_sys_memory` - OS physical memory and page file -- `sys.master_files` - Database file sizes -- `msdb.dbo.sysjobs` - SQL Agent job status -- `msdb.dbo.sysjobhistory` - SQL Agent completed job execution history -- `msdb.dbo.sysjobactivity` - SQL Agent current job activity -- `sys.dm_hadr_availability_group_states` - AG health rollup -- `sys.dm_hadr_availability_replica_states` - Replica operational state -- `sys.dm_hadr_database_replica_states` - Database sync queues and rates -- `sys.dm_hadr_cluster` / `sys.dm_hadr_cluster_members` - WSFC cluster health -- `sys.dm_hadr_database_replica_cluster_states` - Failover readiness -- `sys.dm_hadr_auto_page_repair` - Automatic page repair events -- `sys.dm_hadr_ag_threads` - AG thread usage (SQL Server 2019+) +Monitor Microsoft SQL Server performance, databases, SQL Server Agent jobs, replication, and Always On +Availability Groups. Charts cover connections, throughput, buffer and memory pressure, waits and locks, +per-database transactions and transaction logs, and the health of every job, publication, and availability +group. +All SQL Server editions (Express, Web, Standard, Enterprise, Developer) and Azure SQL Managed Instance are +supported. Azure SQL Database is not supported: required instance-level memory and file views are unavailable, +so the job cannot start. Charts appear only for what the instance actually has and the monitoring login can read: -This collector is supported on all platforms. +| Area | Charts appear | +|:-----|:--------------| +| Instance: connections, batch requests, compilations, SQL errors, buffer manager, memory, process and OS memory | Always | +| Database: transactions, transaction log usage and growth, data and log file sizes, I/O stall, state | For every database | +| Locks and waits | For every lock resource type and wait type observed | +| SQL Server Agent jobs: enabled state, last execution result, duration, age, current run time | When the instance has SQL Server Agent (not on Express) and the login can read `msdb` | +| Replication: publication status, warnings, latency, subscriptions | When the instance is a distributor and the login can read the `distribution` database | +| Always On Availability Groups: group, replica, database replica, WSFC cluster health, automatic page repair | When Always On is enabled on the instance | -This collector supports collecting metrics from multiple instances of this integration, including remote instances. +Three Functions add on-demand troubleshooting from the dashboard: `top-queries` lists the most expensive query +patterns from Query Store or the plan cache, `deadlock-info` shows the most recent deadlock graph, and +`error-info` lists recent SQL errors from an Extended Events session. See Live Data below. + + +The collector connects to the instance over TCP with the Tabular Data Stream (TDS) protocol, using the `dsn` +connection string. It authenticates with a SQL Server login, with Windows integrated authentication when the +DSN carries no credentials (Windows only), or with a Microsoft Entra ID token when `cloud_auth` is configured +for Azure SQL Managed Instance. + +Every `update_every` seconds it runs read-only queries against system views on one connection, each bounded +by `timeout`: + +| Area | Views | +|:-----|:------| +| Counters, sessions, waits, locks | `sys.dm_os_performance_counters`, `sys.dm_exec_sessions`, `sys.dm_exec_requests`, `sys.dm_os_wait_stats`, `sys.dm_tran_locks` | +| Memory and files | `sys.dm_os_process_memory`, `sys.dm_os_sys_memory`, `sys.dm_io_virtual_file_stats`, `sys.master_files`, `sys.databases` | +| SQL Server Agent | `msdb.dbo.sysjobs`, `msdb.dbo.sysjobhistory`, `msdb.dbo.sysjobactivity` | +| Replication | `distribution.dbo.MSreplication_monitordata`, `distribution.dbo.MSpublications`, `distribution.dbo.MSsubscriptions` | +| Always On | `sys.availability_groups`, `sys.availability_replicas`, and the `sys.dm_hadr_*` views for group, replica, database, cluster, failover readiness, page repair and thread state | + +Functions open a second connection the first time one is used, keeping the metrics connection available. +Both workloads still share SQL Server CPU, I/O and locks. The collector never writes to the server: it creates no +objects, changes no settings, and leaves the Extended Events sessions it reads untouched. -The monitoring user requires the VIEW SERVER STATE permission to access DMVs. -SQL Agent job metrics require access to `msdb.dbo.sysjobs`. +This collector is supported on all platforms. -SQL Agent job execution metrics additionally require access to `msdb.dbo.sysjobhistory` and `msdb.dbo.sysjobactivity`. -If those optional `msdb` grants are missing, the collector continues collecting other SQL Server metrics and omits only the affected SQL Agent job metrics. +This collector supports collecting metrics from multiple instances of this integration, including remote instances. -Current job execution time is based on the latest visible SQL Server Agent activity session. -If SQL Server Agent stops or crashes while a job is recorded as running, `sysjobactivity` can report that job as running until Agent creates a newer session or activity/history state changes. +The monitoring login needs these grants. `VIEW SERVER STATE` is the only mandatory one; without an optional +grant the collector keeps running and omits the affected charts. -Last execution warning detection checks failed step history between the previous and latest completed job summary rows. -If SQL Server Agent step history is suppressed or purged before collection, a job that completed successfully with failed intermediate steps can be reported as `ok` instead of `warning`. +| Grant | Needed for | +|:------|:-----------| +| `VIEW SERVER STATE` | All metrics (dynamic management views and performance counters). On SQL Server 2022 and later the narrower `VIEW SERVER PERFORMANCE STATE` can be granted instead. | +| `VIEW ANY DEFINITION` | Data file size and I/O stall values (read through `sys.master_files`; without it those charts exist but stay empty) and Always On Availability Group charts (availability group catalog views) | +| `SELECT` on `msdb.dbo.sysjobs` | SQL Server Agent job status chart | +| `SELECT` on `msdb.dbo.sysjobhistory` and `msdb.dbo.sysjobactivity` | SQL Server Agent job execution charts | +| `SELECT` on `distribution.dbo.MSreplication_monitordata`, `MSpublications` and `MSsubscriptions` | Replication charts (distributor instances only) | +| `VIEW DATABASE STATE` (SQL Server 2016 to 2019) or `VIEW DATABASE PERFORMANCE STATE` (2022 and later) in each user database | `top-queries` reading Query Store; see the Function's prerequisites under Live Data | -Always On AG monitoring requires VIEW ANY DEFINITION for access to availability group catalog views. -On SQL Server 2022+, HADR DMVs may additionally require VIEW SERVER PERFORMANCE STATE. +The login only reads. No `ALTER`, `EXECUTE`, or write permission is needed. Microsoft SQL Server can be monitored further using the following other integrations: @@ -105,20 +99,34 @@ Microsoft SQL Server can be monitored further using the following other integrat #### Auto-Detection -By default, it tries to connect to SQL Server on localhost:1433 without authentication. -You must configure proper credentials for monitoring. - +This integration doesn't support auto-detection. #### Limits -The default configuration for this integration does not impose any limits on data collection. +- SQL Server Agent execution charts (last result, duration, age, current run time) are created only for + enabled jobs. Disabled jobs keep their status chart; set `collect_disabled_jobs: yes` to chart their + executions too. +- The current run time of a job comes from the latest SQL Server Agent activity session, so a job whose run + was cut short by an Agent service crash can show as running until Agent starts a new session. +- A job that succeeded with failed intermediate steps is reported as `warning` only while that step history + still exists; if the history is purged before collection, it is reported as `ok`. +- `top-queries` returns at most `functions.top_queries.limit` query patterns (500) from the last + `functions.top_queries.time_window_days` days (7), with query text cut at 4096 characters. `error-info` + returns at most the same number of rows. `deadlock-info` returns only the most recent deadlock. + #### Performance Impact -The collector executes lightweight queries against system views. -Most queries complete in milliseconds and have minimal impact on server performance. -Enabling `collect_disabled_jobs` expands SQL Agent history/activity queries and adds four charts with eight -dimensions for every disabled job. The administrative job-status chart is always collected. +Negligible on the Agent host. On the monitored instance, each collection runs about 20 to 30 lightweight +queries on one connection against in-memory counters and system views; the per-database and Always On +queries grow with the number of databases and replicas but typically finish in milliseconds. Raise +`update_every` to lower the query rate. + +Functions are heavier and run only when requested: `top-queries` aggregates Query Store runtime statistics +or scans the plan cache, and `error-info` and `deadlock-info` read Extended Events files, which for the +built-in `system_health` session can hold about 1 GB. They use a separate connection but compete with your +workload for SQL Server CPU and I/O. Lower `functions.top_queries.time_window_days`, or set +`functions..disabled: yes` for Functions you do not need. ## Setup @@ -140,32 +148,26 @@ UI configuration requires paid Netdata Cloud plan. ### Prerequisites -#### Create monitoring user +#### Create a monitoring login -Create a SQL Server login with VIEW SERVER STATE permission: +For SQL authentication, create a login with the grants below. Skip this step if you use Windows +Authentication or Microsoft Entra ID (next steps). SQL logins need the instance to run in mixed +authentication mode ("SQL Server and Windows Authentication mode"). ```sql --- Create login CREATE LOGIN netdata_user WITH PASSWORD = 'YourStrongPassword!'; - --- Grant VIEW SERVER STATE (required for DMVs) GRANT VIEW SERVER STATE TO netdata_user; - --- Grant VIEW ANY DEFINITION (required for Always On AG monitoring) +-- Data file sizes, I/O stall, and Always On Availability Groups GRANT VIEW ANY DEFINITION TO netdata_user; --- Grant VIEW SERVER PERFORMANCE STATE (required for HADR DMVs on SQL Server 2022+) --- GRANT VIEW SERVER PERFORMANCE STATE TO netdata_user; - --- Grant access to msdb for SQL Agent job monitoring +-- SQL Server Agent jobs (skip on Express, which has no Agent) USE msdb; CREATE USER netdata_user FOR LOGIN netdata_user; GRANT SELECT ON dbo.sysjobs TO netdata_user; GRANT SELECT ON dbo.sysjobhistory TO netdata_user; GRANT SELECT ON dbo.sysjobactivity TO netdata_user; --- Optional: Grant access to distribution database for replication monitoring --- (only if replication is configured) +-- Replication (distributor instances only) USE distribution; CREATE USER netdata_user FOR LOGIN netdata_user; GRANT SELECT ON dbo.MSreplication_monitordata TO netdata_user; @@ -173,32 +175,21 @@ GRANT SELECT ON dbo.MSpublications TO netdata_user; GRANT SELECT ON dbo.MSsubscriptions TO netdata_user; ``` -**Required permissions:** -- `VIEW SERVER STATE` - Access to dynamic management views - -**Optional permissions:** -- `SELECT on msdb.dbo.sysjobs` - SQL Agent job status monitoring -- `SELECT on msdb.dbo.sysjobhistory` - SQL Agent completed job execution history -- `SELECT on msdb.dbo.sysjobactivity` - SQL Agent current job activity -- `VIEW ANY DEFINITION` - Always On Availability Group monitoring -- `VIEW SERVER PERFORMANCE STATE` - HADR DMVs on SQL Server 2022+ -- `SELECT on distribution.dbo.MSreplication_monitordata` - Replication monitoring -- `SELECT on distribution.dbo.MSpublications` - Publication information -- `SELECT on distribution.dbo.MSsubscriptions` - Subscription counts +To verify, run the collector's first query as the new login; it must return a row: +```sql +EXECUTE AS LOGIN = 'netdata_user'; +SELECT TOP 1 counter_name FROM sys.dm_os_performance_counters; +REVERT; +``` -#### Grant Windows Authentication access (optional) - -If you prefer Windows integrated authentication instead of SQL authentication, grant the -Netdata service account access to SQL Server. -By default, the Netdata service runs as `Local System`. The identity it presents to -SQL Server depends on whether the connection is local or remote: +#### Grant Windows Authentication access -**Local connection (Netdata and SQL Server on the same machine):** +Only when the DSN carries no username and password (Windows only). The Netdata service then authenticates +with its own Windows account, `Local System` by default, and that account needs a SQL Server login. -`Local System` always authenticates as `NT AUTHORITY\SYSTEM`, regardless of whether -the machine is domain-joined or in a workgroup. +For an instance on the same machine, `Local System` is seen as `NT AUTHORITY\SYSTEM`: ```sql CREATE LOGIN [NT AUTHORITY\SYSTEM] FROM WINDOWS; @@ -211,11 +202,8 @@ GRANT SELECT ON dbo.sysjobhistory TO [NT AUTHORITY\SYSTEM]; GRANT SELECT ON dbo.sysjobactivity TO [NT AUTHORITY\SYSTEM]; ``` -**Remote connection (Netdata connects to SQL Server on another machine):** - -On a domain-joined machine, `Local System` authenticates over the network as the -computer account (`DOMAIN\COMPUTERNAME$`). Replace with your actual values -(e.g., `MYDOM\SQLBOX01$`). +For an instance on another machine, a domain-joined Netdata host is seen as its computer account, +`DOMAIN\COMPUTERNAME$` (for example `MYDOM\SQLBOX01$`): ```sql CREATE LOGIN [DOMAIN\COMPUTERNAME$] FROM WINDOWS; @@ -228,13 +216,25 @@ GRANT SELECT ON dbo.sysjobhistory TO [DOMAIN\COMPUTERNAME$]; GRANT SELECT ON dbo.sysjobactivity TO [DOMAIN\COMPUTERNAME$]; ``` -For the default `Local System` service account, remote Windows Authentication works -only on domain-joined machines, where it can authenticate as the computer account. -In workgroups, use a different Windows service account if you need remote Windows -Authentication. +`Local System` cannot authenticate to a remote instance from a workgroup machine. Run the Netdata service +as a domain account instead, or use a SQL login. To see which account SQL Server receives, connect with +Windows Authentication and run `SELECT SYSTEM_USER`. -> **Note**: To verify which account SQL Server sees, connect with Windows Authentication -> and run `SELECT SYSTEM_USER`. + +#### Grant a Microsoft Entra identity access to Azure SQL Managed Instance + +Only when `cloud_auth.provider` is `azure_ad`. The service principal or managed identity that Netdata +signs in with needs a login created from the external provider, named after the app registration or the +managed identity. Create database users and grant the optional database permissions listed above for +SQL Server Agent jobs and replication. See Microsoft's guide to +[Microsoft Entra authentication for Azure SQL](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-aad-configure). + +```sql +-- Azure SQL Managed Instance +CREATE LOGIN [netdata-monitoring] FROM EXTERNAL PROVIDER; +GRANT VIEW SERVER STATE TO [netdata-monitoring]; +GRANT VIEW ANY DEFINITION TO [netdata-monitoring]; +``` @@ -242,7 +242,7 @@ Authentication. #### Options -The following options can be defined globally: update_every, autodetection_retry. +The following options can be defined globally: `update_every`, `autodetection_retry`.
@@ -252,29 +252,100 @@ The following options can be defined globally: update_every, autodetection_retry | Group | Option | Description | Default | Required | |:------|:-----|:------------|:--------|:---------:| -| **Collection** | update_every | Data collection interval (seconds). | 10 | no | -| | autodetection_retry | Autodetection retry interval (seconds). Set 0 to disable. | 0 | no | -| **Target** | dsn | SQL Server DSN (Data Source Name). See [DSN syntax](https://github.com/microsoft/go-mssqldb#connection-parameters-and-dsn). When `cloud_auth.provider` is `azure_ad`, use URL format with `sqlserver://` scheme. | sqlserver://localhost:1433 | yes | -| **Cloud Auth** | cloud_auth.provider | Cloud auth provider (`none` or `azure_ad`). | none | no | -| **Cloud Auth/Azure** | cloud_auth.azure_ad.mode | Azure AD credential mode (`service_principal`, `managed_identity`, or `default`). Required when `cloud_auth.provider` is `azure_ad`. | | yes | -| | cloud_auth.azure_ad.mode_service_principal.tenant_id | Azure tenant ID. Required for `service_principal` mode. | | no | -| | cloud_auth.azure_ad.mode_service_principal.client_id | Azure client ID. Required for `service_principal` mode. | | no | -| | cloud_auth.azure_ad.mode_service_principal.client_secret | Azure client secret for `service_principal` mode. | | no | -| | cloud_auth.azure_ad.mode_managed_identity.client_id | Optional client ID of a user-assigned managed identity (`managed_identity` mode). | | no | -| **Target** | timeout | Query timeout (seconds). | 5 | no | -| **SQL Agent** | collect_disabled_jobs | Collect execution-result, execution-age, execution-duration, and current-runtime charts for disabled SQL Server Agent jobs. Their administrative enabled/disabled status is always collected. Enabling this option adds four charts and eight dimensions per disabled job. Stock last-execution alerts remain gated by the job's enabled state. | no | no | -| **Functions** | functions.top_queries.disabled | Disable the [top-queries](#top-queries) function. | no | no | -| | functions.top_queries.timeout | Query timeout for top-queries function (seconds). Uses collector timeout if not set. | | no | -| | functions.top_queries.limit | Maximum number of queries to return in the top-queries response. | 500 | no | -| | functions.top_queries.time_window_days | Number of days of query statistics to analyze. On the plan-cache fallback this filters by last execution time rather than aggregating a period. Set to 0 to use the default (7), or -1 to include all available data. Smaller positive values improve query performance but show less history. | 7 | no | -| | functions.deadlock_info.disabled | Disable the [deadlock-info](#deadlock-info) function. | no | no | -| | functions.deadlock_info.timeout | Query timeout for deadlock-info function (seconds). Uses collector timeout if not set. | | no | -| | functions.deadlock_info.use_ring_buffer | Use the ring_buffer target instead of event_file for the built-in system_health session on SQL Server or Azure SQL Managed Instance.

WARNING:
• Data is cleared on failover/restart
• Capacity is limited
• XML parsing can increase query CPU

Azure SQL Database is not supported by deadlock-info because it has no built-in system_health session. | no | no | -| | functions.error_info.disabled | Disable the [error-info](#error-info) function. | no | no | -| | functions.error_info.timeout | Query timeout for error-info function (seconds). Uses collector timeout if not set. | | no | -| | functions.error_info.session_name | Extended Events session name capturing error_reported events.
Must be created by administrator with event_file (recommended) or ring_buffer target. | netdata_errors | no | -| | functions.error_info.use_ring_buffer | Use ring_buffer instead of event_file for error events.

WARNING:
• The session must remain running
• Data is cleared on failover/restart
• Capacity is limited
• XML parsing can increase query CPU

Useful when persistent event_file storage is unavailable, including Azure SQL Database without Blob Storage. | no | no | -| **Virtual Node** | vnode | Associates this data collection job with a [Virtual Node](https://learn.netdata.cloud/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts#virtual-nodes). | | no | +| **Base** | update_every | Data collection interval, in seconds. | 10 | no | +| | autodetection_retry | How often to retry the initial connection when the job fails to start, in seconds. Zero disables retries. | 0 | no | +| | [dsn](#option-base-dsn) | Connection string in [go-mssqldb DSN format](https://github.com/microsoft/go-mssqldb#connection-parameters-and-dsn), such as `sqlserver://user:password@host:1433`. With `cloud_auth.provider` set to `azure_ad`, only the `sqlserver://` URL form is accepted. | sqlserver://localhost:1433 | yes | +| | timeout | Query timeout, in seconds. | 5 | no | +| | vnode | Associates this job with a [Virtual Node](https://learn.netdata.cloud/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts#virtual-nodes). | | no | +| **SQL Agent** | [collect_disabled_jobs](#option-sql-agent-collect-disabled-jobs) | Also create execution charts for disabled SQL Server Agent jobs. The enabled/disabled status chart always covers every job. | no | no | +| **Cloud Auth** | cloud_auth.provider | Authentication provider for Azure SQL Managed Instance. `none` uses the credentials in `dsn`; `azure_ad` signs in with a Microsoft Entra ID token. | none | no | +| | [cloud_auth.azure_ad.mode](#option-cloud-auth-cloud-auth-azure-ad-mode) | How Netdata obtains the Microsoft Entra ID token. `service_principal` uses an app registration with a client secret, `managed_identity` the identity of the Azure resource running Netdata, and `default` the Azure SDK credential chain. | default | yes | +| | cloud_auth.azure_ad.mode_service_principal.tenant_id | Directory (tenant) ID of the Microsoft Entra tenant that holds the service principal. Required in `service_principal` mode. | | no | +| | cloud_auth.azure_ad.mode_service_principal.client_id | Application (client) ID of the service principal. Required in `service_principal` mode. | | no | +| | cloud_auth.azure_ad.mode_service_principal.client_secret | Client secret of the service principal. Required in `service_principal` mode. | | no | +| | cloud_auth.azure_ad.mode_managed_identity.client_id | Client ID of a user-assigned managed identity. Leave empty to use the system-assigned identity of the Azure resource. | | no | +| **Functions** | functions.top_queries.disabled | Disable the top-queries Function. | no | no | +| | functions.top_queries.timeout | Query timeout for top-queries requests, in seconds, independent of the metrics `timeout`. Zero uses the built-in 30 seconds. | 30 | no | +| | [functions.top_queries.limit](#option-functions-functions-top-queries-limit) | Maximum number of query patterns top-queries returns. Zero uses the built-in 500. | 500 | no | +| | [functions.top_queries.time_window_days](#option-functions-functions-top-queries-time-window-days) | Days of query statistics top-queries covers. Zero uses the built-in 7 days; -1 covers all retained history. | 7 | no | +| | functions.deadlock_info.disabled | Disable the deadlock-info Function. | no | no | +| | functions.deadlock_info.timeout | Query timeout for deadlock-info requests, in seconds, independent of the metrics `timeout`. Zero uses the built-in 30 seconds. | 30 | no | +| | [functions.deadlock_info.use_ring_buffer](#option-functions-functions-deadlock-info-use-ring-buffer) | Read deadlock reports from the ring_buffer target of the built-in `system_health` session instead of its event_file target. | no | no | +| | functions.error_info.disabled | Disable the error-info Function. | no | no | +| | functions.error_info.timeout | Query timeout for error-info requests, in seconds, independent of the metrics `timeout`. Zero uses the built-in 30 seconds. | 30 | no | +| | [functions.error_info.session_name](#option-functions-functions-error-info-session-name) | Name of the server-scoped Extended Events session that captures `error_reported` events. | netdata_errors | no | +| | [functions.error_info.use_ring_buffer](#option-functions-functions-error-info-use-ring-buffer) | Read error events from the session's ring_buffer target instead of its event_file target. | no | no | + + +##### dsn + +The URL form covers the common cases: + +| Scenario | DSN | +|:---------|:----| +| SQL login | `sqlserver://netdata_user:password@host:1433` | +| Windows Authentication (Windows only) | `sqlserver://host:1433`, with no username and password | +| Named instance | `sqlserver://netdata_user:password@host/INSTANCENAME`; the SQL Server Browser service must be reachable | +| Azure SQL Managed Instance (private endpoint) | `sqlserver://my-instance.dns-zone.database.windows.net:1433?database=master` | +| Encrypted connection to a trusted certificate | append `?encrypt=true` | + +The password may come from a secret store instead of the file, for example `${env:MSSQL_PASSWORD}`. +Percent-encode reserved characters in a literal password (`@` as `%40`). + + + +##### collect_disabled_jobs + +Each disabled job then gets the same four execution charts as an enabled one: last execution status, +duration, age, and current run time. The stock last-execution alerts still fire only for enabled jobs. + + + +##### cloud_auth.azure_ad.mode + +Required in the configuration file whenever `cloud_auth.provider` is `azure_ad`; the configuration form +preselects `default`. The `default` chain tries environment credentials, a managed identity, and local +developer sign-ins in turn, which suits development more than production. + + + +##### functions.top_queries.limit + +The same limit caps the number of rows `error-info` returns. + + + +##### functions.top_queries.time_window_days + +With Query Store, the window selects the runtime statistics intervals to aggregate. On the plan-cache +fallback it keeps only plans executed within the window, because the plan cache has no interval +history. Shorter windows make the query cheaper on busy servers. + + + +##### functions.deadlock_info.use_ring_buffer + +The ring buffer is volatile: its contents are lost on restart or failover and its capacity is small, so +older deadlocks disappear sooner than from the event files, and parsing it costs more CPU on the server. +Applies to SQL Server and Azure SQL Managed Instance. Azure SQL Database has no `system_health` +session, so `deadlock-info` is unavailable there regardless of this option. + + + +##### functions.error_info.session_name + +Create the session as described under Live Data. When it is missing or its target is unavailable, SQL +Server and Managed Instance fall back to the built-in `system_health` session, which records only +selected errors. + + + +##### functions.error_info.use_ring_buffer + +Use it where event files cannot be stored, such as Managed Instance without configured Blob Storage. The session +must be running; events are lost on restart or failover, capacity is limited, and XML parsing costs more +CPU than reading event files. +
@@ -319,7 +390,7 @@ sudo ./edit-config go.d/mssql.conf ###### Basic configuration -Connect to local SQL Server with SQL authentication. +A local instance with a SQL login.
Config @@ -334,14 +405,9 @@ jobs: ###### Windows Authentication -Connect using Windows integrated authentication (Windows only). - -When no username/password is provided in the DSN, the driver uses the Netdata service account's -Windows credentials. By default, the Netdata service runs as `Local System`, which authenticates -to a local SQL Server as `NT AUTHORITY\SYSTEM`. - -See the [Grant Windows Authentication access](#grant-windows-authentication-access-optional) prerequisite -to configure SQL Server for this. +A local instance on Windows, signing in as the Netdata service account. Leave the username and password +out of the DSN and complete the [Grant Windows Authentication access](#grant-windows-authentication-access) +prerequisite.
@@ -357,7 +423,7 @@ jobs: ###### Named instance -Connect to a named SQL Server instance. +A named instance, resolved through the SQL Server Browser service.
Config @@ -372,7 +438,7 @@ jobs: ###### Remote server -Connect to a remote SQL Server. +An instance on another host, reachable on TCP port 1433.
Config @@ -385,10 +451,10 @@ jobs: ```
-###### Azure SQL with service principal +###### Azure SQL Managed Instance with a service principal -Use Microsoft Entra service principal authentication for Azure SQL. `top-queries` and `error-info` -operate in the database selected in the DSN; `deadlock-info` is unavailable on Azure SQL Database. +A managed instance, authenticated with a Microsoft Entra app registration through its private endpoint. +Replace the hostname with the managed instance's fully qualified domain name.
@@ -397,7 +463,7 @@ operate in the database selected in the DSN; `deadlock-info` is unavailable on A ```yaml jobs: - name: azure_sql_sp - dsn: "sqlserver://my-server.database.windows.net:1433?database=mydb" + dsn: "sqlserver://my-instance.dns-zone.database.windows.net:1433?database=master" cloud_auth: provider: azure_ad azure_ad: @@ -405,15 +471,15 @@ jobs: mode_service_principal: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" - client_secret: "super-secret-value" + client_secret: "${env:AZURE_CLIENT_SECRET}" ```
-###### Azure SQL with managed identity +###### Azure SQL Managed Instance with a managed identity -Use managed identity authentication (system-assigned by default). `top-queries` and `error-info` -operate in the database selected in the DSN; `deadlock-info` is unavailable on Azure SQL Database. +Netdata runs on an Azure resource and signs in with its system-assigned managed identity. Set +`mode_managed_identity.client_id` to use a user-assigned identity instead.
@@ -422,7 +488,7 @@ operate in the database selected in the DSN; `deadlock-info` is unavailable on A ```yaml jobs: - name: azure_sql_mi - dsn: "sqlserver://my-server.database.windows.net:1433?database=mydb" + dsn: "sqlserver://my-instance.dns-zone.database.windows.net:1433?database=master" cloud_auth: provider: azure_ad azure_ad: @@ -435,7 +501,7 @@ jobs: > **Note**: When you define multiple jobs, their names must be unique. -Monitoring multiple SQL Server instances. +Several instances monitored by one Agent.
@@ -452,12 +518,10 @@ jobs: ```
-###### With custom function settings - -Configure function-specific settings like timeouts and limits. +###### Metrics only, no Functions -> **Warning**: Query Store may contain unmasked literal values (PII). -> Disable functions if not needed or ensure proper access controls. +The Functions return raw query text, which can contain personal or business data. Disable them where +the dashboard audience must not see it.
@@ -469,12 +533,32 @@ jobs: dsn: "sqlserver://netdata_user:password@localhost:1433" functions: top_queries: - limit: 100 - time_window_days: 7 + disabled: yes deadlock_info: - use_ring_buffer: true + disabled: yes error_info: - session_name: custom_errors + disabled: yes + +``` +
+ +###### Custom Extended Events session for error-info + +The error capture session was created under a name other than `netdata_errors`, with a ring_buffer +target. + + +
+Config + +```yaml +jobs: + - name: local + dsn: "sqlserver://netdata_user:password@localhost:1433" + functions: + error_info: + session_name: app_errors + use_ring_buffer: yes ```
@@ -489,7 +573,7 @@ The following alerts are available: | Alert name | On metric | Description | |:------------|:----------|:------------| | [ mssql_database_log_percent_used ](https://github.com/netdata/netdata/blob/master/src/health/health.d/mssql.conf) | mssql.database_log_percent_used | SQL Server transaction log percent used has been above 90% for the last 15 minutes | -| [ mssql_sql_agent_job_last_execution_warning ](https://github.com/netdata/netdata/blob/master/src/health/health.d/mssql.conf) | mssql.job_last_execution_status | Enabled SQL Server Agent job succeeded but at least one step failed in the last completed execution | +| [ mssql_sql_agent_job_last_execution_warning ](https://github.com/netdata/netdata/blob/master/src/health/health.d/mssql.conf) | mssql.job_last_execution_status | Enabled SQL Server Agent job succeeded, but at least one step failed in the last completed execution | | [ mssql_sql_agent_job_last_execution_failed ](https://github.com/netdata/netdata/blob/master/src/health/health.d/mssql.conf) | mssql.job_last_execution_status | Enabled SQL Server Agent job failed in the last completed execution | @@ -500,49 +584,51 @@ Metrics grouped by *scope*. The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels. +Charts for SQL Server Agent jobs, replication and Always On appear only where the feature exists and the +monitoring login can read it; the Overview lists the grant each one needs. ### Per Microsoft SQL Server instance -These metrics refer to the entire SQL Server instance. +The whole SQL Server instance. This scope has no labels. Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.user_connections | User Connections | user | connections | • | • | -| mssql.session_connections | Session Connections | user, internal | connections | • | • | -| mssql.blocked_processes | Blocked Processes | blocked | processes | • | • | -| mssql.batch_requests | Batch Requests | batch | requests/s | • | • | -| mssql.compilations | SQL Compilations | compilations | compilations/s | • | • | -| mssql.recompilations | SQL Re-Compilations | recompilations | recompilations/s | • | • | -| mssql.auto_param_attempts | Auto-Parameterization Attempts | total, safe, failed | attempts/s | • | • | -| mssql.sql_errors | SQL Errors | errors | errors/s | • | • | -| mssql.buffer_cache_hit_ratio | Buffer Cache Hit Ratio | hit_ratio | percentage | • | • | -| mssql.buffer_page_life_expectancy | Page Life Expectancy | life_expectancy | seconds | • | • | -| mssql.buffer_page_iops | Buffer Page I/O | read, written | pages/s | • | • | -| mssql.buffer_checkpoint_pages | Buffer Checkpoint Pages Flushed | flushed | pages/s | • | • | -| mssql.buffer_page_lookups | Buffer Page Lookups | lookups | lookups/s | • | • | -| mssql.buffer_lazy_writes | Buffer Lazy Writes | lazy_writes | writes/s | • | • | -| mssql.memory_total | Total Server Memory | memory | bytes | • | • | -| mssql.memory_connection | Connection Memory | memory | bytes | • | • | -| mssql.memory_pending_grants | Pending Memory Grants | pending | processes | • | • | -| mssql.memory_external_benefit | External Benefit of Memory | benefit | benefit | • | • | -| mssql.page_splits | Page Splits | page | splits/s | • | • | -| mssql.process_memory_resident | SQL Server Process Resident Memory (Working Set) | resident | bytes | • | • | -| mssql.process_memory_virtual | SQL Server Process Virtual Memory Committed | virtual | bytes | • | • | -| mssql.process_memory_utilization | SQL Server Process Memory Utilization | utilization | percentage | • | • | -| mssql.process_page_faults | SQL Server Process Page Faults | page_faults | faults | • | • | -| mssql.os_memory | OS Physical Memory | used, available | bytes | • | • | -| mssql.os_pagefile | OS Page File | used, available | bytes | • | • | +| mssql.user_connections | User connections | user | connections | • | • | +| mssql.session_connections | Session connections | user, internal | connections | • | • | +| mssql.blocked_processes | Blocked processes | blocked | processes | • | • | +| mssql.batch_requests | Batch requests | batch | requests/s | • | • | +| mssql.compilations | SQL compilations | compilations | compilations/s | • | • | +| mssql.recompilations | SQL re-compilations | recompilations | recompilations/s | • | • | +| mssql.auto_param_attempts | Auto-parameterization attempts | total, safe, failed | attempts/s | • | • | +| mssql.sql_errors | SQL errors | errors | errors/s | • | • | +| mssql.buffer_cache_hit_ratio | Buffer cache hit ratio | hit_ratio | percentage | • | • | +| mssql.buffer_page_life_expectancy | Buffer page life expectancy | life_expectancy | seconds | • | • | +| mssql.buffer_page_iops | Buffer page I/O | read, written | pages/s | • | • | +| mssql.buffer_checkpoint_pages | Buffer checkpoint pages flushed | flushed | pages/s | • | • | +| mssql.buffer_page_lookups | Buffer page lookups | lookups | lookups/s | • | • | +| mssql.buffer_lazy_writes | Buffer lazy writes | lazy_writes | writes/s | • | • | +| mssql.memory_total | Total server memory | memory | bytes | • | • | +| mssql.memory_connection | Connection memory | memory | bytes | • | • | +| mssql.memory_pending_grants | Pending memory grants | pending | processes | • | • | +| mssql.memory_external_benefit | External benefit of memory | benefit | benefit | • | • | +| mssql.page_splits | Page splits | page | splits/s | • | • | +| mssql.process_memory_resident | Process resident memory (working set) | resident | bytes | • | • | +| mssql.process_memory_virtual | Process virtual memory committed | virtual | bytes | • | • | +| mssql.process_memory_utilization | Process memory utilization | utilization | percentage | • | • | +| mssql.process_page_faults | Process page faults | page_faults | faults | • | • | +| mssql.os_memory | OS physical memory | used, available | bytes | • | • | +| mssql.os_pagefile | OS page file | used, available | bytes | • | • | ### Per database -These metrics refer to individual databases. +One database on the instance. Labels: @@ -552,27 +638,27 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.database_active_transactions | Active Transactions | active | transactions | • | • | +| mssql.database_active_transactions | Active transactions | active | transactions | • | • | | mssql.database_transactions | Transactions | transactions | transactions/s | • | • | -| mssql.database_write_transactions | Write Transactions | write | transactions/s | • | • | -| mssql.database_log_flushes | Log Flushes | flushes | flushes/s | • | • | -| mssql.database_log_flushed | Log Bytes Flushed | flushed | bytes/s | • | • | -| mssql.database_log_growths | Log Growths | growths | growths | • | • | -| mssql.database_log_file_size | Transaction Log File Size | used, free | bytes | • | • | -| mssql.database_log_percent_used | Transaction Log Space Utilization | used | percentage | • | • | -| mssql.database_log_truncations_shrinks | Transaction Log Truncations and Shrinks | truncations, shrinks | events/s | • | • | -| mssql.database_io_stall | I/O Stall Time | read, write | ms | • | • | -| mssql.database_data_file_size | Data File Size | size | bytes | • | • | -| mssql.database_backup_restore_throughput | Backup/Restore Throughput | throughput | bytes/s | • | • | -| mssql.database_state | Database State | online, restoring, recovering, pending, suspect, emergency, offline | state | • | • | -| mssql.database_read_only | Database Read-Only Status | read_only, read_write | status | • | • | +| mssql.database_write_transactions | Write transactions | write | transactions/s | • | • | +| mssql.database_log_flushes | Log flushes | flushes | flushes/s | • | • | +| mssql.database_log_flushed | Log bytes flushed | flushed | bytes/s | • | • | +| mssql.database_log_growths | Database log growths | growths | growths | • | • | +| mssql.database_log_file_size | Transaction log file size | used, free | bytes | • | • | +| mssql.database_log_percent_used | Transaction log space utilization | used | percentage | • | • | +| mssql.database_log_truncations_shrinks | Transaction log truncations and shrinks | truncations, shrinks | events/s | • | • | +| mssql.database_io_stall | Database I/O stall time | read, write | ms | • | • | +| mssql.database_data_file_size | Data file size | size | bytes | • | • | +| mssql.database_backup_restore_throughput | Backup/Restore throughput | throughput | bytes/s | • | • | +| mssql.database_state | Database state | online, restoring, recovering, pending, suspect, emergency, offline | state | • | • | +| mssql.database_read_only | Database read-only status | read_only, read_write | status | • | • | ### Per lock stats -These metrics refer to lock statistics by lock resource type (from performance counters). +One lock resource type, as reported by the Locks performance counters. Labels: @@ -582,17 +668,17 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.lock_stats_deadlocks | Deadlocks by Resource Type | deadlocks | deadlocks/s | • | • | -| mssql.lock_stats_waits | Lock Waits by Resource Type | waits | waits/s | • | • | -| mssql.lock_stats_timeouts | Lock Timeouts by Resource Type | timeouts | timeouts/s | • | • | -| mssql.lock_stats_requests | Lock Requests by Resource Type | requests | requests/s | • | • | +| mssql.lock_stats_deadlocks | Deadlocks by lock resource type | deadlocks | deadlocks/s | • | • | +| mssql.lock_stats_waits | Lock waits by lock resource type | waits | waits/s | • | • | +| mssql.lock_stats_timeouts | Lock timeouts by lock resource type | timeouts | timeouts/s | • | • | +| mssql.lock_stats_requests | Lock requests by lock resource type | requests | requests/s | • | • | ### Per lock resource -These metrics refer to lock resource types (from sys.dm_tran_locks). +One lock resource type with locks currently granted or waiting, from `sys.dm_tran_locks`. Labels: @@ -602,14 +688,14 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.locks_by_resource | Lock Count by Resource Type | locks | locks | • | • | +| mssql.locks_by_resource | Locks by resource type | locks | locks | • | • | ### Per wait type -These metrics refer to individual wait types (from sys.dm_os_wait_stats). +One wait type observed since the instance started, grouped into a wait category. Labels: @@ -620,18 +706,18 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.wait_total_time | Total Wait Time | duration | ms | • | • | -| mssql.wait_resource_time | Resource Wait Time | duration | ms | • | • | -| mssql.wait_signal_time | Signal Wait Time | duration | ms | • | • | -| mssql.wait_max_time | Maximum Wait Time | max_time | ms | • | • | -| mssql.wait_count | Wait Count | waits | waits/s | • | • | +| mssql.wait_total_time | Total wait time | duration | ms | • | • | +| mssql.wait_resource_time | Resource wait time | duration | ms | • | • | +| mssql.wait_signal_time | Signal wait time | duration | ms | • | • | +| mssql.wait_max_time | Maximum wait time | max_time | ms | • | • | +| mssql.wait_count | Wait count | waits | waits/s | • | • | ### Per job -These metrics refer to SQL Server Agent jobs. +One SQL Server Agent job. Execution charts exist for enabled jobs, and for disabled jobs only when `collect_disabled_jobs` is set. Not available on Azure SQL Database. Labels: @@ -641,18 +727,18 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.job_status | Job Status | enabled, disabled | status | • | | -| mssql.job_last_execution_status | Job Last Execution Status | unknown, ok, warning, error, canceled | status | • | | -| mssql.job_last_execution_duration | Job Last Execution Duration | duration | seconds | • | | -| mssql.job_last_execution_age | Job Last Execution Age | age | seconds | • | | -| mssql.job_current_execution_time | Job Current Execution Time | duration | seconds | • | | +| mssql.job_status | Job status | enabled, disabled | status | • | | +| mssql.job_last_execution_status | Job last execution status | unknown, ok, warning, error, canceled | status | • | | +| mssql.job_last_execution_duration | Job last execution duration | duration | seconds | • | | +| mssql.job_last_execution_age | Job last execution age | age | seconds | • | | +| mssql.job_current_execution_time | Job current execution time | duration | seconds | • | | ### Per replication -These metrics refer to SQL Server replication publications. +One publication on a distributor instance. Labels: @@ -663,17 +749,17 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.replication_status | Replication Status | started, succeeded, in_progress, idle, retrying, failed | status | • | • | -| mssql.replication_warning | Replication Warnings | expiration, latency, merge_expiration, merge_slow_duration, merge_fast_duration, merge_fast_speed, merge_slow_speed | flags | • | • | -| mssql.replication_latency | Replication Latency | average, best, worst | seconds | • | • | -| mssql.replication_subscriptions | Replication Subscriptions | total, agents_running | subscriptions | • | • | +| mssql.replication_status | Replication status | started, succeeded, in_progress, idle, retrying, failed | status | • | • | +| mssql.replication_warning | Replication warnings | expiration, latency, merge_expiration, merge_slow_duration, merge_fast_duration, merge_fast_speed, merge_slow_speed | flags | • | • | +| mssql.replication_latency | Replication latency | average, best, worst | seconds | • | • | +| mssql.replication_subscriptions | Replication subscriptions | total, agents_running | subscriptions | • | • | ### Per availability group -These metrics refer to Always On Availability Groups. Auto-detected when HADR is enabled. +One Always On Availability Group the instance takes part in. The threads chart requires SQL Server 2019 or later. Labels: @@ -683,16 +769,16 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_sync_health | Availability Group Synchronization Health | not_healthy, partially_healthy, healthy | state | • | • | -| mssql.ag_recovery_health | Availability Group Recovery Health | primary_online, primary_in_progress, secondary_online, secondary_in_progress | state | • | • | -| mssql.ag_threads | Availability Group Threads (SQL Server 2019+) | capture, redo, parallel_redo | threads | • | • | +| mssql.ag_sync_health | Availability group synchronization health | not_healthy, partially_healthy, healthy | state | • | • | +| mssql.ag_recovery_health | Availability group recovery health | primary_online, primary_in_progress, secondary_online, secondary_in_progress | state | • | • | +| mssql.ag_threads | Availability group threads | capture, redo, parallel_redo | threads | • | • | ### Per availability group replica -These metrics refer to per-replica state within an Availability Group. Note: on secondary replicas, the replica states DMV returns only local information. +One replica of an availability group. On a secondary replica the state views describe only the local replica. Labels: @@ -705,16 +791,16 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_replica_role | Availability Group Replica Role | primary, secondary, resolving, unknown | state | • | • | -| mssql.ag_replica_connected_state | Availability Group Replica Connected State | connected, disconnected, unknown | state | • | • | -| mssql.ag_replica_sync_health | Availability Group Replica Synchronization Health | not_healthy, partially_healthy, healthy | state | • | • | +| mssql.ag_replica_role | Availability group replica role | primary, secondary, resolving, unknown | state | • | • | +| mssql.ag_replica_connected_state | Availability group replica connected state | connected, disconnected, unknown | state | • | • | +| mssql.ag_replica_sync_health | Availability group replica synchronization health | not_healthy, partially_healthy, healthy | state | • | • | ### Per availability group database replica -These metrics refer to per-database synchronization within an Availability Group. +One database within an availability group replica; secondary lag requires SQL Server 2016+, and redo rate averages bytes redone over active redo time since SQL Server startup. Labels: @@ -726,36 +812,36 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_db_sync_state | AG Database Synchronization State | not_synchronizing, synchronizing, synchronized, reverting, initializing | state | • | • | -| mssql.ag_db_log_send_queue | AG Database Log Send Queue Size | queue_size | bytes | • | • | -| mssql.ag_db_log_send_rate | AG Database Log Send Rate | send_rate | bytes/s | • | • | -| mssql.ag_db_redo_queue | AG Database Redo Queue Size | queue_size | bytes | • | • | -| mssql.ag_db_redo_rate | AG Database Redo Rate (averaged over active redo time since startup) | redo_rate | bytes/s | • | • | -| mssql.ag_db_filestream_send_rate | AG Database Filestream Send Rate | send_rate | bytes/s | • | • | -| mssql.ag_db_secondary_lag | AG Database Secondary Lag (SQL Server 2016+) | lag | seconds | • | • | -| mssql.ag_db_suspended | AG Database Data Movement Suspended State | active, suspended | state | • | • | -| mssql.ag_db_failover_readiness | AG Database Failover Readiness | ready, not_ready | state | • | • | -| mssql.ag_db_joined_state | AG Database Joined State | joined, not_joined | state | • | • | +| mssql.ag_db_sync_state | AG database synchronization state | not_synchronizing, synchronizing, synchronized, reverting, initializing | state | • | • | +| mssql.ag_db_log_send_queue | AG database log send queue size | queue_size | bytes | • | • | +| mssql.ag_db_log_send_rate | AG database log send rate | send_rate | bytes/s | • | • | +| mssql.ag_db_redo_queue | AG database redo queue size | queue_size | bytes | • | • | +| mssql.ag_db_redo_rate | AG database redo rate | redo_rate | bytes/s | • | • | +| mssql.ag_db_filestream_send_rate | AG database filestream send rate | send_rate | bytes/s | • | • | +| mssql.ag_db_secondary_lag | AG database secondary lag | lag | seconds | • | • | +| mssql.ag_db_suspended | AG database data movement suspended state | active, suspended | state | • | • | +| mssql.ag_db_failover_readiness | AG database failover readiness | ready, not_ready | state | • | • | +| mssql.ag_db_joined_state | AG database joined state | joined, not_joined | state | • | • | ### Per WSFC cluster -These metrics refer to the Windows Server Failover Clustering quorum state. +The Windows Server Failover Cluster hosting the availability groups. This scope has no labels. Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_cluster_quorum_state | WSFC Cluster Quorum State | normal, forced, unknown | state | • | • | +| mssql.ag_cluster_quorum_state | WSFC cluster quorum state | normal, forced, unknown | state | • | • | ### Per WSFC cluster member -These metrics refer to individual WSFC cluster members. +One node of the failover cluster. Labels: @@ -765,15 +851,15 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_cluster_member_state | WSFC Cluster Member State | up, down | state | • | • | -| mssql.ag_cluster_member_quorum_votes | WSFC Cluster Member Quorum Votes | votes | votes | • | • | +| mssql.ag_cluster_member_state | WSFC cluster member state | up, down | state | • | • | +| mssql.ag_cluster_member_quorum_votes | WSFC cluster member quorum votes | votes | votes | • | • | ### Per AG page repair -These metrics refer to automatic page repair events per database in an Availability Group. +One database with automatic page repair events. Labels: @@ -783,15 +869,17 @@ Labels: Metrics: -| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Database | +| Metric | Description | Dimensions | Unit | SQL Server | Azure SQL Managed Instance | |:------|:------------|:----------|:----|:---:|:---:| -| mssql.ag_page_repair | AG Automatic Page Repair Events | successful, failed | repairs | • | • | +| mssql.ag_page_repair | AG automatic page repair events | successful, failed | repairs | • | • | ## Live Data -This collector exposes real-time functions for interactive troubleshooting in the Live tab. +Three Functions run diagnostic queries on demand from the dashboard's Live tab. They keep the metrics +connection available, but share SQL Server CPU, I/O and locks. They return raw query text, so restrict +access to people who may see it. ### Top Queries @@ -800,7 +888,7 @@ Retrieves aggregated SQL query performance metrics from Microsoft SQL Server, pr With Query Store, this function queries `sys.query_store_runtime_stats` and related views across all databases with Query Store enabled, aggregating execution statistics by query hash. It provides comprehensive timing, I/O, memory, and parallelism metrics. -Query Store was introduced in SQL Server 2016 (13.x). When it is missing or turned off everywhere, the function falls back to [`sys.dm_exec_query_stats`](https://learn.microsoft.com/en-us/sql/relational-databases/system-dynamic-management-views/sys-dm-exec-query-stats-transact-sql), the plan cache, and the `Source` column reports which store answered. Changes to Query Store enablement are detected on subsequent requests. The plan cache differs in important ways: +Query Store was introduced in SQL Server 2016 (13.x). When it is missing or turned off everywhere, the function falls back to [`sys.dm_exec_query_stats`](https://learn.microsoft.com/en-us/sql/relational-databases/system-dynamic-management-views/sys-dm-exec-query-stats-transact-sql), the plan cache, and the `Source` column reports which store answered. The plan cache differs in important ways: - It only covers plans that are cached right now. Statistics are lost on restart, recompilation, memory pressure, or `DBCC FREEPROCCACHE`, so they are not query history. - Rows are aggregated by query hash across the whole instance, not per database. `Database` is selected from the cached statements (currently the minimum value), so it is not reliable per-database attribution when the same query hash occurs in multiple databases, and it is empty for ad-hoc or prepared batches that carry no database context. @@ -812,20 +900,20 @@ Use cases: - Analyze I/O patterns (logical reads, physical reads, writes) to detect bottlenecks - Monitor parallelism (DOP) and tempdb usage for capacity planning -Query text is truncated at 4096 characters for display purposes. Columns are detected at runtime from the active source, so metrics that a given SQL Server release or store does not provide are omitted. +Query text is truncated at 4096 characters. Columns that the active source or SQL Server release does not provide are omitted. | Aspect | Description | |:-------|:------------| | Name | `Mssql:top-queries` | | Require Cloud | yes | -| Performance | On SQL Server and Azure SQL Managed Instance, Query Store queries span enabled user databases; the plan-cache fallback reads cached statements across the instance. Azure SQL Database queries use the database selected in the DSN.
• Execution time depends on retained Query Store history or qualifying cached statements and their text
• Aggregation and latest-execution selection process the matching input before the default 500-row result limit
• Error attribution also reads the configured Extended Events target or the system_health fallback. On Standard/Enterprise editions, system_health can retain about 1 GB of event files, which are scanned before selecting recent errors
• Function queries share the collector's database connection, so long scans can delay metric collection | +| Performance | On SQL Server and Azure SQL Managed Instance, Query Store queries span enabled user databases; the plan-cache fallback reads cached statements across the instance.
• Execution time depends on retained Query Store history or qualifying cached statements and their text
• Aggregation and latest-execution selection process the matching input before the default 500-row result limit
• Error attribution also reads the configured Extended Events target or the system_health fallback. On Standard/Enterprise editions, system_health can retain about 1 GB of event files, which are scanned before selecting recent errors
• Functions keep the metrics connection available, but compete for SQL Server CPU, I/O and locks
• Bounded by `functions.top_queries.timeout`, independent of the metrics timeout | | Security | Query text may contain unmasked literal values including potentially sensitive data:
• Personal information in WHERE clauses or INSERT values
• Business data and internal identifiers
• Access should be restricted to authorized personnel only | -| Availability | Available when:
• The collector has successfully connected
• `functions.top_queries.disabled` is false
• Query Store answers when the engine exposes it (SQL Server 2016 (13.x) and later, Azure SQL Managed Instance, or Azure SQL Database) and it is enabled on at least one user database; on Azure SQL Database, in the database selected in the DSN
• Otherwise the plan cache (`sys.dm_exec_query_stats`) answers, which requires `query_hash`, available since SQL Server 2008
• Returns HTTP 403 when required permissions are missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 503 if the collector is still initializing, the function is disabled, or neither Query Store nor the plan cache can answer
• Returns HTTP 500 if the query fails
• Returns HTTP 504 if the query times out | +| Availability | Available when:
• The collector has successfully connected
• `functions.top_queries.disabled` is false
• Query Store answers when the engine exposes it (SQL Server 2016 (13.x) and later, or Azure SQL Managed Instance) and it is enabled on at least one user database
• Otherwise the plan cache (`sys.dm_exec_query_stats`) answers, which requires `query_hash`, available since SQL Server 2008
• Returns HTTP 403 when required permissions are missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 503 if the collector is still initializing, the function is disabled, or neither Query Store nor the plan cache can answer
• Returns HTTP 500 if the query fails
• Returns HTTP 504 if the query times out | #### Prerequisites -##### Plan-cache access +##### Grant plan-cache access Query Store is optional. SQL Server 2008–2014, and newer instances without an enabled Query Store, use the plan cache automatically. Grant the monitoring login the server permission for its version: @@ -838,16 +926,12 @@ GRANT VIEW SERVER STATE TO [netdata_user]; GRANT VIEW SERVER PERFORMANCE STATE TO [netdata_user]; ``` -Azure SQL Database plan-cache permissions depend on the service tier. See the -[Microsoft DMV permissions reference](https://learn.microsoft.com/en-us/sql/relational-databases/system-dynamic-management-views/sys-dm-exec-query-stats-transact-sql#permissions). - ##### Enable Query Store for persistent history On SQL Server 2016 and later, enable Query Store on the databases whose persistent query history you want to inspect. SQL Server and Azure SQL Managed Instance query all enabled user databases. -Azure SQL Database queries only the database selected in the collector DSN. These steps are not -required for the plan-cache fallback. +These steps are not required for the plan-cache fallback. 1. Verify Query Store state: @@ -857,9 +941,6 @@ required for the plan-cache fallback. FROM sys.databases WHERE name NOT IN ('master', 'tempdb', 'model', 'msdb'); - -- Azure SQL Database (run in the database selected in the DSN) - SELECT actual_state_desc - FROM sys.database_query_store_options; ``` 2. Enable Query Store where it is disabled: @@ -870,9 +951,8 @@ required for the plan-cache fallback. 3. Grant the monitoring account the permissions required by your engine: - On SQL Server/Managed Instance, map the login to a database user in every queried user database - before granting the database permission. On Azure SQL Database, grant the permission to the - database principal used by the collector connection. + Map the login to a database user in every queried user database before granting the database + permission. ```sql -- SQL Server 2022+ and Azure SQL Managed Instance 2022+ @@ -887,17 +967,14 @@ required for the plan-cache fallback. CREATE USER [netdata_user] FOR LOGIN [netdata_user]; -- once per database GRANT VIEW DATABASE STATE TO [netdata_user]; - -- Azure SQL Database (run in the database selected in the DSN) - GRANT VIEW DATABASE PERFORMANCE STATE TO [netdata_user]; ``` :::info -- Query Store is available in SQL Server 2016+ and Azure SQL Database +- Query Store is available in SQL Server 2016+ and Azure SQL Managed Instance - Requires ALTER DATABASE permission to enable Query Store - Query Store queries exclude system databases (master, tempdb, model, msdb). The plan cache excludes statements identified as belonging to those databases, but can include statements with unknown database context -- `top-queries` is enabled by default; set `functions.top_queries.disabled: true` only to disable it ::: @@ -1005,7 +1082,7 @@ Query text and wait resource strings are truncated at 4096 characters for displa | Require Cloud | yes | | Performance | Executes on-demand queries against the selected `system_health` event_file or ring_buffer target:
• Not part of regular metric collection
• Overhead is limited to function execution time and XML parsing | | Security | Query text and wait resource strings may include unmasked literal values including sensitive data (PII/secrets):
• SQL literals such as emails, IDs, or tokens
• Schema and table names that may be sensitive in some environments
• Restrict dashboard access to authorized personnel only | -| Availability | Available on SQL Server and Azure SQL Managed Instance when:
• The collector has successfully connected
• `functions.deadlock_info.disabled` is false
• SQL Server 2022+ has `VIEW SERVER PERFORMANCE STATE`; older versions have `VIEW SERVER STATE`
• Azure SQL Database returns HTTP 503 because it has no built-in `system_health` session
• Returns HTTP 200 with empty data when no deadlock is found
• Returns HTTP 403 when permission is missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 500 if the query fails or the required ring-buffer target is unavailable
• Returns HTTP 561 when the deadlock graph cannot be parsed
• Returns HTTP 503 if the collector is still initializing, the function is disabled, or the engine is Azure SQL Database
• Returns HTTP 504 if the query times out | +| Availability | Available on SQL Server and Azure SQL Managed Instance when:
• The collector has successfully connected
• `functions.deadlock_info.disabled` is false
• SQL Server 2022+ has `VIEW SERVER PERFORMANCE STATE`; older versions have `VIEW SERVER STATE`
• Returns HTTP 200 with empty data when no deadlock is found
• Returns HTTP 403 when permission is missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 500 if the query fails or the required ring-buffer target is unavailable
• Returns HTTP 561 when the deadlock graph cannot be parsed
• Returns HTTP 503 if the collector is still initializing or the function is disabled
• Returns HTTP 504 if the query times out | #### Prerequisites @@ -1059,19 +1136,19 @@ Use cases: |:-------|:------------| | Name | `Mssql:error-info` | | Require Cloud | yes | -| Performance | Executes on-demand queries against the configured Extended Events target, falling back to `system_health` when unavailable:
• Also used for error attribution by top-queries, but not by regular metric collection
• event_file reads scan retained files and parse matching errors before returning the newest rows
• The recommended dedicated-session settings of 6 MB per file and three rollover files keep the conservative filesystem scan envelope near 24 MB (current file plus rollovers)
• The system_health fallback has separate retention settings: Standard/Enterprise editions can retain about 1 GB. A small response limit does not bound the files scanned
• Existing sessions keep their own retention settings; the collector does not alter them. Long Function scans can delay metric queries on the shared database connection | +| Performance | Executes on-demand queries against the configured Extended Events target, falling back to `system_health` when unavailable:
• Also used for error attribution by top-queries, but not by regular metric collection
• event_file reads scan retained files and parse matching errors before returning the newest rows
• On SQL Server, the dedicated-session settings of 6 MB per file and three rollover files keep the conservative filesystem scan envelope near 24 MB (current file plus rollovers)
• The system_health fallback has separate retention settings: Standard/Enterprise editions can retain about 1 GB. A small response limit does not bound the files scanned
• Existing sessions keep their own retention settings; the collector does not alter them
• Keeps the metrics connection available, but shares SQL Server CPU, I/O and locks | | Security | Error messages and query text may include unmasked literal values including sensitive data (PII/secrets):
• Restrict dashboard access to authorized personnel only | -| Availability | Available on SQL Server 2012+, Azure SQL Managed Instance, and Azure SQL Database when:
• The collector has successfully connected
• `functions.error_info.disabled` is false
• The configured session is used when its selected target is available; otherwise SQL Server/Managed Instance falls back to the built-in `system_health` target, while Azure SQL Database returns HTTP 503 because it has no built-in `system_health` session
• When `functions.error_info.use_ring_buffer` is false, the configured session has an event_file target. Its configured `filename` is resolved from catalog metadata, so it need not match the session name or be running
• When `functions.error_info.use_ring_buffer` is true, the configured session is running and has a ring_buffer target
• SQL Server 2022+/Managed Instance 2022+ has `VIEW SERVER PERFORMANCE STATE`; older SQL Server has `VIEW SERVER STATE`; Azure SQL Database event_file has `VIEW DATABASE PERFORMANCE STATE`, while ring_buffer has `VIEW DATABASE STATE`
• Returns HTTP 200 with empty data when no errors are found
• Returns HTTP 403 when permission is missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 500 if the query fails
• Returns HTTP 503 if neither the selected target nor the applicable `system_health` fallback is available, or the function is disabled
• Returns HTTP 504 if the query times out | +| Availability | Available on SQL Server 2012+ and Azure SQL Managed Instance when:
• The collector has successfully connected
• `functions.error_info.disabled` is false
• The configured session is used when its selected target is available; otherwise SQL Server/Managed Instance falls back to the built-in `system_health` target
• When `functions.error_info.use_ring_buffer` is false, the configured session has an event_file target. Its configured `filename` is resolved from catalog metadata, so it need not match the session name or be running
• When `functions.error_info.use_ring_buffer` is true, the configured session is running and has a ring_buffer target
• SQL Server 2022+/Managed Instance 2022+ has `VIEW SERVER PERFORMANCE STATE`; older SQL Server has `VIEW SERVER STATE`
• Returns HTTP 200 with empty data when no errors are found
• Returns HTTP 403 when permission is missing
• Returns HTTP 499 when the caller cancels the request
• Returns HTTP 500 if the query fails
• Returns HTTP 503 if neither the selected target nor the applicable `system_health` fallback is available, or the function is disabled
• Returns HTTP 504 if the query times out | #### Prerequisites -##### Create Extended Events session for error capture +##### Create an Extended Events session for error capture Create an Extended Events session that captures `sqlserver.error_reported` with `sql_text` and `query_hash` actions. Choose the engine scope and target that match your deployment and `functions.error_info.use_ring_buffer` setting. -**SQL Server and Azure SQL Managed Instance — event_file (default):** +**SQL Server — event_file (default):** ```sql CREATE EVENT SESSION [netdata_errors] ON SERVER @@ -1112,49 +1189,29 @@ GO Set `functions.error_info.use_ring_buffer: true`. The session starts automatically after restart or failover, but events previously held in memory are lost. -**Azure SQL Database — event_file (default):** +**Azure SQL Managed Instance — event_file (default):** -Create the required database-scoped credential and grant the Database Engine access to an Azure -Storage container first. See Microsoft's [event_file setup guide](https://learn.microsoft.com/en-us/azure/azure-sql/database/xevent-code-event-file). +Managed Instance requires Azure Blob Storage for event files. Create a server-scoped credential in +`master` and grant the Database Engine access to the storage container first. See Microsoft's [event_file setup guide](https://learn.microsoft.com/en-us/azure/azure-sql/database/xevent-code-event-file). ```sql --- Run in the database selected in the collector DSN. -CREATE EVENT SESSION [netdata_errors] ON DATABASE +CREATE EVENT SESSION [netdata_errors] ON SERVER ADD EVENT sqlserver.error_reported( ACTION(sqlserver.sql_text, sqlserver.query_hash) ) ADD TARGET package0.event_file( SET filename=N'https://.blob.core.windows.net//netdata_errors.xel', - max_file_size=6, - max_rollover_files=3 + max_file_size=6 ); GO -ALTER EVENT SESSION [netdata_errors] ON DATABASE STATE = START; -GO -``` - -Azure Storage rollover retention is currently a preview feature and requires creating a new event -session with `max_rollover_files`. Existing sessions are not migrated automatically. - -**Azure SQL Database — ring_buffer:** - -```sql --- Run in the database selected in the collector DSN. -CREATE EVENT SESSION [netdata_errors] ON DATABASE -ADD EVENT sqlserver.error_reported( - ACTION(sqlserver.sql_text, sqlserver.query_hash) -) -ADD TARGET package0.ring_buffer -WITH (STARTUP_STATE = ON); -GO - -ALTER EVENT SESSION [netdata_errors] ON DATABASE STATE = START; +ALTER EVENT SESSION [netdata_errors] ON SERVER STATE = START; GO ``` -Set `functions.error_info.use_ring_buffer: true`. The database-scoped session starts automatically -after restart, failover, or maintenance, but events previously held in memory are lost. +On Managed Instance, `max_rollover_files` does not currently limit retained blobs. Configure an Azure +Storage lifecycle policy to bound retention; `max_file_size` alone does not bound the total data scanned. +Alternatively, use the server-scoped ring buffer example above. ```sql -- SQL Server 2022+ and Azure SQL Managed Instance 2022+ @@ -1163,12 +1220,6 @@ GRANT VIEW SERVER PERFORMANCE STATE TO [netdata_user]; -- SQL Server 2019 and earlier GRANT VIEW SERVER STATE TO [netdata_user]; --- Azure SQL Database (run in the database selected in the DSN) --- event_file target -GRANT VIEW DATABASE PERFORMANCE STATE TO [netdata_user]; - --- ring_buffer target (also satisfies the catalog permission) -GRANT VIEW DATABASE STATE TO [netdata_user]; ``` If you use a different session name, set it in the collector config: @@ -1273,21 +1324,251 @@ If your Netdata runs in a Docker container named "netdata" (replace if different docker logs netdata 2>&1 | grep mssql ``` -### Other Problems +### Known Errors + +#### `check failed: error pinging database: unable to open tcp connection with host 'localhost:1433': dial tcp [::1]:1433: connect: connection refused` + +**When** + +At job start, before any metric is collected. The message ends with the address the driver dialed. + + +**Cause** + +Nothing accepts TCP connections at the address and port in `dsn`. SQL Server may have the TCP/IP protocol +disabled, listen on another port, or sit behind a firewall that blocks port 1433. + + +**Fix** + +Enable TCP/IP in SQL Server Configuration Manager and restart the instance, or point `dsn` at the port the +instance uses. Find it from a working session with: + +```sql +SELECT local_tcp_port FROM sys.dm_exec_connections WHERE session_id = @@SPID; +``` + + +#### `check failed: error pinging database: no instance matching 'INSTANCENAME' returned from host 'localhost'` + +**When** + +The DSN names an instance (`host/INSTANCENAME`) instead of a port. + + +**Cause** + +Named instances are resolved through the SQL Server Browser service on UDP port 1434. The service is +stopped, the port is blocked, or no instance of that name exists on the host. + + +**Fix** + +Check the instance name, start the SQL Server Browser service, or give the instance a static TCP port and +use `host:port` in `dsn`. + + +#### `check failed: error pinging database: mssql: Login failed for user 'netdata_user'. (18456)` + +**Cause** + +The login or password in `dsn` is wrong, the login does not exist on this instance, or the instance +accepts Windows Authentication only and rejects every SQL login. + + +**Fix** + +Check the credentials and that the login exists. For SQL logins, switch the instance to "SQL Server and +Windows Authentication mode" (Server Properties, Security) and restart it. The SQL Server error log +records the reason for every failed login. + + +#### `check failed: error pinging database: mssql: Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'. (18456)` + +**When** + +Windows Authentication (no credentials in `dsn`) against an instance on another machine. + + +**Cause** + +The Netdata service runs as `Local System` on a machine that is not in the instance's domain, so it cannot +present a Windows identity over the network. + + +**Fix** + +Join the Netdata host to the domain and grant its computer account as described under Prerequisites, run +the Netdata service as a domain account with a SQL Server login, or use a SQL login in `dsn`. + + +#### `check failed: batch requests query failed: mssql: VIEW SERVER PERFORMANCE STATE permission was denied on object 'server', database 'master'. (300)` + +**When** + +At job start, right after the connection succeeded. A second line, `mssql: The user does not have +permission to perform this action. (297)`, follows. SQL Server 2019 and earlier name `VIEW SERVER STATE` +instead. + + +**Cause** + +The login lacks `VIEW SERVER STATE`, so the first performance counter query is rejected and the job does +not start. + + +**Fix** + +```sql +GRANT VIEW SERVER STATE TO netdata_user; +``` + +On SQL Server 2022 and later, `GRANT VIEW SERVER PERFORMANCE STATE` is enough for the metrics. + + +#### `SQL Server Agent jobs query failed; job metrics will be unavailable: mssql: The SELECT permission was denied on the object 'sysjobs', database 'msdb', schema 'dbo'. (229)` + +**Cause** + +The login has no access to the SQL Server Agent tables in `msdb`. Every other metric keeps being collected; +the message is logged once. + + +**Fix** + +```sql +USE msdb; +CREATE USER netdata_user FOR LOGIN netdata_user; +GRANT SELECT ON dbo.sysjobs TO netdata_user; +GRANT SELECT ON dbo.sysjobhistory TO netdata_user; +GRANT SELECT ON dbo.sysjobactivity TO netdata_user; +``` + + +#### Some SQL Server Agent jobs have only a status chart, or no charts at all + +**Cause** + +Disabled jobs get execution charts only when `collect_disabled_jobs` is set. On Express there is no SQL +Server Agent, so no job charts exist at all. Missing `SELECT` on `msdb.dbo.sysjobhistory` or +`msdb.dbo.sysjobactivity` removes the execution charts for every job. + + +**Fix** + +Set `collect_disabled_jobs: yes` for disabled jobs, or grant the two missing `msdb` permissions. + + +#### No Always On Availability Group charts appear although Always On is enabled + +**Cause** + +The availability group catalog views return rows only to logins with `VIEW ANY DEFINITION`; without it the +queries succeed with no rows and nothing is logged. The same grant feeds the data file size and I/O stall +charts through `sys.master_files`, so those exist but stay empty. + + +**Fix** + +```sql +GRANT VIEW ANY DEFINITION TO netdata_user; +``` + + +#### `top-queries requires VIEW SERVER STATE for the plan cache, or VIEW DATABASE STATE in every queried user database for Query Store` + +**When** + +Opening `top-queries` in the dashboard; the response is HTTP 403. On SQL Server 2022 and later the message +names `VIEW SERVER PERFORMANCE STATE` and `VIEW DATABASE PERFORMANCE STATE`. + + +**Cause** + +The login can collect metrics but lacks the permission the query statistics source needs: Query Store is +read per database, the plan cache at server level. + + +**Fix** + +Grant the permission the message names, following the `top-queries` prerequisites under Live Data. For +Query Store, the login needs a user and the database-level grant in every user database it should cover. + + +#### `deadlock-info requires VIEW SERVER STATE permission. Grant with: GRANT VIEW SERVER STATE TO [netdata_user];` + +**When** + +Opening `deadlock-info` in the dashboard; the response is HTTP 403. On SQL Server 2022 and later the message +names `VIEW SERVER PERFORMANCE STATE`. + + +**Cause** + +Reading the `system_health` Extended Events session needs the server-level state permission the message +names. + + +**Fix** + +Run the `GRANT` statement from the message as an administrator. + + +#### `error-info requires VIEW SERVER STATE permission. Grant with: GRANT VIEW SERVER STATE TO [netdata_user];` + +**When** + +Opening `error-info` in the dashboard; the response is HTTP 403. The message names `VIEW SERVER PERFORMANCE +STATE` on SQL Server 2022 and later. + + +**Cause** + +Reading the Extended Events session target needs the state permission the message names, at the scope of +the session. + + +**Fix** + +Run the `GRANT` statement from the message as an administrator. + + +#### `error-info not enabled: Extended Events session not found or event_file target missing` + +**When** + +Opening `error-info` in the dashboard; the response is HTTP 503. With `functions.error_info.use_ring_buffer` +set, the message ends in `ring_buffer target missing`. + + +**Cause** + +No Extended Events session named by `functions.error_info.session_name` exists with the expected target, +and no `system_health` fallback is available. + + +**Fix** + +Create the session as described in the `error-info` prerequisites under Live Data, or set +`functions.error_info.session_name` to the name of an existing session and +`functions.error_info.use_ring_buffer` to match its target. + + +#### `top_queries query timed out; Function timeout is 30s (functions.top_queries.timeout), but the request deadline may be shorter` -#### Connection refused +**When** -Ensure SQL Server is running and accepting TCP connections on the configured port. -Check that the SQL Server Browser service is running if using named instances. +Any Function; the response is HTTP 504 and the message names the Function and its timeout option. -#### Login failed +**Cause** -Verify the username and password in the DSN are correct. -Ensure SQL Server is configured for mixed mode authentication if using SQL logins. +The diagnostic query did not finish within `functions..timeout`: a large Query Store history, a big +plan cache, or gigabytes of `system_health` event files to scan. -#### Permission denied +**Fix** -Base metric collection needs `VIEW SERVER STATE`. Functions can require additional version-specific -permissions; see each Function's prerequisites below. +Raise `functions..timeout`, and for `top-queries` lower `functions.top_queries.time_window_days` or +`functions.top_queries.limit`. For `error-info`, create the dedicated session from the prerequisites so +the read stays within its small rollover files instead of the `system_health` history. diff --git a/docs/Collecting Metrics/Prometheus Metric Relabeling.mdx b/docs/Collecting Metrics/Prometheus Metric Relabeling.mdx index f7d549b48f..6050630fb5 100644 --- a/docs/Collecting Metrics/Prometheus Metric Relabeling.mdx +++ b/docs/Collecting Metrics/Prometheus Metric Relabeling.mdx @@ -1,5 +1,5 @@ --- -custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/prometheus/relabel/README.md" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/pkg/relabel/README.md" sidebar_label: "Prometheus Metric Relabeling" learn_status: "Published" toc_max_heading_level: "6" @@ -14,6 +14,22 @@ slug: "/collecting-metrics/prometheus-metric-relabeling" # Metric relabeling +`pkg/relabel` provides the shared Prometheus-compatible rule processor and ordered block pipeline. +They transform a `Record` containing only `Name` and `Labels`; values and metric types remain with the caller. +`__name__` is a virtual label backed by `Name` and MUST NOT also appear in `Labels`. + +`New` compiles rules into a `Processor`; `NewPipeline` compiles name-matched blocks. Both own reusable scratch and +MUST be used serially. Applying rules does not mutate the input label set. A dropped processor result returns the +record at processor entry; a dropped pipeline result returns the record at entry to the dropping block, preserving +changes from earlier successful blocks. Callers MUST discard dropped results and may use the returned record for +diagnostics. Observer callbacks run synchronously and MUST NOT re-enter the same processor or pipeline. + +The shared package does not assemble metric families, select profiles, or validate collector-specific values or +labels. The following configuration guide describes its integration in the Prometheus collector, which owns the +stage order and histogram/summary protection. + +## Prometheus collector integration + Relabeling rewrites, adds, drops, or filters a scraped metric -- its name and its labels -- before charts are built. The same `relabeling` block format has two configuration levels: @@ -185,7 +201,7 @@ Inside a block, `metric_relabel_configs` is a list of rules applied **in order** ## How relabeling runs -The stage order is fixed: +The Prometheus collector fixes the stage order: 1. The job `selector` filters original scraped series. 2. Job relabeling runs and its result is assembled and checked. @@ -202,7 +218,7 @@ Inside either relabeling stage: ### Histogram and summary safety -A histogram or summary is assembled from several series -- `_bucket`/`_sum`/`_count`, or per-quantile series. Netdata +In the Prometheus collector, a histogram or summary is assembled from several series -- `_bucket`/`_sum`/`_count`, or per-quantile series. Netdata will **not** let relabeling silently corrupt one. A rule that would: - split a histogram/summary across multiple metric names, diff --git a/docs/Welcome to Netdata/Monitor Anything.mdx b/docs/Welcome to Netdata/Monitor Anything.mdx index c59bd510d8..b54086d5be 100644 --- a/docs/Welcome to Netdata/Monitor Anything.mdx +++ b/docs/Welcome to Netdata/Monitor Anything.mdx @@ -88,7 +88,7 @@ Need a dedicated integration? [Submit a feature request](https://github.com/netd | [MaxScale](/docs/collecting-metrics/collectors/databases/maxscale) | This collector monitors the activity and performance of MaxScale servers. | | [Meilisearch](/docs/collecting-metrics/collectors/databases/meilisearch) | Track Meilisearch search engine metrics for efficient search performance and management. | | [Memcached](/docs/collecting-metrics/collectors/databases/memcached) | Monitor Memcached metrics for proficient in-memory key-value store operations. | -| [Microsoft SQL Server](/docs/collecting-metrics/collectors/databases/microsoft-sql-server) | This collector monitors the health and performance of Microsoft SQL Server instances. | +| [Microsoft SQL Server](/docs/collecting-metrics/collectors/databases/microsoft-sql-server) | Monitor Microsoft SQL Server performance, databases, SQL Server Agent jobs, replication, and Always On Availability Groups. | | [MongoDB](/docs/collecting-metrics/collectors/databases/mongodb) | This collector monitors MongoDB servers. | | [mosquitto](/docs/collecting-metrics/collectors/databases/mosquitto) | Keep an eye on Mosquitto MQTT broker metrics for efficient IoT message transport and performance. | | [MySQL](/docs/collecting-metrics/collectors/databases/mysql) | This collector monitors the health and performance of MySQL servers and collects general statistics, replication and user metrics. | diff --git a/ingest/generated_map.yaml b/ingest/generated_map.yaml index 04490ef554..b2682a5145 100644 --- a/ingest/generated_map.yaml +++ b/ingest/generated_map.yaml @@ -1196,7 +1196,7 @@ autogen selectors that filter the fallback charts of unmatched metrics. meta_yaml: .nan message: .nan -- custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/prometheus/relabel/README.md +- custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/pkg/relabel/README.md sidebar_label: Prometheus Metric Relabeling learn_status: Published learn_rel_path: Collecting Metrics @@ -3137,9 +3137,11 @@ sidebar_label: Microsoft SQL Server learn_status: Published learn_rel_path: Collecting Metrics/Collectors/Databases - keywords: '[''db'', ''database'', ''mssql'', ''sql server'', ''microsoft'']' - description: This collector monitors the health and performance of Microsoft SQL - Server instances. + keywords: '[''mssql'', ''sql server'', ''microsoft sql server'', ''microsoft'', + ''azure sql'', ''azure sql managed instance'', ''always on'', ''sql server agent'', + ''query store'']' + description: Monitor Microsoft SQL Server performance, databases, SQL Server Agent + jobs, replication, and Always On Availability Groups. meta_yaml: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/mssql/metadata.yaml message: DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE diff --git a/ingest/generated_sidebar_order.json b/ingest/generated_sidebar_order.json index 141241d673..bfd076d596 100644 --- a/ingest/generated_sidebar_order.json +++ b/ingest/generated_sidebar_order.json @@ -1,8 +1,8 @@ { "schema_version": 1, "source": "netdata/docs/.map/map.yaml", - "source_sha256": "d0e76a8b2a2f7602b5832db7ff77c2169aab960a29ace7194a5e3422c113e997", - "source_corpus_sha256": "e04ce50e51f9577a458893038c8de3bf840709bdfde60254e00e8562e1bf7716", + "source_sha256": "7c3c11c1f619d58919a2de0ce2280e362a898f58be3c012cab23d377eb06aa0f", + "source_corpus_sha256": "079c2635732c31dcc30514f2ea19b931514c08304a52cd430290575e623e5f0d", "order": [ { "parent_path": "Alerts & Notifications", @@ -1800,5 +1800,5 @@ "position": 180 } ], - "full_ingest_identity_sha256": "dac8f31f23c01f7fa0b997aab3fa8f9db16dcdb35223aea82bbf19e2eb355eb8" + "full_ingest_identity_sha256": "1fb46a81c4c768cb98ad2dbd94b58ab65ba699578d4330ec01586c1367b59109" } diff --git a/ingest/generated_sidebar_order.json.sha256 b/ingest/generated_sidebar_order.json.sha256 index 68fb0ef1ba..99b4f8ee8d 100644 --- a/ingest/generated_sidebar_order.json.sha256 +++ b/ingest/generated_sidebar_order.json.sha256 @@ -1 +1 @@ -b64c6a3c2ad6743156773c4cb53a7063eed414844c4a85c15515072f94cee372 generated_sidebar_order.json +03c4b86864f056c935c62151b570c207859379a9e2a9ef12ac579282b5eb5715 generated_sidebar_order.json diff --git a/ingest/one_commit_back_file-dict.yaml b/ingest/one_commit_back_file-dict.yaml index 83bfdd982c..c95886c62e 100644 --- a/ingest/one_commit_back_file-dict.yaml +++ b/ingest/one_commit_back_file-dict.yaml @@ -1016,6 +1016,8 @@ learn_path: /docs/exporting-metrics/connectors/prometheus-remote-write - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/exporting/pubsub/README.md learn_path: /docs/exporting-metrics/connectors/google-cloud-pub-sub +- custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/pkg/relabel/README.md + learn_path: /docs/collecting-metrics/prometheus-metric-relabeling - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/aws/README.md learn_path: /docs/collecting-metrics/secrets-management/secret-stores/aws-secrets-manager - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/azure/README.md @@ -1622,8 +1624,6 @@ learn_path: /docs/collecting-metrics/collectors/cloud-and-devops/zerto - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/prometheus/profile-format.md learn_path: /docs/collecting-metrics/prometheus-profile-format -- custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/prometheus/relabel/README.md - learn_path: /docs/collecting-metrics/prometheus-metric-relabeling - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/proxysql/README.md learn_path: /docs/collecting-metrics/collectors/databases/proxysql - custom_edit_url: https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/pulsar/README.md