Feature: Selectively applicable conditions and metrics#395
Conversation
save a list of refrences to entries that a Condition object wont be evaulated against. using Condition.Match with these data will return inconclusive results save a reference to the listdata entry that a Metric object was generated from. if such a reference is present, and the Condition object has it in its skip list, it wont be used when building the perf threshold string add helper function that looks through a condition list, takes out Condition objects where specialized keywords are present, then filters to Condition objects that have a generallized keyword present. These generallized versions can have entries added to their evaluation skip list improve .String() on Condition objects, add more log points, add helper function for slice subtraction use check_drivesize as a testing ground for these changes. If '<drive> used_pct' keyword is present, it will be taken as a specialized keyword, and generallized 'used_pct' containing Conditions will have the drive entry added to their blacklist
…ing skip list for entry. this prevents golangci-lint from complaining this works because golang maps use an internal pointer to heap to save their key-value data. while maps are not directly comparable, using reflect.ValueOf(map).Pointer() to compare their pointers this way, we can save the skipList using map[string]string, and use the internal pointer comparsion to check if we need to skip. see utils.go for more details check_drivesize now passes the entry normally as map[string]string
utility: subtractslice , mapsequal , containsmap condition: testing skiplist
…is intended to give a parseable string output make a new function called DetailString() that gives more debug-like output usable in trace logs
improve log messages around condition and metric entry skipping
use a combined whitelist/blacklist checker in condition.match() and to see if condition is applicable in perfdata string when adding metrics in check_drivesize, use a heuristic to check if a condition is a specialized condition, with drive keyword and matches an entry - if so add it if there are no specialized condition, present add every condition as it is generic
log metric name when using blacklist+whitelist to filter conditions for perfstring
|
Does it have to be hardcoded for the https://omd.consol.de/docs/snclient/checks/commands/check_drivesize/#filter-keywords This works: This doesn't: Here I would have expected to additionally receive the inodes performance data which is set to 77% for Also another thing I noticed, why isn't the metric called like |
Let me see if I can generalize the functions more, so that I can add more metrics to behave like this easily. |
…em more generallizable for further checks conditionlist now has four functions: disableGenerallizedConditionsForEntry filterConditionsUsingKeywords ifKeywordIsPresentAndPermitsEntry filterForSpecializedKeyword checkdata: addBytePercentMetrics and AddPercentMetrics now take an entry parameter, they add it while constructing a metric has three helper functions processMetricsWithSpecializedKeyword transformKeywordsUsingAttributes disableGenerallizedConditionsUsingAttributes Use these helper functions in check_drivesize.go: transformDrivePctMetrics disableGenerallizedConditionsForDrive go through all attributes of percent type, and check if they are specified specifically for a drive. transform them using this method. also specify the entries in other metrics, which enables specialized filtering using drive= and perflabel= for more attributes
unlike AddBytePercentMetrics, it was not adding " %" to the name of the metric this caused problems when matching drive specific inodes_free and inodes_used conditions meant to match with perfdata labels
|
-> This works correctly now
-> This also works correctly now
Warning Here the thresholds for
Warning Here the warning/critical condition for |
2 participants
This PR includes changes to the core Conditon and Metric logic - all features are optional, and do not effect unless used
Core changes
add whitelist and blacklists to Condition struct. -> these are a slice to map[string]string , default golang map is a wrapper to heap memory region used to save key-values, so it does not copy the whole map again. it behaves like a pointer array, but needs helper functions to compare underlying heap table. these are used to store entries for which and restrict/allow entries in Condition.Match
if the entry is in its blacklist, it will not be allowed to check against that. If the Condition has a nonzero length whitelist, it is also effective, only allowed to check entries in the whitelist
these entries come from different checks in snclient, and are added to check.listData. Once they are allocated, the pointer is set and can be added to whitelist/blacklist. Modifying the entry later does not invalidate them in whitelist/blacklist as the underlying pointer to map data is the same
in the Condition.Match a disallowed entry returns inconclusive result - meaning the match is null and void, it does not tell anything about the match. This is the second return argument of Condition.Match
these conditions then influence the return state of the check - each check.listData entry is compared against each Condition in the ok, warning and critical threshold. If they are inconclusive however, they do not modify the results. This many to many check can be somewhat restricted using whitelists and blacklists
add reference the entry object to Metric struct -> which entry generated this metric, if this is known if conditions that check if its whitelisted/blacklisted before using it to build perfdata string
Check_drivesize
use these changes to add selective drive-specific metric and threshold handling to check_drivesize. these come in two forms
this is done using disableGenerallizedConditionsForDrive helper.
This helps to give used_pct thresholds specific to drives - other drives do not include them while building their perf threshold string as they are blacklisted for being generic
keyword, and matches the current entry -
if there is no condition with the specialized keyword - all conditions are generic and pass through the filter
if there is a specialized condition with keyword "drive" and that matches the drive entry - add all conditions with a subcondition passing through the filter using drive entry
The heuristic here might not be fully correct, but looking too deep is not needed now
This helps if the condition is specified using multiple conditions with a group operator. Here, the second condition wont be added to the '/' drive when adding it as a metric.
If it was added, the perfdata generation would detect two conditions in total using 'used_pct' keyword, one global one with value 90 and one specific to '/boot' drive which still went through, with value 50. It would then think it has a range was
@50:90Other changes
add a new processMetricsWithSpecializedKeyword() functon to CheckData -> this calls processMetricsWithSpecializedKeyword for warning and critical thresholds of Metric object
add a new transformKeywordsUsingAttributes() function to CheckData -> this transforms keywords using a format specified. This can be supplied from a list of attributes programmatically changing the keywords
add a new disableGenerallizedConditionsUsingAttributes() function to CheckData -> this this also buids a specialized and generallized keyword using the specified formats. Then it calls disableGenerallizedConditionsForEntry() for ok, warn and crit thresholds of the CheckData
add a new DetailedString() function to Condition, it prints out more fields, use this in logs. Helpful as it also prints the original version of the condition - the keywords might be transformed later, either by the check or to make comparisons with metrics possible
add a new BlacklistWhitelistCheck() function to Condition - fastly checks if condition is accepted through the lists, this is used in Condition.Match()
add a new GetListOfKeywords() function to Condition - recursively gets lists of keywords, useful to ascertain if a condition should be inspected further
add a new disableGenerallizedConditionsForEntry() for ConditionList - checks if specialized keyword is used and if so filters to conditions not using the specialized keyword, and then furhter filters to conditions using the generallized keyword. disables them for the entry. this is used in the specialized metric name matching
add a new filterConditionsUsingKeywords() function to ConditionList - filters the condition list simply
add a new ifKeywordIsPresentAndPermitsEntry() function to ConditionList - used to determine if keyword is present in the condition and is allowed through using Check.Match() . this is used to see if a specialized condition is present in the conditionlist
add a new filterForSpecializedKeyword() function to ConditionList - this does the heuristic described in the second case, but is generallized over the ConditionList. Checks if there is a special condition present using ifKeywordIsPresentAndPermitsEntry() , and if so filters only the condtions using that keyword and matches it. Otherwise the condition list is generic, it includes everyhting
add SubtractSlice[T comparable](op1, op2 []T) (ret []T) to utils.go - used while filtering out slice results from one another
MapsEqual(a, b map[string]string) to utils.go - looks into the actual data pointer of the map, checks if they are the same. maps cannot be directly compared using == operator
ContainsMap(slice []map[string]string, target map[string]string) bool to utils.go - this is used in blacklist/whitelist checks with MapsEqual
Deduplicate[T comparable](slice []T) []T to utils.go - another function to clean up elements of a slice before saving
add logs around Condition checks for check.details, check.listData entries, check.result.Metrics etc.
add logs when a Condition blacklists/whitelists an entry
add tests around whitelist/blacklist functionality