Repository navigation
Stop audited syscalls from waiting on a full audit queue - #504
Open
venkata-stepsecurity wants to merge 1 commit into
Open
venkata-stepsecurity wants to merge 1 commit into
venkata-stepsecurity wants to merge 1 commit into
Conversation
The kernel defaults (backlog limit 64, backlog wait time 60s) make every audited syscall sleep while the agent falls behind, so a job with heavy network traffic slows down instead of the kernel dropping records. Use Auditbeat's default "auto" backpressure strategy: failure mode silent, backlog limit 8192, backlog wait time 0, or a 5000/s rate limit on kernels that cannot set the wait time. Poll the kernel's lost counter every 15s on a separate audit connection, as Auditbeat does, and log new losses.
step-security-bot-int
left a comment
There was a problem hiding this comment.
Please find StepSecurity AI-CodeWise code comments below.
Code Comments
procmon_linux_test.go
- [High]Check error return values from each call to Set methods in configureBackpressure*
In configureBackpressure, each call to methods like SetFailure, SetBacklogLimit, etc., may return an error. Ignoring these errors violates defensive programming practices recommended by authoritative sources such as the Go Blog and Effective Go. Handling the errors immediately prevents hidden failures that can lead to incorrect system behavior or silent misconfiguration. Modify configureBackpressure to check and propagate errors from each Set* method. For example:
err := qc.SetFailure(...)
if err != nil {
return err
}
// Repeat for all subsequent Set* calls
- [High]Use deep equality checks when comparing slices in tests
In the test configureBackpressure(), slice comparisons use fmt.Sprint which is string conversion rather than semantic equivalence. This can produce false positives or negatives. The Go standard library's reflect.DeepEqual provides semantic slice comparison and is recommended as per Go testing best practices. Replace comparisons like 'fmt.Sprint(f.calls) != fmt.Sprint(c.want)' with 'reflect.DeepEqual(f.calls, c.want)'.
Import "reflect" and update assertions accordingly. - [Medium]Improve test coverage by adding negative and boundary test cases
While the test cases cover many configurations, they omit tests for situations such as feature bitmap lacking all relevant flags, zero or negative backlog limits, and large values. Comprehensive testing is critical to catch edge cases and ensure robustness (source: ISTQB and Go testing guidelines). Add test cases with zero or negative BacklogLimit, status with no FeatureBitmap flags set, and very large integer values. This will help verify correct handling of edge conditions. - [Medium]Avoid using fmt.Sprint for logging or comparing values in tests
Using fmt.Sprint converts slices and other complex data types to strings, which is not reliable for structured data comparison. This can obscure problems and reduce maintainability, violating testing principles emphasized in Go's official blog and style guidelines. Use appropriate data comparison or printing alternatives such as reflect.DeepEqual for comparison and testify or cmp packages for richer error messages. - [Medium]Make use of table-driven testing idioms by including test sub-names and proper assertion libraries
While the tests use t.Run and basic assertions, more descriptive assertion failures will improve debugging. Go best practices encourage descriptive errors and assertion helpers (as found in testify or gotest.tools). Use assert or require packages for better assertion messages, or wrap failures with contextual information. Example:
assert.Equal(t, c.want, f.calls, "Calls mismatch in test %q", c.name) - [Low]Add comments describing fakeQueueControl methods' purpose and parameters
Current code methods like SetFailure or SetBacklogLimit do not have doc comments, which hinders maintainability and readability as prescribed by Effective Go and golangci-lint recommendations. Add comments preceding each method describing its function, parameters, and expected behavior. - [Low]Consistently handle error paths in tests
In Test_configureBackpressure_error, the check for f.calls length is less descriptive. Providing explicit error messages aids test failure diagnosis. Update error messages with descriptive failure reasons, e.g.,
if len(f.calls) != 1 {
t.Errorf("Expected 1 call before error, got %d calls: %v", len(f.calls), f.calls)
} - [Low]Use defer to close channels and cleanup resources in test goroutines
In Test_monitorKernelLost_stops, the test could better manage lifecycle with defer statements to avoid subtle leaks. This is aligned with Go concurrency best practices. Add defer statements as appropriate when spawning goroutines; ensure that channel closure and resource cleanup happen reliably.
procmon.go
- [High]Add comments explaining the purpose and usage of new constants for maintainability
While some comments are added for the new constants, they should be more detailed and consistent to improve code readability and maintainability. According to 'Clean Code' by Robert C. Martin, meaningful comments help future developers understand the intent behind code. Expand existing comments to clarify the rationale behind each constant, their relationship to kernel audit settings, and implications of changing these values. - [High]Ensure constants related to kernel audit settings are configurable via configuration files or environment variables
Hardcoding kernel audit parameters reduces flexibility and can make tuning or testing difficult. The Twelve-Factor App principles and best practices suggest configuration should be externalized to allow adjustments without code changes. Refactor the constants auditBacklogLimit, auditFallbackRateLimit, and kernelLostCheckInterval to be initialized from application configuration or environment variables, with fallback defaults. - [Medium]Use a consistent naming convention for atomic counters to clearly indicate their thread-safe nature
Naming atomic variables appropriately improves code clarity about concurrent access. The Go official concurrency patterns recommend naming possibly with suffix 'Atomic' or prefix to denote atomicity. Rename 'kernelLost' field to 'kernelLostAtomic' or similar to reflect its atomic type explicitly. - [Medium]Group related constant declarations in a single block with consistent style
Defining related constants in one block improves code organization and readability as recommended by effective Go style guidelines. Combine the new kernel audit queue constants with existing 'const' declaration block, e.g., merge them below the existing constants at the top of the file. - [Low]Add unit tests or integration tests covering new atomic counter usage to prevent regressions
Testing changes to the ProcessMonitor struct fields can help ensure correctness under concurrency as per best practices in Go testing from the standard library documentation. Add tests that simulate concurrent increments on 'kernelLost' atomic and verify accurate counting. - [Low]Fix minor formatting inconsistencies in comment style and spacing
Consistent comment formatting improves readability; gofmt enforces this standard. Ensure comments start with a capital letter and end with a period; align with existing style.
procmon_linux.go
- [High]Avoid uncontrolled goroutine leaks by ensuring monitorKernelLost goroutine exits appropriately
The MonitorProcesses function launches a goroutine monitorKernelLost which relies on a channel stopLostCheck to terminate. The stopLostCheck is closed via defer when MonitorProcesses returns. If MonitorProcesses never returns (e.g., it loops for the lifetime of the process), the goroutine may leak, or conversely, if MonitorProcesses ends early, the goroutine is stopped promptly. Explicit documentation and testing should confirm proper lifecycle management. This follows best practices for goroutine lifecycle management recommended in "Go Concurrency Patterns" (https://go.dev/blog/concurrency_patterns). Add explicit documentation specifying the lifecycle and ownership of stopLostCheck channel and ensure MonitorProcesses terminates or otherwise coordinates with goroutine termination to prevent leaks. Alternatively, consider context.Context for cancellation. - [High]Check errors returned from WriteLog calls
The calls to WriteLog in configureBackpressure and MonitorProcesses ignore any error returns (if any). While WriteLog may be void, if it could fail (e.g., IO-related), those errors should be handled or at least logged to avoid silent failures. Ignoring errors can hide critical issues. This aligns with error handling best practices from "Effective Go" (https://golang.org/doc/effective_go#errors). If WriteLog returns an error, propagate or handle the error. If it does not return an error, ensure the logging implementation is reliable or consider improving error visibility. - [Medium]Avoid race conditions by protecting shared variables with synchronization primitives
The method noteKernelLost updates p.kernelLost presumably a shared counter without synchronization primitives (mutex or atomic operations). Concurrent goroutine calls can cause race conditions leading to incorrect counts or data corruption. According to "Go Memory Model" (https://golang.org/ref/mem), shared variables accessed by multiple goroutines must be synchronized. Protect access and updates to p.kernelLost using sync.Mutex or atomic package to ensure thread-safe operations. - [Medium]Avoid recreating AuditClient on every unsuccessful status retrieval
In monitorKernelLost, if client.GetStatus() returns an error, client is closed and nil is set. On the next tick, a new client is opened. Frequent errors may cause repeated open/close cycles which are resource-intensive. To improve robustness and performance, implement exponential backoff or limit retries as recommended in "Robustness Principle" (https://en.wikipedia.org/wiki/Robustness_principle). Add retry limits or exponential backoff when re-creating AuditClient to avoid tight loops on failure. - [Medium]Check for possible integer underflow in noteKernelLost when current < last
The logic in noteKernelLost assumes that if current < last, the counter was reset and sets delta = current (effectively missing lost events during the reset). This might undercount lost records. Defensive coding would handle this case explicitly or log a warning. This is consistent with defensive programming principles as discussed in "Secure Programming Cookbook" (https://www.informit.com/articles/article.aspx?p=2053172). Add explicit handling/logging if current < last to indicate possible resets and avoid missing delta calculations. - [Low]Use context.Context for cancellation instead of custom channels where applicable
Using context.Context is the idiomatic way in Go to handle cancellation and timeouts, instead of custom done channels. This improves integration, testability, and adheres to modern Go best practices (https://blog.golang.org/context). Replace stopLostCheck channel with a context.Context passed down to monitorKernelLost and use context cancellation for stopping the goroutine. - [Low]Declare constants explicitly for magic numbers
Values such as 0, 2, and specific numeric limits appear in code (e.g., status.Enabled == 2) without named constants. Using named constants improves readability and maintainability as per "Clean Code" principles (https://www.investigatii.md/uploads/resurse/Clean_Code_Robert_C._Martin.pdf). Define named constants for magic numbers like auditStatusLocked=2 to clarify code intent. - [Low]Add comments describing the meaning of audit status codes
The code compares status.Enabled and status.Failure to numeric values but does not explain what these mean, reducing readability and maintainability, violating best practices from "Go Code Review Comments" (https://github.com/golang/go/wiki/CodeReviewComments). Add inline comments or documentation describing the meaning of these status codes or refer to constants/enums.
Feedback
We appreciate your feedback in helping us improve the service! To provide feedback, please use emojis on this comment. If you find the comments helpful, give them a 👍. If they aren't useful, kindly express that with a 👎. If you have questions or detailed feedback, please create n GitHub issue in StepSecurity/AI-CodeWise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The agent never changes the kernel audit queue settings, so the kernel defaults apply: backlog limit 64 and backlog wait time 60s. When a job makes audited syscalls faster than the agent reads records, the queue fills, and the kernel puts each audited syscall to sleep until there is room. The job slows down rather than the kernel dropping records.
In a workload that streams many small sends over localhost TCP (480 s run, ubuntu-latest), the kernel reported
backlog_wait_time_actualof about 1,150,000 ms. That is roughly 2.4 s of syscalls sleeping per second of the run.Change
This uses the default "auto" backpressure strategy from Elastic Auditbeat (audit_linux.go, initClient). osquery also ships with a zero backlog wait time.
The settings are applied before audit is enabled and before rules are added. They are skipped if audit config is locked. A failure is logged and does not stop monitoring.
Audit rules are unchanged.
Trade-off
Under sustained overload, the kernel now drops records instead of slowing the job. Auditbeat and osquery make the same choice. Losses are now visible in the agent log.
Testing
go mod vendor,go mod verify,go vetandgo test -raceon Linux with Go 1.27.3.TestRunandTest_addAuditRulesfail in a container on both this branch andint, because they need a real runner; CI covers them.On both builds, at start, middle and end of the run, all detection probes were reported: HTTPS connections, a UDP
sendto, and the source file overwrite.