Added item_creation behavior for xmlfilecontent and textfilecontent54 per #315 and #316

[vanderpol]

Added item_creation behavior, similar to new behavior in ntuser test which allows content authors to control when an item is created, and prevent false negatives when text/xml file may not contain the data required to populate an item, and would be overlooked.

[solind]

@vanderpol I need to do some research on whether the default expected behavior isn't already the "filepath_exists" behavior.

[solind]

Hi @vanderpol, thinking about this some more, I think I'm too far removed from the weeds to say for sure at this point whether an OVAL 5.11 interpreter is supposed to make an item if the file exists but the pattern or XPath does not. It certainly could do so, in which case the EntityStateSimpleBaseType check_existence operator becomes impactful on the evaluation. This is important in determining which behavior we should make the default.

@A-Biggs, can you tell us what Joval does in these cases for textfilecontent and xmlfilecontent objects?

[vanderpol]

I did some research last year making sure we didn't have a bug in SCC as we were not creating items when the filepath existed, but somewhere (and it illudes me now), but I had several examples in DISA content that only made sense if the item was created when the pattern was found in the file.

In asking the AI overlords, they agree:

In OVAL 5.12.2, for a textfilecontent54_test to create an item, the pattern must exist within the specified file.An item is only created in the System Characteristics when a unique match for the OVAL Object is found. Since the textfilecontent54_object is defined by the combination of a file (path and filename) and a regular expression pattern, the OVAL Schema documentation specifies that the object identifies specific blocks of text.If the file path exists but the pattern is not found within that file:No item is created in the system characteristics.The object's status in the collected objects section will be flagged as "does not exist".The test's evaluation will then depend on its check_existence attribute (e.g., if set to all_exist, the test would fail because the expected item was not found).

[solind]

@vanderpol does the same hold for xmlfilecontent? And... what about the MacOS schema's plist, plist510 and plist511 objects?

[vanderpol]

@solind the same holds true for xmlfilecontent, and I'm guessing the same for plist and plist511, but haven'd dug into it.

For TextFileContent54 and XMLFileContent there are examples in DISA content which are existence based, of "none_exist", with test tiles like "No 'retries' entries allowed in /etc/pam.d/system-auth or password-auth files", so the files exist, but the only way those checks would work would be if the default was that the entire object has to be populated.

I think that holds true for all OVAL tests, if the items specified in the object are not satisfied, no item is created. This behavior allows for an exception to be made when needed by the content author.

[solind]

@vanderpol well... if an item exists, but the state is matching an entity whose status="does not exist", since the default state entity check_existence is "at least one exists", the test will still return false. Normally this will pan out the same as if there was no item at all.

[vanderpol]

The content in question has no state, the test is check_existence="none_exist", I should have been more clear on my previous comment. In searching current content in our repository, there are 174 textfilecontent54 tests which have a check_existence="none_exist", and they seem to be based on filepaths that exist, but the pattern cannot exist in them.

[solind]

Aha! Yes, that would have a significant impact on the test result! All the more reason to have such a behavior, for all these tests that point to a file and a thing to match within its contents.

And, it sounds like we do want the default behavior to be, in all these cases, that there should be no item if there is not also a content match. Are we agreed?

If so, I think these behaviors should be extended to the plist objects as well.

[vanderpol]

Given that plist and plist510 are both deprecated, I would say any changes would be in plist511, but I'm not sure how the app_id impacts this or if it does? We can add this as an issue to track, but given the lack of macOS implementations, it may or may not make the cut for 5.12.3 unless you or someone else would like to do the pull requests?

[vanderpol]

@solind the plist updates can be tracked separately and discussed on their own. Do you think this PR is ready to be approved for textfilecontent54 and xmlfilecontent? Thanks

[balleman-ctr]

I think this new behavior could be useful when collecting multiple files, for example with a pattern match on filename.

[solind]

@vanderpol sure, good idea, track them on their own. I thought someone was using these tests for MacOS benchmarks but maybe not anymore.

[vanderpol]

FYI, @johnulmer-oval is implementing this in SCC, and we'll post sample results when he's done.