Add a removed records index

[radudoros]

Add a separate removed records index that allows delivering partial information of deleted records. Addresses #396

[clarin-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
🔚	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[twagoo]

It looks good. I wasn't able to easily test it locally, as this requires records that are more than a day old (if maxDaysInSolr==0 then nothing is purged, as this would just delete everything, so the minimal is 1). However, I did notice that the importer 'hangs' in the end, the process doesn't terminate. If you can reproduce this, can you have a look at this? If not, we need to see if there are any specific conditions that may cause this.

[twagoo]

A couple of small suggestions regarding the tombstone page (example screenshot below for reference)

• I think it should be clearer at first sight that the record was not found. Perhaps the "visual title" of the page could be "Record not found" or "Record no longer available" instead of the value of the name field. The record's name could be displayed below that, I would think slightly smaller.
• I think it would be more correct to write "This record was removed from the VLO on [date]"
• Instead of " The information below (...) may help you find it elsewhere" I would write (".. may help you find the resource(s) that your are looking for").
• If there is a landing page among the links, we could show it more prominently, yet in a way that doesn't imply that it is necessarily still available (e.g. "The delete record contained a reference to the following landing page: [url]")

[twagoo]

If I'm not mistaken, the deleteAllFirst config option and deleteFirst option for dataroots do not cause records to be transferred to the morgue. We need to discuss whether this would be desirable.

[twagoo]

And I believed that we agreed in the end about the 404 status code for the morgue record page. I believe this still needs to be implemented?

I was thinking of perhaps splitting this up, but this may need further discussion:

• A 'not found' page with 404 status if the record is requested at the original URL /record/{id}
• A tombstone page, either at a different URL or a variant of /record/{id} with a query parameter that has a regular 200 page

Some notes on that:

• The former (not found page) could lead to the latter (tombstone) in the client, either manually (a clickable link) or automatically (e.g. using meta refresh)
• If we want to link to morgue records e.g. from the search results, we point directly to the tombstone
• The tombstone should point to the active record in case that has reappeared

[radudoros]

I should have addressed all your comments - kindly help me with a rereview @twagoo

[twagoo]

Some wicket things to look at, otherwise all looks good to me!

[twagoo]

Page state must be kept in a model object in Wicket, so correct would be to change this to private IModel<Boolean> removed and set it on initialisation.

[twagoo]

Probably better to use SolrDocumentModel than the generic model, as it has logic to load from the index, and does not store the actual full document (which may cause serialization problems as well)

[twagoo]

Landingpage does not contain a URL, but a json structure that needs to be unwrapped. If I'm not mistaken, this can be done with ResourceInfoModel, for a usage example RecordDetailsResourceInfoPanel, in particular:

final IModel<String> linkModel = new PropertyModel<>(landingPageResourceInfoModel, "href");