Rest: Audit Trail feature

[steph-ieffam]

References

• Fixes DSpace should have an Audit Trail of object changes #8824
• Based on Improve the Event's detail handling #10684 to improve and better manage the audit information through the Event's details
• Related to Rest Contract PR Audit trail feature - REST Contract RestContract#311
• Related to Angular PR Angular: Audit Trail feature dspace-angular#4576

Description

the audit feature can be enabled disabled using the property in audit.cfg called audit.enabled
Includes a Solr Import/Export script documented here: https://wiki.lyrasis.org/display/DSDOC10x/Solr+Data+Management

Instructions for Reviewers

Once Backend and Frontend are both installed please make sure audit.enabled = true and that the new audit core has been deployed to the SOLR service.
Then start with creating/editing an item.

• There's a general section available for admin users in the admin sidebar to check the whole audit logs of the system (audit events related to all the items)
• Another audit section is available through the menu dropdown menu on the item page and only displays the item related events

The audit events include the changed performed on item metadata or dspace objects creations: metadata removed, added (replace is considered as removal and the new adding of a different metadata).
Bundle creation events are displayed as well.
Bitstream creation/deletion is shown and the bitstream checksum is displayed.
Metadata events show the information of the metadata involved (textvalue, authority, place and so on)
Informations about the user who performed these changes are displayed for each event

Checklist

This checklist provides a reminder of what we are going to look for when reviewing your PR. You need not complete this checklist prior to creating your PR (draft PRs are always welcome).
However, reviewers may request that you complete any actions in this list if you have not done so. If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

• My PR is created against the main branch of code (unless it is a backport or is fixing an issue specific to an older branch).
• My PR is small in size (e.g. less than 1,000 lines of code, not including comments & integration tests). Exceptions may be made if previously agreed upon.
• My PR passes Checkstyle validation based on the Code Style Guide.
• My PR includes Javadoc for all new (or modified) public methods and classes. It also includes Javadoc for large or complex private methods.
• My PR passes all tests and includes new/updated Unit or Integration Tests based on the Code Testing Guide.
• My PR includes details on how to test it. I've provided clear instructions to reviewers on how to successfully test this fix or feature.
• If my PR includes new libraries/dependencies (in any pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
• If my PR modifies REST API endpoints, I've opened a separate REST Contract PR related to this change.
• If my PR includes new configurations, I've provided basic technical documentation in the PR itself.
• If my PR fixes an issue ticket, I've linked them together.

[saschaszott]

@steph-ieffam , the recording of change events of an item should ideally begin only once the item enters the workflow, or alternatively when it is moved to the “in archive” state. While the item remains in the workspace, the changes are generally not relevant to admins.

[saschaszott]

@steph-ieffam , we need to decide what happens with the audit trail of a DSpace object when it is deleted.

[saschaszott]

@steph-ieffam , It should be evaluated how much overhead (time) is caused by writing the audit trail during a bulk import or bulk update. How significantly is the performance of operations affected when a large number of changes are made to a large number of DSpace objects?

[steph-ieffam]

@tdonohue all of your feedback are now addressed. Let me know if anything else is spotted

[tdonohue]

@steph-ieffam : Thanks for those updates! All my prior feedback is addressed, but I found a small issue in testing the solr-core-management script.

First off, I want to note that this script works well! I was able to successfully export my audit core to CSV. Then destroy all data in the audit core, and then restore it by reimporting from CSV.

However, the issue I ran into is that this solr-core-management script allows you to specify a --directory from the User Interface. See this screenshot:

This should not be allowed as it lets users from the Admin UI to arbitrarily write these files to whatever directory they want on the server. Additionally, it makes the script un-usable from the Admin UI because a user cannot see the exported content.

So, in my opinion, if we want this script to be available from the Admin UI, then it needs to be updated to work like the export and import scripts. The export script when run from the admin UI will not let the user specify a directory/file, but will let them download the results. The import script when run from the Admin UI will require the user to drag & drop or select a file to import.

The other option here is to simply remove the ability to run the solr-core-management tool from the Admin UI and make it a CLI tool only. We could always enhance it at a later date.

Once this minor issue is fixed, I'm +1 this backend PR.

[steph-ieffam]

Hey @tdonohue Your feedback has been addressed and now the Solr Management Script is no longer available in the scripts list on the UI. Thanks for the feedback

[github-actions]

Hi @steph-ieffam,
Conflicts have been detected against the base branch.
Please resolve these conflicts as soon as you can. Thanks!

[tdonohue]

@steph-ieffam : Thanks for the updates. I've verified my prior feedback has been addressed. However, there are still a few bugs I'm seeing from the User Interface. I've reported them in this review on the User Interface PR

(I suspect a few of those may be backend bugs, but I've not verified where the bug exists)

Beyond that, this PR looks good to me now

[paulo-graca]

Thank you, @steph-ieffam, for this contribution! I’m also following @tdonohue’s comments - when he considers it ready to move forward, you can count on my +1 as well.

[tdonohue]

👍 Thanks @steph-ieffam ! I retested and re-reviewed today. My feedback has been addressed & this is working well overall. I still can reproduce the duplicate dc.identifier.uri bug, but I've figured out it only occurs if you set identifiers.submission.register = true...so I'll move that to a follow-up ticket and assign it to you shortly.

This PR looks good enough as-is. and will be merged as it's already at +3.

Prior feedback was already addressed