[GITFLOW]merging 'release-1.70.0' into 'master'

docs/SERVER_CUSTOMIZATION.md

@@ -6,6 +6,7 @@ For this reason, there are a number of ways in which such a SORMAS instance can
 * **[Server Configuration](#server-configuration):** The basic server configuration is available as a .properties file, needs to be adjusted for every SORMAS instance, and is relevant for both production and development. This should be edited directly after installing the server.
 * **[Feature Configuration](#feature-configuration):** Most SORMAS features are optional and can be turned on or off directly in the database.
 * **[Disease Configuration](#disease-configuration):** SORMAS supports a large number of infectious diseases which can be enabled or disabled and further customized directly in the database.
+* **[Deletion Configuration](#deletion-configuration):** SORMAS can be configured to automatically delete entities in the database.
 * **[Infrastructure Data](#infrastructure-data):** Most infrastructure data (except countries and continents) are not shipped with SORMAS because they are country-specific. Importing the infrastructure data of your country (or creating some dummy data) is one of the first things you should do after setting up a SORMAS server.
 
 Beyond that, the Wiki contains even more customization options:
@@ -44,7 +45,7 @@ Changing these entries overrides that default value. Unlike with features, disea
 
 **Important: If you're using the mobile app, you also need to update the `changedate` to the current date and time whenever you change a disease configuration!** Otherwise the mobile applications will not be notified about the change.
 
-It is possible to adjust the following properties that define how the  diseases are handled:
+It is possible to adjust the following properties that define how the diseases are handled:
 
 * **`active`:** Whether this disease is used in this SORMAS instance. The concrete type of usage is specified by the other properties.
 * **`primaryDisease`:** Primary diseases are enabled for case surveillance while non-primary diseases can only be used for pathogen testing.
@@ -56,6 +57,22 @@ It is possible to adjust the following properties that define how the  diseases
 * **`extendedClassification`:** Whether this disease uses an extended case classification system that allows users to specify whether a case has been clinically, epidemiologically or laboratory-diagnostically confirmed.
 * **`extendedClassificationMulti`:** Whether the three confirmation properties used for extended classification can be specified individually, i.e. users can enter multiple sources of confirmation.
 
+## Deletion Configuration
+SORMAS can be set up to automatically delete entities after a specific time period. There are seven core entities for which automatic deletion can be enabled and configured: *Case, Contact, Event, Event Participant, Immunization, Travel Entry, and Campaign.* This configuration is currently only possible directly in the database via the `deleteconfiguration` table, which already contains a row for each of these entities. The table consists of the following columns:
+
+* **`entityType`:** The name of the entity that supports automatic deletion.
+* **`deletionReference`:** The reference date for the calculation of the date on which deletion takes place (see below).
+* **`deletionPeriod`:** The number of days after which an entity is deleted, starting with the deletion reference. The minimum is 7.
+
+Both `deletionReference` and `deletionPeriod` need to be filled in order for the automatic deletion to take place. Entities for which at least one of these fields is left empty will not be automatically deleted. Deletion is executed via a nightly cron job and might therefore not happen immediately when the deletion date has been reached.
+
+### Deletion Reference
+The `deletionReference` field has three possible values which define the date that is used to calculate whether an entity needs to be deleted (i.e., when the date calculated by subtracting the deletion period from the current date is before the deletion reference date, the entity is deleted).
+
+* **`CREATION`**: The creation date of the entity will be used.
+* **`END`**: The latest change date of the entity itself and any of its depending entities will be used. E.g. for cases, this includes but is not limited to its epi data, symptoms, or hospitalization.
+* **`ORIGIN`**: This is currently only implemented for travel entries and means that the report date of the entity will be used. If this is specified for any other entity, the deletion job will be stopped and throw an error.
+
 ## Infrastructure Data
 When you start a SORMAS server for the first time and the `createDefaultEntities` property is enabled, some default infrastructure data is generated to ensure that the server is usable and the default users can be created.
 It is recommended (and, unless you're working on a demo server, necessary) to archive this default data and import the official infrastructure data of the country or part of the country that you intend to use SORMAS in instead.

docs/SERVER_SETUP.md

@@ -103,6 +103,9 @@ chmod +x $(date +%F)/server-setup.sh
 * Unzip the ZIP archive (e.g. into you download directory)
 * Open Git Bash and navigate to the setup sub-directory
 
+### Auditing
+You can configure the audit logger of SORMAS by providing a Logback [configuration file](https://logback.qos.ch/manual/configuration.html) and setting the `audit.logger.config` property accordingly. An example is provided in `sormas-base/setup/audit-logback.xml`.  Not specifying a value for the property will effectively disable the audit log.
+
 ### Post-Installation Configuration
 
 * Optional: Open ``server-setup.sh`` in a text editor to customize the install paths, database access and ports for the server. The default ports are 6080 (HTTP), 6081 (HTTPS) and 6048 (admin). **Important:** Do not change the name of the database user. The pre-defined name is used in the statements executed in the database.

sormas-api/pom.xml

@@ -2,7 +2,7 @@
 	<parent>
 		<groupId>de.symeda.sormas</groupId>
 		<artifactId>sormas-base</artifactId>
-		<version>1.69.1</version>
+		<version>1.70.0</version>
 		<relativePath>../sormas-base</relativePath>
 	</parent>
 	<modelVersion>4.0.0</modelVersion>

sormas-api/src/main/java/de/symeda/sormas/api/ConfigFacade.java

@@ -88,16 +88,8 @@ public interface ConfigFacade {
 
 	int getInfrastructureSyncThreshold();
 
-	int getDaysAfterCaseGetsArchived();
-
-	int getDaysAfterEventGetsArchived();
-
-	int getDaysAfterEventParticipantGetsArchived();
-
 	int getDaysAfterSystemEventGetsDeleted();
 
-	int getDaysAfterTravelEntryGetsArchived();
-
 	GeoLatLon getCountryCenter();
 
 	boolean isMapUseCountryCenter();
@@ -120,7 +112,7 @@ public interface ConfigFacade {
 
 	PatientDiaryConfig getPatientDiaryConfig();
 
-	void validateExternalUrls();
+	void validateConfigUrls();
 
 	SormasToSormasConfig getS2SConfig();
 
@@ -149,4 +141,8 @@ public interface ConfigFacade {
 	long getDocumentUploadSizeLimitMb();
 
 	long getImportFileSizeLimitMb();
+
+	String getAuditLoggerConfig();
+
+	String getAuditSourceSite();
 }

sormas-api/src/main/java/de/symeda/sormas/api/CoreFacade.java

@@ -28,6 +28,8 @@ public interface CoreFacade<DTO extends EntityDto, INDEX_DTO extends Serializabl
 
     boolean isArchived(String uuid);
 
+    void delete(String uuid);
+
     boolean exists(String uuid);
 
     List<DTO> getAllAfter(Date date, Integer batchSize, String lastSynchronizedUuid);

sormas-api/src/main/java/de/symeda/sormas/api/EditPermissionType.java

@@ -0,0 +1,7 @@
+package de.symeda.sormas.api;
+
+public enum EditPermissionType {
+	ALLOWED,
+	ARCHIVING_STATUS_ONLY,
+	REFUSED
+}

sormas-api/src/main/java/de/symeda/sormas/api/audit/Constants.java

@@ -0,0 +1,72 @@
+package de.symeda.sormas.api.audit;
+
+import org.apache.commons.collections.set.UnmodifiableSet;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Set;
+
+public class Constants {
+
+	public static final Set<String> createPrefix =
+		Collections.unmodifiableSet(new HashSet<>(Arrays.asList("create", "generate", "build", "clone", "calculate")));
+	public static final Set<String> readPrefix = Collections.unmodifiableSet(
+		new HashSet<>(
+			Arrays.asList(
+				"count",
+				"get",
+				"is",
+				"has",
+				"does",
+				"validate",
+				"exists",
+				"read",
+				"import",
+				"find",
+				"query",
+				"load",
+				"check",
+				"uses",
+				"fetch")));
+	public static final Set<String> updatePrefix = Collections.unmodifiableSet(
+		new HashSet<>(
+			Arrays.asList(
+				"update",
+				"post",
+				"set",
+				"archive",
+				"dearchive",
+				"save",
+				"overwrite",
+				"convert",
+				"link",
+				"bulkAssign",
+				"write",
+				"cleanup",
+				"mark")));
+	public static final Set<String> deletePrefix = Collections.unmodifiableSet(new HashSet<>(Arrays.asList("delete", "merge", "remove")));
+	public static final Set<String> executePrefix = Collections.unmodifiableSet(
+		new HashSet<>(
+			Arrays.asList(
+				"send",
+				"start",
+				"share",
+				"signAndEncrypt",
+				"decryptAndVerify",
+				"reject",
+				"report",
+				"unlink",
+				"notify",
+				"register",
+				"cancel",
+				"end",
+				"accept",
+				"sync",
+				"request",
+				"revoke",
+				"reset",
+				"enable",
+				"disable")));
+
+}

sormas-api/src/main/java/de/symeda/sormas/api/campaign/CampaignFacade.java

@@ -5,6 +5,7 @@
 import javax.ejb.Remote;
 
 import de.symeda.sormas.api.CoreFacade;
+import de.symeda.sormas.api.EditPermissionType;
 import de.symeda.sormas.api.campaign.diagram.CampaignDashboardElement;
 
 @Remote
@@ -16,9 +17,9 @@ public interface CampaignFacade extends CoreFacade<CampaignDto, CampaignIndexDto
 
 	List<CampaignDashboardElement> getCampaignDashboardElements(String campaignUuid);
 
-	void deleteCampaign(String uuid);
-
 	List<String> getAllActiveUuids();
 
 	void validate(CampaignReferenceDto campaignReferenceDto);
+
+	EditPermissionType isCampaignEditAllowed(String caseUuid);
 }

sormas-api/src/main/java/de/symeda/sormas/api/campaign/data/CampaignFormDataEntry.java

@@ -18,8 +18,6 @@
 import static de.symeda.sormas.api.utils.FieldConstraints.CHARACTER_LIMIT_DEFAULT;
 
 import java.io.Serializable;
-import java.util.Collection;
-import java.util.Iterator;
 import java.util.Objects;
 
 import javax.validation.constraints.Size;
@@ -78,17 +76,6 @@ public String toString() {
 		return value.toString();
 	}
 
-	public static void removeNullValueEntries(Collection<CampaignFormDataEntry> entries) {
-
-		Iterator<CampaignFormDataEntry> iterator = entries.iterator();
-		while (iterator.hasNext()) {
-			CampaignFormDataEntry entry = iterator.next();
-			if (entry.value == null) {
-				iterator.remove();
-			}
-		}
-	}
-
 	/**
 	 * Needed. Otherwise hibernate will persist whenever loading,
 	 * because hibernate types creates new instances that aren't equal.

sormas-api/src/main/java/de/symeda/sormas/api/caze/CaseDataDto.java

@@ -15,6 +15,7 @@
 
 package de.symeda.sormas.api.caze;
 
+import static de.symeda.sormas.api.CountryHelper.COUNTRY_CODE_FRANCE;
 import static de.symeda.sormas.api.CountryHelper.COUNTRY_CODE_GERMANY;
 import static de.symeda.sormas.api.CountryHelper.COUNTRY_CODE_SWITZERLAND;
 import static de.symeda.sormas.api.utils.FieldConstraints.CHARACTER_LIMIT_BIG;
@@ -251,9 +252,21 @@ public class CaseDataDto extends SormasToSormasShareableDto {
 	@Outbreaks
 	@Required
 	private UserReferenceDto reportingUser;
+	@HideForCountries(countries = {
+		COUNTRY_CODE_FRANCE,
+		COUNTRY_CODE_GERMANY,
+		COUNTRY_CODE_SWITZERLAND })
 	private Date regionLevelDate;
+	@HideForCountries(countries = {
+		COUNTRY_CODE_FRANCE,
+		COUNTRY_CODE_GERMANY,
+		COUNTRY_CODE_SWITZERLAND })
 	private Date nationalLevelDate;
 	@Outbreaks
+	@HideForCountries(countries = {
+		COUNTRY_CODE_FRANCE,
+		COUNTRY_CODE_GERMANY,
+		COUNTRY_CODE_SWITZERLAND })
 	private Date districtLevelDate;
 	@Outbreaks
 	@Required

sormas-api/src/main/java/de/symeda/sormas/api/caze/CaseExportDto.java

@@ -1425,6 +1425,7 @@ public Date getDeathDate() {
 		CaseExportType.CASE_SURVEILLANCE })
 	@ExportProperty(BURIAL_INFO)
 	@ExportGroup(ExportGroupType.SENSITIVE)
+	@HideForCountries
 	public BurialInfoDto getBurialInfo() {
 		return burialInfo;
 	}

sormas-api/src/main/java/de/symeda/sormas/api/caze/CaseFacade.java

@@ -21,7 +21,6 @@
 import java.util.Collection;
 import java.util.Date;
 import java.util.List;
-import java.util.Map;
 import java.util.Random;
 
 import javax.ejb.Remote;
@@ -33,6 +32,7 @@
 import de.symeda.sormas.api.CaseMeasure;
 import de.symeda.sormas.api.CoreFacade;
 import de.symeda.sormas.api.Disease;
+import de.symeda.sormas.api.EditPermissionType;
 import de.symeda.sormas.api.Language;
 import de.symeda.sormas.api.common.Page;
 import de.symeda.sormas.api.contact.ContactReferenceDto;
@@ -91,7 +91,9 @@ List<CaseExportDto> getExportList(
 
 	CaseDataDto getCaseDataByUuid(String uuid);
 
-	CaseDataDto save(@Valid @NotNull CaseDataDto dto, Boolean systemSave) throws ValidationRuntimeException;
+	CaseDataDto save(@Valid @NotNull CaseDataDto dto, boolean systemSave) throws ValidationRuntimeException;
+
+	CoreAndPersonDto<CaseDataDto> save(@Valid @NotNull CoreAndPersonDto<CaseDataDto> dto) throws ValidationRuntimeException;
 
 	void setSampleAssociations(ContactReferenceDto sourceContact, CaseReferenceDto cazeRef);
 
@@ -127,10 +129,10 @@ Long countCasesForMap(
 
 	List<CaseDataDto> getAllCasesOfPerson(String personUuid);
 
-	void deleteCase(String caseUuid) throws ExternalSurveillanceToolException;
-
 	List<String> deleteCases(List<String> caseUuids);
 
+	void deleteWithContacts(String caseUuid);
+
 	void deleteCaseAsDuplicate(String caseUuid, String duplicateOfCaseUuid) throws ExternalSurveillanceToolException;
 
 	Date getOldestCaseOnsetDate();
@@ -141,6 +143,8 @@ Long countCasesForMap(
 
 	boolean isDeleted(String caseUuid);
 
+	boolean isArchived(String caseUuid);
+
 	List<String> getArchivedUuidsSince(Date since);
 
 	List<String> getDeletedUuidsSince(Date since);
@@ -167,9 +171,7 @@ Long countCasesForMap(
 
 	FollowUpPeriodDto calculateFollowUpUntilDate(CaseDataDto caseDto, boolean ignoreOverwrite);
 
-	boolean isCaseEditAllowed(String caseUuid);
-
-	boolean hasPositiveLabResult(String caseUuid);
+	EditPermissionType isCaseEditAllowed(String caseUuid);
 
 	List<CaseFollowUpDto> getCaseFollowUpList(
 		CaseCriteria caseCriteria,
@@ -179,6 +181,12 @@ List<CaseFollowUpDto> getCaseFollowUpList(
 		Integer max,
 		List<SortProperty> sortProperties);
 
+	Page<CaseFollowUpDto> getCaseFollowUpIndexPage(
+		@NotNull CaseFollowUpCriteria criteria,
+		Integer offset,
+		Integer max,
+		List<SortProperty> sortProperties);
+
 	void sendMessage(List<String> caseUuids, String subject, String messageContent, MessageType... messageTypes);
 
 	long countCasesWithMissingContactInformation(List<String> caseUuids, MessageType messageType);
@@ -187,7 +195,7 @@ List<CaseFollowUpDto> getCaseFollowUpList(
 
 	List<String> getUuidsNotShareableWithExternalReportingTools(List<String> caseUuids);
 
-	void saveBulkCase(
+	Integer saveBulkCase(
 		List<String> caseUuidList,
 		@Valid CaseBulkEditData updatedCaseBulkEditData,
 		boolean diseaseChange,
@@ -220,5 +228,9 @@ void saveBulkEditWithFacilities(
 
 	PreviousCaseDto getMostRecentPreviousCase(PersonReferenceDto person, Disease disease, Date startDate);
 
-	Map<ReinfectionDetail, Boolean> cleanUpReinfectionDetails(Map<ReinfectionDetail, Boolean> reinfectionDetails);
+	void archive(String entityUuid, Date endOfProcessingDate, boolean includeContacts);
+
+	void archive(List<String> entityUuids, boolean includeContacts);
+
+	void dearchive(List<String> entityUuids, String dearchiveReason, boolean includeContacts);
 }

sormas-api/src/main/java/de/symeda/sormas/api/caze/CaseFollowUpCriteria.java

@@ -0,0 +1,17 @@
+package de.symeda.sormas.api.caze;
+
+import java.util.Date;
+
+public class CaseFollowUpCriteria extends CaseCriteria {
+
+	private Date referenceDate;
+	private int interval;
+
+	public Date getReferenceDate() {
+		return referenceDate;
+	}
+
+	public int getInterval() {
+		return interval;
+	}
+}