Switch to Inno Setup to build the windows installer

We currently use sbt-native-packager to build the CLI Windows installer,
which does so by generating wsx files and running Wix to compile them to
an msi. But this has a number of complications:

- Wix does not currently create reproducible MSI files
- We currently maintain custom wsx files in addition to having to
  manually transform the wsx files that the plugin generates
- The wsx files only work with Wix3--we can't easily upgrade to new
  versions of Wix that might fix bugs or add new features (like
  reproducibility)
- Even if we could convert the wsx files to a newer version, Wix v6
  switches to a license that is likely not ASF compatible so we are
  essentially stuck with older versions of Wix
- In order to build Wix on Linux, we sort of hack things by replacing
  the Wix light.exe and candle.exe files with scripts that runs them in
  wine, complicating Wix setup

An alternative to Wix is using wixl, a Gnome developed tool to build MSI
files on Linux. This solves some of the above problems, but wixl still
fairly immature and has the same reproducibility issues.

However, switching to Inno Setup to build the Windows installer solves
many of these problems:

- It has an ASF compatible license
- It creates reproducible builds
- It is easy to run under wine without custom scripts
- The configuration file is significantly more simple and supports
  things like paramaterization. The only real complication is we now
  need to manually modify the Path since Inno Setup has no native
  support for this

Note that this no longer uses MSI's, so we lose some of the native
install capabilities that MSI's provide, but users are unlikely to
notice the differences.

One drawback is that Inno Setup isn't supported by sbt-native-packager.
But executing the InnoSetup CLI compiler with the necessary arguments is
fairly straightforward. A new "packageWindowsBin" SBT task is created in
just a handful of lines that executes the compiler, including support
for running under Wine for non-Windows operation systems.

With this change, instead of the old command to build windows installer:

    sbt daffodil-cli/Windows/packageBin

we now run this:

    sbt daffodil-cli/packageWindowsBin

The only setup for this to work is Inno Setup must be manually
downloaded and installed, but this is pretty straightforward and works
well in wine. The setup is also only needed when building windows
installers--if you don't care about that and never run the above command
then you don't need to install Inno Setup or wine.

This also updates the release candidate container to install Inno Setup
and build the new exe file. When we switch to the new tag-based release
process, these changes will need to be copied to that infrastructure.

Compatibility/Deprecation:

The windows installer has switched from an MSI to an EXE. Installing the
new version will not uninstall the previous version. Previously
installed Daffodil MSIs should be uninstalled prior to installing new
EXEs.

DAFFODIL-2980

Author: stevedlawrence

containers/release-candidate/Dockerfile

@@ -40,37 +40,27 @@ RUN \
     unzip \
     vim-minimal \
     wine \
-    winetricks && \
+    winetricks \
+    xorg-x11-server-Xvfb && \
   dnf clean all
 
 # Enable sbt-pgp plugin
 COPY src/plugins.sbt /root/.sbt/1.0/plugins/
 
-# WIX is a Microsoft Windows Installer creation tool kit.
-#
-# Install wix, including changes to allow WiX to run in wine on Linux. See
-# src/wix_wine.sh for more details on why we need to do this and how it works
-#
-# Updating WIX should be done only if there is a specific need (for security, or other compelling reason)
-# because it is likely things will break and the release scripts/process will have to adapt.
-# The WIX version 3.11.2 is hard coded into these script lines as tokens wix3112rtm and wix311.
-#
-# In order to ensure we are downloading and using the exact WIX binary we have tested and trust
-# we verify the sha512 is the same as the one expected. This protects us from if someone
-# was to get github credentials allowing them to change the wix binaries on github.
-# If WIX is updated to a newer version, this sha512 will need to be recomputed.
-ARG WIXSHA=6fd961c85e1e6adafb99ef50c9659e3eb83c84ecaf49f523e556788845b206e1857aba2c39409405d4cda1df9b30a552ce5aab808be5f8368a37a447d78d1a05
-#
+# Download and install Inno Setup via wine. Even though we provide options to
+# make Inno Setup a headless install, wine still thinks it wants to open a
+# window so we need install with a temporary virtual X server with xvfb-run.
+# Also ensure it is installed in the directory where the deafult Daffodil SBT
+# configuration expects it--different wine configurations could install it
+# somewhere else.
 RUN \
-  curl -sS -L https://github.com/wixtoolset/wix3/releases/download/wix3112rtm/wix311-binaries.zip -o wix311-binaries.zip && \
-  echo "$WIXSHA wix311-binaries.zip" | sha512sum --quiet -c - && \
-  mkdir /opt/wix311 && \
-  unzip -q wix311-binaries.zip -d /opt/wix311/ && \
-  rm wix311-binaries.zip
-RUN mv /opt/wix311/{candle.exe,real-candle.exe}
-RUN mv /opt/wix311/{light.exe,real-light.exe}
-COPY src/wix_wine.sh /opt/wix311/candle.exe
-COPY src/wix_wine.sh /opt/wix311/light.exe
+  INNO_SETUP_EXE="innosetup-6.4.1.exe" && \
+  EXPECTED_SHA=5dfc9999e2feafa28754baaf80cf73ac96414228b94a1132a919554a822c892810197305d9355885b9ac408c214691cd45279fc2df3a891fbebc4f8eb86bac87 && \
+  curl -sS -L https://files.jrsoftware.org/is/6/$INNO_SETUP_EXE -o $INNO_SETUP_EXE && \
+  echo "$EXPECTED_SHA $INNO_SETUP_EXE" | sha512sum --quiet -c - && \
+  winetricks -q win10 && \
+  xvfb-run wine $INNO_SETUP_EXE /VERYSILENT /SUPPRESSMSGBOXES /TASKS= '/DIR=C:\\Program Files (x86)\\Inno Setup 6' && \
+  rm $INNO_SETUP_EXE
 
 # Install a pinned version of sbt. This is used only as a bootstrap so does not need to be updated.
 # This version of sbt is not used to build any Daffodil project parts, as they specify their own sbt versions.

containers/release-candidate/src/daffodil-release-candidate

@@ -295,7 +295,7 @@ case $PROJECT_REPO in
       "+compile" \
       "+$SBT_PUBLISH" \
       "daffodil-cli/Rpm/packageBin" \
-      "daffodil-cli/Windows/packageBin" \
+      "daffodil-cli/packageWindowsBin" \
       "daffodil-cli/Universal/packageBin" \
       "daffodil-cli/Universal/packageZipTarball" \
       "unidoc" \
@@ -306,9 +306,8 @@ case $PROJECT_REPO in
     cp daffodil-cli/target/universal/apache-daffodil-*.tgz $DAFFODIL_RELEASE_DIR/bin/
     cp daffodil-cli/target/universal/apache-daffodil-*.zip $DAFFODIL_RELEASE_DIR/bin/
     cp daffodil-cli/target/rpm/RPMS/noarch/apache-daffodil-*.rpm $DAFFODIL_RELEASE_DIR/bin/
-    MSI_NAME=$(basename $DAFFODIL_RELEASE_DIR/bin/*.zip .zip).msi
-    cp daffodil-cli/target/windows/Daffodil.msi $DAFFODIL_RELEASE_DIR/bin/$MSI_NAME
-    chmod -x $DAFFODIL_RELEASE_DIR/bin/$MSI_NAME
+    cp daffodil-cli/target/windows/apache-daffodil-*.exe $DAFFODIL_RELEASE_DIR/bin/
+    chmod -x $DAFFODIL_RELEASE_DIR/bin/*.exe
 
     echo "Embedding RPM Signature..."
     rpmsign --define "_gpg_name $PGP_SIGNING_KEY_ID" --define "_binary_filedigest_algorithm 10" --addsign $DAFFODIL_RELEASE_DIR/bin/*.rpm

containers/release-candidate/src/wix_wine.sh

@@ -15,20 +15,21 @@
  * limitations under the License.
  */
 
-import scala.xml.Attribute
-import scala.xml.transform.RewriteRule
-import scala.xml.transform.RuleTransformer
+import scala.sys.process.Process
+
+import sbt.internal.util.Util.isWindows
 
 enablePlugins(JavaAppPackaging)
 enablePlugins(RpmPlugin)
-enablePlugins(WindowsPlugin)
+
+lazy val packageWindowsBin = taskKey[Unit]("Generate windows installer")
+lazy val isccPath = settingKey[String]("Path to the Inno Setup ISCC.exe file")
 
 executableScriptName := "daffodil"
 
 Universal / packageName := "apache-daffodil-" + version.value + "-bin" //tarball name
 Linux / packageName := executableScriptName.value
 Rpm / packageName := "apache-" + executableScriptName.value
-Windows / packageName := executableScriptName.value
 
 val optSourceDateEpoch = scala.util.Properties.envOrNone("SOURCE_DATE_EPOCH")
 
@@ -158,184 +159,47 @@ rpmPrefix := Some(defaultLinuxInstallLocation.value)
 // Windows configuration
 //
 
-//
-// Here we set the variables that are supported by the sbt Native Packager plug-in.
-// We also get fairly aggressive in editing/modifying the XML in order
-// to control and use some specific features that are supported by WiX
-// but which are not properly suported by the sbt plug-in.
-//
-
-// Force the correct installation directory name. This overwrites
-// 'daffodil-cli', which is the directory that we invoke sbt in.
-// The sbt WiX plug-in incorrectly assumes that the directory of
-// invocation is the same name as the direcotry you eventually
-// want to install into.
-Windows / name := "Daffodil"
-
-// The Windows packager sbt plug-in maps the packageSummary variable
-// into the WiX productName field. Another strange choice.
-Windows / packageSummary := "Daffodil"
-
-// The Windows packager sbt plug-in limits the length of the packageDescription
-// field to a single line. Use the short packageSummary from the RPM config.
-Windows / packageDescription := (Rpm / packageSummary).value
-
-// Use the same version number as in the rpm, which has SNAPSHOT removed if it
-// exists. Windows version numbers has no concept of a "snapshot build", only
-// major, minor, patch, and build. So Windows MSI versions do not differentiate
-// between snapshots and non-snapshots.
-Windows / version := (Rpm / version).value
-
-// Required and critical GUIDs. Ironically the ProductId is unique
-// to a given release, but UpgradeId must NEVER change! This may
-// seem conter-intuitive, but the UpgradeId is actually what ties
-// the product to it's upgrades and the product is actually unique
-// each time it is released, so there is some semblance of logic
-// to this scheme.
-wixProductUpgradeId := "4C966AFF-585E-4E17-8CC2-059FD70FEC77"
-
-// Light options. Bring in standard dialog boxes and localization.
-// The suppression of ICE61 is required as we *DO* permit
-// re-installation of the same version. Despite the presence of
-// specific XML to enable this, the WiX compiler and linker
-// complain about it unless you specifically suppress the warning.
-lightOptions ++= Seq(
-  "-sval", // validation does not currently work under Wine, this disables that
-  "-sice:ICE61",
-  "-loc",
-  ((Windows / sourceDirectory).value / "Product_en-us.wxl").toString
-)
-
-// Build an RTF version of the license file for display in the license
-// acceptance dialog box. This file will also be sent to the
-// printer when and if the user asks for hard copy via the 'print' button.
-wixProductLicense := {
-  // Make sure the target direcotry exists.
-  (Windows / target).value.mkdirs()
-
-  // This target file doesn't exist until placed there by the build.
-  val targetLicense = (Windows / target).value / "LICENSE.rtf"
-  val sourceLicense = baseDirectory.value / "bin.LICENSE"
-  // somehow convert sourceLicense into RTF and store at targetLicense
-  val rtfHeader = """{\rtf {\fonttbl {\f0 Arial;}} \f0\fs18"""
-  val rtfFooter = """}"""
-
-  val licenseLines = scala.io.Source.fromFile(sourceLicense, "UTF-8").getLines
-  val writer = new java.io.PrintWriter(targetLicense, "UTF-8")
-  // windows style line endings in the license are required by the WiX toolkit
-  writer.write(rtfHeader + "\r\n")
-  licenseLines.foreach { line =>
-    writer.write(line + """\line""" + "\r\n")
-  }
-  writer.write(rtfFooter + "\r\n")
-  writer.close
-  Option(targetLicense)
-}
-
-// Use the wixFiles variable to add in the Daffodil-specific dialog
-// boxes and sequence.
-wixFiles ++= Seq(
-  (Windows / sourceDirectory).value / "WixUI_Daffodil.wxs"
-)
-
-// The sbt Native Packager plug-in assumes that we want to give the user
-// a Feature Tree to select from. One of the 'features' that the plug-in
-// offers up is a set of all shortcuts and menu links. Daffodil is
-// actually a command-line executable, so we do not include
-// configuration links as they are unnecessary. From a practical
-// standpoint the user must invoke a command shell in a window
-// before they can invoke Daffodil anyway.
-wixFeatures := {
-  val features = wixFeatures.value
-  features.filter { _.id != "AddConfigLinks" }
+/**
+ * If building on a non-Windows machine, uses winepath to convert a Unix file path to a Windows
+ * path that can be used with wine. If this is Windows, just return the file path as a string
+ */
+def winpath(file: File): String = {
+  if (isWindows) file.toString
+  else Process(Seq("winepath", "-w", file.toString)).!!
 }
 
-// Make sure that we don't use an MSI installer that is older than
-// version 2.0. It also fixes the comment attribute that hangs
-// out on the Package keyword.
-wixPackageInfo := wixPackageInfo.value.copy(
-  installerVersion = "200",
-  comments = "!(loc.Comments)"
-)
-
-// Fix the XML that is associated with the installable files and directories.
-wixProductConfig := {
-  // Pick up the generated code.
-  val pc = wixProductConfig.value
-
-  // Replace the default headline banner and Welcome/Exit screen
-  // bitmaps with the custom ones we developed for Daffodil.
-  val banner = <WixVariable Id="WixUIBannerBmp" Value={
-    ((Windows / sourceDirectory).value / "banner.bmp").toString
-  } />
-  val dialog = <WixVariable Id="WixUIDialogBmp" Value={
-    ((Windows / sourceDirectory).value / "dialog.bmp").toString
-  } />
-
-  // Reference the Daffodil-specific User Interface (dialog box) sequence.
-  val ui = <UI><UIRef Id="WixUI_Daffodil" /></UI>
-
-  // Make sure we abort if we are not installing on Windows 95 or later.
-  val osCondition =
-    <Condition Message="!(loc.OS2Old)"><![CDATA[Installed OR (VersionNT >= 400)]]></Condition>
-
-  // Define icons (ID should not be longer than 18 chars and must end with ".exe")
-  val icon = Seq(
-    <Icon Id="Daffodil.ico" SourceFile={
-      ((Windows / sourceDirectory).value / "apache-daffodil.ico").toString
-    } />,
-    <Property Id="ARPPRODUCTICON" Value="Daffodil.ico" />
-  )
-
-  // String together the additional XML around the generated directory and file lists.
-  val pcGroup = pc.asInstanceOf[scala.xml.Group]
-  val newNodes = osCondition ++ icon ++ pcGroup.nodes ++ dialog ++ banner ++ ui
-  val pcWithNewNodes = pcGroup.copy(nodes = newNodes)
-
-  // Change (edit) some items inside the directory/files list.
-  val pcRewriteRule = new RewriteRule {
-    override def transform(n: scala.xml.Node): Seq[scala.xml.Node] = n match {
-
-      // We want to comply with the Windows standard pattern of
-      // installing at /Program Files/ManufacturerName/Application
-      // This case effectively inserts the manufacturer name into
-      // the XML as a directory to comply with the standard.
-      case e: scala.xml.Elem if (e \ "@Name").text == "PFiles" => {
-        val apacheDir = <Directory Id="ProgramFilesApache" Name="!(loc.ManufacturerName)" />
-        val apacheDirWithChild = apacheDir.copy(child = e.child)
-        e.copy(child = apacheDirWithChild)
-      }
-
-      // We *ARE* going to allow the user to repair and reinstall
-      // the same exact version, so we need to add an attribute
-      // to the MajorUpgrade keyword.  This will trigger an 'ICE61'
-      // error that we suppress on the 'light' linker command line.
-      case e: scala.xml.Elem if e.label == "MajorUpgrade" => {
-        e % scala.xml.Attribute("", "AllowSameVersionUpgrades", "yes", e.attributes)
-      }
-
-      // Fixup for registry key.
-      case e: scala.xml.Elem if e.label == "RegistryValue" => {
-        val attribs = e.attributes.remove("Key")
-        e % scala.xml.Attribute(
-          "",
-          "Key",
-          """Software\Apache\Installed Products\Daffodil""",
-          attribs
-        )
-      }
-
-      // The WixUI_FeatureTree reference has to be removed so that
-      // our custom Daffodil UI can operate properly.
-      case e: scala.xml.Elem
-          if e.label == "UIRef" && (e \ "@Id").text == "WixUI_FeatureTree" => {
-        scala.xml.NodeSeq.Empty
-      }
-      case `n` => n
+/**
+ * Set the default path to the ISCC compiler to the Windows path. Since we run it with wine on
+ * Linux, this path should work on both Windows and Linux. Depending on the wine config, windows
+ * version, or how Inno Setup is installed, this might need to be set to a different value.
+ */
+isccPath := "C:\\Program Files (x86)\\Inno Setup 6\\ISCC.exe"
+
+packageWindowsBin := {
+  (Universal / stage).value
+
+  // if SOURCE_DATE_EPOCH is defined, then we use that as the TouchDate/Time settings to set
+  // embedded timestamps, which allows for reproducible builds
+  val (touchDate, touchTime) = optSourceDateEpoch
+    .map { epoch =>
+      val fmt = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
+      fmt.setTimeZone(java.util.TimeZone.getTimeZone("UTC"))
+      val time = fmt.format(new java.util.Date(epoch.toLong * 1000))
+      val dateTime = time.split(" ")
+      (dateTime(0), dateTime(1))
     }
-  }
-
-  // Now apply all the edits in the RewriteRule defined above.
-  val newXml = new RuleTransformer(pcRewriteRule).transform(pcWithNewNodes)
-  <xml:group>{newXml}</xml:group>
+    .getOrElse(("current", "current"))
+
+  val optWine = if (!isWindows) Some("wine") else None
+  val isccCmd = optWine ++: Seq(
+    isccPath.value,
+    "/O" + winpath(target.value / "windows"),
+    "/F" + (Universal / packageName).value,
+    "/DVERSION=" + version.value,
+    "/DBASEDIR=" + winpath(baseDirectory.value),
+    "/DTOUCHDATE=" + touchDate,
+    "/DTOUCHTIME=" + touchTime,
+    winpath(baseDirectory.value / "src" / "windows" / "apache-daffodil.iss")
+  )
+  Process(isccCmd).!!
 }

daffodil-cli/build.sbt

@@ -0,0 +1,102 @@
+; Licensed to the Apache Software Foundation (ASF) under one or more
+; contributor license agreements.  See the NOTICE file distributed with
+; this work for additional information regarding copyright ownership.
+; The ASF licenses this file to You under the Apache License, Version 2.0
+; (the "License"); you may not use this file except in compliance with
+; the License.  You may obtain a copy of the License at
+;
+;     http://www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[Setup]
+; Do not change the AppID, this GUID uniquely identifies Daffodil
+AppId={{4C966AFF-585E-4E17-8CC2-059FD70FEC77}
+AppName=Apache Daffodil
+AppPublisher=Apache Daffodil <dev@apache.daffodil.org>
+AppPublisherURL=https://daffodil.apache.org
+AppVerName=Apache Daffodil v{#VERSION}
+AppVersion={#VERSION}
+ChangesEnvironment=yes
+Compression=none
+DefaultDirName={autopf}\Apache\Daffodil
+DisableProgramGroupPage=yes
+DisableWelcomePage=no
+LicenseFile={#BASEDIR}\bin.LICENSE
+PrivilegesRequired=admin
+SetupIconFile={#BASEDIR}\src\windows\apache-daffodil.ico
+UninstallDisplayIcon={app}\apache-daffodil.ico
+TimeStampsInUTC=yes
+TouchDate={#TOUCHDATE}
+TouchTime={#TOUCHTIME}
+WizardImageFile={#BASEDIR}\src\windows\dialog.bmp
+WizardStyle=modern
+
+[Languages]
+Name: "english"; MessagesFile: "compiler:Default.isl"
+
+[Files]
+Source: "{#BASEDIR}\target\universal\stage\*"; DestDir: "{app}"; \
+  Flags: ignoreversion recursesubdirs touch
+Source: "{#BASEDIR}\src\windows\apache-daffodil.ico"; DestDir: "{app}"; \
+  Flags: ignoreversion touch
+
+[InstallDelete]
+; Inno Setup does not delete files from previous installations when upgrading
+; to a new version. For most of our files this is fine since we'll just overwrite
+; the old files with  the new files. But the Daffodil CLI adds everything in the
+; lib directory to the classpath, so we need to make sure to remove old jars from
+; previous installs. Rather than trying to list all possible old jars, just
+; delete the entire lib directory. Note that if users put plugin jars in this
+; directory, they will also be deleted, but they shouldn't do that--plugin jars
+; should live in a separate directory and added to DAFFODIL_CLASSPATH.
+Type: files; Name: "{app}\lib\*"
+
+[Code]
+// Inno Setup does not have a built-in way to add the {app}\bin directory to
+// PATH. This is important since this is a CLI tool so it won't be started by
+// things like shortscuts or the start menu so really does want to be on the
+// PATH. The below code modifies PATH to add or remove the bin directory
+// depending on if we are installing or uninstalling, making sure not to re-add
+// it if a previous install already added it.
+procedure ModifyPath(Path: string; Installing: boolean);
+var
+  CurrentPaths: string;
+  PathList: TStringList;
+  PathIndex: Integer;
+begin
+  RegQueryStringValue(HKEY_LOCAL_MACHINE, 'System\CurrentControlSet\Control\Session Manager\Environment', 'Path', CurrentPaths);
+
+  PathList := TStringList.Create;
+  PathList.Delimiter := ';';
+  PathList.StrictDelimiter := True;
+  PathList.CaseSensitive := False;
+  PathList.DelimitedText := CurrentPaths;
+
+  PathIndex := PathList.IndexOf(Path);
+
+  if Installing and (PathIndex = -1) then
+    PathList.Add(Path)
+  else if not Installing and (PathIndex <> -1) then
+    PathList.Delete(PathIndex);
+
+  RegWriteStringValue(HKEY_LOCAL_MACHINE, 'System\CurrentControlSet\Control\Session Manager\Environment', 'Path', PathList.DelimitedText);
+
+  PathList.Free;
+end;
+
+procedure CurStepChanged(CurStep: TSetupStep);
+begin
+  if CurStep = ssPostInstall then
+    ModifyPath(ExpandConstant('{app}\bin'), true);
+end;
+
+procedure CurUninstallStepChanged(CurStep: TUninstallStep);
+begin
+  if CurStep = usPostUninstall then
+    ModifyPath(ExpandConstant('{app}\bin'), false);
+end;

daffodil-cli/src/windows/Product_en-us.wxl

@@ -36,7 +36,6 @@ object Rat {
 
     // images used for the windows installer
     file("daffodil-cli/src/windows/apache-daffodil.ico"),
-    file("daffodil-cli/src/windows/banner.bmp"),
     file("daffodil-cli/src/windows/dialog.bmp"),
 
     // generated code examples