pact with aeson 2 (#1565)

* pact with aeson-2 * ... * update ethereum pin * [WIP] ea * update ethereum pin * update freeze file * More changes to setup Nix for the new aeson2 work (#1581) * use LegacyValue * pact-json * Revert "[WIP] ea" This reverts commit 50a42ad. * confirmed and updated empty-block-tests results * confirmed and updated new-block-0 results * ... empty-block-tests * ProofPactEvent * fix roundtrip tests for PactEvent * fix block time in spv tests * ... * HasTextRepresentation and consistent show instances in Payload * legacy encoding in mkPayloadWithTextOld * update ea and genesis headers * update pact pin * ghc 9.4 * update freeze file * update mac build * update pact pin * update pact and pact-json pins * update compiler versions * fix encoding of pact exceptions * [TEMPORARY] add some debug logging to test * [TEMPORARY] include tests in container image * [TEMPORARY] build image with ghc-9.4.4 * build with ghc-9.6 * build with ghc-9.6 in CI * create docker file for 9.6.1 * remove use of deprecated TypeInType * make ghci happy * update cabal to 3.10 * ... * temporarily disable build with 9.4 * ... * ... * sort db keys using legacy ordering * fix new-block-0 golden outputs * update pact pin * ... * update pact pin * update github actions * update pact pin * update pact pin * update pact pin * update pact pin * update pact pin * update pact pin * wip error forking * errors back in chainweb, future proof read init modules * update pact and massiv pin * update pact pin * bump limit for replay * update pact pin * forking tests, fix forking for pact47 * some more strictness * allow-newer ghc-prim * cleanup cabal.project * pact pin update * update pact pin * more strictness * fix nix build -- step 1 * fix nix build -- step 2 * fix nix build -- step 3 * fix nix build -- step 4 * fix nix build -- step 5 * fix nix build -- step 6 * fix nix build -- step 7 * fix nix build -- step 8 * Fix source-repository-package sha256s * update pact pin * nix fixes to "pact with aeson 2" (#1638) * update ethereum package * default-flake.nix fixups - upgrade compiler to 9.6.1 - pkgs: add config = {}; and overlays = [];, for purity - disable cabal.project.freeze (in nix only) when ghc != 8.10.7 - export more attributes that are useful for debugging * bump ethereum commit again * bump ethereum package * update ethereum pin * update pact pin * update pact pin * relax some redundant upper bounds * update pact and pact-json pins * replace aeson encode by pact-json encode in single chain tests * update pact pin * allow newer base-compat-batteries * remove redundant source-repository-package entry for massiv * Fix bench * update pin * remove support for GHC < 9.6 * freeze file for GHC 9.6 * fix JSON encoding for latest pact pin * enable fork * make tests compile * ok but fix tests for real this time * fix legacy behavior of block module cache * update genesis headers * update nix pins for pact and pact-json * update nix hashes of pact(-json) and nix compiler to ghc962 * update hackage.nix flake input * Fix fork heights * Apply suggestions from code review Co-authored-by: Edmund Noble <[email protected]> * remove unused module Chainweb.BlockHeader.Genesis * remove redundant import * use ghc-9.6.2 in Dockerfile * use ghc 9.6.2 in CI * update freeze file * Update the Pact repository pin --------- Co-authored-by: John Wiegley <[email protected]> Co-authored-by: Jose <[email protected]> Co-authored-by: Enis Bayramoğlu <[email protected]> Co-authored-by: chessai <[email protected]> Co-authored-by: Edmund Noble <[email protected]> Co-authored-by: chessai <[email protected]> Co-authored-by: Edmund Noble <[email protected]>
kadena-io · Jul 27, 2023 · 760c171 · 760c171
1 parent c123bda
commit 760c171
Show file tree

Hide file tree

Showing 117 changed files with 1,721 additions and 1,387 deletions.
diff --git a/.github/workflows/applications.yml b/.github/workflows/applications.yml
@@ -119,18 +119,21 @@ jobs:
         fi
 
         echo "tag-suffix=$TAG_SUFFIX" >> $GITHUB_OUTPUT
+
+    # BUILD MATRIX
+    #
     - name: Create matrix
       id: set-matrix
       run: |
         MATRIX="$(jq -c '.' <<EOF
         {
-          "ghc": ["8.10.7", "9.0.2"],
+          "ghc": ["9.6.2"],
           "cabal": ["3.10"],
           "os": ["ubuntu-20.04", "ubuntu-22.04"],
           "use-freeze-file": ["false"],
           "include": [
             {
-              "ghc": "8.10.7",
+              "ghc": "9.6.2",
               "cabal": "3.10",
               "os" : "ubuntu-20.04",
               "use-freeze-file": "true"
@@ -281,14 +284,6 @@ jobs:
           ghc-options: -debug
         EOF
 
-    - name: Set GHC Eventlog Flag
-      if: needs.config.outputs.eventlog == 'True'
-      run: |
-        cat >> cabal.project.local <<EOF
-        package chainweb
-          ghc-options: -eventlog
-        EOF
-
     # dist cache
     # the cache-key forces uploading of cache at least once a day, which ensures that
     # upstream dependency changes are captured regularly.
@@ -327,7 +322,7 @@ jobs:
         diff -w <(git show HEAD:cabal.project.freeze) cabal.project.freeze || true
     - name: Sync from cabal cache
       if: env.USE_CABAL_CACHE == 'true'
-      uses: larskuhtz/cabal-cache-action@ecc751af6d31b3ed5a3b4aefdefd0ed6ef4cb414
+      uses: larskuhtz/cabal-cache-action@018b7ae0c480ba3dc4fa0cfa3a0bc1f05b7724b3
       with:
         bucket: "kadena-cabal-cache"
         region: "us-east-1"
@@ -624,10 +619,10 @@ jobs:
         # when adding more than one build, use a different package name or
         # different tags
         include:
-        - ghc: "8.10.7"
+        - ghc: "9.6.2"
           os: "ubuntu-20.04"
           use-freeze-file: "false"
-        - ghc: "8.10.7"
+        - ghc: "9.6.2"
           os: "ubuntu-20.04"
           use-freeze-file: "true"
     env:
@@ -648,7 +643,7 @@ jobs:
 
     - name: Create Dockerfile
       run: |
-        cat > Dockerfile <<EOF
+        cat > Dockerfile <<DEOF
         FROM ubuntu:${OS#ubuntu-}
         LABEL com.chainweb.docker.image.compiler="ghc-${{ matrix.ghc }}"
         LABEL com.chainweb.docker.image.os="${OS}"
@@ -658,10 +653,27 @@ jobs:
         LABEL com.chainweb.docker.image.debug="${{ needs.config.outputs.debug }}"
         LABEL com.chainweb.docker.image.eventlog="${{ needs.config.outputs.eventlog }}"
         LABEL com.chainweb.docker.image.revision="${{ needs.config.outputs.git-sha-short }}"
-        RUN apt-get update && apt-get install -y ca-certificates libgmp10 libssl1.1 libsnappy1v5 zlib1g liblz4-1 libbz2-1.0 libgflags2.2 zstd locales && rm -rf /var/lib/apt/lists/* && locale-gen en_US.UTF-8 && update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
+        RUN <<EOF
+        apt-get update &&
+        apt-get install -y \
+          ca-certificates \
+          libgmp10 \
+          libssl1.1 \
+          libsnappy1v5 \
+          zlib1g \
+          liblz4-1 \
+          libbz2-1.0 \
+          libgflags2.2 \
+          zstd \
+          locales && 
+          rm -rf /var/lib/apt/lists/* &&
+          locale-gen en_US.UTF-8 &&
+          update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
+        EOF
         ENV LANG=en_US.UTF-8
         WORKDIR /chainweb
         COPY chainweb/chainweb-node .
+        COPY chainweb/chainweb-tests .
         COPY chainweb/LICENSE .
         COPY chainweb/README.md .
         COPY chainweb/CHANGELOG.md .
@@ -671,7 +683,7 @@ jobs:
         COPY chainweb/cabal.project.freeze .
         STOPSIGNAL SIGTERM
         ENTRYPOINT [ "/chainweb/chainweb-node" ]
-        EOF
+        DEOF
 
     - name: Set frozen tag
       id: frozen-tag
@@ -684,7 +696,7 @@ jobs:
 
     - name: Docker meta
       id: meta
-      uses: docker/metadata-action@v3
+      uses: docker/metadata-action@v4
       with:
         images: ghcr.io/kadena-io/chainweb-node
         flavor: |
@@ -715,7 +727,7 @@ jobs:
 
     - name: Build and push
       id: docker_build
-      uses: docker/build-push-action@v2
+      uses: docker/build-push-action@v4
       with:
         push: true
         context: .

diff --git a/.github/workflows/macos.yaml b/.github/workflows/macos.yaml
@@ -12,7 +12,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        ghc: ["8.10.7", "9.0"]
+        ghc: ["9.6.1"]
         cabal: ["3.10"]
         os: ["macOS-latest"]
         cabalcache: ["true"]
@@ -82,7 +82,7 @@ jobs:
         cabal freeze
     - name: Sync from cabal cache
       if: matrix.cabalcache == 'true'
-      uses: larskuhtz/cabal-cache-action@ecc751af6d31b3ed5a3b4aefdefd0ed6ef4cb414
+      uses: larskuhtz/cabal-cache-action@018b7ae0c480ba3dc4fa0cfa3a0bc1f05b7724b3
       with:
         bucket: "kadena-cabal-cache"
         region: "us-east-1"

diff --git a/Dockerfile b/Dockerfile
@@ -47,7 +47,7 @@
 # images.
 
 ARG UBUNTU_VERSION=20.04
-ARG GHC_VERSION=8.10.7
+ARG GHC_VERSION=9.6.2
 
 # ############################################################################ #
 # Chainweb Application Runtime Image

diff --git a/Setup.hs b/Setup.hs
@@ -1,5 +1,5 @@
 -- ------------------------------------------------------ --
--- Copyright © 2019 Kadena LLC <[email protected]>
+-- Copyright © 2019-2023 Kadena LLC <[email protected]>
 -- Copyright © 2019 Colin Woodbury <[email protected]>
 -- Copyright © 2015-2018 Lars Kuhtz <[email protected]>
 -- Copyright © 2014 AlephCloud Systems, Inc.
@@ -113,14 +113,8 @@ import Distribution.Simple.Setup
 import Distribution.Text
 import Distribution.Types.LocalBuildInfo
 import Distribution.Types.UnqualComponentName
-
-#if MIN_VERSION_Cabal(3,6,0)
 import Distribution.Utils.Path
-#endif
-
-#if MIN_VERSION_Cabal(3,2,0)
 import Distribution.Utils.ShortText
-#endif
 
 import System.Process
 
@@ -170,13 +164,8 @@ mkPkgInfoModules hooks = hooks
 prettyLicense :: I.InstalledPackageInfo -> String
 prettyLicense = either prettyShow prettyShow . I.license
 
-#if MIN_VERSION_Cabal(3,2,0)
 ft :: ShortText -> String
 ft = fromShortText
-#else
-ft :: String -> String
-ft = id
-#endif
 
 -- -------------------------------------------------------------------------- --
 -- Cabal 2.0
@@ -242,11 +231,7 @@ trim = f . f
 -- -------------------------------------------------------------------------- --
 -- VCS
 
-#if defined (MIN_VERSION_Cabal) && MIN_VERSION_Cabal(3,4,0)
 getVCS :: IO (Maybe KnownRepoType)
-#else
-getVCS :: IO (Maybe RepoType)
-#endif
 getVCS = getCurrentDirectory >>= getVcsOfDir
   where
     getVcsOfDir d = do
@@ -445,12 +430,7 @@ licenseFilesText pkgDesc =
     B.intercalate "\n------------------------------------------------------------\n" <$> mapM fileTextStr
         (licenseFiles pkgDesc)
   where
-#if MIN_VERSION_Cabal(3,6,0)
     fileTextStr = fileText . getSymbolicPath
-#else
-    fileTextStr = fileText
-#endif
-
     fileText file = doesFileExist file >>= \x -> if x
         then B.readFile file
         else return ""

diff --git a/bench/Chainweb/Pact/Backend/Bench.hs b/bench/Chainweb/Pact/Backend/Bench.hs
@@ -5,6 +5,7 @@
 {-# LANGUAGE RecordWildCards #-}
 {-# LANGUAGE ScopedTypeVariables #-}
 {-# LANGUAGE TypeFamilies #-}
+{-# LANGUAGE TypeOperators #-}
 
 module Chainweb.Pact.Backend.Bench
   ( bench )

diff --git a/bench/Chainweb/Pact/Backend/ForkingBench.hs b/bench/Chainweb/Pact/Backend/ForkingBench.hs
@@ -37,6 +37,7 @@ import Data.List.NonEmpty (NonEmpty)
 import qualified Data.List.NonEmpty as NEL
 import Data.Map.Strict (Map)
 import qualified Data.Map.Strict as M
+import Data.String
 import Data.Text (Text)
 import qualified Data.Text as T
 import Data.Text.Encoding
@@ -396,11 +397,8 @@ createCoinAccount v meta name = do
     sender00Keyset <- NEL.fromList <$> getKeyset "sender00"
     nameKeyset <- NEL.fromList <$> getKeyset name
     let attach = attachCaps "sender00" name 1000.0
-    let theData = object [T.pack name .= fmap (formatB16PubKey . fst) (attach nameKeyset)]
-    res <- mkExec (T.pack theCode) theData meta
-      (NEL.toList $ attach sender00Keyset)
-      (Just $ Pact.NetworkId $ toText (_versionName v))
-      Nothing
+    let theData = object [fromString name .= fmap (formatB16PubKey . fst) (attach nameKeyset)]
+    res <- mkExec (T.pack theCode) theData meta (NEL.toList $ attach sender00Keyset) (Just $ Pact.NetworkId $ toText (_versionName v)) Nothing
     pure (nameKeyset, res)
   where
     theCode = printf "(coin.transfer-create \"sender00\" \"%s\" (read-keyset \"%s\") 1000.0)" name name

diff --git a/cabal.project b/cabal.project
@@ -32,16 +32,14 @@ package chainweb
 
 package pact
     ghc-options: -Wwarn
+    optimization: True
     -- avoid conflict with cryptonite during linking
     flags: +cryptonite-ed25519 -build-tool
 
 package rocksdb-haskell-kadena
     ghc-options: -Wwarn -optc-w -optcxx-w
 
-package aeson
-    flags: +cffi
-
-package cryptonite
+package crypton
     flags: +support_pclmuldq
 
 package vault
@@ -61,8 +59,14 @@ package yet-another-logger
 source-repository-package
     type: git
     location: https://github.com/kadena-io/pact.git
-    tag: 2c295ca3c50e0f521c2fb636085107f802f27d92
-    --sha256: 1jjjdmyj4w84djc6zwwdbx6435rxb4p9s5ym5szfvmcfx7kdzkic
+    tag: 7f79433de962fe4b55dfa3f7e1baf9fc3ed703a9
+    --sha256: sha256-OHrldtQTIUtLEzYPh7SYpMFToYO1z5WTrHTKPVwKyMw=
+
+source-repository-package
+    type: git
+    location: https://github.com/kadena-io/pact-json.git
+    tag: 1d260bfaa48312b54851057885de4c43c420e35f
+    --sha256: sha256-EIIsBSy+1nJBs1/QtbnNrJFserA19R43ZUV+rX4l+Ds=
 
 source-repository-package
     type: git
@@ -80,7 +84,7 @@ source-repository-package
     type: git
     location: https://github.com/kadena-io/rosetta.git
     tag: 6c8dd2eea1f6d0dba925646dbcb6e07feeccbfd5
-    --sha256: sha256-EE8A/5wJYqIfx1uz/v3QjWCFgaeVLf/fi1r03A3w8qY=
+    --sha256: 19pjy06xrx2siggzybcmly0qaq4ds3yzxcsvqwgs4qh9kkzh0kqh
 
 source-repository-package
     type: git
@@ -100,14 +104,28 @@ source-repository-package
   tag: f596ea2cc23908fe364569499948c58061fbaa74
   --sha256: 16p28bgcadnd9dimny1syias3bap0v6x8dz15bqrxg7ws63ay1rz
 
--- Required for GHC-9:
+-- Required for backward compatibility:
 
 -- ixset-typed FIX (no PR yet)
 source-repository-package
     type: git
     location: https://github.com/larskuhtz/ixset-typed
     tag: d8019c3404d6f3b3c0b0416e9899cfdf614ef425
-    --sha256: sha256-AwS/uqzUQrTUgxEHNjpX65JKFcaOuZN+03mI16JXuiY=
+    --sha256: 09msayidg23rsdz97fcfqqalm4pbawx3c1qihgab8hnlmjxby103
+
+-- Inherited from pact:
+source-repository-package
+    type: git
+    location: https://github.com/hackage-package-forks/trifecta
+    tag: f991ffb74a1a1ab86f14e751d7c4f4ba549785b3
+    --sha256: 0flnwyjm40zcssq5xfn9h5p24f07npgapr6cx1ni43vx0sbjkqlv
+
+-- Patch merged into master (upcoming version 10.0). We are currently using 9.2
+source-repository-package
+    type: git
+    location: https://github.com/larskuhtz/sbv
+    tag: b66e3a04c20f753213fe7e5115a95b3fe34109f9
+    --sha256: 0dca5pl56nz8ijnqavnpxw5f47qmpalszd5w0ag8bq3fd0l3839m
 
 -- -------------------------------------------------------------------------- --
 -- Relaxed Bounds
@@ -116,7 +134,23 @@ source-repository-package
 
 allow-newer: token-bucket:*
 allow-newer: ixset-typed:*
-allow-newer: hashable:*
+
+-- TODO: I think this fixed?
+allow-newer: rosetta:*
+
+-- Servant is notoriously forcing outdated upper bounds onto its users.
+-- It is usually safe to just ignore those.
+--
+allow-newer: servant-server:*
+allow-newer: servant-client-core:*
+allow-newer: servant-client:*
+allow-newer: servant:*
+
+-- these are more liberal than necessary, but since everything works fine
+-- with this there's no reason to constrain it more than necessary.
+allow-newer: *:base
+allow-newer: *:tempalte-haskell
+allow-newer: *:ghc-prim
 
 -- wai-middleware-validation dependencies
 
@@ -125,6 +159,7 @@ allow-newer: validation:*
 allow-newer: these:*
 allow-newer: regex-base:*
 allow-newer: regex-tdfa:*
+allow-newer: base-compat-batteries:*
 
 -- -------------------------------------------------------------------------- --
 -- Upper Bounds
@@ -133,19 +168,14 @@ allow-newer: regex-tdfa:*
 -- these upper bounds are required in order to not break payload validation
 constraints: base16-bytestring <1
 constraints: prettyprinter <1.6.1
-constraints: hashable <1.3.1
 constraints: base64-bytestring <1.1
 
--- other pact induced bounds (not relevant for on-chain semantics)
-constraints: megaparsec <9.3
-
--- contiguous depends on primitive-unlifted, newer versions of which
--- require GHC >= 9.4
-constraints: contiguous == 0.6.2.0
-constraints: run-st == 0.1.1.0
+-- we have to add these because the old versions that we use would prevent
+-- us to use recent versions of other packages
+allow-newer: base64-bytestring:*
+allow-newer: base16-bytestring:*
 
--- -------------------------------------------------------------------------- --
--- direct-sqlite 2.3.27
+-- other pact induced bounds (not relevant for on-chain semantics)
+-- constraints: megaparsec <9.3
+allow-newer: prettyprinter:*
 
--- TODO remove once the bounds are upgraded in pact.
-allow-newer: pact:direct-sqlite