Merge pull request #626 from metosin/power-merge-schemas

Introduce two-phase Schema compilation

CHANGELOG.md

@@ -12,19 +12,22 @@ We use [Break Versioning][breakver]. The version numbers follow a `<major>.<mino
 
 [breakver]: https://github.com/ptaoussanis/encore/blob/master/BREAK-VERSIONING.md
 
-## UNREALEASED
+## UNRELEASED
+
+**BREAKING**: `compile-request-coercers` returns a map with `:data` and `:coerce` instead of plain `:coerce` function
+**BREAKING**: Parameter and Response schemas are acculated into vector in route data - to be merged properly into compiled result, fixes [#422](https://github.com/metosin/reitit/issues/422) - merging multiple schemas together works with `Malli` and `Schema`, partially with `data-spec` but not with `spec`.
 
 ```clojure
 [metosin/schema-tools "0.13.1"] is available but we use "0.13.0"
 [com.fasterxml.jackson.core/jackson-core "2.15.1"] is available but we use "2.14.2"
 [com.fasterxml.jackson.core/jackson-databind "2.15.1"] is available but we use "2.14.2"
 ```
 
-## 0.7.3-alpha4 (2023-05-17)
+## 0.7.0-alpha4 (2023-05-17)
 
 * OpenAPI 3 parameter descriptions get populated from malli/spec/schema descriptions. [#612](https://github.com/metosin/reitit/issues/612)
 
-## 0.7.3-alpha3 (2023-05-05)
+## 0.7.0-alpha3 (2023-05-05)
 
 * Compile `reitit.Trie` with Java 1.8 target for compatibility
 

doc/advanced/configuring_routers.md

@@ -2,18 +2,21 @@
 
 Routers can be configured via options. The following options are available for the `reitit.core/router`:
 
-| key           | description
-|---------------|-------------
-| `:path`       | Base-path for routes
-| `:routes`     | Initial resolved routes (default `[]`)
-| `:data`       | Initial route data (default `{}`)
-| `:spec`       | clojure.spec definition for a route data, see `reitit.spec` on how to use this
-| `:syntax`     | Path-parameter syntax as keyword or set of keywords (default #{:bracket :colon})
-| `:expand`     | Function of `arg opts => data` to expand route arg to route data (default `reitit.core/expand`)
-| `:coerce`     | Function of `route opts => route` to coerce resolved route, can throw or return `nil`
-| `:meta-merge` | Function which follows the signature of `meta-merge.core/meta-merge`, useful for when you want to have more control over the meta merging
-| `:compile`    | Function of `route opts => result` to compile a route handler
-| `:validate`   | Function of `routes opts => ()` to validate route (data) via side-effects
-| `:conflicts`  | Function of `{route #{route}} => ()` to handle conflicting routes
-| `:exception`  | Function of `Exception => Exception ` to handle creation time exceptions (default `reitit.exception/exception`)
-| `:router`     | Function of `routes opts => router` to override the actual router implementation
+| key             | description
+|-----------------|-------------
+| `:path`         | Base-path for routes
+| `:routes`       | Initial resolved routes (default `[]`)
+| `:data`         | Initial route data (default `{}`)
+| `:spec`         | clojure.spec definition for a route data, see `reitit.spec` on how to use this
+| `:syntax`       | Path-parameter syntax as keyword or set of keywords (default #{:bracket :colon})
+| `:expand`       | Function of `arg opts => data` to expand route arg to route data (default `reitit.core/expand`)
+| `:coerce`       | Function of `route opts => route` to coerce resolved route, can throw or return `nil`
+| `:compile`      | Function of `route opts => result` to compile a route handler
+| `:validate`     | Function of `routes opts => ()` to validate route (data) via side-effects
+| `:conflicts`    | Function of `{route #{route}} => ()` to handle conflicting routes
+| `:exception`    | Function of `Exception => Exception ` to handle creation time exceptions (default `reitit.exception/exception`)
+| `:meta-merge`   | Function of `left right => merged` to merge route-data (default `meta-merge.core/meta-merge`)
+| `:update-paths` | Sequence of Vectors with elements `update-path` and `function`, used to preprocess route data
+| `:router`       | Function of `routes opts => router` to override the actual router implementation
+
+

modules/reitit-core/src/reitit/coercion.cljc

@@ -94,8 +94,9 @@
                                   (conj (:content model) [:default (:body model)])
                                   [[:default model]])
             format->coercer (some->> (for [[format schema] format-schema-pairs
-                                           :when schema]
-                                       [format (-request-coercer coercion (case style :request :body style) (->open schema))])
+                                           :when schema
+                                           :let [type (case style :request :body style)]]
+                                       [format (-request-coercer coercion type (->open schema))])
                                      (filter second)
                                      (seq)
                                      (into {}))]
@@ -117,7 +118,8 @@
 (defn response-coercer [coercion {:keys [content body]} {:keys [extract-response-format serialize-failed-result]
                                                          :or {extract-response-format extract-response-format-default}}]
   (if coercion
-    (let [per-format-coercers (some->> (for [[format schema] content]
+    (let [per-format-coercers (some->> (for [[format schema] content
+                                             :when schema]
                                          [format (-response-coercer coercion schema)])
                                        (filter second)
                                        (seq)
@@ -152,25 +154,23 @@
       response)))
 
 (defn request-coercers [coercion parameters opts]
-  (some->> (for [[k v] parameters
-                 :when v]
+  (some->> (for [[k v] parameters, :when v]
              [k (request-coercer coercion k v opts)])
-           (filter second)
-           (seq)
-           (into {})))
+           (filter second) (seq) (into {})))
 
 (defn response-coercers [coercion responses opts]
   (some->> (for [[status model] responses]
              [status (response-coercer coercion model opts)])
-           (filter second)
-           (seq)
-           (into {})))
+           (filter second) (seq) (into {})))
+
+(defn -compile-parameters [data coercion]
+  (impl/path-update data [[[:parameters any?] #(-compile-model coercion % nil)]]))
 
 ;;
 ;; api-docs
 ;;
 
-(defn -warn-unsupported-coercions [{:keys [parameters responses] :as data}]
+(defn -warn-unsupported-coercions [{:keys [parameters responses] :as _data}]
   (when (:request parameters)
     (println "WARNING [reitit.coercion]: swagger apidocs don't support :request coercion"))
   (when (some :content (vals responses))
@@ -204,17 +204,29 @@
 
 (defn compile-request-coercers
   "A router :compile implementation which reads the `:parameters`
-  and `:coercion` data to create compiled coercers into Match under
-  `:result. A pre-requisite to use [[coerce!]]."
-  [[_ {:keys [parameters coercion]}] opts]
+  and `:coercion` data to both compile the schemas and create compiled coercers
+  into Match under `:result with the following keys:
+
+   | key       | description
+   | ----------|-------------
+   | `:data`   | data with compiled schemas
+   | `:coerce` | function of `Match -> coerced parameters` to coerce parameters
+
+  A pre-requisite to use [[coerce!]].
+
+  NOTE: this is not needed with ring/http, where the coercion compilation is
+  managed in the request coercion middleware/interceptors."
+  [[_ {:keys [parameters coercion] :as data}] opts]
   (if (and parameters coercion)
-    (request-coercers coercion parameters opts)))
+    (let [{:keys [parameters] :as data} (-compile-parameters data coercion)]
+      {:data data
+       :coerce (request-coercers coercion parameters opts)})))
 
 (defn coerce!
-  "Returns a map of coerced input parameters using pre-compiled
-  coercers under `:result` (provided by [[compile-request-coercers]].
-  Throws `ex-info` if parameters can't be coerced
-  If coercion or parameters are not defined, return `nil`"
+  "Returns a map of coerced input parameters using pre-compiled coercers in `Match`
+  under path `[:result :coerce]` (provided by [[compile-request-coercers]].
+  Throws `ex-info` if parameters can't be coerced. If coercion or parameters
+  are not defined, returns `nil`"
   [match]
-  (if-let [coercers (:result match)]
+  (if-let [coercers (-> match :result :coerce)]
     (coerce-request coercers match)))

modules/reitit-core/src/reitit/core.cljc

@@ -307,27 +307,30 @@
    :coerce (fn coerce [route _] route)
    :compile (fn compile [[_ {:keys [handler]}] _] handler)
    :exception exception/exception
+   :update-paths [[[:parameters any?] impl/accumulate]]
    :conflicts (fn throw! [conflicts] (exception/fail! :path-conflicts conflicts))})
 
 (defn router
   "Create a [[Router]] from raw route data and optionally an options map.
   Selects implementation based on route details. The following options
   are available:
 
-  | key          | description
-  | -------------|-------------
-  | `:path`      | Base-path for routes
-  | `:routes`    | Initial resolved routes (default `[]`)
-  | `:data`      | Initial route data (default `{}`)
-  | `:spec`      | clojure.spec definition for a route data, see `reitit.spec` on how to use this
-  | `:syntax`    | Path-parameter syntax as keyword or set of keywords (default #{:bracket :colon})
-  | `:expand`    | Function of `arg opts => data` to expand route arg to route data (default `reitit.core/expand`)
-  | `:coerce`    | Function of `route opts => route` to coerce resolved route, can throw or return `nil`
-  | `:compile`   | Function of `route opts => result` to compile a route handler
-  | `:validate`  | Function of `routes opts => ()` to validate route (data) via side-effects
-  | `:conflicts` | Function of `{route #{route}} => ()` to handle conflicting routes
-  | `:exception` | Function of `Exception => Exception ` to handle creation time exceptions (default `reitit.exception/exception`)
-  | `:router`    | Function of `routes opts => router` to override the actual router implementation"
+  | key             | description
+  | ----------------|-------------
+  | `:path`         | Base-path for routes
+  | `:routes`       | Initial resolved routes (default `[]`)
+  | `:data`         | Initial route data (default `{}`)
+  | `:spec`         | clojure.spec definition for a route data, see `reitit.spec` on how to use this
+  | `:syntax`       | Path-parameter syntax as keyword or set of keywords (default #{:bracket :colon})
+  | `:expand`       | Function of `arg opts => data` to expand route arg to route data (default `reitit.core/expand`)
+  | `:coerce`       | Function of `route opts => route` to coerce resolved route, can throw or return `nil`
+  | `:compile`      | Function of `route opts => result` to compile a route handler
+  | `:validate`     | Function of `routes opts => ()` to validate route (data) via side-effects
+  | `:conflicts`    | Function of `{route #{route}} => ()` to handle conflicting routes
+  | `:exception`    | Function of `Exception => Exception ` to handle creation time exceptions (default `reitit.exception/exception`)
+  | `:meta-merge`   | Function of `left right => merged` to merge route-data (default `meta-merge.core/meta-merge`)
+  | `:update-paths` | Sequence of Vectors with elements `update-path` and `function`, used to preprocess route data
+  | `:router`       | Function of `routes opts => router` to override the actual router implementation"
   ([raw-routes]
    (router raw-routes {}))
   ([raw-routes opts]