Introduces a new LogicalType: FILE

LogicalTypes.md

@@ -635,7 +635,172 @@ The type has two type parameters:
 
 The sort order used for `GEOGRAPHY` is undefined. When writing data, no min/max
 statistics should be saved for this type and if such non-compliant statistics
-are found during reading, they must be ignored. 
+are found during reading, they must be ignored.
+
+### FILE
+
+`FILE` annotates a group that represents a reference to a range of bytes, which may
+be stored inline in the value, elsewhere within the current file, or in an external file. It
+is intended for use cases such as storing file inventories, manifests, and unstructured
+data references (e.g., images or audio files stored in object storage).
+
+The annotated group may contain the following fields, identified by name case sensitively,
+not by field order. Field IDs, if they exist, may also be used for projection. Every field
+is optional both in the schema and in the data: a writer may omit any field from the group
+definition, and any field that is present has a field repetition type of `OPTIONAL`.
+A group need only define the fields it uses (for example, an inline-only group may define
+just `inline`, and an external reference may define just `path`).
+
+| Field          | Type       |
+|----------------|------------|
+| `path`         | STRING     |
+| `offset`       | INT64      |
+| `size`         | INT64      |
+| `content_type` | STRING     |
+| `checksum`     | STRING     |
+| `inline`       | BYTE_ARRAY |
+
+A value resolves to bytes determined by `inline` / `path` / `offset` / `size`;
+`content_type` and `checksum` are metadata describing whatever is resolved.
+
+#### Fields
+
+For the descriptions below, a field is *set* when it is present in the `FILE` group
+and its value is non-null (and, for string fields, non-empty). A field is *not set*
+when it is absent from the group, or is present but null or empty.
+
+##### path
+
+An opaque path string to an external file (e.g., `s3://bucket/file.jpg`). No special
+encoding (e.g., URI encoding) is applied on top of the user-provided data. If `path` is
+not set, the value refers to the current file (a self-reference).
+
+##### offset
+
+A byte offset indicating the start of the byte range within the referenced data.
+If not set, readers must treat the value as 0.
+If set and non-zero, readers must seek to this offset to retrieve the referenced data.
+`offset` must be set for a self-reference (`path` not set); it is optional for an
+external reference (`path` set).
+
+##### size
+
+The byte length of the referenced data. Must be zero or a positive integer if set; a
+value of 0 indicates empty referenced data. `size` must be set whenever `offset` is set.
+It may be omitted only for a whole-file external reference (`path` set, `offset` not set),
+in which case the range runs to the end of the referenced file. Because a self-reference
+always sets `offset`, it always sets `size` as well.
+
+##### content_type
+
+The media type (MIME type) of the resolved bytes (e.g., `image/png`).
+
+##### checksum
+
+A self-describing integrity token for the resolved bytes, of the form
+`<algorithm>:base64(<digest bytes>)`. Readers should ignore unknown 
+algorithms. The recognized algorithms are:
+
+| Algorithm | Notes                                                           |
+|-----------|-----------------------------------------------------------------|
+| `ETAG`    | the object-store eTag — equality-only, not recomputable         |
+| `MD5`     | the usual S3/HTTP eTag and Content-MD5                          |
+| `CRC32`   | Parquet's page-checksum algorithm (gzip/zlib)                   |
+| `CRC32C`  | common in object stores, hardware-accelerated                   |
+| `SHA-256` | e.g. S3 additional checksums                                    |
+
+`checksum` applies to the resolved bytes, except for `ETAG`, which is the
+object-store eTag for the whole file referenced by `path`.
+
+##### inline
+
+The referenced bytes stored inline in the value. If `inline` is set, it supplies the
+bytes and any locator fields (`path`, `offset`, `size`) that are set are provenance
+only.
+
+#### Resolution
+
+A value resolves to bytes based on which of `inline`, `path`, `offset`, and `size` are
+set:
+
+| `inline` | `path` | `offset` | `size` | Resolves to                                           |
+|----------|--------|----------|--------|-------------------------------------------------------|
+| set      | –      | –        | –      | the inline bytes                                      |
+| –        | set    | –        | –      | whole external file at `path`                         |
+| –        | set    | set      | -      | external `path`, `[offset, EOF)`                      |
+| –        | set    | –        | set    | external `path`, `[0, size)`                          |
+| –        | set    | set      | set    | external `path`, `[offset, offset + size)`            |
+| –        | -      | set      | -      | invalid                                               |
+| –        | -      | -        | set    | invalid                                               |
+| –        | –      | set      | set    | this file, `[offset, offset + size)` (self-reference) |
+| –        | –      | –        | –      | nothing — invalid                                     |
+
+`size` must be set whenever `offset` is set, so any offset-based read always carries an
+explicit `size`. A self-reference (`path` not set) must set `offset`, and therefore also
+`size`. `size` may be omitted only for a whole-file external reference, where the range
+runs to the end of the referenced file.
+
+A self-reference points within the same Parquet file using `offset` and `size` (both
+required); the bytes are written between column chunks and are not otherwise referenced by
+the footer. A self-reference is when `path` is not set, never an absolute path back to
+the current file, so a file containing self-references is renamed or relocated as a
+single unit.
+
+The bytes referenced by a self-reference are compressed with the same `CompressionCodec`
+as the one specified for the `inline` column.
+
+#### Validation
+
+* A value must resolve to some referenced data. It resolves only if `inline`, `path`, or
+  `offset` is set; if none of them are set, the value does not resolve and is invalid, even
+  if `size` is set. Use column nullability to represent a null value.
+* A self-reference (`path` not set) must set `offset`. A value with neither `path` nor
+  `offset` set (and not `inline`) does not resolve and is invalid.
+* `size` must be set whenever `offset` is set. A value that sets `offset` without `size`
+  is invalid. Because a self-reference must set `offset`, it must also set `size`.
+* If `inline` is set, it supplies the bytes; producers may instead treat `inline` and the
+  locator fields as mutually exclusive.
+* Field names within a `FILE`-annotated group must not be renamed.
+* Additional metadata about the file (e.g., modification timestamp) must
+  be stored adjacent to this group by engines or table formats, not inside it.
+
+Statistics may be collected for the individual fields of a `FILE`-annotated group
+according to the sort order of each field's logical type.
+
+This is an example of a `FILE`-annotated group that defines all fields:
+
+```
+optional group my_file (FILE) {
+  optional binary path (STRING);
+  optional int64 offset;
+  optional int64 size;
+  optional binary content_type (STRING);
+  optional binary checksum (STRING);
+  optional binary inline;
+}
+```
+
+Because every field is optional, a group need only define the fields it uses. A group
+whose values are always stored inline may define just `inline`:
+
+```
+optional group inline_file (FILE) {
+  optional binary inline;
+  optional binary content_type (STRING);
+}
+```
+
+A group whose values are always whole external files may define just `path` and optionally
+`content_type` and `checksum` for validation:
+
+```
+optional group external_file (FILE) {
+  optional binary path (STRING);
+  optional binary content_type (STRING);
+  optional binary checksum (STRING);
+}
+```
+
 
 ## Nested Types
 

src/main/thrift/parquet.thrift

@@ -468,6 +468,18 @@ struct GeographyType {
   2: optional EdgeInterpolationAlgorithm algorithm;
 }
 
+/**
+ * File logical type annotation
+ *
+ * Annotates a group that represents a reference to a file, or to a range of
+ * bytes that may be stored inline, elsewhere in this file, or in an external
+ * file.
+ *
+ * See LogicalTypes.md for details.
+ */
+struct FileType {
+}
+
 /**
  * LogicalType annotations to replace ConvertedType.
  *
@@ -501,6 +513,7 @@ union LogicalType {
   16: VariantType VARIANT     // no compatible ConvertedType
   17: GeometryType GEOMETRY   // no compatible ConvertedType
   18: GeographyType GEOGRAPHY // no compatible ConvertedType
+  19: FileType FILE           // no compatible ConvertedType
 }
 
 /**