Skip to content

Document Parquet Features by Version #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alamb merged 20 commits into from
Jun 23, 2026
+261 −0
Merged

Document Parquet Features by Version #186

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
21fb211
Document Parquet Versions
alamb Jun 4, 2026
430ffce
Update content/en/docs/File Format/versions.md
alamb Jun 5, 2026
7f37c8e
Use forward compatibile terminology
alamb Jun 5, 2026
3af9dfb
document LogicalType union, and Nano timestamp
alamb Jun 5, 2026
390bad3
Apply suggestion from @etseidl
alamb Jun 5, 2026
c438950
Note PARE footer
alamb Jun 5, 2026
9c73309
Merge branch 'alamb/update_v2' of github.com:alamb/parquet-site into …
alamb Jun 5, 2026
45b2b94
Consistently list physical types with logical types
alamb Jun 5, 2026
9e31493
Merge remote-tracking branch 'origin/production' into alamb/update_v2
alamb Jun 9, 2026
d8c63bd
Clarfiy Semantic Versioning
alamb Jun 9, 2026
df8c1ca
tweak
alamb Jun 9, 2026
8915933
Avoid ascribing semantic meaning to the version field
alamb Jun 9, 2026
0b3a17f
Remove other mention of V1 and V2 to reduce confusion
alamb Jun 10, 2026
8cb0942
Follow wording from @stseidl to avoid implying semantic meaning to v1/v2
alamb Jun 10, 2026
1126b53
remove V1/V2 columns to avoid confusion
alamb Jun 10, 2026
86e6937
Merge remote-tracking branch 'origin/production' into alamb/update_v2
alamb Jun 16, 2026
829d6cf
revert package json
alamb Jun 16, 2026
ae621fe
Update content/en/docs/File Format/versions.md
alamb Jun 16, 2026
9faf941
fix link
alamb Jun 16, 2026
9218b90
Update status of IEEE 754 total order and NaN counts
alamb Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 265 additions & 0 deletions content/en/docs/File Format/versions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,265 @@
---
title: "Parquet format versions"
linkTitle: "Format versions"
weight: 9
---

This page describes how features are added to the [Parquet format
specification](https://github.com/apache/parquet-format) and how they affect
reader and writer compatibility. See the
[Implementation status](../implementationstatus/) page for which implementations
(arrow, parquet-java, arrow-rs, etc.) support each feature.

*Note*: If you find out-of-date information, please open an issue or pull request.

## Feature compatibility

The Parquet format spec [classifies changes] by their effect on reader and
writer compatibility. Changes differ in their *forward* compatibility — whether
an older reader can read files that use a newer feature.

**Forward compatible** features remain **readable by older readers**, with a
possibly degraded experience: some metadata may be missing or performance may
suffer, but the reader does not fail. Examples:

* **Bloom filters**: a reader that ignores them skips the pruning metadata but
still reads the data correctly.
* **Logical type annotations** such as `VARIANT`: an older reader reads the
underlying physical column (e.g. `BYTE_ARRAY`) as raw bytes without applying
the logical type.

**Forward incompatible** features make the data **unreadable** to older software.
Examples:

Backwards incompatible features make the data **unreadable** to older software
that does not support them. Examples:

Comment thread
alamb marked this conversation as resolved.
Outdated
* **New encodings** (e.g. the `DELTA_*` encodings, `BYTE_STREAM_SPLIT`,
`RLE_DICTIONARY`): a reader that does not implement them cannot decode the
column values.
* **Data Page V2 headers**: a reader that only understands `DataPageHeader`
cannot parse `DataPageHeaderV2` pages.

[classifies changes]: https://github.com/apache/parquet-format/blob/master/CONTRIBUTING.md#compatibility-and-feature-enablement

## `FileMetadata` version field

Each Parquet file has a `version` field in the [`thrift FileMetadata`] that

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've said this elsewhere, but I don't think we can rely on a version in the metadata, as it cannot convey changes to the metadata itself.

I think we should just say this field conveys no meaningful information any longer.

@alamb alamb Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to make changes to the metadata, I think new magic bytes in teh footer would be an alternate.

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's one option, but I think we'll always want 'PAR', so that limits how many changes we can make with the remaining byte ('1' and 'E' are taken, so I guess that leaves 254 more, fewer if we restrict ourselves to ASCII 🤣). I proposed in the M/L augmenting the header, which would be a forwards compatible change that leaves us leeway to use anything we want to convey versioning info.

@alamb alamb Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PAR2 for maximum confusion!

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You owe me a new keyboard! I just sprayed coffee all over it 🤣

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See also some other ideas of encoding the parquet-format version in the metadata itself

declares which features the file may use, and thus what a reader **must** support

@emkornfield emkornfield Jun 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my issue is the linear nature of versioning here it imposes on readers. In a perfect world every reader would implement everything they needed to up as it is released. But this means a reader needs to move in lock-step with the major version of the header. For example, it lets say we release the following features:

  1. Backward incompatible feature that isn't strictly better for everyone (V3)
  2. Awesome new encoding that a lot more people care about (V4)

By this spec, any writer would need to write V4. This gives readers two choices:

  1. Cheat and try to read the data anyways (this makes version less useful in general, and I think one of the reasons some writer always wrote "1". Readers were capable of reading new encodings (and pretty cleanly detecting when they couldn't so people ignored the guidance).
  2. Implement both V3 and V4 before they get the benefit of V4 (this might have much longer delays given a lot of parquet implementations are volunteer driven).

Is there a way to reconcile this?

@alamb alamb Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really nice description of the core tradeoffs with using Versions (and I think one of the major points we need to resolve to arrive at agreement about how to move versioning forward)

I am not sure there are simple ways to resolve this tradeoff and I believe there are a variety of different opinions

Thus, What I plan to do is to update this PR to REMOVE the a statement on semantic versioning, and keep it focused on what the current state of the spec /features is. We can then continue to have the fun (FUN!) discussion about how to version / signal new features in the corresponding mailing list thread.

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified in d8c63bd that Parquet does NOT follow semantic versioning

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, here is a proposal for how to encode the parquet-format version directly in the meatdata

@emkornfield emkornfield Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also apache/parquet-format#588 to not need version info directly in metadata (although we could add it).

to read it.

**Note**: Many writers set the version field to `1` even for files that use
format 2.0 features, which has caused [confusion and interoperability
issues][closing-out-2.0].

@jorisvandenbossche jorisvandenbossche Jun 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first paragraph giving meaning to the "version" metadata field seems a bit confusing/misleading, together with the note, and moreover with the fact that the thrift itself specifically says this should be hardcoded to "1":

https://github.com/apache/parquet-format/blob/74001e41f5c5a1856b29be115f9c992cab16a4bf/src/main/thrift/parquet.thrift#L1365-L1374

...
struct FileMetaData {
  /** Version of this file
    *
    * As of December 2025, there is no agreed upon consensus of what constitutes
    * version 2 of the file. For maximum compatibility with readers, writers should
    * always populate "1" for version. For maximum compatibility with writers,
    * readers should accept "1" and "2" interchangeably.  All other versions are
    * reserved for potential future use-cases.
    */
  1: required i32 version
...

Or when keeping the text here, the note in the thrift file should be updated to match better?

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right -- I will back this description off to match what is in the thrift

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 8915933


## `parquet-format` release versions

The Thrift definition is released independently of implementations such as
Comment thread
alamb marked this conversation as resolved.
parquet-java or arrow-rs, following the Apache release process and
[semantic versioning]:

1. The major version corresponds to the [`thrift FileMetadata`] `version` field.
2. Minor releases (e.g. `2.10.0` to `2.11.0`) may add compatible
Comment thread
alamb marked this conversation as resolved.
Outdated
features, but never incompatible ones. The minor version is not recorded in the
file itself.

## Adding new features

New features are added by discussion and voting on the [parquet dev mailing list]
(full process [here]). Once approved, a feature is added to the spec and ships in
Comment thread
alamb marked this conversation as resolved.
Outdated
the next parquet-format release.

[parquet dev mailing list]: https://lists.apache.org/list.html?dev@parquet.apache.org
[semantic versioning]: https://semver.org/
[`thrift FileMetadata`]: https://github.com/apache/parquet-format/blob/c42c2cb4ecfccb38153375e24b702a82fd763cc0/src/main/thrift/parquet.thrift#L1365-L1373
[here]: https://github.com/apache/parquet-format/blob/master/CONTRIBUTING.md#additionschanges-to-the-format
[closing-out-2.0]: https://lists.apache.org/thread/0bdyyb7qobrxx94x8v7t5z7g2ksnpyr2

## Forward incompatible features by version

Forward incompatible features and the format version each became available in:

* **V1**: the original Parquet format (1.0).
* **V2**: format version 2.0.

| Feature | V1 | V2 | Released in | Source | Notes |

@emkornfield emkornfield Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we somehow integrate this as YAML data + rendering like we do for feature compatibility?

@alamb alamb Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy to convert this to be data driven rendering rather than an explicit table (rather I would get Claude to do it).

But here it may make less sense as I expect this page to be the only consumer/producer of this data, and the features don't seem to need the same type of cross referencing / forced deduplication of the status page

@emkornfield emkornfield Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it allows use to mark have richer hyper links on the other page. For the compatibility chart, we already had to populate some of this data:

https://github.com/apache/parquet-site/blob/production/data/implementations/features/encodings.yaml#L33

@emkornfield emkornfield Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also means one more page necessary to update when we add something new?

@alamb alamb Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will update to be data driven

@emkornfield emkornfield Jun 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb I'm fine with this as a follow-up or we can always reconcile separately.

@alamb alamb Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks -- I did some research, and I think it would be relatively straightforward to update this page to be data driven.

Upon reflection I would like to do it as a follow on PR to minimize the diff in this PR (I think unifying the data would result in changes to more files and would be harder to track)

| ------------------------------------------ | ---- | ---- | ----------------------------- | --- | ------------------------- |
| [BOOLEAN] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [INT32] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [INT64] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [INT96 (deprecated)] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [FLOAT] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [DOUBLE] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [BYTE_ARRAY] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [FIXED_LEN_BYTE_ARRAY] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [Data Page V1] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [Data Page V2] | | ✅ | [2.0.0] | [1.0.0..2.0.0] | |
| [PLAIN] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [PLAIN_DICTIONARY] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [RLE] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [BIT_PACKED (deprecated)] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [RLE_DICTIONARY] | | ✅ | [2.0.0] | [1.0.0..2.0.0] | |
| [DELTA_BINARY_PACKED] | | ✅ | [2.0.0] | [1.0.0..2.0.0] | |
| [DELTA_LENGTH_BYTE_ARRAY] | | ✅ | [2.0.0] | [1.0.0..2.0.0] | |
| [DELTA_BYTE_ARRAY] | | ✅ | [2.0.0] | [1.0.0..2.0.0] | |
| [BYTE_STREAM_SPLIT] | | ✅ | [2.8.0] | [2.7.0..2.8.0] | [Approved 2019-12-03] |
| [BYTE_STREAM_SPLIT<br/>(Additional Types)] | | ✅ | [2.11.0] | [2.10.0..2.11.0] | [Approved 2024-03-18] |
| [UNCOMPRESSED] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [SNAPPY] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [GZIP] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [LZO] | ✅ | ✅ | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [BROTLI] | | ✅ | [2.4.0] | [2.3.1..2.4.0] | |
| [LZ4 (deprecated)] | | ✅ | [2.4.0] | [2.3.1..2.4.0] | |
| [LZ4_RAW] | | ✅ | [2.9.0] | [2.8.0..2.9.0] | |
| [ZSTD] | | ✅ | [2.4.0] | [2.3.1..2.4.0] | |
| [Modular encryption] | | ✅ | [2.7.0] | [2.6.0..2.7.0] | [Approved 2019-01-16] |


> **Note:** Compression codecs and modular encryption are not gated by the
Comment thread
alamb marked this conversation as resolved.
Outdated
> `version` field — a reader needs support for the specific codec or for
> encryption regardless of version.

## Forward compatible additions

@wgtmac wgtmac Jun 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to include the new IEEE754TotalOrder column order just released in 2.13.0? I think it is a compatible change if readers just regard the new column order as an unknown order and ignore its min/max stats.

@alamb alamb Jun 23, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9218b90


Older readers can read files that use these features but may not understand the
new information.

| Feature | Released in | Source | Notes |
| ------------------------------------------- | ----------------------------- | --- | ------------------------- |

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also from my own notes:

  • Addition of the LogicalType union in favor of ConvertedType enum was in 2.4.0.
  • Formal deprecation of ConvertedType in 2.9.0
  • Addition of NANOS to TimeUnit 2.6.0

@alamb alamb Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also from my own notes:

  • Addition of the LogicalType union in favor of ConvertedType enum was in 2.4.0.
  • Formal deprecation of ConvertedType in 2.9.0
  • Addition of NANOS to TimeUnit 2.6.0

Good call -- added rows for these two:

  • Addition of the LogicalType union in favor of ConvertedType enum was in 2.4.0.
  • Addition of NANOS to TimeUnit 2.6.0

I added this as a note instead of a new row, as it doesn't change what is actually written into files, instead I think it signals a future incompatible change (no longer write converted type)

  • Formal deprecation of ConvertedType in 2.9.0

| [xxHash-based bloom filters] | [2.7.0] | [2.6.0..2.7.0] | [Approved 2019-09-09] |
| [Bloom filter length] | [2.10.0] | [2.9.0..2.10.0] | |
| [Page index] | [2.4.0] | [2.3.1..2.4.0] | |
| [Page CRC32 checksum] | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [Size statistics] | [2.10.0] | [2.9.0..2.10.0] | [Approved 2023-11-14] |
| [Geospatial statistics] | [2.11.0] | [2.10.0..2.11.0] | [Approved 2025-02-09] |
| [Binary protocol extensions] | [2.11.0] | [2.10.0..2.11.0] | [Approved 2024-09-06] |
| [IEEE 754 total order and NaN counts] | not yet released | [#514] | [Approved 2026-05-26] |
| [STRING] | [1.0.0] | [1.0.0][tree-1.0.0] | |
Comment thread
alamb marked this conversation as resolved.
Outdated
| [ENUM] | [2.0.0] | [1.0.0..2.0.0] | |
| [UUID] | [2.6.0] | [2.5.0..2.6.0] | |
Comment thread
alamb marked this conversation as resolved.
Outdated
| [Signed and unsigned integer logical types] | [2.2.0] | [2.1.0..2.2.0] | |
| [DECIMAL (INT32)] | [2.1.0] | [2.0.0..2.1.0] | |
| [DECIMAL (INT64)] | [2.1.0] | [2.0.0..2.1.0] | |
| [DECIMAL (BYTE_ARRAY)] | [2.1.0] | [2.0.0..2.1.0] | |
| [DECIMAL (FIXED_LEN_BYTE_ARRAY)] | [2.1.0] | [2.0.0..2.1.0] | |
| [FLOAT16] | [2.10.0] | [2.9.0..2.10.0] | [Approved 2023-10-13] |
| [DATE] | [2.2.0] | [2.1.0..2.2.0] | |
| [TIME (INT32)] | [2.2.0] | [2.1.0..2.2.0] | |
| [TIME (INT64)] | [2.4.0] | [2.3.1..2.4.0] | |
| [TIMESTAMP (INT64)] | [2.2.0] | [2.1.0..2.2.0] | |
| [INTERVAL] | [2.2.0] | [2.1.0..2.2.0] | |
| [JSON] | [2.2.0] | [2.1.0..2.2.0] | |
| [BSON] | [2.2.0] | [2.1.0..2.2.0] | |
| [VARIANT] | [2.12.0] | [2.11.0..2.12.0] | [Approved 2025-08-24] |
| [Variant shredding] | [2.12.0] | [2.11.0..2.12.0] | [Approved 2025-08-24] |
| [GEOMETRY] | [2.11.0] | [2.10.0..2.11.0] | [Approved 2025-02-09] |
| [GEOGRAPHY] | [2.11.0] | [2.10.0..2.11.0] | [Approved 2025-02-09] |
| [LIST] | [1.0.0] | [1.0.0][tree-1.0.0] | |

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My notes say LIST and MAP were not formally codified with the three-level structure until 2.3.1 (apache/parquet-format@0e2e0a4)

@alamb alamb Jun 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can tell, LIST and MAP are actually defined in the 1.0.0 spec (using the old, deprecated "ConvertedType" annotations)

It seems like apache/parquet-format@0e2e0a4 added the types to the LogicalType.md type structure

🤔

@etseidl etseidl Jun 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the original LIST was underspecified, and implementers diverged in how they were represented in the schema. To enable nullable lists with nullable elements, the current 3-level structure in the schema was introduced. We still suffer with having to parse older, now non-compliant lists (see for example apache/arrow-rs#8496).

I'm just pointing this out for historical purposes 🤓

| [MAP] | [1.0.0] | [1.0.0][tree-1.0.0] | |
| [UNKNOWN (always null)] | [2.4.0] | [2.3.1..2.4.0] | |

[PLAIN]: https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0
[PLAIN_DICTIONARY]: https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
[RLE]: https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3
[RLE_DICTIONARY]: https://github.com/apache/parquet-format/blob/master/Encodings.md#dictionary-encoding-plain_dictionary--2-and-rle_dictionary--8
[DELTA_BINARY_PACKED]: https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5
[DELTA_LENGTH_BYTE_ARRAY]: https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-length-byte-array-delta_length_byte_array--6
[DELTA_BYTE_ARRAY]: https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-strings-delta_byte_array--7
[BYTE_STREAM_SPLIT]: https://github.com/apache/parquet-format/blob/master/Encodings.md#byte-stream-split-byte_stream_split--9
[Modular encryption]: https://github.com/apache/parquet-format/blob/master/Encryption.md
[xxHash-based bloom filters]: https://github.com/apache/parquet-format/blob/master/BloomFilter.md
[Page index]: https://github.com/apache/parquet-format/blob/master/PageIndex.md
[FLOAT16]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#float16
[VARIANT]: https://github.com/apache/parquet-format/blob/master/VariantEncoding.md
[GEOMETRY]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#geometry
[GEOGRAPHY]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#geography

[1.0.0]: https://github.com/apache/parquet-format/releases/tag/parquet-format-1.0.0
[2.0.0]: https://github.com/apache/parquet-format/releases/tag/parquet-format-2.0.0
[2.8.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.8.0
[2.11.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.11.0
[2.4.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.4.0
[2.9.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.9.0
[2.7.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.7.0
[2.10.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.10.0
[2.12.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.12.0

[Approved 2019-12-03]: https://lists.apache.org/thread/xs5qt2odm299pxgqb22mty2csc1so5yr
[Approved 2024-03-18]: https://lists.apache.org/thread/nlsj0ftxy7y4ov1678rgy5zc7dmogg6q
[Approved 2019-01-16]: https://lists.apache.org/thread/l8zcwnbrnhjh3w2k1lyb0v6ct5lnzr0h
[Approved 2019-09-09]: https://lists.apache.org/thread/ktdx1xp0d2gjfgkcvd29zxvt3cgg88bo
[Approved 2023-11-14]: https://lists.apache.org/thread/wgobz41mfldbhqpg9q4mdwypghg2cxg2
[Approved 2023-10-13]: https://lists.apache.org/thread/gyvqcx9ssxkjlrwogqwy7n4z6ofdm871
[Approved 2025-08-24]: https://lists.apache.org/thread/obn1yzhgm5zlznwrdpg7f66mswwooxw7
[Approved 2025-02-09]: https://lists.apache.org/thread/s6s714c98cn9gg22mnk5nsn7xymym8xo

[1.0.0..2.0.0]: https://github.com/apache/parquet-format/compare/parquet-format-1.0.0...parquet-format-2.0.0
[2.7.0..2.8.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.7.0...apache-parquet-format-2.8.0
[2.10.0..2.11.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.10.0...apache-parquet-format-2.11.0
[2.3.1..2.4.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.3.1...apache-parquet-format-2.4.0
[2.8.0..2.9.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.8.0...apache-parquet-format-2.9.0
[2.6.0..2.7.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.6.0...apache-parquet-format-2.7.0
[2.9.0..2.10.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.9.0...apache-parquet-format-2.10.0
[2.11.0..2.12.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.11.0...apache-parquet-format-2.12.0

[2.1.0]: https://github.com/apache/parquet-format/releases/tag/parquet-format-2.1.0
[2.2.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.2.0
[2.6.0]: https://github.com/apache/parquet-format/releases/tag/apache-parquet-format-2.6.0
[2.0.0..2.1.0]: https://github.com/apache/parquet-format/compare/parquet-format-2.0.0...parquet-format-2.1.0
[2.1.0..2.2.0]: https://github.com/apache/parquet-format/compare/parquet-format-2.1.0...apache-parquet-format-2.2.0
[2.5.0..2.6.0]: https://github.com/apache/parquet-format/compare/apache-parquet-format-2.5.0...apache-parquet-format-2.6.0

[tree-1.0.0]: https://github.com/apache/parquet-format/tree/parquet-format-1.0.0

[Variant shredding]: https://github.com/apache/parquet-format/blob/master/VariantShredding.md
[Geospatial statistics]: https://github.com/apache/parquet-format/blob/master/Geospatial.md#statistics
[Binary protocol extensions]: https://github.com/apache/parquet-format/blob/master/BinaryProtocolExtensions.md
[#514]: https://github.com/apache/parquet-format/pull/514
[Approved 2024-09-06]: https://lists.apache.org/thread/x3472kldrq5kjnld9ztj1jozz25f40hg
[Approved 2026-05-26]: https://lists.apache.org/thread/h0k0hqo0sojqphnbnrkp8b0gmwdzq9on

[Bloom filter length]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L933
[Page CRC32 checksum]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L829
[Size statistics]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L202
[IEEE 754 Column Order]: https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L1061
[STRING]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#string
[ENUM]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#enum
[UUID]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#uuid
[8, 16, 32, 64 bit signed and unsigned INT]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#signed-integers
[DECIMAL (INT32)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
[DECIMAL (INT64)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
[DECIMAL (BYTE_ARRAY)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
[DECIMAL (FIXED_LEN_BYTE_ARRAY)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
[DATE]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#date
[TIME (INT32)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#time
[TIME (INT64)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#time
[TIMESTAMP (INT64)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#timestamp
[INTERVAL]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#interval
[JSON]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#json
[BSON]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#bson
[LIST]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists
[MAP]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#maps
[UNKNOWN (always null)]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#unknown-always-null
[Signed and unsigned integer logical types]: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#signed-integers
[IEEE 754 total order and NaN counts]: https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L1061

[BOOLEAN]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L33
[INT32]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L34
[INT64]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L35
[INT96 (deprecated)]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L36
[FLOAT]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L37
[DOUBLE]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L38
[BYTE_ARRAY]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L39
[FIXED_LEN_BYTE_ARRAY]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L40
[Data Page V1]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L671
[Data Page V2]: https://github.com/apache/parquet-format/blob/96656a543a2165d57cc1c9abefaad7f9aeb563a5/src/main/thrift/parquet.thrift#L724
[BIT_PACKED (deprecated)]: https://github.com/apache/parquet-format/blob/master/Encodings.md#bit-packed-deprecated-bit_packed--4
[BYTE_STREAM_SPLIT<br/>(Additional Types)]: https://github.com/apache/parquet-format/blob/master/Encodings.md#byte-stream-split-byte_stream_split--9
Comment thread
alamb marked this conversation as resolved.
[UNCOMPRESSED]: https://github.com/apache/parquet-format/blob/master/Compression.md#uncompressed
[SNAPPY]: https://github.com/apache/parquet-format/blob/master/Compression.md#snappy
[GZIP]: https://github.com/apache/parquet-format/blob/master/Compression.md#gzip
[LZO]: https://github.com/apache/parquet-format/blob/master/Compression.md#lzo
[BROTLI]: https://github.com/apache/parquet-format/blob/master/Compression.md#brotli
[LZ4 (deprecated)]: https://github.com/apache/parquet-format/blob/master/Compression.md#lz4
[LZ4_RAW]: https://github.com/apache/parquet-format/blob/master/Compression.md#lz4_raw
[ZSTD]: https://github.com/apache/parquet-format/blob/master/Compression.md#zstd
6 changes: 3 additions & 3 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
"homepage": "https://github.com/apache/parquet-site#readme",
"devDependencies": {
"@mermaid-js/mermaid-cli": "^11.4.2",
"autoprefixer": "^10.4.18",
"autoprefixer": "^10.5.0",
Comment thread
alamb marked this conversation as resolved.
Outdated
"bootstrap": "^5.3.6",
"postcss": "^8.4.35",
"postcss-cli": "^11.0.0"
"postcss": "^8.5.15",
"postcss-cli": "^11.0.1"
}
}
Loading