Replies: 2 comments 2 replies
-
|
Given parquet's support for nested data, it's certainly possible. Are you referring to temporal attributes per geometry or per coordinate? Either is possible, though the latter might be harder while the data is stored in WKB |
Beta Was this translation helpful? Give feedback.
-
|
Per geometry would be the usual use case. I think it's quite common to have a fixed set of geometries with associated temporal data. Another example might be a collection of geographical regions with attributes of daily average temperature, rainfall, windspeed etc. You don't want to store repeated copies of the geometry for each timeslice - that's what you would need to do with a shapefile and it quickly gets out of hand. With the geopackage/spatiallite option the data is normalised into two separate tables and a relation or view created so that the geoprocessing tools know how the data fits together. These are both definitely not "big data" solutions though. I wondered if or how geoparquet planned to deal with this. It's very helpful to be able to ship the combined data as one file, it's also easier on less technical users of the data who wouldn't necessarily know how to join two datasets back together. |
Beta Was this translation helpful? Give feedback.
-
I believe I talked to Anita Graser (of movingpandas) at Foss4g about this last year for trajectory/movement data, so that was why it was on my mind.
It would be pretty straightforward to have a nested list column that stores any sequence of attributes that map to one geometry. You might just want some metadata that describes the time range of each value (if regularly gridded) or another column with the same nesting that describes the time value of each attribute value. Either way you'd be able to map a list of attributes to a single geometry without storing repeated geometry copies or denormalizing into tables. |
Beta Was this translation helpful? Give feedback.
-
|
Yes there are a few options. You could also have a column which is an array
of timestamp+value pairs.
Whilst actively working with this type of data I'm happy joining,
collecting, exploding as suits really.
It's transferring that data to someone else in a format that they/their GIS
understands that's of interest to me here.
I guess what I'm really interested in is if there's a standard for vector
geospatial temporal data that might emerge from geoparquet.
Particularly from the viewpoint of ArcGis and QGis users
…On Mon, 20 Feb 2023 at 20:28, Kyle Barron ***@***.***> wrote:
Per geometry would be the usual use case
I believe I talked to Anita Graser (of movingpandas) at Foss4g about this
last year for trajectory/movement data, so that was why it was on my mind.
I wondered if or how geoparquet planned to deal with this. It's very
helpful to be able to ship the combined data as one file, it's also easier
on less technical users of the data who wouldn't necessarily know how to
join two datasets back together.
It would be pretty straightforward to have a nested list column that
stores any sequence of attributes that map to one geometry. You might just
want some metadata that describes the time range of each value (if
regularly gridded) or another column with the same nesting that describes
the time value of each attribute value. Either way you'd be able to map a
list of attributes to a single geometry without storing repeated geometry
copies or denormalizing into tables.
—
Reply to this email directly, view it on GitHub
<#166 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAUOA5YXB5PUHXJLI5WSSXTWYPHX5ANCNFSM6AAAAAAVCA334M>
.
You are receiving this because you authored the thread.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
Does the format have any plans to support compact representation of large datasets containing a vector spatial reference set with child data set(s) representing attributes of those geospatial features which change over time e.g. a fixed road network with hourly traffic volume counts?
Maybe this compression would just come for free using the run length encoding in parquet? Or would there be a notion of "related" data as per spatialite/geopackage?
There doesn't appear to be a current spatial file format which supports this well enough to work across all potential user applications.
Shapefiles require flat repeated geo-features and thus end up as huge files, spatialite seems to work well with QGIS but not ArcGis and geopackage vice versa!
Beta Was this translation helpful? Give feedback.
All reactions