fix: handle reserved words - #51
Conversation
|
@pnadolny13 happy with this as-is, but it would be good to review this point in a future issue:
For downstream analytics use it would be great to create in a case-insensitive way, even if that means heavier-handed name conforming. We dialled back the default conforming in the SDK, so it'd be up to us in |
|
@kgpayne yeah 100%. I'm not sure how it should be handled in the general case but these changes mimic what the PPW variant does and what I think is reasonable in the snowflake case. This is my summary, while its fresh in my head: I have a stream that has a property I've used products in the past that keep the exact casing of the source when writing to snowflake meaning lowercase properties would stay lowercase in the snowflake table within quotes. This was a huge pain to query, even the snowflake console had issues querying those columns using a preview back then (probably resolved by now). |
Closes #48
Sqlalchemy handles some of the quoting for reserved words but we also build sql manually so those dont get handled. Also when doing column diffing like to decided whether a column exists or not we need to use uppercase column names because we created them as uppercase with quotes so case matters.
I found a bug with the COPY logic when the stream has schema updates. Logged it in #53 and draft PR in #52. Once that is fixed we can add a schema change to the test records with no key properties so we assert alters work with reserved words when no key properties are being used.