Special Character Normalization

[MatteoGisondi]

Description
Endpoints containing special characters are parsed such that they are treated as words when split.

OpenAPI Spec File
The problem can be seen in this (admittedly poorly written) API spec: https://geofeux.sopfeu.qc.ca/sopfeu-api/swagger/v1/swagger.json

Additional context

Currently generated client:

client/api
├── __init__.py
├── contours_administratifs_administrative_perimeters
│   ├── __init__.py
│   ├── get_public_codes_geographiques.py
│   ├── get_public_contour_mrc.py
│   └── get_public_contour_municipalite.py
├── couches_de_donn_é_es_r_é_serv_é_à_lusage_de_la_compagnie_lg2
│   ├── __init__.py
│   ├── get_lg2_frontiere_quebec.py
│   ├── get_lg2_limite_nordique.py
│   ├── get_lg2_mrc.py
│   ├── get_lg2_municipalites.py
│   ├── get_lg2_parc_nationaux_canada.py
│   ├── get_lg2_parc_nationaux_quebec.py
│   ├── get_lg2_parc_regionaux.py
│   ├── get_lg2_pourvoiries.py
│   ├── get_lg2_regions_administratives.py
│   ├── get_lg2_reserves_fauniques.py
│   ├── get_lg2_zones_danger_incendie.py
│   ├── get_lg2_zones_intensive.py
│   └── get_lg2_zones_nordique.py
├── danger_dincendie_fire_danger
│   ├── __init__.py
│   └── get_public_danger_incendie.py
├── feux_fires
│   ├── __init__.py
│   ├── get_public_feu_contours.py
│   ├── get_public_feux.py
│   ├── get_public_feux_englobes.py
│   ├── get_public_feux_etat_situation.py
│   └── get_public_feux_sources_ignition.py
├── mesures_pr_é_ventives_restrictions
│   ├── __init__.py
│   └── get_public_mesures_preventives.py
├── signaler_un_feu_r_é_serv_é_à_lusage_de_la_compagnie_lg2
│   ├── __init__.py
│   └── post_lg2_signaler_feu.py
└── utilisateurs_r_é_serv_é_à_lusage_de_la_compagnie_lg2
    ├── __init__.py
    ├── delete_lg2_utilisateur.py
    ├── delete_lg2_utilisateur_abonnement_local.py
    ├── get_lg2_utilisateur.py
    ├── post_lg2_utilisateur.py
    └── post_lg2_utilisateur_abonnement_local.py

Desired:

client/api
├── __init__.py
├── contours_administratifs_administrative_perimeters
│   ├── __init__.py
│   ├── get_public_codes_geographiques.py
│   ├── get_public_contour_mrc.py
│   └── get_public_contour_municipalite.py
├── couches_de_donnees_reserve_a_lusage_de_la_compagnie_lg2
│   ├── __init__.py
│   ├── get_lg2_frontiere_quebec.py
│   ├── get_lg2_limite_nordique.py
│   ├── get_lg2_mrc.py
│   ├── get_lg2_municipalites.py
│   ├── get_lg2_parc_nationaux_canada.py
│   ├── get_lg2_parc_nationaux_quebec.py
│   ├── get_lg2_parc_regionaux.py
│   ├── get_lg2_pourvoiries.py
│   ├── get_lg2_regions_administratives.py
│   ├── get_lg2_reserves_fauniques.py
│   ├── get_lg2_zones_danger_incendie.py
│   ├── get_lg2_zones_intensive.py
│   └── get_lg2_zones_nordique.py
├── danger_dincendie_fire_danger
│   ├── __init__.py
│   └── get_public_danger_incendie.py
├── feux_fires
│   ├── __init__.py
│   ├── get_public_feu_contours.py
│   ├── get_public_feux.py
│   ├── get_public_feux_englobes.py
│   ├── get_public_feux_etat_situation.py
│   └── get_public_feux_sources_ignition.py
├── mesures_preventives_restrictions
│   ├── __init__.py
│   └── get_public_mesures_preventives.py
├── signaler_un_feu_reserve_a_lusage_de_la_compagnie_lg2
│   ├── __init__.py
│   └── post_lg2_signaler_feu.py
└── utilisateurs_reserve_a_lusage_de_la_compagnie_lg2
    ├── __init__.py
    ├── delete_lg2_utilisateur.py
    ├── delete_lg2_utilisateur_abonnement_local.py
    ├── get_lg2_utilisateur.py
    ├── post_lg2_utilisateur.py
    └── post_lg2_utilisateur_abonnement_local.py

Notably for example, utilisateurs_r_é_serv_é_à_lusage_de_la_compagnie_lg2 should be utilisateurs_reserve_a_lusage_de_la_compagnie_lg2, as it's quite illegible otherwise.

[josteinl]

I have a similar API I have to work with: https://test.ngu.no/api/nadag/innmelding/openapi/v1

It is full of class properties that contain non-ascii unicode characters.

Example (removed lots of stuff) of the OpenAPI file:

---
openapi: 3.1.0
components:
  schemas:
    GeotekniskUnders:
      type: object
      properties:
        område:
          $ref: "#/components/schemas/Polygon"
          type: object
        undersøkelsePeriodeFra:
          type: string
          format: date-time
          examples:
          - 2022-03-10T12:15:50-04:00
        sammensattområde:
          $ref: "#/components/schemas/MultiPolygon"
          type: object
        undersøkelsePeriodeTil:
          type: string
          format: date-time
          examples:
          - 2022-03-10T12:15:50-04:00
        undersøkelseÅrAntatt:
          type: integer
          format: int32

So in this example, I get a class definition like this:

class GeotekniskUnders:
    område: Union[Unset, "Polygon"] = UNSET
    unders_ø_kelse_periode_fra: Union[Unset, datetime.datetime] = UNSET
    sammensattområde: Union[Unset, "MultiPolygon"] = UNSET
    unders_ø_kelse_periode_til: Union[Unset, datetime.datetime] = UNSET
    unders_ø_kelse_å_r_antatt: Union[Unset, int] = UNSET

I am wondering why some characters (ø) or camel case handling (undersøkelseÅrAntatt -> unders_ø_kelse_å_r_antatt) result in these extra underscores _?

Just to be clear. I wish I had a class like this:

class GeotekniskUnders:
    område: Union[Unset, "Polygon"] = UNSET
    undersøkelse_periode_fra: Union[Unset, datetime.datetime] = UNSET
    sammensattområde: Union[Unset, "MultiPolygon"] = UNSET
    undersøkelse_periode_til: Union[Unset, datetime.datetime] = UNSET
    undersøkelse_år_antatt: Union[Unset, int] = UNSET

Sorry if I hijack your discussion @MatteoGisondi, but this seems related to me.

[MatteoGisondi]

This seems exactly related and it's a good additional example. I invite you to test my patch: #1285 to see if this fits your use-case.

[josteinl]

I tested your patch.

I get a new result without the extra underscores (_):

class GeotekniskUnders:
    omrade: Union[Unset, "Polygon"] = UNSET
    undersokelse_periode_fra: Union[Unset, datetime.datetime] = UNSET
    sammensattomrade: Union[Unset, "MultiPolygon"] = UNSET
    undersokelse_periode_til: Union[Unset, datetime.datetime] = UNSET
    undersokelse_ar_antatt: Union[Unset, int] = UNSET

But I would like to keep the Unicode characters and not normalise them to their base character.

Could your normalisation be a configuration option?