Use Narwhals instead of Pandas

[akmalsoliev]

Is your feature request related to a problem? Please describe.
Currently, there is a limitation in the adoption of a new DataFrame library due to the dominance of a single DataFrame library (pandas). Narwhals is a DataFrame agnostic library that is gaining popularity for its simplicity and ability to use multiple DataFrame libraries through a single API. However, users who prefer different libraries face a significant hurdle as they are forced to use Pandas and/or Spark with GE.

Describe the solution you'd like
A potential solution would be to integrate Narwhals into Great Expectations (GE), allowing users to leverage GE's powerful data validation capabilities with their preferred DataFrame library. This integration would provide a more flexible and inclusive environment for data practitioners, enabling them to use GE seamlessly with libraries such as Pandas, Spark, Polars and etc.. through the Narwhals API.

Describe alternatives you've considered
There are currently no alternatives, users would be forced to convert into Pandas to use GE, which is already not compatible with numpy>2.0

Additional context
Narwhals page:
https://github.com/narwhals-dev/narwhals

[adeola-ak]

Hi there, thank you for sharing this with us. I'll pass this feedback to the team—please stay tuned for updates.

[akmalsoliev]

@adeola-ak great news, looking forward to an update, this would open GE to a great number of communities.

[Bidek56]

@akmalsoliev It's not happening, Polars was rejected from the enhancement list.
If you need a different DF, use alternatives to GE, they support other DFs.

[MarcoGorelli]

@Bidek56 are you sure this is true? They seem open to it in Polars support for Great Expectations

In fact, maybe this issue can be considered a duplicate of that discussion?

[Bidek56]

Please see the comment on #5914