Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Luqum is using the global lexer by default that may conflict with other libraries using ply #108

Open
Morikko opened this issue Jan 3, 2025 · 1 comment

Comments

@Morikko
Copy link

Morikko commented Jan 3, 2025

Problem

import luqum.parser

lucene_query = "field: value"

luqum.parser.parser.parse(lucene_query)
# No error

import plyara

luqum.parser.parser.parse(lucene_query)
# No error

plyara.Plyara()

luqum.parser.parser.parse(lucene_query)
# Error: luqum.exceptions.ParseSyntaxError: Syntax error in input : unexpected  'field' at position 0!

As opposed to #72, the example is failing in a single thread mode.

Also using luqum.thread.parse(lucene_query) may still fail depending of the ordering:

  1. Before plyara.Plyara(): No error as the new global lexer was not changed yet
  2. After plyara.Plyara(): Same error as lex.lexer.clone() is used instead of luqum.parser.lexer

Cause

luqum is defining a global lexer in https://github.com/jurismarches/luqum/blob/master/luqum/parser.py#L248 that is used "magically" by PLY.

The PLY documentation has a section Multiple Parsers and Lexers for multiple "lexer" usage.

Workaround

luqum.parser.parser.parse(raw_lucene_query, lexer=luqum.parser.lexer)`

Idea

  1. luqum.thread.parse should use luqum.parser.lexer instead of lex.lexer.clone(). Also the lexer parameter in luqum.thread.parse is not even usable as a workaround.

  2. Provide a parser interface that automatically use the luqum.parser.lexer. Although, there is a workaround, I still believe the user should not have to know PLY internal in order to use luqum in a global environment where other libraries may use PLY. It is the solution implemented by plyara.Plyara, so plyara is still parsing correctly even if luqum.parser would be imported first.

@alexgarel
Copy link
Member

@Morikko it's a good idea ! Do you want to champion to propose a PR ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants