search for words which have the feature/misc name with any value

Orange-OpenSource · Jul 20, 2023 · df15c72 · df15c72
1 parent 2b0c982
commit df15c72
Show file tree

Hide file tree

Showing 12 changed files with 1,410 additions and 25 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,5 +1,9 @@
 # Changes
 
+## Version 2.22.4
+* `Feat:VerbForm=` or `Misc:SpaceAfter=` (no value given) searches for words which have the feature/misc name with any value (use `not Feat:VerbForm:` to look for words which do not have the given feature)
+* new tests
+
 ## Version 2.22.3
 * bug concerning --rootdir corrected
 

diff --git a/README.md b/README.md
@@ -26,7 +26,7 @@ The editor provides the following functionalities:
 * adding Translit= values to the MISC column (transliterating the FORM column) see section [Transliteration](#transliteration)
 * finding similar or identical sentence in a list of CoNLL-U files, see section [Find Similar Sentences](#find-similar-sentences)
 
-Current version: 2.22.3 (see [change history](CHANGES.md))
+Current version: 2.22.4 (see [change history](CHANGES.md))
 
 ConlluEditor can also be used as front-end to display the results of dependency parsing in the same way as the editor.
 * dependency tree/dependency hedge
@@ -327,7 +327,7 @@ In order to create a multiword token, use the `compose <wordid> <length>`
 command. Click on the multiword token bar (at the bottom of the dependency
 tree/graph to open a dialogue which allows to edit or delete the token (i.e. the `n-m` line).
 
-All operations which change the tokenisation of the sentence will create a `incoherent # text and forms` warning. This is because the è# text = ....` 
+All operations which change the tokenisation of the sentence will create a `incoherent # text and forms` warning. This is because the è# text = ....`
 metadata must be coherent with the concatenation of forms (taken into account `SpacesAfter`/`SpaceAfter` fields in the MISC column.
 Unless earlier versions, the `# text ...` is no longer updated automatically, but must be adapted manually using the `edit metadata` button.
 
@@ -498,7 +498,7 @@ see [Mass Editing](doc/mass_editing.md)
 
 ## Metadata editing
 
-The CoNLL-U format provides some special comment lines to indicate whether the current sentence is the beginning of a new document, new paragraph, 
+The CoNLL-U format provides some special comment lines to indicate whether the current sentence is the beginning of a new document, new paragraph,
 the sentence itself, as well as its sentence id, translations (mostly into English) or transliterations.
 Clicking on `edit metadata` opens the Metadata dialogue.
 For translations, the translations must be prefixed with the language code as shown in the screen shot.

diff --git a/doc/mass_editing.md b/doc/mass_editing.md
@@ -32,9 +32,13 @@ Examples:
   * `IsEmpty` (no value, true if the current node is empty)
   * `IsMWT` (no value, true if the current node is a MWT)
 
-`Form:`, `Lemma:` and `Xpos:` can contain simple regular expression (only the character ')' cannot be used
+`Form:`, `Lemma:` and `Xpos:` can contain simple regular expression (only the character ')' cannot be used.
+
+To check for any Feat or Misc value, leave the value empty:
+  * `Feat:Gender:` true if the current word has the feature `Gender` with any value
+
 In order to check for the absence of a given Featurename in the Feature or Misc column, use the following:
-  * `Feat:Gender:` true if the cyurrent word has no feature `Gender`
+  * `not Feat:Gender:` true if the current word has no feature `Gender`
 
 `EUD` cannot deal (yet) with empty word ids (`n.m`)
 

diff --git a/gui/index.html b/gui/index.html
@@ -3,7 +3,7 @@
     <!--
     This library is under the 3-Clause BSD License
 
-    Copyright (c) 2018-2022, Orange S.A.
+    Copyright (c) 2018-2023, Orange S.A.
 
     Redistribution and use in source and binary forms, with or without modification,
     are permitted provided that the following conditions are met:
@@ -31,7 +31,7 @@
     THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
     author Johannes Heinecke
-    version 2.20.0 as of 13th December 2022
+    version 2.22.4 as of 20th July 2023
     -->
 
     <head>
@@ -225,7 +225,7 @@
                        <button class="editbuttons mybutton" id="replacego">search & replace</button>
                     </div>
                 </td></tr>
-                
+
                 <tr id="grewmatchsearchandreplace"><td>
                     <div>
                         <input class="inputfield" placeholder="pattern {N [upos=NOUN]} without {V -[nsubj]-> N}" title="grewmatch pattern" size="70" type="text" id="grewmatchsearchexpression" value="">
@@ -513,9 +513,12 @@ <h3>Complex search</h3>
                         </ul>
                         In order to check for the absence of a given Featurename in the Feature or Misc column, use the following:
                         <ul>
-                            <li><span class="userinputdoc">Feat:Gender:</span> true if the cyurrent word has no feature <span class="userinputdoc">Gender</span>
+                            <li><span class="userinputdoc">not Feat:Gender:</span> true if the current word has no feature <span class="userinputdoc">Gender</span>.
+                        </ul>
+                        In order to check any Feature (more Misc) value, do not specify the value:
+                        <ul>
+                            <li><span class="userinputdoc">Feat:Gender:</span> true if the current word has feature <span class="userinputdoc">Gender</span> with any value.
                         </ul>
-
                         In addition to key keys listed above, four functions are available to take the context of the token into account:
                         <ul>
                             <li>  <span class="userinputdoc">child()</span> child of current token</li>
@@ -546,7 +549,7 @@ <h3>Complex search</h3>
                             <li> <span class="userinputdoc">@Deprel=prec(@Deprel)</span>: true, if the current word and the preceding word have the same <span class="userinputdoc">deprel</span> value
                             <li> <span class="userinputdoc">@Xpos=head(head(@Feat:Featname))</span> true if the `XPOS` of the current word has the same value as the feature <span class="userinputdoc">Featname</span> of the head of its head.
                             <li> <span class="userinputdoc">@Feat:Gender=head(@Feat:Gender) and not Upos:DET</span> true if the head and the current word have the same value for the feature <span class="userinputdoc">Gender</span> and the current word is not a <span class="userinputdoc">DET</span>
-
+                        </ul>
 
                         <h3>Search and replace</h3>
 

diff --git a/pom.xml b/pom.xml
@@ -32,13 +32,13 @@
     THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
     author Johannes Heinecke
-    version 2.22.3 as of 4th June 2023
+    version 2.22.4 as of 20th July 2023
     -->
 
     <modelVersion>4.0.0</modelVersion>
     <groupId>com.orange.labs</groupId>
     <artifactId>ConlluEditor</artifactId>
-    <version>2.22.3</version>
+    <version>2.22.4</version>
     <packaging>jar</packaging>
 
     <properties>

diff --git a/src/main/java/com/orange/labs/conllparser/CEvalVisitor.java b/src/main/java/com/orange/labs/conllparser/CEvalVisitor.java
@@ -28,7 +28,7 @@
  THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
  @author Johannes Heinecke
- @version 2.18.1 as of 29th October 2022
+ @version 2.22.4 as of 20th July 2023
  */
 package com.orange.labs.conllparser;
 
@@ -305,10 +305,15 @@ public Boolean visitCheckFeat(ConditionsParser.CheckFeatContext ctx) {
         }
         boolean rtc ;
         if (fv.length == 2) {
-            rtc = use.matchesFeatureValue(fv[0], fv[1]);
+            if ("!".equals(fv[1])) {
+                // feature must not be in word
+                rtc = !use.getFeatures().containsKey(fv[0]);
+            } else {
+                rtc = use.matchesFeatureValue(fv[0], fv[1]);
+            }
         } else {
-            // feature must not be in word
-            rtc = !use.getFeatures().containsKey(fv[0]);
+            // mist must be in word with any key
+            rtc = use.getFeatures().containsKey(fv[0]);
         }
         return rtc;
     }
@@ -323,10 +328,15 @@ public Boolean visitCheckMisc(ConditionsParser.CheckMiscContext ctx) {
         }
         boolean rtc;
         if (fv.length == 2) {
-            rtc = use.matchesMiscValue(fv[0], fv[1]);
+            if ("!".equals(fv[1])) {
+                // misc must not be in word
+                rtc = !use.getMisc().containsKey(fv[0]);
+            } else {
+                rtc = use.matchesMiscValue(fv[0], fv[1]);
+            }
         } else {
-            // feature must not be in word
-            rtc = !use.getMisc().containsKey(fv[0]);
+            // mist must be in word with any key
+            rtc = use.getMisc().containsKey(fv[0]);
         }
         return rtc;
     }
@@ -459,7 +469,7 @@ public Boolean visitOder(ConditionsParser.OderContext ctx) {
     @Override
     public Boolean visitValcompare(ConditionsParser.ValcompareContext ctx) {
         CGetVisitor getvisitor = new CGetVisitor(cword, wordlists);
-       
+
         //System.err.println("GET VALUES FOR COMPARISON");
         //String left = getvisitor.visit(ctx.columnname(0));  // get value of left columnname
         //String right = getvisitor.visit(ctx.columnname(1));  // get value of right columnname
@@ -480,12 +490,12 @@ public Boolean visitValcompare(ConditionsParser.ValcompareContext ctx) {
         }
         return false;
     }
-    
-    
+
+
     @Override
     public Boolean visitValcompatible(ConditionsParser.ValcompatibleContext ctx) {
         CGetVisitor getvisitor = new CGetVisitor(cword, wordlists);
-       
+
         //System.err.println("GET VALUES FOR COMPATIBILITY");
         //String left = getvisitor.visit(ctx.columnname(0));  // get value of left columnname
         //String right = getvisitor.visit(ctx.columnname(1));  // get value of right columnname

diff --git a/src/main/java/com/orange/labs/conllparser/ConllWord.java b/src/main/java/com/orange/labs/conllparser/ConllWord.java
@@ -1547,7 +1547,7 @@ public boolean hasFeature(String name) {
         }
         return features.containsKey(name);
     }
-    
+
     // check whether feature with value is present
     public boolean hasFeature(String name, String val) {
         if (features.isEmpty()) {

diff --git a/src/test/java/TestConllFile.java b/src/test/java/TestConllFile.java
@@ -453,6 +453,30 @@ public void test20repl07() throws IOException, ConllException {
         applyRule("Upos:VERB", "feat:\"InlfClass=\"+this(Feat_Number) feat:\"Number=\"", "rule26.conllu");
     }
 
+    @Test
+    public void test20repl08() throws IOException, ConllException {
+        name("repl 08");
+        applyRule("Feat:VerbForm:Inf", "misc:\"VF=INF\"", "rule27.conllu");
+    }
+
+    @Test
+    public void test20repl09() throws IOException, ConllException {
+        name("repl 09");
+        applyRule("Feat:VerbForm:", "misc:\"VF=ANY\"", "rule28.conllu");
+    }
+
+    @Test
+    public void test20repl10() throws IOException, ConllException {
+        name("repl 09");
+        applyRule("not Feat:Number:", "misc:\"NoNumber=True\"", "rule30.conllu");
+    }
+
+    @Test
+    public void test20repl11() throws IOException, ConllException {
+        name("repl 11");
+        applyRule("Misc:SpaceAfter:", "feat:\"SPACEAFTER=ANY\"", "rule29.conllu");
+    }
+
     @Test
     public void test21value01() throws IOException, ConllException {
         name("value 01");