unmask.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bde150d-1677-412b-8f56-3d67a369521d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from unmask import umaskall"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49e2c39a-0701-4666-b354-8fc0efe8acc1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 从字符串执行\n",
    "text='''\n",
    "Spatial navigation, which was a process 1performed exclusively by the human brain and perceptual system, has now been surrendered to technology. However, in doing so, we [RB] surrendered our agency.\n",
    "There are structures in the brain dedicated to complex pathfinding tasks. In particular, the hippocampus is deeply 3associated with supporting spatial memory, spatial navigation and mental mapping.\n",
    "Anthropologists have gone so far as to suggest that navigation needs might have been the 4origin for all memories. As we age, our memory declines. And 5while there is no silver bullet for healthy aging, neuroscientists agree that one of the key 6ingredients of successful aging is staying mentally active. Studies show that we can actually 7exercise the hippocampus memory through exploration and spatial navigation. \n",
    "By following a set of digital turn-by-turn directions, GPS navigation apps treat us as 8passive passengers rather than active explorers, 9removing our agency to make decisions. 10In _turn this inhibits our ability to create proper mental maps of the surrounding environment, and 11negatively impacts the hippocampus, which is critical for brain health.\n",
    "As we explore new environments, the brain 12constantly maps out our surroundings. And we use all of our senses to learn about and remember our surroundings, not just vision. In particular, evolution 13rewarded auditory navigation: as hunter-gatherers we listened to the environment and moved towards sounds, 14allowing us to avoid predators, track prey and locate water sources. Our brains ultimately could 15generate detailed maps of our environment through active exploration, a hard-won evolutionary skill that never went away. However, as civilizations expanded and grew, it became more 16difficult to navigate over long distances only through sensory experiences and memory. One cannot sail to another continent 16on senses alone. So, maps were a revolution for humanity.\n",
    "The rise of satellite-enabled GPS was revolutionary for navigation, and with the rise of mobile phones, anyone can have their 18personal navigator. GPS navigation apps enable egocentric navigation with easy-to-follow turn-by-turn directions: “Turn right in 100 meters, then turn left,” all the way to the destination. 19However, multiple experiments have shown that this easy egocentric navigation reduces spatial 20awareness and mental mapping when 21compared to more traditional forms of allocentric navigation like paper maps.\n",
    "'''\n",
    "top_k = 50\n",
    "split_stences = True\n",
    "single_mask = False\n",
    "outputfile = \"output.txt\"\n",
    "with open(outputfile, \"w\", encoding=\"utf-8\") as output:\n",
    "    umaskall(text, top_k=top_k, split_stences=split_stences, single_mask=single_mask, io=output)\n",
    "with open(outputfile, encoding=\"utf-8\") as output:\n",
    "    print(output.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61f8fb83-a444-4d72-a336-1e0c71a5d734",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 从文件执行\n",
    "top_k = 50\n",
    "split_stences = True\n",
    "single_mask = False\n",
    "inputfile = \"article.txt\"\n",
    "outputfile = \"output.txt\"\n",
    "\n",
    "with open(inputfile, encoding=\"utf-8\") as f:\n",
    "    text = f.read()\n",
    "with open(outputfile, \"w\", encoding=\"utf-8\") as output:\n",
    "    umaskall(text, top_k=top_k, split_stences=split_stences, single_mask=single_mask, io=output)\n",
    "with open(outputfile, encoding=\"utf-8\") as output:\n",
    "    print(output.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64552ab7-0542-44b5-9e3b-00acf51fff7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 交互执行\n",
    "top_k = 50\n",
    "split_stences = True\n",
    "single_mask = False\n",
    "print(\"## 输入句子，挖空的词用数字或下划线开头或用中括号指定词性，输入q退出。回车执行。 ##\")\n",
    "while True:\n",
    "    text = input(\"⮞ 请输入句子：\")\n",
    "    if text == \"q\":\n",
    "        break\n",
    "    else:\n",
    "        umaskall(text, top_k=top_k, split_stences=split_stences, single_mask=single_mask)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b040a0a-c622-4c47-abde-0228d83b4df6",
   "metadata": {},
   "source": [
    "* 修改`text`字符串中的内容或者直接编辑`article.txt`\n",
    "* 可以修改`top_k`, `split_stences`, `single_mask`的值\n",
    "* 结果输出在`output.txt`\n",
    "***\n",
    "* 挖空的词可以用数字或下划线`_`开头，如：`It's a _great tool` 或 `It's 1pretty 2useful`\n",
    "* 挖空的词也可以用中括号指定词性，`|`表示逻辑或，如：`it is a [RB] useful __.` 或 `[NN|PRP] is [adj]`\n",
    "* 默认自动补全末尾句号，如不希望补全，在末尾加反斜杠`\\`，如：`I am _`和`I am _\\`的区别\n",
    "* 当输入是完整句子而非短语时效果会更好，整段话或者整篇文章亦可。`split_stences`控制自动拆分成单句或段落多次输入\n",
    "\n",
    "### 词性粗标签\n",
    "\n",
    "Tag|Description|释义|例子\n",
    " - | - | - | - \n",
    "ADJ|adjective|形容词|new, good, high, special, big, local\n",
    "ADP|adposition|介词|on, of, at, with, by, into, under\n",
    "ADV|adverb|副词|really, already, still, early, now\n",
    "CONJ|conjunction|连词|and, or, but, if, while, although\n",
    "DET|determiner, article|限定词|the, a, some, most, every, no, which\n",
    "NOUN|noun|名词|year, home, costs, time, Africa\n",
    "NUM|numeral|数词|twenty-four, fourth, 1991, 14:24\n",
    "PRT|particle|虚词|at, on, out, over per, that, up, with\n",
    "PRON|pronoun|代词|he, their, her, its, my, I, us\n",
    "VERB|verb|动词|is, say, told, given, playing, would\n",
    ".|punctuation marks|标点符号|. , ; !\n",
    "X|other|其它|ersatz, esprit, dunno, gr8, univeristy\n",
    "### 词性细标签\n",
    "\n",
    "Tag|Description|释义|例子\n",
    " - | - | - | - \n",
    "CC|Coordinating conjunction|连词|and, or,but, if, while,although\n",
    "CD|Cardinal number|数词|twenty-four, fourth, 1991,14:24\n",
    "DT|Determiner|限定词|the, a, some, most,every, no\n",
    "EX|Existential there|存在量词|there, there’s\n",
    "FW|Foreign word|外来词|dolce, ersatz, esprit, quo,maitre\n",
    "IN|Preposition or subordinating conjunction|介词连词|on, of,at, with,by,into, under\n",
    "JJ|Adjective|形容词|new,good, high, special, big, local\n",
    "JJR|Adjective, comparative|比较级词语|bleaker braver breezier briefer brighter brisker\n",
    "JJS|Adjective, superlative|最高级词语|calmest cheapest choicest classiest cleanest clearest\n",
    "LS|List item marker|标记|A A. B B. C C. D E F First G H I J K\n",
    "MD|Modal|情态动词|can cannot could couldn’t\n",
    "NN|Noun, singular or mass|名词|year,home, costs, time, education\n",
    "NNS|Noun, plural|名词复数|undergraduates scotches\n",
    "NNP|Proper noun, singular|专有名词|Alison,Africa,April,Washington\n",
    "NNPS|Proper noun, plural|专有名词复数|Americans Americas Amharas Amityvilles\n",
    "PDT|Predeterminer|前限定词|all both half many\n",
    "POS|Possessive ending|所有格标记|’ ‘s\n",
    "PRP|Personal pronoun|人称代词|hers herself him himself hisself\n",
    "PRP\\$|Possessive pronoun|所有格|her his mine my our ours\n",
    "RB|Adverb|副词|occasionally unabatingly maddeningly\n",
    "RBR|Adverb, comparative|副词比较级|further gloomier grander\n",
    "RBS|Adverb, superlative|副词最高级|best biggest bluntest earliest\n",
    "RP|Particle|虚词|aboard about across along apart\n",
    "SYM|Symbol|符号|% & ’ ” ”. ) )\n",
    "TO|to|词to|to\n",
    "UH|Interjection|感叹词|Goodbye Goody Gosh Wow\n",
    "VB|Verb, base form|动词|ask assemble assess\n",
    "VBD|Verb, past tense|动词过去式|dipped pleaded swiped\n",
    "VBG|Verb, gerund or present participle|动词现在分词|telegraphing stirring focusing\n",
    "VBN|Verb, past participle|动词过去分词|multihulled dilapidated aerosolized\n",
    "VBP|Verb, non-3rd person singular present|动词现在式非第三人称时态|predominate wrap resort sue\n",
    "VBZ|Verb, 3rd person singular present|动词现在式第三人称时态|bases reconstructs marks\n",
    "WDT|Wh-determiner|Wh限定词|who,which,when,what,where,how\n",
    "WP|Wh-pronoun|WH代词|that what whatever\n",
    "WP\\$|Possessive wh-pronoun|WH代词所有格|whose\n",
    "WRB|Wh-adverb|WH副词|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d6dc4cb-783b-4935-8408-886241a40271",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}