Skip to content

Commit dd0d578

Browse files
author
wl5719
committed
Initial commit
0 parents  commit dd0d578

20 files changed

+1240
-0
lines changed

‎.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
.idea
2+
__pycache__
3+
notebooks
4+
outputs

‎README.md

+241
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
<div align="center">
2+
3+
# NoRA - Notion Research Assistant
4+
5+
[![python](https://img.shields.io/badge/-Python-blue?logo=python&logoColor=white)](https://github.com/pre-commit/pre-commit)
6+
[![license](https://img.shields.io/badge/License-MIT-green.svg?labelColor=gray)](https://github.com/ashleve/lightning-hydra-template#license)
7+
8+
A Notion template to help you keep track of the papers you read, their authors,
9+
your notes and more 🚀⚡🔥
10+
11+
</div>
12+
13+
<br>
14+
15+
## 📌  Introduction
16+
17+
This project was built as a [Notion](https://www.notion.so)-based alternative to
18+
reference management softwares such as Zotero and Mendeley.
19+
20+
It is composed of the **NoRA Notion template** for you to build on top of, as
21+
well as **NoRA-Tools** to programmatically:
22+
- 🔥 upload an arxiv paper to your NoRA library from its ID, URL or title
23+
- 🔥 move all your Zotero library to your NoRA library
24+
25+
### 🧪  NoRA template
26+
27+
The NoRA template provides you with a structure of interconnected databases to
28+
keep track of your research papers and notes.
29+
More specifically, the template contains the following databases:
30+
- `🏗️ Projects`
31+
- `📜 Papers`
32+
- `👤 People`
33+
- `🏢 Affiliations`
34+
- `🤹 Conferences & journals`
35+
- `🧲 Key topics`
36+
37+
The inner workings of the template are quite straightforward, the best way to
38+
get familiar with it is probably to play with it 😉 !
39+
40+
### 🛠  NoRA-Tools
41+
42+
The NoRA-Tools provide functionalities to programmatically upload data to your
43+
NoRA template. For now, the two main functionalities are uploading your whole
44+
Zotero library to NoRA and uploading an arxiv paper (and associated metadata)
45+
to NoRA.
46+
47+
You can freely modify or extend the NoRA template. However, keep in mind that if
48+
you want to use NoRA-Tools after modifying some sensitive page fields, you will
49+
need to adjust your [Notion configuration](###notion-configuration) accordingly.
50+
51+
<br>
52+
53+
## 🧱  Installation
54+
55+
### Installing the template
56+
57+
Simply duplicate the [NoRA template](https://silent-switch-780.notion.site/Template-research-library-286d3393a7e845c6a689a5c693790987) to your personal Notion account.
58+
59+
### Installing NoRA tools
60+
61+
First, install the NoRA-Tools locally on your machine.
62+
63+
```bash
64+
# clone project
65+
git clone [email protected]:drprojects/nora.git
66+
cd nora
67+
68+
# create a 'nora' conda environment with required dependencies
69+
conda env create -f nora.yml
70+
```
71+
72+
### Notion configuration
73+
74+
Next, you will need to configure your NoRA-Tools to connect to your NoRA databases in Notion.
75+
To this end, follow the official guide to [Create a Notion Integration](https://developers.notion.com/docs/create-a-notion-integration).
76+
Following all steps in this page you will:
77+
- Create an integration and get a **Token**
78+
- For each database in the NoRA template:
79+
- Share the database with your integration
80+
- Save the **database ID**
81+
82+
You can then save the **Token** and the **database IDs** in the `configs/notion/default.yaml` file:
83+
84+
````yaml
85+
# Adapt to your own Notion library.
86+
# See https://developers.notion.com/docs/create-a-notion-integration
87+
token: "your_integration_token"
88+
papers_db_id: "your_papers_database_id"
89+
people_db_id: "your_people_database_id"
90+
affiliations_db_id: "your_affiliations_database_id"
91+
venues_db_id: "your_venues_database_id"
92+
topics_db_id: "your_topics_database_id"
93+
````
94+
95+
<details>
96+
<summary><b>⚠️ Want to modify some field names in NoRA ?️</b></summary>
97+
98+
By default, NoRA-Tools expect the attribute fields (eg column names in Notion) of your papers, people, etc. to have specific values.
99+
If you want to adjust those, you will also need to adjust the `configs/notion/default.yaml` file:
100+
101+
````yaml
102+
# If you happen to modify your field names in Notion, update the
103+
# following database-specific keys
104+
person_keys:
105+
name: 'Name'
106+
affiliations: '🏢 Affiliations'
107+
papers: '📜 Papers'
108+
website: 'Website'
109+
110+
paper_keys:
111+
name: 'Name'
112+
authors: '👤 Authors'
113+
abstract: 'Abstract'
114+
topics: '🧲 Key topics'
115+
url: 'URL'
116+
to_read: 'Reading status'
117+
year: 'Year'
118+
venue: '🤹 Venue'
119+
120+
affiliation_keys:
121+
name: 'Name'
122+
123+
venue_keys:
124+
name: 'Name'
125+
````
126+
127+
</details>
128+
129+
<details>
130+
<summary><b>
131+
⚠️ Want to add your own 🤹 Conferences & journals ?️</b></summary>
132+
133+
By default, when parsing an arxiv paper, NoRA-Tools will try to figure out
134+
which `🤹 Conferences & journals` to place it under. However, one can hardly
135+
account for all possible conference and journal names, nor for all the slight
136+
formatting differences used to describe how a paper was published. Yet, we
137+
attempt to group the most frequent ones using a predefined `VENUES` dictionary
138+
in `nora/utils/venues.py`.
139+
140+
If many papers from your library are from a conference or journal absent from
141+
this dictionary, and you would like them to be grouped under the same
142+
`🤹 Conferences & journals` item, you can simply append your own entries in
143+
`VENUES`, using the following format:
144+
145+
````python
146+
'lowercase_match_to_search_for_in_arxiv_metadata': 'shorthand_under_which_to_group'
147+
````
148+
149+
</details>
150+
151+
### Zotero configuration (optional)
152+
153+
If you intend to move your whole Zotero library to Notion, you will need to configure your NoRA-Tools accordingly.
154+
To this end, you will need to:
155+
- Get your Zotero **library ID** by checking the UserID in your [profile settings](https://www.zotero.org/settings/keys)
156+
- Create a Zotero **API key** in your [profile settings](https://www.zotero.org/settings/keys)
157+
158+
You can then save your Zotero **library ID** and **API key** in the `configs/zotero/default.yaml` file:
159+
160+
````yaml
161+
# Adapt to your own Zotero library
162+
library_id: 'your_library_id'
163+
api_token: 'your_api_key'
164+
````
165+
166+
By default, the `collections` (eg folders) in your Zotero library will be used to populate the `Key Topics` field of your papers in NoRA.
167+
If you want to exclude some of your collections from this behavior, your can list them in:
168+
169+
````yaml
170+
ignored_collections: ['collection name 1', 'collection name 2']
171+
````
172+
173+
### Proxy configuration (optional)
174+
175+
If your machine has a proxy, you will need to configure your NoRA-Tools to use it.
176+
To this end, specify your `$HTTP_PROXY` and `$HTTPS_PROXY` in the `configs/proxy/default.yaml` file:
177+
178+
````yaml
179+
http_proxy: 'your_http_proxy'
180+
https_proxy: 'your_https_proxy'
181+
````
182+
183+
<br>
184+
185+
## ⚡  Using NoRA-Tools
186+
187+
### Uploading an arxiv paper to NoRA
188+
189+
From its arxiv ID:
190+
191+
```bash
192+
python nora/arxiv_to_nora.py arxiv.id="'2204.07548'"
193+
```
194+
195+
From its url:
196+
197+
```bash
198+
python nora/arxiv_to_nora.py arxiv.id="https://arxiv.org/abs/2204.07548"
199+
```
200+
201+
From its title:
202+
203+
```bash
204+
python nora/arxiv_to_nora.py arxiv.title="Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"
205+
```
206+
207+
### Uploading your entire Zotero library to NoRA
208+
209+
```bash
210+
python nora/zotero_to_nora.py
211+
```
212+
213+
<br>
214+
215+
## License
216+
217+
NoRA is licensed under the MIT License.
218+
219+
```
220+
MIT License
221+
222+
Copyright (c) 2021 ashleve
223+
224+
Permission is hereby granted, free of charge, to any person obtaining a copy
225+
of this software and associated documentation files (the "Software"), to deal
226+
in the Software without restriction, including without limitation the rights
227+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
228+
copies of the Software, and to permit persons to whom the Software is
229+
furnished to do so, subject to the following conditions:
230+
231+
The above copyright notice and this permission notice shall be included in all
232+
copies or substantial portions of the Software.
233+
234+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
235+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
236+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
237+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
238+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
239+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
240+
SOFTWARE.
241+
```

‎configs/arxiv/default.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Specify one of those when uploading an arxiv paper to NoRA
2+
id: null
3+
title: null

‎configs/config.yaml

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# @package _global_
2+
3+
# specify here default configuration
4+
# order of defaults determines the order in which configs override each other
5+
defaults:
6+
- _self_
7+
- arxiv: default.yaml
8+
- notion: default.yaml
9+
- proxy: default.yaml
10+
- zotero: default.yaml

‎configs/notion/default.yaml

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Adapt to your own Notion library.
2+
# See https://developers.notion.com/docs/create-a-notion-integration
3+
token: ???
4+
papers_db_id: ???
5+
people_db_id: ???
6+
affiliations_db_id: ???
7+
venues_db_id: ???
8+
topics_db_id: ???
9+
10+
# If you happen to modify your field names in Notion, update the
11+
# following database-specific keys
12+
person_keys:
13+
name: 'Name'
14+
affiliations: '🏢 Affiliations'
15+
papers: '📜 Papers'
16+
website: 'Website'
17+
18+
paper_keys:
19+
name: 'Name'
20+
authors: '👤 Authors'
21+
abstract: 'Abstract'
22+
topics: '🧲 Key topics'
23+
url: 'URL'
24+
to_read: 'Reading status'
25+
year: 'Year'
26+
venue: '🤹 Venue'
27+
28+
affiliation_keys:
29+
name: 'Name'
30+
31+
venue_keys:
32+
name: 'Name'
33+
34+
# Not supposed to be edited
35+
max_text_length: 2000

‎configs/proxy/default.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Adapt to your local proxy if need be
2+
http_proxy: ''
3+
https_proxy: ''

‎configs/zotero/default.yaml

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Adapt to your own Zotero library
2+
library_id: ???
3+
api_token: ???
4+
ignored_collections: []
5+
6+
# Leave untouched
7+
paper_types: ['bookSection', 'conferencePaper', 'journalArticle', 'report']
8+
venue_keys_per_type:
9+
bookSection: ['bookTitle']
10+
conferencePaper: ['proceedingsTitle', 'conferenceName']
11+
journalArticle: ['publicationTitle', 'journalAbbreviation']
12+
report: []

‎nora.yml

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: nora
2+
channels:
3+
- defaults
4+
dependencies:
5+
- bzip2=1.0.8
6+
- ca-certificates=2023.01.10
7+
- certifi=2022.12.7
8+
- libffi=3.4.2
9+
- ncurses=6.4
10+
- openssl=1.1.1t
11+
- pip=23.0.1
12+
- python=3.11.2
13+
- readline=8.2
14+
- setuptools=65.6.3
15+
- sqlite=3.41.1
16+
- tk=8.6.12
17+
- tzdata=2023c
18+
- wheel=0.38.4
19+
- xz=5.2.10
20+
- zlib=1.2.13
21+
- pip:
22+
- antlr4-python3-runtime==4.9.3
23+
- anyio==3.6.2
24+
- arxiv==1.4.3
25+
- attrs==19.3.0
26+
- bibtexparser==1.4.0
27+
- charset-normalizer==3.1.0
28+
- emoji==2.2.0
29+
- feedparser==6.0.10
30+
- h11==0.14.0
31+
- html5lib==1.1
32+
- httpcore==0.16.3
33+
- httpx==0.23.3
34+
- hydra-core==1.3.2
35+
- idna==3.4
36+
- mistletoe-ebp==0.10.0
37+
- notion-client==2.0.0
38+
- notional==0.8.0
39+
- omegaconf==2.3.0
40+
- packaging==23.0
41+
- pydantic==1.10.7
42+
- pyparsing==3.0.9
43+
- pyrootutils==1.0.4
44+
- python-dotenv==1.0.0
45+
- pytz==2023.3
46+
- pyyaml==6.0
47+
- pyzotero==1.5.5
48+
- requests==2.28.2
49+
- rfc3986==1.5.0
50+
- sgmllib3k==1.0.0
51+
- six==1.16.0
52+
- sniffio==1.3.0
53+
- typing-extensions==4.5.0
54+
- urllib3==1.26.15
55+
- webencodings==0.5.1
56+
prefix: /Users/wl5719/opt/anaconda3/envs/nora

‎nora/__init__.py

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
import nora.utils
2+
import nora.parsers

0 commit comments

Comments
 (0)