Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why not use sqlite? #6

Open
Profpatsch opened this issue Jan 22, 2022 · 1 comment
Open

why not use sqlite? #6

Profpatsch opened this issue Jan 22, 2022 · 1 comment

Comments

@Profpatsch
Copy link

Looking at

def create_index(self) -> None:
print('Creating index once')
# create index
col_index_to_skip = {1, 2, 3, 4, 5, 6} # everything before "pos" and after the last "genus" column
for row_idx, row in enumerate(self.data):
for col_idx, word in enumerate(row):
if col_idx in col_index_to_skip:
continue
if not word:
continue
word_low = word.lower()
if row_idx not in self.index[word_low]:
# liste, weil Reihenfolge der Indexes erhalten bleiben muss
self.index[word_low].append(row_idx)
# save index file
output = ''
for k, v in self.index.items():
indexes = '\t'.join(str(x) for x in v)
output += f'{k}\t{indexes}\n'
with open(INDEX_FILE_PATH, 'w', encoding='utf-8') as f:
f.write(output)
def load_index(self) -> None:
with open(INDEX_FILE_PATH, encoding='utf8') as f:
lines = [l.strip() for l in f.readlines()]
for line in lines:
split = line.split('\t')
word_low = split[0]
self.index[word_low] = [int(x) for x in split[1:] if x]

this logic screams for just writing import sqlite3 (comes with the python interpreter!), creating a table in memory (or on disk), maybe even with a column index, and writing some simple sql queries.

@gambolputty
Copy link
Owner

Hello, if you submit a PR, I'll be happy to take a look at it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants