Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot sort discoveries by "category" nor "snippet" #142

Open
alaabenfatma opened this issue Jul 9, 2021 · 5 comments
Open

Cannot sort discoveries by "category" nor "snippet" #142

alaabenfatma opened this issue Jul 9, 2021 · 5 comments
Labels
bug Something isn't working

Comments

@alaabenfatma
Copy link
Contributor

alaabenfatma commented Jul 9, 2021

It enters an infinite loop.

Tested client: sqlite

Reproduce the bug: While on discoveries view page (http://0.0.0.0:5000/discoveries?url=https://github.com/user/repo), any attempt to sort discoveries via "category" or "snippet" fails on the server side resulting in an infinite loop on the UI (no matter the number of discoveries).

@alaabenfatma alaabenfatma self-assigned this Jul 9, 2021
@alaabenfatma alaabenfatma changed the title Cannot sort discoveries by "category" nor "snippet" [wip] Cannot sort discoveries by "category" nor "snippet" Jul 9, 2021
@alaabenfatma alaabenfatma added the bug Something isn't working label Jul 9, 2021
@marcorosa
Copy link
Member

Can you give more context, please?
Is it an issue that happens with both the clients? How can we reproduce it (e.g., a script to test this bug)? Did you test it with a repo giving only a few discoveries (so that it's very easy to see that it enters an infinite loop)?

@alaabenfatma
Copy link
Contributor Author

Can you give more context, please?
Is it an issue that happens with both the clients? How can we reproduce it (e.g., a script to test this bug)? Did you test it with a repo giving only a few discoveries (so that it's very easy to see that it enters an infinite loop)?

Hi @marcorosa, I updated the description

@marcorosa
Copy link
Member

Ok, but can you be more specific, please? 😄
At first I though it was related to the data structure we are using (could have been not sortable). Yet, I sorted manually some discoveries and it works fine

conn = sqlite3.connect('data.db')
cursor = conn.cursor()
repo_url = '...'
cursor.execute('select * from discoveries where repo_url=?', (repo_url,))
discoveries = cursor.fetchall()
sorted(discoveries, key=lambda d:d[4])  # 4 is the snippet position

So, where is this issue coming from? what methods are you using? are you trying to do this sort with the python library or directly in-browser with javascript? are you trying to sort all the discoveries or a repo or just the paginated ones?

@alaabenfatma
Copy link
Contributor Author

Thank you for your comment @marcorosa
No, it is not related to the data structure nor the database.

So, where is this issue coming from?

The issue is coming from the /get_discoveries route/function. Precisely this line:

key=lambda i: States[i[order_by]].value,

are you trying to do this sort with the python library or directly in-browser with javascript?

The bug occurs when we try to sort the discoveries on the browser (javascript).

The bug can be reproduced manually without writing a script, by sorting the discoveries via the UI using these built-in buttons:
image

are you trying to sort all the discoveries or a repo or just the paginated ones?

I tried both 'all discoveries' and only the paginated ones, but none of them has worked.

Fix

Problem

Sorting discoveries (manually, from the UI, using the built-in sorting buttons), results in this error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2088, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2073, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/root/apps/new_cdigger/credential-digger-main/ui/server.py", line 396, in get_discoveries
    'data': sorted([{'snippet': keys[0],
  File "/root/apps/new_cdigger/credential-digger-main/ui/server.py", line 408, in <lambda>
    key=lambda i: States[i[order_by]].value,
  File "/usr/lib/python3.8/enum.py", line 349, in __getitem__
    return cls._member_map_[name]
KeyError: 'crypto_key'

crypto_key in our case is one of the categories we have at hand. One of the discoveries happens to be a crytpo_key.

Fix

The problem comes from line:

key=lambda i: States[i[order_by]].value,

We replaced States[i[order_by]].value by order_by and it works just fine as for our first experiments, we will test it further more.

@marcorosa
Copy link
Member

Ordering discoveries by category still breaks the UI

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/i355397/git/credential-digger/ui/server.py", line 447, in get_discoveries
    'data': sorted([{'snippet': keys[0],
  File "/Users/i355397/git/credential-digger/ui/server.py", line 460, in <lambda>
    key=lambda i: States[i[order_by]].value,
  File "/usr/local/Cellar/[email protected]/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/enum.py", line 432, in __getitem__
    return cls._member_map_[name]
KeyError: 'password'

@marcorosa marcorosa changed the title [wip] Cannot sort discoveries by "category" nor "snippet" Cannot sort discoveries by "category" nor "snippet" Jun 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants