启动报错：sqlite3.OperationalError: no such table: metadata #205
Labels
dependencies
Pull requests that update a dependency file
Comments
|
I also encountered this problem. It may be caused by the version of Python. The version of SQLAlchemy number under Python 3.7 cannot be too high, or the FLASK data connection will make an error and Scrapydweb cannot be installed. I used version 3.7 and it reported an error. After reinstalling version 3.6, it ran normally. You can try it, good luck~~(^v^)~~ 说人话，换python版本试试 |
|
I am running into the same issue. Here is a Dockerfile I am using to try to get scrapyd-web up and running. FROM python:3.7-slim
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install git -y
COPY requirements.txt .
RUN python3 -m pip install --no-cache-dir -r requirements.txt
RUN python3 -m pip install --upgrade git+https://github.com/my8100/scrapydweb.git
ENTRYPOINT scrapydwebThe |
|
Just tried using python 3.6 and there is an error, although it is different. FROM python:3.6-slim
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install git -y
COPY requirements.txt .
RUN python3 -m pip install --no-cache-dir -r requirements.txt
RUN python3 -m pip install --upgrade git+https://github.com/my8100/scrapydweb.git
ENTRYPOINT scrapydwebLogs: |
|
I haven't encountered this problem, but I found a tip in the report: ATTENTION: Have you open the service of scrapyd normally? Scrapydweb is an extended function of Scrapyd, which provides visual operation services on the basis of Scrapyd. I recommend the following steps to start service of scrapydweb:
It is also the first time for me to contact Scrapydweb. If there are any mistakes, please forgive me. Good luck~ |
|
As it turns out I guess I was using a version of SQLAlchemy that was too recent. I updated my Full pip>=19.1.1
APScheduler>=3.5.3
flask>=1.0.2
flask-compress>=1.4.0
Flask-SQLAlchemy>=2.3.2
logparser==0.8.2
requests>=2.21.0
setuptools>=40.6.3
six>=1.12.0
SQLAlchemy>=1.2.15,<1.4.0
w3lib>=1.17.0
logparserDockerfile: FROM python:3.6-slim
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install git -y
RUN mkdir /app
WORKDIR /app
COPY requirements.txt .
RUN python3 -m pip install -r requirements.txt
RUN python3 -m pip install --upgrade git+https://github.com/my8100/scrapydweb.git
ENTRYPOINT ["scrapydweb"]Thanks to this thread for pointing me in that direction: #178 |
|
thanks, work perfect! |
|
Hi, after running the requirements.txt and scrapydweb.git installation. `[2023-02-17 07:53:15,571] INFO in apscheduler.scheduler: Scheduler started ATTENTION: The config file 'scrapydweb_settings_v10.py' has been copied to current working directory. Error in atexit._run_exitfuncs: The above exception was the direct cause of the following exception: Traceback (most recent call last): |
|
I was using python 3.9 but then switching to 3.6 with upgrading sqlalchemy to 1.4 shows the scrapydweb gui perfectly on the address. But still get error in the logs of scrapydweb
|
|
Does author exist? This project needs update for latest version. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Please have a try for the latest scrapydweb v1.4.1, which supports Python 3.6 to 3.9 |
|
Fixed in scrapydweb v1.4.1 |
|
确认是SQLAlchemy版本问题，按照requirements.txt里的版本要求安装就可以 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
scrapy==1.8.0
scrapyd==1.2.1
scrapyd-client
scrapy-splash==0.7.2
scrapydweb==1.4.0
logparser
logs
[2022-11-09 15:29:13,804] INFO in apscheduler.scheduler: Scheduler started
[2022-11-09 15:29:13,814] INFO in scrapydweb.run: ScrapydWeb version: 1.4.0
[2022-11-09 15:29:13,814] INFO in scrapydweb.run: Use 'scrapydweb -h' to get help
[2022-11-09 15:29:13,814] INFO in scrapydweb.run: Main pid: 18
[2022-11-09 15:29:13,814] DEBUG in scrapydweb.run: Loading default settings from /usr/local/lib/python3.7/site-packages/scrapydweb/default_settings.py
Traceback (most recent call last):
File "/usr/local/bin/scrapydweb", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/scrapydweb/run.py", line 32, in main
app = create_app()
File "/usr/local/lib/python3.7/site-packages/scrapydweb/init.py", line 81, in create_app
handle_db(app)
File "/usr/local/lib/python3.7/site-packages/scrapydweb/init.py", line 123, in handle_db
db.create_all()
File "/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/extension.py", line 868, in create_all
self._call_for_binds(bind_key, "create_all")
File "/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/extension.py", line 839, in _call_for_binds
engine = self.engines[key]
File "/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/extension.py", line 628, in engines
app = current_app._get_current_object() # type: ignore[attr-defined]
File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 513, in _get_current_object
raise RuntimeError(unbound_message) from None
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context(). See the documentation for more information.
[2022-11-09 15:29:13,834] INFO in sqlalchemy.engine.Engine: BEGIN (implicit)
2022-11-09 15:29:13,834 INFO sqlalchemy.engine.Engine BEGIN (implicit)
[2022-11-09 15:29:13,838] INFO in sqlalchemy.engine.Engine: SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
2022-11-09 15:29:13,838 INFO sqlalchemy.engine.Engine SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
2022-11-09 15:29:13,838 INFO sqlalchemy.engine.Engine [generated in 0.00034s] ('1.4.0', 1, 0)
[2022-11-09 15:29:13,838] INFO in sqlalchemy.engine.Engine: [generated in 0.00034s] ('1.4.0', 1, 0)
[2022-11-09 15:29:13,839] INFO in sqlalchemy.engine.Engine: ROLLBACK
2022-11-09 15:29:13,839 INFO sqlalchemy.engine.Engine ROLLBACK
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: metadata
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/scrapydweb/common.py", line 85, in handle_metadata
metadata = Metadata.query.filter_by(version=version).first()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2823, in first
return self.limit(1)._iter().first()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2918, in _iter
execution_options={"_sa_orm_load_options": self.load_options},
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
self, multiparams, params, execution_options
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1582, in _execute_clauseelement
cache_hit=cache_hit,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1944, in execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2125, in handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from=e
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 208, in raise
raise exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: metadata
[SQL: SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?]
[parameters: ('1.4.0', 1, 0)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
The text was updated successfully, but these errors were encountered: