.dockerignore

@@ -1,3 +1,3 @@
 node_modules
 .gitignore
-.git
+.git

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,33 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**.
+A clear and concise description of what the bug is.
+
+**To Reproduce**.
+Steps to reproduce the behavior:
+1. Set essential settings.py
+2. Provide sequence of actions you made in spider
+3. Make log-file with error
+4. See error
+
+**Expected behavior**.
+A clear and concise description of what you expected to happen.
+
+**Screenshots**.
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Python version: [e.g. 3.11.4]
+ - Scrapy version: [e.g. 2.10]
+ - Scrapy-puppeteer version: [e.g. 0.1.0]
+
+**Additional context**.
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for scrapy-puppeteer-service
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**.
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**.
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**.
+Add any other context or screenshots about the feature request here.

.github/pull_request_template.md

@@ -0,0 +1,37 @@
+# Title
+
+Please, provide meaningful title 
+
+# Description
+
+Please include a summary of the changes and the related issue. Please also include relevant motivation and context.
+List any dependencies that are required for this change.
+
+## Type of change
+
+Please delete options that are not relevant.
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] This change requires a documentation update
+
+# How Has This Been Tested?
+
+Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.
+Please also list any relevant details for your test configuration
+
+- [ ] Test A
+- [ ] Test B
+
+**Test Configuration**:
+* Software version:
+* Hardware:
+
+# Checklist:
+
+- [ ] My code follows the style guidelines of this project
+- [ ] I have performed a self-review of my code
+- [ ] I have commented my code, particularly in hard-to-understand areas
+- [ ] I have made corresponding changes to the documentation
+- [ ] My changes generate no new warnings

.gitignore

@@ -1,2 +1,3 @@
 .idea
 node_modules
+.gitignore

CODE_OF_CONDUCT.md

@@ -0,0 +1,47 @@
+# *Code Of Conduct*
+
+---
+Just like any other technical community, Scrapy-puppeteer-service team and community is made up
+of many people from all over the world. We want to join them into one huge community of *Scrapy-puppeteer-service*.
+
+As there are many diverse people with different points of view, we decided to write rules for contributing,
+issue declaring and discussion. We hope everybody would follow them and enjoy spending time within the community!
+
+This code of conduct applies to all spaces managed by Scrapy-puppeteer-service project and to all members
+of its community.
+
+If you believe someone is violating the code of conduct, we ask that you report it by
+emailing us.
+
+* Be friendly and patient.
+* Be welcoming. We strive to be a community that welcomes and supports people of all backgrounds and identities.
+This includes, but is not limited to members of any race, ethnicity, culture, national origin, colour, immigration
+status, social and economic class, educational level, sex, age, size, family status, political belief, religion,
+and mental and physical ability.
+* Be considerate. Your work will be used by other people, and you in turn will depend on the work of others.
+Any decision you take will affect users and colleagues, and you should take those consequences into account
+when making decisions. Remember that we're a world-wide community, so you might not be communicating in
+someone else's primary language.
+* Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior or
+bad manners. We might all experience some frustration now and then, but we cannot allow that frustration to
+turn into a personal attack. It’s important to remember that a community where people feel uncomfortable or
+threatened is not a productive one. Members of the Scrapy-puppeteer-service community should be respectful when dealing
+with other members as well as with people outside the community.
+* Be careful in the words that you choose. We are a community of professionals, and we conduct ourselves
+professionally. Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary
+behavior aren't acceptable. This includes, but is not limited to:
+  1. Violent threats or language directed against another person.
+  2. Discriminatory jokes and language.
+  3. Posting (or threatening to post) other people's personally identifying information ("doxing").
+  4. Personal insults.
+  5. Advocating for, or encouraging, any of the above behavior.
+  6. Repeated harassment of others. In general, if someone asks you to stop, then stop.
+* When we disagree, try to understand why. Disagreements, both social and technical, happen all the time and
+Scrapy-puppeteer-service is no exception. It is important that we resolve disagreements and differing views constructively.
+Remember that we’re different. The strength of Scrapy-puppeteer-service comes from its varied community, people from a
+wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why
+someone holds a viewpoint doesn’t mean that they’re wrong. Don’t forget that it is human to err and blaming
+each other doesn’t get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.
+
+
+Original text from [DJango](https://www.djangoproject.com/conduct/).

CONTRIBUTING.md

@@ -0,0 +1,20 @@
+# Contributing to Scrapy-puppeteer-service
+
+---
+As an open source project, Scrapy-puppeteer-service welcomes contributions of many forms.
+
+Examples of contributions include:
+
+* Code patches
+* Documentation improvements
+* Bug reports and patch reviews
+
+**Warning: any pull request without informative title, meaningful description will be closed!**
+
+`Please formalize your pull request (PR)` you will get.
+
+---
+# Code of Conduct
+
+As a contributor, you can help us keep the Scrapy-puppeteer-service community open and inclusive.
+Please read and follow our [Code Of Conduct](https://github.com/ispras/scrapy-puppeteer-service/blob/master/CODE_OF_CONDUCT.md).

Dockerfile

@@ -1,25 +1,43 @@
-FROM node:10.16.3
+FROM node:18
 
-RUN apt-get update && \
-    apt-get -y install xvfb gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 \
-      libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 \
-      libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 \
-      libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 \
-      libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget && \
-    rm -rf /var/lib/apt/lists/*
+ENV USER=root
+ENV DISPLAY=:1
+ENV RESOLUTION=1080x720
+
+# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
+# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer
+# installs, work.
+RUN apt-get update \
+    && apt-get install -y wget gnupg tightvncserver xfce4 xfce4-goodies xfonts-base dbus-x11 novnc \
+    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor -o /usr/share/keyrings/googlechrome-linux-keyring.gpg \
+    && sh -c 'echo "deb [arch=amd64 signed-by=/usr/share/keyrings/googlechrome-linux-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
+    && apt-get update \
+    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-khmeros fonts-kacst fonts-freefont-ttf libxss1 \
+      --no-install-recommends \
+    && ln /usr/share/novnc/vnc.html /usr/share/novnc/index.html \
+    && rm -rf /var/lib/apt/lists/*
 
 RUN mkdir -p /app
 COPY . /app/
 WORKDIR /app
 ENV NODE_PATH=/app/node_modules
-RUN npm install
 
-# Add user so we don't need --no-sandbox.
 RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
     && mkdir -p /home/pptruser/Downloads \
     && chown -R pptruser:pptruser /home/pptruser \
-    && chown -R pptruser:pptruser /app/node_modules
+    && chown -R pptruser:pptruser /app \
+    && chmod +x start_vnc.sh \
+    && chmod +x start_container.sh
 
 USER pptruser
+
+RUN yarn install
+
+# puppeteer-service
 EXPOSE 3000
-CMD npm start
+# VNC-server
+EXPOSE 5901
+# NoVNC
+EXPOSE 5900
+
+CMD ./start_container.sh