diff --git a/README.md b/README.md index 3f178dc..4b527ef 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,12 @@ -This project allows users of the RAMPART project [https://github.com/artic-network/rampart](https://github.com/artic-network/rampart) to take automatic screenshots of RAMPART chart outputs. +This application allows users of the RAMPART project [https://github.com/artic-network/rampart](https://github.com/artic-network/rampart) to take automatic screenshots of RAMPART chart outputs using puppeteer https://github.com/puppeteer/puppeteer](https://github.com/puppeteer/puppeteer). + +The application will also save the table data in the RAMPART reports tab into separate .csv files. --- ## Requirements -You need to have Node.js installed on your local machine, this is required to handle npm packages. +You need to have Node.js (at least version 10.18.1) installed on your local machine, this is required to handle npm packages. Note that RAMPART also requires Node.js to be at least version 10 or above so you should be able to rely on the version of Node.js that is already running your rampart project. https://nodejs.org/en/download/ @@ -28,9 +30,9 @@ Once RAMPART is running, navigate to the root of this project and run: node app.js ``` -The application will then automatically take screenshots of the charts in your active RAMPART session. Screenshots are saved as .png files in the project root under `/images`. Depending on how many charts you have in your project, it may take some time to finish taking all of the screenshots. +The application will then automatically take screenshots of the charts in your active RAMPART session. Screenshots are saved as .png files in the project root under `/outputs`. Depending on how many charts you have in your project, it may take some time to finish taking all of the screenshots. Any screenshots that fail (see below for possible errors) will have "-failed" appended to the filename. -Once the program has started, puppeteer will open a temporary Chrome window to start taking screenshots. The window will automatically close again when the program has finished. +The .csv table data in the RAMPART reports tab will also be saved into the `/outputs` directory. --- @@ -52,8 +54,32 @@ When no arguments are passed the application will assume your RAMPART project is --- -## Additional Project Notes +## Fullpage screenshot issues + +There is currently a known chromium bug that is affecting the behaviour of some fullpage screenshots, see here for more information [https://github.com/puppeteer/puppeteer/issues/1576](https://github.com/puppeteer/puppeteer/issues/1576). This seems to affect pages that have a large viewport height and causes the fullpage screenshot to duplicate content. + +This application currently uses a workaround to take the fullpage screenshot until the above bug is fixed. This workaround uses Jimp [https://www.npmjs.com/package/jimp](https://www.npmjs.com/package/jimp) to merge several screenshots into a fullpage image. + +--- + +## Timeout error + +When running the application, you may occasionally experience this error: "TimeoutError: waiting for selector "[selector]" failed". This could be related to a potential puppeteer bug: [https://github.com/puppeteer/puppeteer/issues/4072](https://github.com/puppeteer/puppeteer/issues/4072). + +When this error occurs the application will try to continue taking screenshots of the remaining charts so you will need to take the screenshot of the failed chart manually. + +--- + +## Why does the application sometimes fail to screenshot every chart? + +When viewing your image outputs you might occasionally notice that the application did not take a screenshot of every chart, or just took a screenshot of the tab bar containing the chart (in which case the image name will have "-failed" appended). This is related to the above mentioned timeout error [https://github.com/puppeteer/puppeteer/issues/4072](https://github.com/puppeteer/puppeteer/issues/4072). If this happens, try running the application again or take a manual screenshot of the charts that were missed. + +Any charts that fail to produce a successful screenshot will have "-failed" appened to the image name in the outputs folder (e.g. Mayinga-failed.png) and you will therefore need to take a screenshot of this image manually. + +--- + +## Compatibility -This project uses puppeteer [https://github.com/puppeteer/puppeteer](https://github.com/puppeteer/puppeteer) to programatically interact with the browser. +All features have been tested against Rampart version 1.1.0. This application relies on the structure of the Rampart page being constant, therefore future updates to Rampart may cause errors. --- diff --git a/app.js b/app.js index f43a5c8..8d6ca53 100644 --- a/app.js +++ b/app.js @@ -1,58 +1,60 @@ -const fs = require("fs"); -const puppeteer = require("puppeteer"); -const screenshot = require("./screenshot/screenshot"); -const saveReport = require("./screenshot/save_report"); +const rimraf = require("rimraf"); + +const puppeteerConnect = require("./src/puppeteer_connect"); +const createDirectories = require("./src/create_directories"); +const screenshot = require("./src/screenshot"); +const saveReport = require("./src/save_report"); // process arguments let url = process.argv[2]; if (!url) { console.log( - "No arguments passed to specify url. Defaulting to http://localhost:3000" + "No arguments passed to specify url. Defaulting to http://localhost:3000 \n" ); url = "http://localhost:3000"; } -const puppeteerConnect = async (url) => { - const browser = await puppeteer.launch({ - // headless: false, - // slowMo: 250, - }); +const runApp = async (url) => { + const { browser, page } = await puppeteerConnect.puppeteerConnect(url); - const page = await browser.newPage(); - await page.setViewport({ width: 1980, height: 50000 }); // setting large height to account for case where there are many charts + const { + imagesDirectory, + reportsDirectory, + tempImagesDirectory, + } = await createDirectories.createDirectories(); + // take screenshots of charts try { - await page.goto(url, { - waitUntil: "networkidle2", - }); + await screenshot.takeScreenshots( + page, + imagesDirectory, + tempImagesDirectory + ); } catch (err) { console.log( - "Having trouple establishing a connection. If you are attempting to connect to localhost try entering the url as your local ip address" + "\x1b[36m%s\x1b[0m", + " \n Something went wrong taking screenshots. Printing error... \n" ); console.log(err); - browser.close(); - return; } - // get the current date in YYYY-MM-DD format (for creating directories) - let timeStamp = Date.now(); - let dateObject = new Date(timeStamp); - let date = dateObject.getDate(); - let month = dateObject.getMonth() + 1; - let year = dateObject.getFullYear(); - - const directory = `./outputs/${ - year + "-" + month + "-" + date + "-" + timeStamp - }`; - if (!fs.existsSync(directory)) { - fs.mkdirSync(directory, { recursive: true }); + // save the information in the 'reports' tab as a .csv + try { + await saveReport.saveReport(page, reportsDirectory); + } catch (err) { + console.log( + "\x1b[36m%s\x1b[0m", + "Something went wrong saving the chart. Printing error... \n" + ); + console.log(err); } - await screenshot.takeScreenshots(page, directory); - // await saveReport.saveReport(page, directory); - console.log("All operations completed"); + console.log("cleaning up temporary files and folders"); + await rimraf(tempImagesDirectory, function () { + console.log("\x1b[36m%s\x1b[0m", "all operations completed"); + }); browser.close(); }; -puppeteerConnect(url); +runApp(url); diff --git a/package-lock.json b/package-lock.json index 798da92..ceea81b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -4,6 +4,335 @@ "lockfileVersion": 1, "requires": true, "dependencies": { + "@babel/runtime": { + "version": "7.12.5", + "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.12.5.tgz", + "integrity": "sha512-plcc+hbExy3McchJCEQG3knOsuh3HH+Prx1P6cLIkET/0dLuQDEnrT+s27Axgc9bqfsmNUNHfscgMUdBpC9xfg==", + "requires": { + "regenerator-runtime": "^0.13.4" + } + }, + "@jimp/bmp": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/bmp/-/bmp-0.16.1.tgz", + "integrity": "sha512-iwyNYQeBawrdg/f24x3pQ5rEx+/GwjZcCXd3Kgc+ZUd+Ivia7sIqBsOnDaMZdKCBPlfW364ekexnlOqyVa0NWg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "bmp-js": "^0.1.0" + } + }, + "@jimp/core": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/core/-/core-0.16.1.tgz", + "integrity": "sha512-la7kQia31V6kQ4q1kI/uLimu8FXx7imWVajDGtwUG8fzePLWDFJyZl0fdIXVCL1JW2nBcRHidUot6jvlRDi2+g==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "any-base": "^1.1.0", + "buffer": "^5.2.0", + "exif-parser": "^0.1.12", + "file-type": "^9.0.0", + "load-bmfont": "^1.3.1", + "mkdirp": "^0.5.1", + "phin": "^2.9.1", + "pixelmatch": "^4.0.2", + "tinycolor2": "^1.4.1" + } + }, + "@jimp/custom": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/custom/-/custom-0.16.1.tgz", + "integrity": "sha512-DNUAHNSiUI/j9hmbatD6WN/EBIyeq4AO0frl5ETtt51VN1SvE4t4v83ZA/V6ikxEf3hxLju4tQ5Pc3zmZkN/3A==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/core": "^0.16.1" + } + }, + "@jimp/gif": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/gif/-/gif-0.16.1.tgz", + "integrity": "sha512-r/1+GzIW1D5zrP4tNrfW+3y4vqD935WBXSc8X/wm23QTY9aJO9Lw6PEdzpYCEY+SOklIFKaJYUAq/Nvgm/9ryw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "gifwrap": "^0.9.2", + "omggif": "^1.0.9" + } + }, + "@jimp/jpeg": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/jpeg/-/jpeg-0.16.1.tgz", + "integrity": "sha512-8352zrdlCCLFdZ/J+JjBslDvml+fS3Z8gttdml0We759PnnZGqrnPRhkOEOJbNUlE+dD4ckLeIe6NPxlS/7U+w==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "jpeg-js": "0.4.2" + } + }, + "@jimp/plugin-blit": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blit/-/plugin-blit-0.16.1.tgz", + "integrity": "sha512-fKFNARm32RoLSokJ8WZXHHH2CGzz6ire2n1Jh6u+XQLhk9TweT1DcLHIXwQMh8oR12KgjbgsMGvrMVlVknmOAg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-blur": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blur/-/plugin-blur-0.16.1.tgz", + "integrity": "sha512-1WhuLGGj9MypFKRcPvmW45ht7nXkOKu+lg3n2VBzIB7r4kKNVchuI59bXaCYQumOLEqVK7JdB4glaDAbCQCLyw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-circle": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-circle/-/plugin-circle-0.16.1.tgz", + "integrity": "sha512-JK7yi1CIU7/XL8hdahjcbGA3V7c+F+Iw+mhMQhLEi7Q0tCnZ69YJBTamMiNg3fWPVfMuvWJJKOBRVpwNTuaZRg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-color": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-color/-/plugin-color-0.16.1.tgz", + "integrity": "sha512-9yQttBAO5SEFj7S6nJK54f+1BnuBG4c28q+iyzm1JjtnehjqMg6Ljw4gCSDCvoCQ3jBSYHN66pmwTV74SU1B7A==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "tinycolor2": "^1.4.1" + } + }, + "@jimp/plugin-contain": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-contain/-/plugin-contain-0.16.1.tgz", + "integrity": "sha512-44F3dUIjBDHN+Ym/vEfg+jtjMjAqd2uw9nssN67/n4FdpuZUVs7E7wadKY1RRNuJO+WgcD5aDQcsvurXMETQTg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-cover": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-cover/-/plugin-cover-0.16.1.tgz", + "integrity": "sha512-YztWCIldBAVo0zxcQXR+a/uk3/TtYnpKU2CanOPJ7baIuDlWPsG+YE4xTsswZZc12H9Kl7CiziEbDtvF9kwA/Q==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-crop": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-crop/-/plugin-crop-0.16.1.tgz", + "integrity": "sha512-UQdva9oQzCVadkyo3T5Tv2CUZbf0klm2cD4cWMlASuTOYgaGaFHhT9st+kmfvXjKL8q3STkBu/zUPV6PbuV3ew==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-displace": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-displace/-/plugin-displace-0.16.1.tgz", + "integrity": "sha512-iVAWuz2+G6Heu8gVZksUz+4hQYpR4R0R/RtBzpWEl8ItBe7O6QjORAkhxzg+WdYLL2A/Yd4ekTpvK0/qW8hTVw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-dither": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-dither/-/plugin-dither-0.16.1.tgz", + "integrity": "sha512-tADKVd+HDC9EhJRUDwMvzBXPz4GLoU6s5P7xkVq46tskExYSptgj5713J5Thj3NMgH9Rsqu22jNg1H/7tr3V9Q==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-fisheye": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-fisheye/-/plugin-fisheye-0.16.1.tgz", + "integrity": "sha512-BWHnc5hVobviTyIRHhIy9VxI1ACf4CeSuCfURB6JZm87YuyvgQh5aX5UDKtOz/3haMHXBLP61ZBxlNpMD8CG4A==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-flip": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-flip/-/plugin-flip-0.16.1.tgz", + "integrity": "sha512-KdxTf0zErfZ8DyHkImDTnQBuHby+a5YFdoKI/G3GpBl3qxLBvC+PWkS2F/iN3H7wszP7/TKxTEvWL927pypT0w==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-gaussian": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-gaussian/-/plugin-gaussian-0.16.1.tgz", + "integrity": "sha512-u9n4wjskh3N1mSqketbL6tVcLU2S5TEaFPR40K6TDv4phPLZALi1Of7reUmYpVm8mBDHt1I6kGhuCJiWvzfGyg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-invert": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-invert/-/plugin-invert-0.16.1.tgz", + "integrity": "sha512-2DKuyVXANH8WDpW9NG+PYFbehzJfweZszFYyxcaewaPLN0GxvxVLOGOPP1NuUTcHkOdMFbE0nHDuB7f+sYF/2w==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-mask": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-mask/-/plugin-mask-0.16.1.tgz", + "integrity": "sha512-snfiqHlVuj4bSFS0v96vo2PpqCDMe4JB+O++sMo5jF5mvGcGL6AIeLo8cYqPNpdO6BZpBJ8MY5El0Veckhr39Q==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-normalize": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-normalize/-/plugin-normalize-0.16.1.tgz", + "integrity": "sha512-dOQfIOvGLKDKXPU8xXWzaUeB0nvkosHw6Xg1WhS1Z5Q0PazByhaxOQkSKgUryNN/H+X7UdbDvlyh/yHf3ITRaw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-print": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-print/-/plugin-print-0.16.1.tgz", + "integrity": "sha512-ceWgYN40jbN4cWRxixym+csyVymvrryuKBQ+zoIvN5iE6OyS+2d7Mn4zlNgumSczb9GGyZZESIgVcBDA1ezq0Q==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "load-bmfont": "^1.4.0" + } + }, + "@jimp/plugin-resize": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-resize/-/plugin-resize-0.16.1.tgz", + "integrity": "sha512-u4JBLdRI7dargC04p2Ha24kofQBk3vhaf0q8FwSYgnCRwxfvh2RxvhJZk9H7Q91JZp6wgjz/SjvEAYjGCEgAwQ==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-rotate": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-rotate/-/plugin-rotate-0.16.1.tgz", + "integrity": "sha512-ZUU415gDQ0VjYutmVgAYYxC9Og9ixu2jAGMCU54mSMfuIlmohYfwARQmI7h4QB84M76c9hVLdONWjuo+rip/zg==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-scale": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-scale/-/plugin-scale-0.16.1.tgz", + "integrity": "sha512-jM2QlgThIDIc4rcyughD5O7sOYezxdafg/2Xtd1csfK3z6fba3asxDwthqPZAgitrLgiKBDp6XfzC07Y/CefUw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-shadow": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-shadow/-/plugin-shadow-0.16.1.tgz", + "integrity": "sha512-MeD2Is17oKzXLnsphAa1sDstTu6nxscugxAEk3ji0GV1FohCvpHBcec0nAq6/czg4WzqfDts+fcPfC79qWmqrA==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugin-threshold": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-threshold/-/plugin-threshold-0.16.1.tgz", + "integrity": "sha512-iGW8U/wiCSR0+6syrPioVGoSzQFt4Z91SsCRbgNKTAk7D+XQv6OI78jvvYg4o0c2FOlwGhqz147HZV5utoSLxA==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1" + } + }, + "@jimp/plugins": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/plugins/-/plugins-0.16.1.tgz", + "integrity": "sha512-c+lCqa25b+4q6mJZSetlxhMoYuiltyS+ValLzdwK/47+aYsq+kcJNl+TuxIEKf59yr9+5rkbpsPkZHLF/V7FFA==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/plugin-blit": "^0.16.1", + "@jimp/plugin-blur": "^0.16.1", + "@jimp/plugin-circle": "^0.16.1", + "@jimp/plugin-color": "^0.16.1", + "@jimp/plugin-contain": "^0.16.1", + "@jimp/plugin-cover": "^0.16.1", + "@jimp/plugin-crop": "^0.16.1", + "@jimp/plugin-displace": "^0.16.1", + "@jimp/plugin-dither": "^0.16.1", + "@jimp/plugin-fisheye": "^0.16.1", + "@jimp/plugin-flip": "^0.16.1", + "@jimp/plugin-gaussian": "^0.16.1", + "@jimp/plugin-invert": "^0.16.1", + "@jimp/plugin-mask": "^0.16.1", + "@jimp/plugin-normalize": "^0.16.1", + "@jimp/plugin-print": "^0.16.1", + "@jimp/plugin-resize": "^0.16.1", + "@jimp/plugin-rotate": "^0.16.1", + "@jimp/plugin-scale": "^0.16.1", + "@jimp/plugin-shadow": "^0.16.1", + "@jimp/plugin-threshold": "^0.16.1", + "timm": "^1.6.1" + } + }, + "@jimp/png": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/png/-/png-0.16.1.tgz", + "integrity": "sha512-iyWoCxEBTW0OUWWn6SveD4LePW89kO7ZOy5sCfYeDM/oTPLpR8iMIGvZpZUz1b8kvzFr27vPst4E5rJhGjwsdw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/utils": "^0.16.1", + "pngjs": "^3.3.3" + } + }, + "@jimp/tiff": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/tiff/-/tiff-0.16.1.tgz", + "integrity": "sha512-3K3+xpJS79RmSkAvFMgqY5dhSB+/sxhwTFA9f4AVHUK0oKW+u6r52Z1L0tMXHnpbAdR9EJ+xaAl2D4x19XShkQ==", + "requires": { + "@babel/runtime": "^7.7.2", + "utif": "^2.0.1" + } + }, + "@jimp/types": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/types/-/types-0.16.1.tgz", + "integrity": "sha512-g1w/+NfWqiVW4CaXSJyD28JQqZtm2eyKMWPhBBDCJN9nLCN12/Az0WFF3JUAktzdsEC2KRN2AqB1a2oMZBNgSQ==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/bmp": "^0.16.1", + "@jimp/gif": "^0.16.1", + "@jimp/jpeg": "^0.16.1", + "@jimp/png": "^0.16.1", + "@jimp/tiff": "^0.16.1", + "timm": "^1.6.1" + } + }, + "@jimp/utils": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/@jimp/utils/-/utils-0.16.1.tgz", + "integrity": "sha512-8fULQjB0x4LzUSiSYG6ZtQl355sZjxbv8r9PPAuYHzS9sGiSHJQavNqK/nKnpDsVkU88/vRGcE7t3nMU0dEnVw==", + "requires": { + "@babel/runtime": "^7.7.2", + "regenerator-runtime": "^0.13.3" + } + }, "@types/node": { "version": "14.11.2", "resolved": "https://registry.npmjs.org/@types/node/-/node-14.11.2.tgz", @@ -24,6 +353,50 @@ "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz", "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g==" }, + "ajv": { + "version": "6.12.6", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.12.6.tgz", + "integrity": "sha512-j3fVLgvTo527anyYyJOGTYJbG+vnnQYvE0m5mmkc1TK+nxAppkCLMIL0aZ4dblVCNoGShhm+kzE4ZUykBoMg4g==", + "requires": { + "fast-deep-equal": "^3.1.1", + "fast-json-stable-stringify": "^2.0.0", + "json-schema-traverse": "^0.4.1", + "uri-js": "^4.2.2" + } + }, + "any-base": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/any-base/-/any-base-1.1.0.tgz", + "integrity": "sha512-uMgjozySS8adZZYePpaWs8cxB9/kdzmpX6SgJZ+wbz1K5eYk5QMYDVJaZKhxyIHUdnnJkfR7SVgStgH7LkGUyg==" + }, + "asn1": { + "version": "0.2.4", + "resolved": "https://registry.npmjs.org/asn1/-/asn1-0.2.4.tgz", + "integrity": "sha512-jxwzQpLQjSmWXgwaCZE9Nz+glAG01yF1QnWgbhGwHI5A6FRIEY6IVqtHhIepHqI7/kyEyQEagBC5mBEFlIYvdg==", + "requires": { + "safer-buffer": "~2.1.0" + } + }, + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=" + }, + "asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=" + }, + "aws-sign2": { + "version": "0.7.0", + "resolved": "https://registry.npmjs.org/aws-sign2/-/aws-sign2-0.7.0.tgz", + "integrity": "sha1-tG6JCTSpWR8tL2+G1+ap8bP+dqg=" + }, + "aws4": { + "version": "1.11.0", + "resolved": "https://registry.npmjs.org/aws4/-/aws4-1.11.0.tgz", + "integrity": "sha512-xh1Rl34h6Fi1DC2WWKfxUTVqRsNnr6LsKz2+hfwDxQJWmrx8+c7ylaqBMcHfl1U1r2dsifOvKX3LQuLNZ+XSvA==" + }, "balanced-match": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.0.tgz", @@ -34,6 +407,19 @@ "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.3.1.tgz", "integrity": "sha512-mLQ4i2QO1ytvGWFWmcngKO//JXAQueZvwEKtjgQFM4jIK0kU+ytMfplL8j+n5mspOfjHwoAg+9yhb7BwAHm36g==" }, + "bcrypt-pbkdf": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz", + "integrity": "sha1-pDAdOJtqQ/m2f/PKEaP2Y342Dp4=", + "requires": { + "tweetnacl": "^0.14.3" + } + }, + "bignumber.js": { + "version": "2.4.0", + "resolved": "https://registry.npmjs.org/bignumber.js/-/bignumber.js-2.4.0.tgz", + "integrity": "sha1-g4qZLan51zfg9LLbC+YrsJ3Qxeg=" + }, "bl": { "version": "4.0.3", "resolved": "https://registry.npmjs.org/bl/-/bl-4.0.3.tgz", @@ -44,6 +430,11 @@ "readable-stream": "^3.4.0" } }, + "bmp-js": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/bmp-js/-/bmp-js-0.1.0.tgz", + "integrity": "sha1-4Fpj95amwf8l9Hcex62twUjAcjM=" + }, "brace-expansion": { "version": "1.1.11", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", @@ -67,21 +458,52 @@ "resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz", "integrity": "sha1-DTM+PwDqxQqhRUq9MO+MKl2ackI=" }, + "buffer-equal": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/buffer-equal/-/buffer-equal-0.0.1.tgz", + "integrity": "sha1-kbx0sR6kBbyRa8aqkI+q+ltKrEs=" + }, + "caseless": { + "version": "0.12.0", + "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", + "integrity": "sha1-G2gcIf+EAzyCZUMJBolCDRhxUdw=" + }, "chownr": { "version": "1.1.4", "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz", "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==" }, + "combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "requires": { + "delayed-stream": "~1.0.0" + } + }, "concat-map": { "version": "0.0.1", "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz", "integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=" }, + "core-util-is": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz", + "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac=" + }, "csv-writer": { "version": "1.6.0", "resolved": "https://registry.npmjs.org/csv-writer/-/csv-writer-1.6.0.tgz", "integrity": "sha512-NOx7YDFWEsM/fTRAJjRpPp8t+MKRVvniAg9wQlUKx20MFrPs73WLJhFf5iteqrxNYnsy924K3Iroh3yNHeYd2g==" }, + "dashdash": { + "version": "1.14.1", + "resolved": "https://registry.npmjs.org/dashdash/-/dashdash-1.14.1.tgz", + "integrity": "sha1-hTz6D3y+L+1d4gMmuN1YEDX24vA=", + "requires": { + "assert-plus": "^1.0.0" + } + }, "debug": { "version": "4.2.0", "resolved": "https://registry.npmjs.org/debug/-/debug-4.2.0.tgz", @@ -90,11 +512,30 @@ "ms": "2.1.2" } }, + "delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=" + }, "devtools-protocol": { "version": "0.0.799653", "resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.799653.tgz", "integrity": "sha512-t1CcaZbvm8pOlikqrsIM9GOa7Ipp07+4h/q9u0JXBWjPCjHdBl9KkddX87Vv9vBHoBGtwV79sYQNGnQM6iS5gg==" }, + "dom-walk": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/dom-walk/-/dom-walk-0.1.2.tgz", + "integrity": "sha512-6QvTW9mrGeIegrFXdtQi9pk7O/nSK6lSdXW2eqUspN5LWD7UTji2Fqw5V2YLjBpHEoU9Xl/eUWNpDeZvoyOv2w==" + }, + "ecc-jsbn": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/ecc-jsbn/-/ecc-jsbn-0.1.2.tgz", + "integrity": "sha1-OoOpBOVDUyh4dMVkt1SThoSamMk=", + "requires": { + "jsbn": "~0.1.0", + "safer-buffer": "^2.1.0" + } + }, "end-of-stream": { "version": "1.4.4", "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz", @@ -103,6 +544,21 @@ "once": "^1.4.0" } }, + "es6-promise": { + "version": "3.3.1", + "resolved": "https://registry.npmjs.org/es6-promise/-/es6-promise-3.3.1.tgz", + "integrity": "sha1-oIzd6EzNvzTQJ6FFG8kdS80ophM=" + }, + "exif-parser": { + "version": "0.1.12", + "resolved": "https://registry.npmjs.org/exif-parser/-/exif-parser-0.1.12.tgz", + "integrity": "sha1-WKnS1ywCwfbwKg70qRZicrd2CSI=" + }, + "extend": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", + "integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==" + }, "extract-zip": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/extract-zip/-/extract-zip-2.0.1.tgz", @@ -114,6 +570,21 @@ "yauzl": "^2.10.0" } }, + "extsprintf": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/extsprintf/-/extsprintf-1.3.0.tgz", + "integrity": "sha1-lpGEQOMEGnpBT4xS48V06zw+HgU=" + }, + "fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==" + }, + "fast-json-stable-stringify": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz", + "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==" + }, "fd-slicer": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/fd-slicer/-/fd-slicer-1.1.0.tgz", @@ -122,6 +593,11 @@ "pend": "~1.2.0" } }, + "file-type": { + "version": "9.0.0", + "resolved": "https://registry.npmjs.org/file-type/-/file-type-9.0.0.tgz", + "integrity": "sha512-Qe/5NJrgIOlwijpq3B7BEpzPFcgzggOTagZmkXQY4LA6bsXKTUstK7Wp12lEJ/mLKTpvIZxmIuRcLYWT6ov9lw==" + }, "find-up": { "version": "4.1.0", "resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz", @@ -131,6 +607,21 @@ "path-exists": "^4.0.0" } }, + "forever-agent": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/forever-agent/-/forever-agent-0.6.1.tgz", + "integrity": "sha1-+8cfDEGt6zf5bFd60e1C2P2sypE=" + }, + "form-data": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-2.3.3.tgz", + "integrity": "sha512-1lLKB2Mu3aGP1Q/2eCOx0fNbRMe7XdwktwOruhfqqd0rIJWwN4Dh+E3hrPSlDCXnSR7UtZ1N38rVXm+6+MEhJQ==", + "requires": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.6", + "mime-types": "^2.1.12" + } + }, "fs-constants": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz", @@ -149,6 +640,23 @@ "pump": "^3.0.0" } }, + "getpass": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/getpass/-/getpass-0.1.7.tgz", + "integrity": "sha1-Xv+OPmhNVprkyysSgmBOi6YhSfo=", + "requires": { + "assert-plus": "^1.0.0" + } + }, + "gifwrap": { + "version": "0.9.2", + "resolved": "https://registry.npmjs.org/gifwrap/-/gifwrap-0.9.2.tgz", + "integrity": "sha512-fcIswrPaiCDAyO8xnWvHSZdWChjKXUanKKpAiWWJ/UTkEi/aYKn5+90e7DE820zbEaVR9CE2y4z9bzhQijZ0BA==", + "requires": { + "image-q": "^1.1.1", + "omggif": "^1.0.10" + } + }, "glob": { "version": "7.1.6", "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz", @@ -162,6 +670,39 @@ "path-is-absolute": "^1.0.0" } }, + "global": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/global/-/global-4.3.2.tgz", + "integrity": "sha1-52mJJopsdMOJCLEwWxD8DjlOnQ8=", + "requires": { + "min-document": "^2.19.0", + "process": "~0.5.1" + } + }, + "har-schema": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/har-schema/-/har-schema-2.0.0.tgz", + "integrity": "sha1-qUwiJOvKwEeCoNkDVSHyRzW37JI=" + }, + "har-validator": { + "version": "5.1.5", + "resolved": "https://registry.npmjs.org/har-validator/-/har-validator-5.1.5.tgz", + "integrity": "sha512-nmT2T0lljbxdQZfspsno9hgrG3Uir6Ks5afism62poxqBM6sDnMEuPmzTq8XN0OEwqKLLdh1jQI3qyE66Nzb3w==", + "requires": { + "ajv": "^6.12.3", + "har-schema": "^2.0.0" + } + }, + "http-signature": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/http-signature/-/http-signature-1.2.0.tgz", + "integrity": "sha1-muzZJRFHcvPZW2WmCruPfBj7rOE=", + "requires": { + "assert-plus": "^1.0.0", + "jsprim": "^1.2.2", + "sshpk": "^1.7.0" + } + }, "https-proxy-agent": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz", @@ -176,6 +717,11 @@ "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.13.tgz", "integrity": "sha512-4vf7I2LYV/HaWerSo3XmlMkp5eZ83i+/CDluXi/IGTs/O1sejBNhTtnxzmRZfvOUqj7lZjqHkeTvpgSFDlWZTg==" }, + "image-q": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/image-q/-/image-q-1.1.1.tgz", + "integrity": "sha1-/IQJlmRGC5DKhi2TALa/u7+/gFY=" + }, "inflight": { "version": "1.0.6", "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", @@ -190,6 +736,94 @@ "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==" }, + "ip-regex": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/ip-regex/-/ip-regex-1.0.3.tgz", + "integrity": "sha1-3FiQdvZZ9BnCIgOaMzFvHHOH7/0=" + }, + "is-function": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/is-function/-/is-function-1.0.2.tgz", + "integrity": "sha512-lw7DUp0aWXYg+CBCN+JKkcE0Q2RayZnSvnZBlwgxHBQhqt5pZNVy4Ri7H9GmmXkdu7LUthszM+Tor1u/2iBcpQ==" + }, + "is-plain-obj": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-1.1.0.tgz", + "integrity": "sha1-caUMhCnfync8kqOQpKA7OfzVHT4=" + }, + "is-typedarray": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-typedarray/-/is-typedarray-1.0.0.tgz", + "integrity": "sha1-5HnICFjfDBsR3dppQPlgEfzaSpo=" + }, + "isstream": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/isstream/-/isstream-0.1.2.tgz", + "integrity": "sha1-R+Y/evVa+m+S4VAOaQ64uFKcCZo=" + }, + "jimp": { + "version": "0.16.1", + "resolved": "https://registry.npmjs.org/jimp/-/jimp-0.16.1.tgz", + "integrity": "sha512-+EKVxbR36Td7Hfd23wKGIeEyHbxShZDX6L8uJkgVW3ESA9GiTEPK08tG1XI2r/0w5Ch0HyJF5kPqF9K7EmGjaw==", + "requires": { + "@babel/runtime": "^7.7.2", + "@jimp/custom": "^0.16.1", + "@jimp/plugins": "^0.16.1", + "@jimp/types": "^0.16.1", + "regenerator-runtime": "^0.13.3" + } + }, + "jpeg-js": { + "version": "0.4.2", + "resolved": "https://registry.npmjs.org/jpeg-js/-/jpeg-js-0.4.2.tgz", + "integrity": "sha512-+az2gi/hvex7eLTMTlbRLOhH6P6WFdk2ITI8HJsaH2VqYO0I594zXSYEP+tf4FW+8Cy68ScDXoAsQdyQanv3sw==" + }, + "jsbn": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/jsbn/-/jsbn-0.1.1.tgz", + "integrity": "sha1-peZUwuWi3rXyAdls77yoDA7y9RM=" + }, + "json-schema": { + "version": "0.2.3", + "resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.2.3.tgz", + "integrity": "sha1-tIDIkuWaLwWVTOcnvT8qTogvnhM=" + }, + "json-schema-traverse": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-0.4.1.tgz", + "integrity": "sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg==" + }, + "json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha1-Epai1Y/UXxmg9s4B1lcB4sc1tus=" + }, + "jsprim": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/jsprim/-/jsprim-1.4.1.tgz", + "integrity": "sha1-MT5mvB5cwG5Di8G3SZwuXFastqI=", + "requires": { + "assert-plus": "1.0.0", + "extsprintf": "1.3.0", + "json-schema": "0.2.3", + "verror": "1.10.0" + } + }, + "load-bmfont": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/load-bmfont/-/load-bmfont-1.4.1.tgz", + "integrity": "sha512-8UyQoYmdRDy81Brz6aLAUhfZLwr5zV0L3taTQ4hju7m6biuwiWiJXjPhBJxbUQJA8PrkvJ/7Enqmwk2sM14soA==", + "requires": { + "buffer-equal": "0.0.1", + "mime": "^1.3.4", + "parse-bmfont-ascii": "^1.0.3", + "parse-bmfont-binary": "^1.0.5", + "parse-bmfont-xml": "^1.1.4", + "phin": "^2.9.1", + "xhr": "^2.0.1", + "xtend": "^4.0.0" + } + }, "locate-path": { "version": "5.0.0", "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz", @@ -198,6 +832,80 @@ "p-locate": "^4.1.0" } }, + "merge-img": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/merge-img/-/merge-img-2.1.3.tgz", + "integrity": "sha512-PA8caQPleTulKV7s2CXGvzTPxVoT5wkFCB0jf0RVzIYG+job08wibDA9lMYlpcHjCOvup3p1UPk1JgTwgJnu6g==", + "requires": { + "is-plain-obj": "^1.1.0", + "jimp": "0.2.27" + }, + "dependencies": { + "bmp-js": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/bmp-js/-/bmp-js-0.0.1.tgz", + "integrity": "sha1-WtAUcJnROp84qnuZrx1ueGZu038=" + }, + "file-type": { + "version": "3.9.0", + "resolved": "https://registry.npmjs.org/file-type/-/file-type-3.9.0.tgz", + "integrity": "sha1-JXoHg4TR24CHvESdEH1SpSZyuek=" + }, + "jimp": { + "version": "0.2.27", + "resolved": "https://registry.npmjs.org/jimp/-/jimp-0.2.27.tgz", + "integrity": "sha1-Qe9Qgti2MgHVR0fgT+i8rLryVHQ=", + "requires": { + "bignumber.js": "^2.1.0", + "bmp-js": "0.0.1", + "es6-promise": "^3.0.2", + "exif-parser": "^0.1.9", + "file-type": "^3.1.0", + "jpeg-js": "^0.2.0", + "load-bmfont": "^1.2.3", + "mime": "^1.3.4", + "pixelmatch": "^4.0.0", + "pngjs": "^3.0.0", + "read-chunk": "^1.0.1", + "request": "^2.65.0", + "stream-to-buffer": "^0.1.0", + "tinycolor2": "^1.1.2", + "url-regex": "^3.0.0" + } + }, + "jpeg-js": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/jpeg-js/-/jpeg-js-0.2.0.tgz", + "integrity": "sha1-U+RI7J0mPmgyZkZ+lELSxaLvVII=" + } + } + }, + "mime": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz", + "integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==" + }, + "mime-db": { + "version": "1.44.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.44.0.tgz", + "integrity": "sha512-/NOTfLrsPBVeH7YtFPgsVWveuL+4SjjYxaQ1xtM1KMFj7HdxlBlxeyNLzhyJVx7r4rZGJAZ/6lkKCitSc/Nmpg==" + }, + "mime-types": { + "version": "2.1.27", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.27.tgz", + "integrity": "sha512-JIhqnCasI9yD+SsmkquHBxTSEuZdQX5BuQnS2Vc7puQQQ+8yiP5AY5uWhpdv4YL4VM5c6iliiYWPgJ/nJQLp7w==", + "requires": { + "mime-db": "1.44.0" + } + }, + "min-document": { + "version": "2.19.0", + "resolved": "https://registry.npmjs.org/min-document/-/min-document-2.19.0.tgz", + "integrity": "sha1-e9KC4/WELtKVu3SM3Z8f+iyCRoU=", + "requires": { + "dom-walk": "^0.1.0" + } + }, "minimatch": { "version": "3.0.4", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz", @@ -206,6 +914,19 @@ "brace-expansion": "^1.1.7" } }, + "minimist": { + "version": "1.2.5", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.5.tgz", + "integrity": "sha512-FM9nNUYrRBAELZQT3xeZQ7fmMOBg6nWNmJKTcgsJeaLstP/UODVpGsr5OhXhhXg6f+qtJ8uiZ+PUxkDWcgIXLw==" + }, + "mkdirp": { + "version": "0.5.5", + "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.5.tgz", + "integrity": "sha512-NKmAlESf6jMGym1++R0Ra7wvhV+wFW63FaSOFPwRahvea0gMUcGUhVeAg/0BC0wiv9ih5NYPB1Wn1UEI1/L+xQ==", + "requires": { + "minimist": "^1.2.5" + } + }, "mkdirp-classic": { "version": "0.5.3", "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz", @@ -216,6 +937,16 @@ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" }, + "oauth-sign": { + "version": "0.9.0", + "resolved": "https://registry.npmjs.org/oauth-sign/-/oauth-sign-0.9.0.tgz", + "integrity": "sha512-fexhUFFPTGV8ybAtSIGbV6gOkSv8UtRbDBnAyLQw4QPKkgNlsH2ByPGtMUqdWkos6YCRmAqViwgZrJc/mRDzZQ==" + }, + "omggif": { + "version": "1.0.10", + "resolved": "https://registry.npmjs.org/omggif/-/omggif-1.0.10.tgz", + "integrity": "sha512-LMJTtvgc/nugXj0Vcrrs68Mn2D1r0zf630VNtqtpI1FEO7e+O9FP4gqs9AcnBaSEeoHIPm28u6qgPR0oyEpGSw==" + }, "once": { "version": "1.4.0", "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", @@ -245,6 +976,35 @@ "resolved": "https://registry.npmjs.org/p-try/-/p-try-2.2.0.tgz", "integrity": "sha512-R4nPAVTAU0B9D35/Gk3uJf/7XYbQcyohSKdvAxIRSNghFl4e71hVoGnBNQz9cWaXxO2I10KTC+3jMdvvoKw6dQ==" }, + "pako": { + "version": "1.0.11", + "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz", + "integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==" + }, + "parse-bmfont-ascii": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/parse-bmfont-ascii/-/parse-bmfont-ascii-1.0.6.tgz", + "integrity": "sha1-Eaw8P/WPfCAgqyJ2kHkQjU36AoU=" + }, + "parse-bmfont-binary": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/parse-bmfont-binary/-/parse-bmfont-binary-1.0.6.tgz", + "integrity": "sha1-0Di0dtPp3Z2x4RoLDlOiJ5K2kAY=" + }, + "parse-bmfont-xml": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/parse-bmfont-xml/-/parse-bmfont-xml-1.1.4.tgz", + "integrity": "sha512-bjnliEOmGv3y1aMEfREMBJ9tfL3WR0i0CKPj61DnSLaoxWR3nLrsQrEbCId/8rF4NyRF0cCqisSVXyQYWM+mCQ==", + "requires": { + "xml-parse-from-string": "^1.0.0", + "xml2js": "^0.4.5" + } + }, + "parse-headers": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/parse-headers/-/parse-headers-2.0.3.tgz", + "integrity": "sha512-QhhZ+DCCit2Coi2vmAKbq5RGTRcQUOE2+REgv8vdyu7MnYx2eZztegqtTx99TZ86GTIwqiy3+4nQTWZ2tgmdCA==" + }, "path-exists": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz", @@ -260,6 +1020,24 @@ "resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz", "integrity": "sha1-elfrVQpng/kRUzH89GY9XI4AelA=" }, + "performance-now": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/performance-now/-/performance-now-2.1.0.tgz", + "integrity": "sha1-Ywn04OX6kT7BxpMHrjZLSzd8nns=" + }, + "phin": { + "version": "2.9.3", + "resolved": "https://registry.npmjs.org/phin/-/phin-2.9.3.tgz", + "integrity": "sha512-CzFr90qM24ju5f88quFC/6qohjC144rehe5n6DH900lgXmUe86+xCKc10ev56gRKC4/BkHUoG4uSiQgBiIXwDA==" + }, + "pixelmatch": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/pixelmatch/-/pixelmatch-4.0.2.tgz", + "integrity": "sha1-j0fc7FARtHe2fbA8JDvB8wheiFQ=", + "requires": { + "pngjs": "^3.0.0" + } + }, "pkg-dir": { "version": "4.2.0", "resolved": "https://registry.npmjs.org/pkg-dir/-/pkg-dir-4.2.0.tgz", @@ -268,6 +1046,16 @@ "find-up": "^4.0.0" } }, + "pngjs": { + "version": "3.4.0", + "resolved": "https://registry.npmjs.org/pngjs/-/pngjs-3.4.0.tgz", + "integrity": "sha512-NCrCHhWmnQklfH4MtJMRjZ2a8c80qXeMlQMv2uVp9ISJMTt562SbGd6n2oq0PaPgKm7Z6pL9E2UlLIhC+SHL3w==" + }, + "process": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/process/-/process-0.5.2.tgz", + "integrity": "sha1-FjjYqONML0QKkduVq5rrZ3/Bhc8=" + }, "progress": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz", @@ -278,6 +1066,11 @@ "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz", "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==" }, + "psl": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/psl/-/psl-1.8.0.tgz", + "integrity": "sha512-RIdOzyoavK+hA18OGGWDqUTsCLhtA7IcZ/6NCs4fFJaHBDab+pDDmDIByWFRQJq2Cd7r1OoQxBGKOaztq+hjIQ==" + }, "pump": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.0.tgz", @@ -287,6 +1080,11 @@ "once": "^1.3.1" } }, + "punycode": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.1.1.tgz", + "integrity": "sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==" + }, "puppeteer": { "version": "5.3.1", "resolved": "https://registry.npmjs.org/puppeteer/-/puppeteer-5.3.1.tgz", @@ -305,6 +1103,16 @@ "ws": "^7.2.3" } }, + "qs": { + "version": "6.5.2", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.2.tgz", + "integrity": "sha512-N5ZAX4/LxJmF+7wN74pUD6qAh9/wnvdQcjq9TZjevvXzSUo7bfmw91saqMjzGS2xq91/odN2dW/WOl7qQHNDGA==" + }, + "read-chunk": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/read-chunk/-/read-chunk-1.0.1.tgz", + "integrity": "sha1-X2jKswfmY/GZk1J9m1icrORmEZQ=" + }, "readable-stream": { "version": "3.6.0", "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.0.tgz", @@ -315,6 +1123,45 @@ "util-deprecate": "^1.0.1" } }, + "regenerator-runtime": { + "version": "0.13.7", + "resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.7.tgz", + "integrity": "sha512-a54FxoJDIr27pgf7IgeQGxmqUNYrcV338lf/6gH456HZ/PhX+5BcwHXG9ajESmwe6WRO0tAzRUrRmNONWgkrew==" + }, + "request": { + "version": "2.88.2", + "resolved": "https://registry.npmjs.org/request/-/request-2.88.2.tgz", + "integrity": "sha512-MsvtOrfG9ZcrOwAW+Qi+F6HbD0CWXEh9ou77uOb7FM2WPhwT7smM833PzanhJLsgXjN89Ir6V2PczXNnMpwKhw==", + "requires": { + "aws-sign2": "~0.7.0", + "aws4": "^1.8.0", + "caseless": "~0.12.0", + "combined-stream": "~1.0.6", + "extend": "~3.0.2", + "forever-agent": "~0.6.1", + "form-data": "~2.3.2", + "har-validator": "~5.1.3", + "http-signature": "~1.2.0", + "is-typedarray": "~1.0.0", + "isstream": "~0.1.2", + "json-stringify-safe": "~5.0.1", + "mime-types": "~2.1.19", + "oauth-sign": "~0.9.0", + "performance-now": "^2.1.0", + "qs": "~6.5.2", + "safe-buffer": "^5.1.2", + "tough-cookie": "~2.5.0", + "tunnel-agent": "^0.6.0", + "uuid": "^3.3.2" + }, + "dependencies": { + "uuid": { + "version": "3.4.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.4.0.tgz", + "integrity": "sha512-HjSDRw6gZE5JMggctHBcjVak08+KEVhSIiDzFnT9S9aegmp85S/bReBVTb4QTFaRNptJ9kuYaNhnbNEOkbKb/A==" + } + } + }, "rimraf": { "version": "3.0.2", "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-3.0.2.tgz", @@ -328,6 +1175,45 @@ "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==" }, + "safer-buffer": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==" + }, + "sax": { + "version": "1.2.4", + "resolved": "https://registry.npmjs.org/sax/-/sax-1.2.4.tgz", + "integrity": "sha512-NqVDv9TpANUjFm0N8uM5GxL36UgKi9/atZw+x7YFnQ8ckwFGKrl4xX4yWtrey3UJm5nP1kUbnYgLopqWNSRhWw==" + }, + "sshpk": { + "version": "1.16.1", + "resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.16.1.tgz", + "integrity": "sha512-HXXqVUq7+pcKeLqqZj6mHFUMvXtOJt1uoUx09pFW6011inTMxqI8BA8PM95myrIyyKwdnzjdFjLiE6KBPVtJIg==", + "requires": { + "asn1": "~0.2.3", + "assert-plus": "^1.0.0", + "bcrypt-pbkdf": "^1.0.0", + "dashdash": "^1.12.0", + "ecc-jsbn": "~0.1.1", + "getpass": "^0.1.1", + "jsbn": "~0.1.0", + "safer-buffer": "^2.0.2", + "tweetnacl": "~0.14.0" + } + }, + "stream-to": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/stream-to/-/stream-to-0.2.2.tgz", + "integrity": "sha1-hDBgmNhf25kLn6MAsbPM9V6O8B0=" + }, + "stream-to-buffer": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/stream-to-buffer/-/stream-to-buffer-0.1.0.tgz", + "integrity": "sha1-JnmdkDqyAlyb1VCsRxcbAPjdgKk=", + "requires": { + "stream-to": "~0.2.0" + } + }, "string_decoder": { "version": "1.3.0", "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", @@ -364,6 +1250,38 @@ "resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz", "integrity": "sha1-DdTJ/6q8NXlgsbckEV1+Doai4fU=" }, + "timm": { + "version": "1.7.1", + "resolved": "https://registry.npmjs.org/timm/-/timm-1.7.1.tgz", + "integrity": "sha512-IjZc9KIotudix8bMaBW6QvMuq64BrJWFs1+4V0lXwWGQZwH+LnX87doAYhem4caOEusRP9/g6jVDQmZ8XOk1nw==" + }, + "tinycolor2": { + "version": "1.4.2", + "resolved": "https://registry.npmjs.org/tinycolor2/-/tinycolor2-1.4.2.tgz", + "integrity": "sha512-vJhccZPs965sV/L2sU4oRQVAos0pQXwsvTLkWYdqJ+a8Q5kPFzJTuOFwy7UniPli44NKQGAglksjvOcpo95aZA==" + }, + "tough-cookie": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-2.5.0.tgz", + "integrity": "sha512-nlLsUzgm1kfLXSXfRZMc1KLAugd4hqJHDTvc2hDIwS3mZAfMEuMbc03SujMF+GEcpaX/qboeycw6iO8JwVv2+g==", + "requires": { + "psl": "^1.1.28", + "punycode": "^2.1.1" + } + }, + "tunnel-agent": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz", + "integrity": "sha1-J6XeoGs2sEoKmWZ3SykIaPD8QP0=", + "requires": { + "safe-buffer": "^5.0.1" + } + }, + "tweetnacl": { + "version": "0.14.5", + "resolved": "https://registry.npmjs.org/tweetnacl/-/tweetnacl-0.14.5.tgz", + "integrity": "sha1-WuaBd/GS1EViadEIr6k/+HQ/T2Q=" + }, "unbzip2-stream": { "version": "1.4.3", "resolved": "https://registry.npmjs.org/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz", @@ -373,6 +1291,30 @@ "through": "^2.3.8" } }, + "uri-js": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/uri-js/-/uri-js-4.4.0.tgz", + "integrity": "sha512-B0yRTzYdUCCn9n+F4+Gh4yIDtMQcaJsmYBDsTSG8g/OejKBodLQ2IHfN3bM7jUsRXndopT7OIXWdYqc1fjmV6g==", + "requires": { + "punycode": "^2.1.0" + } + }, + "url-regex": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/url-regex/-/url-regex-3.2.0.tgz", + "integrity": "sha1-260eDJ4p4QXdCx8J9oYvf9tIJyQ=", + "requires": { + "ip-regex": "^1.0.1" + } + }, + "utif": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/utif/-/utif-2.0.1.tgz", + "integrity": "sha512-Z/S1fNKCicQTf375lIP9G8Sa1H/phcysstNrrSdZKj1f9g58J4NMgb5IgiEZN9/nLMPDwF0W7hdOe9Qq2IYoLg==", + "requires": { + "pako": "^1.0.5" + } + }, "util-deprecate": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", @@ -383,6 +1325,16 @@ "resolved": "https://registry.npmjs.org/uuid/-/uuid-8.3.1.tgz", "integrity": "sha512-FOmRr+FmWEIG8uhZv6C2bTgEVXsHk08kE7mPlrBbEe+c3r9pjceVPgupIfNIhc4yx55H69OXANrUaSuu9eInKg==" }, + "verror": { + "version": "1.10.0", + "resolved": "https://registry.npmjs.org/verror/-/verror-1.10.0.tgz", + "integrity": "sha1-OhBcoXBTr1XW4nDB+CiGguGNpAA=", + "requires": { + "assert-plus": "^1.0.0", + "core-util-is": "1.0.2", + "extsprintf": "^1.2.0" + } + }, "wrappy": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", @@ -393,6 +1345,41 @@ "resolved": "https://registry.npmjs.org/ws/-/ws-7.3.1.tgz", "integrity": "sha512-D3RuNkynyHmEJIpD2qrgVkc9DQ23OrN/moAwZX4L8DfvszsJxpjQuUq3LMx6HoYji9fbIOBY18XWBsAux1ZZUA==" }, + "xhr": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/xhr/-/xhr-2.5.0.tgz", + "integrity": "sha512-4nlO/14t3BNUZRXIXfXe+3N6w3s1KoxcJUUURctd64BLRe67E4gRwp4PjywtDY72fXpZ1y6Ch0VZQRY/gMPzzQ==", + "requires": { + "global": "~4.3.0", + "is-function": "^1.0.1", + "parse-headers": "^2.0.0", + "xtend": "^4.0.0" + } + }, + "xml-parse-from-string": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/xml-parse-from-string/-/xml-parse-from-string-1.0.1.tgz", + "integrity": "sha1-qQKekp09vN7RafPG4oI42VpdWig=" + }, + "xml2js": { + "version": "0.4.23", + "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.4.23.tgz", + "integrity": "sha512-ySPiMjM0+pLDftHgXY4By0uswI3SPKLDw/i3UXbnO8M/p28zqexCUoPmQFrYD+/1BzhGJSs2i1ERWKJAtiLrug==", + "requires": { + "sax": ">=0.6.0", + "xmlbuilder": "~11.0.0" + } + }, + "xmlbuilder": { + "version": "11.0.1", + "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-11.0.1.tgz", + "integrity": "sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==" + }, + "xtend": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", + "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==" + }, "yauzl": { "version": "2.10.0", "resolved": "https://registry.npmjs.org/yauzl/-/yauzl-2.10.0.tgz", diff --git a/package.json b/package.json index 5d302ad..f8b6ec5 100644 --- a/package.json +++ b/package.json @@ -5,14 +5,16 @@ "main": "app.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1", - "start": "node app.js", - "build": "nexe" + "start": "node app.js" }, "author": "", "license": "ISC", "dependencies": { "csv-writer": "^1.6.0", + "jimp": "^0.16.1", + "merge-img": "^2.1.3", "puppeteer": "^5.3.1", + "rimraf": "^3.0.2", "uuid": "^8.3.1" }, "devDependencies": {} diff --git a/screenshot/save_report.js b/screenshot/save_report.js deleted file mode 100644 index deae8eb..0000000 --- a/screenshot/save_report.js +++ /dev/null @@ -1,92 +0,0 @@ -const { table } = require("console"); -const fs = require("fs"); -const { v4: uuidv4 } = require("uuid"); -const createCsvWriter = require("csv-writer").createObjectCsvWriter; - -const saveReport = async (page, directory) => { - const reportButtonElement = await page.$( - `#root > div > div > div.sc-AxirZ.aHDei > div.buttons > button` - ); - - if (reportButtonElement === null) { - console.log("No report tab was found"); - return; - } - - await reportButtonElement.click(); // open the report tab - - await page.waitForSelector( - `#root > div > div > div.sc-fzqBZW.ksbfDg.open > div` - ); - - // get the headers from the first table - const tableHeadData = await page.evaluate(() => { - const thead = Array.from( - document.querySelectorAll( - "#root > div > div > div.sc-fzqBZW.ksbfDg.open > div > div.sc-fzqNJr.hSLfNI > table:nth-child(2) > thead > tr > th" - ) - ); - return thead.map((th) => th.innerText); - }); - - // format header data to be inserted into csv - const csvHeaders = tableHeadData.map((header, index) => { - return { - id: index, - title: header, - }; - }); - console.log(tableHeadData); - console.log(csvHeaders); - - // get the rows from the first table - const tableRowData = await page.evaluate(() => { - const tbody = Array.from( - document.querySelectorAll( - "#root > div > div > div.sc-fzqBZW.ksbfDg.open > div > div.sc-fzqNJr.hSLfNI > table:nth-child(2) > tbody > tr" - ) - ); - return tbody.map((tr) => tr.innerText); - }); - //console.log(tableRowData); - - // extract content from each table row - const parsedRowData = []; - tableRowData.forEach((element) => { - const splitRow = element.split("\t"); - parsedRowData.push(splitRow); - }); - console.log(parsedRowData); - - const csvRows = []; - parsedRowData.map((row) => { - let obj = {}; - row.map((inner, index) => { - obj[index] = inner; - }); - csvRows.push(obj); - }); - console.log(csvRows); - - // write data to csv - const csvWriter = createCsvWriter({ - path: `${directory}/report.csv`, - header: csvHeaders, - }); - - // const csvRows = [ - // { 1: "test1", 2: "test2", 3: "test3", 4: "test4", 5: "test5", 6: "test6" }, - // { 1: "test1", 2: "test2", 3: "test3", 4: "test4", 5: "test5", 6: "test6" }, - // ]; - - await csvWriter - .writeRecords(csvRows) // returns a promise - .then(() => { - console.log("saved report as .csv"); - }) - .catch((error) => { - console.error(error); - }); -}; - -exports.saveReport = saveReport; diff --git a/screenshot/screenshot.js b/screenshot/screenshot.js deleted file mode 100644 index 1b3a465..0000000 --- a/screenshot/screenshot.js +++ /dev/null @@ -1,132 +0,0 @@ -const fs = require("fs"); -const { v4: uuidv4 } = require("uuid"); - -const takeScreenshots = async (page, directory) => { - // count how many divs are clickable buttons (to open the chart tabs) - let i = 1; - let blocksLeftToCount = true; - while (blocksLeftToCount) { - const element = await page.$(`#root > div > div > div:nth-child(${i})`); - - if (element === null) { - // element will be null when it finds no more dom elements - i--; // do this to avoid adding an extra element to the count - blocksLeftToCount = false; - } - i++; - } - const foundElementsCount = i; - - // open all of the tabs containing charts (necessary to take screenshots) - for (let i = 1; i < foundElementsCount + 1; i++) { - const element = await page.$( - `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > svg` - ); - - if (element === null) { - // element will be null if it doesn't contain the button to open the chart tab - continue; - } - await element.click(); // open the chart tab and wait for the element to appear in the dom - await page.waitForSelector( - `#root > div > div > div:nth-child(${i}) > div > div:nth-child(2)` - ); - - // remove this repetition - const chartTabName = await page.$( - `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > span` - ); - let chartTabNameText = await page.evaluate( - (element) => element.innerHTML, - chartTabName - ); - - console.log(`Opening chart tab: ${chartTabNameText}, please wait...`); - } - - // store an array of elements pointing towards the tabs containing charts, also store the tab names - let chartTabGroupElements = []; - let chartTabGroupNames = []; - for (let i = 1; i < foundElementsCount + 1; i++) { - const element = await page.$( - `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > svg` - ); - - if (element === null) { - // element will be null if it doesn't contain the svg button to open the chart tab - continue; - } - - const chartTabName = await page.$( - `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > span` - ); - let chartTabNameText = await page.evaluate( - (element) => element.innerHTML, - chartTabName - ); - - // if there are duplicate tab names, append a unique id to the end of the name - if ( - chartTabGroupNames.includes(chartTabNameText) || - fs.existsSync(`./${directory}/${chartTabNameText}.png`) - ) { - chartTabNameText += `_${uuidv4()}`; - } - - chartTabGroupNames.push(chartTabNameText); - chartTabGroupElements.push(`#root > div > div > div:nth-child(${i}) > div`); - } - - // screenshot each tab containing charts - for (let i = 0; i < chartTabGroupElements.length; i++) { - console.log(`Taking screenshot: ${chartTabGroupNames[i]}, please wait...`); - const chartTab = await page.$(chartTabGroupElements[i]); - await chartTab.screenshot({ - path: `${directory}/${chartTabGroupNames[i]}.png`, - omitBackground: true, - }); - } - - // screenshot the first set of charts in the top header (append unique id to name if it already exists in the directory) - const headerCharts = await page.$( - "#root > div > div > div:nth-child(2) > div" - ); - await headerCharts.screenshot({ - path: fs.existsSync(`./${directory}/header_charts.png`) - ? `${directory}/header_charts_${uuidv4()}.png` - : `${directory}/header_charts.png`, - omitBackground: true, - }); - - // screenshot the top header - const topHeader = await page.$("#root > div > div > div"); - await topHeader.screenshot({ - path: fs.existsSync(`./${directory}/top_header.png`) - ? `${directory}/top_header${uuidv4()}.png` - : `${directory}/top_header.png`, - omitBackground: true, - }); - - // detect the size of the page when all chart tabs are open so we can take a full page screenshot - // commented out setViewport as the chart tabs get closed when changing the viewport size (this is an issue with RAMPART not this app) - const rootElement = await page.$("#root"); - const boundingBox = await rootElement.boundingBox(); - const { width, height } = boundingBox; - - // await page.setViewport({ width: width, height: height }); - if (height > page.viewport().height) { - console.log( - `Viewport too small to fit images on screen. You need to increase the viewport height in app.js line 20. Please set the height larger than ${height}` - ); - } - - // screenshot the entire page (append unique id to name if it already exists in the directory) - await rootElement.screenshot({ - path: fs.existsSync(`./${directory}/full_page.png`) - ? `${directory}/full_page${uuidv4()}.png` - : `${directory}/full_page.png`, - omitBackground: true, - }); -}; - -exports.takeScreenshots = takeScreenshots; diff --git a/src/create_directories.js b/src/create_directories.js new file mode 100644 index 0000000..ba7d439 --- /dev/null +++ b/src/create_directories.js @@ -0,0 +1,32 @@ +const fs = require("fs"); + +const createDirectories = async () => { + // get the current date in YYYY-MM-DD format (for creating directories) + let timeStamp = Date.now(); + let dateObject = new Date(timeStamp); + let date = dateObject.getDate(); + let month = dateObject.getMonth() + 1; + let year = dateObject.getFullYear(); + + const imagesDirectory = `./outputs/${ + year + "-" + month + "-" + date + "-" + timeStamp + }`; + const reportsDirectory = `${imagesDirectory}/reports`; + + if (!fs.existsSync(reportsDirectory)) { + fs.mkdirSync(reportsDirectory, { recursive: true }); + } + + const tempImagesDirectory = "./outputs/tempImages"; + if (!fs.existsSync(tempImagesDirectory)) { + fs.mkdirSync(tempImagesDirectory, { recursive: true }); + } + + return { + imagesDirectory: imagesDirectory, + reportsDirectory: reportsDirectory, + tempImagesDirectory: tempImagesDirectory, + }; +}; + +exports.createDirectories = createDirectories; diff --git a/src/puppeteer_connect.js b/src/puppeteer_connect.js new file mode 100644 index 0000000..a34bad8 --- /dev/null +++ b/src/puppeteer_connect.js @@ -0,0 +1,29 @@ +const puppeteer = require("puppeteer"); + +const puppeteerConnect = async (url) => { + const browser = await puppeteer.launch({ + // headless: false, + // slowMo: 250, + }); + + const page = await browser.newPage(); + await page.setViewport({ width: 1980, height: 1000 }); + page.setDefaultTimeout(10000); + + try { + await page.goto(url, { + waitUntil: "networkidle2", + }); + } catch (err) { + console.log( + "Having trouple establishing a connection. If you are attempting to connect to localhost try entering the url as your local ip address" + ); + console.log(err); + browser.close(); + return; + } + + return { browser: browser, page: page }; +}; + +exports.puppeteerConnect = puppeteerConnect; diff --git a/src/save_report.js b/src/save_report.js new file mode 100644 index 0000000..69ea365 --- /dev/null +++ b/src/save_report.js @@ -0,0 +1,125 @@ +const fs = require("fs"); +const { v4: uuidv4 } = require("uuid"); +const createCsvWriter = require("csv-writer").createObjectCsvWriter; + +const saveReport = async (page, directory) => { + const openReportTab = async () => { + const reportButtonElement = await page.$( + `#root > div > div > div:nth-child(1) > div.buttons > button` + ); + if (reportButtonElement === null) { + console.log("No report tab was found"); + return; + } + await reportButtonElement.click(); + await page.waitForSelector(`#root > div > div > .open > div`); + }; + + const countTables = async () => { + const tables = await page.$$( + `#root > div > div > .open > div > :nth-child(2) > table` + ); + return tables.length; + }; + + const getTableName = async (tableRef) => { + return await page.$eval( + `${tableRef} > caption`, + (tableCaption) => tableCaption.innerText + ); + }; + + const getTableHeaderData = async (tableRef) => { + const tableHeadData = await page.evaluate((tableRef) => { + const thead = Array.from( + document.querySelectorAll(`${tableRef} > thead > tr > th`) + ); + return thead.map((thead) => thead.innerText); + }, tableRef); + + // an alternative method to above (more abstract and less clear what it does) + // const tableHeadData = await page.$$eval( + // `${tableRef} > thead > tr > th`, + // (theads) => theads.map((th) => th.innerText) + // ); + + // format header data to be inserted into csv + return tableHeadData.map((header, index) => { + return { + id: index, + title: header, + }; + }); + }; + + const getTableRowData = async (tableRef) => { + // returns a tab separated string of row data + return await page.evaluate((tableRef) => { + const tbody = Array.from( + document.querySelectorAll(`${tableRef} > tbody > tr`) + ); + return tbody.map((tr) => tr.innerText); + }, tableRef); + }; + + const parseTableData = async (tableRowData) => { + // convert each table row data into an array of rowItems + const parsedRowData = []; + tableRowData.forEach((row) => { + const splitRow = row.split("\t"); + parsedRowData.push(splitRow); + }); + + const csvRows = []; + parsedRowData.map((row) => { + let rowObject = {}; + row.map((rowItem, index) => { + rowObject[index] = rowItem; // index will be the csv tab number where the rowItem exists + }); + csvRows.push(rowObject); + }); + + return csvRows; + }; + + const writeTableToCsv = async (tableName, csvHeaders, csvRows) => { + const csvWriter = createCsvWriter({ + path: fs.existsSync(`./${directory}/report_${tableName}.csv`) + ? `${directory}/report_${tableName}_${uuidv4()}.csv` + : `${directory}/report_${tableName}.csv`, + header: csvHeaders, + }); + + await csvWriter + .writeRecords(csvRows) + .then(() => { + console.log(`saved '${tableName}' as .csv`); + }) + .catch((error) => { + console.log( + `something went wrong trying to write report '${tableName}' as .csv \n${error}` + ); + }); + }; + + // module flow starts here + await openReportTab(); + const tableCount = await countTables(); + if (tableCount === 0) { + console.log("no tables were found"); + return; + } + + // loop through each table and save the data as csv + for (let i = 1; i < tableCount + 1; i++) { + const tableRef = `#root > div > div > .open > div > div:nth-child(2) > table:nth-of-type(${i})`; + + const tableName = await getTableName(tableRef); + const csvHeaders = await getTableHeaderData(tableRef); + const tableRowData = await getTableRowData(tableRef); + const csvRows = await parseTableData(tableRowData); + await writeTableToCsv(tableName, csvHeaders, csvRows); + } +}; + +exports.saveReport = saveReport; diff --git a/src/screenshot.js b/src/screenshot.js new file mode 100644 index 0000000..5295d4c --- /dev/null +++ b/src/screenshot.js @@ -0,0 +1,258 @@ +const fs = require("fs"); +const { v4: uuidv4 } = require("uuid"); +const Jimp = require("jimp"); +const mergeImg = require("merge-img"); + +const takeScreenshots = async (page, directory, tempImagesDirectory) => { + const getElementText = async (elementRef) => { + // get the name of the current chart tab + const elementTextRef = await page.$(elementRef); + let text = await page.evaluate( + (element) => element.innerHTML, + elementTextRef + ); + return text; + }; + + const countElements = async () => { + // count how many divs exist in the dom (so we know what to iterate over when opening chart tabs) + let i = 1; + let blocksLeftToCount = true; + while (blocksLeftToCount) { + const divRef = `#root > div > div > div:nth-child(${i})`; + const element = await page.$(divRef); + + if (element === null) { + // element will be null when it finds no more dom elements + i--; // do this to avoid adding an extra element to the count + blocksLeftToCount = false; + } + i++; + } + return i; + }; + + // open all of the tabs containing charts (necessary to take screenshots) + const openChartTabs = async (domElementCount) => { + for (let i = 1; i < domElementCount + 1; i++) { + // a recursive function that will attempt to open the chart tab 2 times if it fails for some reason + let attempts = 0; + const openTabAndWait = async () => { + // get the button to open the tab again (else the node detaches on the second attempt) + const element = await page.$(tabButtonRef); + await element.click(); + console.log(`chart tab button ${chartTabNameText} clicked`); + + try { + console.log( + "waiting for element to appear in the dom, please wait..." + ); + await page.waitForSelector( + `#root > div > div > div:nth-child(${i}) > div > div:nth-child(2)` + ); + console.log(`Successfully opened chart tab ${chartTabNameText} \n`); + return true; + } catch (err) { + attempts++; + if (attempts <= 1) { + console.log( + `experienced a timeout. Trying to open tab ${chartTabNameText} again` + ); + await openTabAndWait(); + } else { + console.log( + `timeout waiting for tab ${chartTabNameText} to open. Moving on to next chart... \n` + ); + failedScreenshots.push(chartTabNameText); + return false; + } + } + }; + + const tabButtonRef = `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > svg`; + const element = await page.$(tabButtonRef); + if (element === null) continue; // element will be null if it doesn't contain the button to open the chart tab + + const chartTabNameText = await getElementText( + `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > span` + ); + + const tabOpened = await openTabAndWait(); + if (tabOpened === false) continue; + } + }; + + // store references to the elements containing the chart tabs + // also store the names of the chart tabs + const storeChartElementRefs = async (domElementCount) => { + for (let i = 1; i < domElementCount + 1; i++) { + const element = await page.$( + `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > svg` + ); + + // element will be null if it doesn't contain the svg button to open the chart tab + if (element === null) continue; + + const chartTabNameText = await getElementText( + `#root > div > div > div:nth-child(${i}) > div > div.infoRow > div:nth-child(1) > span` + ); + + // if there are duplicate tab names, append a unique id to the end of the name + if ( + chartTabGroupNames.includes(chartTabNameText) || + fs.existsSync(`./${directory}/${chartTabNameText}.png`) + ) { + chartTabNameText += `_${uuidv4()}`; + } + + chartTabGroupNames.push(chartTabNameText); + chartTabGroupElements.push( + `#root > div > div > div:nth-child(${i}) > div` + ); + } + }; + + const takeTabScreenshots = async ( + chartTabGroupElements, + chartTabGroupNames, + failedScreenshots + ) => { + for (let i = 0; i < chartTabGroupElements.length; i++) { + console.log( + `Taking screenshot: ${chartTabGroupNames[i]}, please wait...` + ); + const chartTab = await page.$(chartTabGroupElements[i]); + + // append 'failed' to image name if the chart tab failed to open + const appendToImageName = failedScreenshots.includes( + chartTabGroupNames[i] + ) + ? "-failed" + : ""; + + await chartTab.screenshot({ + path: `${directory}/${chartTabGroupNames[i]}${appendToImageName}.png`, + omitBackground: true, + }); + } + }; + + const takeHeaderScreenshot = async () => { + // screenshot the top header + const topHeader = await page.$("#root > div > div > div"); + await topHeader.screenshot({ + path: fs.existsSync(`./${directory}/top_header.png`) + ? `${directory}/top_header${uuidv4()}.png` + : `${directory}/top_header.png`, + omitBackground: true, + }); + + // screenshot the first set of charts in the top header (append unique id to name if it already exists in the directory) + const headerCharts = await page.$( + "#root > div > div > div:nth-child(2) > div" + ); + await headerCharts.screenshot({ + path: fs.existsSync(`./${directory}/header_charts.png`) + ? `${directory}/header_charts_${uuidv4()}.png` + : `${directory}/header_charts.png`, + omitBackground: true, + }); + }; + + const scrollDown = async () => { + const isEnd = await page.evaluate(() => { + window.scrollBy(0, window.innerHeight); + return window.scrollY >= document.body.clientHeight - window.innerHeight; + }); + return isEnd; + }; + + const takeFullpageScreenshot = async () => { + console.log("\nTaking full page screenshot, please wait..."); + try { + const { pagesCount, extraPixels, viewport } = await page.evaluate(() => { + window.scrollTo(0, 0); + return { + pagesCount: Math.ceil( + document.body.clientHeight / window.innerHeight + ), + extraPixels: document.body.clientHeight % window.innerHeight, + viewport: { + height: window.innerHeight, + width: window.innerWidth, + }, + clientHeight: document.body.clientHeight, + innerHeight: window.innerHeight, + }; + }); + + const images = []; + for (let i = 0; i < pagesCount; i += 1) { + const image = await page.screenshot({ + path: `./${tempImagesDirectory}/temp_${i}.png`, + }); + await scrollDown(); + images.push(image); + } + + if (pagesCount === 1) { + const image = await Jimp.read(images[0]); + image.write( + fs.existsSync(`./${directory}/full_page_screenshot.png`) + ? `${directory}/full_page_screenshot_${uuidv4()}.png` + : `${directory}/full_page_screenshot.png`, + () => { + console.log("successfully took full page screenshot \n"); + } + ); + return; + } + + // crop the last image so that we can append it to the end of the full page png + const cropped = await Jimp.read(images.pop()) + .then((image) => + image.crop( + 0, + viewport.height - extraPixels - 16, + viewport.width, + extraPixels + ) + ) + .then((image) => image.getBufferAsync(Jimp.AUTO)); + images.push(cropped); + const mergedImage = await (0, mergeImg)(images, { + direction: true, + }); + await mergedImage.write( + fs.existsSync(`./${directory}/full_page_screenshot.png`) + ? `${directory}/full_page_screenshot_${uuidv4()}.png` + : `${directory}/full_page_screenshot.png` + ); + console.log("successfully took full page screenshot \n"); + } catch (err) { + console.log( + "something went wrong taking the full page screenshot, printing error... \n " + + err + ); + } + }; + + // module flow starts here + const chartTabGroupElements = []; // stores the chart tab dom elements + const chartTabGroupNames = []; // stores the name of each chart tab + const failedScreenshots = []; // stores the names of any charts that failed to screenshot + + const domElementCount = await countElements(); + await openChartTabs(domElementCount); + await storeChartElementRefs(domElementCount); + + await takeTabScreenshots( + chartTabGroupElements, + chartTabGroupNames, + failedScreenshots + ); + await takeHeaderScreenshot(); + await takeFullpageScreenshot(); +}; + +exports.takeScreenshots = takeScreenshots;