Full application prototype

README.md

@@ -1,10 +1,12 @@
-This project allows users of the RAMPART project [https://github.com/artic-network/rampart](https://github.com/artic-network/rampart) to take automatic screenshots of RAMPART chart outputs.
+This application allows users of the RAMPART project [https://github.com/artic-network/rampart](https://github.com/artic-network/rampart) to take automatic screenshots of RAMPART chart outputs using puppeteer https://github.com/puppeteer/puppeteer](https://github.com/puppeteer/puppeteer).
+
+The application will also save the table data in the RAMPART reports tab into separate .csv files.
 
 ---
 
 ## Requirements
 
-You need to have Node.js installed on your local machine, this is required to handle npm packages.
+You need to have Node.js (at least version 10.18.1) installed on your local machine, this is required to handle npm packages. Note that RAMPART also requires Node.js to be at least version 10 or above so you should be able to rely on the version of Node.js that is already running your rampart project.
 
 https://nodejs.org/en/download/
 
@@ -28,9 +30,9 @@ Once RAMPART is running, navigate to the root of this project and run:
 node app.js
 ```
 
-The application will then automatically take screenshots of the charts in your active RAMPART session. Screenshots are saved as .png files in the project root under `/images`. Depending on how many charts you have in your project, it may take some time to finish taking all of the screenshots.
+The application will then automatically take screenshots of the charts in your active RAMPART session. Screenshots are saved as .png files in the project root under `/outputs`. Depending on how many charts you have in your project, it may take some time to finish taking all of the screenshots. Any screenshots that fail (see below for possible errors) will have "-failed" appended to the filename.
 
-Once the program has started, puppeteer will open a temporary Chrome window to start taking screenshots. The window will automatically close again when the program has finished.
+The .csv table data in the RAMPART reports tab will also be saved into the `/outputs` directory.
 
 ---
 
@@ -52,8 +54,32 @@ When no arguments are passed the application will assume your RAMPART project is
 
 ---
 
-## Additional Project Notes
+## Fullpage screenshot issues
+
+There is currently a known chromium bug that is affecting the behaviour of some fullpage screenshots, see here for more information [https://github.com/puppeteer/puppeteer/issues/1576](https://github.com/puppeteer/puppeteer/issues/1576). This seems to affect pages that have a large viewport height and causes the fullpage screenshot to duplicate content.
+
+This application currently uses a workaround to take the fullpage screenshot until the above bug is fixed. This workaround uses Jimp [https://www.npmjs.com/package/jimp](https://www.npmjs.com/package/jimp) to merge several screenshots into a fullpage image.
+
+---
+
+## Timeout error
+
+When running the application, you may occasionally experience this error: "TimeoutError: waiting for selector "[selector]" failed". This could be related to a potential puppeteer bug: [https://github.com/puppeteer/puppeteer/issues/4072](https://github.com/puppeteer/puppeteer/issues/4072).
+
+When this error occurs the application will try to continue taking screenshots of the remaining charts so you will need to take the screenshot of the failed chart manually.
+
+---
+
+## Why does the application sometimes fail to screenshot every chart?
+
+When viewing your image outputs you might occasionally notice that the application did not take a screenshot of every chart, or just took a screenshot of the tab bar containing the chart (in which case the image name will have "-failed" appended). This is related to the above mentioned timeout error [https://github.com/puppeteer/puppeteer/issues/4072](https://github.com/puppeteer/puppeteer/issues/4072). If this happens, try running the application again or take a manual screenshot of the charts that were missed.
+
+Any charts that fail to produce a successful screenshot will have "-failed" appened to the image name in the outputs folder (e.g. Mayinga-failed.png) and you will therefore need to take a screenshot of this image manually.
+
+---
+
+## Compatibility
 
-This project uses puppeteer [https://github.com/puppeteer/puppeteer](https://github.com/puppeteer/puppeteer) to programatically interact with the browser.
+All features have been tested against Rampart version 1.1.0. This application relies on the structure of the Rampart page being constant, therefore future updates to Rampart may cause errors.
 
 ---

app.js

@@ -1,58 +1,60 @@
-const fs = require("fs");
-const puppeteer = require("puppeteer");
-const screenshot = require("./screenshot/screenshot");
-const saveReport = require("./screenshot/save_report");
+const rimraf = require("rimraf");
+
+const puppeteerConnect = require("./src/puppeteer_connect");
+const createDirectories = require("./src/create_directories");
+const screenshot = require("./src/screenshot");
+const saveReport = require("./src/save_report");
 
 // process arguments
 let url = process.argv[2];
 if (!url) {
 	console.log(
-		"No arguments passed to specify url. Defaulting to http://localhost:3000"
+		"No arguments passed to specify url. Defaulting to http://localhost:3000 \n"
 	);
 	url = "http://localhost:3000";
 }
 
-const puppeteerConnect = async (url) => {
-	const browser = await puppeteer.launch({
-		// headless: false,
-		// slowMo: 250,
-	});
+const runApp = async (url) => {
+	const { browser, page } = await puppeteerConnect.puppeteerConnect(url);
 
-	const page = await browser.newPage();
-	await page.setViewport({ width: 1980, height: 50000 }); // setting large height to account for case where there are many charts
+	const {
+		imagesDirectory,
+		reportsDirectory,
+		tempImagesDirectory,
+	} = await createDirectories.createDirectories();
 
+	// take screenshots of charts
 	try {
-		await page.goto(url, {
-			waitUntil: "networkidle2",
-		});
+		await screenshot.takeScreenshots(
+			page,
+			imagesDirectory,
+			tempImagesDirectory
+		);
 	} catch (err) {
 		console.log(
-			"Having trouple establishing a connection. If you are attempting to connect to localhost try entering the url as your local ip address"
+			"\x1b[36m%s\x1b[0m",
+			" \n Something went wrong taking screenshots. Printing error... \n"
 		);
 		console.log(err);
-		browser.close();
-		return;
 	}
 
-	// get the current date in YYYY-MM-DD format (for creating directories)
-	let timeStamp = Date.now();
-	let dateObject = new Date(timeStamp);
-	let date = dateObject.getDate();
-	let month = dateObject.getMonth() + 1;
-	let year = dateObject.getFullYear();
-
-	const directory = `./outputs/${
-		year + "-" + month + "-" + date + "-" + timeStamp
-	}`;
-	if (!fs.existsSync(directory)) {
-		fs.mkdirSync(directory, { recursive: true });
+	// save the information in the 'reports' tab as a .csv
+	try {
+		await saveReport.saveReport(page, reportsDirectory);
+	} catch (err) {
+		console.log(
+			"\x1b[36m%s\x1b[0m",
+			"Something went wrong saving the chart. Printing error... \n"
+		);
+		console.log(err);
 	}
 
-	await screenshot.takeScreenshots(page, directory);
-	// await saveReport.saveReport(page, directory);
-	console.log("All operations completed");
+	console.log("cleaning up temporary files and folders");
+	await rimraf(tempImagesDirectory, function () {
+		console.log("\x1b[36m%s\x1b[0m", "all operations completed");
+	});
 
 	browser.close();
 };
 
-puppeteerConnect(url);
+runApp(url);