Skip to content

Commit

Permalink
MacOS with two execute stage version
Browse files Browse the repository at this point in the history
  • Loading branch information
Naibo_Mac_M2 committed Dec 23, 2023
1 parent 476cec0 commit 2fb6283
Show file tree
Hide file tree
Showing 8 changed files with 74 additions and 27 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,46 @@ Due to the complex security settings of MacOS, the issue of being unable to open

https://github.com/NaiboWang/EasySpider/wiki/MacOS-Guide

For the Arm version, if it shows "the package is damaged", you need to use the following command to modify the package attributes:
The main steps are as follows:

- Design phase - Apple Arm chip version of MacOS

1. For the Arm version, if it shows "The package is damaged", you need to do the following to run EasySpider:

2. Open the terminal command line window.

3. Switch to the EasySpider software directory, such as:

cd ~/Downloads/EasySpider_MacOS

4. In the EasySpider directory, use the following command to modify the software package attributes:

xattr -c YourEasySpider.appFilePath

xattr -cr Your EasySpider.app file path

For example:

xattr -cr /Users/your_username/Downloads/EasySpider_MacOS/EasySpider.app

Then try to open it again.
xattr -c EasySpider.app

You can now open and use the software.


- Design phase - Intel chip version of MacOS

1. Due to MacOS's security policy, the system does not allow EasySpider to run when it is first opened, and it will prompt you to move it to the trash. At this point, you need to click "Cancel".

2. Then open System Settings -> Security & Privacy.

3. Click "Open Anyway" (if you can't see it, scroll to the bottom).

Now, you can design tasks as you would in other operating systems.


- Execution phase

When executing the xattr command, if an error like the one below occurs, you can ignore it. After the execution is finished, you can open the software:
The operation is the same as the design phase of the Intel version. When running the 'easyspider_executestage' program for the first time, you need to set "Always Allow" in System Settings -> Security & Privacy, and re-run the "./easyspider_executestage EID" command, and click "Open Anyway" to run the task.

xattr: [Errno 13] Permission denied: 'EasySpider.app/Contents/Resources/app/node_modules/node-window-manager/build/node_gyp_bins/python3'

File access permissions must be granted, but microphone permissions are not needed at all. The author is also unclear why microphone access would be requested, so it can be refused.
During the execution of the task, if an error similar to the following occurs, it can be ignored:

During the execution of tasks, if an error similar to the one below occurs, it can also be ignored:
Traceback (most recent call last): File "multiprocessing/resource_tracker.py", line 209, in main KeyError: '/mp-5dxyey7c'

Traceback (most recent call last):
File "multiprocessing/resource_tracker.py", line 209, in main
KeyError: '/mp-5dxyey7c'
File access permissions must be granted, but microphone permissions are not needed at all. The author is also unclear why microphone access permissions would be requested, so you can refuse.
1 change: 1 addition & 0 deletions .temp_to_pub/EasySpider_MacOS/Sample Tasks/306.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id":306,"name":"XML Example","url":"https://www.chinanews.com.cn/rss/scroll-news.xml","links":"https://www.chinanews.com.cn/rss/scroll-news.xml","create_time":"2023-12-23 10:47:31","update_time":"2023-12-23 11:07:16","version":"0.6.0","saveThreshold":10,"quitWaitTime":60,"environment":0,"maximizeWindow":0,"maxViewLength":15,"recordLog":1,"outputFormat":"xlsx","saveName":"current_time","dataWriteMode":1,"inputExcel":"","startFromExit":0,"pauseKey":"p","containJudge":false,"browser":"chrome","removeDuplicate":0,"desc":"https://www.chinanews.com.cn/rss/scroll-news.xml","inputParameters":[{"id":0,"name":"urlList_0","nodeId":1,"nodeName":"打开网页","value":"https://www.chinanews.com.cn/rss/scroll-news.xml","desc":"要采集的网址列表,多行以\\n分开","type":"text","exampleValue":"https://www.chinanews.com.cn/rss/scroll-news.xml"}],"outputParameters":[{"id":0,"name":"自定义参数_1","desc":"","type":"text","recordASField":1,"exampleValue":"自定义值"}],"graph":[{"index":0,"id":0,"parentId":0,"type":-1,"option":0,"title":"root","sequence":[1,3],"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0},"isInLoop":false},{"id":1,"index":1,"parentId":0,"type":0,"option":1,"title":"打开网页","sequence":[],"isInLoop":false,"position":0,"parameters":{"useLoop":false,"xpath":"","wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"url":"https://www.chinanews.com.cn/rss/scroll-news.xml","links":"https://www.chinanews.com.cn/rss/scroll-news.xml","maxWaitTime":10,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"cookies":""}},{"id":3,"index":2,"parentId":2,"type":0,"option":3,"title":"提取数据","sequence":[],"isInLoop":true,"position":0,"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"clear":0,"newLine":1,"params":[{"nodeType":0,"contentType":0,"relative":true,"name":"自定义参数_1","desc":"","iframe":false,"extractType":0,"relativeXPath":"","recordASField":1,"allXPaths":[],"exampleValues":[{"num":0,"value":"自定义值"}],"default":"","beforeJS":"1","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"paraType":"text","splitLine":0}]}},{"id":2,"index":3,"parentId":0,"type":1,"option":8,"title":"循环 - 不固定元素列表","sequence":[2],"isInLoop":false,"position":1,"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"/rss/channel/item","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"loopType":1,"pathList":"","textList":"","code":"","waitTime":0,"exitCount":0,"exitElement":"//body","historyWait":2,"breakMode":0,"breakCode":"","breakCodeWaitTime":0,"skipCount":0}},{"id":-1,"index":4,"parentId":0,"type":0,"option":3,"title":"提取数据","sequence":[],"isInLoop":false,"position":1,"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"clear":0,"newLine":1,"params":[{"nodeType":0,"contentType":0,"relative":false,"name":"自定义参数_1","desc":"","iframe":false,"extractType":0,"relativeXPath":"/rss/channel/item","recordASField":1,"allXPaths":[],"exampleValues":[{"num":0,"value":"自定义值"}],"default":"","beforeJS":"","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"paraType":"text","splitLine":0}]}}]}
1 change: 1 addition & 0 deletions .temp_to_pub/EasySpider_MacOS/Sample Tasks/307.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id":307,"name":"京东全球版-专业的综合网上购物商城","url":"https://www.jd.com","links":"https://www.jd.com","create_time":"2023-12-23 11:16:19","update_time":"2023-12-23 11:32:41","version":"0.6.0","saveThreshold":10,"quitWaitTime":60,"environment":0,"maximizeWindow":0,"maxViewLength":15,"recordLog":1,"outputFormat":"xlsx","saveName":"current_time","dataWriteMode":1,"inputExcel":"","startFromExit":0,"pauseKey":"p","containJudge":false,"browser":"chrome","removeDuplicate":0,"desc":"https://www.jd.com","inputParameters":[{"id":0,"name":"urlList_0","nodeId":1,"nodeName":"打开网页","value":"https://www.jd.com","desc":"要采集的网址列表,多行以\\n分开","type":"text","exampleValue":"https://www.jd.com"},{"id":1,"name":"inputText_1","nodeName":"输入文字","nodeId":2,"desc":"要输入的文本,如京东搜索框输入:电脑","type":"text","exampleValue":"Field[\"参数\"]TTTeval(\"self.a\")TTTJS(\"return new Date().getMonth()+1\")TTTJS(\"return new Date()\")","value":"Field[\"参数\"]TTTeval(\"self.a\")TTTJS(\"return new Date().getMonth()+1\")TTTJS(\"return new Date()\")"},{"id":2,"name":"loopText_2","nodeId":4,"nodeName":"循环输入文字","desc":"要输入的文本/网址,多行以\\n分开","type":"text","exampleValue":"JS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2023\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2024\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2025","value":"JS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2023\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2024\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2025"}],"outputParameters":[{"id":0,"name":"执行Python代码","desc":"自定义操作返回的数据","type":"text","recordASField":0,"exampleValue":""},{"id":1,"name":"参数","desc":"","type":"text","recordASField":1,"exampleValue":"手机"},{"id":2,"name":"参数2_链接地址","desc":"","type":"text","recordASField":1,"exampleValue":"https://shouji.jd.com/"}],"graph":[{"index":0,"id":0,"parentId":0,"type":-1,"option":0,"title":"root","sequence":[1,3,6,2,4],"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0},"isInLoop":false},{"id":1,"index":1,"parentId":0,"type":0,"option":1,"title":"打开网页","sequence":[],"isInLoop":false,"position":0,"parameters":{"useLoop":false,"xpath":"","wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"url":"https://www.jd.com","links":"https://www.jd.com","maxWaitTime":10,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"cookies":""}},{"id":4,"index":2,"parentId":0,"type":0,"option":4,"title":"输入文字","sequence":[],"isInLoop":false,"position":3,"parameters":{"history":4,"tabIndex":-1,"useLoop":false,"xpath":"//*[@id=\"key\"]","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"value":"Field[\"参数\"]TTTeval(\"self.a\")TTTJS(\"return new Date().getMonth()+1\")TTTJS(\"return new Date()\")","index":0,"allXPaths":["/html/body/div[4]/div[1]/div[2]/div[1]/input[1]","//input[contains(., '')]","id(\"key\")","//INPUT[@class='text']","/html/body/div[last()-6]/div/div[last()-2]/div/input"]}},{"id":2,"index":3,"parentId":0,"type":0,"option":5,"title":"执行Python代码","sequence":[],"isInLoop":false,"position":1,"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"clear":0,"newLine":1,"codeMode":5,"code":"self.a = 222","waitTime":0,"recordASField":0,"paraType":"text","emailConfig":{"host":"","port":465,"username":"","password":"","from":"","to":"","subject":"","content":""}}},{"id":5,"index":4,"parentId":0,"type":1,"option":8,"title":"循环输入文字","sequence":[5],"isInLoop":false,"position":4,"parameters":{"history":3,"tabIndex":-1,"useLoop":false,"xpath":"//*[@id=\"key\"]","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"loopType":3,"pathList":"","textList":"JS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2023\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2024\nJS(\"function getCurrentMonth() {var now = new Date(); var month = now.getMonth() + 1; return month;} return getCurrentMonth()\")/2025","code":"","waitTime":0,"exitCount":0,"exitElement":"//body","historyWait":2,"breakMode":0,"breakCode":"","breakCodeWaitTime":0,"skipCount":0,"allXPaths":["/html/body/div[4]/div[1]/div[2]/div[1]/input[1]","//input[contains(., '')]","id(\"key\")","//INPUT[@class='text']","/html/body/div[last()-6]/div/div[last()-2]/div/input"]}},{"id":7,"index":5,"parentId":5,"type":0,"option":4,"title":"输入文字","sequence":[],"isInLoop":true,"position":0,"parameters":{"history":3,"tabIndex":-1,"useLoop":true,"xpath":"//*[@id=\"key\"]","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"value":"result 1JS(\"return 22\")","index":0,"allXPaths":["/html/body/div[4]/div[1]/div[2]/div[1]/input[1]","//input[contains(., '')]","id(\"key\")","//INPUT[@class='text']","/html/body/div[last()-6]/div/div[last()-2]/div/input"]}},{"id":3,"index":6,"parentId":0,"type":1,"option":8,"title":"循环采集数据","sequence":[7],"isInLoop":false,"position":2,"parameters":{"history":3,"tabIndex":-1,"useLoop":false,"xpath":"/html/body/div[5]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/a","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"loopType":1,"pathList":"","code":"","waitTime":0,"exitCount":0,"exitElement":"//body","historyWait":2,"breakMode":0,"breakCode":"","breakCodeWaitTime":0,"skipCount":0,"allXPaths":["/html/body/div[5]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/a[1]","//a[contains(., '手机')]","/html/body/div[last()-5]/div/div[last()-4]/div/div[last()-2]/div/div/div/div[last()-1]/div[last()-12]/a[last()-1]"]}},{"id":6,"index":7,"parentId":3,"type":0,"option":3,"title":"提取数据","sequence":[],"isInLoop":true,"position":0,"parameters":{"history":3,"tabIndex":-1,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"clear":0,"newLine":1,"params":[{"nodeType":1,"contentType":0,"relative":true,"name":"参数","desc":"","extractType":0,"relativeXPath":"","allXPaths":"","exampleValues":[{"num":0,"value":"手机"}],"unique_index":"tain0ppyd1rlqhi7699","iframe":false,"default":"","paraType":"text","recordASField":1,"beforeJS":"","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"splitLine":0},{"nodeType":2,"contentType":0,"relative":true,"name":"参数2_链接地址","desc":"","relativeXPath":"","allXPaths":"","exampleValues":[{"num":0,"value":"https://shouji.jd.com/"}],"unique_index":"tain0ppyd1rlqhi7699","iframe":false,"default":"","paraType":"text","recordASField":1,"beforeJS":"","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"splitLine":0}]}}]}
1 change: 1 addition & 0 deletions .temp_to_pub/EasySpider_MacOS/Sample Tasks/308.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id":308,"name":"京东全球版-专业的综合网上购物商城","url":"https://www.jd.com","links":"https://www.jd.com","create_time":"2023-12-23 14:21:24","update_time":"2023-12-23 14:23:36","version":"0.6.0","saveThreshold":10,"quitWaitTime":60,"environment":0,"maximizeWindow":0,"maxViewLength":15,"recordLog":1,"outputFormat":"xlsx","saveName":"current_time","dataWriteMode":1,"inputExcel":"","startFromExit":0,"pauseKey":"p","containJudge":false,"browser":"chrome","removeDuplicate":1,"desc":"https://www.jd.com","inputParameters":[{"id":0,"name":"urlList_0","nodeId":1,"nodeName":"打开网页","value":"https://www.jd.com","desc":"要采集的网址列表,多行以\\n分开","type":"text","exampleValue":"https://www.jd.com"}],"outputParameters":[{"id":0,"name":"参数1_链接文本","desc":"","type":"text","recordASField":1,"exampleValue":"手机"},{"id":1,"name":"参数2_链接地址","desc":"","type":"text","recordASField":1,"exampleValue":"https://shouji.jd.com/"}],"graph":[{"index":0,"id":0,"parentId":0,"type":-1,"option":0,"title":"root","sequence":[1,2],"parameters":{"history":1,"tabIndex":0,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0},"isInLoop":false},{"id":1,"index":1,"parentId":0,"type":0,"option":1,"title":"打开网页","sequence":[],"isInLoop":false,"position":0,"parameters":{"useLoop":false,"xpath":"","wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"url":"https://www.jd.com","links":"https://www.jd.com","maxWaitTime":10,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"cookies":""}},{"id":2,"index":2,"parentId":0,"type":1,"option":8,"title":"循环采集数据","sequence":[3],"isInLoop":false,"position":1,"parameters":{"history":4,"tabIndex":-1,"useLoop":false,"xpath":"/html/body/div[5]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div/a","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"scrollType":0,"scrollCount":1,"scrollWaitTime":1,"loopType":1,"pathList":"","code":"","waitTime":0,"exitCount":0,"exitElement":"//body","historyWait":2,"breakMode":0,"breakCode":"","breakCodeWaitTime":0,"skipCount":0,"allXPaths":["/html/body/div[5]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]/a[1]","//a[contains(., '手机')]","/html/body/div[last()-6]/div/div[last()-4]/div/div[last()-2]/div/div/div/div[last()-1]/div[last()-12]/a[last()-1]"]}},{"id":3,"index":3,"parentId":2,"type":0,"option":3,"title":"提取数据","sequence":[],"isInLoop":true,"position":0,"parameters":{"history":4,"tabIndex":-1,"useLoop":false,"xpath":"","iframe":false,"wait":0,"waitType":0,"beforeJS":"","beforeJSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"waitElement":"","waitElementTime":10,"waitElementIframeIndex":0,"clear":0,"newLine":1,"params":[{"nodeType":1,"contentType":8,"relative":true,"name":"参数1_链接文本","desc":"","extractType":0,"relativeXPath":"","allXPaths":"","exampleValues":[{"num":0,"value":"手机"}],"unique_index":"m5moh4pro4rlqhoa60d","iframe":false,"default":"","paraType":"text","recordASField":1,"beforeJS":"","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"splitLine":0},{"nodeType":2,"contentType":0,"relative":true,"name":"参数2_链接地址","desc":"","relativeXPath":"","allXPaths":"","exampleValues":[{"num":0,"value":"https://shouji.jd.com/"}],"unique_index":"m5moh4pro4rlqhoa60d","iframe":false,"default":"","paraType":"text","recordASField":1,"beforeJS":"","beforeJSWaitTime":0,"JS":"","JSWaitTime":0,"afterJS":"","afterJSWaitTime":0,"downloadPic":0,"splitLine":0}]}}]}
Loading

0 comments on commit 2fb6283

Please sign in to comment.