Updated scrape?

[billdenney]

Hi @dpastoor, I was looking for a way to gather the whole DDMoRe repository for some testing with nlmixr2, and I came across this. Any chance that the scripts for the scraping still exist and that they could be rerun?

[dpastoor]

Original scrape was with https://github.com/dpastoor/ddmore_scraping/blob/master/index.js but I also feel like I rewrote something similar with rvest a couple years ago/later sitting in an rmd... Maybe in like the bbi repo or the like.

I can take a look

[billdenney]

How would I run that? (I don't usually do stuff with javascript code.)

[billdenney]

I ran it on Linux. It created the directories but didn't populate them with files.

sudo apt install node npm
npm install nightmare
npm audit fix --force
git clone [email protected]:dpastoor/ddmore_scraping.git
cd ddmore_scraping
node index.js

It printed out many lines like this:

$ node index.js
http://repository.ddmore.eu/model/DDMODEL00000234#Files
http://repository.ddmore.eu/model/DDMODEL00000235#Files
http://repository.ddmore.eu/model/DDMODEL00000236#Files
http://repository.ddmore.eu/model/DDMODEL00000237#Files

[dpastoor]

sorry for the late reply - I wonder if the ddmore website has changed its markup some. I have seen some stuff breaking/go missing in the last year or two so I unfortunately wouldn't be surprised if this doesn't work exactly now.

I can try to take a look soon, nudge me if I don't respond :-)

[liangqiongyue]

Now the ddmore website is indeed no longer available, I have to sigh that you are really prescient and thank you for keeping the resources on here! By the way, in addition to ddmore resources on github here now, are there any other resources that have not been uploaded? I would be grateful if you could take the time to update the resources.

[shanmdphd]

I am also grateful. It's a pity that ddmore website is down for now.

[zyfff3]

thanks a lot , you have save these model.