Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LS Metrics statistic text files outputs do not exported (big rasters). #5

Open
rafaguimaraess opened this issue Nov 12, 2018 · 6 comments

Comments

@rafaguimaraess
Copy link

Hello,

I have been working with Cerrado biome and the enhanced LS Metrics (new version) for some landscape metrics (patch size, functional connectivity). The problem is: the rasters outputs were exported to the folder, but the statistic text files not. I would like to know what it is probably happening because the LS Metrics keeps going on processing even after the rasters maps were exported.

Obs.: Raster input size (columns and rows): 66977, 85490

Thanks for the package. It has been useful mainly with big landscapes!

Best wishes,

Rafaela

@rafaguimaraess rafaguimaraess changed the title LS Metrics statistic text files output doesn't exported (big rasters). LS Metrics statistic text files outputs do not exported (big rasters). Nov 12, 2018
@bniebuhr
Copy link
Contributor

Hi @rafaguimaraess
Thanks for your report. This behavior is strange, since it seems that this functionality runs for small maps (or even for other large maps we have used as input). But I'll test that with the dataset you sent me and try to find out what is happening.

Best
Bernardo

@JohnWRRC
Copy link
Contributor

JohnWRRC commented Nov 12, 2018 via email

@bniebuhr
Copy link
Contributor

Hello, Bernardo, the site size is very large 66k lines by 80k columns. This takes time to process. Rafaela, what is the solution you are working on? Have you tried to process a smaller sample with your own test data?

John, I talked to Rafaela and, in principle, there is no limitation in GRASS or LSMetrics for calculating metrics for such a map (although it takes time). Therefore, I think the issue is not exactly the size.

"I'd like to know what's probably happening because LS Metrics continue to process even after the rasters maps have been exported" Can you specify this better? It may not have reached the end of the process and so TXT is not in the folder.

That's a good hypothesis. Rafaela, are you sure the process got to an end (did the user interface tell you that the "calculations have finished")?

I would like to receive the data too
Rafela sent me that in private, I'll forward it to you, John.

The strange thing is that the process works for smaller regions or maps, even in her computer.
But it would be good to test that here and there.

As a temporary solution to her problem, she ran r.stats after processing the maps; however, our problem is not solved yet.

B

@rafaguimaraess
Copy link
Author

Hello John and Bernardo,

After the LS Metrics download (corrected version), I started to test the package choosing a metric once a time (patch size). The maps outputs had been exported in two hours while text files hadn't exported in 7 days. So, I stopped the process (before the message "calculations have finished") because I just decided to test the package with a small landscape and got it well (the maps and text files exported).
It seems as the LS Metrics are processing more slowly to export text files than rasters maps. This "behavior" is so strange for me because I expect the opposite.
The function "r.stats" which I tested with the maps outputs (patch size) worked well and fastly (15minutes).

Which email can I share the input?

Thanks for help and recommendations.

Best

Rafaela

PC features:
Intel Core I7 2.4GHz, 8 GB RAM, 1TB HD, GeForce 2GB (dedicated).

@bniebuhr
Copy link
Contributor

Hi @rafaguimaraess

How have you been?
I guess you already solved this problem, but I think I found out the reason why the text file export was not working. We created a function to calculate areas in hectares, based on r.stats, but I just found out the way it saves the information in the text file is completely inefficient. For maps with small number of patches this is not an issue, but for large extent maps such as yours, this is a problem.

r.stats, in turn, is much more efficient in writing the output file.

I'll find out some way of dealing with that and I keep you (and other users) informed here.

Best,
Bernardo

@rafaguimaraess
Copy link
Author

Hi @bniebuhr,

I'm glad to know about the solution. I solved the problems with your suggestion of using r.stats. Thanks for your help.

Best,

Rafaela

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants