"Unstable" Release Discussion

[kubilus1]

Discussion for the latest 'unstable' cut. This release is automatically cut from the main branch and is a good place to checkout pre-released features and fixes in a pre-packaged way.

As the name implied there could be issues, but feedback is welcome here!

https://github.com/kubilus1/autoortho/releases/tag/unstable

[marloscaida]

Kubilus1, I am going to offer you a humble contribution that I hope will be useful to you. Not as a criticism since AO seems to me to be an immense contribution to the improvement of XP, but with the aim of improving it.

In principle, I didn't notice an improvement in performance (I can't say there isn't one) but I think there is a lot of information that is not useful to the user and takes up processing time. I am referring to the LOGS tab (in fact the integration of the LOGs seemed like a very good success to me).

There is an accumulation of information that is not useful for the user and that could either be eliminated or summarized, presenting the data in a clearer and more practical way. For example:

INFO:aostats:STATS: ID: {'cur_mem_mb': 1218, 'chunk_miss': 16097, 'req_ok': 11810, 'bytes_dl': 139071632, 'tile_mem_miss': 6206, 'tile_mem_hits': 13759, 'count': 212981 , 'chunk_hit': 210030, 'partial_mm_averages': {0: 0.0}, 'partial_mm_counts': {0: 7263}, 'mm_counts': {0: 175, 1: 633, 2: 681, 3: 1419, 4: 8065}, 'mm_averages': {0: 0.041, 1: 0.019, 2: 0.005, 3: 0.002, 4: 0.001}, 'retry_chunk_count': 1781, 'backup_chunk_count': 8850, 'minutes_to_start': 0.0}

A decoding of this. Something that was more understandable. Should I change the font of my OrthoPhotos? What do I do with the files that are in the image cache if I change the fonts? Is something going on with your internet access speed? Anyway, something that allows me to evaluate AO's behavior and be able to initiate some corrective action.
Printing logs can consume processor time and from what I see, repetition is almost a constant. Even if I fly the same route. It seems that AO can't do anything more in the visual improvement of the terrain.

I insist, this is not a criticism, but rather an attempt to make the interface more user-friendly and take better advantage of the processor's resources.
I hope you find this small contribution useful. Since I think his contribution to the x-plane community is HUGE.

Thank you very much for your brilliant work and the new UNSTABLE version is very nice.

Best regards
carlos

[kubilus1]

Those stats are already made into nice graphs for you, just point your browser to localhost:5000.

I would argue though that the information shown is summarized and vital to what is going on with the service.

[kubilus1]

And thanks for testing out the pre-release of this!

[marloscaida]

Kubilus, I think that language differences do not help in this discussion (I find myself needing to use a translator, since my English is very poor).

I'm going to try to be clearer because I get the impression that my point was not understood.

The truth is that the only thing that interests me about AO is the notable improvement it introduces to XP. For this, performance is essential, especially for those who have problems flying the simulator and using AO.

What I'm saying is that, to the user (my case) 'cur_mem_mb': 1218, 'chunk_miss': 16097, it doesn't tell me anything. I don't know what to do with this information. This is just an example.

I guess for the development part, this is vital to know what's going on. As a developer, I would also like to have a detailed and useful log to see what is happening on the user side. How much is the responsibility of the software and how much is the responsibility of the human being who is operating it.

What I propose is not to build beautiful statistical graphs. They don't interest me as a user. If they are available, I will surely be able to see them sometime, but I don't think I will do it very often. They do not contribute to the objective of flying XP in an environment as real as AO provides me.

I propose not to generate a LOG (at the user's discretion, as an option that can be turned on or off) to avoid intensive use of the processor.

And I also propose that you guide the user to improve the experience, with some data that you can collect and decipher.

For example, the method that @chezhj taught me was tremendously useful to make the most of AO.

In fact, I use it on each of my flights, but I see hundreds of users "lost in the desert" without knowing how to make this fantastic software work correctly.

I would like AO to tell me: you have lost a lot of "chunks" using Bing, why don't you try with "xx" which can benefit you more due to your proximity to the servers... (it's just a stupid example to convey the perspective from the user). Useful information from the user's point of view, not the programmer's.

What does 'retry_chunk_count': 1781 mean? is good or bad? What should I do to increase or decrease that value? What if I can't do anything, why show it to me? Do you understand?

I know that our profession would be a true paradise if we didn't have to deal with two terrible things: hardware and users...

Especially that clumsy mass that is between the keyboard and the chair.

I insist. This is not a criticism. It is the attempt to provide an opinion to improve the product.

Do you know why, we programmers, if we are given the choice between the cabin where the passengers go on an airplane and the cabin where the pilots are, we always choose the latter?

Because it is very difficult for us to side with the user who is our main recipient.

Because we like everything complicated, not simple and pleasant.

Users will always choose a comfortable passenger cabin, with attentive and courteous staff, to spend the flight as best as possible.

A big hug, as we Latinos say.

Regards
carlos

[kubilus1]

I do truly appreciate the feedback. What needs to be understood, though:

• 'retry_chunk_count' may or may not mean something should or shouldn't be tweaked. It really depends on many other things, but it is useful, and arguably necessary information to track. I could see in some situations raising some warnings over thresholds, but when that is appropriate really depends. It may be worse to say "Hey you might want to tweak this one value" when that may not apply in all situations.
• The amount of logs shown with debug OFF is very low, so it's no real impact on the system. With debug ON is another story, but that shouldn't be used normally.
• I'd still consider this project fairly new/green, so I'm not too concerned if some things aren't the easiest. The easiest things would be to use default scenery with Xplane, or use MSFS 2020. The issue is there are many variables outside the scope of what this project can control that yet effect user results. I definitely see that many many issues would be resolved by users fully reading the existing documentation and FAQ. Many more issues would be resolved by using Linux ;)

[rayw45]

"The amount of logs shown with debug OFF is very low, so it's no real impact on the system. With debug ON is another story, but that shouldn't be used normally."

So as I have asked elsewhere (see the other thread asking this question ), why do I have to enable this debug for my Windows 10, XP12, AO 6.1 install to work at all (using wither Dokan or WinFSP) without an unable to run error and crash inXP12 ? Many users are reporting this so surely the cause must be apparent to you the developers. So once again I ask when will this be fixed? I also praise this AO system for XP12 as it is a huge plus, but I do need to be able to switch the very time consuming debug option to False. It never ever was needed to enable debug in AO 4.4

[Jeff2122]

I switched back and forth between v0.5.1-3 and this untamed release. I still have misplaced tiles with this version, not so much (almost never) any ugly tiles with v0.5. I have set my maxwait = 4.0 and I'm asking for ZL14 with Bing.
Note that I am not flying the same route. I have been traversing across the U.S. and so all tiles are downloaded fresh.

[kubilus1]

ZL14 with Bing

How so are you doing that? Are you making your own tiles with Ortho4XP? If you are just adjusting the min_zoom setting, you really shouldn't bump that much if at all. That can definitely cause exponentially more stuff to be pulled in. Likely major cause of lag and issues on your part.

I'll say though that increasing maxwait really at all (beyond default 0.5) indicates some kind of networking issue on your end. Possibly you have some kind of scanning, firewall, etc, or poor wifi/internet connection. Look in the FAQ about how you may want to set Windows Defender scanning exceptions for key directories. Defender is AGRESSIVE and many many small time projects are impacted.

This may not be your issue, but perhaps clearing out your config and starting with defaults is probably good on occasion.

The main change of this last update is the removal of the CMD window and therefore I retain control of shutdown and can put dirs back in place cleanly.

[kubilus1]

In comparison, even on my low end test Windows 10 laptop (i7600u, external AMD5600 CPU, 80Mbps internet) I have to drop maxwait down to like 0.2 to even get much in the way of missed/green tiles and can fly around with time speed up (alt-t) set to 16 times.

On my linux machine, it's all even faster.

[Jeff2122]

No, I am relying only on AO, no custom ortho at all.
Ok, I will set the ZL12 and maxwait back to 0.5 and try it again.

[Jeff2122]

I set maxwait = 0.5 and ZL=12 and tried v0.6.unstable first. The top screenie shows multiple repeated tiles that disappear if you zoom in and out.
Flushed the cache, and then with v0.5.1-3 I had none of that, but I did have one green plain tile in sight. When I turned around to look again, that tile it was gone. (By the time I took this snapshot.)

Not sure if any of this is important, but it is a data point if nothing else.

Side note: I have always set my maxwait to 3.0 but today, with 0.5, I did see longer loading times with both versions. I will leave it there for now to see if there is anything there - my guess is if it waits longer, it doesn't have to retry so much thus shortening loading times. Could that be possible?

[Jeff2122]

I noted top right corner of second shot has plain green tiles, I suppose if I waited and turned around a few times, they would disappear too.
Nothing's perfect.

[kubilus1]

I'm not really worried about occasional green tiles, that's just a missed fetch. That tile repeating thing I haven't been able to reproduce yet.

[kubilus1]

Could you send me a STATS log line example from when you get a lot of repeating tiles, and another when you don't get the repeating tiles? That would be helpful. Thanks!!

[Jeff2122]

A STATS log line example. I will be happy to, please tell me where to find this example. What is STATS?

[Jeff2122]

I'd like to add for the record that I do see some ZL shift with v0.5.1-3 but only at high altitudes. The distant tiles (20nm+) shift to low res and back again when you go higher and lower. I am not flying to do this, just in external view zooming up and down with SHF+arrow keys.
The difference being that with v0.6.x this happens with close tiles too. Never with v0.5.1-3. What can cause this effect?

[Jeff2122]

I am getting the same results each time I try the new version. Different locations produce the same results. v0.5.1-3 isn't a problem for me.
I am not an ortho expert. How do these zoom layers work anyway? Not sure how else to explain that the scenery looks like carp with the new version.

[rayw45]

Yes all fine but I still cannot get an answer as to the necessity to enable Debug for many people. Surely the developers must know why this is happening, likely due to some needed delay in the running of the code???. Can I expect this to be fixed soon?? Why no response to my posts here which have been polite and simply trying to help both myself and many others who seem to have this problem which was not present in 4.4. I have systematically spent hours disabled plugins etc and nothing will prevent the need for debug = True. So I ask again with a big PLEASE can somebody in the know here help with this??

[jonaseberle]

@rayw45 Let's just say that this project has a wide userbase from very different backgrounds. I think we mostly ignore what is not interesting, especially if it totally misses the topic. The topic "debug mode hides deeper root problems for some Windows users" is in fact an interesting tale but totally not actionable and the developer(s) don't know what causes it.

95% of "problem reports" are utter useless to the devs and just noise for those that try to actively follow. We (those that care about the project and are not just consumers) could probably improve the docs and counter misinformation that is being spread more but we are just swamped by the volume of posts. Also we are on version 0.x – things are moving fast.

It would contribute more to fix the problem to use the latest Git version, run it with Python, make it fail and poke at the logs and the code to get an idea what could cause it. If you open a proper issue and make other users with the same problem aware of it and do some structured crowd trouble shooting maybe it could become actionable. I suggest to move the real problems there and open new discussion forums here or on the dedicated .org forum (for support questions).

Please don't expect any help and take it as what it is. I am sorry that I have derailed the topic now. But it can't get any worse than what it already is. Shoutout to Jeff who is so actively helping by giving support and testing!

[jonaseberle]

@Jeff2122

How do these zoom layers work anyway?

I'll try:
AutoOrtho is a file system driver.
When X-Plane tries to access a texture (.dds), AutoOrtho produces that file on the fly and delivers it (just what a file system driver has to do).
By the part of the .dds file that X-Plane tries to read AutoOrtho tries to produce the fitting zoom level ("MipMap" in .dds). That is because producing the whole texture at the highest zoom level would be costly so it tries to avoid that.
If anything fails (imagery server throttles requests, imagery is not available, ...) or takes too long (the setting max_wait), AutoOrtho assembles a .dds based on what it has: That can for example be a zoomed up version of a lower ZL that gets delivered to X-Plane as a higher ZL (<- these are the plaid tiles) – or (in the worst case) the "green" fallback textures.

[rayw45]

I appreciate your response jonaseberie, many thanks. However, I have absolutely no idea how to run it in Python on my PC with XP12. I have played with Pythion on my CoreELEC KODI 20 TV streaming box and know a little of Python coding, but not used it on my Windows 10 PC. I am otherwise an ordinary XP12 user who runs it in Windows 10, which as a power user I keep properly managed and maintained in good condition. Looks then like I need to just keep running AO 6.1 with the long loading delays of debug = True for now and hope the cause is eventually traced.

If there is anything straight forward I can do to help then let me know, but was hoping it would be an easy symptom to fault trace with it being just a needed delay. All I needed to know was that so far the cause is unknown by the developers as you have now stated, but this answer had not been forthcoming so far and is an important issue suffered by many with no apparent reason.

I love AO and look forward to these various problems, and indeed this debug one, being fixed with future versions. I applaud the development team on this superb project so please do not take this as a complaint or a moan.

[Jeff2122]

@jonaseberle I appreciate you for giving that explanation. It is basically as I thought. It is giving what it can.
I don't want to push anyone's button, but I think a big question still remains - after looking at the big picture, a change was made between v0.5.1-3 and v0.6.1 that causes plaid tiles, and the same thing is going on with the latest unstable version. Is it possible to isolate that particular change and determine why it is doing this and/or is it a worthwhile move forward considering this side-effect? It doesn't please the eye to me, but I guess some don't have this issue, as I have seen that some people are very happy with the latest version v0.6.1.

[marloscaida]

ATTENTION: Please verify that in the Custom Scenery folder there are no folders created by AO that repeat folders of the type z_ao_xx (where xx represents the region).

For example: z_ao_na_bk or similar folder.

This causes XP to detect them and add them to the first line of the scenery_pack.ini file, so you CANNOT see your custom or third-party airports.

Something went wrong and AO "duplicated" the symlinks. There should only be one AO folder per region.

This happened to me today and it took me a while to realize what was wrong.

Regards
carlos

[kubilus1]

That's as intended. If there are duplicate folders that are not obviously created by AO (marked with a file flag) then I have the program move and rename them, since I don't want to go and delete a bunch of stuff. That will be up to the user to decide.

[kubilus1]

I just dropped an update to the 'unstable' release that should fix tile duplication. @Jeff2122 and @jonaseberle I think you both had experienced this previously, want to take a look?

[jonaseberle]

Yes, that has fixed it for me. Thanks!

[Jeff2122]

No more repeating tiles here either, just the plaid tiles remain.

[Jeff2122]

EDIT: This pic is due to the mistake made by me using the wrong version - the new version does indeed fix the repeating tiles.
It doesn't take long - I loaded a new airport not previously cached and this is what I see when I exit the cockpit. I am running the source version, didn't try the exec.

[kubilus1]

I noticed that the unstable version won't start unless there is a valid .autoortho file in place. If there is none, the program creates one at startup but does not present itself to the user to enter the path values. If you manually edit the paths, the program will start properly.

@Jeff2122 Thanks for checking that out. Can you explain this further? I just tried removing the ~/.autoortho file and starting and the program starts right up. Of course it needs configure, but that's to be expected. Are you showing something different?

[kubilus1]

Why don't I see any plaid tiles with v0.5.1-3? Maybe I'm not understanding something?

Hard to say, but there are some fundamental differences. I know that the older version has a much longer wait time so it's possible there are duplicate tiles, just hidden more there.

[Jeff2122]

Well, you remember I made a boo boo somewhere with the source code so I got the exec this time and installed it yesterday, using the Windows Installer. When I renamed my v0.5 .autoortho file for preservation, the program would not appear when I clicked on it (running as admin). I checked and the newly created .autoortho was there so tried to open it again. Clicking does nothing and each time the exec would not open. I then edited the file to give it proper paths in my text editor, and it opens normally. Magic.
I just now retraced the steps above to see if I'm full of anything smelly, and the same thing happened. It will not open. Edit the new file and it works fine. Strange.

[Jeff2122]

I uninstalled and reinstalled the exe the same way as yesterday. First startup log below.

INFO:main:Setup logs: C:\Users\geoff.autoortho-data\logs, level: 20
INFO:autoortho:AutoOrtho version: 2023.10.08.18.37.53
INFO:aostats:Creating stats object
WARNING:autoortho:No installed sceneries detected. Exiting.

I now edit the paths and it starts fine. Note that I installed as administrator both times, and also run the exec as admin always.

[Jeff2122]

So I tried the source version again. I managed to figure out how to get a fresh copy of the repo (that was my mistake yesterday) and it works just the same way as above. I don't understand why but it is very repeatable just by deleting/renaming the .autoortho file.

[kubilus1]

Are you trying to run the program in headless mode (-H switch)? I wouldn't expect that to work if not configured.

…

On Tue, Oct 10, 2023 at 2:33 PM Jeff2122 ***@***.***> wrote: So I tried the source version again. I managed to figure out how to get a fresh copy of the repo (that was my mistake yesterday) and it works just the same way as above. I don't understand why but it is very repeatable just by deleting/renaming the .autoortho file. — Reply to this email directly, view it on GitHub <#399 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVEDLC7EK2WKFFX625FKJTX6WIGXAVCNFSM6AAAAAA5LIL25GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TENBUHA3TK> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[kubilus1]

@Jeff2122 the info you provide is very valuable, concise, and specific. It's a useful and welcome contribution. Much appreciated!

[kubilus1]

I was able to replicate this. Should be fixed with #411 once that is in.

[kubilus1]

Should be good to go now.

[Jeff2122]

Thanks for your diligence. I will grab the latest and keep on playing.
Got it. Works fine now. Nice job.

[kubilus1]

Awesome thanks for verifying!

[Jeff2122]

What happens starting in versions v0.6.0 and v0.6.1 is the plaid tiles at any airport I load, but usually clears up if I reload the same airport twice. This is probably sits fine for those who cache their scenery and fly the same routes, but I never fly the same routes. I make my way around different countries as much as possible so the issue shows up all the time for me. I have my maxwait set for 5 seconds just for fun. Doesn't change anything that I can tell. I tried 0.5 and 3.0 too.

[Jeff2122]

Well, that's fine in your eyes, not mine. Yes, Windows 10. Defender is not interfering, it has an exception for all my folders and the AO exec.
Anyway v0.5.1-3 does not have this issue. Dokan or WinFSP, no difference. I must be the only one that doesn't like it, so I'll stay with the older version.
By the way, I set up v0.7.0 and it works fine. Just has the plaid tiles.

[rayw45]

Just to add that I have carefully ensured Defender real time scanning is disabled which it already was and I have tried temporarily totally disabling Kaspersky Protection for 15 minutes whilst I tested AO with debug = False. Still the same unable to run XP12 error message when it is reading new scenery and crashes back to desktop. All my AO folders are added to Kaspersky, and the disabled Defender, exclusions too so my problem I am now sure it not caused by Defender nor Kaspersky and I cannot so far see any other A/V protection running anywhere else, though not ruling it out but unlikely.

Cannot find new version 0.7 in the download site page ??

[jonaseberle]

Jeff's log file shows that something has been downloaded but not the individual requests (I don't know currently how to make them show up). Since the max_wait is high, the only reason for plaid/upscaled textures should be failed downloads.

[jonaseberle]

@rayw45 you again hijacked a thread. It is really hard to have a conversation like that.

[rayw45]

Sorry not intended I was answering kubilus1 question about disabling Defender. Sorry if I misunderstood,