Hardcoded default limit of max 500 search results is not obvious

[arknu]

I just had a strange issue where searching was returning unexpected results. Eventually, it turned out that Examine has a default hardcoded limit of only returning 500 search results:

Examine/src/Examine.Core/Search/QueryOptions.cs

Line 7 in 56c8971

public const int DefaultMaxResults = 500;

I would expect that not passing in QueryOptions would simply return all the results. If I want to limit the results, I'll pass in QueryOptions as appropriate.

I find the current behavior very strange and unintuitive. There is nothing in the API design to indicate that there is a limit, I had to check the source code to find out.

[Shazwazza]

The more results you try to return from Lucene, the higher the performance hit. This goes up exponentially. It is an anti-pattern to try to return all results. If you want to keep returning results, you should use Skip/Take and keep iterating. @nzdev has also added "deep paging" https://shazwazza.github.io/Examine/articles/paging.html#deep-paging for further performance improvements for this.

If you need more results than 500, use Skip/Take options but be advised that performance will degrade the more you try to load in a query (even if there aren't that many results).