-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial implementation. Provide catalog health check and surgery for four existing issues. #1
Commits on May 23, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 1983559 - Browse repository at this point
Copy the full SHA 1983559View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2b6bea - Browse repository at this point
Copy the full SHA e2b6beaView commit details
Commits on Jul 8, 2019
-
Add catalog checkup for rid/uid and data.
Check consistency of mapping from path to rid and vice versa, also check catalog metadata for its contents.
Configuration menu - View commit details
-
Copy full SHA for c92f16d - Browse repository at this point
Copy the full SHA c92f16dView commit details -
Configuration menu - View commit details
-
Copy full SHA for f9d1ccd - Browse repository at this point
Copy the full SHA f9d1ccdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 08fba6d - Browse repository at this point
Copy the full SHA 08fba6dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a9eae33 - Browse repository at this point
Copy the full SHA a9eae33View commit details -
Configuration menu - View commit details
-
Copy full SHA for 85894cf - Browse repository at this point
Copy the full SHA 85894cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 635e963 - Browse repository at this point
Copy the full SHA 635e963View commit details -
Configuration menu - View commit details
-
Copy full SHA for d4f0f05 - Browse repository at this point
Copy the full SHA d4f0f05View commit details -
Configuration menu - View commit details
-
Copy full SHA for b00244c - Browse repository at this point
Copy the full SHA b00244cView commit details -
Configuration menu - View commit details
-
Copy full SHA for a9dae03 - Browse repository at this point
Copy the full SHA a9dae03View commit details -
Improve naming in various places.
- Split catalog module into surgery and checkup. - Use unhealthy instead of abberation - Use healthcheck instead of checkup - shorten zopectl command name
Configuration menu - View commit details
-
Copy full SHA for a380f49 - Browse repository at this point
Copy the full SHA a380f49View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8cd7185 - Browse repository at this point
Copy the full SHA 8cd7185View commit details -
Configuration menu - View commit details
-
Copy full SHA for 36fc03f - Browse repository at this point
Copy the full SHA 36fc03fView commit details -
The object that was used to catalog these entries is gone and cannot be found at path any longer.
Configuration menu - View commit details
-
Copy full SHA for 14c4310 - Browse repository at this point
Copy the full SHA 14c4310View commit details -
Configuration menu - View commit details
-
Copy full SHA for 04e96a5 - Browse repository at this point
Copy the full SHA 04e96a5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6cef3fa - Browse repository at this point
Copy the full SHA 6cef3faView commit details -
Add commit and post-surgery healthcheck.
Only commit in surgery and only if the post-surgery health check is successful. Also add '--dry-run` argument to prevent committing for a testrun.
Configuration menu - View commit details
-
Copy full SHA for 60ccfb3 - Browse repository at this point
Copy the full SHA 60ccfb3View commit details -
Add UID index length to healthcheck.
The UID index contains one UID per plone object. The UID for each object is unique, thus we know that for each object in the catalog there will also be one entry in the UID index. Thus we can use this index as a sample to check the catalog index health.
Configuration menu - View commit details
-
Copy full SHA for 130e2a9 - Browse repository at this point
Copy the full SHA 130e2a9View commit details -
Add command tests, slightly rework arguments.
Default dryrun to true for checkup. Checkup must never cause persistent changes and thus never commit.
Configuration menu - View commit details
-
Copy full SHA for 271b2f5 - Browse repository at this point
Copy the full SHA 271b2f5View commit details -
Improve surgery index removal.
Remove manually from indices as unindex_object aborts too early if the internal data is inconsistent.
Configuration menu - View commit details
-
Copy full SHA for 879b7f3 - Browse repository at this point
Copy the full SHA 879b7f3View commit details -
Rename index_data to catalog_data.
We mostly check the catalogs internal data strucuture and not the indices. I think not mentioning indices in the method name is better.
Configuration menu - View commit details
-
Copy full SHA for 5ca56ee - Browse repository at this point
Copy the full SHA 5ca56eeView commit details -
Consistently call index uuid_index.
This hopefully helps a bit to avoid confusion between the uuid_index, an index in the catalog and the catalogs internal uid attribute which contains the path->rid mapping.
Configuration menu - View commit details
-
Copy full SHA for a80d0be - Browse repository at this point
Copy the full SHA a80d0beView commit details -
Provide get_physical_path and get_rid methods.
Consistently provide these methods as helper methods on the base test class.
Configuration menu - View commit details
-
Copy full SHA for 7edcd7d - Browse repository at this point
Copy the full SHA 7edcd7dView commit details -
Add healthcheck for uuid index data.
Check rids in index and unindex against each other and agains the catalogs uid mapping (path->rid). This seems to change some of our assumptions regarding how we should update catalog index length.
Configuration menu - View commit details
-
Copy full SHA for 9be3479 - Browse repository at this point
Copy the full SHA 9be3479View commit details -
Add helper to make unhealty extra rid after move.
Creates an unhealthy catalog the same way as happens in production. Should then be fixable by surgery. Also document in what way the catalog is unhealthy and provide entry points for further analysis while developing fixes.
Configuration menu - View commit details
-
Copy full SHA for 5e45e2d - Browse repository at this point
Copy the full SHA 5e45e2dView commit details -
Configuration menu - View commit details
-
Copy full SHA for b648547 - Browse repository at this point
Copy the full SHA b648547View commit details -
Configuration menu - View commit details
-
Copy full SHA for e581946 - Browse repository at this point
Copy the full SHA e581946View commit details -
Configuration menu - View commit details
-
Copy full SHA for c4c7544 - Browse repository at this point
Copy the full SHA c4c7544View commit details -
Configuration menu - View commit details
-
Copy full SHA for 00c9df9 - Browse repository at this point
Copy the full SHA 00c9df9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a5d787 - Browse repository at this point
Copy the full SHA 4a5d787View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c470ec - Browse repository at this point
Copy the full SHA 1c470ecView commit details -
Add surgery for unindex_object call.
Some indices seem to be ablte to maintain their internal integrity thus we can use the official `unindex_object` method to remove a rid from these indices.
Configuration menu - View commit details
-
Copy full SHA for 189ce41 - Browse repository at this point
Copy the full SHA 189ce41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1850ae6 - Browse repository at this point
Copy the full SHA 1850ae6View commit details -
Configuration menu - View commit details
-
Copy full SHA for ae77525 - Browse repository at this point
Copy the full SHA ae77525View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9efc04a - Browse repository at this point
Copy the full SHA 9efc04aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d91876 - Browse repository at this point
Copy the full SHA 7d91876View commit details -
Pass a linked
Length
to decrease during removal.Allow to pass a `Length` that has to be decreased once an index row is empty. Slightly refactors coupling of `index._length` to the index which is most reliably used to determine the correct length as this varies heavily between different index implementations.
Configuration menu - View commit details
-
Copy full SHA for 5eeed1d - Browse repository at this point
Copy the full SHA 5eeed1dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 72557a3 - Browse repository at this point
Copy the full SHA 72557a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 612f06d - Browse repository at this point
Copy the full SHA 612f06dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a482db9 - Browse repository at this point
Copy the full SHA a482db9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 77dda25 - Browse repository at this point
Copy the full SHA 77dda25View commit details -
Configuration menu - View commit details
-
Copy full SHA for ff66ab7 - Browse repository at this point
Copy the full SHA ff66ab7View commit details -
Configuration menu - View commit details
-
Copy full SHA for f25ff7e - Browse repository at this point
Copy the full SHA f25ff7eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c58b2ec - Browse repository at this point
Copy the full SHA c58b2ecView commit details -
Configuration menu - View commit details
-
Copy full SHA for eb45986 - Browse repository at this point
Copy the full SHA eb45986View commit details -
Configuration menu - View commit details
-
Copy full SHA for 18008c9 - Browse repository at this point
Copy the full SHA 18008c9View commit details -
Add surgery to drop extra rid not in indexes.
Re-use existing surgery for extra rid, that is not in the UUID index, but in rid-->path mapping and catalog metadata.
Configuration menu - View commit details
-
Copy full SHA for db45c54 - Browse repository at this point
Copy the full SHA db45c54View commit details -
Process indexing queue after fixes.
Some fixes reindex objects, to guarantee an up-to date catalog before post-surgery healthcheck we process the indexing queue.
Configuration menu - View commit details
-
Copy full SHA for a897511 - Browse repository at this point
Copy the full SHA a897511View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f8bb48 - Browse repository at this point
Copy the full SHA 1f8bb48View commit details -
Configuration menu - View commit details
-
Copy full SHA for cf7e754 - Browse repository at this point
Copy the full SHA cf7e754View commit details -
Fix for acquired objects moved into their parents.
Correct special case handling for objects that were moved into one of their parent objects and thus can still be found via acquisition when their former path is used as their id was not changed. We assumed that the redirector is involved, this is not correct. Thus drop redirector check and instead make sure that the acquisition chain of the wrapped and unwrapped object is different.
Configuration menu - View commit details
-
Copy full SHA for a802308 - Browse repository at this point
Copy the full SHA a802308View commit details -
Configuration menu - View commit details
-
Copy full SHA for 381f8af - Browse repository at this point
Copy the full SHA 381f8afView commit details -
Always doom transaction for healthcheck.
Explicitly doom transaction in healthcheck command, factor out healthcheck helper function that can be called from surgery.
Configuration menu - View commit details
-
Copy full SHA for 99e5af3 - Browse repository at this point
Copy the full SHA 99e5af3View commit details -
Configuration menu - View commit details
-
Copy full SHA for f95437e - Browse repository at this point
Copy the full SHA f95437eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 894a421 - Browse repository at this point
Copy the full SHA 894a421View commit details -
Rename IndexSurgery to SurgeryStep.
Improves readability by avoiding to overload the name surgery.
Configuration menu - View commit details
-
Copy full SHA for e5ec455 - Browse repository at this point
Copy the full SHA e5ec455View commit details -
Configuration menu - View commit details
-
Copy full SHA for 83c668d - Browse repository at this point
Copy the full SHA 83c668dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ff140bd - Browse repository at this point
Copy the full SHA ff140bdView commit details -
Test surgery lookup by doctor is correct.
Also improve testing and add another assertion here and there.
Configuration menu - View commit details
-
Copy full SHA for 9f2195b - Browse repository at this point
Copy the full SHA 9f2195bView commit details