[ID-741]Fix Case-Insensitive Sorting#742
Conversation
petyos
left a comment
There was a problem hiding this comment.
It does not fix sorting as sorting is applied here:
Stephen said, can you add a case insensitivity flag to the Core BB, applied to the Mongo query. He passed me this:
Directory Sort that needs case insensitivity applied
core-building-block/driven/storage/adapter.go
Line 1544 in 01e7e72
This reverts commit 4487c45.
|
Hi @petyos The documents are then sorted using the normalized fields. These normalized fields exist only temporarily in the pipeline and won't appear in the final output unless explicitly included in a $project stage. |
* [ID-680] Single identity within an organisation (rokwire#684) * Update CHANGELOG.md * Define accounts identifies collection * Define account identity - in progress * Define properties * Add indexes * Update CHANGELOG.md * Change to tenants accounts * Update structs * Update indexes * Migrate data - in progress * Use transaction * Verify that the transaction works * Prepare complex query * Show only the repeatable ones * Process duplicate accounts - in progress * Remove commented code * Work with struct * Organise code * Prepare founded items * Prepare founded items - in progress * Load all accounts * Load the accounts prevent missing any fields * Get full accounts objects * Get full accounts objects - part 2 * Construct tenants accounts * Segment by org - in progress * We need to load the apps orgs object from the database as we will need them * Segment by org - in progress * Segment by org * Print data * Simplify structure data - in progress * Simplify structure data - in progress * Simplify structure data * Construct tenant accounts for org - in progress * Construct tenant accounts for org - in progress * Add env variable flag * Construct tenant accounts - in progress * Fix error * Construct tenant account from account * Construct tenant accounts - in progress * Check if contains identity * Create tenant accounts * Verify that do not repeate * Add account to tenant account * Replace item * Add only the auth types which are not already in the mixed tenant account * Merge memberships - in progress * Merge memberships - in progress * More stuff * Mix tenant accounts * More stuff * Insert tenant accounts * Mark accounts as processed * Iterate only not migrated records * Check if need to apply processing * Prepare oepration for huge data - in progress * Work on the query * Work on query * Work on the query * Work on the query.. * Query is in progress * Introduce two phases * Use $merge instead of $out * Process by org * Process for every organization * More stuff * Add appropriate comments * Set appropriate log * Start the migration before the watch threads * Set long timeout * Disable storing system data * Sleep for 1 second * Remove shared identities from the docs * Generate gen_types with the latest tool version * Remove shared profile * Remove shared identities * Account to profile is one to one relation * Update model - in progress * Update model * Start using tenant accounts collection step by step * Use app org id from the params * Mark storage functions as deprecated * Define accountFromStorage for tenant account type * Tenant accounts in progress * Optimise code * Work with memberships - in progress * Do not use pointer * Construct memberships * Set current membership * Register account - in progress * Construct account * Mark function as deprecated * Insert account * Use tenants accounts * Use tenants.. * More readable code * We must handle accounts on org level * Check if it is "sign-in" or "org-sign-up" or "app-sign-up" - in progress * Handle different operation - in progress * app-sign-up operation is not supported * Set current app org id * External auth - in progress * Various fixes * Use tenant accounts * Fix credential when external * More APIs - in progress * More APIs - in progress * More APIs - in progress * Fix update profile API * Fix update privacy API * Fix get preferences API * Update preferences is in progress * Fix update preferences * Disable this for now * Fix mocks * Anonymous accounts - in progress * Use v2 function * Create anonymous account * Fix get system configs API * Fix update username API * Fix add follow API * Fix get mfa types API * Fix get public accounts API * Fix get public accounts API - part 2 * Fix find accounts API - in progress * Fix find accounts API - in progress * Fix find accounts API * Disable for now * Fix update account permission * Fix update account roles * Fix update account groups * Revert comment * Fix mfa types * Fix delete mfa type * Fix scopes * Fix refresh tokens API * Fix can sign in API * Use tenants accounts * Use tenants accounts * Link issues * Link issues - part 2 * Link issues - part 3 * Fix get account * Use tenants accounts * Use tenants accounts * Fix system configs * Fix system configs - part 2 * Use tenants accounts * Fix search params * Fix counts API. * Groups in progress * Add accounts to group * Remove accounts from group * Fix create group * Fix update group * Use tenant accoutns * Fix devices * in progress * Use account v2 * Fix grant permissions to account * in progress * Fix revoke permissions * Fix grant roles * Fix revoke roles * Use tenants accounts * More issues * set "code" * Fix mocks * secrets * Fix groups * Not used and also no more relevant as one account has a profile * docs * in progress * secrets * add Apps to the docs in the PartialAccount * add Apps to the conversion * secrets * change Apps to App * update gen version * Do not make changes in go.mod and go.sum * Use v1.16.2 tool * Fix broken files - remove the two files with the same names - part 1 * Fix broken files - part 2 * Fix broken files - part 3 arh.. * Fix broken files - part 4 * Fix other broken files - part 1 * Fix other broken files - part 2 * Regenerate * App is renamed to PartialApp * Account has apps but not app * add Apps do docs * add apps in the structure * add apps to DeleteAccount * Fix build * API in progress * API in progress * API in progress * Prepare API * Delete Membership * secrets * fix * fix comments * fix * secrets * fix * secrets * Give better name * Fix lint issue * Fix more lint issues * Disable verified as not used * Remove unused deprecated functions * Fix verified * Verified is omitempty --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> Co-authored-by: Stefan Vitanov <stefan@inabyte.com> * Fix docker build * Prepare v1.35.0 * Fix storeSystemData (rokwire#686) * check if collection exists - functionality in progress * create auth type collection function * set the Changelog.md * fix lint issues * set the mocks * resolve comments --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> * [ID-687] Handle Create Admin account API issue (rokwire#688) * Update CHANGELOG.md * Fix is in progress * Do not pass parameter which is not used * Check if the account exists for this app * Determine operation * App sign up is in progress * App sign up is in progress * Return account * Once we have tenants accounts this means that phone is also valid * In progress * API is done --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.35.1 * [ID-690] Accounts to tenants account migration improvement (rokwire#691) * Update CHANGELOG.md * Split in separate function * In progress * Reuse function * Process by pieces * Revert test code --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.36.0 * [ID-692] Migration issue (rokwire#693) * Update CHANGELOG.md * User orgs_accounts * Use migrated 2 * We care only org account * External user could be nil for some reasons, prevent nil pointer --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.37.0 * [ID-694] Fix nil pointer (rokwire#695) * Update CHANGELOG.md * Fix nil pointer * Update CHANGELOG.md --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.37.1 * fix some build errors * [ID-698] Clean up accounts migration legacy code (rokwire#699) * Update CHANGELOG.md * Remove migration * Remove more stuff --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.37.2 * fix remaining build errors * remove UIUC specific pieces from migration * [ID-700]Update filter accounts API to accept many UINs (rokwire#701) * convert string into slice and set the docs * fix the docs * set the Changelog.md * fix * Generate docs --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * migration updates * remove phase2 from tenant account migration, change external_ids to use identifiers in account search API examples * use preferences on org signup external, remove unused function params in auth * more unused function param cleanup, use tenantAccounts coll instead of accounts coll * [ID-702]Expose Create multiple accounts admin API (rokwire#703) * add Changelog.md * Define the new API in the adapter * Docs in progress * Mark the old one as deprecated * Docs are done * creating the struct and the object for the porpose of the API * structure * in progress * add permission to authorisation policy * secrets * fix the name and the Changelog * fix the function name * fix the comments of the CreateAccounts function * change the name of the struct * at the transaction for the whole function * one transaction for all operations * secrets * fix lint issue * secrets * fix --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.38.0 * batch credential collection migration, need to batch accounts and login sessions * batch account identifiers migration * [ID-704]fix v2 create accounts (rokwire#705) * fix the API so all the accounts could be saved * set the Changelog.md * fix * secrets --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> * Prepare v1.38.1 * fix account and credential migration batching bugs * update go version to 1.22 * update changelog, merge duplicate any tenant account app memberships during auth type migration * fix bbs, tps, and admin account search by params * make sure account identifiers are not returned in account search if requestor is not scoped for identifiers * fix Account, PartialAccount schema validation bugs * fix FindAccount bug when attempting operationAppSignUp * batch tenant account migration * change migrated field back to migrated_2 for tenant account migration * remove auth type migration * [ID-710] Admin app account creation update (rokwire#711) * Update CHANGELOG.md * Mock external login for quick development * Has admin app access - in progress * Has admin app access - in progress * Implement hasAdminAppAccess function * Remove TODO comment * Revert commented code * In progress * Add membership - in progress * More stuff * Revert commented mock * Update to go1.22 --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.39.0 * [ID-712]Add "admin_app_access_roles" to the system POST and PUT app_orgs (rokwire#713) * add to Changelog.md * add to the docs but nit generated * Gen doc & types * in progress * fix * resolve comments (must generate docs) * go sum * change workflows build to1.22.3 * fix * resolve comments --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Gen docs & types * Fix build * fot fix * [ID-335] Remove account from the system - all BBs (rokwire#354) * add service accounts * add refresh service account token handler * start updating token management * finish updating service account endpoints * start adding system endpoints * updates to service account system endpoints * more updates to service account system endpoints * finish service account system endpoints * update changelog * add signature auth from auth library * continue adding signature auth for service account tokens * register service account and add service cred bug fixes * signature auth bug fixes * bugfixes * flag accounts for deletion, add get deleted accounts endpoint * updated changelog * add delete accounts timer * updated docs, service account storage model, implemented missing storage functions * add optional search params for get service accounts * bugfixes and improvements * remove system endpoints used for testing * don't delete accounts on get removed accounts * add delete accounts period env var, add bbs permissions auth * use profile pointer in account * improve get deleted accounts handler * removed commented code * added service account id to ServiceReg and AuthServiceReg * upgrade auth lib to 1.0.5 * upgrade auth lib to 1.0.5 * added bbs access-token endpoint * reverted account profile to use struct, updated docs * get deleted accounts by app_org_id * use all in tokens for null appIDs, orgIDs * updates for upcoming core-auth-library update [rokwire#335] * update service accounts and tokens to use all string [rokwire#335] * fix bbs permissions policy * fix secrets baseline * update go version, upgrade dependencies * change deleted flags to date_deleted timestamps, change BBs GET deleted-accounts API response to partial memberships * refactor bbs/deleted-accounts to bbs/deleted-memberships, improve response schema * add DeletedMembershipContext to send additional data to each service about which data to delete per deleted membership * bug fixes * upgrade dependencies * move deleted memberships to separate collection * bug fixes * add get_deleted-memberships bbs auth policy description * Do not fail if a context is not passed * Service id is not used * Filter by time - in progress * Filter by time - in progress * Fix mock * Filter by time - done * Do not delete the removed memberships for now - we allow filtering by date --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * update service to 1.41.0 * [ID-718] Provide external ids on get deleted memberships BBs API (rokwire#719) * Update CHANGELOG.md * Add external ids field * Give appropriate name * Give appropriate name - part 2 * Remove unused field * Revert app id * Define delete account API entity - docs * Use the new entity on the delete account API. * Use the new entity * Now we can remove the unnecessary field * Fix validation * Add external ids field * Fix nullable * Return external ids * Set external ids * Fix github build --------- Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * Prepare v1.42.0 * [ID-724]Get the account ids with FERPA filed true (rokwire#725) * set the Changelog.md * set the API docs * in progress * set the ids in the query * in progress * fix farpa field type * get farpa ids if the farpa field is false. If its true it wont be shown * set the mocks * fix * fix lint issue * update the go version * Update .secrets.baseline * secrets * secrets * secrets * resolve comments in progress * resolve the comments * mocks * secrets * Update secrets baseline * fix * secrets fix * fix secret * FIX * It is many but not one * Keep them separated * Add index * Put it in the function * That's wrong * Fix ferpa handling * Add ferpa to equals * Do not return empty * Add field * Update ferpa field --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> * update version to 1.43.0 * update secrets baseline using v1.5.0 * [rokwire#727] Set up privacy settings for account directory data visibility (rokwire#728) * add visibility settings check function * update API documentation for /accounts/public and /account/privacy, fix lint issues * update Privacy and PrivacyNullable schemas * finish integration of visibility status checking into GET /accounts/public handler * add field visibility update to PUT /account/privacy * GET /accounts/public bug fixes * add new profile fields to API models and internal model * add GET/account/private API) * revert Account.GetPublicProfile back to using reflect, implement visibility checking for unstructured properties, add missing fields to PublicProfile * update identity BB profile data parsing, add IdentityBBProfileFields to ApplicationOrganization model * update changelog * Prepare v1.44.0 * [rokwire#729] Fix Privacy.FieldVisibility validation (rokwire#730) allow null values in Privacy.FieldVisibility * update version to 1.44.1 * [rokwire#734] Public account filtering by unstructured properties (rokwire#735) * add public accounts filtering by unstructured properties * update changelog * sort public accounts by last name, first name, use slices.SortFunc instead of sort.Slice * update version to 1.45.0 * [rokwire#733] GET services/accounts returns unsorted response (rokwire#737) * fix public account sorting bug - move sorting to storage adapter * optimize public accounts pipeline match stages, remove tenantAccounts collection profile index for sorting (unused) * update version to 1.45.1 * [ID-741]Fix Case-Insensitive Sorting (rokwire#742) * add the Changelog.md * fix sorting * sort * Revert "sort" This reverts commit 4487c45. * fix sort * in progress * fix * remove empty rows --------- Co-authored-by: Stefan Vitanov <stefan@inabyte.com> * update the version to 1.45.2 --------- Co-authored-by: Petyo Stoyanov <petyo.st.stoyanov@gmail.com> Co-authored-by: Petyo Stoyanov <petyo.stoyanov@inabit.eu> Co-authored-by: Stefan Vitanov <stefan@inabyte.com> Co-authored-by: stefanvitanov <75947010+stefanvit@users.noreply.github.com>
Description
Fix Case-Insensitive Sorting
**Resolves #741
Review Time Estimate
Type of changes
Checklist: