Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Janusupgrade to 1.0.0 with a smart marker logic to reduce Cassandra Calls #3849

Open
wants to merge 54 commits into
base: master
Choose a base branch
from
Open

Janusupgrade to 1.0.0 with a smart marker logic to reduce Cassandra Calls #3849

wants to merge 54 commits into from

Conversation

sriram-atlan
Copy link

Change description

Changes in effort to reduce indexsearch endpoint slowness.

  • Upgrade janusgraph to 1.0.0
  • Add a marker logic to lookup Cassandra only when property exist
  • Marker logic does not include IN ward relations to vertex
  • Uses Janus graph prefetch with properties

Type of change

  • Bug fix (fixes an issue)
  • New feature (adds functionality)

Related issues

https://atlanhq.atlassian.net/browse/PLT-2809

Fix #1

Checklists

Development

  • Lint rules pass locally
  • Application changes have been tested thoroughly
  • Automated tests covering modified code pass

Security

  • Security impact of change has been considered
  • Code follows company security practices and guidelines

Code review

  • Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
  • "Ready for review" label attached and reviewers assigned
  • Changes have been reviewed by at least one other contributor
  • Pull request linked to task tracker where applicable

sriram-atlan and others added 30 commits November 19, 2024 08:14
@sriram-atlan @aarshi0301
PLT-2568 optimise vertex fetch from janusgraph
364c4aa
@aarshi0301
remove method
8c7ca54
@aarshi0301
Fix property name
3df42f5
@aarshi0301
add null cehck
0bb62fc
@aarshi0301
remove submit call
9b966a6
@aarshi0301
handle cocurrent modification exception
b0530b3
@aarshi0301
update libraries
52e645d
@aarshi0301
remove scrub results
c323ad7
@aarshi0301
fix results
9741f59
@aarshi0301
add dummy commit
1545164
@aarshi0301
Merge pull request #3767 from atlanhq/dummy
8259f67 
add dummy commit
@aarshi0301
add branch
953bce5
@aarshi0301
add janus branch
c6c284b
@aarshi0301
scrub results
d27ea4d
@aarshi0301
add flag for janusgraph optimisation
0015ef7
@aarshi0301
add flag while retrieving from cassandra
54103c8
@aarshi0301
log ES query error
b903c7b
@aarshi0301
fix map vertex method
dc77657
@aarshi0301
add configuration in flag
ec4a32d
@aarshi0301
fix return type
f51f8f6
@aarshi0301
add flag for collapsed result
391e510
@aarshi0301
fix updated by and createdby field
8751387
@aarshi0301
fix if condition
6f1549d
@aarshi0301
remove branch deployment
7feef65
@sriram-atlan
use multi query to improve response time of post elastic query lookups
bd1ddb0
@sriram-atlan
Merge remote-tracking branch 'origin/janusoptimisation' into janusopt…
0bc01d6 
…imisation

# Conflicts:
#	repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
@aarshi0301
import function
650cfd1
@sriram-atlan
remove the mapVertex method that uses valueMap
f20e859
@sriram-atlan
build this branch as docker image
0022dae
@sriram-atlan
close thread bound janusgraph transaction
1be3658
sriram-atlan and others added 17 commits November 21, 2024 23:57
@sriram-atlan
revert entity audit listener
d73d858
@sriram-atlan
multi query with janusgraph 1.0.0
0af18e1
@sriram-atlan
add attributes part of multi query
4e39d43
@sriram-atlan
include janusoptimisation-multiQuery-1.0.0 in ecr build push
9c5a643
@sriram-atlan
include janusoptimisation-1.0.0 in ecr build push
01e5b3b
@sriram-atlan
include janusupgrade in ecr build push
ee86e6e
@sriram-atlan
bulk fetch with auto batching
e042b4f
@sriram-atlan
use properties method with janusgraph than gremlin
9ee5645
@sriram-atlan
use edge fetch always
c8578e4
@sriram-atlan
log edge and edge properties
a47bdee
@sriram-atlan
lookup only when marker exists or when attribute relationship is inwards
4dd2743
@sriram-atlan
include all policy attributes in prefetch exclude
76fe1c8
@sriram-atlan
revert using prefetch in mapSystemAttributes and toAtlasToObjectId
358a7bc
@sriram-atlan
remove logs. update comments
5c6ad1b
@aarshi0301
fix business attribute
b95c0e6
@aarshi0301
add check for business attributes
4cb985a
@aarshi0301
code cleanup
8eef377
@aarshi0301 aarshi0301 changed the base branch from beta to master December 11, 2024 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sriram-atlan @aarshi0301