feat: add tracking logs by andrey-canon · Pull Request #173 · open-craft/openedx-completion-aggregator

andrey-canon · 2023-12-04T22:10:43Z

Description:
This changes allow to emit events when an aggregator is created or updated, and basically I'm proposing this to integrate those new events with the event-routing-backends library, which takes advantages of the tracking events to publish that data to a Xapi or caliper service. The default behavior that I'm proposing has the following events

Progress events, those will be emitted when the block has not reached the 100% of completion

edx.completion_aggregator.progress.course
edx.completion_aggregator.progress.chapter
edx.completion_aggregator.progress.sequential
edx.completion_aggregator.progress.vertical

Completion events, those will be emitted when the block has reached the 100% of completion

edx.completion_aggregator.completion.course
edx.completion_aggregator.completion.chapter
edx.completion_aggregator.completion.sequential
edx.completion_aggregator.completion.vertical

Testing instructions:

Create a section with a subsection and a unit
Complete the unit
Check server logs

Reviewers:

tag reviewer @Agrendalath

Merge checklist:

Post merge:

Create a tag
Check new version is pushed to PyPI after tag-triggered build is
finished.
Delete working branch (if not needed anymore)

Author concerns: List any concerns about this PR - inelegant
solutions, hacks, quick-and-dirty implementations, concerns about
migrations, etc.

Agrendalath

Thank you for preparing this PR, @andrey-canon!

Agrendalath · 2023-12-05T14:39:26Z

test_settings.py

 ]
 USE_TZ = True

+EVENT_TRACKING_ENABLED = True


This variable is unused. Should we use it to gate this feature?

This is used by the eventtracking library, in the test scenario I'm implementing the default django_tracker that is added by this AppConfig, that uses this method to register the tracker, after checking the DJANGO_ENABLED_SETTING_NAME which is EVENT_TRACKING_ENABLED as you can see in this line

Agrendalath · 2023-12-05T14:54:21Z

completion_aggregator/models.py

+            tracker.emit(
+                event_name,
+                {
+                    "user_id": obj.user_id,
+                    "course_id": str(obj.course_key),
+                    "block_id": str(obj.block_key),
+                    "modified": obj.modified,
+                    "created": obj.created,
+                    "earned": obj.earned,
+                    "possible": obj.possible,
+                    "percent": obj.percent,
+                    "type": event_type,
+                }
+            )


What is the cost of this operation (especially when we use multiple event-tracking backends)? Is it possible to emit multiple events in bulk?

I want to be particularly cautious about this, as we had faced a partial outage before due to the high volume of additional operations.

After running a basic test in my local I got the following

As you can see the time is in the order of 0.002 seconds, that test was performed with the following EVENT_TRACKING_BACKENDS value

{'tracking_logs': {'ENGINE': 'eventtracking.backends.routing.RoutingBackend', 'OPTIONS': {'backends': {'logger': {'ENGINE': 'eventtracking.backends.logger.LoggerBackend', 'OPTIONS': {'name': 'tracking', 'max_event_size': 50000}}}, 'processors': [{'ENGINE': 'common.djangoapps.track.shim.LegacyFieldMappingProcessor'}, {'ENGINE': 'common.djangoapps.track.shim.PrefixedEventProcessor'}]}}, 'segmentio': {'ENGINE': 'eventtracking.backends.routing.RoutingBackend', 'OPTIONS': {'backends': {'segment': {'ENGINE': 'eventtracking.backends.segment.SegmentBackend'}}, 'processors': [{'ENGINE': 'eventtracking.processors.whitelist.NameWhitelistProcessor', 'OPTIONS': {'whitelist': []}}, {'ENGINE': 'common.djangoapps.track.shim.GoogleAnalyticsProcessor'}]}}, 'xapi': {'ENGINE': 'eventtracking.backends.async_routing.AsyncRoutingBackend', 'OPTIONS': {'backend_name': 'xapi', 'processors': [{'ENGINE': 'eventtracking.processors.whitelist.NameWhitelistProcessor', 'OPTIONS': {'whitelist': ['edx.problem.completed', 'showanswer', 'edx.completion.block_completion.changed', 'edx.grades.problem.submitted', 'edx.course.enrollment.activated', 'edx.course.enrollment.deactivated', 'edx.course.grade.passed.first_time', 'edx.course.grade.now_passed', 'edx.course.grade.now_failed', 'load_video', 'edx.video.loaded', 'play_video', 'edx.video.played', 'stop_video', 'edx.video.stopped', 'complete_video', 'edx.video.completed', 'pause_video', 'edx.video.paused']}}], 'backends': {'xapi': {'ENGINE': 'event_routing_backends.backends.events_router.EventsRouter', 'OPTIONS': {'processors': [{'ENGINE': 'event_routing_backends.processors.xapi.transformer_processor.XApiProcessor', 'OPTIONS': {}}], 'backend_name': 'xapi'}}}}}, 'caliper': {'ENGINE': 'eventtracking.backends.async_routing.AsyncRoutingBackend', 'OPTIONS': {'backend_name': 'caliper', 'processors': [{'ENGINE': 'eventtracking.processors.whitelist.NameWhitelistProcessor', 'OPTIONS': {'whitelist': ['edx.course.enrollment.activated', 'edx.course.enrollment.deactivated', 'edx.ui.lms.link_clicked', 'edx.ui.lms.sequence.outline.selected', 'edx.ui.lms.outline.selected', 'edx.ui.lms.sequence.next_selected', 'edx.ui.lms.sequence.previous_selected', 'edx.ui.lms.sequence.tab_selected', 'showanswer', 'edx.problem.hint.demandhint_displayed', 'problem_check', 'load_video', 'edx.video.loaded', 'play_video', 'edx.video.played', 'complete_video', 'edx.video.completed', 'stop_video', 'edx.video.stopped', 'pause_video', 'edx.video.paused', 'seek_video', 'edx.video.position.changed', 'edx.course.grade.passed.first_time', 'edx.course.grade.now_passed', 'edx.course.grade.now_failed']}}], 'backends': {'caliper': {'ENGINE': 'event_routing_backends.backends.events_router.EventsRouter', 'OPTIONS': {'processors': [{'ENGINE': 'event_routing_backends.processors.caliper.transformer_processor.CaliperProcessor', 'OPTIONS': {}}, {'ENGINE': 'event_routing_backends.processors.caliper.envelope_processor.CaliperEnvelopeProcessor', 'OPTIONS': {'sensor_id': 'https://localhost:18000'}}], 'backend_name': 'caliper'}}}}}}

If I'm not wrong tracking_logs and segmentio comes by default, xapi and caliper are added by the event-routing-backends library.

Finally I'm not aware of a bulk emit capability, I couldn't find anything related to that in the eventtracking library and all the implementation that I found are using the single emit method

Agrendalath · 2023-12-05T15:21:57Z

completion_aggregator/settings/common.py

+        "progress": [
+            "course",
+            "chapter",
+            "sequential",
+            "vertical",
+        ],
+        "completion": [
+            "course",
+            "chapter",
+            "sequential",
+            "vertical",
+        ]


Suggested change

"progress": [

"course",

"chapter",

"sequential",

"vertical",

],

"completion": [

"course",

"chapter",

"sequential",

"vertical",

]

"progress": {

"course",

"chapter",

"sequential",

"vertical",

},

"completion": {

"course",

"chapter",

"sequential",

"vertical",

}

Agrendalath · 2023-12-05T15:29:59Z

completion_aggregator/models.py

+            event = "progress" if obj.percent < 1 else "completion"
+            event_type = obj.aggregation_name
+
+            if event_type not in settings.ALLOWED_COMPLETION_AGGREGATOR_EVENT_TYPES.get(event, []):


Suggested change

if event_type not in settings.ALLOWED_COMPLETION_AGGREGATOR_EVENT_TYPES.get(event, []):

if event_type not in settings.ALLOWED_COMPLETION_AGGREGATOR_EVENT_TYPES.get(event, {}):

Agrendalath · 2023-12-07T22:39:23Z

@andrey-canon, I will review and test this PR next week.

feat: add tracking logs

ead8f61

andrey-canon force-pushed the and/add_tracking_logs branch from d0f60e3 to ead8f61 Compare December 4, 2023 22:14

Agrendalath reviewed Dec 5, 2023

View reviewed changes

feat: addressing pr comments

a82984e

andrey-canon requested a review from Agrendalath December 5, 2023 18:10

This was referenced Dec 5, 2023

feat: add tracking logs nelc/openedx-completion-aggregator#1

Open

feat: add aggregator events openedx/event-routing-backends#377

Closed

feat: add aggregator events nelc/event-routing-backends#6

Closed

This was referenced Jun 3, 2024

Plugin: add aggregated completion events openedx/openedx-aspects#222

Open

Add tracking logs nelc/openedx-completion-aggregator#2

Closed

pomegranited mentioned this pull request Jun 13, 2024

Add tracking logs #206

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add tracking logs#173

feat: add tracking logs#173
andrey-canon wants to merge 2 commits intoopen-craft:masterfrom
nelc:and/add_tracking_logs

andrey-canon commented Dec 4, 2023 •

edited

Loading

Uh oh!

Agrendalath left a comment

Uh oh!

Agrendalath Dec 5, 2023

Uh oh!

andrey-canon Dec 5, 2023

Uh oh!

Agrendalath Dec 5, 2023

Uh oh!

andrey-canon Dec 5, 2023

Uh oh!

Agrendalath Dec 5, 2023

Uh oh!

Agrendalath Dec 5, 2023

Uh oh!

Agrendalath commented Dec 7, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	if event_type not in settings.ALLOWED_COMPLETION_AGGREGATOR_EVENT_TYPES.get(event, []):
	if event_type not in settings.ALLOWED_COMPLETION_AGGREGATOR_EVENT_TYPES.get(event, {}):

Conversation

andrey-canon commented Dec 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Agrendalath left a comment

Choose a reason for hiding this comment

Uh oh!

Agrendalath Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

andrey-canon Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

Agrendalath Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

andrey-canon Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

Agrendalath Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

Agrendalath Dec 5, 2023

Choose a reason for hiding this comment

Uh oh!

Agrendalath commented Dec 7, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrey-canon commented Dec 4, 2023 •

edited

Loading