2023/05/16/from-domain-events-to-infrastructure-thinking-out-loud-about-possible-approaches-i-dont-hate/

[giscus[bot]]

2023/05/16/from-domain-events-to-infrastructure-thinking-out-loud-about-possible-approaches-i-dont-hate/

On the topics of domain-driven design and event-driven architecture, something that’s been on my mind for quite some time, is the process of handling domain events, as well as potentially translating them into integration events.

https://blog.codingmilitia.com/2023/05/16/from-domain-events-to-infrastructure-thinking-out-loud-about-possible-approaches-i-dont-hate/?utm_source=substack&utm_medium=email

[Rastusik]

Hi Joao! Thanks for the interesting article and great analysis of the event publishing problem! ;) I'm curious why exactly are you inclining to go with the approach #3. I feel like you try to publish correct integration events asap and that's why you prefer solution #3. But why is this such a big priority for you? I think that a processing lag between persisting of the aggregate and publishing integration events is completely acceptable thing, if he integration events are being processed in the same order in which the aggregate was being persisted. In this regard I would intuitively choose solution #2. I wouldn't really mind additional infrastructure hops/cost, because I think it would be neglectable and I think that it's better to have more shorter and smaller transactions for better scalability, since database is almost always the bottleneck. But as you mentioned, maybe #3 is better suited for you exact use case. It's just interesting for me to observe the difference in prioritization of which system/architecture properties are more important. Anyway, keep writing more ;)

[joaofbantunes]

Hey, thanks! Glad it was interesting 🙂

The cons I found with option 2 aren't really about lag for the integration event consumers though.
The main problem is, if the integration event carries more information than the domain one, it means we'll need to query the database for additional information. If between publishing the domain event and consuming it to transform into the integration one, the data changed, the integration event now doesn't reflect the exact state of the aggregate at the time the domain event was triggered. This might or might not be a problem, but at least it's a bit weird 😅.

There are other reasons as well, less deal breakers, but still relevant. Regarding the infrastructure cost, being negligible probably depends a bit on scale. With option 2, if we want to publish 1 million events, we're now publishing 2 million, which isn't nothing.
Besides that, there's a possible optimization opportunity we miss, to reduce load on the database. In option 3, we might be able to take advantage of having the aggregate already in memory, to use to map the domain event into the integration one. With option 2, we'll always need to query the database to get the additional required data.

But as you say, it's all a matter of prioritizing the characteristics we find more important, there's no perfect solution, just one that fits better a scenario given some constraints.

[Rastusik]

Oh, I understand now. In our company, we address the problem with queries differently. We collect data for integration events in process managers (so they basically react to events as they happen) and the PMs store current data. After completing the data collection as a reaction to the events, the whole integration event is sent away. The PMs react to specific ids, and are state machines, and usually don't process concurrent users' workloads so I think there is a really low probability of hitting the problems you mention (the integration event is sent only after receiving data from each partial event, which should theoretically have the most current data, but it still depends on the workload). Anyway, this works for us, but I believe that there might be use cases where this approach wouldn't be ideal.

Regarding the infrastructure costs with 1 vs 2 million messages, I don't thing that messaging is so expensive, but that is not the point. The main point for me is that if you have 1 million messages, that means that you have 1 million transactions in the database, which inevitably has to show as the main bottleneck. It still depends on how many of them would be run concurrently, but still. Regarding saving db performance with in memory aggregates in #3, we still can save the database by using a different physical db for the process managers.

But I still get your point and I understand that we're talking about trade offs. I think the future will show which approach makes more sense in general and the main decision factor would be how both approaches perform under high scale :)

[joaofbantunes]

Interesting approaches you're describing. Pretty cool there are so many ways to achieve the same, and there's no perfect option 🙂