How to deal with backup and restore points in event-sourcing?
Our tool
In our team we are working on a tool that uses event sourcing. Events are used to build (runtime) and rebuild (startup) the state of aggregates in our system. Different microservices (viewstores, repositories...) tap into the stream of events to build their state.
Problem
I am working on an issue for which a user can choose to create a backup point for one of these aggregates. At a later point a user may choose to return to this point.
Proposed solution
My idea is to create two events:
BackupCreatedEventBackupRestoredEvent
The BackupCreatedEvent will contain the identifier of the aggregate, a timestamp and a backup identifier. When triggering a restore, the state of the aggregate will be discarded. Next, a replay will happen of all events in the event store that apply to only that aggregate. The replay will stop, as soon as a BackupCreatedEvent has been hit that has matching aggregate and backup identifiers. This should bring back the aggregate in the desired state.
Whenever you restart the application and you have to rebuild all the aggregates, I intend to do the following. If during event replay I hit a BackupCreatedEvent for an aggregate X, it will ignore any subsequent event applying to X, until I hit the matching BackupRestoredEvent.
Question and considerations
When I read about event sourcing, is that you are not allowed to alter any events in your event store. For example, you should not mark them as to be ignored or to be skipped at replay time. Neither should you remove them from your event-store.
We have considered using snapshots, but we have many services (e.g. viewstores) that can only incrementally apply events and have no capability to load a snapshot of an aggregate. We would have to build that capability.
My question is mainly: Is it okay to ignore events on replay?
domain-driven-design event-sourcing
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
Our tool
In our team we are working on a tool that uses event sourcing. Events are used to build (runtime) and rebuild (startup) the state of aggregates in our system. Different microservices (viewstores, repositories...) tap into the stream of events to build their state.
Problem
I am working on an issue for which a user can choose to create a backup point for one of these aggregates. At a later point a user may choose to return to this point.
Proposed solution
My idea is to create two events:
BackupCreatedEventBackupRestoredEvent
The BackupCreatedEvent will contain the identifier of the aggregate, a timestamp and a backup identifier. When triggering a restore, the state of the aggregate will be discarded. Next, a replay will happen of all events in the event store that apply to only that aggregate. The replay will stop, as soon as a BackupCreatedEvent has been hit that has matching aggregate and backup identifiers. This should bring back the aggregate in the desired state.
Whenever you restart the application and you have to rebuild all the aggregates, I intend to do the following. If during event replay I hit a BackupCreatedEvent for an aggregate X, it will ignore any subsequent event applying to X, until I hit the matching BackupRestoredEvent.
Question and considerations
When I read about event sourcing, is that you are not allowed to alter any events in your event store. For example, you should not mark them as to be ignored or to be skipped at replay time. Neither should you remove them from your event-store.
We have considered using snapshots, but we have many services (e.g. viewstores) that can only incrementally apply events and have no capability to load a snapshot of an aggregate. We would have to build that capability.
My question is mainly: Is it okay to ignore events on replay?
domain-driven-design event-sourcing
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago
add a comment |
Our tool
In our team we are working on a tool that uses event sourcing. Events are used to build (runtime) and rebuild (startup) the state of aggregates in our system. Different microservices (viewstores, repositories...) tap into the stream of events to build their state.
Problem
I am working on an issue for which a user can choose to create a backup point for one of these aggregates. At a later point a user may choose to return to this point.
Proposed solution
My idea is to create two events:
BackupCreatedEventBackupRestoredEvent
The BackupCreatedEvent will contain the identifier of the aggregate, a timestamp and a backup identifier. When triggering a restore, the state of the aggregate will be discarded. Next, a replay will happen of all events in the event store that apply to only that aggregate. The replay will stop, as soon as a BackupCreatedEvent has been hit that has matching aggregate and backup identifiers. This should bring back the aggregate in the desired state.
Whenever you restart the application and you have to rebuild all the aggregates, I intend to do the following. If during event replay I hit a BackupCreatedEvent for an aggregate X, it will ignore any subsequent event applying to X, until I hit the matching BackupRestoredEvent.
Question and considerations
When I read about event sourcing, is that you are not allowed to alter any events in your event store. For example, you should not mark them as to be ignored or to be skipped at replay time. Neither should you remove them from your event-store.
We have considered using snapshots, but we have many services (e.g. viewstores) that can only incrementally apply events and have no capability to load a snapshot of an aggregate. We would have to build that capability.
My question is mainly: Is it okay to ignore events on replay?
domain-driven-design event-sourcing
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Our tool
In our team we are working on a tool that uses event sourcing. Events are used to build (runtime) and rebuild (startup) the state of aggregates in our system. Different microservices (viewstores, repositories...) tap into the stream of events to build their state.
Problem
I am working on an issue for which a user can choose to create a backup point for one of these aggregates. At a later point a user may choose to return to this point.
Proposed solution
My idea is to create two events:
BackupCreatedEventBackupRestoredEvent
The BackupCreatedEvent will contain the identifier of the aggregate, a timestamp and a backup identifier. When triggering a restore, the state of the aggregate will be discarded. Next, a replay will happen of all events in the event store that apply to only that aggregate. The replay will stop, as soon as a BackupCreatedEvent has been hit that has matching aggregate and backup identifiers. This should bring back the aggregate in the desired state.
Whenever you restart the application and you have to rebuild all the aggregates, I intend to do the following. If during event replay I hit a BackupCreatedEvent for an aggregate X, it will ignore any subsequent event applying to X, until I hit the matching BackupRestoredEvent.
Question and considerations
When I read about event sourcing, is that you are not allowed to alter any events in your event store. For example, you should not mark them as to be ignored or to be skipped at replay time. Neither should you remove them from your event-store.
We have considered using snapshots, but we have many services (e.g. viewstores) that can only incrementally apply events and have no capability to load a snapshot of an aggregate. We would have to build that capability.
My question is mainly: Is it okay to ignore events on replay?
domain-driven-design event-sourcing
domain-driven-design event-sourcing
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 5 hours ago
DeLaChanceDeLaChance
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 5 hours ago
DeLaChanceDeLaChance
1
asked 5 hours ago
DeLaChanceDeLaChance
1
1
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
DeLaChance is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago
add a comment |
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
DeLaChance is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54249692%2fhow-to-deal-with-backup-and-restore-points-in-event-sourcing%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
DeLaChance is a new contributor. Be nice, and check out our Code of Conduct.
DeLaChance is a new contributor. Be nice, and check out our Code of Conduct.
DeLaChance is a new contributor. Be nice, and check out our Code of Conduct.
DeLaChance is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54249692%2fhow-to-deal-with-backup-and-restore-points-in-event-sourcing%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are you backing up a single aggregate or the whole system? You can restore the whole system up to some point in time. But i don't see how you can restore a single aggregate - those events you ignore in the aggregate logic may not be ignored elsewhere, read models should be affected alredy.
– Roman Eremin
3 hours ago
I am backing up only a single aggregate. The backup should not affect other aggregates in any way.
– DeLaChance
1 hour ago