Best way to update a collection of SQL data
I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
All help appreciated!
c# sql .net entity-framework relational-database
add a comment |
I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
All help appreciated!
c# sql .net entity-framework relational-database
add a comment |
I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
All help appreciated!
c# sql .net entity-framework relational-database
I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
All help appreciated!
c# sql .net entity-framework relational-database
c# sql .net entity-framework relational-database
asked Jan 19 at 23:41
camuscamus
276
276
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.
IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...
Note: the query is not executed yet!
You want to update the database such, that after your update your database contains the items from your user interface elements.
- User interface elements that are not in the database yet will be added
- Database elements that are not in the user interface need to be removed
- User interface elements that are also in the database need to be updated.
You didn't write how you decide whether a user interface element is also in the database.
Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.
var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);
var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))
The last one: remove all items that have an Id that is not in your user interface elements anymore.
Note: we still have not executed any query!
Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items
var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();
By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.
dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);
To update in entity framework, the proper method would be to fetch the data and then change the properties.
Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.
Proper way:
foreach(var itemToUpdate in updateList)
{
var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
// TODO: update changed properties of the fetchedItem with values from itemToUpdate
}
Dangerous method:
foreach(var itemToUpdate in updateList)
{
dbContext.Entry(itemToUpdate).State = entityState.Modified;
}
Finally:
dbContext.SaveChanges();
Improved delete method
You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.
When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?
To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.
Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.
If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.
The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54272322%2fbest-way-to-update-a-collection-of-sql-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.
IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...
Note: the query is not executed yet!
You want to update the database such, that after your update your database contains the items from your user interface elements.
- User interface elements that are not in the database yet will be added
- Database elements that are not in the user interface need to be removed
- User interface elements that are also in the database need to be updated.
You didn't write how you decide whether a user interface element is also in the database.
Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.
var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);
var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))
The last one: remove all items that have an Id that is not in your user interface elements anymore.
Note: we still have not executed any query!
Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items
var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();
By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.
dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);
To update in entity framework, the proper method would be to fetch the data and then change the properties.
Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.
Proper way:
foreach(var itemToUpdate in updateList)
{
var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
// TODO: update changed properties of the fetchedItem with values from itemToUpdate
}
Dangerous method:
foreach(var itemToUpdate in updateList)
{
dbContext.Entry(itemToUpdate).State = entityState.Modified;
}
Finally:
dbContext.SaveChanges();
Improved delete method
You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.
When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?
To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.
Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.
If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.
The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.
add a comment |
So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.
IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...
Note: the query is not executed yet!
You want to update the database such, that after your update your database contains the items from your user interface elements.
- User interface elements that are not in the database yet will be added
- Database elements that are not in the user interface need to be removed
- User interface elements that are also in the database need to be updated.
You didn't write how you decide whether a user interface element is also in the database.
Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.
var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);
var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))
The last one: remove all items that have an Id that is not in your user interface elements anymore.
Note: we still have not executed any query!
Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items
var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();
By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.
dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);
To update in entity framework, the proper method would be to fetch the data and then change the properties.
Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.
Proper way:
foreach(var itemToUpdate in updateList)
{
var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
// TODO: update changed properties of the fetchedItem with values from itemToUpdate
}
Dangerous method:
foreach(var itemToUpdate in updateList)
{
dbContext.Entry(itemToUpdate).State = entityState.Modified;
}
Finally:
dbContext.SaveChanges();
Improved delete method
You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.
When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?
To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.
Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.
If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.
The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.
add a comment |
So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.
IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...
Note: the query is not executed yet!
You want to update the database such, that after your update your database contains the items from your user interface elements.
- User interface elements that are not in the database yet will be added
- Database elements that are not in the user interface need to be removed
- User interface elements that are also in the database need to be updated.
You didn't write how you decide whether a user interface element is also in the database.
Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.
var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);
var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))
The last one: remove all items that have an Id that is not in your user interface elements anymore.
Note: we still have not executed any query!
Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items
var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();
By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.
dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);
To update in entity framework, the proper method would be to fetch the data and then change the properties.
Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.
Proper way:
foreach(var itemToUpdate in updateList)
{
var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
// TODO: update changed properties of the fetchedItem with values from itemToUpdate
}
Dangerous method:
foreach(var itemToUpdate in updateList)
{
dbContext.Entry(itemToUpdate).State = entityState.Modified;
}
Finally:
dbContext.SaveChanges();
Improved delete method
You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.
When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?
To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.
Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.
If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.
The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.
So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.
IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...
Note: the query is not executed yet!
You want to update the database such, that after your update your database contains the items from your user interface elements.
- User interface elements that are not in the database yet will be added
- Database elements that are not in the user interface need to be removed
- User interface elements that are also in the database need to be updated.
You didn't write how you decide whether a user interface element is also in the database.
Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.
var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);
var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))
The last one: remove all items that have an Id that is not in your user interface elements anymore.
Note: we still have not executed any query!
Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items
var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();
By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.
dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);
To update in entity framework, the proper method would be to fetch the data and then change the properties.
Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.
Proper way:
foreach(var itemToUpdate in updateList)
{
var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
// TODO: update changed properties of the fetchedItem with values from itemToUpdate
}
Dangerous method:
foreach(var itemToUpdate in updateList)
{
dbContext.Entry(itemToUpdate).State = entityState.Modified;
}
Finally:
dbContext.SaveChanges();
Improved delete method
You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.
When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?
To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.
Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.
If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.
The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.
answered Jan 23 at 8:31
Harald CoppoolseHarald Coppoolse
12.1k12960
12.1k12960
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54272322%2fbest-way-to-update-a-collection-of-sql-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown