Best way to update a collection of SQL data












0















I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
All help appreciated!










share|improve this question



























    0















    I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
    Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
    So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
    All help appreciated!










    share|improve this question

























      0












      0








      0








      I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
      Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
      So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
      All help appreciated!










      share|improve this question














      I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI.
      Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table.
      So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure.
      All help appreciated!







      c# sql .net entity-framework relational-database






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Jan 19 at 23:41









      camuscamus

      276




      276
























          1 Answer
          1






          active

          oldest

          votes


















          1














          So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.



          IEnumerable<MyClass> userInterfaceElements = ...
          IQueryable<MyClass> databaseElements = ...


          Note: the query is not executed yet!



          You want to update the database such, that after your update your database contains the items from your user interface elements.




          • User interface elements that are not in the database yet will be added

          • Database elements that are not in the user interface need to be removed

          • User interface elements that are also in the database need to be updated.


          You didn't write how you decide whether a user interface element is also in the database.
          Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.



          var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
          var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

          var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
          var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))


          The last one: remove all items that have an Id that is not in your user interface elements anymore.



          Note: we still have not executed any query!



          Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items



          var addList = itemsToAdd.ToList();
          var updateList = itemsToUpdate.ToList();
          var removeList = itemsToRemove.ToList();


          By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.



          dbContext.MyClasses.RemoveRange(removeList);
          dbContext.MyClasses.AddRange(addList);


          To update in entity framework, the proper method would be to fetch the data and then change the properties.



          Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.



          Proper way:



          foreach(var itemToUpdate in updateList)
          {
          var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
          // TODO: update changed properties of the fetchedItem with values from itemToUpdate
          }


          Dangerous method:



          foreach(var itemToUpdate in updateList)
          {
          dbContext.Entry(itemToUpdate).State = entityState.Modified;
          }


          Finally:



          dbContext.SaveChanges();


          Improved delete method



          You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.



          When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?



          To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.



          Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.



          If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.



          The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.






          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54272322%2fbest-way-to-update-a-collection-of-sql-data%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.



            IEnumerable<MyClass> userInterfaceElements = ...
            IQueryable<MyClass> databaseElements = ...


            Note: the query is not executed yet!



            You want to update the database such, that after your update your database contains the items from your user interface elements.




            • User interface elements that are not in the database yet will be added

            • Database elements that are not in the user interface need to be removed

            • User interface elements that are also in the database need to be updated.


            You didn't write how you decide whether a user interface element is also in the database.
            Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.



            var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
            var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

            var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
            var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))


            The last one: remove all items that have an Id that is not in your user interface elements anymore.



            Note: we still have not executed any query!



            Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items



            var addList = itemsToAdd.ToList();
            var updateList = itemsToUpdate.ToList();
            var removeList = itemsToRemove.ToList();


            By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.



            dbContext.MyClasses.RemoveRange(removeList);
            dbContext.MyClasses.AddRange(addList);


            To update in entity framework, the proper method would be to fetch the data and then change the properties.



            Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.



            Proper way:



            foreach(var itemToUpdate in updateList)
            {
            var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
            // TODO: update changed properties of the fetchedItem with values from itemToUpdate
            }


            Dangerous method:



            foreach(var itemToUpdate in updateList)
            {
            dbContext.Entry(itemToUpdate).State = entityState.Modified;
            }


            Finally:



            dbContext.SaveChanges();


            Improved delete method



            You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.



            When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?



            To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.



            Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.



            If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.



            The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.






            share|improve this answer




























              1














              So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.



              IEnumerable<MyClass> userInterfaceElements = ...
              IQueryable<MyClass> databaseElements = ...


              Note: the query is not executed yet!



              You want to update the database such, that after your update your database contains the items from your user interface elements.




              • User interface elements that are not in the database yet will be added

              • Database elements that are not in the user interface need to be removed

              • User interface elements that are also in the database need to be updated.


              You didn't write how you decide whether a user interface element is also in the database.
              Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.



              var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
              var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

              var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
              var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))


              The last one: remove all items that have an Id that is not in your user interface elements anymore.



              Note: we still have not executed any query!



              Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items



              var addList = itemsToAdd.ToList();
              var updateList = itemsToUpdate.ToList();
              var removeList = itemsToRemove.ToList();


              By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.



              dbContext.MyClasses.RemoveRange(removeList);
              dbContext.MyClasses.AddRange(addList);


              To update in entity framework, the proper method would be to fetch the data and then change the properties.



              Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.



              Proper way:



              foreach(var itemToUpdate in updateList)
              {
              var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
              // TODO: update changed properties of the fetchedItem with values from itemToUpdate
              }


              Dangerous method:



              foreach(var itemToUpdate in updateList)
              {
              dbContext.Entry(itemToUpdate).State = entityState.Modified;
              }


              Finally:



              dbContext.SaveChanges();


              Improved delete method



              You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.



              When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?



              To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.



              Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.



              If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.



              The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.






              share|improve this answer


























                1












                1








                1







                So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.



                IEnumerable<MyClass> userInterfaceElements = ...
                IQueryable<MyClass> databaseElements = ...


                Note: the query is not executed yet!



                You want to update the database such, that after your update your database contains the items from your user interface elements.




                • User interface elements that are not in the database yet will be added

                • Database elements that are not in the user interface need to be removed

                • User interface elements that are also in the database need to be updated.


                You didn't write how you decide whether a user interface element is also in the database.
                Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.



                var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
                var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

                var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
                var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))


                The last one: remove all items that have an Id that is not in your user interface elements anymore.



                Note: we still have not executed any query!



                Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items



                var addList = itemsToAdd.ToList();
                var updateList = itemsToUpdate.ToList();
                var removeList = itemsToRemove.ToList();


                By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.



                dbContext.MyClasses.RemoveRange(removeList);
                dbContext.MyClasses.AddRange(addList);


                To update in entity framework, the proper method would be to fetch the data and then change the properties.



                Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.



                Proper way:



                foreach(var itemToUpdate in updateList)
                {
                var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
                // TODO: update changed properties of the fetchedItem with values from itemToUpdate
                }


                Dangerous method:



                foreach(var itemToUpdate in updateList)
                {
                dbContext.Entry(itemToUpdate).State = entityState.Modified;
                }


                Finally:



                dbContext.SaveChanges();


                Improved delete method



                You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.



                When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?



                To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.



                Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.



                If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.



                The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.






                share|improve this answer













                So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.



                IEnumerable<MyClass> userInterfaceElements = ...
                IQueryable<MyClass> databaseElements = ...


                Note: the query is not executed yet!



                You want to update the database such, that after your update your database contains the items from your user interface elements.




                • User interface elements that are not in the database yet will be added

                • Database elements that are not in the user interface need to be removed

                • User interface elements that are also in the database need to be updated.


                You didn't write how you decide whether a user interface element is also in the database.
                Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.



                var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
                var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

                var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
                var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))


                The last one: remove all items that have an Id that is not in your user interface elements anymore.



                Note: we still have not executed any query!



                Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items



                var addList = itemsToAdd.ToList();
                var updateList = itemsToUpdate.ToList();
                var removeList = itemsToRemove.ToList();


                By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.



                dbContext.MyClasses.RemoveRange(removeList);
                dbContext.MyClasses.AddRange(addList);


                To update in entity framework, the proper method would be to fetch the data and then change the properties.



                Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.



                Proper way:



                foreach(var itemToUpdate in updateList)
                {
                var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
                // TODO: update changed properties of the fetchedItem with values from itemToUpdate
                }


                Dangerous method:



                foreach(var itemToUpdate in updateList)
                {
                dbContext.Entry(itemToUpdate).State = entityState.Modified;
                }


                Finally:



                dbContext.SaveChanges();


                Improved delete method



                You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.



                When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?



                To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.



                Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.



                If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.



                The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Jan 23 at 8:31









                Harald CoppoolseHarald Coppoolse

                12.1k12960




                12.1k12960






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54272322%2fbest-way-to-update-a-collection-of-sql-data%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Callistus III

                    Plistias Cous

                    Index Sanctorum