Most efficient container for uint16_t to uint16_t map












2















I am developing program for very processing power limited machine where I want to map uint16_t keys to uint16_t values.



I am currently using std::map use non-safe reading:



std::map<uint16_t, uint16_t> m;
//fill m only once

while(true){
auto x = m[y];
}


The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?



Edit:
Some information:




  • Total number of items is less than 500

  • Insertion is done only once

  • Querying values is done more than 250 times / sec

  • Keys and values are unique

  • Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB

  • 100 MHz single core processor










share|improve this question




















  • 3





    Have you tried std::unordered_map?

    – NathanOliver
    Jan 18 at 17:35








  • 2





    I think more information about the task at hand and restrictions would help.

    – Philipp
    Jan 18 at 17:35






  • 2





    Which constraints are required for the container?

    – JVApen
    Jan 18 at 17:35






  • 11





    uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

    – Remy Lebeau
    Jan 18 at 17:36








  • 4





    How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

    – Zan Lynx
    Jan 18 at 17:40


















2















I am developing program for very processing power limited machine where I want to map uint16_t keys to uint16_t values.



I am currently using std::map use non-safe reading:



std::map<uint16_t, uint16_t> m;
//fill m only once

while(true){
auto x = m[y];
}


The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?



Edit:
Some information:




  • Total number of items is less than 500

  • Insertion is done only once

  • Querying values is done more than 250 times / sec

  • Keys and values are unique

  • Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB

  • 100 MHz single core processor










share|improve this question




















  • 3





    Have you tried std::unordered_map?

    – NathanOliver
    Jan 18 at 17:35








  • 2





    I think more information about the task at hand and restrictions would help.

    – Philipp
    Jan 18 at 17:35






  • 2





    Which constraints are required for the container?

    – JVApen
    Jan 18 at 17:35






  • 11





    uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

    – Remy Lebeau
    Jan 18 at 17:36








  • 4





    How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

    – Zan Lynx
    Jan 18 at 17:40
















2












2








2








I am developing program for very processing power limited machine where I want to map uint16_t keys to uint16_t values.



I am currently using std::map use non-safe reading:



std::map<uint16_t, uint16_t> m;
//fill m only once

while(true){
auto x = m[y];
}


The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?



Edit:
Some information:




  • Total number of items is less than 500

  • Insertion is done only once

  • Querying values is done more than 250 times / sec

  • Keys and values are unique

  • Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB

  • 100 MHz single core processor










share|improve this question
















I am developing program for very processing power limited machine where I want to map uint16_t keys to uint16_t values.



I am currently using std::map use non-safe reading:



std::map<uint16_t, uint16_t> m;
//fill m only once

while(true){
auto x = m[y];
}


The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?



Edit:
Some information:




  • Total number of items is less than 500

  • Insertion is done only once

  • Querying values is done more than 250 times / sec

  • Keys and values are unique

  • Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB

  • 100 MHz single core processor







c++ performance dictionary






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 18 at 18:08







Humam Helfawi

















asked Jan 18 at 17:33









Humam HelfawiHumam Helfawi

11.7k63598




11.7k63598








  • 3





    Have you tried std::unordered_map?

    – NathanOliver
    Jan 18 at 17:35








  • 2





    I think more information about the task at hand and restrictions would help.

    – Philipp
    Jan 18 at 17:35






  • 2





    Which constraints are required for the container?

    – JVApen
    Jan 18 at 17:35






  • 11





    uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

    – Remy Lebeau
    Jan 18 at 17:36








  • 4





    How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

    – Zan Lynx
    Jan 18 at 17:40
















  • 3





    Have you tried std::unordered_map?

    – NathanOliver
    Jan 18 at 17:35








  • 2





    I think more information about the task at hand and restrictions would help.

    – Philipp
    Jan 18 at 17:35






  • 2





    Which constraints are required for the container?

    – JVApen
    Jan 18 at 17:35






  • 11





    uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

    – Remy Lebeau
    Jan 18 at 17:36








  • 4





    How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

    – Zan Lynx
    Jan 18 at 17:40










3




3





Have you tried std::unordered_map?

– NathanOliver
Jan 18 at 17:35







Have you tried std::unordered_map?

– NathanOliver
Jan 18 at 17:35






2




2





I think more information about the task at hand and restrictions would help.

– Philipp
Jan 18 at 17:35





I think more information about the task at hand and restrictions would help.

– Philipp
Jan 18 at 17:35




2




2





Which constraints are required for the container?

– JVApen
Jan 18 at 17:35





Which constraints are required for the container?

– JVApen
Jan 18 at 17:35




11




11





uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

– Remy Lebeau
Jan 18 at 17:36







uint16_t can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y]; Such an array would take up only 128K of memory at most.

– Remy Lebeau
Jan 18 at 17:36






4




4





How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

– Zan Lynx
Jan 18 at 17:40







How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.

– Zan Lynx
Jan 18 at 17:40














3 Answers
3






active

oldest

votes


















3














Without some more context about your map,



if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.



if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.



if you're using not a lot of data and require it to be flexible, std::vector is a good choice



Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.






share|improve this answer








New contributor




Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




























    3
















    • Total number of items is less than 500

    • Insertion is done only once




    Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.



    Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).






    share|improve this answer































      1














      Usually a binary tree (std::map which you currently use) provides adequate performance. Your CPU budget must be really small.



      A binary search approach probably won't be much faster. A hashtable (std::unordered_map) might be a solution.



      Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.



      std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
      m.emplace(123, 234);
      m.emplace(2345, 345);

      std::cout << m.find(123)->second; // don't use operator to avoid creating entries
      std::cout << m.find(2345)->second;


      To check the quality of the hashing function, iterate over all the buckets by calling bucket_size() and add up anything above 1 (i.e. a collision).






      share|improve this answer























        Your Answer






        StackExchange.ifUsing("editor", function () {
        StackExchange.using("externalEditor", function () {
        StackExchange.using("snippets", function () {
        StackExchange.snippets.init();
        });
        });
        }, "code-snippets");

        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "1"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54258881%2fmost-efficient-container-for-uint16-t-to-uint16-t-map%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        3














        Without some more context about your map,



        if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.



        if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.



        if you're using not a lot of data and require it to be flexible, std::vector is a good choice



        Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.






        share|improve this answer








        New contributor




        Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.

























          3














          Without some more context about your map,



          if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.



          if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.



          if you're using not a lot of data and require it to be flexible, std::vector is a good choice



          Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.






          share|improve this answer








          New contributor




          Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
          Check out our Code of Conduct.























            3












            3








            3







            Without some more context about your map,



            if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.



            if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.



            if you're using not a lot of data and require it to be flexible, std::vector is a good choice



            Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.






            share|improve this answer








            New contributor




            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.










            Without some more context about your map,



            if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.



            if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.



            if you're using not a lot of data and require it to be flexible, std::vector is a good choice



            Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.







            share|improve this answer








            New contributor




            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.









            share|improve this answer



            share|improve this answer






            New contributor




            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.









            answered Jan 18 at 17:45









            DanielDaniel

            373114




            373114




            New contributor




            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.





            New contributor





            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.






            Daniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
            Check out our Code of Conduct.

























                3
















                • Total number of items is less than 500

                • Insertion is done only once




                Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.



                Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).






                share|improve this answer




























                  3
















                  • Total number of items is less than 500

                  • Insertion is done only once




                  Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.



                  Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).






                  share|improve this answer


























                    3












                    3








                    3









                    • Total number of items is less than 500

                    • Insertion is done only once




                    Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.



                    Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).






                    share|improve this answer















                    • Total number of items is less than 500

                    • Insertion is done only once




                    Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.



                    Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Jan 18 at 17:54









                    Matteo ItaliaMatteo Italia

                    100k15142239




                    100k15142239























                        1














                        Usually a binary tree (std::map which you currently use) provides adequate performance. Your CPU budget must be really small.



                        A binary search approach probably won't be much faster. A hashtable (std::unordered_map) might be a solution.



                        Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.



                        std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
                        m.emplace(123, 234);
                        m.emplace(2345, 345);

                        std::cout << m.find(123)->second; // don't use operator to avoid creating entries
                        std::cout << m.find(2345)->second;


                        To check the quality of the hashing function, iterate over all the buckets by calling bucket_size() and add up anything above 1 (i.e. a collision).






                        share|improve this answer




























                          1














                          Usually a binary tree (std::map which you currently use) provides adequate performance. Your CPU budget must be really small.



                          A binary search approach probably won't be much faster. A hashtable (std::unordered_map) might be a solution.



                          Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.



                          std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
                          m.emplace(123, 234);
                          m.emplace(2345, 345);

                          std::cout << m.find(123)->second; // don't use operator to avoid creating entries
                          std::cout << m.find(2345)->second;


                          To check the quality of the hashing function, iterate over all the buckets by calling bucket_size() and add up anything above 1 (i.e. a collision).






                          share|improve this answer


























                            1












                            1








                            1







                            Usually a binary tree (std::map which you currently use) provides adequate performance. Your CPU budget must be really small.



                            A binary search approach probably won't be much faster. A hashtable (std::unordered_map) might be a solution.



                            Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.



                            std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
                            m.emplace(123, 234);
                            m.emplace(2345, 345);

                            std::cout << m.find(123)->second; // don't use operator to avoid creating entries
                            std::cout << m.find(2345)->second;


                            To check the quality of the hashing function, iterate over all the buckets by calling bucket_size() and add up anything above 1 (i.e. a collision).






                            share|improve this answer













                            Usually a binary tree (std::map which you currently use) provides adequate performance. Your CPU budget must be really small.



                            A binary search approach probably won't be much faster. A hashtable (std::unordered_map) might be a solution.



                            Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.



                            std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
                            m.emplace(123, 234);
                            m.emplace(2345, 345);

                            std::cout << m.find(123)->second; // don't use operator to avoid creating entries
                            std::cout << m.find(2345)->second;


                            To check the quality of the hashing function, iterate over all the buckets by calling bucket_size() and add up anything above 1 (i.e. a collision).







                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Jan 18 at 21:54









                            rustyxrustyx

                            29.9k697139




                            29.9k697139






























                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Stack Overflow!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54258881%2fmost-efficient-container-for-uint16-t-to-uint16-t-map%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Liquibase includeAll doesn't find base path

                                How to use setInterval in EJS file?

                                Petrus Granier-Deferre