Most efficient container for uint16_t to uint16_t map
I am developing program for very processing power limited machine where I want to map uint16_t
keys to uint16_t
values.
I am currently using std::map
use non-safe reading:
std::map<uint16_t, uint16_t> m;
//fill m only once
while(true){
auto x = m[y];
}
The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?
Edit:
Some information:
- Total number of items is less than 500
- Insertion is done only once
- Querying values is done more than 250 times / sec
- Keys and values are unique
- Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB
- 100 MHz single core processor
c++ performance dictionary
|
show 9 more comments
I am developing program for very processing power limited machine where I want to map uint16_t
keys to uint16_t
values.
I am currently using std::map
use non-safe reading:
std::map<uint16_t, uint16_t> m;
//fill m only once
while(true){
auto x = m[y];
}
The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?
Edit:
Some information:
- Total number of items is less than 500
- Insertion is done only once
- Querying values is done more than 250 times / sec
- Keys and values are unique
- Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB
- 100 MHz single core processor
c++ performance dictionary
3
Have you triedstd::unordered_map
?
– NathanOliver
Jan 18 at 17:35
2
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
2
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
11
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes?uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.
– Remy Lebeau
Jan 18 at 17:36
4
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40
|
show 9 more comments
I am developing program for very processing power limited machine where I want to map uint16_t
keys to uint16_t
values.
I am currently using std::map
use non-safe reading:
std::map<uint16_t, uint16_t> m;
//fill m only once
while(true){
auto x = m[y];
}
The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?
Edit:
Some information:
- Total number of items is less than 500
- Insertion is done only once
- Querying values is done more than 250 times / sec
- Keys and values are unique
- Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB
- 100 MHz single core processor
c++ performance dictionary
I am developing program for very processing power limited machine where I want to map uint16_t
keys to uint16_t
values.
I am currently using std::map
use non-safe reading:
std::map<uint16_t, uint16_t> m;
//fill m only once
while(true){
auto x = m[y];
}
The performance is still non-acceptable for the requirements. Is there a better solution in term of execution speed?
Edit:
Some information:
- Total number of items is less than 500
- Insertion is done only once
- Querying values is done more than 250 times / sec
- Keys and values are unique
- Ram is very limited, overall ram is 512KB, free ram for this part of code is less than 50KB
- 100 MHz single core processor
c++ performance dictionary
c++ performance dictionary
edited Jan 18 at 18:08
Humam Helfawi
asked Jan 18 at 17:33
Humam HelfawiHumam Helfawi
11.7k63598
11.7k63598
3
Have you triedstd::unordered_map
?
– NathanOliver
Jan 18 at 17:35
2
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
2
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
11
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes?uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.
– Remy Lebeau
Jan 18 at 17:36
4
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40
|
show 9 more comments
3
Have you triedstd::unordered_map
?
– NathanOliver
Jan 18 at 17:35
2
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
2
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
11
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes?uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.
– Remy Lebeau
Jan 18 at 17:36
4
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40
3
3
Have you tried
std::unordered_map
?– NathanOliver
Jan 18 at 17:35
Have you tried
std::unordered_map
?– NathanOliver
Jan 18 at 17:35
2
2
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
2
2
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
11
11
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.– Remy Lebeau
Jan 18 at 17:36
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes? uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.– Remy Lebeau
Jan 18 at 17:36
4
4
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40
|
show 9 more comments
3 Answers
3
active
oldest
votes
Without some more context about your map,
if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.
if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.
if you're using not a lot of data and require it to be flexible, std::vector is a good choice
Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.
New contributor
add a comment |
- Total number of items is less than 500
- Insertion is done only once
Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.
Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).
add a comment |
Usually a binary tree (std::map
which you currently use) provides adequate performance. Your CPU budget must be really small.
A binary search approach probably won't be much faster. A hashtable (std::unordered_map
) might be a solution.
Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.
std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
m.emplace(123, 234);
m.emplace(2345, 345);
std::cout << m.find(123)->second; // don't use operator to avoid creating entries
std::cout << m.find(2345)->second;
To check the quality of the hashing function, iterate over all the buckets by calling bucket_size()
and add up anything above 1 (i.e. a collision).
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54258881%2fmost-efficient-container-for-uint16-t-to-uint16-t-map%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Without some more context about your map,
if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.
if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.
if you're using not a lot of data and require it to be flexible, std::vector is a good choice
Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.
New contributor
add a comment |
Without some more context about your map,
if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.
if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.
if you're using not a lot of data and require it to be flexible, std::vector is a good choice
Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.
New contributor
add a comment |
Without some more context about your map,
if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.
if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.
if you're using not a lot of data and require it to be flexible, std::vector is a good choice
Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.
New contributor
Without some more context about your map,
if you intend to use a lot of keys, a big array like suggested before would be easy enough to handle since there wouldn't be collisions, but if you're not going to use all the memory, it might be wasteful.
if you intend to use a fair amount of data, but not enough that there'd be too many hash collisions, std::unordered_map has amortized O(1) lookups, and if you don't care about the order they're stored in, that could be a good guess.
if you're using not a lot of data and require it to be flexible, std::vector is a good choice
Seeing as all we know is that it's a uin16_t to uint16_t map, there's no one best answer.
New contributor
New contributor
answered Jan 18 at 17:45
DanielDaniel
373114
373114
New contributor
New contributor
add a comment |
add a comment |
- Total number of items is less than 500
- Insertion is done only once
Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.
Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).
add a comment |
- Total number of items is less than 500
- Insertion is done only once
Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.
Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).
add a comment |
- Total number of items is less than 500
- Insertion is done only once
Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.
Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).
- Total number of items is less than 500
- Insertion is done only once
Keep a fixed-size (500) array of key-value pairs plus a "valid items count", if the actual size is known only at runtime; populate it with the data and sort it; then you can simply do a binary search.
Given that the elements are few, depending on the specific processor it may even be more convenient to just do a linear search (perhaps keeping the most used elements first if you happen to have clues about the most frequently looked-up values).
answered Jan 18 at 17:54
Matteo ItaliaMatteo Italia
100k15142239
100k15142239
add a comment |
add a comment |
Usually a binary tree (std::map
which you currently use) provides adequate performance. Your CPU budget must be really small.
A binary search approach probably won't be much faster. A hashtable (std::unordered_map
) might be a solution.
Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.
std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
m.emplace(123, 234);
m.emplace(2345, 345);
std::cout << m.find(123)->second; // don't use operator to avoid creating entries
std::cout << m.find(2345)->second;
To check the quality of the hashing function, iterate over all the buckets by calling bucket_size()
and add up anything above 1 (i.e. a collision).
add a comment |
Usually a binary tree (std::map
which you currently use) provides adequate performance. Your CPU budget must be really small.
A binary search approach probably won't be much faster. A hashtable (std::unordered_map
) might be a solution.
Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.
std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
m.emplace(123, 234);
m.emplace(2345, 345);
std::cout << m.find(123)->second; // don't use operator to avoid creating entries
std::cout << m.find(2345)->second;
To check the quality of the hashing function, iterate over all the buckets by calling bucket_size()
and add up anything above 1 (i.e. a collision).
add a comment |
Usually a binary tree (std::map
which you currently use) provides adequate performance. Your CPU budget must be really small.
A binary search approach probably won't be much faster. A hashtable (std::unordered_map
) might be a solution.
Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.
std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
m.emplace(123, 234);
m.emplace(2345, 345);
std::cout << m.find(123)->second; // don't use operator to avoid creating entries
std::cout << m.find(2345)->second;
To check the quality of the hashing function, iterate over all the buckets by calling bucket_size()
and add up anything above 1 (i.e. a collision).
Usually a binary tree (std::map
which you currently use) provides adequate performance. Your CPU budget must be really small.
A binary search approach probably won't be much faster. A hashtable (std::unordered_map
) might be a solution.
Just make sure to size it properly so as to avoid rehashing and stay just below 1.0 load factor.
std::unordered_map<uint16_t, uint16_t> m(503); // 503 is a prime
m.emplace(123, 234);
m.emplace(2345, 345);
std::cout << m.find(123)->second; // don't use operator to avoid creating entries
std::cout << m.find(2345)->second;
To check the quality of the hashing function, iterate over all the buckets by calling bucket_size()
and add up anything above 1 (i.e. a collision).
answered Jan 18 at 21:54
rustyxrustyx
29.9k697139
29.9k697139
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54258881%2fmost-efficient-container-for-uint16-t-to-uint16-t-map%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
Have you tried
std::unordered_map
?– NathanOliver
Jan 18 at 17:35
2
I think more information about the task at hand and restrictions would help.
– Philipp
Jan 18 at 17:35
2
Which constraints are required for the container?
– JVApen
Jan 18 at 17:35
11
uint16_t
can hold values 0..65535, so why not just use a fixed array where the keys are the indexes?uint16_t m[65535]; ... uint16_t y = ...; auto x = m[y];
Such an array would take up only 128K of memory at most.– Remy Lebeau
Jan 18 at 17:36
4
How many items? I once did some optimization work and found that for 10 items or less, with small keys (like integers) a simple linear search was faster than maps, unordered maps or binary search. Just start at the front and search until the end or you find it. Don't even keep a length value, just always search the 10 items and fill unused slots with zeros or -1 or whatever. With the fixed size array and the right compiler (and CPU obviously) you can even get that unrolled and vectorized for you.
– Zan Lynx
Jan 18 at 17:40