How to add __len__ to an object without __len__ on its data type definition?
According to the documentation, this does not work because of this:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:
>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()
https://docs.python.org/3/reference/datamodel.html#special-method-lookup
I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.
This is the generator:
def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser
I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():
def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1
https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py
Then, when trying to add the __len__ attribute to get_sections, hence the error:
get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff
TypeError: object of type 'function' has no len()
python python-3.x generator
add a comment |
According to the documentation, this does not work because of this:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:
>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()
https://docs.python.org/3/reference/datamodel.html#special-method-lookup
I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.
This is the generator:
def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser
I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():
def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1
https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py
Then, when trying to add the __len__ attribute to get_sections, hence the error:
get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff
TypeError: object of type 'function' has no len()
python python-3.x generator
1
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customizelen(), you need to define a class.
– Barmar
Jan 18 at 23:32
2
Yeah, easiest thing here is to just write a custom-class wrapper with a__len__that delegates everything else you need to the generator object
– juanpa.arrivillaga
Jan 18 at 23:33
add a comment |
According to the documentation, this does not work because of this:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:
>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()
https://docs.python.org/3/reference/datamodel.html#special-method-lookup
I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.
This is the generator:
def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser
I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():
def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1
https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py
Then, when trying to add the __len__ attribute to get_sections, hence the error:
get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff
TypeError: object of type 'function' has no len()
python python-3.x generator
According to the documentation, this does not work because of this:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:
>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()
https://docs.python.org/3/reference/datamodel.html#special-method-lookup
I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.
This is the generator:
def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser
I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():
def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1
https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py
Then, when trying to add the __len__ attribute to get_sections, hence the error:
get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff
TypeError: object of type 'function' has no len()
python python-3.x generator
python python-3.x generator
asked Jan 18 at 23:28
useruser
2,34532152
2,34532152
1
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customizelen(), you need to define a class.
– Barmar
Jan 18 at 23:32
2
Yeah, easiest thing here is to just write a custom-class wrapper with a__len__that delegates everything else you need to the generator object
– juanpa.arrivillaga
Jan 18 at 23:33
add a comment |
1
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customizelen(), you need to define a class.
– Barmar
Jan 18 at 23:32
2
Yeah, easiest thing here is to just write a custom-class wrapper with a__len__that delegates everything else you need to the generator object
– juanpa.arrivillaga
Jan 18 at 23:33
1
1
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize
len(), you need to define a class.– Barmar
Jan 18 at 23:32
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize
len(), you need to define a class.– Barmar
Jan 18 at 23:32
2
2
Yeah, easiest thing here is to just write a custom-class wrapper with a
__len__ that delegates everything else you need to the generator object– juanpa.arrivillaga
Jan 18 at 23:33
Yeah, easiest thing here is to just write a custom-class wrapper with a
__len__ that delegates everything else you need to the generator object– juanpa.arrivillaga
Jan 18 at 23:33
add a comment |
1 Answer
1
active
oldest
votes
You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:
class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)
def __len__(self):
return self.length
def __iter__(self):
yield from self.it
Now you just change your invalid attempt to set a length of:
get_sections.__len__ = lambda: calculated_length
to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:
get_sections = KnownLengthIterator(get_sections, calculated_length)
No other code needs to change.
1
I would makeKnownLengthIteratoran actual iterator. So just__iter__(self): return iter(self.it)and__next__(self): return next(self.it)
– juanpa.arrivillaga
Jan 18 at 23:51
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define__next__,__iter__must be the identity function (doing nothing butreturn self). Python code actually relies on being able to detect iterators by testingiter(x) is xand having it be true only for iterators. The advantage toyield fromis that the delegation means even coroutine behaviors likesendcontinue to work (at the expense of making the class technically an iterable, not an iterator).
– ShadowRanger
Jan 18 at 23:54
1
Ah, this is a good point. But you could just usereturn selfin__iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.
– juanpa.arrivillaga
Jan 18 at 23:56
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54262704%2fhow-to-add-len-to-an-object-without-len-on-its-data-type-definition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:
class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)
def __len__(self):
return self.length
def __iter__(self):
yield from self.it
Now you just change your invalid attempt to set a length of:
get_sections.__len__ = lambda: calculated_length
to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:
get_sections = KnownLengthIterator(get_sections, calculated_length)
No other code needs to change.
1
I would makeKnownLengthIteratoran actual iterator. So just__iter__(self): return iter(self.it)and__next__(self): return next(self.it)
– juanpa.arrivillaga
Jan 18 at 23:51
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define__next__,__iter__must be the identity function (doing nothing butreturn self). Python code actually relies on being able to detect iterators by testingiter(x) is xand having it be true only for iterators. The advantage toyield fromis that the delegation means even coroutine behaviors likesendcontinue to work (at the expense of making the class technically an iterable, not an iterator).
– ShadowRanger
Jan 18 at 23:54
1
Ah, this is a good point. But you could just usereturn selfin__iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.
– juanpa.arrivillaga
Jan 18 at 23:56
add a comment |
You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:
class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)
def __len__(self):
return self.length
def __iter__(self):
yield from self.it
Now you just change your invalid attempt to set a length of:
get_sections.__len__ = lambda: calculated_length
to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:
get_sections = KnownLengthIterator(get_sections, calculated_length)
No other code needs to change.
1
I would makeKnownLengthIteratoran actual iterator. So just__iter__(self): return iter(self.it)and__next__(self): return next(self.it)
– juanpa.arrivillaga
Jan 18 at 23:51
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define__next__,__iter__must be the identity function (doing nothing butreturn self). Python code actually relies on being able to detect iterators by testingiter(x) is xand having it be true only for iterators. The advantage toyield fromis that the delegation means even coroutine behaviors likesendcontinue to work (at the expense of making the class technically an iterable, not an iterator).
– ShadowRanger
Jan 18 at 23:54
1
Ah, this is a good point. But you could just usereturn selfin__iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.
– juanpa.arrivillaga
Jan 18 at 23:56
add a comment |
You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:
class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)
def __len__(self):
return self.length
def __iter__(self):
yield from self.it
Now you just change your invalid attempt to set a length of:
get_sections.__len__ = lambda: calculated_length
to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:
get_sections = KnownLengthIterator(get_sections, calculated_length)
No other code needs to change.
You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:
class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)
def __len__(self):
return self.length
def __iter__(self):
yield from self.it
Now you just change your invalid attempt to set a length of:
get_sections.__len__ = lambda: calculated_length
to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:
get_sections = KnownLengthIterator(get_sections, calculated_length)
No other code needs to change.
edited Jan 19 at 0:27
answered Jan 18 at 23:47
ShadowRangerShadowRanger
59.5k55595
59.5k55595
1
I would makeKnownLengthIteratoran actual iterator. So just__iter__(self): return iter(self.it)and__next__(self): return next(self.it)
– juanpa.arrivillaga
Jan 18 at 23:51
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define__next__,__iter__must be the identity function (doing nothing butreturn self). Python code actually relies on being able to detect iterators by testingiter(x) is xand having it be true only for iterators. The advantage toyield fromis that the delegation means even coroutine behaviors likesendcontinue to work (at the expense of making the class technically an iterable, not an iterator).
– ShadowRanger
Jan 18 at 23:54
1
Ah, this is a good point. But you could just usereturn selfin__iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.
– juanpa.arrivillaga
Jan 18 at 23:56
add a comment |
1
I would makeKnownLengthIteratoran actual iterator. So just__iter__(self): return iter(self.it)and__next__(self): return next(self.it)
– juanpa.arrivillaga
Jan 18 at 23:51
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define__next__,__iter__must be the identity function (doing nothing butreturn self). Python code actually relies on being able to detect iterators by testingiter(x) is xand having it be true only for iterators. The advantage toyield fromis that the delegation means even coroutine behaviors likesendcontinue to work (at the expense of making the class technically an iterable, not an iterator).
– ShadowRanger
Jan 18 at 23:54
1
Ah, this is a good point. But you could just usereturn selfin__iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.
– juanpa.arrivillaga
Jan 18 at 23:56
1
1
I would make
KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)– juanpa.arrivillaga
Jan 18 at 23:51
I would make
KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)– juanpa.arrivillaga
Jan 18 at 23:51
1
1
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define
__next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).– ShadowRanger
Jan 18 at 23:54
@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define
__next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).– ShadowRanger
Jan 18 at 23:54
1
1
Ah, this is a good point. But you could just use
return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.– juanpa.arrivillaga
Jan 18 at 23:56
Ah, this is a good point. But you could just use
return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.– juanpa.arrivillaga
Jan 18 at 23:56
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54262704%2fhow-to-add-len-to-an-object-without-len-on-its-data-type-definition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize
len(), you need to define a class.– Barmar
Jan 18 at 23:32
2
Yeah, easiest thing here is to just write a custom-class wrapper with a
__len__that delegates everything else you need to the generator object– juanpa.arrivillaga
Jan 18 at 23:33