How to add __len__ to an object without __len__ on its data type definition?












4















According to the documentation, this does not work because of this:




For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:



>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()


https://docs.python.org/3/reference/datamodel.html#special-method-lookup




I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.



This is the generator:



def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser


I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():




def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1


https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py




Then, when trying to add the __len__ attribute to get_sections, hence the error:



get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff


TypeError: object of type 'function' has no len()










share|improve this question


















  • 1





    What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

    – Barmar
    Jan 18 at 23:32






  • 2





    Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

    – juanpa.arrivillaga
    Jan 18 at 23:33
















4















According to the documentation, this does not work because of this:




For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:



>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()


https://docs.python.org/3/reference/datamodel.html#special-method-lookup




I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.



This is the generator:



def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser


I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():




def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1


https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py




Then, when trying to add the __len__ attribute to get_sections, hence the error:



get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff


TypeError: object of type 'function' has no len()










share|improve this question


















  • 1





    What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

    – Barmar
    Jan 18 at 23:32






  • 2





    Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

    – juanpa.arrivillaga
    Jan 18 at 23:33














4












4








4


1






According to the documentation, this does not work because of this:




For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:



>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()


https://docs.python.org/3/reference/datamodel.html#special-method-lookup




I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.



This is the generator:



def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser


I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():




def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1


https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py




Then, when trying to add the __len__ attribute to get_sections, hence the error:



get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff


TypeError: object of type 'function' has no len()










share|improve this question














According to the documentation, this does not work because of this:




For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:



>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()


https://docs.python.org/3/reference/datamodel.html#special-method-lookup




I had tried this on a function generator, which does not have __len__, but I knew beforehand its length, then, I tried monkey patch it with something like c.__len__ = lambda: 5, but it kept saying the generator object had no length.



This is the generator:



def get_sections(loaded_config_file):
for module_file, config_parser in loaded_config_file.items():
for section in config_parser.sections():
yield section, module_file, config_parser


I was passing the generator (which has no length) to this other function (yet, another generator), which requires the iterable length by calling len():




def sequence_timer(sequence, info_frequency=0):
i = 0
start = time.time()
if_counter = start
length = len(sequence)
for elem in sequence:
now = time.time()
if now - if_counter < info_frequency:
yield elem, None
else:
pi = ProgressInfo(now - start, float(i)/length)
if_counter += info_frequency
yield elem, pi
i += 1


https://github.com/arp2600/Etc/blob/60c5af803faecb2d14b5dd3041254ef00a5a79a9/etc.py




Then, when trying to add the __len__ attribute to get_sections, hence the error:



get_sections.__len__ = lambda: calculated_length
for stuff, progress in sequence_timer( get_sections ):
section, module_file, config_parser = stuff


TypeError: object of type 'function' has no len()







python python-3.x generator






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Jan 18 at 23:28









useruser

2,34532152




2,34532152








  • 1





    What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

    – Barmar
    Jan 18 at 23:32






  • 2





    Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

    – juanpa.arrivillaga
    Jan 18 at 23:33














  • 1





    What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

    – Barmar
    Jan 18 at 23:32






  • 2





    Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

    – juanpa.arrivillaga
    Jan 18 at 23:33








1




1





What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

– Barmar
Jan 18 at 23:32





What's your question? You quoted the section of the documentation that says it's not expected to work. If you want to customize len(), you need to define a class.

– Barmar
Jan 18 at 23:32




2




2





Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

– juanpa.arrivillaga
Jan 18 at 23:33





Yeah, easiest thing here is to just write a custom-class wrapper with a __len__ that delegates everything else you need to the generator object

– juanpa.arrivillaga
Jan 18 at 23:33












1 Answer
1






active

oldest

votes


















4














You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:



class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)

def __len__(self):
return self.length

def __iter__(self):
yield from self.it


Now you just change your invalid attempt to set a length of:



get_sections.__len__ = lambda: calculated_length


to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:



get_sections = KnownLengthIterator(get_sections, calculated_length)


No other code needs to change.






share|improve this answer





















  • 1





    I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

    – juanpa.arrivillaga
    Jan 18 at 23:51






  • 1





    @juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

    – ShadowRanger
    Jan 18 at 23:54








  • 1





    Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

    – juanpa.arrivillaga
    Jan 18 at 23:56













Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54262704%2fhow-to-add-len-to-an-object-without-len-on-its-data-type-definition%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









4














You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:



class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)

def __len__(self):
return self.length

def __iter__(self):
yield from self.it


Now you just change your invalid attempt to set a length of:



get_sections.__len__ = lambda: calculated_length


to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:



get_sections = KnownLengthIterator(get_sections, calculated_length)


No other code needs to change.






share|improve this answer





















  • 1





    I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

    – juanpa.arrivillaga
    Jan 18 at 23:51






  • 1





    @juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

    – ShadowRanger
    Jan 18 at 23:54








  • 1





    Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

    – juanpa.arrivillaga
    Jan 18 at 23:56


















4














You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:



class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)

def __len__(self):
return self.length

def __iter__(self):
yield from self.it


Now you just change your invalid attempt to set a length of:



get_sections.__len__ = lambda: calculated_length


to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:



get_sections = KnownLengthIterator(get_sections, calculated_length)


No other code needs to change.






share|improve this answer





















  • 1





    I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

    – juanpa.arrivillaga
    Jan 18 at 23:51






  • 1





    @juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

    – ShadowRanger
    Jan 18 at 23:54








  • 1





    Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

    – juanpa.arrivillaga
    Jan 18 at 23:56
















4












4








4







You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:



class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)

def __len__(self):
return self.length

def __iter__(self):
yield from self.it


Now you just change your invalid attempt to set a length of:



get_sections.__len__ = lambda: calculated_length


to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:



get_sections = KnownLengthIterator(get_sections, calculated_length)


No other code needs to change.






share|improve this answer















You can't add it to an existing object, so make your own wrapper class that has a class level definition you control:



class KnownLengthIterator:
def __init__(self, it, length):
self.it = it
self.length = int(length)

def __len__(self):
return self.length

def __iter__(self):
yield from self.it


Now you just change your invalid attempt to set a length of:



get_sections.__len__ = lambda: calculated_length


to a valid rewrapping that makes get_sections continue to be a valid generator (yield from will delegate all iteration behaviors to the wrapped generator), while exposing a length too:



get_sections = KnownLengthIterator(get_sections, calculated_length)


No other code needs to change.







share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 19 at 0:27

























answered Jan 18 at 23:47









ShadowRangerShadowRanger

59.5k55595




59.5k55595








  • 1





    I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

    – juanpa.arrivillaga
    Jan 18 at 23:51






  • 1





    @juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

    – ShadowRanger
    Jan 18 at 23:54








  • 1





    Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

    – juanpa.arrivillaga
    Jan 18 at 23:56
















  • 1





    I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

    – juanpa.arrivillaga
    Jan 18 at 23:51






  • 1





    @juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

    – ShadowRanger
    Jan 18 at 23:54








  • 1





    Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

    – juanpa.arrivillaga
    Jan 18 at 23:56










1




1





I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

– juanpa.arrivillaga
Jan 18 at 23:51





I would make KnownLengthIterator an actual iterator. So just __iter__(self): return iter(self.it) and __next__(self): return next(self.it)

– juanpa.arrivillaga
Jan 18 at 23:51




1




1





@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

– ShadowRanger
Jan 18 at 23:54







@juanpa.arrivillaga: You could do it, but that's not the way to do it; you've just violated the iterator protocol. If you define __next__, __iter__ must be the identity function (doing nothing but return self). Python code actually relies on being able to detect iterators by testing iter(x) is x and having it be true only for iterators. The advantage to yield from is that the delegation means even coroutine behaviors like send continue to work (at the expense of making the class technically an iterable, not an iterator).

– ShadowRanger
Jan 18 at 23:54






1




1





Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

– juanpa.arrivillaga
Jan 18 at 23:56







Ah, this is a good point. But you could just use return self in __iter__. I suppose maintaining coroutine behaviors might be worth it, depending on the use-case.

– juanpa.arrivillaga
Jan 18 at 23:56




















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54262704%2fhow-to-add-len-to-an-object-without-len-on-its-data-type-definition%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Callistus III

Ostreoida

Plistias Cous