Why does functools.lru_cache not cache __call__ while working on normal methods
I have been trying to make functools.lru_cache instance specific as described in this answer, but their solution fails when used on the __call__ method.
class test:
def __init__(self):
self.method = lru_cache()(self.method)
self.__call__ = lru_cache()(self.__call__)
def method(self, x):
print('method', end=' ')
return x
def __call__(self, x):
print('__call__', end=' ')
return x
b = test()
# b.method is cached as expected
print(b.method(1)) # method 1
print(b.method(1)) # 1
# __call__ is executed every time
print(b(1)) # __call__ 1
print(b(1)) # __call__ 1
So the results of __call__ are not getting cached when wrapped using this method. The cache on __call__ does not even register the function having been called, and unhashable values do not throw errors.
print(b.method.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
print(b.__call__.cache_info())
# CacheInfo(hits=0, misses=0, maxsize=128, currsize=0)
print(b.call({})) # __call__ {}
print(b.method({})) # ... TypeError: unhashable type: 'dict'
python caching magic-methods
asked Nov 22 '18 at 2:38
WillWill
149311
add a comment |
I have been trying to make functools.lru_cache instance specific as described in this answer, but their solution fails when used on the __call__ method.
class test:
def __init__(self):
self.method = lru_cache()(self.method)
self.__call__ = lru_cache()(self.__call__)
def method(self, x):
print('method', end=' ')
return x
def __call__(self, x):
print('__call__', end=' ')
return x
b = test()
# b.method is cached as expected
print(b.method(1)) # method 1
print(b.method(1)) # 1
# __call__ is executed every time
print(b(1)) # __call__ 1
print(b(1)) # __call__ 1
So the results of __call__ are not getting cached when wrapped using this method. The cache on __call__ does not even register the function having been called, and unhashable values do not throw errors.
print(b.method.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
print(b.__call__.cache_info())
# CacheInfo(hits=0, misses=0, maxsize=128, currsize=0)
print(b.call({})) # __call__ {}
print(b.method({})) # ... TypeError: unhashable type: 'dict'
python caching magic-methods
asked Nov 22 '18 at 2:38
WillWill
149311
1
i have tried you code, and if applying@lru_cachedecorator to__call__, it works fine, but for you case, you are assigning__call__to self when initialize, so the lru version of__call__is added inself.__dict__, without actually changing__call__method.
– Enix
Nov 22 '18 at 4:37
@Enix If you mean decorating__call__normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)
– Will
Nov 22 '18 at 18:22
add a comment |
I have been trying to make functools.lru_cache instance specific as described in this answer, but their solution fails when used on the __call__ method.
class test:
def __init__(self):
self.method = lru_cache()(self.method)
self.__call__ = lru_cache()(self.__call__)
def method(self, x):
print('method', end=' ')
return x
def __call__(self, x):
print('__call__', end=' ')
return x
b = test()
# b.method is cached as expected
print(b.method(1)) # method 1
print(b.method(1)) # 1
# __call__ is executed every time
print(b(1)) # __call__ 1
print(b(1)) # __call__ 1
So the results of __call__ are not getting cached when wrapped using this method. The cache on __call__ does not even register the function having been called, and unhashable values do not throw errors.
print(b.method.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
print(b.__call__.cache_info())
# CacheInfo(hits=0, misses=0, maxsize=128, currsize=0)
print(b.call({})) # __call__ {}
print(b.method({})) # ... TypeError: unhashable type: 'dict'
python caching magic-methods
asked Nov 22 '18 at 2:38
WillWill
149311
I have been trying to make functools.lru_cache instance specific as described in this answer, but their solution fails when used on the __call__ method.
class test:
def __init__(self):
self.method = lru_cache()(self.method)
self.__call__ = lru_cache()(self.__call__)
def method(self, x):
print('method', end=' ')
return x
def __call__(self, x):
print('__call__', end=' ')
return x
b = test()
# b.method is cached as expected
print(b.method(1)) # method 1
print(b.method(1)) # 1
# __call__ is executed every time
print(b(1)) # __call__ 1
print(b(1)) # __call__ 1
So the results of __call__ are not getting cached when wrapped using this method. The cache on __call__ does not even register the function having been called, and unhashable values do not throw errors.
print(b.method.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
print(b.__call__.cache_info())
# CacheInfo(hits=0, misses=0, maxsize=128, currsize=0)
print(b.call({})) # __call__ {}
print(b.method({})) # ... TypeError: unhashable type: 'dict'
python caching magic-methods
python caching magic-methods
asked Nov 22 '18 at 2:38
WillWill
149311
asked Nov 22 '18 at 2:38
WillWill
149311
edited Nov 22 '18 at 18:23
Will
asked Nov 22 '18 at 2:38
WillWill
149311
asked Nov 22 '18 at 2:38
WillWill
149311
asked Nov 22 '18 at 2:38
WillWill
149311
149311
1
i have tried you code, and if applying@lru_cachedecorator to__call__, it works fine, but for you case, you are assigning__call__to self when initialize, so the lru version of__call__is added inself.__dict__, without actually changing__call__method.
– Enix
Nov 22 '18 at 4:37
@Enix If you mean decorating__call__normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)
– Will
Nov 22 '18 at 18:22
add a comment |
1
i have tried you code, and if applying@lru_cachedecorator to__call__, it works fine, but for you case, you are assigning__call__to self when initialize, so the lru version of__call__is added inself.__dict__, without actually changing__call__method.
– Enix
Nov 22 '18 at 4:37
@Enix If you mean decorating__call__normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)
– Will
Nov 22 '18 at 18:22
1
1
i have tried you code, and if applying
@lru_cache decorator to __call__ , it works fine, but for you case, you are assigning __call__ to self when initialize, so the lru version of __call__ is added in self.__dict__, without actually changing __call__ method.– Enix
Nov 22 '18 at 4:37
i have tried you code, and if applying
@lru_cache decorator to __call__ , it works fine, but for you case, you are assigning __call__ to self when initialize, so the lru version of __call__ is added in self.__dict__, without actually changing __call__ method.– Enix
Nov 22 '18 at 4:37
@Enix If you mean decorating
__call__ normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)– Will
Nov 22 '18 at 18:22
@Enix If you mean decorating
__call__ normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)– Will
Nov 22 '18 at 18:22
add a comment |
1 Answer
1
active
oldest
votes
This is due to the difference between class attributes and instance attributes. When accessing an attribute (such as method) python first checks for an instance attribute. If you have not assigned to self.method it will not find one. Then class attributes are checked, which is equivalent to self.__class__.method. The value of this function is not changed by assigning to self.method, that only updates the instance attribute.
However, b(1) becomes b.__class__.__call__(b, 1) which uses the original class definition of __call__ and b.__call__(1) will be cached the same way as method since it uses the instance definition.
answered Nov 22 '18 at 18:56
WillWill
149311
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423121%2fwhy-does-functools-lru-cache-not-cache-call-while-working-on-normal-methods%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
This is due to the difference between class attributes and instance attributes. When accessing an attribute (such as method) python first checks for an instance attribute. If you have not assigned to self.method it will not find one. Then class attributes are checked, which is equivalent to self.__class__.method. The value of this function is not changed by assigning to self.method, that only updates the instance attribute.
However, b(1) becomes b.__class__.__call__(b, 1) which uses the original class definition of __call__ and b.__call__(1) will be cached the same way as method since it uses the instance definition.
answered Nov 22 '18 at 18:56
WillWill
149311
add a comment |
This is due to the difference between class attributes and instance attributes. When accessing an attribute (such as method) python first checks for an instance attribute. If you have not assigned to self.method it will not find one. Then class attributes are checked, which is equivalent to self.__class__.method. The value of this function is not changed by assigning to self.method, that only updates the instance attribute.
However, b(1) becomes b.__class__.__call__(b, 1) which uses the original class definition of __call__ and b.__call__(1) will be cached the same way as method since it uses the instance definition.
answered Nov 22 '18 at 18:56
WillWill
149311
add a comment |
This is due to the difference between class attributes and instance attributes. When accessing an attribute (such as method) python first checks for an instance attribute. If you have not assigned to self.method it will not find one. Then class attributes are checked, which is equivalent to self.__class__.method. The value of this function is not changed by assigning to self.method, that only updates the instance attribute.
However, b(1) becomes b.__class__.__call__(b, 1) which uses the original class definition of __call__ and b.__call__(1) will be cached the same way as method since it uses the instance definition.
answered Nov 22 '18 at 18:56
WillWill
149311
This is due to the difference between class attributes and instance attributes. When accessing an attribute (such as method) python first checks for an instance attribute. If you have not assigned to self.method it will not find one. Then class attributes are checked, which is equivalent to self.__class__.method. The value of this function is not changed by assigning to self.method, that only updates the instance attribute.
However, b(1) becomes b.__class__.__call__(b, 1) which uses the original class definition of __call__ and b.__call__(1) will be cached the same way as method since it uses the instance definition.
answered Nov 22 '18 at 18:56
WillWill
149311
edited Nov 22 '18 at 19:29
answered Nov 22 '18 at 18:56
WillWill
149311
answered Nov 22 '18 at 18:56
WillWill
149311
answered Nov 22 '18 at 18:56
WillWill
149311
149311
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423121%2fwhy-does-functools-lru-cache-not-cache-call-while-working-on-normal-methods%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
i have tried you code, and if applying
@lru_cachedecorator to__call__, it works fine, but for you case, you are assigning__call__to self when initialize, so the lru version of__call__is added inself.__dict__, without actually changing__call__method.– Enix
Nov 22 '18 at 4:37
@Enix If you mean decorating
__call__normally then yes that works fine for one instance, but as soon as you have two instances then the cache is shared, and resetting it on one resets it on the other instance, so using the decorator normally does not work. (I have tried not to make this question about how to have instance specific caches, since the linked question covers that)– Will
Nov 22 '18 at 18:22