How can I periodically execute a function with asyncio?
I'm migrating from tornado
to asyncio
, and I can't find the asyncio
equivalent of tornado
's PeriodicCallback
. (A PeriodicCallback
takes two arguments: the function to run and the number of milliseconds between calls.)
- Is there such an equivalent in
asyncio
? - If not, what would be the cleanest way to implement this without running the risk of getting a
RecursionError
after a while?
python python-3.x tornado python-3.5 python-asyncio
add a comment |
I'm migrating from tornado
to asyncio
, and I can't find the asyncio
equivalent of tornado
's PeriodicCallback
. (A PeriodicCallback
takes two arguments: the function to run and the number of milliseconds between calls.)
- Is there such an equivalent in
asyncio
? - If not, what would be the cleanest way to implement this without running the risk of getting a
RecursionError
after a while?
python python-3.x tornado python-3.5 python-asyncio
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Just addawait asyncio.sleep(time)
to your function.
– shongololo
Jul 2 '16 at 5:16
Same with Twisted, noLoopingCall
implementation.
– zgoda
Jun 29 '18 at 14:16
add a comment |
I'm migrating from tornado
to asyncio
, and I can't find the asyncio
equivalent of tornado
's PeriodicCallback
. (A PeriodicCallback
takes two arguments: the function to run and the number of milliseconds between calls.)
- Is there such an equivalent in
asyncio
? - If not, what would be the cleanest way to implement this without running the risk of getting a
RecursionError
after a while?
python python-3.x tornado python-3.5 python-asyncio
I'm migrating from tornado
to asyncio
, and I can't find the asyncio
equivalent of tornado
's PeriodicCallback
. (A PeriodicCallback
takes two arguments: the function to run and the number of milliseconds between calls.)
- Is there such an equivalent in
asyncio
? - If not, what would be the cleanest way to implement this without running the risk of getting a
RecursionError
after a while?
python python-3.x tornado python-3.5 python-asyncio
python python-3.x tornado python-3.5 python-asyncio
asked May 29 '16 at 16:17
2Cubed2Cubed
1,44141333
1,44141333
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Just addawait asyncio.sleep(time)
to your function.
– shongololo
Jul 2 '16 at 5:16
Same with Twisted, noLoopingCall
implementation.
– zgoda
Jun 29 '18 at 14:16
add a comment |
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Just addawait asyncio.sleep(time)
to your function.
– shongololo
Jul 2 '16 at 5:16
Same with Twisted, noLoopingCall
implementation.
– zgoda
Jun 29 '18 at 14:16
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Just add
await asyncio.sleep(time)
to your function.– shongololo
Jul 2 '16 at 5:16
Just add
await asyncio.sleep(time)
to your function.– shongololo
Jul 2 '16 at 5:16
Same with Twisted, no
LoopingCall
implementation.– zgoda
Jun 29 '18 at 14:16
Same with Twisted, no
LoopingCall
implementation.– zgoda
Jun 29 '18 at 14:16
add a comment |
5 Answers
5
active
oldest
votes
For Python versions below 3.5:
import asyncio
@asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
3
Even in Tornado, I'd recommend a loop like this instead of aPeriodicCallback
for applications that make use of coroutines.
– Ben Darnell
May 29 '16 at 18:17
7
Just a quick note: Don’t directly createTask
instances; use theensure_future()
function or theAbstractEventLoop.create_task()
method. From the asyncio documentation.
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of thestop
function. I.e.:loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
Or you can just call it likeloop.call_later(5, task.cancel)
.
– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-levelasyncio.create_task()
to createTask
s.
– mhchia
Jan 23 at 8:37
add a comment |
When you feel that something should happen "in background" of your asyncio program, asyncio.Task
might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start
we just start task that calls some functions and sleeps some time in endless loop. On stop
we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func
, you possibly would need to run it in executor.
The most complete and clear answer so far! Thanks. Is it good idea to require thefunc
to be a coroutine, so we can:await self.func()
in the_run
method?
– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
add a comment |
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
await for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks()
iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
Theasyncio
event loop has atime()
method that could be used in place of thetime
module.
– krs013
May 26 '17 at 4:34
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
add a comment |
Based on @A. Jesse Jiryu Davis response (with @Torkel Bjørnson-Langen and @ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
@asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
periodic
should probably useloop.time()
in preference totime.time()
becauseloop.time()
is the time reference internally used byasyncio.sleep()
.loop.time()
returns monotonic time, whiletime.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.
– user4815162342
Feb 4 '18 at 21:29
add a comment |
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
@periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37512182%2fhow-can-i-periodically-execute-a-function-with-asyncio%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
For Python versions below 3.5:
import asyncio
@asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
3
Even in Tornado, I'd recommend a loop like this instead of aPeriodicCallback
for applications that make use of coroutines.
– Ben Darnell
May 29 '16 at 18:17
7
Just a quick note: Don’t directly createTask
instances; use theensure_future()
function or theAbstractEventLoop.create_task()
method. From the asyncio documentation.
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of thestop
function. I.e.:loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
Or you can just call it likeloop.call_later(5, task.cancel)
.
– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-levelasyncio.create_task()
to createTask
s.
– mhchia
Jan 23 at 8:37
add a comment |
For Python versions below 3.5:
import asyncio
@asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
3
Even in Tornado, I'd recommend a loop like this instead of aPeriodicCallback
for applications that make use of coroutines.
– Ben Darnell
May 29 '16 at 18:17
7
Just a quick note: Don’t directly createTask
instances; use theensure_future()
function or theAbstractEventLoop.create_task()
method. From the asyncio documentation.
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of thestop
function. I.e.:loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
Or you can just call it likeloop.call_later(5, task.cancel)
.
– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-levelasyncio.create_task()
to createTask
s.
– mhchia
Jan 23 at 8:37
add a comment |
For Python versions below 3.5:
import asyncio
@asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python versions below 3.5:
import asyncio
@asyncio.coroutine
def periodic():
while True:
print('periodic')
yield from asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
For Python 3.5 and above:
import asyncio
async def periodic():
while True:
print('periodic')
await asyncio.sleep(1)
def stop():
task.cancel()
loop = asyncio.get_event_loop()
loop.call_later(5, stop)
task = loop.create_task(periodic())
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
edited Aug 13 '18 at 11:02
Lonami
5371614
5371614
answered May 29 '16 at 16:53
A. Jesse Jiryu DavisA. Jesse Jiryu Davis
18.4k33349
18.4k33349
3
Even in Tornado, I'd recommend a loop like this instead of aPeriodicCallback
for applications that make use of coroutines.
– Ben Darnell
May 29 '16 at 18:17
7
Just a quick note: Don’t directly createTask
instances; use theensure_future()
function or theAbstractEventLoop.create_task()
method. From the asyncio documentation.
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of thestop
function. I.e.:loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
Or you can just call it likeloop.call_later(5, task.cancel)
.
– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-levelasyncio.create_task()
to createTask
s.
– mhchia
Jan 23 at 8:37
add a comment |
3
Even in Tornado, I'd recommend a loop like this instead of aPeriodicCallback
for applications that make use of coroutines.
– Ben Darnell
May 29 '16 at 18:17
7
Just a quick note: Don’t directly createTask
instances; use theensure_future()
function or theAbstractEventLoop.create_task()
method. From the asyncio documentation.
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of thestop
function. I.e.:loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
Or you can just call it likeloop.call_later(5, task.cancel)
.
– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-levelasyncio.create_task()
to createTask
s.
– mhchia
Jan 23 at 8:37
3
3
Even in Tornado, I'd recommend a loop like this instead of a
PeriodicCallback
for applications that make use of coroutines.– Ben Darnell
May 29 '16 at 18:17
Even in Tornado, I'd recommend a loop like this instead of a
PeriodicCallback
for applications that make use of coroutines.– Ben Darnell
May 29 '16 at 18:17
7
7
Just a quick note: Don’t directly create
Task
instances; use the ensure_future()
function or the AbstractEventLoop.create_task()
method. From the asyncio documentation.– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
Just a quick note: Don’t directly create
Task
instances; use the ensure_future()
function or the AbstractEventLoop.create_task()
method. From the asyncio documentation.– Torkel Bjørnson-Langen
Jan 5 '17 at 22:38
A lambda may be used instead instead of the
stop
function. I.e.: loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
A lambda may be used instead instead of the
stop
function. I.e.: loop.call_later(5, lambda: task.cancel())
– Torkel Bjørnson-Langen
Jan 5 '17 at 22:41
11
11
Or you can just call it like
loop.call_later(5, task.cancel)
.– ReWrite
Jan 19 '17 at 1:32
Or you can just call it like
loop.call_later(5, task.cancel)
.– ReWrite
Jan 19 '17 at 1:32
Just a note for Python 3.7: From the asyncio doc, we should use the high-level
asyncio.create_task()
to create Task
s.– mhchia
Jan 23 at 8:37
Just a note for Python 3.7: From the asyncio doc, we should use the high-level
asyncio.create_task()
to create Task
s.– mhchia
Jan 23 at 8:37
add a comment |
When you feel that something should happen "in background" of your asyncio program, asyncio.Task
might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start
we just start task that calls some functions and sleeps some time in endless loop. On stop
we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func
, you possibly would need to run it in executor.
The most complete and clear answer so far! Thanks. Is it good idea to require thefunc
to be a coroutine, so we can:await self.func()
in the_run
method?
– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
add a comment |
When you feel that something should happen "in background" of your asyncio program, asyncio.Task
might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start
we just start task that calls some functions and sleeps some time in endless loop. On stop
we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func
, you possibly would need to run it in executor.
The most complete and clear answer so far! Thanks. Is it good idea to require thefunc
to be a coroutine, so we can:await self.func()
in the_run
method?
– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
add a comment |
When you feel that something should happen "in background" of your asyncio program, asyncio.Task
might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start
we just start task that calls some functions and sleeps some time in endless loop. On stop
we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func
, you possibly would need to run it in executor.
When you feel that something should happen "in background" of your asyncio program, asyncio.Task
might be good way to do it. You can read this post to see how to work with tasks.
Here's possible implementation of class that executes some function periodically:
import asyncio
from contextlib import suppress
class Periodic:
def __init__(self, func, time):
self.func = func
self.time = time
self.is_started = False
self._task = None
async def start(self):
if not self.is_started:
self.is_started = True
# Start task to call func periodically:
self._task = asyncio.ensure_future(self._run())
async def stop(self):
if self.is_started:
self.is_started = False
# Stop task and await it stopped:
self._task.cancel()
with suppress(asyncio.CancelledError):
await self._task
async def _run(self):
while True:
await asyncio.sleep(self.time)
self.func()
Let's test it:
async def main():
p = Periodic(lambda: print('test'), 1)
try:
print('Start')
await p.start()
await asyncio.sleep(3.1)
print('Stop')
await p.stop()
await asyncio.sleep(3.1)
print('Start')
await p.start()
await asyncio.sleep(3.1)
finally:
await p.stop() # we should stop task finally
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
Start
test
test
test
Stop
Start
test
test
test
[Finished in 9.5s]
As you see on start
we just start task that calls some functions and sleeps some time in endless loop. On stop
we just cancel that task. Note, that task should be stopped at the moment program finished.
One more important thing that your callback shouldn't take much time to be executed (or it'll freeze your event loop). If you're planning to call some long-running func
, you possibly would need to run it in executor.
edited May 23 '17 at 12:02
Community♦
11
11
answered May 29 '16 at 20:27
Mikhail GerasimovMikhail Gerasimov
13.4k43765
13.4k43765
The most complete and clear answer so far! Thanks. Is it good idea to require thefunc
to be a coroutine, so we can:await self.func()
in the_run
method?
– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
add a comment |
The most complete and clear answer so far! Thanks. Is it good idea to require thefunc
to be a coroutine, so we can:await self.func()
in the_run
method?
– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
The most complete and clear answer so far! Thanks. Is it good idea to require the
func
to be a coroutine, so we can: await self.func()
in the _run
method?– Sergey Belash
Jan 30 '17 at 7:23
The most complete and clear answer so far! Thanks. Is it good idea to require the
func
to be a coroutine, so we can: await self.func()
in the _run
method?– Sergey Belash
Jan 30 '17 at 7:23
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
@SergeyBelash, sure, it'll be ok. Note only that since we cancel task at random time, your func may be also cancelled at random time. It means every await line inside your function can potentially raise CancelledError. But it's actual for every async function at all (just like KeyboardInterrupt can be raised randomly in regular non-async code).
– Mikhail Gerasimov
Jan 30 '17 at 8:02
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
I worry with this (and other answers) that the repeat rate won't be exactly the time value. If func takes an appreciable time to execute it won't even be close, and over a long period it will drift even if func takes negligible time.
– Ian Goldby
Dec 4 '17 at 11:29
add a comment |
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
await for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks()
iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
Theasyncio
event loop has atime()
method that could be used in place of thetime
module.
– krs013
May 26 '17 at 4:34
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
add a comment |
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
await for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks()
iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
Theasyncio
event loop has atime()
method that could be used in place of thetime
module.
– krs013
May 26 '17 at 4:34
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
add a comment |
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
await for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks()
iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
There is no built-in support for periodic calls, no.
Just create your own scheduler loop that sleeps and executes any tasks scheduled:
import math, time
async def scheduler():
while True:
# sleep until the next whole second
now = time.time()
await asyncio.sleep(math.ceil(now) - now)
# execute any scheduled tasks
await for task in scheduled_tasks(time.time()):
await task()
The scheduled_tasks()
iterator should produce tasks that are ready to be run at the given time. Note that producing the schedule and kicking off all the tasks could in theory take longer than 1 second; the idea here is that the scheduler yields all tasks that should have started since the last check.
answered May 29 '16 at 16:41
Martijn Pieters♦Martijn Pieters
708k13624732296
708k13624732296
Theasyncio
event loop has atime()
method that could be used in place of thetime
module.
– krs013
May 26 '17 at 4:34
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
add a comment |
Theasyncio
event loop has atime()
method that could be used in place of thetime
module.
– krs013
May 26 '17 at 4:34
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
The
asyncio
event loop has a time()
method that could be used in place of the time
module.– krs013
May 26 '17 at 4:34
The
asyncio
event loop has a time()
method that could be used in place of the time
module.– krs013
May 26 '17 at 4:34
1
1
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
@krs013: That's a different clock; it doesn't necessarily give you real-world time (it depends on the event loop implementation, and can measure CPU time ticks or another monotonically increasing clock measure). Because it is not guaranteed to provide a measure in seconds, it should not be used here.
– Martijn Pieters♦
May 26 '17 at 6:29
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
Oh, good point, thanks. I figured that it would be good enough for interval timing, but it looks like no guarantee is made for accuracy in sleeping threads. The implementations I've seen seem to just use the machines uptime in nanoseconds, but yeah, you're right. I think I have some code to fix now...
– krs013
May 26 '17 at 6:37
add a comment |
Based on @A. Jesse Jiryu Davis response (with @Torkel Bjørnson-Langen and @ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
@asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
periodic
should probably useloop.time()
in preference totime.time()
becauseloop.time()
is the time reference internally used byasyncio.sleep()
.loop.time()
returns monotonic time, whiletime.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.
– user4815162342
Feb 4 '18 at 21:29
add a comment |
Based on @A. Jesse Jiryu Davis response (with @Torkel Bjørnson-Langen and @ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
@asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
periodic
should probably useloop.time()
in preference totime.time()
becauseloop.time()
is the time reference internally used byasyncio.sleep()
.loop.time()
returns monotonic time, whiletime.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.
– user4815162342
Feb 4 '18 at 21:29
add a comment |
Based on @A. Jesse Jiryu Davis response (with @Torkel Bjørnson-Langen and @ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
@asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
Based on @A. Jesse Jiryu Davis response (with @Torkel Bjørnson-Langen and @ReWrite comments) this is an improvement which avoids drift.
import time
import asyncio
@asyncio.coroutine
def periodic(period):
def g_tick():
t = time.time()
count = 0
while True:
count += 1
yield max(t + count * period - time.time(), 0)
g = g_tick()
while True:
print('periodic', time.time())
yield from asyncio.sleep(next(g))
loop = asyncio.get_event_loop()
task = loop.create_task(periodic(1))
loop.call_later(5, task.cancel)
try:
loop.run_until_complete(task)
except asyncio.CancelledError:
pass
answered Jan 11 '18 at 9:53
Wojciech MigdaWojciech Migda
40749
40749
periodic
should probably useloop.time()
in preference totime.time()
becauseloop.time()
is the time reference internally used byasyncio.sleep()
.loop.time()
returns monotonic time, whiletime.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.
– user4815162342
Feb 4 '18 at 21:29
add a comment |
periodic
should probably useloop.time()
in preference totime.time()
becauseloop.time()
is the time reference internally used byasyncio.sleep()
.loop.time()
returns monotonic time, whiletime.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.
– user4815162342
Feb 4 '18 at 21:29
periodic
should probably use loop.time()
in preference to time.time()
because loop.time()
is the time reference internally used by asyncio.sleep()
. loop.time()
returns monotonic time, while time.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.– user4815162342
Feb 4 '18 at 21:29
periodic
should probably use loop.time()
in preference to time.time()
because loop.time()
is the time reference internally used by asyncio.sleep()
. loop.time()
returns monotonic time, while time.time()
returns wallclock time. The two will differ e.g. when a system administrator modifies the date on the system, or when NTP adjusts wallclock time.– user4815162342
Feb 4 '18 at 21:29
add a comment |
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
@periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
add a comment |
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
@periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
add a comment |
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
@periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
Alternative version with decorator for python 3.7
import asyncio
import time
def periodic(period):
def scheduler(fcn):
async def wrapper(*args, **kwargs):
while True:
asyncio.create_task(fcn(*args, **kwargs))
await asyncio.sleep(period)
return wrapper
return scheduler
@periodic(2)
async def do_something(*args, **kwargs):
await asyncio.sleep(5) # Do some heavy calculation
print(time.time())
if __name__ == '__main__':
asyncio.run(do_something('Maluzinha do papai!', secret=42))
answered Nov 14 '18 at 5:07
Fernando José Esteves de SouzaFernando José Esteves de Souza
1
1
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f37512182%2fhow-can-i-periodically-execute-a-function-with-asyncio%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
gist.github.com/hirokiky/f4dae78b6d637f078e1c
– Arunmu
May 29 '16 at 16:22
Why do you need to move from tornado? They can work together, no? tornadoweb.org/en/stable/asyncio.html
– cricket_007
May 29 '16 at 16:29
Just add
await asyncio.sleep(time)
to your function.– shongololo
Jul 2 '16 at 5:16
Same with Twisted, no
LoopingCall
implementation.– zgoda
Jun 29 '18 at 14:16