You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Store Celery task events in a datastore like Redis #1192
Is your feature request related to a problem? Please describe.
Storing a large number of tasks e.g. 1M+ in memory makes Flower super slow. Additionally, given the fact that internally it uses an LRUCache whenever we schedule a ton of tasks (more than the in memory cache holds) we lose information about the tasks, old tasks that have been scheduled but not picked up by workers yet will be cleared out (this is common with lazy-queues that are consumed slowly).
Describe the solution you'd like
I think it would be great to shift the cache to a datastore like Redis, this would make Flower more responsive and allow for storing task events as long as they are needed.
Would you be open to something like that?
Reacted by Brian Atkinson, Gabriel Felvinczi, Peter Slovak, Pablo A. García, Jakub Stawowy, IProudNoob, logofn, denisolyanyuk, Igor Fominykh, Florian Demmer and 1 more
Activity