-
Notifications
You must be signed in to change notification settings - Fork 537
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
parameterize pandas datetime caching
Summary: Detection algortihms is memory intensive and recently started ooming after onboarding ~6k prophet detectors (D53922840). Strobelight points to convert_listlike_datetimes using [11%](https://fburl.com/scuba/pyperf_alloc/on_demand/65bwy03b) heap now and [20%](https://fburl.com/scuba/pyperf_alloc/on_demand/rzwgukjx) during the period the tier was OOMing. Pandas [caches](https://github.com/pandas-dev/pandas/blob/main/pandas/core/tools/datetimes.py#L1008) datetime conversions internally by default to avoid recomputation. This behaviour is parameterized. This diff turns it off to since DA is memory bound so it's desirable to trade some compute for more memory buffer. There will be no performance implications for other users of Kats. Differential Revision: D53941075 fbshipit-source-id: ec72922b5351937a24c6846f979a86a74509c7b0
- Loading branch information
1 parent
5be3003
commit 6141d6c
Showing
1 changed file
with
17 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters