reflector: add generation tracking for stale cache entry detection#921
Open
kaddynator wants to merge 4 commits into
Open
reflector: add generation tracking for stale cache entry detection#921kaddynator wants to merge 4 commits into
kaddynator wants to merge 4 commits into
Conversation
Each watch reconnect increments a _generation counter on the reflector. Cache entries are stamped with the generation in which they were last seen via a parallel _entry_generations dict (raw k8s resource dicts are not modified). A new is_stale(ref_key) method returns True when an entry's generation is behind the current watch cycle, indicating it has not been confirmed since the last reconnect. Callers such as poll() can use this to decide whether to re-query the API rather than act on potentially stale data. No behaviour change for existing callers that do not call is_stale().
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
After a watch reconnect, the reflector cache may still contain entries from the previous watch cycle that have not yet been confirmed by a new event. There is currently no way for callers to distinguish a freshly-confirmed entry from one that may be stale (e.g. a pod that was deleted while the watch was down).
What this adds
A lightweight generation counter on
ResourceReflector:self._generation— incremented after each successful_list_and_update, marking a new watch cycleself._entry_generations— a parallel dict stamping each cache entry with the generation in which it was last seen. Raw k8s resource dicts inself.resourcesare not modified.is_stale(ref_key)— returnsTruewhen an entry carries a generation older than the current watch cycleThread safety note
self.resourcesandself._entry_generationsare updated on consecutive lines, so there is a brief window of inconsistency between the two dicts.is_stale()is advisory: a caller that races here will either see the old generation (slightly conservative — may re-query the API when it did not need to) or the new one (correct). It will never see an inconsistent state that causes a crash or incorrect action. This matches the existing comment onself.resourcesthat callers accept a potentially out-of-date view.How to use
No breaking changes
Existing callers that do not call
is_stale()are unaffected. The generation counter adds two small dicts and one integer increment per watch cycle.