Evidence Freshness Score
CoreCube uses freshness to resolve a common search problem: when two results are similarly relevant, the newer evidence is often more useful. At the same time, an older policy, decision, or technical reference should not disappear merely because it is old.
The freshness score therefore follows three principles:
- Relevance comes first. Freshness cannot turn an unrelated result into a good result.
- Age applies a bounded penalty. Old evidence is demoted, never excluded.
- The source version's age matters. Re-embedding, reindexing, or syncing unchanged content does not make it look new.
The calculation at a glance
Two independent signals meet at the end of retrieval:
The semantic rank answers how well the evidence matches the query. The freshness multiplier answers how much age should demote it. CoreCube multiplies them to produce the final score.
Where freshness runs in the retrieval pipeline
Freshness is applied after semantic reranking and before CoreCube selects the final chunks:
CoreCube does not use freshness to remove or reorder candidates before the reranker. Doing so could hide highly relevant historical evidence before semantic relevance has been evaluated.
The reranker returns an ordered candidate pool without immediately reducing it to the requested final result count. The freshness finalizer converts the reranker positions into rank relevance, applies each document's freshness multiplier, sorts by the combined final score, and only then selects the final top-K chunks.
By default, the pool reaching the freshness finalizer is:
max(24, 3 × finalTopK)
It is capped by the number of retrieved candidates and the reranker's supported maximum. If the reranker is disabled or unavailable, the same finalizer uses the fused vector and full-text retrieval order instead. Freshness is therefore applied exactly once in every outcome.
Step 1: determine the evidence date
CoreCube resolves one durable evidence date for each content version. It uses the first valid value in this order:
- Modification date stored inside the document.
- Modification date reported by the source system or connector.
- Creation date stored inside the document.
- Creation date reported by the source system or connector.
- The time CoreCube first observed that exact content version.
For example, CoreCube can read PDF Info/XMP dates, Office and OpenDocument package metadata, RTF document information, HTML metadata, and Markdown front matter. Formats without a safe universal date contract use connector metadata or the first-observed fallback.
The evidence date belongs to the content version. These operations do not reset it:
- Re-embedding with another model.
- Rebuilding a vector index.
- Changing the chunking strategy.
- Syncing a source when the content did not change.
This prevents routine processing from making historical evidence appear new.
Step 2: calculate the freshness multiplier
The freshness multiplier uses the evidence age, a half-life, and a maximum penalty:
age = max(0, ranking time - evidence date)
decay = 2 ^ (-age / half-life)
freshness multiplier = 1 - maximum penalty × (1 - decay)
If a connection disables recency with a half-life of 0, the multiplier is always 1.0.
The default maximum penalty is 0.4675. That creates a permanent lower bound:
minimum multiplier = 1 - 0.4675 = 0.5325
Even extremely old evidence therefore retains at least 53.25% of its rank relevance. Freshness never drives a result toward zero.
90-day half-life
The organization default is a 90-day half-life. This does not mean that CoreCube cuts the entire score in half every 90 days.
It means that every 90 days, the remaining distance between the current multiplier and the 53.25% minimum is halved.
| Evidence age | Freshness multiplier | Plain-language meaning |
|---|---|---|
| New | 1.0000 — 100% | No age penalty |
| 30 days | 0.9036 — 90.36% | Small age penalty |
| 45 days | 0.8631 — 86.31% | About 29% of the way to the maximum |
| 90 days | 0.7663 — 76.63% | Halfway to the maximum penalty |
| 180 days | 0.6494 — 64.94% | Three-quarters of the way to the maximum |
| 360 days | 0.5617 — 56.17% | Close to the permanent minimum |
| Extremely old | 0.5325 — 53.25% | Maximum possible freshness penalty reached |
The curve starts gently and slows as it approaches the lower bound. Evidence never crosses below that bound, regardless of its age.
Step 3: convert semantic rank into relevance
Reranker providers return scores on different scales. Some return values from zero to one, while others return logits that may be negative. Multiplying those raw scores by freshness could produce incorrect ordering.
CoreCube instead uses the result's ordinal rank:
rank relevance = exp(-0.09 × (rank - 1))
| Semantic rank | Rank relevance |
|---|---|
| 1 | 1.0000 |
| 2 | 0.9139 |
| 3 | 0.8353 |
| 5 | 0.6977 |
| 8 | 0.5326 |
| 9 | 0.4868 |
Rank 1 always begins with full relevance. Each lower position decreases smoothly, without hard ranking bands or sudden boundaries.
When reranking is unavailable, CoreCube uses the fused hybrid-search order as the semantic rank.
Step 4: combine relevance and freshness
The final calculation is:
final score = rank relevance × freshness multiplier
Consider two results using the default 90-day half-life:
| Result | Semantic rank | Evidence age | Rank relevance | Freshness | Final score |
|---|---|---|---|---|---|
| A | 1 | 30 days | 1.0000 | 0.9036 | 0.9036 |
| B | 2 | New | 0.9139 | 1.0000 | 0.9139 |
Result B is slightly less relevant semantically, but its evidence is new. It finishes just ahead of the 30-day-old rank-1 result.
More precisely, with the default settings, a completely fresh rank-2 result overtakes rank 1 when the rank-1 evidence is approximately 26.4 days old.
How much can freshness change the order?
Freshness is deliberately limited. With the default rank curve and maximum penalty:
- A fresh result can move ahead by at most approximately seven positions.
- A fresh rank-8 result can only narrowly overtake an extremely old rank-1 result.
- Rank 9 can never overtake rank 1 through freshness alone.
This limit applies continuously at every depth. There are no arbitrary boundaries where adjacent results suddenly behave differently.
Choosing a half-life
The half-life describes how quickly information from a source becomes less current. It is separate from the connection's sync-health window.
| Source type | Typical direction | Reason |
|---|---|---|
| News, incidents, release status | Shorter | Information becomes outdated quickly |
| Tickets, project updates, operational records | Short to medium | Recent state is usually more useful |
| Product and engineering documentation | Medium | Changes matter, but older material may remain useful |
| Policies, standards, legal or archival records | Longer or disabled | Age alone is a weak signal of validity |
The initial settings are:
| Setting | Initial value | Meaning |
|---|---|---|
| Organization default half-life | 2160 hours | 90 days |
| Connection half-life | Inherit | Use the organization default |
Connection half-life 0 | Disabled | Apply no freshness penalty |
| Maximum freshness penalty | 0.4675 | Old evidence retains at least 53.25% |
| Rank-decay constant | 0.09 | Limits maximum movement to about seven ranks |
A connection can inherit the organization default, override it with a value from 24 to 8,760
hours, or use 0 for timeless reference content.
Use 90 days unless you can clearly explain how quickly information from that source becomes outdated. A very short half-life is a strong ranking decision, not a way to make a connector sync more often.
Freshness is not connection health
These settings answer different questions:
| Concern | Question |
|---|---|
| Connection health | Is CoreCube successfully receiving updates from this source? |
| Evidence freshness | How old is this content version, and should age affect its rank? |
| Sync schedule | How often should CoreCube check the source for changed content? |
Changing the ranking half-life does not change the sync schedule. Likewise, a successful sync does not make unchanged documents appear newer.
Missing, invalid, and future dates
- Missing date: CoreCube uses the first time it observed that content version.
- Malformed date: CoreCube records the rejection and tries the next timestamp source.
- Known sentinel date: A format-specific sentinel such as
1601-01-01is rejected. - Unusually old valid date: The date remains valid. CoreCube does not make it appear newer merely because it predates an arbitrary cutoff.
- Future date: Age is clamped to zero, so the document cannot receive more than a
1.0multiplier.
Duplicate content from multiple sources
CoreCube ranks identical content once so duplicates do not occupy several reranker and result slots. Only duplicate sources visible to the caller participate in the decision.
The canonical evidence date uses the newest valid native or source-system modification date. The primary visible source supplies the connection half-life. Source authority and stable identifiers resolve remaining ties.
What the diagnostics show
Query diagnostics expose the values needed to explain the final order:
- Effective evidence date and how it was obtained.
- Evidence age and captured ranking time.
- Effective half-life and maximum penalty.
- Semantic rank and rank relevance.
- Freshness multiplier and final score.
- Ranking algorithm version.
- Rejected metadata or observation-fallback reason.
Quick reference
| Question | Answer |
|---|---|
| Does freshness remove old documents? | No. It applies a bounded ranking penalty. |
| Does a 90-day half-life cut the score in half? | No. It halves the remaining distance to the 53.25% minimum. |
| Does syncing reset evidence age? | Only when the source content version actually changes. |
| Does re-embedding reset evidence age? | No. |
| What happens when no source date exists? | CoreCube uses the content version's first-observed time. |
| Can a future timestamp increase a score above 1.0? | No. Age is clamped to zero. |
| Can freshness overpower semantic relevance? | Only within the bounded maximum of approximately seven ranks. |
| Is ranking half-life the same as source-health SLA? | No. They are separate settings with different purposes. |