Point-in-time retrieval: one policy, three right answers

Use point-in-time retrieval to answer for the right date, preserve the old rule, and show exactly which version applied.

On April 10, a customer asks a support agent, “What was the refund window for my March 20 order?” The knowledge base holds two versions of the refund policy. Both talk about refund windows, both match the question closely, and only one was in force on March 20.

A fictional policy built for illustration. All dates fall in one year and use the policy's stated time zone.

Illustrative timeline, not a product interface: the two validity intervals with recorded and last-modified markers, and how a current, as-of, and what-changed question each resolve to a different version.

The fictional policy adds one rule: the window in force on the order date governs that order. Under that rule, the answer is 14 days. An agent that returns the current version says 30 days, cites a real document, and sounds completely sure of itself.

That gap is what point-in-time retrieval closes. It means fetching the version of a source that was valid at a specific date, not the version that's newest or closest in wording. The work starts before retrieval, with one decision: which date is this question actually about?

Three questions that share one policy

The same policy can answer three different questions. Each needs a different retrieval behavior, and each has a different right answer in the fixture above.

The first column is the real decision. A question tied to a dated event, such as an order, a signed contract, or an incident, is usually an as-of question. The user rarely says “as of.” If it's unclear which date governs, the agent should ask instead of quietly picking one.

Grounding still matters here. An answer that cites the data behind it is the baseline. Time adds a second test: the citation has to point to the version in force on the right date.

“What's the refund window?” means the version in force now

A current answer needs the effective version, the one in force at the moment of asking. That isn't always the most recent document. In the fixture, v2 was recorded on March 25 but didn't take effect until April 1.

Ask the same question on March 28 and the difference shows. A retriever that prefers the newest document returns 30 days. The right current answer on March 28 is 14 days, because v1 is still in force.

The last modified date causes a similar slip. Someone fixed formatting in v2 on April 8, so its modified date reads April 8. Its effective date is still April 1. An index with a single “last updated” field can't tell a typo fix from a policy change.

Database researchers separate these with two clocks. Valid time is when a fact is true in the business. Recorded time, also called transaction time, is when the system stored it. Neeraj Yadav's MemStrata preprint traces this bi-temporal idea to Snodgrass and Ahn's 1985 taxonomy, later standardized in SQL:2011's system-versioned and application-period tables.

For the refund policy, each version needs at least three dates: when it takes effect, when it stops, and when it was recorded. Edits that don't change the rule shouldn't touch the first two.

“What was the window for my March 20 order?” needs a validity interval

A validity interval is the span of time during which a version is in force. Writing the boundaries precisely is most of the work. “Through March 31” should become “until the start of April 1,” in the policy's stated time zone.

For this contiguous pair, start included and end excluded assigns the April 1 boundary to v2. Other histories still need checks for gaps and overlaps. Half-open notation alone doesn't prevent bad version metadata.

Resolving the question is then mechanical. March 20 falls inside v1's interval, so the answer is 14 days, cited to v1. The agent should say which date it resolved against, so the customer can see why the answer isn't today's 30 days.

Two limits keep that answer honest. First, the order-date rule belongs to this fictional policy. Real terms might key on delivery, invoice, or contract date, or add transition clauses. That's the policy owner's call, not the retriever's.

Second, 14 days is the window, not a refund decision. Whether this customer can still get a refund on April 10 depends on when the count starts, any exceptions, and the order's facts. Retrieval answers “which rule applied.” Whatever process decides eligibility answers the rest.

Why the closest match isn't the version in force

Similarity search ranks passages by what they're about, not when they applied. RAG, or retrieval augmented generation, usually starts there. The MemStrata preprint explains that RAG “retrieves both the stale and the current value with near-identical embedding similarity” when a fact changes. Its example is a service port that moved from 8000 to 8080.

The two refund versions have the same problem. They share nearly every word except the number. Similarity alone has no basis for choosing between them.

That doesn't mean every retrieval system fails at this. Many pipelines attach effective-date metadata and filter on it before or after ranking. Some designs fold time into ranking instead, which helps, but ranking prefers one result without ruling the other out. If a wrong-date version must never answer, you want an explicit interval check you can inspect.

MemStrata's own design shows the storage side. It retires superseded facts instead of deleting them, recording when each was valid and what replaced it. The paper also states its limits. Its benchmarks use ingestion order as a stand-in for time, and an as-of retrieval mode is future work.

“What changed?” compares versions, not passages

A what-changed question doesn't want a single version. It wants the transition between two, stated as a small record. That's the field, old and new values, the effective date, and both version IDs.

For the fixture, that record reads: refund window, 14 days to 30 days, effective April 1, from v1 to v2. Two tempting errors sit nearby. One reports April 8 because that's the last edit. The other reports March 25 because that's when v2 appeared.

March 25 can still be useful, labeled as the date the change was recorded or announced. It just isn't the date the rule changed.

VersionRAG, a preprint by Daniel Huwiler, Kurt Stockinger, and Jonathan Fürst at Zurich University of Applied Sciences, takes this approach for technical documents. It models version nodes and change nodes, and uses version-aware filtering to keep retrieval inside a specified version. The authors also name a limit: the system handles discrete version numbers but can't map calendar time to versions.

That limit belongs to that architecture, not to versioned retrieval in general. It's also a useful warning for your own data. If versions carry labels but no effective dates, you can answer “what changed in v2” but not “what changed in April.” The fixture needs both.

Showing the reader which version produced an answer is a provenance job. For the broader controls around that evidence, see provenance, identity scope, and audit in agent memory. A provenance trail still needs the right effective date; it doesn't select that date for you.

When the old version isn't there

An as-of answer needs an authoritative historical source. If v1 was overwritten and no authorized copy remains, retrieval can't establish what it said. A similar old ticket or a model's memory isn't a verified replacement.

The honest answer in that case is partial. The agent can say the current window is 30 days, effective April 1, per v2. It can say a different rule may have applied before that date, and that it can't confirm what it was. Then it routes the question to someone who can.

Keeping history isn't the same as keeping everything forever. Retention schedules, contracts, and privacy obligations can require deleting old records. As-of answers only work inside whatever your organization is allowed to keep, so include a date before the earliest retained version in your tests.

Access is a separate check too. A version being valid on March 20 doesn't mean today's user may read it. An internal draft, or a superseded internal price list, stays behind the permissions that apply when the question is asked.

Completeness and version history are also different properties. Work on closing gaps in the knowledge base adds answers that were missing. Version discipline keeps existing answers dated instead of overwritten. Which source wins a conflict, and who keeps each source fresh, are separate decisions again.

What to test before you trust an as-of answer

The fixture turns into an acceptance set without much effort. Each case below states the date it's asked on, the question, and the output that should count as a pass. These are proposed test cases, not results from any system.

A pass needs every field, not just the number. A response that says “14 days” without naming v1 and the date it used hasn't shown its work. You can't audit it later.

Use the recorded-but-not-effective row to expose a current-only shortcut. Then use the historical row to check whether the system can return v1 after v2 becomes current. Passing one doesn't establish the other.

When evaluating Computer, by DevRev, ask whether the proposed setup can reproduce those different answers and cite each version's in-force dates. If the history is missing or access is denied, can it preserve those limits in the output? Ask DevRev which parts depend on source metadata, configuration, or human review. No native temporal-query or version-store capability is established by these questions.

Time-aware retrieval sits within the wider architecture for organizational memory. Here, the narrower requirement is that changing today's answer must not erase the evidence for yesterday's.

Keep the historical answer testable

Freeze the fictional policy versions and expected outputs before you run the acceptance set. Save each actual answer beside its cited version, effective interval, and date used. When a later version becomes current, rerun the historical rows without rewriting their expected answers to match it.

A current answer can change correctly while the March 20 answer stays at 14 days. Your test should prove both, not force them to agree.

Once the dated answer resolves correctly, set controls for how agent memory preserves and audits its evidence as policies change.