When Query Rewrite Changes the User's Question in a RAG Pipeline

I started investigating Quivr's RAG pipeline under the following local configuration:

• qwen2:7b as the main LLM
• qwen2.5-coder:32b for comparison
• nomic-embed-text for embeddings

I used this simple question:

When is the support account reactivated?

The source material contained one fact:

The support account is reactivated immediately after a successful card payment.

Running the script for the first time, I got this answer:

The support account is reactivated immediately after a successful card payment. The exact date of reinstatement would depend on when such a successful card payment occurs.

The answer above did provide the correct operational fact. However, the sentence after it is unnecessary and slightly confusing. Looking at the trace I added, the correct source was retrieved but the question changed before retrieval. The system rewrote the question as:

Determine the date when the support account will be reinstated.

The original question asks about the condition or timing of reactivation while the rewritten version changed the request to be about a calendar date.

I continued the same test to 10 runs in total. The rewrite changed the question to calendar-date intent in all 10.

Another example I got:

The support account is reactivated immediately after a successful card payment. The exact date of reinstatement would depend on when such a successful card payment occurs.

Date of Reinstatement:

• Immediately after a successful card payment

Note: Since there's no specific date mentioned for the transaction that triggers reactivation, we cannot determine an exact date without additional context or information about when the next successful card payment will take place.

In all the 10 runs of this case with qwen2:7b, the rewrite stage was the first failure stage.

I then ran it 11 times with qwen2.5-coder:32b and the rewrite stage preserved the original question. This led me to initially think that rewrite reliability might be model-dependent under the tested configurations.

I decided to test more question types then retest selected failures with the other model and a strengthened rewrite prompt.

But before continuing the tests, I traced what the rewrite stage actually does in Quivr to understand why it exists.

What rewrite does in Quivr

The rewrite stage reformulates the user’s current question into a standalone task which is then used for retrieval.

In Quivr, it does 4 things:

1. It takes the current task.
2. It includes the filtered chat history.
3. It asks the LLM to reformulate the task.
4. It replaces the task definition with the LLM’s response.

Strengthened Prompt Attempt

I wanted to separate prompt contract from model behaviour instead of changing the model and prompt at the same time. I changed only the rewrite prompt to see whether the result changed.

I used this strengthened prompt:

Given a chat history and the latest user task which might reference context in the chat history, formulate a standalone task which can be understood without the chat history.

Preserve the meaning of the original task exactly.
Do not add, remove, narrow, broaden or infer requirements.

Preserve the requested answer type and all timing, conditions, qualifiers, constraints and modality.
Do not convert a question about when something happens into a request for a calendar date unless the original task explicitly asks for a calendar date.

If the task is already standalone and contains no unresolved references, return it unchanged.

Do NOT complete the task.
Do not output your reasoning, just the task.

I ran it 10 times and it still changed the original question

When is the support account reactivated?

to:

Determine the date when the support account was reactivated.

Strengthening the rewrite contract didn't improve this case with qwen2:7b. The original prompt produced semantic drift and the strengthened prompt also produced semantic drift, despite explicitly instructing the model not to convert a timing question into a calendar-date request and to leave an already standalone task unchanged.

At this point, I thought the observed rewrite behaviour was model-dependent under the tested configurations and strengthening the rewrite prompt alone didn't fix the 7B behaviour.

Test more cases

I restored the original rewrite prompt and tested different question types to see whether the qwen2:7b failure extended beyond the original question. I isolated the rewrite stage and ignored generation because rewrite was my focus for this experiment.

Case 1: How long does account reactivation take?

This question tests a different requested answer type "duration". I ran it 5 times and all preserved the semantic meaning. The exact wording varied but all runs still asked how long reactivation takes rather than asking what date it happens.

Rewrite:
Determine the duration required for an account reactivation process.

This is evidence that the 7B rewrite issue is not simply “it corrupts temporal questions.” At this point I suspected the issue might be more specifically related to how the model interprets the ambiguous word "when."

Case 2: What causes the support account to be reactivated?

I ran this question 5 times and it preserved cause/condition intent. It didn't turn the question into a date, duration or procedure request.

Rewrite:
Identify the factors that lead to the restoration of a support account's status.

Case 3: Can the support account be reactivated before payment succeeds?

I ran this question 6 times and it led to semantic preservation failure. It changed the requested outcome "account reactivated" into process initiation "process initiated". However, it preserved the "before payment succeeds" constraint.

Rewrite:
Can the process for reviving a support account be initiated prior to the successful completion of a payment?

This showed that my interpretation after the 2 previous cases was too narrow. The problem wasn't just when being interpreted as a date.

Case 4: What do I need to do to reactivate the support account?

I ran this question 5 times and it preserved the meaning "what the user must do to reactivate the account".

Rewrite:
What steps are required to reactivate a support account?

Case 5: On what date will the support account be reactivated?

I ran it 5 times and all runs preserved the calendar-date intent. I observed no semantic drift here unlike with the original when question.

Rewrite:
Determine the specific date when the support account will be reinstated.

I also repeated this case 5 times with the strengthened prompt and all runs preserved the explicit calendar-date intent.

Case 6: When should I contact support if the account is not reactivated?

I ran it 5 times and got different results.

For 2 runs:

Rewrite:
Determine the appropriate time to reach out to customer support for account reactivation issues.

The rewrite preserved the customer support action and the timing. "Account reactivation issues" could include many situations besides the account remaining unactivated so the rewrite didn't preserve the specific escalation condition.

For 3 runs:

Rewrite:
When should you reach out to customer support if your account remains inactive?

All 5 retained the support escalation timing but 2 broadened the trigger from the specific condition “if the account is not reactivated” to general “account reactivation issues”.

Retest failed cases

Case 3: Can the support account be reactivated before payment succeeds?

Change model to qwen2.5-coder:32b

I ran it 5 times with qwen2.5-coder:32b and all runs preserved the meaning.

Rewrite:
Can a support account be reactivated before payment is successful?

Change to strengthened prompt

I then ran the case with qwen2:7b and changed only the rewrite prompt. Across 16 runs, 12 preserved the operational meaning and 4 showed semantic drift. The failed runs included the original process-initiation shift and a separate change from support account to support service for an account.

12 runs preserved the operational meaning:

Rewrite:
Can the support account be reactivated prior to the successful completion of a payment process?

2 failed runs had the same failure as the original prompt:

Rewrite:
Can the process for reviving a support account be initiated prior to the successful completion of a payment?

1 failed run changed "support account" to "support service for an account":

Rewrite:
Can the support service for an account be reinstated prior to the successful completion of a payment process?

1 failed run that produced a strange rewrite:

Rewrite:
Can the support account's activation status be changed to reactivation prior to the successful completion of a payment process?

For this case, strengthening the rewrite contract substantially changed the observed qwen2:7b behaviour. Most runs preserved the operational distinction, whereas none of the 6 original-prompt runs did. Semantic drift still occurred in 4 of 16 strengthened-prompt runs so the prompt change didn't make rewriting reliable.

Case 6: When should I contact support if the account is not reactivated?

I ran this case 6 times with the strengthened rewrite prompt. All 6 runs preserved the contact-support action and timing intent but none preserved the exact escalation condition. "If the account is not reactivated" became either:

Rewrite:
Determine the appropriate time to reach out to customer support for account reactivation issues.

or:

Rewrite:
Determine the appropriate time to reach out to customer support for account reactivation status.

Someone could contact support about reactivation status or a reactivation issue even when the account hasn't yet met the specific condition of remaining unreactivated.

The strengthened prompt therefore didn't improve this case in the observed sample. It produced worse semantic preservation than the original prompt sample. Adding more explicit semantic-preservation instructions didn't reliably fix the issue.

Conclusion

Under the tested Quivr configuration, qwen2:7b showed repeatable but selective semantic-preservation failures across several temporal and operational question types. qwen2.5-coder:32b didn't reproduce the 2 failure patterns tested against it.

Strengthening the rewrite contract had case-dependent effects: no improvement on the original "When" case, partial improvement on the prerequisite case and no improvement on the escalation-threshold case.

Explicit calendar-date intent was preserved correctly under both prompts so the problem wasn't simply an inability to handle temporal or date-oriented questions.

This investigation left me with a different architectural question: Why is Quivr invoking an LLM rewrite for already-standalone single-turn questions when the rewrite prompt itself says to leave them unchanged if no reformulation is needed?