Desired result
Note the list of citations at the end.
Background
I can use perplexity sonar models in Open WebUI via openrouter, and their responses have inline [k] references, but they don’t list the actual citations.
Here I would like to hack that into the code.
First we do a test request to openrouter:
export OPENROUTER_API_KEY="blabla"
curl https://openrouter.ai/api/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d '{
"model": "perplexity/sonar",
"messages": [
{
"role": "user",
"content": "what’s the latest news around perplexity the app?"
}
]
}'… which shows me that we get citations at the top-level, same as choices
Code changes
I then worked through middleware.py and ended up in process_chat_response() - post_response_handler() - stream_body_handler().
Adding the following three changes, and then restarting Open WebUI should render a citations list as shown in the example screenshot above.
diff --git a/backend/open_webui/utils/middleware.py b/backend/open_webui/utils/middleware.py
index 4070bc697..22914bffc 100644
--- a/backend/open_webui/utils/middleware.py
+++ b/backend/open_webui/utils/middleware.py
@@ -1577,6 +1577,8 @@ async def process_chat_response(
response_tool_calls = []
+ # cpbotha: init citations list
+ citations_list = None
async for line in response.body_iterator:
line = line.decode("utf-8") if isinstance(line, bytes) else line
data = line
@@ -1617,6 +1619,12 @@ async def process_chat_response(
},
)
else:
+ # cpbotha: check here if there are any citations and store them for rendering
+ if data.get("citations", None):
+ citations_list = "\n\n**Citations**\n" + "\n".join(
+ [f"{i + 1}. {c}" for i, c in enumerate(data.get("citations", []))]
+ )
+
choices = data.get("choices", [])
if not choices:
error = data.get("error", {})
@@ -1827,6 +1835,20 @@ async def process_chat_response(
log.debug("Error: ", e)
continue
+ # cpbotha: if we have a citations list, add it to the content[_blocks]
+ if citations_list is not None:
+ # follow exactly the same append to content AND content_blocks pattern as above for delta.get("content")
+ content = f"{content}{citations_list}"
+ if not content_blocks:
+ content_blocks.append(
+ {
+ "type": "text",
+ "content": "",
+ }
+ )
+
+ content_blocks[-1]["content"] += f"{citations_list}"
+
if content_blocks:
# Clean up the last text block
if content_blocks[-1]["type"] == "text":