refactor(connection): 合并旧逻辑新的业务逻辑代码

update：仅在 MQTT 服务可用时显示设备在线状态列

README.md

@@ -183,8 +183,8 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
 #### 🚀 部署方式选择
 | 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 | 
 |---------|------|---------|---------|---------|---------|
-| **最简化安装** | 智能对话、IOT、MCP、视觉感知 | 低配置环境，数据存储在配置文件，无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G，如果全API，要2核2G | - | 
+| **最简化安装** | 智能对话、单智能体管理 | 低配置环境，数据存储在配置文件，无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G，如果全API，要2核2G | - | 
-| **全模块安装** | 智能对话、IOT、MCP接入点、声纹识别、视觉感知、OTA、智控台 | 完整功能体验，数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 
+| **全模块安装** | 智能对话、多用户管理、多智能体管理、智控台界面操作 | 完整功能体验，数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 
 
 常见问题及相关教程，可参考[这个链接](./docs/FAQ.md)
 
@@ -211,10 +211,10 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 
 | 模块名称 | 入门全免费设置 | 流式配置 |
 |:---:|:---:|:---:|
-| ASR(语音识别) | FunASR(本地) | 👍FunASR(本地GPU模式) |
+| ASR(语音识别) | FunASR(本地) | 👍XunfeiStreamASR(讯飞流式) |
-| LLM(大模型) | ChatGLMLLM(智谱glm-4-flash) | 👍AliLLM(qwen3-235b-a22b-instruct-2507) 或 👍DoubaoLLM(doubao-1-5-pro-32k-250115) |
+| LLM(大模型) | glm-4-flash(智谱) | 👍qwen-flash(阿里百炼) |
-| VLLM(视觉大模型) | ChatGLMVLLM(智谱glm-4v-flash) | 👍QwenVLVLLM(千问qwen2.5-vl-3b-instructh) |
+| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen2.5-vl-3b-instructh(阿里百炼) |
-| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) 或 👍AliyunStreamTTS(阿里云流式语音合成) |
+| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山流式) |
 | Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
 | Memory(记忆功能) | mem_local_short(本地短期记忆） | mem_local_short（本地短期记忆） |
 
@@ -260,7 +260,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 ---
 
 ## 产品生态 👬
-小智是一个生态，当你使用这个产品时，也可以看看其他在这个生态圈的[优秀项目](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE)
+小智是一个生态，当你使用这个产品时，也可以看看其他在这个生态圈的[优秀项目](https://github.com/78/xiaozhi-esp32/blob/main/README_zh.md#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE)
 
 ---
 

README_de.md

@@ -181,8 +181,8 @@ Dieses Projekt bietet zwei Bereitstellungsmethoden. Bitte wählen Sie basierend
 #### 🚀 Auswahl der Bereitstellungsmethode
 | Bereitstellungsmethode | Funktionen | Anwendungsszenarien | Deployment-Dokumente | Konfigurationsanforderungen | Video-Tutorials |
 |---------|------|---------|---------|---------|---------|
-| **Vereinfachte Installation** | Intelligenter Dialog, IOT, MCP, visuelle Wahrnehmung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 Kerne 4GB bei Verwendung von `FunASR`, 2 Kerne 2GB bei allen APIs | - |
+| **Vereinfachte Installation** | Intelligenter Dialog, Einzel-Agenten-Verwaltung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 Kerne 4GB bei Verwendung von `FunASR`, 2 Kerne 2GB bei allen APIs | - |
-| **Vollständige Modulinstallation** | Intelligenter Dialog, IOT, MCP-Endpunkte, Stimmabdruckerkennung, visuelle Wahrnehmung, OTA, intelligente Steuerkonsole | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](./docs/dev-ops-integration.md) | 4 Kerne 8GB bei Verwendung von `FunASR`, 2 Kerne 4GB bei allen APIs| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
+| **Vollständige Modulinstallation** | Intelligenter Dialog, Mehrbenutzerverwaltung, Mehr-Agenten-Verwaltung, Intelligente Steuerkonsole-Bedienung | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](./docs/dev-ops-integration.md) | 4 Kerne 8GB bei Verwendung von `FunASR`, 2 Kerne 4GB bei allen APIs| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
 
 Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](./docs/FAQ.md)
 
@@ -209,10 +209,10 @@ Websocket-Schnittstellenadresse: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 
 | Modulname | Einstiegslevel Kostenlose Einstellungen | Streaming-Konfiguration |
 |:---:|:---:|:---:|
-| ASR (Spracherkennung) | FunASR (Lokal) | 👍FunASR (Lokaler GPU-Modus) |
+| ASR (Spracherkennung) | FunASR (Lokal) | 👍XunfeiStreamASR (Xunfei-Streaming) |
-| LLM (Großes Modell) | ChatGLMLLM (Zhipu glm-4-flash) | 👍AliLLM (qwen3-235b-a22b-instruct-2507) oder 👍DoubaoLLM (doubao-1-5-pro-32k-250115) |
+| LLM (Großes Modell) | glm-4-flash (Zhipu) | 👍qwen-flash (Alibaba Bailian) |
-| VLLM (Vision Large Model) | ChatGLMVLLM (Zhipu glm-4v-flash) | 👍QwenVLVLLM (Qwen qwen2.5-vl-3b-instructh) |
+| VLLM (Vision Large Model) | glm-4v-flash (Zhipu) | 👍qwen2.5-vl-3b-instructh (Alibaba Bailian) |
-| TTS (Sprachsynthese) | ✅LinkeraiTTS (Lingxi-Streaming) | 👍HuoshanDoubleStreamTTS (Volcano Dual-Stream-Sprachsynthese) oder 👍AliyunStreamTTS (Alibaba Cloud Streaming-Sprachsynthese) |
+| TTS (Sprachsynthese) | ✅LinkeraiTTS (Lingxi-Streaming) | 👍HuoshanDoubleStreamTTS (Volcano-Streaming) |
 | Intent (Absichtserkennung) | function_call (Funktionsaufruf) | function_call (Funktionsaufruf) |
 | Memory (Gedächtnisfunktion) | mem_local_short (Lokales Kurzzeitgedächtnis) | mem_local_short (Lokales Kurzzeitgedächtnis) |
 
@@ -258,7 +258,7 @@ Wenn Sie ein Softwareentwickler sind, finden Sie hier einen [Offenen Brief an En
 ---
 
 ## Produktökosystem 👬
-Xiaozhi ist ein Ökosystem. Wenn Sie dieses Produkt verwenden, können Sie sich auch andere [hervorragende Projekte](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE) in diesem Ökosystem ansehen
+Xiaozhi ist ein Ökosystem. Wenn Sie dieses Produkt verwenden, können Sie sich auch andere [hervorragende Projekte](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#related-open-source-projects) in diesem Ökosystem ansehen
 
 ---
 

README_en.md

@@ -181,8 +181,8 @@ This project provides two deployment methods. Please choose based on your specif
 #### 🚀 Deployment Method Selection
 | Deployment Method | Features | Applicable Scenarios | Deployment Docs | Configuration Requirements | Video Tutorials | 
 |---------|------|---------|---------|---------|---------|
-| **Simplified Installation** | Intelligent dialogue, IOT, MCP, visual perception | Low-configuration environments, data stored in config files, no database required | [①Docker Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - | 
+| **Simplified Installation** | Intelligent dialogue, single agent management | Low-configuration environments, data stored in config files, no database required | [①Docker Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - | 
-| **Full Module Installation** | Intelligent dialogue, IOT, MCP endpoints, voiceprint recognition, visual perception, OTA, intelligent control console | Complete functionality experience, data stored in database |[①Docker Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](./docs/dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 
+| **Full Module Installation** | Intelligent dialogue, multi-user management, multi-agent management, intelligent console interface operation | Complete functionality experience, data stored in database |[①Docker Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](./docs/dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 
 
 
 > 💡 Note: Below is a test platform deployed with the latest code. You can burn and test if needed. Concurrent users: 6, data will be cleared daily.
@@ -208,10 +208,10 @@ Websocket Interface Address: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 
 | Module Name | Entry Level Free Settings | Streaming Configuration |
 |:---:|:---:|:---:|
-| ASR(Speech Recognition) | FunASR(Local) | 👍FunASRServer or 👍DoubaoStreamASR |
+| ASR(Speech Recognition) | FunASR(Local) | 👍XunfeiStreamASR(Xunfei Streaming) |
-| LLM(Large Model) | ChatGLMLLM(Zhipu glm-4-flash) | 👍DoubaoLLM(Volcano doubao-1-5-pro-32k-250115) |
+| LLM(Large Model) | glm-4-flash(Zhipu) | 👍qwen-flash(Alibaba Bailian) |
-| VLLM(Vision Large Model) | ChatGLMVLLM(Zhipu glm-4v-flash) | 👍QwenVLVLLM(Qwen qwen2.5-vl-3b-instructh) |
+| VLLM(Vision Large Model) | glm-4v-flash(Zhipu) | 👍qwen2.5-vl-3b-instructh(Alibaba Bailian) |
-| TTS(Speech Synthesis) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano dual-stream speech synthesis) |
+| TTS(Speech Synthesis) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
 | Intent(Intent Recognition) | function_call(Function calling) | function_call(Function calling) |
 | Memory(Memory function) | mem_local_short(Local short-term memory) | mem_local_short(Local short-term memory) |
 
@@ -256,7 +256,7 @@ If you are a software developer, here is an [Open Letter to Developers](docs/con
 ---
 
 ## Product Ecosystem 👬
-Xiaozhi is an ecosystem. When using this product, you can also check out other [excellent projects](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE) in this ecosystem
+Xiaozhi is an ecosystem. When using this product, you can also check out other [excellent projects](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#related-open-source-projects) in this ecosystem
 
 | Project Name | Project Address | Project Description |
 |:---------------------|:--------|:--------|

README_vi.md

@@ -182,8 +182,8 @@ Dự án này cung cấp hai phương pháp triển khai, vui lòng chọn theo
 #### 🚀 Lựa chọn phương pháp triển khai
 | Phương pháp triển khai | Đặc điểm | Tình huống áp dụng | Tài liệu triển khai | Yêu cầu cấu hình | Video hướng dẫn |
 |---------|------|---------|---------|---------|---------|
-| **Cài đặt tối giản** | Đối thoại thông minh, IOT, MCP, cảm nhận thị giác | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 nhân 4GB nếu dùng `FunASR`, 2 nhân 2GB nếu toàn API | - |
+| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 nhân 4GB nếu dùng `FunASR`, 2 nhân 2GB nếu toàn API | - |
-| **Cài đặt toàn bộ module** | Đối thoại thông minh, IOT, điểm truy cập MCP, nhận dạng giọng nói, cảm nhận thị giác, OTA, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](./docs/dev-ops-integration.md) | 4 nhân 8GB nếu dùng `FunASR`, 2 nhân 4GB nếu toàn API| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
+| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](./docs/dev-ops-integration.md) | 4 nhân 8GB nếu dùng `FunASR`, 2 nhân 4GB nếu toàn API| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
 
 Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](./docs/FAQ.md)
 
@@ -210,10 +210,10 @@ Công cụ kiểm tra dịch vụ: https://2662r3426b.vicp.fun/test/
 
 | Tên module | Cài đặt miễn phí cho người mới | Cấu hình streaming |
 |:---:|:---:|:---:|
-| ASR(Nhận dạng giọng nói) | FunASR(Local) | 👍FunASR(Chế độ GPU cục bộ) |
+| ASR(Nhận dạng giọng nói) | FunASR(Local) | 👍XunfeiStreamASR(Xunfei Streaming) |
-| LLM(Mô hình lớn) | ChatGLMLLM(Zhipu glm-4-flash) | 👍AliLLM(qwen3-235b-a22b-instruct-2507) hoặc 👍DoubaoLLM(doubao-1-5-pro-32k-250115) |
+| LLM(Mô hình lớn) | glm-4-flash(Zhipu) | 👍qwen-flash(Alibaba Bailian) |
-| VLLM(Mô hình lớn thị giác) | ChatGLMVLLM(Zhipu glm-4v-flash) | 👍QwenVLVLLM(Qwen qwen2.5-vl-3b-instructh) |
+| VLLM(Mô hình lớn thị giác) | glm-4v-flash(Zhipu) | 👍qwen2.5-vl-3b-instructh(Alibaba Bailian) |
-| TTS(Tổng hợp giọng nói) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Tổng hợp giọng nói streaming kép Volcano) hoặc 👍AliyunStreamTTS(Tổng hợp giọng nói streaming Alibaba Cloud) |
+| TTS(Tổng hợp giọng nói) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
 | Intent(Nhận dạng ý định) | function_call(Gọi hàm) | function_call(Gọi hàm) |
 | Memory(Chức năng bộ nhớ) | mem_local_short(Bộ nhớ ngắn hạn cục bộ) | mem_local_short(Bộ nhớ ngắn hạn cục bộ) |
 
@@ -259,7 +259,7 @@ Nếu bạn là một nhà phát triển phần mềm, đây có một [Lá thư
 ---
 
 ## Hệ sinh thái sản phẩm 👬
-Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạn cũng có thể xem các [dự án xuất sắc](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE) khác trong hệ sinh thái này
+Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạn cũng có thể xem các [dự án xuất sắc](https://github.com/78/xiaozhi-esp32?tab=readme-ov-file#related-open-source-projects) khác trong hệ sinh thái này
 
 ---
 

docs/FAQ.md

@@ -38,10 +38,10 @@ conda install conda-forge::ffmpeg
 
 | 模块名称 | 入门全免费设置 | 流式配置 |
 |:---:|:---:|:---:|
-| ASR(语音识别) | FunASR(本地) | 👍FunASR(本地GPU模式) |
+| ASR(语音识别) | FunASR(本地) | 👍XunfeiStreamASR(讯飞流式) |
-| LLM(大模型) | ChatGLMLLM(智谱glm-4-flash) | 👍AliLLM(qwen3-235b-a22b-instruct-2507) 或 👍DoubaoLLM(doubao-1-5-pro-32k-250115) |
+| LLM(大模型) | glm-4-flash(智谱) | 👍qwen-flash(阿里百炼) |
-| VLLM(视觉大模型) | ChatGLMVLLM(智谱glm-4v-flash) | 👍QwenVLVLLM(千问qwen2.5-vl-3b-instructh) |
+| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen2.5-vl-3b-instructh(阿里百炼) |
-| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) 或 👍AliyunStreamTTS(阿里云流式语音合成) |
+| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山流式) |
 | Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
 | Memory(记忆功能) | mem_local_short(本地短期记忆） | mem_local_short（本地短期记忆） |
 
@@ -69,6 +69,7 @@ VAD:
 ### 9、编译固件相关教程
 1、[如何自己编译小智固件](./firmware-build.md)<br/>
 2、[如何基于虾哥编译好的固件修改OTA地址](./firmware-setting.md)<br/>
+3、[单模块部署如何配置固件OTA自动升级](./ota-upgrade-guide.md)<br/>
 
 ### 10、拓展相关教程
 1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/>

docs/huoshan-streamTTS-voice-cloning.md

@@ -23,6 +23,8 @@
 
 ### 2.将音色资源ID分配给系统账号
 
+使用超级管理员账号登录智控台，点击顶部`参数字典`，在下拉菜单中，点击`系统功能配置`页面。在页面上勾选`音色克隆`，点击保存配置。即可在顶部菜单看到`音色克隆`按钮。
+
 使用超级管理员账号登录智控台，点击顶部【音色克隆】、【音色资源】。
 
 点击新增按钮，在【平台名称】选择“火山双流式语音合成”；

docs/mqtt-gateway-integration.md

@@ -76,6 +76,7 @@ MQTT_PORT=1883            # MQTT服务器端口
 UDP_PORT=8884             # UDP服务器端口
 API_PORT=8007             # 管理API端口
 MQTT_SIGNATURE_KEY=test   # MQTT签名密钥
+SERVER_SECRET=Te1st12134  # 服务器密钥，请保持和智控台（server.secret）一致或者和xiaozhi-server里（server.auth_key）保持一致
 ```
 请注意`PUBLIC_IP`配置，确保其与实际公网IP一致，如果有域名就填域名。
 
@@ -85,6 +86,13 @@ MQTT_SIGNATURE_KEY=test   # MQTT签名密钥
 - 注意不要用简单的密码，比如`123456`、`test`等。
 - 注意不要用简单的密码，比如`123456`、`test`等。
 
+`SERVER_SECRET` 是用生成websocket连接的认证信息。
+
+1、如果你是全模块部署，且你的智控台的参数管理里`server.auth.enabled`设置成了`true`，那么，`SERVER_SECRET`需要和智控台（`server.secret`）保持一致。
+
+2、如果你是单模块部署，且你在配置文件里把`server.auth.enabled`设置成了`true`，那么，`SERVER_SECRET`需要和配置文件里（`server.auth_key`）保持一致。
+
+
 6. 启动MQTT网关
 ```
 # 启动服务

docs/ota-upgrade-guide.md

@@ -0,0 +1,142 @@
+# 单模块部署固件OTA自动升级配置指南
+
+本教程将指导你如何在**单模块部署**场景下配置固件OTA自动升级功能，实现设备固件的自动更新。
+
+如果你已经使用**全模块部署**，请忽略本教程。
+
+## 功能介绍
+
+在单模块部署中，xiaozhi-server内置了OTA固件管理功能，可以自动检测设备版本并下发升级固件。系统会根据设备型号和当前版本，自动匹配并推送最新的固件版本。
+
+## 前提条件
+
+- 你已经成功进行**单模块部署**并运行xiaozhi-server
+- 设备能够正常连接到服务器
+
+## 第一步 准备固件文件
+
+### 1. 创建固件存放目录
+
+固件文件需要放在`data/bin/`目录下。如果该目录不存在，请手动创建：
+
+```bash
+mkdir -p data/bin
+```
+
+### 2. 固件文件命名规则
+
+固件文件必须遵循以下命名格式：
+
+```
+{设备型号}_{版本号}.bin
+```
+
+**命名规则说明：**
+- `设备型号`：设备的型号名称，例如 `lichuang-dev`、`bread-compact-wifi` 等
+- `版本号`：固件版本号，必须以数字开头，支持数字、字母、点号、下划线和短横线，例如 `1.6.6`、`2.0.0` 等
+- 文件扩展名必须是 `.bin`
+
+**命名示例：**
+```
+bread-compact-wifi_1.6.6.bin
+lichuang-dev_2.0.0.bin
+```
+
+### 3. 放置固件文件
+
+将准备好的固件文件（.bin文件）复制到`data/bin/`目录下：
+
+重要的事情说三遍：升级的bin文件是`xiaozhi.bin`，不是全量固件文件`merged-binary.bin`!
+
+重要的事情说三遍：升级的bin文件是`xiaozhi.bin`，不是全量固件文件`merged-binary.bin`!
+
+重要的事情说三遍：升级的bin文件是`xiaozhi.bin`，不是全量固件文件`merged-binary.bin`!
+
+```bash
+cp xiaozhi.bin data/bin/设备型号_版本号.bin
+```
+
+例如：
+```bash
+cp xiaozhi.bin data/bin/bread-compact-wifi_1.6.6.bin
+```
+
+## 第二步 配置公网访问地址（仅公网部署需要）
+
+**注意：此步骤仅适用于单模块公网部署的场景。**
+
+如果你的xiaozhi-server是公网部署（使用公网IP或域名），**必须**配置`server.vision_explain`参数，因为OTA固件下载地址会使用该配置的域名和端口。
+
+如果你是局域网部署，可以跳过此步骤。
+
+### 为什么要配置这个参数？
+
+在单模块部署中，系统生成固件下载地址时，会使用`vision_explain`配置的域名和端口作为基础地址。如果不配置或配置错误，设备将无法访问固件下载地址。
+
+### 配置方法
+
+打开`data/.config.yaml`文件，找到`server`配置段，设置`vision_explain`参数：
+
+```yaml
+server:
+  vision_explain: http://你的域名或IP:端口号/mcp/vision/explain
+```
+
+**配置示例：**
+
+局域网部署（默认）：
+```yaml
+server:
+  vision_explain: http://192.168.1.100:8003/mcp/vision/explain
+```
+
+公网域名部署：
+```yaml
+server:
+  vision_explain: http://yourdomain.com:8003/mcp/vision/explain
+```
+
+### 注意事项
+
+- 域名或IP必须是设备能够访问的地址
+- 如果使用Docker部署，不能使用Docker内部地址（如127.0.0.1或localhost）
+- 如果你使用了nginx反向代理，请填写对外的地址和端口号，不是本项目运行的端口号
+
+
+## 常见问题
+
+### 1. 设备收不到固件更新
+
+**可能原因和解决方法：**
+
+- 检查固件文件命名是否符合规则：`{型号}_{版本号}.bin`
+- 检查固件文件是否正确放置在`data/bin/`目录
+- 检查设备型号是否与固件文件名中的型号匹配
+- 检查固件版本号是否高于设备当前版本
+- 查看服务器日志，确认OTA请求是否正常处理
+
+### 2. 设备报告下载地址无法访问
+
+**可能原因和解决方法：**
+
+- 检查`server.vision_explain`配置的域名或IP是否正确
+- 确认端口号配置正确（默认8003）
+- 如果是公网部署，确保设备能够访问该公网地址
+- 如果是Docker部署，确保不是使用了内部地址（127.0.0.1）
+- 检查防火墙是否开放了对应端口
+- 如果你使用了nginx反向代理，请填写对外的地址和端口号，不是本项目运行的端口号
+
+### 3. 如何确认设备当前版本
+
+查看OTA请求日志，日志中会显示设备上报的版本号：
+
+```
+[ota_handler] - 设备 AA:BB:CC:DD:EE:FF 固件已是最新: 1.6.6
+```
+
+### 4. 固件文件放置后没有生效
+
+系统有30秒的缓存时间（默认），可以：
+- 等待30秒后再让设备发起OTA请求
+- 重启xiaozhi-server服务
+- 调整`firmware_cache_ttl`配置为更短的时间

main/manager-api/src/main/java/xiaozhi/common/redis/RedisKeys.java

@@ -159,4 +159,11 @@ public class RedisKeys {
     public static String getKnowledgeBaseCacheKey(String datasetId) {
         return "knowledge:base:" + datasetId;
     }
+
+    /**
+     * 获取临时注册设备标记key
+     */
+    public static String getTmpRegisterMacKey(String deviceId) {
+        return "tmp_register_mac:" + deviceId;
+    }
 }

main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java

@@ -44,6 +44,7 @@ import xiaozhi.modules.agent.entity.AgentEntity;
 import xiaozhi.modules.agent.entity.AgentTemplateEntity;
 import xiaozhi.modules.agent.service.AgentChatAudioService;
 import xiaozhi.modules.agent.service.AgentChatHistoryService;
+import xiaozhi.modules.agent.service.AgentChatSummaryService;
 import xiaozhi.modules.agent.service.AgentContextProviderService;
 import xiaozhi.modules.agent.service.AgentPluginMappingService;
 import xiaozhi.modules.agent.service.AgentService;
@@ -66,6 +67,7 @@ public class AgentController {
     private final AgentChatAudioService agentChatAudioService;
     private final AgentPluginMappingService agentPluginMappingService;
     private final AgentContextProviderService agentContextProviderService;
+    private final AgentChatSummaryService agentChatSummaryService;
     private final RedisUtils redisUtils;
 
     @GetMapping("/list")
@@ -119,6 +121,27 @@ public class AgentController {
         return new Result<>();
     }
 
+    @PostMapping("/chat-summary/{sessionId}/save")
+    @Operation(summary = "根据会话ID生成聊天记录总结并保存（异步执行）")
+    public Result<Void> generateAndSaveChatSummary(@PathVariable String sessionId) {
+        try {
+            // 异步执行总结生成任务，立即返回成功响应
+            new Thread(() -> {
+                try {
+                    agentChatSummaryService.generateAndSaveChatSummary(sessionId);
+                    System.out.println("异步执行会话 " + sessionId + " 的聊天记录总结完成");
+                } catch (Exception e) {
+                    System.err.println("异步执行会话 " + sessionId + " 的聊天记录总结失败: " + e.getMessage());
+                }
+            }).start();
+
+            // 立即返回成功响应，不等待总结生成完成
+            return new Result<Void>().ok(null);
+        } catch (Exception e) {
+            return new Result<Void>().error("启动异步总结生成任务失败: " + e.getMessage());
+        }
+    }
+
     @PutMapping("/{id}")
     @Operation(summary = "更新智能体")
     @RequiresPermissions("sys:role:normal")
@@ -186,6 +209,7 @@ public class AgentController {
         List<AgentChatHistoryDTO> result = agentChatHistoryService.getChatHistoryBySessionId(id, sessionId);
         return new Result<List<AgentChatHistoryDTO>>().ok(result);
     }
+
     @GetMapping("/{id}/chat-history/user")
     @Operation(summary = "获取智能体聊天记录（用户）")
     @RequiresPermissions("sys:role:normal")

main/manager-api/src/main/java/xiaozhi/modules/agent/dao/AiAgentChatHistoryDao.java

@@ -1,6 +1,9 @@
 package xiaozhi.modules.agent.dao;
 
+import java.util.List;
+
 import org.apache.ibatis.annotations.Mapper;
+import org.apache.ibatis.annotations.Param;
 
 import com.baomidou.mybatisplus.core.mapper.BaseMapper;
 
@@ -15,12 +18,6 @@ import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
  */
 @Mapper
 public interface AiAgentChatHistoryDao extends BaseMapper<AgentChatHistoryEntity> {
-    /**
-     * 根据智能体ID删除音频
-     *
-     * @param agentId 智能体ID
-     */
-    void deleteAudioByAgentId(String agentId);
 
     /**
      * 根据智能体ID删除聊天历史记录
@@ -35,4 +32,19 @@ public interface AiAgentChatHistoryDao extends BaseMapper<AgentChatHistoryEntity
      * @param agentId 智能体ID
      */
     void deleteAudioIdByAgentId(String agentId);
+
+    /**
+     * 根据智能体ID获取所有音频ID列表
+     *
+     * @param agentId 智能体ID
+     * @return 音频ID列表
+     */
+    List<String> getAudioIdsByAgentId(String agentId);
+
+    /**
+     * 批量删除音频
+     *
+     * @param audioIds 音频ID列表
+     */
+    void deleteAudioByIds(@Param("audioIds") List<String> audioIds);
 }

main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentChatSummaryDTO.java

@@ -0,0 +1,45 @@
+package xiaozhi.modules.agent.dto;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import lombok.Data;
+
+/**
+ * 智能体聊天记录总结DTO
+ */
+@Data
+@Schema(description = "智能体聊天记录总结对象")
+public class AgentChatSummaryDTO {
+
+    @Schema(description = "会话ID")
+    private String sessionId;
+
+    @Schema(description = "智能体ID")
+    private String agentId;
+
+    @Schema(description = "总结内容")
+    private String summary;
+
+    @Schema(description = "总结状态")
+    private boolean success;
+
+    @Schema(description = "错误信息")
+    private String errorMessage;
+
+    public AgentChatSummaryDTO() {
+        this.success = true;
+    }
+
+    public AgentChatSummaryDTO(String sessionId, String agentId, String summary) {
+        this.sessionId = sessionId;
+        this.agentId = agentId;
+        this.summary = summary;
+        this.success = true;
+    }
+
+    public AgentChatSummaryDTO(String sessionId, String errorMessage) {
+        this.sessionId = sessionId;
+        this.errorMessage = errorMessage;
+        this.success = false;
+    }
+
+}

main/manager-api/src/main/java/xiaozhi/modules/agent/service/AgentChatSummaryService.java

@@ -0,0 +1,15 @@
+package xiaozhi.modules.agent.service;
+
+/**
+ * 智能体聊天记录总结服务接口
+ */
+public interface AgentChatSummaryService {
+
+    /**
+     * 根据会话ID生成聊天记录总结并保存到智能体记忆
+     * 
+     * @param sessionId 会话ID
+     * @return 保存结果
+     */
+    boolean generateAndSaveChatSummary(String sessionId);
+}

main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java

@@ -17,6 +17,7 @@ import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
 import xiaozhi.modules.agent.entity.AgentEntity;
 import xiaozhi.modules.agent.service.AgentChatAudioService;
 import xiaozhi.modules.agent.service.AgentChatHistoryService;
+import xiaozhi.modules.agent.service.AgentChatSummaryService;
 import xiaozhi.modules.agent.service.AgentService;
 import xiaozhi.modules.agent.service.biz.AgentChatHistoryBizService;
 import xiaozhi.modules.device.entity.DeviceEntity;
@@ -36,6 +37,7 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
     private final AgentService agentService;
     private final AgentChatHistoryService agentChatHistoryService;
     private final AgentChatAudioService agentChatAudioService;
+    private final AgentChatSummaryService agentChatSummaryService;
     private final RedisUtils redisUtils;
     private final DeviceService deviceService;
 
@@ -50,7 +52,8 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
     public Boolean report(AgentChatHistoryReportDTO report) {
         String macAddress = report.getMacAddress();
         Byte chatType = report.getChatType();
-        Long reportTimeMillis = null != report.getReportTime() ? report.getReportTime() * 1000 : System.currentTimeMillis();
+        Long reportTimeMillis = null != report.getReportTime() ? report.getReportTime() * 1000
+                : System.currentTimeMillis();
         log.info("小智设备聊天上报请求: macAddress={}, type={} reportTime={}", macAddress, chatType, reportTimeMillis);
 
         // 根据设备MAC地址查询对应的默认智能体，判断是否需要上报
@@ -105,7 +108,8 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
     /**
      * 组装上报数据
      */
-    private void saveChatText(AgentChatHistoryReportDTO report, String agentId, String macAddress, String audioId, Long reportTime) {
+    private void saveChatText(AgentChatHistoryReportDTO report, String agentId, String macAddress, String audioId,
+            Long reportTime) {
         // 构建聊天记录实体
         AgentChatHistoryEntity entity = AgentChatHistoryEntity.builder()
                 .macAddress(macAddress)

main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentChatHistoryServiceImpl.java

@@ -84,7 +84,16 @@ public class AgentChatHistoryServiceImpl extends ServiceImpl<AiAgentChatHistoryD
     @Transactional(rollbackFor = Exception.class)
     public void deleteByAgentId(String agentId, Boolean deleteAudio, Boolean deleteText) {
         if (deleteAudio) {
-            baseMapper.deleteAudioByAgentId(agentId);
+            // 分批删除音频,避免超时
+            List<String> audioIds = baseMapper.getAudioIdsByAgentId(agentId);
+            if (audioIds != null && !audioIds.isEmpty()) {
+                int batchSize = 1000; // 每批删除1000条
+                for (int i = 0; i < audioIds.size(); i += batchSize) {
+                    int end = Math.min(i + batchSize, audioIds.size());
+                    List<String> batch = audioIds.subList(i, end);
+                    baseMapper.deleteAudioByIds(batch);
+                }
+            }
         }
         if (deleteAudio && !deleteText) {
             baseMapper.deleteAudioIdByAgentId(agentId);
@@ -107,7 +116,7 @@ public class AgentChatHistoryServiceImpl extends ServiceImpl<AiAgentChatHistoryD
                 // 添加此行，确保查询结果按照创建时间降序排列
                 // 使用id的原因：数据形式，id越大的创建时间就越晚，所以使用id的结果和创建时间降序排列结果一样
                 // id作为降序排列的优势，性能高，有主键索引，不用在排序的时候重新进行排除扫描比较
-                .orderByDesc(AgentChatHistoryEntity::getId); 
+                .orderByDesc(AgentChatHistoryEntity::getId);
 
         // 构建分页查询，查询前50页数据
         Page<AgentChatHistoryEntity> pageParam = new Page<>(0, 50);

main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentChatSummaryServiceImpl.java

@@ -0,0 +1,423 @@
+package xiaozhi.modules.agent.service.impl;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Service;
+
+import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
+
+import lombok.RequiredArgsConstructor;
+import xiaozhi.modules.agent.dto.AgentChatHistoryDTO;
+import xiaozhi.modules.agent.dto.AgentChatSummaryDTO;
+import xiaozhi.modules.agent.dto.AgentMemoryDTO;
+import xiaozhi.modules.agent.dto.AgentUpdateDTO;
+import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
+import xiaozhi.modules.agent.service.AgentChatHistoryService;
+import xiaozhi.modules.agent.service.AgentChatSummaryService;
+import xiaozhi.modules.agent.service.AgentService;
+import xiaozhi.modules.agent.vo.AgentInfoVO;
+import xiaozhi.modules.device.entity.DeviceEntity;
+import xiaozhi.modules.device.service.DeviceService;
+import xiaozhi.modules.llm.service.LLMService;
+import xiaozhi.modules.model.entity.ModelConfigEntity;
+import xiaozhi.modules.model.service.ModelConfigService;
+
+/**
+ * 智能体聊天记录总结服务实现类
+ * 实现Python端mem_local_short.py中的总结逻辑
+ */
+@Service
+@RequiredArgsConstructor
+public class AgentChatSummaryServiceImpl implements AgentChatSummaryService {
+
+    private static final Logger log = LoggerFactory.getLogger(AgentChatSummaryServiceImpl.class);
+
+    private final AgentChatHistoryService agentChatHistoryService;
+    private final AgentService agentService;
+    private final DeviceService deviceService;
+    private final LLMService llmService;
+    private final ModelConfigService modelConfigService;
+
+    // 总结规则常量
+    private static final int MAX_SUMMARY_LENGTH = 1800; // 最大总结长度
+    private static final Pattern JSON_PATTERN = Pattern.compile("\\{.*?\\}", Pattern.DOTALL);
+    private static final Pattern DEVICE_CONTROL_PATTERN = Pattern.compile("设备控制|设备操作|控制设备|设备状态",
+            Pattern.CASE_INSENSITIVE);
+    private static final Pattern WEATHER_PATTERN = Pattern.compile("天气|温度|湿度|降雨|气象", Pattern.CASE_INSENSITIVE);
+    private static final Pattern DATE_PATTERN = Pattern.compile("日期|时间|星期|月份|年份", Pattern.CASE_INSENSITIVE);
+
+    private AgentChatSummaryDTO generateChatSummary(String sessionId) {
+        try {
+            System.out.println("开始生成会话 " + sessionId + " 的聊天记录总结");
+
+            // 1. 根据sessionId获取聊天记录
+            List<AgentChatHistoryDTO> chatHistory = getChatHistoryBySessionId(sessionId);
+            if (chatHistory == null || chatHistory.isEmpty()) {
+                return new AgentChatSummaryDTO(sessionId, "未找到该会话的聊天记录");
+            }
+
+            // 2. 获取智能体信息
+            String agentId = getAgentIdFromSession(sessionId, chatHistory);
+            if (StringUtils.isBlank(agentId)) {
+                return new AgentChatSummaryDTO(sessionId, "无法获取智能体信息");
+            }
+
+            // 3. 提取关键对话内容
+            List<String> meaningfulMessages = extractMeaningfulMessages(chatHistory);
+            if (meaningfulMessages.isEmpty()) {
+                return new AgentChatSummaryDTO(sessionId, "没有有效的对话内容可总结");
+            }
+
+            // 4. 生成总结（generateSummaryFromMessages方法已包含长度限制逻辑）
+            String summary = generateSummaryFromMessages(meaningfulMessages, agentId);
+
+            System.out.println("成功生成会话 " + sessionId + " 的聊天记录总结，长度: " + summary.length() + " 字符");
+            return new AgentChatSummaryDTO(sessionId, agentId, summary);
+
+        } catch (Exception e) {
+            System.err.println("生成会话 " + sessionId + " 的聊天记录总结时发生错误: " + e.getMessage());
+            return new AgentChatSummaryDTO(sessionId, "生成总结时发生错误: " + e.getMessage());
+        }
+    }
+
+    @Override
+    public boolean generateAndSaveChatSummary(String sessionId) {
+        try {
+            // 1. 生成总结
+            AgentChatSummaryDTO summaryDTO = generateChatSummary(sessionId);
+            if (!summaryDTO.isSuccess()) {
+                System.err.println("生成总结失败: " + summaryDTO.getErrorMessage());
+                return false;
+            }
+
+            // 2. 获取设备信息（通过会话关联的设备）
+            DeviceEntity device = getDeviceBySessionId(sessionId);
+            if (device == null) {
+                System.err.println("未找到与会话 " + sessionId + " 关联的设备");
+                return false;
+            }
+
+            // 3. 更新智能体记忆
+            AgentMemoryDTO memoryDTO = new AgentMemoryDTO();
+            memoryDTO.setSummaryMemory(summaryDTO.getSummary());
+
+            // 调用现有接口更新记忆
+            agentService.updateAgentById(device.getAgentId(),
+                    new AgentUpdateDTO() {
+                        {
+                            setSummaryMemory(summaryDTO.getSummary());
+                        }
+                    });
+
+            System.out.println("成功保存会话 " + sessionId + " 的聊天记录总结到智能体 " + device.getAgentId());
+            return true;
+
+        } catch (Exception e) {
+            System.err.println("保存会话 " + sessionId + " 的聊天记录总结时发生错误: " + e.getMessage());
+            return false;
+        }
+    }
+
+    /**
+     * 根据会话ID获取聊天记录
+     */
+    private List<AgentChatHistoryDTO> getChatHistoryBySessionId(String sessionId) {
+        try {
+            // 这里需要根据sessionId获取聊天记录
+            // 由于现有接口需要agentId，我们需要先找到关联的agentId
+            String agentId = findAgentIdBySessionId(sessionId);
+            if (StringUtils.isBlank(agentId)) {
+                return null;
+            }
+            return agentChatHistoryService.getChatHistoryBySessionId(agentId, sessionId);
+        } catch (Exception e) {
+            System.err.println("获取会话 " + sessionId + " 的聊天记录失败: " + e.getMessage());
+            return null;
+        }
+    }
+
+    /**
+     * 根据会话ID查找关联的智能体ID
+     */
+    private String findAgentIdBySessionId(String sessionId) {
+        try {
+            // 查询该会话的第一条记录获取agentId
+            QueryWrapper<AgentChatHistoryEntity> wrapper = new QueryWrapper<>();
+            wrapper.select("agent_id")
+                    .eq("session_id", sessionId)
+                    .last("LIMIT 1");
+
+            AgentChatHistoryEntity entity = agentChatHistoryService.getOne(wrapper);
+            return entity != null ? entity.getAgentId() : null;
+        } catch (Exception e) {
+            System.err.println("根据会话ID " + sessionId + " 查找智能体ID失败: " + e.getMessage());
+            return null;
+        }
+    }
+
+    /**
+     * 从会话中获取智能体ID
+     */
+    private String getAgentIdFromSession(String sessionId, List<AgentChatHistoryDTO> chatHistory) {
+        // 直接从数据库查询智能体ID
+        return findAgentIdBySessionId(sessionId);
+    }
+
+    /**
+     * 提取有意义的对话内容（只提取用户消息，排除AI回复）
+     */
+    private List<String> extractMeaningfulMessages(List<AgentChatHistoryDTO> chatHistory) {
+        List<String> meaningfulMessages = new ArrayList<>();
+
+        for (AgentChatHistoryDTO message : chatHistory) {
+            // 只处理用户消息（chatType = 1）
+            if (message.getChatType() != null && message.getChatType() == 1) {
+                String content = extractContentFromMessage(message);
+                if (isMeaningfulMessage(content)) {
+                    meaningfulMessages.add(content);
+                }
+            }
+        }
+
+        return meaningfulMessages;
+    }
+
+    /**
+     * 从消息中提取内容（处理JSON格式）
+     */
+    private String extractContentFromMessage(AgentChatHistoryDTO message) {
+        String content = message.getContent();
+        if (StringUtils.isBlank(content)) {
+            return "";
+        }
+
+        // 处理JSON格式内容（与前端ChatHistoryDialog.vue逻辑一致）
+        Matcher matcher = JSON_PATTERN.matcher(content);
+        if (matcher.find()) {
+            String jsonContent = matcher.group();
+            // 简化处理：提取JSON中的文本内容
+            return extractTextFromJson(jsonContent);
+        }
+
+        return content;
+    }
+
+    /**
+     * 从JSON中提取文本内容
+     */
+    private String extractTextFromJson(String jsonContent) {
+        // 简化处理：提取"content"字段的值
+        Pattern contentPattern = Pattern.compile("\"content\"\s*:\s*\"([^\"]*)\"");
+        Matcher matcher = contentPattern.matcher(jsonContent);
+        if (matcher.find()) {
+            return matcher.group(1);
+        }
+        return jsonContent;
+    }
+
+    /**
+     * 判断是否为有意义的消息
+     */
+    private boolean isMeaningfulMessage(String content) {
+        if (StringUtils.isBlank(content)) {
+            return false;
+        }
+
+        // 排除设备控制信息
+        if (DEVICE_CONTROL_PATTERN.matcher(content).find()) {
+            return false;
+        }
+
+        // 排除日期天气等无关内容
+        if (WEATHER_PATTERN.matcher(content).find() || DATE_PATTERN.matcher(content).find()) {
+            return false;
+        }
+
+        // 排除过短的消息
+        return content.length() >= 5;
+    }
+
+    /**
+     * 从消息生成总结
+     */
+    private String generateSummaryFromMessages(List<String> messages, String agentId) {
+        if (messages.isEmpty()) {
+            return "本次对话内容较少，没有需要总结的重要信息。";
+        }
+
+        // 构建完整的对话内容
+        StringBuilder conversation = new StringBuilder();
+        for (int i = 0; i < messages.size(); i++) {
+            conversation.append("消息").append(i + 1).append(": ").append(messages.get(i)).append("\n");
+        }
+
+        try {
+            // 获取当前智能体的历史记忆
+            String historyMemory = getCurrentAgentMemory(agentId);
+
+            // 调用LLM服务进行智能总结，传递agentId以获取正确的模型配置
+            String summary = callJavaLLMForSummaryWithHistory(conversation.toString(), historyMemory, agentId);
+
+            // 应用总结规则：限制最大长度
+            if (summary.length() > MAX_SUMMARY_LENGTH) {
+                summary = summary.substring(0, MAX_SUMMARY_LENGTH) + "...";
+            }
+
+            return summary;
+        } catch (Exception e) {
+            System.err.println("调用Java端LLM服务失败: " + e.getMessage());
+            throw new RuntimeException("LLM服务不可用，无法生成聊天总结");
+        }
+    }
+
+    /**
+     * 获取当前智能体的历史记忆
+     */
+    private String getCurrentAgentMemory(String agentId) {
+        try {
+            if (StringUtils.isBlank(agentId)) {
+                return null;
+            }
+
+            // 获取智能体信息
+            AgentInfoVO agentInfo = agentService.getAgentById(agentId);
+            if (agentInfo == null) {
+                return null;
+            }
+
+            // 返回智能体的当前总结记忆
+            return agentInfo.getSummaryMemory();
+        } catch (Exception e) {
+            System.err.println("获取智能体历史记忆失败，agentId: " + agentId + ", 错误: " + e.getMessage());
+            return null;
+        }
+    }
+
+    /**
+     * 调用Java端LLM服务进行智能总结（支持历史记忆合并）
+     */
+    private String callJavaLLMForSummaryWithHistory(String conversation, String historyMemory, String agentId) {
+        try {
+            // 获取智能体配置，从中提取记忆总结的模型ID
+            String modelId = getMemorySummaryModelId(agentId);
+
+            if (StringUtils.isBlank(modelId)) {
+                System.out.println("未找到记忆总结的LLM模型配置，使用默认LLM服务");
+                return llmService.generateSummaryWithHistory(conversation, historyMemory, null, null);
+            }
+
+            // 使用指定的模型ID调用LLM服务（支持历史记忆合并）
+            String summary = llmService.generateSummaryWithHistory(conversation, historyMemory, null, modelId);
+
+            if (StringUtils.isNotBlank(summary) && !summary.equals("服务暂不可用") && !summary.equals("总结生成失败")) {
+                return summary;
+            }
+
+            throw new RuntimeException("Java端LLM服务返回异常: " + summary);
+
+        } catch (Exception e) {
+            System.err.println("调用Java端LLM服务异常，agentId: " + agentId + ", 错误: " + e.getMessage());
+            throw e;
+        }
+    }
+
+    /**
+     * 调用Java端LLM服务进行智能总结
+     */
+    private String callJavaLLMForSummary(String conversation, String agentId) {
+        try {
+            // 获取智能体配置，从中提取记忆总结的模型ID
+            String modelId = getMemorySummaryModelId(agentId);
+
+            if (StringUtils.isBlank(modelId)) {
+                System.out.println("未找到记忆总结的LLM模型配置，使用默认LLM服务");
+                return llmService.generateSummary(conversation);
+            }
+
+            // 使用指定的模型ID调用LLM服务
+            String summary = llmService.generateSummaryWithModel(conversation, modelId);
+
+            if (StringUtils.isNotBlank(summary) && !summary.equals("服务暂不可用") && !summary.equals("总结生成失败")) {
+                return summary;
+            }
+
+            throw new RuntimeException("Java端LLM服务返回异常: " + summary);
+
+        } catch (Exception e) {
+            System.err.println("调用Java端LLM服务异常，agentId: " + agentId + ", 错误: " + e.getMessage());
+            throw e;
+        }
+    }
+
+    /**
+     * 获取记忆总结的LLM模型ID
+     */
+    private String getMemorySummaryModelId(String agentId) {
+        try {
+            if (StringUtils.isBlank(agentId)) {
+                return null;
+            }
+
+            // 获取智能体信息
+            AgentInfoVO agentInfo = agentService.getAgentById(agentId);
+            if (agentInfo == null) {
+                return null;
+            }
+
+            // 获取智能体的记忆模型ID
+            String memModelId = agentInfo.getMemModelId();
+            if (StringUtils.isBlank(memModelId)) {
+                return null;
+            }
+
+            // 获取记忆模型配置
+            ModelConfigEntity memModelConfig = modelConfigService.getModelByIdFromCache(memModelId);
+            if (memModelConfig == null || memModelConfig.getConfigJson() == null) {
+                return null;
+            }
+
+            // 从记忆模型配置中提取对应的LLM模型ID
+            Map<String, Object> configMap = memModelConfig.getConfigJson();
+            String llmModelId = (String) configMap.get("llm");
+
+            if (StringUtils.isBlank(llmModelId)) {
+                // 如果记忆模型没有配置独立的LLM，则使用智能体的默认LLM模型
+                return agentInfo.getLlmModelId();
+            }
+
+            return llmModelId;
+        } catch (Exception e) {
+            System.err.println("获取记忆总结LLM模型ID失败，agentId: " + agentId + ", 错误: " + e.getMessage());
+            return null;
+        }
+    }
+
+    /**
+     * 根据会话ID获取设备信息
+     */
+    private DeviceEntity getDeviceBySessionId(String sessionId) {
+        try {
+            // 查询该会话的第一条记录获取macAddress
+            QueryWrapper<AgentChatHistoryEntity> wrapper = new QueryWrapper<>();
+            wrapper.select("mac_address")
+                    .eq("session_id", sessionId)
+                    .last("LIMIT 1");
+
+            AgentChatHistoryEntity entity = agentChatHistoryService.getOne(wrapper);
+            if (entity != null && StringUtils.isNotBlank(entity.getMacAddress())) {
+                return deviceService.getDeviceByMacAddress(entity.getMacAddress());
+            }
+            return null;
+        } catch (Exception e) {
+            System.err.println("根据会话ID " + sessionId + " 查找设备信息失败: " + e.getMessage());
+            return null;
+        }
+    }
+}

main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java

@@ -106,6 +106,15 @@ public class ConfigServiceImpl implements ConfigService {
 
     @Override
     public Map<String, Object> getAgentModels(String macAddress, Map<String, String> selectedModule) {
+        // 检查是否为管理控制台请求
+        String redisKey = RedisKeys.getTmpRegisterMacKey(macAddress);
+        Object isAdminRequest = redisUtils.get(redisKey);
+        
+        if (isAdminRequest != null && "true".equals(isAdminRequest)) {
+            // 管理控制台请求，返回getConfig的结果
+            redisUtils.delete(redisKey); // 使用后清理
+            return (Map<String, Object>) getConfig(true);
+        }
         // 根据MAC地址查找设备
         DeviceEntity device = deviceService.getDeviceByMacAddress(macAddress);
         if (device == null) {

main/manager-api/src/main/java/xiaozhi/modules/device/service/DeviceService.java

@@ -98,4 +98,14 @@ public interface DeviceService extends BaseService<DeviceEntity> {
      */
     void updateDeviceConnectionInfo(String agentId, String deviceId, String appVersion);
 
+    /**
+     * 生成WebSocket认证token
+     *
+     * @param clientId 客户端ID
+     * @param username 用户名(通常为deviceId)
+     * @return 认证token字符串
+     * @throws Exception 生成token时的异常
+     */
+    String generateWebSocketToken(String clientId, String username) throws Exception;
+
 }

main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java

@@ -518,7 +518,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
      * @param username 用户名 (通常为deviceId/macAddress)
      * @return 认证token字符串
      */
-    private String generateWebSocketToken(String clientId, String username)
+    public String generateWebSocketToken(String clientId, String username)
             throws NoSuchAlgorithmException, InvalidKeyException {
         // 从系统参数获取密钥
         String secretKey = sysParamsService.getValue(Constant.SERVER_SECRET, false);

main/manager-api/src/main/java/xiaozhi/modules/llm/service/LLMService.java

@@ -0,0 +1,70 @@
+package xiaozhi.modules.llm.service;
+
+/**
+ * LLM服务接口
+ * 支持多种大模型调用
+ */
+public interface LLMService {
+
+    /**
+     * 生成聊天记录总结
+     * 
+     * @param conversation   对话内容
+     * @param promptTemplate 提示词模板
+     * @return 总结结果
+     */
+    String generateSummary(String conversation, String promptTemplate);
+
+    /**
+     * 生成聊天记录总结（使用默认提示词）
+     * 
+     * @param conversation 对话内容
+     * @return 总结结果
+     */
+    String generateSummary(String conversation);
+
+    /**
+     * 生成聊天记录总结（指定模型ID）
+     * 
+     * @param conversation 对话内容
+     * @param modelId      模型ID
+     * @return 总结结果
+     */
+    String generateSummaryWithModel(String conversation, String modelId);
+
+    /**
+     * 生成聊天记录总结（指定模型ID和提示词模板）
+     * 
+     * @param conversation   对话内容
+     * @param promptTemplate 提示词模板
+     * @param modelId        模型ID
+     * @return 总结结果
+     */
+    String generateSummary(String conversation, String promptTemplate, String modelId);
+
+    /**
+     * 生成聊天记录总结（包含历史记忆合并）
+     * 
+     * @param conversation   对话内容
+     * @param historyMemory  历史记忆
+     * @param promptTemplate 提示词模板
+     * @param modelId        模型ID
+     * @return 总结结果
+     */
+    String generateSummaryWithHistory(String conversation, String historyMemory, String promptTemplate, String modelId);
+
+    /**
+     * 检查服务是否可用
+     * 
+     * @return 是否可用
+     */
+    boolean isAvailable();
+
+    /**
+     * 检查指定模型的服务是否可用
+     * 
+     * @param modelId 模型ID
+     * @return 是否可用
+     */
+    boolean isAvailable(String modelId);
+}

main/manager-api/src/main/java/xiaozhi/modules/llm/service/impl/OpenAIStyleLLMServiceImpl.java

@@ -0,0 +1,305 @@
+package xiaozhi.modules.llm.service.impl;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.commons.lang3.StringUtils;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpEntity;
+import org.springframework.http.HttpHeaders;
+import org.springframework.http.HttpMethod;
+import org.springframework.http.MediaType;
+import org.springframework.http.ResponseEntity;
+import org.springframework.stereotype.Service;
+import org.springframework.web.client.RestTemplate;
+
+import cn.hutool.json.JSONArray;
+import cn.hutool.json.JSONObject;
+import cn.hutool.json.JSONUtil;
+import lombok.extern.slf4j.Slf4j;
+import xiaozhi.modules.llm.service.LLMService;
+import xiaozhi.modules.model.entity.ModelConfigEntity;
+import xiaozhi.modules.model.service.ModelConfigService;
+
+/**
+ * OpenAI风格API的LLM服务实现
+ * 支持阿里云、DeepSeek、ChatGLM等兼容OpenAI API的模型
+ */
+@Slf4j
+@Service
+public class OpenAIStyleLLMServiceImpl implements LLMService {
+
+    @Autowired
+    private ModelConfigService modelConfigService;
+
+    private final RestTemplate restTemplate = new RestTemplate();
+
+    private static final String DEFAULT_SUMMARY_PROMPT = "你是一个经验丰富的记忆总结者，擅长将对话内容进行总结摘要，遵循以下规则：\n1、总结用户的重要信息，以便在未来的对话中提供更个性化的服务\n2、不要重复总结，不要遗忘之前记忆，除非原来的记忆超过了1800字，否则不要遗忘、不要压缩用户的历史记忆\n3、用户操控的设备音量、播放音乐、天气、退出、不想对话等和用户本身无关的内容，这些信息不需要加入到总结中\n4、聊天内容中的今天的日期时间、今天的天气情况与用户事件无关的数据，这些信息如果当成记忆存储会影响后续对话，这些信息不需要加入到总结中\n5、不要把设备操控的成果结果和失败结果加入到总结中，也不要把用户的一些废话加入到总结中\n6、不要为了总结而总结，如果用户的聊天没有意义，请返回原来的历史记录也是可以的\n7、只需要返回总结摘要，严格控制在1800字内\n8、不要包含代码、xml，不需要解释、注释和说明，保存记忆时仅从对话提取信息，不要混入示例内容\n9、如果提供了历史记忆，请将新对话内容与历史记忆进行智能合并，保留有价值的历史信息，同时添加新的重要信息\n\n历史记忆：\n{history_memory}\n\n新对话内容：\n{conversation}";
+
+    @Override
+    public String generateSummary(String conversation) {
+        return generateSummary(conversation, null, null);
+    }
+
+    @Override
+    public String generateSummaryWithModel(String conversation, String modelId) {
+        return generateSummary(conversation, null, modelId);
+    }
+
+    @Override
+    public String generateSummary(String conversation, String promptTemplate, String modelId) {
+        if (!isAvailable()) {
+            log.warn("LLM服务不可用，无法生成总结");
+            return "LLM服务不可用，无法生成总结";
+        }
+
+        try {
+            // 从智控台获取LLM模型配置
+            ModelConfigEntity llmConfig;
+            if (modelId != null && !modelId.trim().isEmpty()) {
+                // 通过具体模型ID获取配置
+                llmConfig = modelConfigService.getModelByIdFromCache(modelId);
+            } else {
+                // 保持向后兼容，使用默认配置
+                llmConfig = getDefaultLLMConfig();
+            }
+
+            if (llmConfig == null || llmConfig.getConfigJson() == null) {
+                log.error("未找到可用的LLM模型配置，modelId: {}", modelId);
+                return "未找到可用的LLM模型配置";
+            }
+
+            JSONObject configJson = llmConfig.getConfigJson();
+            String baseUrl = configJson.getStr("base_url");
+            String model = configJson.getStr("model_name");
+            String apiKey = configJson.getStr("api_key");
+            Double temperature = configJson.getDouble("temperature");
+            Integer maxTokens = configJson.getInt("max_tokens");
+
+            if (StringUtils.isBlank(baseUrl) || StringUtils.isBlank(apiKey)) {
+                log.error("LLM配置不完整，baseUrl或apiKey为空");
+                return "LLM配置不完整，无法生成总结";
+            }
+
+            // 构建提示词
+            String prompt = (promptTemplate != null ? promptTemplate : DEFAULT_SUMMARY_PROMPT).replace("{conversation}",
+                    conversation);
+
+            // 构建请求体
+            Map<String, Object> requestBody = new HashMap<>();
+            requestBody.put("model", model != null ? model : "gpt-3.5-turbo");
+
+            Map<String, Object>[] messages = new Map[1];
+            Map<String, Object> message = new HashMap<>();
+            message.put("role", "user");
+            message.put("content", prompt);
+            messages[0] = message;
+
+            requestBody.put("messages", messages);
+            requestBody.put("temperature", temperature != null ? temperature : 0.7);
+            requestBody.put("max_tokens", maxTokens != null ? maxTokens : 2000);
+
+            // 发送HTTP请求
+            HttpHeaders headers = new HttpHeaders();
+            headers.setContentType(MediaType.APPLICATION_JSON);
+            headers.set("Authorization", "Bearer " + apiKey);
+
+            HttpEntity<Map<String, Object>> entity = new HttpEntity<>(requestBody, headers);
+
+            // 构建完整的API URL
+            String apiUrl = baseUrl;
+            if (!apiUrl.endsWith("/chat/completions")) {
+                if (!apiUrl.endsWith("/")) {
+                    apiUrl += "/";
+                }
+                apiUrl += "chat/completions";
+            }
+
+            ResponseEntity<String> response = restTemplate.exchange(
+                    apiUrl, HttpMethod.POST, entity, String.class);
+
+            if (response.getStatusCode().is2xxSuccessful()) {
+                JSONObject responseJson = JSONUtil.parseObj(response.getBody());
+                JSONArray choices = responseJson.getJSONArray("choices");
+                if (choices != null && choices.size() > 0) {
+                    JSONObject choice = choices.getJSONObject(0);
+                    JSONObject messageObj = choice.getJSONObject("message");
+                    return messageObj.getStr("content");
+                }
+            } else {
+                log.error("LLM API调用失败，状态码：{}，响应：{}", response.getStatusCode(), response.getBody());
+            }
+        } catch (Exception e) {
+            log.error("调用LLM服务生成总结时发生异常，modelId: {}", modelId, e);
+        }
+
+        return "生成总结失败，请稍后重试";
+    }
+
+    @Override
+    public String generateSummary(String conversation, String promptTemplate) {
+        return generateSummary(conversation, promptTemplate, null);
+    }
+
+    @Override
+    public String generateSummaryWithHistory(String conversation, String historyMemory, String promptTemplate,
+            String modelId) {
+        if (!isAvailable()) {
+            log.warn("LLM服务不可用，无法生成总结");
+            return "LLM服务不可用，无法生成总结";
+        }
+
+        try {
+            // 从智控台获取LLM模型配置
+            ModelConfigEntity llmConfig;
+            if (modelId != null && !modelId.trim().isEmpty()) {
+                // 通过具体模型ID获取配置
+                llmConfig = modelConfigService.getModelByIdFromCache(modelId);
+            } else {
+                // 保持向后兼容，使用默认配置
+                llmConfig = getDefaultLLMConfig();
+            }
+
+            if (llmConfig == null || llmConfig.getConfigJson() == null) {
+                log.error("未找到可用的LLM模型配置，modelId: {}", modelId);
+                return "未找到可用的LLM模型配置";
+            }
+
+            JSONObject configJson = llmConfig.getConfigJson();
+            String baseUrl = configJson.getStr("base_url");
+            String model = configJson.getStr("model_name");
+            String apiKey = configJson.getStr("api_key");
+
+            if (StringUtils.isBlank(baseUrl) || StringUtils.isBlank(apiKey)) {
+                log.error("LLM配置不完整，baseUrl或apiKey为空");
+                return "LLM配置不完整，无法生成总结";
+            }
+
+            // 构建提示词，包含历史记忆
+            String prompt = (promptTemplate != null ? promptTemplate : DEFAULT_SUMMARY_PROMPT)
+                    .replace("{history_memory}", historyMemory != null ? historyMemory : "无历史记忆")
+                    .replace("{conversation}", conversation);
+
+            // 构建请求体
+            Map<String, Object> requestBody = new HashMap<>();
+            requestBody.put("model", model != null ? model : "gpt-3.5-turbo");
+
+            Map<String, Object>[] messages = new Map[1];
+            Map<String, Object> message = new HashMap<>();
+            message.put("role", "user");
+            message.put("content", prompt);
+            messages[0] = message;
+
+            requestBody.put("messages", messages);
+            requestBody.put("temperature", 0.2);
+            requestBody.put("max_tokens", 2000);
+
+            // 发送HTTP请求
+            HttpHeaders headers = new HttpHeaders();
+            headers.setContentType(MediaType.APPLICATION_JSON);
+            headers.set("Authorization", "Bearer " + apiKey);
+
+            HttpEntity<Map<String, Object>> entity = new HttpEntity<>(requestBody, headers);
+
+            // 构建完整的API URL
+            String apiUrl = baseUrl;
+            if (!apiUrl.endsWith("/chat/completions")) {
+                if (!apiUrl.endsWith("/")) {
+                    apiUrl += "/";
+                }
+                apiUrl += "chat/completions";
+            }
+
+            ResponseEntity<String> response = restTemplate.exchange(
+                    apiUrl, HttpMethod.POST, entity, String.class);
+
+            if (response.getStatusCode().is2xxSuccessful()) {
+                JSONObject responseJson = JSONUtil.parseObj(response.getBody());
+                JSONArray choices = responseJson.getJSONArray("choices");
+                if (choices != null && choices.size() > 0) {
+                    JSONObject choice = choices.getJSONObject(0);
+                    JSONObject messageObj = choice.getJSONObject("message");
+                    return messageObj.getStr("content");
+                }
+            } else {
+                log.error("LLM API调用失败，状态码：{}，响应：{}", response.getStatusCode(), response.getBody());
+            }
+        } catch (Exception e) {
+            log.error("调用LLM服务生成总结时发生异常，modelId: {}", modelId, e);
+        }
+
+        return "生成总结失败，请稍后重试";
+    }
+
+    @Override
+    public boolean isAvailable() {
+        try {
+            ModelConfigEntity defaultLLMConfig = getDefaultLLMConfig();
+            if (defaultLLMConfig == null || defaultLLMConfig.getConfigJson() == null) {
+                return false;
+            }
+
+            JSONObject configJson = defaultLLMConfig.getConfigJson();
+            String baseUrl = configJson.getStr("base_url");
+            String apiKey = configJson.getStr("api_key");
+
+            return baseUrl != null && !baseUrl.trim().isEmpty() &&
+                    apiKey != null && !apiKey.trim().isEmpty();
+        } catch (Exception e) {
+            log.error("检查LLM服务可用性时发生异常：", e);
+            return false;
+        }
+    }
+
+    @Override
+    public boolean isAvailable(String modelId) {
+        try {
+            if (modelId == null || modelId.trim().isEmpty()) {
+                return isAvailable();
+            }
+
+            // 通过具体模型ID获取配置
+            ModelConfigEntity modelConfig = modelConfigService.getModelByIdFromCache(modelId);
+            if (modelConfig == null || modelConfig.getConfigJson() == null) {
+                log.warn("未找到指定的LLM模型配置，modelId: {}", modelId);
+                return false;
+            }
+
+            JSONObject configJson = modelConfig.getConfigJson();
+            String baseUrl = configJson.getStr("base_url");
+            String apiKey = configJson.getStr("api_key");
+
+            return baseUrl != null && !baseUrl.trim().isEmpty() &&
+                    apiKey != null && !apiKey.trim().isEmpty();
+        } catch (Exception e) {
+            log.error("检查LLM服务可用性时发生异常，modelId: {}", modelId, e);
+            return false;
+        }
+    }
+
+    /**
+     * 从智控台获取默认的LLM模型配置
+     */
+    private ModelConfigEntity getDefaultLLMConfig() {
+        try {
+            // 获取所有启用的LLM模型配置
+            List<ModelConfigEntity> llmConfigs = modelConfigService.getEnabledModelsByType("LLM");
+            if (llmConfigs == null || llmConfigs.isEmpty()) {
+                return null;
+            }
+
+            // 优先返回默认配置，如果没有默认配置则返回第一个启用的配置
+            for (ModelConfigEntity config : llmConfigs) {
+                if (config.getIsDefault() != null && config.getIsDefault() == 1) {
+                    return config;
+                }
+            }
+
+            return llmConfigs.get(0);
+        } catch (Exception e) {
+            log.error("获取LLM模型配置时发生异常：", e);
+            return null;
+        }
+    }
+}

main/manager-api/src/main/java/xiaozhi/modules/model/service/ModelConfigService.java

@@ -55,4 +55,12 @@ public interface ModelConfigService extends BaseService<ModelConfigEntity> {
      * @return TTS平台列表(id和modelName)
      */
     List<Map<String, Object>> getTtsPlatformList();
+
+    /**
+     * 根据模型类型获取所有启用的模型配置
+     *
+     * @param modelType 模型类型（如：LLM, TTS, ASR等）
+     * @return 启用的模型配置列表
+     */
+    List<ModelConfigEntity> getEnabledModelsByType(String modelType);
 }

main/manager-api/src/main/java/xiaozhi/modules/model/service/impl/ModelConfigServiceImpl.java

@@ -502,4 +502,22 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
     public List<Map<String, Object>> getTtsPlatformList() {
         return modelConfigDao.getTtsPlatformList();
     }
+
+    /**
+     * 根据模型类型获取所有启用的模型配置
+     */
+    @Override
+    public List<ModelConfigEntity> getEnabledModelsByType(String modelType) {
+        if (StringUtils.isBlank(modelType)) {
+            return null;
+        }
+
+        List<ModelConfigEntity> entities = modelConfigDao.selectList(
+                new QueryWrapper<ModelConfigEntity>()
+                        .eq("model_type", modelType)
+                        .eq("is_enabled", 1)
+                        .orderByAsc("sort"));
+
+        return entities;
+    }
 }

main/manager-api/src/main/java/xiaozhi/modules/security/config/ShiroConfig.java

@@ -89,7 +89,7 @@ public class ShiroConfig {
         filterMap.put("/config/**", "server");
         filterMap.put("/agent/chat-history/report", "server");
         filterMap.put("/agent/chat-history/download/**", "anon");
-        filterMap.put("/agent/saveMemory/**", "server");
+        filterMap.put("/agent/chat-summary/**", "server");
         filterMap.put("/agent/play/**", "anon");
         filterMap.put("/voiceClone/play/**", "anon");
         filterMap.put("/**", "oauth2");

main/manager-api/src/main/java/xiaozhi/modules/sys/controller/ServerSideManageController.java

@@ -31,6 +31,8 @@ import xiaozhi.modules.sys.dto.ServerActionResponseDTO;
 import xiaozhi.modules.sys.enums.ServerActionEnum;
 import xiaozhi.modules.sys.service.SysParamsService;
 import xiaozhi.modules.sys.utils.WebSocketClientManager;
+import xiaozhi.modules.device.service.DeviceService;
+import xiaozhi.common.redis.RedisUtils;
 
 /**
  * 服务端管理控制器
@@ -41,6 +43,8 @@ import xiaozhi.modules.sys.utils.WebSocketClientManager;
 @AllArgsConstructor
 public class ServerSideManageController {
     private final SysParamsService sysParamsService;
+    private final DeviceService deviceService;
+    private final RedisUtils redisUtils;
     private static final ObjectMapper objectMapper;
     static {
         objectMapper = new ObjectMapper();
@@ -85,9 +89,22 @@ public class ServerSideManageController {
             return false;
         }
         String serverSK = sysParamsService.getValue(Constant.SERVER_SECRET, true);
+
+        String deviceId = UUID.randomUUID().toString();
+        String clientId = UUID.randomUUID().toString();
+
+        String redisKey = xiaozhi.common.redis.RedisKeys.getTmpRegisterMacKey(deviceId);
+        redisUtils.set(redisKey, "true", 300); // 5分钟有效期
+
         WebSocketHttpHeaders headers = new WebSocketHttpHeaders();
-        headers.add("device-id", UUID.randomUUID().toString());
+        headers.add("device-id", deviceId);
-        headers.add("client-id", UUID.randomUUID().toString());
+        headers.add("client-id", clientId);
+        try {
+            String token = deviceService.generateWebSocketToken(clientId, deviceId);
+            headers.add("authorization", "Bearer " + token);
+        } catch (Exception e) {
+            throw new RenException(ErrorCode.WEB_SOCKET_CONNECT_FAILED);
+        }
 
         try (WebSocketClientManager client = new WebSocketClientManager.Builder()
                 .connectTimeout(3, TimeUnit.SECONDS)

main/manager-api/src/main/resources/db/changelog/202511221450.sql

@@ -0,0 +1,17 @@
+-- 更新HuoshanDoubleStreamTTS供应器配置，增加开启链接复用选项
+UPDATE `ai_model_provider`
+SET fields = '[{"key": "ws_url", "type": "string", "label": "WebSocket地址"}, {"key": "appid", "type": "string", "label": "应用ID"}, {"key": "access_token", "type": "string", "label": "访问令牌"}, {"key": "resource_id", "type": "string", "label": "资源ID"}, {"key": "speaker", "type": "string", "label": "默认音色"}, {"key": "enable_ws_reuse", "type": "boolean", "label": "是否开启链接复用", "default": true}, {"key": "speech_rate", "type": "number", "label": "语速(-50~100)"}, {"key": "loudness_rate", "type": "number", "label": "音量(-50~100)"}, {"key": "pitch", "type": "number", "label": "音高(-12~12)"}]'
+WHERE id = 'SYSTEM_TTS_HSDSTTS';
+
+UPDATE `ai_model_config` SET 
+`doc_link` = 'https://console.volcengine.com/speech/service/10007',
+`remark` = '火山引擎语音合成服务配置说明：
+1. 访问 https://www.volcengine.com/ 注册并开通火山引擎账号
+2. 访问 https://console.volcengine.com/speech/service/10007 开通语音合成大模型，购买音色
+3. 在页面底部获取appid和access_token
+5. 资源ID固定为：volc.service_type.10029（大模型语音合成及混音）
+6. 链接复用：开启WebSocket连接复用，默认true减少链接损耗（注意：复用后设备处于聆听状态时空闲链接会占并发数）
+7. 语速：-50~100，可不填，正常默认值0，可填-50~100
+8. 音量：-50~100，可不填，正常默认值0，可填-50~100
+9. 音高：-12~12，可不填，正常默认值0，可填-12~12
+10. 填入配置文件中' WHERE `id` = 'TTS_HuoshanDoubleStreamTTS';

main/manager-api/src/main/resources/db/changelog/202512161529.sql

@@ -0,0 +1 @@
+INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (311, 'enable_websocket_ping', 'false', 'boolean', 1, '是否启用WebSocket心跳保活机制');

main/manager-api/src/main/resources/db/changelog/202512192245.sql

@@ -0,0 +1,2 @@
+-- 为智能体聊天历史记录添加音频ID索引
+ALTER TABLE ai_agent_chat_history ADD INDEX idx_ai_agent_chat_history_audio_id (audio_id);

main/manager-api/src/main/resources/db/changelog/202512221117.sql

@@ -0,0 +1,10 @@
+-- 更新豆包流式ASR供应器，增加end_window_size配置
+delete from `ai_model_provider` where id = 'SYSTEM_ASR_DoubaoStreamASR';
+INSERT INTO `ai_model_provider` (`id`, `model_type`, `provider_code`, `name`, `fields`, `sort`, `creator`, `create_date`, `updater`, `update_date`) VALUES
+('SYSTEM_ASR_DoubaoStreamASR', 'ASR', 'doubao_stream', '火山引擎语音识别(流式)', '[{"key":"appid","label":"应用ID","type":"string"},{"key":"access_token","label":"访问令牌","type":"string"},{"key":"cluster","label":"集群","type":"string"},{"key":"boosting_table_name","label":"热词文件名称","type":"string"},{"key":"correct_table_name","label":"替换词文件名称","type":"string"},{"key":"output_dir","label":"输出目录","type":"string"},{"key":"end_window_size","label":"静音判定时长(ms)","type":"number"}]', 3, 1, NOW(), 1, NOW());
+
+
+-- 更新豆包流式ASR模型配置，增加end_window_size默认值
+UPDATE `ai_model_config` SET
+`config_json` = JSON_SET(`config_json`, '$.end_window_size', 200)
+WHERE `id` = 'ASR_DoubaoStreamASR' AND JSON_EXTRACT(`config_json`, '$.end_window_size') IS NULL;

main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml

@@ -424,6 +424,13 @@ databaseChangeLog:
             encoding: utf8
             path: classpath:db/changelog/202511131023.sql
   - changeSet:
+      id: 202511221450
+      author: RanChen
+      changes:
+        - sqlFile:
+            encoding: utf8
+            path: classpath:db/changelog/202511221450.sql
+  - changeSet:         
       id: 202512031517
       author: rainv123
       changes:
@@ -445,3 +452,24 @@ databaseChangeLog:
         - sqlFile:
             encoding: utf8
             path: classpath:db/changelog/202512131453.sql
+  - changeSet:
+      id: 202512161529
+      author: RanChen
+      changes:
+        - sqlFile:
+            encoding: utf8
+            path: classpath:db/changelog/202512161529.sql
+  - changeSet:
+      id: 202512192245
+      author: hrz
+      changes:
+        - sqlFile:
+            encoding: utf8
+            path: classpath:db/changelog/202512192245.sql
+  - changeSet:
+      id: 202512221117
+      author: RanChen
+      changes:
+        - sqlFile:
+            encoding: utf8
+            path: classpath:db/changelog/202512221117.sql

main/manager-api/src/main/resources/mapper/agent/AiAgentChatHistoryDao.xml

@@ -22,13 +22,18 @@
     created_at, updated_at
   </sql>
 
-  <delete id="deleteAudioByAgentId">
+  <select id="getAudioIdsByAgentId" resultType="java.lang.String">
-    DELETE FROM ai_agent_chat_audio 
+    SELECT DISTINCT audio_id
-    WHERE id IN (
+    FROM ai_agent_chat_history
-      SELECT audio_id 
+    WHERE agent_id = #{agentId} AND audio_id IS NOT NULL
-      FROM ai_agent_chat_history 
+  </select>
-      WHERE agent_id = #{agentId}
+
-    )
+  <delete id="deleteAudioByIds">
+    DELETE FROM ai_agent_chat_audio
+    WHERE id IN
+    <foreach collection="audioIds" item="id" open="(" separator="," close=")">
+      #{id}
+    </foreach>
   </delete>
   
   <update id="deleteAudioIdByAgentId">

main/manager-web/src/components/HeaderBar.vue

@@ -4,7 +4,7 @@
       <!-- 左侧元素 -->
       <div class="header-left" @click="goHome">
         <img loading="lazy" alt="" src="@/assets/xiaozhi-logo.png" class="logo-img" />
-        <img loading="lazy" alt="" src="@/assets/xiaozhi-ai.png" class="brand-img" />
+        <img loading="lazy" alt="" :src="xiaozhiAiIcon" class="brand-img" />
       </div>
 
       <!-- 中间导航菜单 -->
@@ -257,6 +257,24 @@ export default {
           return this.$t("language.zhCN");
       }
     },
+    // 根据当前语言获取对应的xiaozhi-ai图标
+    xiaozhiAiIcon() {
+      const currentLang = this.currentLanguage;
+      switch (currentLang) {
+        case "zh_CN":
+          return require("@/assets/xiaozhi-ai.png");
+        case "zh_TW":
+          return require("@/assets/xiaozhi-ai_zh_TW.png");
+        case "en":
+          return require("@/assets/xiaozhi-ai_en.png");
+        case "de":
+          return require("@/assets/xiaozhi-ai_de.png");
+        case "vi":
+          return require("@/assets/xiaozhi-ai_vi.png");
+        default:
+          return require("@/assets/xiaozhi-ai.png");
+      }
+    },
     // 用户菜单选项
     userMenuOptions() {
       return [

main/manager-web/src/i18n/de.js

@@ -821,7 +821,7 @@ export default {
   'modelConfig.rag': 'RAG',
   'modelConfig.modelId': 'Modell-ID',
   'modelConfig.modelName': 'Modellname',
-  'modelConfig.provider': 'Anbieter',
+  'modelConfig.provider': 'Schnittstellentyp',
   'modelConfig.unknown': 'Unbekannt',
   'modelConfig.isEnabled': 'Aktiviert',
   'modelConfig.isDefault': 'Standard',

main/manager-web/src/i18n/en.js

@@ -821,7 +821,7 @@ export default {
   'modelConfig.rag': 'RAG',
   'modelConfig.modelId': 'Model ID',
   'modelConfig.modelName': 'Model Name',
-  'modelConfig.provider': 'Provider',
+  'modelConfig.provider': 'Interface Type',
   'modelConfig.unknown': 'Unknown',
   'modelConfig.isEnabled': 'Enabled',
   'modelConfig.isDefault': 'Default',

main/manager-web/src/i18n/vi.js

@@ -821,7 +821,7 @@ export default {
   'modelConfig.rag': 'RAG',
   'modelConfig.modelId': 'ID mô hình',
   'modelConfig.modelName': 'Tên mô hình',
-  'modelConfig.provider': 'Nhà cung cấp',
+  'modelConfig.provider': 'Loại giao diện',
   'modelConfig.unknown': 'Không xác định',
   'modelConfig.isEnabled': 'Đã bật',
   'modelConfig.isDefault': 'Mặc định',

main/manager-web/src/i18n/zh_CN.js

@@ -821,7 +821,7 @@ export default {
   'modelConfig.rag': '知识库',
   'modelConfig.modelId': '模型ID',
   'modelConfig.modelName': '模型名称',
-  'modelConfig.provider': '提供商',
+  'modelConfig.provider': '接口类型',
   'modelConfig.unknown': '未知',
   'modelConfig.isEnabled': '是否启用',
   'modelConfig.isDefault': '是否默认',

main/manager-web/src/i18n/zh_TW.js

@@ -821,7 +821,7 @@ export default {
   'modelConfig.rag': '知識庫',
   'modelConfig.modelId': '模型ID',
   'modelConfig.modelName': '模型名稱',
-  'modelConfig.provider': '提供商',
+  'modelConfig.provider': '接口類型',
   'modelConfig.unknown': '未知',
   'modelConfig.isEnabled': '是否啟用',
   'modelConfig.isDefault': '是否默認',

main/manager-web/src/views/DeviceManagement.vue

@@ -35,7 +35,7 @@
               <el-table-column :label="$t('device.bindTime')" prop="bindTime" align="center"></el-table-column>
               <el-table-column :label="$t('device.lastConversation')" prop="lastConversation"
                 align="center"></el-table-column>
-              <el-table-column :label="$t('device.deviceStatus')" prop="deviceStatus" align="center">
+              <el-table-column v-if="mqttServiceAvailable" :label="$t('device.deviceStatus')" prop="deviceStatus" align="center">
                 <template slot-scope="scope">
                   <el-tag v-if="scope.row.deviceStatus === 'online'" type="success">{{ $t('device.online') }}</el-tag>
                   <el-tag v-else type="danger">{{ $t('device.offline') }}</el-tag>
@@ -147,6 +147,7 @@ export default {
       loading: false,
       userApi: null,
       firmwareTypes: [],
+      mqttServiceAvailable: false, // MQTT服务是否可用
     };
   },
   computed: {
@@ -392,12 +393,21 @@ export default {
 
             // 直接使用解析后的数据作为设备状态映射（不需要devices字段包装）
             if (statusData && typeof statusData === 'object') {
+              // 成功获取到设备状态
+              this.mqttServiceAvailable = true;
               // 更新设备状态
               this.updateDeviceStatusFromResponse(statusData);
+            } else {
+              // 数据格式不正确，MQTT服务不可用
+              this.mqttServiceAvailable = false;
             }
           } catch (error) {
-            // JSON解析失败，忽略状态更新
+            // JSON解析失败，MQTT服务不可用
+            this.mqttServiceAvailable = false;
           }
+        } else {
+          // 接口调用失败，MQTT服务不可用
+          this.mqttServiceAvailable = false;
         }
       });
     },

main/manager-web/src/views/login.vue

@@ -10,7 +10,7 @@
             gap: 10px;
           ">
           <img loading="lazy" alt="" src="@/assets/xiaozhi-logo.png" style="width: 45px; height: 45px" />
-          <img loading="lazy" alt="" src="@/assets/xiaozhi-ai.png" style="height: 18px" />
+          <img loading="lazy" alt="" :src="xiaozhiAiIcon" style="height: 18px" />
         </div>
       </el-header>
       <div class="login-person">
@@ -192,6 +192,24 @@ export default {
           return this.$t("language.zhCN");
       }
     },
+    // 根据当前语言获取对应的xiaozhi-ai图标
+    xiaozhiAiIcon() {
+      const currentLang = this.currentLanguage;
+      switch (currentLang) {
+        case "zh_CN":
+          return require("@/assets/xiaozhi-ai.png");
+        case "zh_TW":
+          return require("@/assets/xiaozhi-ai_zh_TW.png");
+        case "en":
+          return require("@/assets/xiaozhi-ai_en.png");
+        case "de":
+          return require("@/assets/xiaozhi-ai_de.png");
+        case "vi":
+          return require("@/assets/xiaozhi-ai_vi.png");
+        default:
+          return require("@/assets/xiaozhi-ai.png");
+      }
+    },
   },
   data() {
     return {

main/manager-web/src/views/register.vue

@@ -5,7 +5,7 @@
       <el-header>
         <div style="display: flex;align-items: center;margin-top: 15px;margin-left: 10px;gap: 10px;">
           <img loading="lazy" alt="" src="@/assets/xiaozhi-logo.png" style="width: 45px;height: 45px;" />
-          <img loading="lazy" alt="" src="@/assets/xiaozhi-ai.png" style="height: 18px;" />
+          <img loading="lazy" alt="" :src="xiaozhiAiIcon" style="height: 18px;" />
         </div>
       </el-header>
       <div class="login-person">
@@ -108,7 +108,7 @@
           <div style="font-size: 14px;color: #979db1;">
             {{ $t('register.agreeTo') }}
             <div style="display: inline-block;color: #5778FF;cursor: pointer;">{{ $t('register.userAgreement') }}</div>
-            {{ $t('register.and') }}
+            {{ $t('login.and') }}
             <div style="display: inline-block;color: #5778FF;cursor: pointer;">{{ $t('register.privacyPolicy') }}</div>
           </div>
         </div>
@@ -127,6 +127,7 @@ import Api from '@/apis/api';
 import VersionFooter from '@/components/VersionFooter.vue';
 import { getUUID, goToPage, showDanger, showSuccess, sm2Encrypt, validateMobile } from '@/utils';
 import { mapState } from 'vuex';
+import i18n from '@/i18n';
 
 // 导入语言切换功能
 
@@ -142,6 +143,28 @@ export default {
       mobileAreaList: state => state.pubConfig.mobileAreaList,
       sm2PublicKey: state => state.pubConfig.sm2PublicKey,
     }),
+    // 获取当前语言
+    currentLanguage() {
+      return i18n.locale || "zh_CN";
+    },
+    // 根据当前语言获取对应的xiaozhi-ai图标
+    xiaozhiAiIcon() {
+      const currentLang = this.currentLanguage;
+      switch (currentLang) {
+        case "zh_CN":
+          return require("@/assets/xiaozhi-ai.png");
+        case "zh_TW":
+          return require("@/assets/xiaozhi-ai_zh_TW.png");
+        case "en":
+          return require("@/assets/xiaozhi-ai_en.png");
+        case "de":
+          return require("@/assets/xiaozhi-ai_de.png");
+        case "vi":
+          return require("@/assets/xiaozhi-ai_vi.png");
+        default:
+          return require("@/assets/xiaozhi-ai.png");
+      }
+    },
     canSendMobileCaptcha() {
       return this.countdown === 0 && validateMobile(this.form.mobile, this.form.areaCode);
     }

main/manager-web/src/views/retrievePassword.vue

@@ -5,7 +5,7 @@
       <el-header>
         <div style="display: flex;align-items: center;margin-top: 15px;margin-left: 10px;gap: 10px;">
           <img loading="lazy" alt="" src="@/assets/xiaozhi-logo.png" style="width: 45px;height: 45px;" />
-          <img loading="lazy" alt="" src="@/assets/xiaozhi-ai.png" style="height: 18px;" />
+          <img loading="lazy" alt="" :src="xiaozhiAiIcon" style="height: 18px;" />
         </div>
       </el-header>
       <div class="login-person">
@@ -83,7 +83,7 @@
             <div style="font-size: 14px;color: #979db1;">
               {{ $t('retrievePassword.agreeTo') }}
               <div style="display: inline-block;color: #5778FF;cursor: pointer;">{{ $t('register.userAgreement') }}</div>
-              {{ $t('register.and') }}
+              {{ $t('login.and') }}
               <div style="display: inline-block;color: #5778FF;cursor: pointer;">{{ $t('register.privacyPolicy') }}</div>
             </div>
           </div>
@@ -103,6 +103,7 @@ import Api from '@/apis/api';
 import VersionFooter from '@/components/VersionFooter.vue';
 import { getUUID, goToPage, showDanger, showSuccess, validateMobile, sm2Encrypt } from '@/utils';
 import { mapState } from 'vuex';
+import i18n from '@/i18n';
 
 // 导入语言切换功能
 import { changeLanguage } from '@/i18n';
@@ -118,6 +119,28 @@ export default {
       mobileAreaList: state => state.pubConfig.mobileAreaList,
       sm2PublicKey: state => state.pubConfig.sm2PublicKey
     }),
+    // 获取当前语言
+    currentLanguage() {
+      return i18n.locale || "zh_CN";
+    },
+    // 根据当前语言获取对应的xiaozhi-ai图标
+    xiaozhiAiIcon() {
+      const currentLang = this.currentLanguage;
+      switch (currentLang) {
+        case "zh_CN":
+          return require("@/assets/xiaozhi-ai.png");
+        case "zh_TW":
+          return require("@/assets/xiaozhi-ai_zh_TW.png");
+        case "en":
+          return require("@/assets/xiaozhi-ai_en.png");
+        case "de":
+          return require("@/assets/xiaozhi-ai_de.png");
+        case "vi":
+          return require("@/assets/xiaozhi-ai_vi.png");
+        default:
+          return require("@/assets/xiaozhi-ai.png");
+      }
+    },
     canSendMobileCaptcha() {
       return this.countdown === 0 && validateMobile(this.form.mobile, this.form.areaCode);
     }

main/xiaozhi-server/app.py

@@ -3,11 +3,11 @@ import uuid
 import signal
 import asyncio
 from aioconsole import ainput
-from config.settings import load_config
+from config.config_loader import load_config
 from config.logger import setup_logging
 from core.utils.util import get_local_ip, validate_mcp_endpoint
 from core.http_server import SimpleHttpServer
-from core.websocket_server import WebSocketServer
+from core.xiaozhi_server_facade import XiaozhiServerFacade
 from core.utils.util import check_ffmpeg_installed
 from core.utils.gc_manager import get_gc_manager
 
@@ -68,9 +68,9 @@ async def main():
     gc_manager = get_gc_manager(interval_seconds=300)
     await gc_manager.start()
 
-    # 启动 WebSocket 服务器
+    # 启动小智服务器门面（支持WebSocket和MQTT）
-    ws_server = WebSocketServer(config)
+    xiaozhi_server = XiaozhiServerFacade(config)
-    ws_task = asyncio.create_task(ws_server.start())
+    xiaozhi_task = asyncio.create_task(xiaozhi_server.start())
     # 启动 Simple http 服务器
     ota_server = SimpleHttpServer(config)
     ota_task = asyncio.create_task(ota_server.start())
@@ -100,24 +100,52 @@ async def main():
             logger.bind(tag=TAG).error("mcp接入点不符合规范")
             config["mcp_endpoint"] = "你的接入点 websocket地址"
 
-    # 获取WebSocket配置，使用安全的默认值
+    # 显示协议连接信息
-    websocket_port = 8000
+    connection_info = xiaozhi_server.get_connection_info()
-    server_config = config.get("server", {})
+    
-    if isinstance(server_config, dict):
+    # WebSocket信息
-        websocket_port = int(server_config.get("port", 8000))
+    websocket_info = connection_info.get('websocket', {})
-
+    if websocket_info.get('enabled', False):
-    logger.bind(tag=TAG).info(
+        websocket_port = websocket_info.get('port', 8000)
-        "Websocket地址是\tws://{}:{}/xiaozhi/v1/",
+        logger.bind(tag=TAG).info(
-        get_local_ip(),
+            "WebSocket地址是\tws://{}:{}/xiaozhi/v1/",
-        websocket_port,
+            get_local_ip(),
-    )
+            websocket_port,
-
+        )
-    logger.bind(tag=TAG).info(
+    
-        "=======上面的地址是websocket协议地址，请勿用浏览器访问======="
+    # MQTT信息
-    )
+    mqtt_info = connection_info.get('mqtt', {})
-    logger.bind(tag=TAG).info(
+    if mqtt_info.get('enabled', False):
-        "如想测试websocket请用谷歌浏览器打开test目录下的test_page.html"
+        mqtt_port = mqtt_info.get('port', 1883)
-    )
+        udp_port = mqtt_info.get('udp_port', 1883)
+        logger.bind(tag=TAG).info(
+            "MQTT地址是\t\tmqtt://{}:{}",
+            get_local_ip(),
+            mqtt_port,
+        )
+        logger.bind(tag=TAG).info(
+            "UDP音频端口是\t{}:{}",
+            get_local_ip(),
+            udp_port,
+        )
+    
+    # 显示启用的协议
+    enabled_protocols = xiaozhi_server.config.get('enabled_protocols', [])
+    logger.bind(tag=TAG).info(f"启用的协议: {', '.join(enabled_protocols)}")
+    
+    if 'websocket' in enabled_protocols:
+        logger.bind(tag=TAG).info(
+            "=======上面的WebSocket地址请勿用浏览器访问======="
+        )
+        logger.bind(tag=TAG).info(
+            "如想测试WebSocket请用谷歌浏览器打开test目录下的test_page.html"
+        )
+    
+    if 'mqtt' in enabled_protocols:
+        logger.bind(tag=TAG).info(
+            "=======MQTT客户端ID格式: GID_test@@@mac_address@@@uuid======="
+        )
+    
     logger.bind(tag=TAG).info(
         "=============================================================\n"
     )
@@ -127,18 +155,28 @@ async def main():
     except asyncio.CancelledError:
         print("任务被取消，清理资源中...")
     finally:
+        # 停止小智服务器
+        try:
+            await xiaozhi_server.stop()
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"停止小智服务器失败: {e}")
+        
         # 停止全局GC管理器
         await gc_manager.stop()
 
         # 取消所有任务（关键修复点）
         stdin_task.cancel()
-        ws_task.cancel()
+        xiaozhi_task.cancel()
         if ota_task:
             ota_task.cancel()
 
         # 等待任务终止（必须加超时）
+        tasks_to_wait = [stdin_task, xiaozhi_task]
+        if ota_task:
+            tasks_to_wait.append(ota_task)
+            
         await asyncio.wait(
-            [stdin_task, ws_task, ota_task] if ota_task else [stdin_task, ws_task],
+            tasks_to_wait,
             timeout=3.0,
             return_when=asyncio.ALL_COMPLETED,
         )

main/xiaozhi-server/config.yaml

@@ -31,16 +31,70 @@ server:
   auth:
     # 是否启用认证
     enabled: false
+    # 设备的token，可以在编译固件的环节，写入你自己定义的token
+    # 固件上的token和以下的token如果能对应，才能连接本服务端
+    tokens:
+      - token: "your-token1" # 设备1的token
+        name: "your-device-name1"  # 设备1标识
+      - token: "your-token2"  # 设备2的token
+        name: "your-device-name2" # 设备2标识
     # 白名单设备ID列表
     # 如果属于白名单内的设备，不校验token，直接放行
     allowed_devices:
       - "11:22:33:44:55:66"
- # MQTT网关配置，用于通过OTA下发到设备，根据mqtt_gateway的.env文件配置，格式为host:port
+  # MQTT网关配置，用于通过OTA下发到设备，根据mqtt_gateway的.env文件配置，格式为host:port
   mqtt_gateway: null
   # MQTT签名密钥，用于生成MQTT连接密码，根据mqtt_gateway的.env文件配置
   mqtt_signature_key: null
   # UDP网关配置
   udp_gateway: null
+
+# #####################################################################################
+# #############################协议配置（Protocol Configuration）########################
+# 支持WebSocket和MQTT两种协议，可以单独启用或同时启用
+protocols:
+  # 启用的协议列表，可选值: ["websocket", "mqtt"]
+  enabled_protocols: ["websocket"]  # 默认只启用WebSocket
+  # WebSocket协议开关
+  websocket_enabled: true
+  # MQTT协议开关
+  mqtt_enabled: false
+
+# MQTT服务器配置（仅在mqtt_enabled为true时生效）
+mqtt_server:
+  # 是否启用MQTT服务器
+  enabled: false
+  # MQTT服务器监听地址
+  host: 0.0.0.0
+  # MQTT服务器端口
+  port: 1883
+  # UDP音频传输端口（通常与MQTT端口相同）
+  udp_port: 1883
+  # 公网IP地址（用于UDP音频传输配置）
+  # 如果使用docker部署或公网部署，请设置为实际的公网IP或域名
+  public_ip: localhost
+  # 最大连接数
+  max_connections: 1000
+  # 心跳检查间隔（秒）
+  heartbeat_interval: 30
+  # 最大消息载荷大小（字节）
+  max_payload_size: 8192
+
+# MQTT协议使用说明：
+# 1. 客户端ID格式：GID_test@@@mac_address@@@uuid 或 GID_test@@@mac_address
+#    例如：GID_test@@@aa:bb:cc:dd:ee:ff@@@unique_uuid_123
+# 2. 连接地址：mqtt://your.server.ip:1883
+# 3. 音频传输：通过UDP加密传输，配置信息在hello消息中返回
+# 4. 消息格式：JSON格式，支持hello、音频、文本等消息类型
+# 
+# 启用MQTT的配置示例：
+# protocols:
+#   enabled_protocols: ["websocket", "mqtt"]  # 同时启用两种协议
+#   mqtt_enabled: true
+# mqtt_server:
+#   enabled: true
+#   port: 1883
+#   public_ip: "your.server.ip"  # 替换为实际IP
 log:
   # 设置控制台输出的日志格式，时间、日志级别、标签、消息
   log_format: "<green>{time:YYMMDD HH:mm:ss}</green>[{version}_{selected_module}][<light-blue>{extra[tag]}</light-blue>]-<level>{level}</level>-<light-green>{message}</light-green>"
@@ -69,6 +123,9 @@ enable_greeting: true
 enable_stop_tts_notify: false
 # 说完话是否开启提示音，音效地址
 stop_tts_notify_voice: "config/assets/tts_notify.mp3"
+# 是否启用WebSocket心跳保活机制
+enable_websocket_ping: false
+
 
 # TTS音频发送延迟配置
 # tts_audio_send_delay: 控制音频包发送间隔
@@ -282,12 +339,15 @@ Memory:
     # 如果这里不填，则会默认使用selected_module.LLM的模型作为意图识别的思考模型
     # 如果你的不想使用selected_module.LLM记忆存储，这里最好使用独立的LLM作为意图识别，例如使用免费的ChatGLMLLM
     llm: ChatGLMLLM
-
 ASR:
   FunASR:
     type: fun_local
     model_dir: models/SenseVoiceSmall
     output_dir: tmp/
+    # 队列最大大小（可选，默认100）
+    # 当并发请求超过此值时，会返回"服务繁忙"提示
+    # 建议根据服务器性能调整，GPU 服务器可适当增大
+    queue_max_size: 100
   FunASRServer:
     # 独立部署FunASR，使用FunASR的API服务，只需要五句话
     # 第一句：mkdir -p ./funasr-runtime-resources/models
@@ -343,6 +403,8 @@ ASR:
     # 热词、替换词使用流程：https://www.volcengine.com/docs/6561/155738
     boosting_table_name: （选填）你的热词文件名称
     correct_table_name: （选填）你的替换词文件名称
+    # 静音判定时长(ms)，默认200ms
+    end_window_size: 200
     output_dir: tmp/
   TencentASR:
     # token申请地址：https://console.cloud.tencent.com/cam/capi
@@ -677,6 +739,8 @@ TTS:
     access_token: 你的火山引擎语音合成服务access_token
     resource_id: volc.service_type.10029
     speaker: zh_female_wanwanxiaohe_moon_bigtts
+    # 开启WebSocket连接复用，默认复用（注意：复用后设备处于聆听状态时空闲链接会占并发数）
+    enable_ws_reuse: True
     speech_rate: 0
     loudness_rate: 0
     pitch: 0

main/xiaozhi-server/config/config_loader.py

@@ -1,8 +1,229 @@
 import os
 import yaml
 from collections.abc import Mapping
+from typing import Any, Dict, Optional, Type, TypeVar, Union, get_type_hints, get_origin, get_args
+from dataclasses import dataclass, field, fields, MISSING
+import inspect
 from config.manage_api_client import init_service, get_server_config, get_agent_models
 
+T = TypeVar('T')
+
+
+class ConfigDict(dict):
+    """增强的配置字典，支持点号访问和嵌套获取"""
+    
+    def __init__(self, data: Dict[str, Any] = None):
+        super().__init__()
+        if data:
+            for key, value in data.items():
+                if isinstance(value, dict):
+                    self[key] = ConfigDict(value)
+                else:
+                    self[key] = value
+    
+    def __getattr__(self, key: str) -> Any:
+        """支持点号访问"""
+        try:
+            return self[key]
+        except KeyError:
+            raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{key}'")
+    
+    def __setattr__(self, key: str, value: Any) -> None:
+        """支持点号设置"""
+        self[key] = value
+    
+    def __getitem__(self, key: str) -> Any:
+        """重写[]访问，支持嵌套路径，找不到抛出KeyError"""
+        if '.' in key:
+            keys = key.split('.')
+            current = self
+            for k in keys:
+                if not isinstance(current, (dict, ConfigDict)):
+                    raise KeyError(f"Cannot access '{k}' on non-dict object")
+                current = super(ConfigDict, current).__getitem__(k)
+            return current
+        return super().__getitem__(key)
+    
+    def get(self, key: str, default: Any = None) -> Any:
+        """重写get方法，支持嵌套路径"""
+        try:
+            return self[key]
+        except KeyError:
+            return default
+    
+    def __setitem__(self, key: str, value: Any) -> None:
+        """重写[]设置，支持嵌套路径"""
+        if '.' in key:
+            keys = key.split('.')
+            current = self
+            for k in keys[:-1]:
+                if k not in current:
+                    current[k] = ConfigDict()
+                elif not isinstance(current[k], (dict, ConfigDict)):
+                    current[k] = ConfigDict()
+                current = current[k]
+            
+            if isinstance(value, dict) and not isinstance(value, ConfigDict):
+                value = ConfigDict(value)
+            super(ConfigDict, current).__setitem__(keys[-1], value)
+        else:
+            if isinstance(value, dict) and not isinstance(value, ConfigDict):
+                value = ConfigDict(value)
+            super().__setitem__(key, value)
+
+
+class ConfigField:
+    """配置字段，模仿dataclass的field功能"""
+    
+    def __init__(self, default=MISSING, default_factory=MISSING, prefix: str = None):
+        self.default = default
+        self.default_factory = default_factory
+        self.prefix = prefix
+        
+        if default is not MISSING and default_factory is not MISSING:
+            raise ValueError("Cannot specify both default and default_factory")
+
+
+def config_field(default=MISSING, default_factory=MISSING, prefix: str = None):
+    """创建配置字段"""
+    return ConfigField(default, default_factory, prefix)
+
+
+def _is_config_class(cls: Type) -> bool:
+    """检查类是否是ConfigurationProperties装饰的配置类"""
+    return hasattr(cls, '_config_prefix') and hasattr(cls, '_inject_config')
+
+
+def _create_nested_config_instance(config_class: Type, config: ConfigDict, config_path: str):
+    """创建嵌套配置类实例"""
+    try:
+        # 获取嵌套配置数据 - 直接传递整个config，让嵌套类自己处理前缀
+        # 因为嵌套类有自己的prefix，它会从config中正确提取数据
+        return config_class(config)
+    except Exception as e:
+        # 如果创建失败，返回None或抛出更详细的错误
+        raise ValueError(f"Failed to create nested config instance for {config_class.__name__} at path '{config_path}': {e}")
+
+
+def ConfigurationProperties(prefix: str = "", auto_inject: bool = True):
+    """
+    配置属性装饰器，模仿Spring Boot的@ConfigurationProperties
+    
+    Args:
+        prefix: 配置前缀，如 'server.database'
+        auto_inject: 是否自动注入配置
+    """
+    def decorator(cls: Type[T]) -> Type[T]:
+        if not inspect.isclass(cls):
+            raise TypeError("ConfigurationProperties can only be applied to classes")
+        
+        # 保存原始的__init__方法
+        original_init = cls.__init__ if hasattr(cls, '__init__') else None
+        
+        # 获取类的类型注解
+        type_hints = get_type_hints(cls)
+        
+        def new_init(self, config: ConfigDict = None, **kwargs):
+            # 如果有原始的__init__，先调用它
+            if original_init and original_init is not object.__init__:
+                try:
+                    original_init(self)
+                except TypeError:
+                    # 如果原始__init__不接受参数，忽略
+                    pass
+            
+            if config is None:
+                # 如果没有传入config，尝试从全局获取
+                config = getattr(self.__class__, '_global_config', None)
+                if config is None:
+                    return
+            
+            # 注入配置
+            self._inject_config(config, prefix, **kwargs)
+        
+        def _inject_config(self, config: ConfigDict, config_prefix: str = "", **overrides):
+            """注入配置到实例属性"""
+            # 处理类属性
+            for attr_name in dir(self.__class__):
+                if attr_name.startswith('_'):
+                    continue
+                    
+                attr_value = getattr(self.__class__, attr_name)
+                if isinstance(attr_value, ConfigField):
+                    # 确定配置路径
+                    field_prefix = attr_value.prefix or config_prefix
+                    config_path = f"{field_prefix}.{attr_name}" if field_prefix else attr_name
+                    
+                    # 从overrides或config获取值
+                    if attr_name in overrides:
+                        value = overrides[attr_name]
+                    else:
+                        # 检查是否有类型注解，如果是嵌套配置类则特殊处理
+                        attr_type = type_hints.get(attr_name)
+                        if attr_type and inspect.isclass(attr_type) and _is_config_class(attr_type):
+                            # 嵌套配置类，创建实例
+                            try:
+                                value = _create_nested_config_instance(attr_type, config, config_path)
+                            except ValueError as e:
+                                # 如果创建失败，使用默认值
+                                print(f"Warning: {e}")
+                                if attr_value.default_factory is not MISSING:
+                                    value = attr_value.default_factory()
+                                else:
+                                    value = attr_value.default
+                        else:
+                            # 普通类型，使用默认值逻辑
+                            if attr_value.default_factory is not MISSING:
+                                default_val = attr_value.default_factory()
+                            else:
+                                default_val = attr_value.default
+                            
+                            value = config.get(config_path, default_val)
+                    
+                    setattr(self, attr_name, value)
+            
+            # 处理类型注解的属性
+            for attr_name, attr_type in type_hints.items():
+                if hasattr(self, attr_name):
+                    continue  # 已经通过ConfigField处理过了
+                
+                config_path = f"{config_prefix}.{attr_name}" if config_prefix else attr_name
+                
+                if attr_name in overrides:
+                    value = overrides[attr_name]
+                else:
+                    # 检查是否是嵌套的ConfigurationProperties类
+                    if inspect.isclass(attr_type) and _is_config_class(attr_type):
+                        # 创建嵌套配置类实例
+                        try:
+                            value = _create_nested_config_instance(attr_type, config, config_path)
+                        except ValueError as e:
+                            # 如果创建失败，使用None或默认值
+                            print(f"Warning: {e}")
+                            value = None
+                    else:
+                        # 普通类型，直接从配置获取
+                        value = config.get(config_path)
+                
+                if value is not None:
+                    setattr(self, attr_name, value)
+        
+        # 添加方法到类
+        cls.__init__ = new_init
+        cls._inject_config = _inject_config
+        cls._config_prefix = prefix
+        
+        # 添加类方法用于设置全局配置
+        @classmethod
+        def set_global_config(cls, config: ConfigDict):
+            cls._global_config = config
+        
+        cls.set_global_config = set_global_config
+        
+        return cls
+    
+    return decorator
+
 
 def get_project_dir():
     """获取项目根目录"""
@@ -22,6 +243,9 @@ def load_config():
     # 检查缓存
     cached_config = cache_manager.get(CacheType.CONFIG, "main_config")
     if cached_config is not None:
+        # 确保返回的是ConfigDict类型
+        if not isinstance(cached_config, ConfigDict):
+            cached_config = ConfigDict(cached_config)
         return cached_config
 
     default_config_path = get_project_dir() + "config.yaml"
@@ -45,6 +269,10 @@ def load_config():
     else:
         # 合并配置
         config = merge_configs(default_config, custom_config)
+    
+    # 转换为ConfigDict
+    config = ConfigDict(config)
+    
     # 初始化目录
     ensure_directories(config)
 
@@ -82,7 +310,7 @@ async def get_config_from_api_async(config):
     # 如果服务器没有prompt_template，则从本地配置读取
     if not config_data.get("prompt_template"):
         config_data["prompt_template"] = config.get("prompt_template")
-    return config_data
+    return ConfigDict(config_data)
 
 
 async def get_private_config_from_api(config, device_id, client_id):
@@ -160,3 +388,218 @@ def merge_configs(default_config, custom_config):
             merged[key] = value
 
     return merged
+
+
+# 导出主要的类和函数
+__all__ = [
+    'ConfigDict', 
+    'ConfigField', 
+    'config_field', 
+    'ConfigurationProperties',
+    'load_config',
+    'get_project_dir',
+    'merge_configs'
+]
+
+
+# 配置类定义
+
+@ConfigurationProperties(prefix="server.database")
+class DatabaseConfig:
+    """数据库配置类"""
+    host: str = config_field(default="localhost")
+    port: int = config_field(default=3306)
+    username: str = config_field(default="root")
+    password: str = config_field(default="")
+    database: str = config_field(default="xiaozhi")
+
+
+@ConfigurationProperties(prefix="server.redis")
+class RedisConfig:
+    """Redis配置类"""
+    host: str = config_field(default="localhost")
+    port: int = config_field(default=6379)
+    password: str = config_field(default="")
+    db: int = config_field(default=0)
+
+
+@ConfigurationProperties(prefix="mqtt_server")
+class MQTTServerConfig:
+    """MQTT服务器配置类"""
+    enabled: bool = config_field(default=False)
+    host: str = config_field(default="0.0.0.0")
+    port: int = config_field(default=1883)
+    udp_port: int = config_field(default=1883)
+    public_ip: str = config_field(default="localhost")
+    max_connections: int = config_field(default=1000)
+    heartbeat_interval: int = config_field(default=30)
+    max_payload_size: int = config_field(default=8192)
+
+
+@ConfigurationProperties(prefix="server")
+class ServerConfig:
+    """服务器配置类"""
+    ip: str = config_field(default="0.0.0.0")
+    port: int = config_field(default=8080)
+    http_port: int = config_field(default=8081)
+    auth_key: str = config_field(default="")
+    vision_explain: str = config_field(default="")
+    
+    # 嵌套配置类
+    database: DatabaseConfig = config_field(default_factory=lambda: DatabaseConfig())
+    redis: RedisConfig = config_field(default_factory=lambda: RedisConfig())
+    mqtt_server: MQTTServerConfig = config_field(default_factory=lambda: MQTTServerConfig())
+
+
+@ConfigurationProperties(prefix="asr.whisper")
+class WhisperConfig:
+    """Whisper ASR配置类"""
+    model: str = config_field(default="base")
+    language: str = config_field(default="zh")
+    device: str = config_field(default="cpu")
+
+
+@ConfigurationProperties(prefix="asr")
+class ASRConfig:
+    """ASR配置类"""
+    provider: str = config_field(default="whisper")
+    # 嵌套配置
+    whisper: WhisperConfig = config_field(default_factory=lambda: WhisperConfig())
+
+
+@ConfigurationProperties(prefix="selected_module")
+class SelectedModuleConfig:
+    """选中模块配置类"""
+    ASR: str = config_field(default="")
+    TTS: str = config_field(default="")
+    LLM: str = config_field(default="")
+    VLLM: str = config_field(default="")
+    VAD: str = config_field(default="")
+    Memory: str = config_field(default="")
+    Intent: str = config_field(default="")
+
+
+@ConfigurationProperties(prefix="log")
+class LogConfig:
+    """日志配置类"""
+    log_dir: str = config_field(default="tmp")
+    level: str = config_field(default="INFO")
+
+
+@ConfigurationProperties(prefix="protocols")
+class ProtocolConfig:
+    """协议配置类"""
+    enabled_protocols: list = config_field(default_factory=lambda: ["websocket"])
+    websocket_enabled: bool = config_field(default=True)
+    mqtt_enabled: bool = config_field(default=False)
+
+
+@ConfigurationProperties(prefix="")
+class MainConfig:
+    """主配置类，包含常用的顶级配置"""
+    read_config_from_api: bool = config_field(default=False)
+    exit_commands: list = config_field(default_factory=list)
+    close_connection_no_voice_time: int = config_field(default=120)
+    xiaozhi: str = config_field(default="")
+    prompt: str = config_field(default="")
+    delete_audio: bool = config_field(default=True)
+    
+    # 协议配置
+    protocols: ProtocolConfig = config_field(default_factory=lambda: ProtocolConfig())
+
+
+@ConfigurationProperties(prefix="voiceprint")
+class VoiceprintConfig:
+    """声纹配置类"""
+    enabled: bool = config_field(default=False)
+    model_path: str = config_field(default="")
+    threshold: float = config_field(default=0.5)
+
+
+# 全局配置实例
+_global_config_dict: ConfigDict = None
+_server_config: ServerConfig = None
+_selected_module_config: SelectedModuleConfig = None
+_log_config: LogConfig = None
+_main_config: MainConfig = None
+_voiceprint_config: VoiceprintConfig = None
+_mqtt_server_config: MQTTServerConfig = None
+_protocol_config: ProtocolConfig = None
+
+
+def get_config_instance(config_class: Type[T]) -> T:
+    """获取配置类实例的工厂方法"""
+    global _global_config_dict
+    if _global_config_dict is None:
+        _global_config_dict = load_config()
+    
+    return config_class(_global_config_dict)
+
+
+def get_server_config() -> ServerConfig:
+    """获取服务器配置实例"""
+    global _server_config
+    if _server_config is None:
+        _server_config = get_config_instance(ServerConfig)
+    return _server_config
+
+
+def get_selected_module_config() -> SelectedModuleConfig:
+    """获取选中模块配置实例"""
+    global _selected_module_config
+    if _selected_module_config is None:
+        _selected_module_config = get_config_instance(SelectedModuleConfig)
+    return _selected_module_config
+
+
+def get_log_config() -> LogConfig:
+    """获取日志配置实例"""
+    global _log_config
+    if _log_config is None:
+        _log_config = get_config_instance(LogConfig)
+    return _log_config
+
+
+def get_main_config() -> MainConfig:
+    """获取主配置实例"""
+    global _main_config
+    if _main_config is None:
+        _main_config = get_config_instance(MainConfig)
+    return _main_config
+
+
+def get_voiceprint_config() -> VoiceprintConfig:
+    """获取声纹配置实例"""
+    global _voiceprint_config
+    if _voiceprint_config is None:
+        _voiceprint_config = get_config_instance(VoiceprintConfig)
+    return _voiceprint_config
+
+
+def get_mqtt_server_config() -> MQTTServerConfig:
+    """获取MQTT服务器配置实例"""
+    global _mqtt_server_config
+    if _mqtt_server_config is None:
+        _mqtt_server_config = get_config_instance(MQTTServerConfig)
+    return _mqtt_server_config
+
+
+def get_protocol_config() -> ProtocolConfig:
+    """获取协议配置实例"""
+    global _protocol_config
+    if _protocol_config is None:
+        _protocol_config = get_config_instance(ProtocolConfig)
+    return _protocol_config
+
+
+def refresh_config():
+    """刷新所有配置实例"""
+    global _global_config_dict, _server_config, _selected_module_config, _log_config, _main_config, _voiceprint_config, _mqtt_server_config, _protocol_config
+    _global_config_dict = None
+    _server_config = None
+    _selected_module_config = None
+    _log_config = None
+    _main_config = None
+    _voiceprint_config = None
+    _mqtt_server_config = None
+    _protocol_config = None

main/xiaozhi-server/config/manage_api_client.py

@@ -53,6 +53,7 @@ class ManageApiClient:
     async def _ensure_async_client(cls):
         """确保异步客户端已创建（为每个事件循环创建独立的客户端）"""
         import asyncio
+
         try:
             loop = asyncio.get_running_loop()
             loop_id = id(loop)
@@ -115,6 +116,7 @@ class ManageApiClient:
     async def _execute_async_request(cls, method: str, endpoint: str, **kwargs) -> Dict:
         """带重试机制的异步请求执行器"""
         import asyncio
+
         retry_count = 0
 
         while retry_count <= cls.max_retries:
@@ -138,6 +140,7 @@ class ManageApiClient:
     def safe_close(cls):
         """安全关闭所有异步连接池"""
         import asyncio
+
         for client in list(cls._async_clients.values()):
             try:
                 asyncio.run(client.aclose())
@@ -149,7 +152,9 @@ class ManageApiClient:
 
 async def get_server_config() -> Optional[Dict]:
     """获取服务器基础配置"""
-    return await ManageApiClient._instance._execute_async_request("POST", "/config/server-base")
+    return await ManageApiClient._instance._execute_async_request(
+        "POST", "/config/server-base"
+    )
 
 
 async def get_agent_models(
@@ -167,17 +172,15 @@ async def get_agent_models(
     )
 
 
-async def save_mem_local_short(mac_address: str, short_momery: str) -> Optional[Dict]:
+async def generate_and_save_chat_summary(session_id: str) -> Optional[Dict]:
+    """生成并保存聊天记录总结"""
     try:
         return await ManageApiClient._instance._execute_async_request(
-            "PUT",
+            "POST",
-            f"/agent/saveMemory/" + mac_address,
+            f"/agent/chat-summary/{session_id}/save",
-            json={
-                "summaryMemory": short_momery,
-            },
         )
     except Exception as e:
-        print(f"存储短期记忆到服务器失败: {e}")
+        print(f"生成并保存聊天记录总结失败: {e}")
         return None
 
 

main/xiaozhi-server/config/mqtt_config_example.yaml

@@ -0,0 +1,54 @@
+# MQTT协议配置示例
+# 将此配置添加到你的主配置文件中
+
+# 协议配置
+protocols:
+  enabled_protocols: ["websocket", "mqtt"]  # 启用的协议列表
+  websocket_enabled: true                   # WebSocket协议开关
+  mqtt_enabled: true                        # MQTT协议开关
+
+# MQTT服务器配置
+mqtt_server:
+  enabled: true                    # 是否启用MQTT服务器
+  host: "0.0.0.0"                 # 监听地址
+  port: 1883                      # MQTT端口
+  udp_port: 1883                  # UDP端口（用于音频传输）
+  public_ip: "your.server.ip"     # 公网IP地址
+  max_connections: 1000           # 最大连接数
+  heartbeat_interval: 30          # 心跳检查间隔（秒）
+  max_payload_size: 8192          # 最大消息载荷大小
+
+# 服务器配置（扩展）
+server:
+  ip: "0.0.0.0"
+  port: 8080                      # WebSocket端口
+  http_port: 8081
+  auth_key: ""
+  vision_explain: ""
+  
+  # MQTT服务器配置（嵌套）
+  mqtt_server:
+    enabled: true
+    host: "0.0.0.0"
+    port: 1883
+    udp_port: 1883
+    public_ip: "localhost"
+    max_connections: 1000
+    heartbeat_interval: 30
+    max_payload_size: 8192
+
+# 使用示例：
+# 1. 启动多协议服务器：
+#    python main_multi_protocol.py
+#
+# 2. WebSocket客户端连接：
+#    ws://your.server.ip:8080/
+#
+# 3. MQTT客户端连接：
+#    mqtt://your.server.ip:1883
+#    客户端ID格式：GID_test@@@mac_address@@@uuid
+#    或：GID_test@@@mac_address
+#
+# 4. UDP音频传输：
+#    客户端通过MQTT接收UDP配置后，使用UDP发送音频数据
+

main/xiaozhi-server/core/api/base_handler.py

@@ -10,7 +10,15 @@ class BaseHandler:
     def _add_cors_headers(self, response):
         """添加CORS头信息"""
         response.headers["Access-Control-Allow-Headers"] = (
-            "client-id, content-type, device-id"
+            "client-id, content-type, device-id, authorization"
         )
         response.headers["Access-Control-Allow-Credentials"] = "true"
         response.headers["Access-Control-Allow-Origin"] = "*"
+
+    async def handle_options(self, request):
+        """处理OPTIONS请求，添加CORS头信息"""
+        response = web.Response(body=b"", content_type="text/plain")
+        self._add_cors_headers(response)
+        # 添加允许的方法
+        response.headers["Access-Control-Allow-Methods"] = "GET, POST, OPTIONS"
+        return response

main/xiaozhi-server/core/api/ota_handler.py

@@ -3,15 +3,46 @@ import time
 import base64
 import hashlib
 import hmac
+import os
+import re
+import glob
+from typing import Dict, List, Tuple
 from aiohttp import web
 
 from core.auth import AuthManager
-from core.utils.util import get_local_ip
+from core.utils.util import get_local_ip, get_vision_url
 from core.api.base_handler import BaseHandler
 
 TAG = __name__
 
 
+def _safe_basename(filename: str) -> str:
+    # Prevent directory traversal
+    return os.path.basename(filename)
+
+
+def _parse_version(ver: str) -> Tuple[int, ...]:
+    # conservative parser: split by non-digit, keep numeric parts
+    parts = re.findall(r"\d+", ver)
+    return tuple(int(p) for p in parts) if parts else (0,)
+
+
+def _is_higher_version(a: str, b: str) -> bool:
+    """Return True if version string a > b (semver-like numeric compare)."""
+    ta = _parse_version(a)
+    tb = _parse_version(b)
+    # compare tuple lexicographically, but allow different lengths
+    maxlen = max(len(ta), len(tb))
+    for i in range(maxlen):
+        ai = ta[i] if i < len(ta) else 0
+        bi = tb[i] if i < len(tb) else 0
+        if ai > bi:
+            return True
+        if ai < bi:
+            return False
+    return False
+
+
 class OTAHandler(BaseHandler):
     def __init__(self, config: dict):
         super().__init__(config)
@@ -23,6 +54,54 @@ class OTAHandler(BaseHandler):
         expire_seconds = auth_config.get("expire_seconds")
         self.auth = AuthManager(secret_key=secret_key, expire_seconds=expire_seconds)
 
+        # firmware storage
+        self.bin_dir = os.path.join(os.getcwd(), "data", "bin")
+        # cache structure: { 'updated_at': timestamp, 'ttl': seconds, 'files_by_model': { model: [(version, filename), ...] } }
+        self._bin_cache: Dict = {
+            "updated_at": 0,
+            "ttl": config.get("firmware_cache_ttl", 30),
+            "files_by_model": {},
+        }
+
+    def _refresh_bin_cache_if_needed(self):
+        now = int(time.time())
+        ttl = int(self._bin_cache.get("ttl", 30))
+        if now - int(
+            self._bin_cache.get("updated_at", 0)
+        ) < ttl and self._bin_cache.get("files_by_model"):
+            return
+
+        files_by_model: Dict[str, List[Tuple[str, str]]] = {}
+        try:
+            if not os.path.isdir(self.bin_dir):
+                os.makedirs(self.bin_dir, exist_ok=True)
+
+            # match files like model_1.2.3.bin (allow dots, dashes, underscores in model and version)
+            pattern = os.path.join(self.bin_dir, "*.bin")
+            for path in glob.glob(pattern):
+                fname = os.path.basename(path)
+                # filename format: {model}_{version}.bin
+                m = re.match(r"^(.+?)_([0-9][A-Za-z0-9\.\-_]*)\.bin$", fname)
+                if not m:
+                    # skip files not conforming to naming rule
+                    continue
+                model = m.group(1)
+                version = m.group(2)
+                files_by_model.setdefault(model, []).append((version, fname))
+
+            # sort versions for each model descending
+            for model, items in files_by_model.items():
+                items.sort(key=lambda it: _parse_version(it[0]), reverse=True)
+
+            self._bin_cache["files_by_model"] = files_by_model
+            self._bin_cache["updated_at"] = now
+            self.logger.bind(tag=TAG).info(
+                f"Firmware cache refreshed: {len(files_by_model)} models"
+            )
+        except Exception as e:
+            self.logger.bind(tag=TAG).error(f"刷新固件缓存失败: {e}")
+            # keep previous cache if any
+
     def generate_password_signature(self, content: str, secret_key: str) -> str:
         """生成MQTT密码签名
 
@@ -62,7 +141,14 @@ class OTAHandler(BaseHandler):
             return f"ws://{local_ip}:{port}/xiaozhi/v1/"
 
     async def handle_post(self, request):
-        """处理 OTA POST 请求"""
+        """处理 OTA POST 请求
+
+        This handler will:
+        - read device id/client id (as before)
+        - attempt to determine device model and current firmware version (prefer headers, fallback to body)
+        - check data/bin for newer firmware for that model
+        - if found a newer firmware, set firmware.url to the download endpoint
+        """
         try:
             data = await request.text()
             self.logger.bind(tag=TAG).debug(f"OTA请求方法: {request.method}")
@@ -81,33 +167,76 @@ class OTAHandler(BaseHandler):
             else:
                 raise Exception("OTA请求ClientID为空")
 
-            data_json = json.loads(data)
+            data_json = {}
+            try:
+                data_json = json.loads(data) if data else {}
+            except Exception:
+                data_json = {}
 
             server_config = self.config["server"]
-            port = int(server_config.get("port", 8000))
+            # Distinguish ports:
+            # - websocket_port is used to construct websocket URL (server["port"])
+            # - http_port is used to construct OTA download URLs (server["http_port"])
+            websocket_port = int(server_config.get("port", 8000))
+            http_port = int(server_config.get("http_port", 8003))
             local_ip = get_local_ip()
 
+            # Determine device model (prefer headers)
+            device_model = ""
+            # header candidates
+            for h in ("device-model", "device_model", "model"):
+                if h in request.headers:
+                    device_model = request.headers.get(h, "").strip()
+                    break
+            # body fallback
+            if not device_model:
+                try:
+                    if "board" in data_json and isinstance(data_json["board"], dict):
+                        device_model = data_json["board"].get("type", "")
+                    elif "model" in data_json:
+                        device_model = data_json.get("model", "")
+                except Exception:
+                    device_model = ""
+            if not device_model:
+                device_model = "default"
+
+            # Determine device current version (prefer headers)
+            device_version = ""
+            for h in (
+                "device-version",
+                "device_version",
+                "firmware-version",
+                "app-version",
+                "application-version",
+            ):
+                if h in request.headers:
+                    device_version = request.headers.get(h, "").strip()
+                    break
+            if not device_version:
+                try:
+                    device_version = data_json.get("application", {}).get("version", "")
+                except Exception:
+                    device_version = ""
+            if not device_version:
+                device_version = "0.0.0"
+
             return_json = {
                 "server_time": {
                     "timestamp": int(round(time.time() * 1000)),
                     "timezone_offset": server_config.get("timezone_offset", 8) * 60,
                 },
                 "firmware": {
-                    "version": data_json["application"].get("version", "1.0.0"),
+                    "version": device_version,
                     "url": "",
                 },
             }
 
+            # existing mqtt/websocket logic (unchanged)
             mqtt_gateway_endpoint = server_config.get("mqtt_gateway")
 
             if mqtt_gateway_endpoint:  # 如果配置了非空字符串
-                # 尝试从请求数据中获取设备型号
+                # 尝试从请求数据中获取设备型号（已解析 above）
-                device_model = "default"
                 try:
-                    if "device" in data_json and isinstance(data_json["device"], dict):
-                        device_model = data_json["device"].get("model", "default")
-                    elif "model" in data_json:
-                        device_model = data_json["model"]
                     group_id = f"GID_{device_model}".replace(":", "_").replace(" ", "_")
                 except Exception as e:
                     self.logger.bind(tag=TAG).error(f"获取设备型号失败: {e}")
@@ -159,20 +288,61 @@ class OTAHandler(BaseHandler):
                             token = self.auth.generate_token(client_id, device_id)
                     else:
                         token = self.auth.generate_token(client_id, device_id)
+                # NOTE: use websocket_port here
                 return_json["websocket"] = {
-                    "url": self._get_websocket_url(local_ip, port),
+                    "url": self._get_websocket_url(local_ip, websocket_port),
                     "token": token,
                 }
                 self.logger.bind(tag=TAG).info(
                     f"未配置MQTT网关，为设备 {device_id} 下发WebSocket配置"
                 )
-                self.logger.bind(tag=TAG).info(f"{return_json}")
+
+            # Now check firmware files for updates
+            try:
+                self._refresh_bin_cache_if_needed()
+                files_by_model = self._bin_cache.get("files_by_model", {})
+                candidates = files_by_model.get(device_model, [])
+
+                self.logger.bind(tag=TAG).info(
+                    f"查找型号 {device_model} 的固件，找到 {len(candidates)} 个候选"
+                )
+
+                chosen_url = ""
+                chosen_version = device_version
+
+                # candidates are sorted descending by version
+                for ver, fname in candidates:
+                    if _is_higher_version(ver, device_version):
+                        # build download url (only allow download via our download endpoint)
+                        chosen_version = ver
+                        # Use get_vision_url to get the base URL and replace the path
+                        vision_url = get_vision_url(self.config)
+                        # Replace the path from "/mcp/vision/explain" to "/xiaozhi/ota/download/{fname}"
+                        chosen_url = vision_url.replace(
+                            "/mcp/vision/explain", f"/xiaozhi/ota/download/{fname}"
+                        )
+                        break
+
+                if chosen_url:
+                    return_json["firmware"]["version"] = chosen_version
+                    return_json["firmware"]["url"] = chosen_url
+                    self.logger.bind(tag=TAG).info(
+                        f"为设备 {device_id} 下发固件 {chosen_version} [如果地址前缀有误，请检查配置文件中的server.vision_explain]-> {chosen_url} "
+                    )
+                else:
+                    self.logger.bind(tag=TAG).info(
+                        f"设备 {device_id} 固件已是最新: {device_version}"
+                    )
+
+            except Exception as e:
+                self.logger.bind(tag=TAG).error(f"检查固件版本时出错: {e}")
 
             response = web.Response(
                 text=json.dumps(return_json, separators=(",", ":")),
                 content_type="application/json",
             )
         except Exception as e:
+            self.logger.bind(tag=TAG).error(f"OTA POST处理异常: {e}")
             return_json = {"success": False, "message": "request error."}
             response = web.Response(
                 text=json.dumps(return_json, separators=(",", ":")),
@@ -187,8 +357,9 @@ class OTAHandler(BaseHandler):
         try:
             server_config = self.config["server"]
             local_ip = get_local_ip()
-            port = int(server_config.get("port", 8000))
+            # use websocket port for websocket URL
-            websocket_url = self._get_websocket_url(local_ip, port)
+            websocket_port = int(server_config.get("port", 8000))
+            websocket_url = self._get_websocket_url(local_ip, websocket_port)
             message = f"OTA接口运行正常，向设备发送的websocket地址是：{websocket_url}"
             response = web.Response(text=message, content_type="text/plain")
         except Exception as e:
@@ -197,3 +368,48 @@ class OTAHandler(BaseHandler):
         finally:
             self._add_cors_headers(response)
             return response
+
+    async def handle_download(self, request):
+        """
+        下载固件接口
+        URL: /xiaozhi/ota/download/{filename}
+        - 只允许下载 data/bin 目录下的 .bin 文件
+        - filename 必须是 basename 且匹配安全的模式
+        """
+        try:
+            fname = request.match_info.get("filename", "")
+            if not fname:
+                raise web.HTTPBadRequest(text="filename required")
+
+            # sanitize
+            fname = _safe_basename(fname)
+            # pattern: allow letters, numbers, dot, underscore, dash
+            if not re.match(r"^[A-Za-z0-9\.\-_]+\.bin$", fname):
+                raise web.HTTPBadRequest(text="invalid filename")
+
+            file_path = os.path.join(self.bin_dir, fname)
+            # ensure realpath is under bin_dir
+            file_real = os.path.realpath(file_path)
+            bin_dir_real = os.path.realpath(self.bin_dir)
+            if (
+                not file_real.startswith(bin_dir_real + os.sep)
+                and file_real != bin_dir_real
+            ):
+                raise web.HTTPForbidden(text="forbidden")
+
+            if not os.path.isfile(file_real):
+                raise web.HTTPNotFound(text="file not found")
+
+            # use FileResponse to stream file
+            resp = web.FileResponse(path=file_real)
+        except web.HTTPError as e:
+            resp = e
+        except Exception as e:
+            self.logger.bind(tag=TAG).error(f"固件下载异常: {e}")
+            resp = web.Response(text="download error", status=500)
+        finally:
+            try:
+                self._add_cors_headers(resp)
+            except Exception:
+                pass
+            return resp

main/xiaozhi-server/core/api/vision_handler.py

@@ -2,6 +2,7 @@ import json
 import copy
 from aiohttp import web
 from config.logger import setup_logging
+from core.api.base_handler import BaseHandler
 from core.utils.util import get_vision_url, is_valid_image_file
 from core.utils.vllm import create_instance
 from config.config_loader import get_private_config_from_api
@@ -16,10 +17,9 @@ TAG = __name__
 MAX_FILE_SIZE = 5 * 1024 * 1024
 
 
-class VisionHandler:
+class VisionHandler(BaseHandler):
     def __init__(self, config: dict):
-        self.config = config
+        super().__init__(config)
-        self.logger = setup_logging()
         # 初始化认证工具
         self.auth = AuthToken(config["server"]["auth_key"])
 
@@ -172,11 +172,3 @@ class VisionHandler:
         finally:
             self._add_cors_headers(response)
             return response
-
-    def _add_cors_headers(self, response):
-        """添加CORS头信息"""
-        response.headers["Access-Control-Allow-Headers"] = (
-            "client-id, content-type, device-id"
-        )
-        response.headers["Access-Control-Allow-Credentials"] = "true"
-        response.headers["Access-Control-Allow-Origin"] = "*"

main/xiaozhi-server/core/auth.py

@@ -10,6 +10,140 @@ class AuthenticationError(Exception):
     pass
 
 
+class AuthMiddleware:
+    """
+    认证中间件
+    用于 WebSocket/MQTT 连接认证
+    集成 AuthManager 的 token 验证逻辑，支持多种认证方式
+    """
+
+    def __init__(self, config: dict):
+        """
+        初始化认证中间件
+        
+        Args:
+            config: 配置字典，包含认证相关配置
+        """
+        self.config = config
+        server_config = config.get("server", {})
+        auth_config = server_config.get("auth", {})
+        
+        self.enabled = auth_config.get("enabled", False)
+        self.tokens = auth_config.get("tokens", [])
+        self.allowed_devices = set(auth_config.get("allowed_devices", []))
+        
+        # 获取 auth_key 用于 HMAC token 验证
+        self.auth_key = server_config.get("auth_key", "")
+        expire_seconds = auth_config.get("expire_seconds", None)
+        
+        # 创建 AuthManager 实例用于 HMAC token 验证
+        if self.auth_key:
+            self._auth_manager = AuthManager(
+                secret_key=self.auth_key,
+                expire_seconds=expire_seconds
+            )
+        else:
+            self._auth_manager = None
+
+    def authenticate(self, device_id: str, token: str = None, client_id: str = None) -> bool:
+        """
+        验证设备认证（同步方法）
+        
+        Args:
+            device_id: 设备 ID
+            token: 认证令牌（可以是静态 token 或 HMAC token）
+            client_id: 客户端 ID（用于 HMAC token 验证）
+            
+        Returns:
+            bool: 认证是否通过
+        """
+        if not self.enabled:
+            return True
+        
+        # 1. 检查白名单
+        if device_id and device_id in self.allowed_devices:
+            return True
+        
+        # 2. 检查静态 token
+        if token:
+            # 移除 Bearer 前缀（如果有）
+            if token.startswith("Bearer "):
+                token = token[7:]
+            
+            for token_config in self.tokens:
+                if token_config.get("token") == token:
+                    return True
+        
+        # 3. 检查 HMAC token（需要 AuthManager）
+        if token and self._auth_manager and client_id and device_id:
+            if self._auth_manager.verify_token(token, client_id, device_id):
+                return True
+        
+        return False
+
+    async def authenticate_async(self, headers: dict) -> bool:
+        """
+        从 headers 中提取信息并进行异步认证
+        
+        Args:
+            headers: HTTP 请求头字典
+            
+        Returns:
+            bool: 认证是否通过
+            
+        Raises:
+            AuthenticationError: 认证失败时抛出
+        """
+        if not self.enabled:
+            return True
+        
+        device_id = headers.get("device-id")
+        client_id = headers.get("client-id")
+        authorization = headers.get("authorization", "")
+        
+        # 提取 token
+        token = None
+        if authorization:
+            if authorization.startswith("Bearer "):
+                token = authorization[7:]
+            else:
+                token = authorization
+        
+        # 执行认证
+        if self.authenticate(device_id, token, client_id):
+            return True
+        
+        raise AuthenticationError(f"认证失败: device_id={device_id}")
+
+    def authenticate_websocket(self, websocket) -> bool:
+        """
+        WebSocket 连接认证
+        
+        Args:
+            websocket: WebSocket 连接对象
+            
+        Returns:
+            bool: 认证是否通过
+        """
+        if not self.enabled:
+            return True
+        
+        headers = dict(websocket.request.headers)
+        device_id = headers.get("device-id")
+        client_id = headers.get("client-id")
+        authorization = headers.get("authorization", "")
+        
+        # 提取 token
+        token = None
+        if authorization:
+            if authorization.startswith("Bearer "):
+                token = authorization[7:]
+            else:
+                token = authorization
+        
+        return self.authenticate(device_id, token, client_id)
+
+
 class AuthManager:
     """
     统一授权认证管理器

main/xiaozhi-server/core/components/adapters/asr_adapter.py

@@ -0,0 +1,105 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import asr
+from core.utils.modules_initialize import initialize_asr
+from config.logger import setup_logging
+
+logger = setup_logging()
+TAG = __name__
+
+
+class ASRAdapter(Component):
+    """
+    ASR组件适配器：将现有ASR组件包装为新的组件接口
+    
+    支持两种模式：
+    1. 共享实例模式：使用 SharedASRManager 的全局共享实例
+    2. 独立实例模式：每个连接创建独立的 ASR 实例（原有逻辑）
+    """
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.ASR, config)
+        self._asr_instance = None
+        self._delete_audio = config.get("delete_audio", True)
+        self._using_shared = False  # 是否使用共享实例
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化ASR组件"""
+        try:
+            # 获取ASR配置
+            selected_module = self.config.get("selected_module", {}).get("ASR")
+            if not selected_module:
+                raise ValueError("未配置ASR模块")
+            
+            # 检查是否有全局共享 ASR 管理器
+            shared_manager = getattr(context, 'shared_asr_manager', None)
+            
+            if shared_manager and shared_manager.is_ready():
+                # 使用共享实例模式
+                logger.bind(tag=TAG).info(f"使用共享 ASR 实例: {selected_module}")
+                from core.providers.asr.shared_asr_proxy import SharedASRProxy
+                self._asr_instance = SharedASRProxy(shared_manager)
+                self._using_shared = True
+            else:
+                # 使用独立实例模式（原有逻辑）
+                logger.bind(tag=TAG).info(f"使用独立 ASR 实例: {selected_module}")
+                self._asr_instance = initialize_asr(self.config)
+                self._using_shared = False
+            
+            # 注册资源以便清理（仅非共享实例）
+            if not self._using_shared:
+                self.add_resource(self._asr_instance)
+            
+            # 打开音频通道（如果需要）
+            if hasattr(self._asr_instance, 'open_audio_channels'):
+                await self._asr_instance.open_audio_channels(context)
+            
+            logger.bind(tag=TAG).info(
+                f"ASR组件初始化完成: {selected_module}, "
+                f"共享模式: {self._using_shared}"
+            )
+            
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"ASR组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理ASR组件"""
+        if self._asr_instance:
+            try:
+                # 如果是共享实例，不需要关闭（由服务器统一管理）
+                if self._using_shared:
+                    logger.bind(tag=TAG).debug("共享 ASR 实例，跳过清理")
+                else:
+                    # 关闭独立 ASR 实例
+                    if hasattr(self._asr_instance, 'close'):
+                        await self._asr_instance.close()
+                    
+                    # 清理音频文件
+                    if hasattr(self._asr_instance, 'cleanup_audio_files'):
+                        self._asr_instance.cleanup_audio_files()
+                    
+                    logger.bind(tag=TAG).info("ASR组件清理完成")
+                
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"ASR组件清理失败: {e}")
+            finally:
+                self._asr_instance = None
+                self._using_shared = False
+    
+    @property
+    def asr_instance(self):
+        """获取ASR实例"""
+        return self._asr_instance
+
+
+class ASRFactory(ComponentFactory):
+    """ASR组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return ASRAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.ASR
+
+

main/xiaozhi-server/core/components/adapters/intent_adapter.py

@@ -0,0 +1,85 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import intent
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class IntentAdapter(Component):
+    """Intent组件适配器：将现有Intent组件包装为新的组件接口"""
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.INTENT, config)
+        self._intent_instance = None
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化Intent组件"""
+        try:
+            # 获取Intent配置
+            selected_module = self.config.get("selected_module", {}).get("Intent")
+            if not selected_module:
+                raise ValueError("未配置Intent模块")
+            
+            # 获取Intent类型
+            intent_type = (
+                selected_module
+                if "type" not in self.config["Intent"][selected_module]
+                else self.config["Intent"][selected_module]["type"]
+            )
+            
+            # 创建Intent实例
+            self._intent_instance = intent.create_instance(
+                intent_type,
+                self.config["Intent"][selected_module],
+            )
+            
+            # 注册资源以便清理
+            self.add_resource(self._intent_instance)
+            
+            # 设置LLM（如果需要）
+            if intent_type in ["intent_llm", "function_call"]:
+                llm_component = context.components.get('llm')
+                if llm_component and hasattr(llm_component, 'llm_instance'):
+                    if hasattr(self._intent_instance, 'set_llm'):
+                        self._intent_instance.set_llm(llm_component.llm_instance)
+            
+            logger.info(f"Intent组件初始化完成: {intent_type}")
+            
+        except Exception as e:
+            logger.error(f"Intent组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理Intent组件"""
+        if self._intent_instance:
+            try:
+                # 关闭Intent实例
+                if hasattr(self._intent_instance, 'close'):
+                    await self._intent_instance.close()
+                elif hasattr(self._intent_instance, 'cleanup'):
+                    await self._intent_instance.cleanup()
+                
+                logger.info("Intent组件清理完成")
+                
+            except Exception as e:
+                logger.error(f"Intent组件清理失败: {e}")
+            finally:
+                self._intent_instance = None
+    
+    @property
+    def intent_instance(self):
+        """获取Intent实例"""
+        return self._intent_instance
+
+
+class IntentFactory(ComponentFactory):
+    """Intent组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return IntentAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.INTENT
+
+

main/xiaozhi-server/core/components/adapters/llm_adapter.py

@@ -0,0 +1,78 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import llm
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class LLMAdapter(Component):
+    """LLM组件适配器：将现有LLM组件包装为新的组件接口"""
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.LLM, config)
+        self._llm_instance = None
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化LLM组件"""
+        try:
+            # 获取LLM配置
+            selected_module = self.config.get("selected_module", {}).get("LLM")
+            if not selected_module:
+                raise ValueError("未配置LLM模块")
+            
+            # 获取LLM类型
+            llm_type = (
+                selected_module
+                if "type" not in self.config["LLM"][selected_module]
+                else self.config["LLM"][selected_module]["type"]
+            )
+            
+            # 创建LLM实例
+            self._llm_instance = llm.create_instance(
+                llm_type,
+                self.config["LLM"][selected_module],
+            )
+            
+            # 注册资源以便清理
+            self.add_resource(self._llm_instance)
+            
+            logger.info(f"LLM组件初始化完成: {llm_type}")
+            
+        except Exception as e:
+            logger.error(f"LLM组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理LLM组件"""
+        if self._llm_instance:
+            try:
+                # 关闭LLM实例
+                if hasattr(self._llm_instance, 'close'):
+                    await self._llm_instance.close()
+                elif hasattr(self._llm_instance, 'cleanup'):
+                    await self._llm_instance.cleanup()
+                
+                logger.info("LLM组件清理完成")
+                
+            except Exception as e:
+                logger.error(f"LLM组件清理失败: {e}")
+            finally:
+                self._llm_instance = None
+    
+    @property
+    def llm_instance(self):
+        """获取LLM实例"""
+        return self._llm_instance
+
+
+class LLMFactory(ComponentFactory):
+    """LLM组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return LLMAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.LLM
+
+

main/xiaozhi-server/core/components/adapters/memory_adapter.py

@@ -0,0 +1,96 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import memory
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class MemoryAdapter(Component):
+    """Memory组件适配器：将现有Memory组件包装为新的组件接口"""
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.MEMORY, config)
+        self._memory_instance = None
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化Memory组件"""
+        try:
+            # 获取Memory配置
+            selected_module = self.config.get("selected_module", {}).get("Memory")
+            if not selected_module:
+                raise ValueError("未配置Memory模块")
+            
+            # 获取Memory类型
+            memory_type = (
+                selected_module
+                if "type" not in self.config["Memory"][selected_module]
+                else self.config["Memory"][selected_module]["type"]
+            )
+            
+            # 创建Memory实例
+            self._memory_instance = memory.create_instance(
+                memory_type,
+                self.config["Memory"][selected_module],
+                self.config.get("summaryMemory", None),
+            )
+            
+            # 注册资源以便清理
+            self.add_resource(self._memory_instance)
+            
+            # 初始化记忆模块
+            if hasattr(self._memory_instance, 'init_memory'):
+                # 需要LLM实例来初始化记忆
+                llm_component = context.components.get('llm')
+                if llm_component and hasattr(llm_component, 'llm_instance'):
+                    self._memory_instance.init_memory(
+                        role_id=context.device_id,
+                        llm=llm_component.llm_instance,
+                        summary_memory=self.config.get("summaryMemory", None),
+                        save_to_file=not self.config.get("read_config_from_api", False),
+                    )
+            
+            logger.info(f"Memory组件初始化完成: {memory_type}")
+            
+        except Exception as e:
+            logger.error(f"Memory组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理Memory组件"""
+        if self._memory_instance:
+            try:
+                # 保存记忆
+                if hasattr(self._memory_instance, 'save_memory'):
+                    # 这里需要获取对话历史，暂时跳过
+                    pass
+                
+                # 关闭Memory实例
+                if hasattr(self._memory_instance, 'close'):
+                    await self._memory_instance.close()
+                elif hasattr(self._memory_instance, 'cleanup'):
+                    await self._memory_instance.cleanup()
+                
+                logger.info("Memory组件清理完成")
+                
+            except Exception as e:
+                logger.error(f"Memory组件清理失败: {e}")
+            finally:
+                self._memory_instance = None
+    
+    @property
+    def memory_instance(self):
+        """获取Memory实例"""
+        return self._memory_instance
+
+
+class MemoryFactory(ComponentFactory):
+    """Memory组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return MemoryAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.MEMORY
+
+

main/xiaozhi-server/core/components/adapters/tts_adapter.py

@@ -0,0 +1,90 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import tts
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class TTSAdapter(Component):
+    """TTS组件适配器：将现有TTS组件包装为新的组件接口"""
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.TTS, config)
+        self._tts_instance = None
+        self._delete_audio = config.get("delete_audio", True)
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化TTS组件"""
+        try:
+            # 获取TTS配置
+            selected_module = self.config.get("selected_module", {}).get("TTS")
+            if not selected_module:
+                raise ValueError("未配置TTS模块")
+            
+            # 获取TTS类型
+            tts_type = (
+                selected_module
+                if "type" not in self.config["TTS"][selected_module]
+                else self.config["TTS"][selected_module]["type"]
+            )
+            
+            # 创建TTS实例
+            self._tts_instance = tts.create_instance(
+                tts_type,
+                self.config["TTS"][selected_module],
+                str(self._delete_audio).lower() in ("true", "1", "yes"),
+            )
+            
+            # 注册资源以便清理
+            self.add_resource(self._tts_instance)
+            
+            # 打开音频通道
+            if hasattr(self._tts_instance, 'open_audio_channels'):
+                await self._tts_instance.open_audio_channels(context)
+            
+            # 设置兼容属性（用于向后兼容）
+            if hasattr(context, 'tts'):
+                context.tts = self._tts_instance
+            
+            logger.info(f"TTS组件初始化完成: {tts_type}")
+            
+        except Exception as e:
+            logger.error(f"TTS组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理TTS组件"""
+        if self._tts_instance:
+            try:
+                # 关闭TTS实例
+                if hasattr(self._tts_instance, 'close'):
+                    await self._tts_instance.close()
+                
+                # 清理音频文件
+                if hasattr(self._tts_instance, 'cleanup_audio_files'):
+                    self._tts_instance.cleanup_audio_files()
+                
+                logger.info("TTS组件清理完成")
+                
+            except Exception as e:
+                logger.error(f"TTS组件清理失败: {e}")
+            finally:
+                self._tts_instance = None
+    
+    @property
+    def tts_instance(self):
+        """获取TTS实例"""
+        return self._tts_instance
+
+
+class TTSFactory(ComponentFactory):
+    """TTS组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return TTSAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.TTS
+
+

main/xiaozhi-server/core/components/adapters/vad_adapter.py

@@ -0,0 +1,78 @@
+from typing import Any, Dict
+from core.components.component_manager import Component, ComponentType, ComponentFactory
+from core.utils import vad
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class VADAdapter(Component):
+    """VAD组件适配器：将现有VAD组件包装为新的组件接口"""
+    
+    def __init__(self, config: Dict[str, Any]):
+        super().__init__(ComponentType.VAD, config)
+        self._vad_instance = None
+    
+    async def _do_initialize(self, context: Any) -> None:
+        """初始化VAD组件"""
+        try:
+            # 获取VAD配置
+            selected_module = self.config.get("selected_module", {}).get("VAD")
+            if not selected_module:
+                raise ValueError("未配置VAD模块")
+            
+            # 获取VAD类型
+            vad_type = (
+                selected_module
+                if "type" not in self.config["VAD"][selected_module]
+                else self.config["VAD"][selected_module]["type"]
+            )
+            
+            # 创建VAD实例
+            self._vad_instance = vad.create_instance(
+                vad_type,
+                self.config["VAD"][selected_module],
+            )
+            
+            # 注册资源以便清理
+            self.add_resource(self._vad_instance)
+            
+            logger.info(f"VAD组件初始化完成: {vad_type}")
+            
+        except Exception as e:
+            logger.error(f"VAD组件初始化失败: {e}")
+            raise
+    
+    async def _do_cleanup(self) -> None:
+        """清理VAD组件"""
+        if self._vad_instance:
+            try:
+                # 关闭VAD实例
+                if hasattr(self._vad_instance, 'close'):
+                    await self._vad_instance.close()
+                elif hasattr(self._vad_instance, 'cleanup'):
+                    await self._vad_instance.cleanup()
+                
+                logger.info("VAD组件清理完成")
+                
+            except Exception as e:
+                logger.error(f"VAD组件清理失败: {e}")
+            finally:
+                self._vad_instance = None
+    
+    @property
+    def vad_instance(self):
+        """获取VAD实例"""
+        return self._vad_instance
+
+
+class VADFactory(ComponentFactory):
+    """VAD组件工厂"""
+    
+    def create(self, config: Dict[str, Any]) -> Component:
+        return VADAdapter(config)
+    
+    def get_component_type(self) -> ComponentType:
+        return ComponentType.VAD
+
+

main/xiaozhi-server/core/components/component_manager.py

@@ -0,0 +1,235 @@
+import asyncio
+import weakref
+from abc import ABC, abstractmethod
+from typing import Any, Callable, Dict, Optional, Type, TypeVar, Generic
+from enum import Enum
+from config.logger import setup_logging
+
+T = TypeVar('T')
+
+logger = setup_logging()
+
+
+class ComponentType(Enum):
+    """组件类型枚举"""
+    TTS = "tts"
+    ASR = "asr" 
+    VAD = "vad"
+    LLM = "llm"
+    MEMORY = "memory"
+    INTENT = "intent"
+
+
+class ComponentState(Enum):
+    """组件状态枚举"""
+    UNINITIALIZED = "uninitialized"
+    INITIALIZING = "initializing"
+    READY = "ready"
+    ERROR = "error"
+    CLEANING = "cleaning"
+    CLEANED = "cleaned"
+
+
+class Component(ABC):
+    """组件基类：定义统一的组件接口和生命周期管理"""
+    
+    def __init__(self, component_type: ComponentType, config: Dict[str, Any]):
+        self.component_type = component_type
+        self.config = config
+        self.state = ComponentState.UNINITIALIZED
+        self._initialization_lock = asyncio.Lock()
+        self._cleanup_lock = asyncio.Lock()
+        self._dependencies: Dict[str, 'Component'] = {}
+        self._dependents: weakref.WeakSet['Component'] = weakref.WeakSet()
+        self._resources: list = []  # 存储需要清理的资源
+        
+    @abstractmethod
+    async def _do_initialize(self, context: Any) -> None:
+        """子类实现具体的初始化逻辑"""
+        pass
+    
+    @abstractmethod
+    async def _do_cleanup(self) -> None:
+        """子类实现具体的清理逻辑"""
+        pass
+    
+    async def initialize(self, context: Any) -> None:
+        """初始化组件（带锁保护）"""
+        async with self._initialization_lock:
+            if self.state != ComponentState.UNINITIALIZED:
+                return
+                
+            try:
+                self.state = ComponentState.INITIALIZING
+                logger.info(f"正在初始化组件: {self.component_type.value}")
+                
+                # 初始化依赖组件
+                await self._initialize_dependencies(context)
+                
+                # 执行具体初始化
+                await self._do_initialize(context)
+                
+                self.state = ComponentState.READY
+                logger.info(f"组件初始化完成: {self.component_type.value}")
+                
+            except Exception as e:
+                self.state = ComponentState.ERROR
+                logger.error(f"组件初始化失败: {self.component_type.value}, 错误: {e}")
+                raise
+    
+    async def cleanup(self) -> None:
+        """清理组件（带锁保护）"""
+        async with self._cleanup_lock:
+            if self.state in [ComponentState.CLEANING, ComponentState.CLEANED]:
+                return
+                
+            try:
+                self.state = ComponentState.CLEANING
+                logger.info(f"正在清理组件: {self.component_type.value}")
+                
+                # 清理依赖此组件的其他组件
+                await self._cleanup_dependents()
+                
+                # 执行具体清理
+                await self._do_cleanup()
+                
+                # 清理资源
+                await self._cleanup_resources()
+                
+                self.state = ComponentState.CLEANED
+                logger.info(f"组件清理完成: {self.component_type.value}")
+                
+            except Exception as e:
+                logger.error(f"组件清理失败: {self.component_type.value}, 错误: {e}")
+                # 即使清理失败，也标记为已清理，避免重复清理
+                self.state = ComponentState.CLEANED
+    
+    def add_dependency(self, name: str, component: 'Component') -> None:
+        """添加依赖组件"""
+        self._dependencies[name] = component
+        component._dependents.add(self)
+    
+    def add_resource(self, resource: Any) -> None:
+        """添加需要清理的资源"""
+        self._resources.append(resource)
+    
+    async def _initialize_dependencies(self, context: Any) -> None:
+        """初始化依赖组件"""
+        for name, dep in self._dependencies.items():
+            if dep.state == ComponentState.UNINITIALIZED:
+                await dep.initialize(context)
+    
+    async def _cleanup_dependents(self) -> None:
+        """清理依赖此组件的其他组件"""
+        for dependent in list(self._dependents):
+            await dependent.cleanup()
+    
+    async def _cleanup_resources(self) -> None:
+        """清理所有注册的资源"""
+        for resource in self._resources:
+            try:
+                if hasattr(resource, 'close'):
+                    if asyncio.iscoroutinefunction(resource.close):
+                        await resource.close()
+                    else:
+                        resource.close()
+                elif hasattr(resource, 'cleanup'):
+                    if asyncio.iscoroutinefunction(resource.cleanup):
+                        await resource.cleanup()
+                    else:
+                        resource.cleanup()
+            except Exception as e:
+                logger.warning(f"清理资源时出错: {e}")
+        self._resources.clear()
+
+
+class ComponentFactory(ABC):
+    """组件工厂基类"""
+    
+    @abstractmethod
+    def create(self, config: Dict[str, Any]) -> Component:
+        """创建组件实例"""
+        pass
+    
+    @abstractmethod
+    def get_component_type(self) -> ComponentType:
+        """获取组件类型"""
+        pass
+
+
+class ComponentManager:
+    """
+    组件管理器：统一管理连接期内的组件实例生命周期。
+    支持分类管理、依赖注入、按需懒加载与统一清理。
+    """
+
+    def __init__(self, config: Dict[str, Any]):
+        self._config = config
+        self._components: Dict[str, Component] = {}
+        self._factories: Dict[ComponentType, ComponentFactory] = {}
+        self._initialization_order: list[ComponentType] = []
+        
+    def register_factory(self, factory: ComponentFactory) -> None:
+        """注册组件工厂"""
+        component_type = factory.get_component_type()
+        self._factories[component_type] = factory
+        logger.debug(f"注册组件工厂: {component_type.value}")
+    
+    def set_initialization_order(self, order: list[ComponentType]) -> None:
+        """设置组件初始化顺序"""
+        self._initialization_order = order
+    
+    async def get_component(self, component_type: ComponentType, context: Any) -> Optional[Component]:
+        """获取组件实例（按需创建）"""
+        key = component_type.value
+        
+        if key not in self._components:
+            factory = self._factories.get(component_type)
+            if factory is None:
+                logger.warning(f"未找到组件工厂: {component_type.value}")
+                return None
+                
+            try:
+                instance = factory.create(self._config)
+                await instance.initialize(context)
+                self._components[key] = instance
+                logger.info(f"组件创建并初始化完成: {component_type.value}")
+            except Exception as e:
+                logger.error(f"组件创建失败: {component_type.value}, 错误: {e}")
+                return None
+                
+        return self._components.get(key)
+    
+    def get(self, component_name: str) -> Optional[Component]:
+        """获取已初始化的组件实例（兼容接口）"""
+        return self._components.get(component_name)
+    
+    async def initialize_all(self, context: Any) -> None:
+        """按顺序初始化所有组件"""
+        for component_type in self._initialization_order:
+            await self.get_component(component_type, context)
+    
+    async def cleanup_all(self) -> None:
+        """清理所有组件（逆序清理）"""
+        # 按逆序清理，确保依赖关系正确
+        for component_type in reversed(self._initialization_order):
+            key = component_type.value
+            if key in self._components:
+                component = self._components[key]
+                await component.cleanup()
+                del self._components[key]
+        
+        # 清理可能遗漏的组件
+        remaining_components = list(self._components.values())
+        for component in remaining_components:
+            await component.cleanup()
+        
+        self._components.clear()
+        logger.info("所有组件已清理完成")
+    
+    def get_component_status(self) -> Dict[str, str]:
+        """获取所有组件状态"""
+        return {
+            name: component.state.value 
+            for name, component in self._components.items()
+        }

main/xiaozhi-server/core/components/component_registry.py

@@ -0,0 +1,75 @@
+from typing import Dict, Any
+from core.components.component_manager import ComponentManager, ComponentType
+from core.components.adapters.tts_adapter import TTSFactory
+from core.components.adapters.asr_adapter import ASRFactory
+from core.components.adapters.vad_adapter import VADFactory
+from core.components.adapters.llm_adapter import LLMFactory
+from core.components.adapters.memory_adapter import MemoryFactory
+from core.components.adapters.intent_adapter import IntentFactory
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class ComponentRegistry:
+    """组件注册器：统一管理所有组件工厂的注册"""
+    
+    _factories_registered = False
+    
+    @classmethod
+    def create_component_manager(cls, config: Dict[str, Any]) -> ComponentManager:
+        """创建并配置组件管理器"""
+        manager = ComponentManager(config)
+        
+        # 只在第一次时记录注册日志
+        if not cls._factories_registered:
+            logger.info("注册组件工厂")
+            cls._factories_registered = True
+        
+        # 注册所有组件工厂（每个manager都需要注册，但不重复记录日志）
+        manager.register_factory(TTSFactory())
+        manager.register_factory(ASRFactory())
+        manager.register_factory(VADFactory())
+        manager.register_factory(LLMFactory())
+        manager.register_factory(MemoryFactory())
+        manager.register_factory(IntentFactory())
+        
+        # 设置组件初始化顺序（考虑依赖关系）
+        # VAD -> ASR -> LLM -> Memory/Intent -> TTS
+        initialization_order = [
+            ComponentType.VAD,
+            ComponentType.ASR,
+            ComponentType.LLM,
+            ComponentType.MEMORY,
+            ComponentType.INTENT,
+            ComponentType.TTS,
+        ]
+        manager.set_initialization_order(initialization_order)
+        
+        if not cls._factories_registered:
+            logger.info("组件管理器创建完成，已注册所有组件工厂")
+        
+        return manager
+    
+    @staticmethod
+    def get_required_components(config: Dict[str, Any]) -> list[ComponentType]:
+        """根据配置获取需要的组件类型"""
+        required = []
+        selected_modules = config.get("selected_module", {})
+        
+        if selected_modules.get("VAD"):
+            required.append(ComponentType.VAD)
+        if selected_modules.get("ASR"):
+            required.append(ComponentType.ASR)
+        if selected_modules.get("LLM"):
+            required.append(ComponentType.LLM)
+        if selected_modules.get("TTS"):
+            required.append(ComponentType.TTS)
+        if selected_modules.get("Memory"):
+            required.append(ComponentType.MEMORY)
+        if selected_modules.get("Intent"):
+            required.append(ComponentType.INTENT)
+        
+        return required
+
+

main/xiaozhi-server/core/connection.py

@@ -24,23 +24,25 @@ from core.utils.modules_initialize import (
     initialize_tts,
     initialize_asr,
 )
-from core.handle.reportHandle import report
+# from core.handle.reportHandle import report  # 旧架构，已被新架构替代
 from core.providers.tts.default import DefaultTTS
 from concurrent.futures import ThreadPoolExecutor
 from core.utils.dialogue import Message, Dialogue
 from core.providers.asr.dto.dto import InterfaceType
-from core.handle.textHandle import handleTextMessage
+# from core.handle.textHandle import handleTextMessage  # 旧架构，已被新架构替代
 from core.providers.tools.unified_tool_handler import UnifiedToolHandler
 from plugins_func.loadplugins import auto_import_modules
-from plugins_func.register import Action
+from plugins_func.register import Action, ActionResponse
-from core.auth import AuthenticationError
+from core.auth import AuthMiddleware, AuthenticationError
-from config.config_loader import get_private_config_from_api
+from config.config_loader import get_private_config_from_api, ConfigDict
 from core.providers.tts.dto.dto import ContentType, TTSMessageDTO, SentenceType
 from config.logger import setup_logging, build_module_string, create_connection_logger
 from config.manage_api_client import DeviceNotFoundException, DeviceBindException
 from core.utils.prompt_manager import PromptManager
 from core.utils.voiceprint_provider import VoiceprintProvider
 from core.utils import textUtils
+from core.context.session_context import SessionContext
+from core.components.component_registry import ComponentRegistry
 
 TAG = __name__
 
@@ -144,7 +146,7 @@ class ConnectionHandler:
         self.iot_descriptors = {}
         self.func_handler = None
 
-        self.cmd_exit = self.config["exit_commands"]
+        self.cmd_exit = self.config.get("exit_commands", [])
 
         # 是否在聊天结束后关闭连接
         self.close_after_chat = False
@@ -164,6 +166,11 @@ class ConnectionHandler:
 
         # 初始化提示词管理器
         self.prompt_manager = PromptManager(self.config, self.logger)
+        
+        # 新增：会话上下文与组件管理器（会话级清理）
+        self.session_context: SessionContext = SessionContext()
+        self.session_context.config = self.config
+        self.component_manager = ComponentRegistry.create_component_manager(self.config)
 
     async def handle_connection(self, ws):
         try:
@@ -187,6 +194,11 @@ class ConnectionHandler:
 
             # 认证通过,继续处理
             self.websocket = ws
+            
+            # 更新会话上下文关键信息
+            self.session_context.headers = self.headers
+            self.session_context.device_id = self.device_id
+            self.session_context.client_ip = self.client_ip
 
             # 检查是否来自MQTT连接
             request_path = ws.request.path
@@ -201,7 +213,8 @@ class ConnectionHandler:
             # 启动超时检查任务
             self.timeout_task = asyncio.create_task(self._check_timeout())
 
-            self.welcome_msg = self.config["xiaozhi"]
+            # 新的配置访问方式 - 使用点号访问
+            self.welcome_msg = self.config.xiaozhi
             self.welcome_msg["session_id"] = self.session_id
 
             # 在后台初始化配置和组件（完全不阻塞主循环）
@@ -232,6 +245,18 @@ class ConnectionHandler:
                     self.logger.bind(tag=TAG).error(
                         f"强制关闭连接时出错: {close_error}"
                     )
+            finally:
+                # 会话级组件与回调清理（容错）
+                try:
+                    if hasattr(self, "component_manager") and self.component_manager:
+                        await self.component_manager.cleanup_all()
+                except Exception as e:
+                    self.logger.bind(tag=TAG).error(f"组件清理失败: {e}")
+                try:
+                    if hasattr(self, "session_context") and self.session_context:
+                        await self.session_context.run_cleanup()
+                except Exception as e:
+                    self.logger.bind(tag=TAG).error(f"会话清理回调执行失败: {e}")
 
     async def _save_and_close(self, ws):
         """保存记忆并关闭连接"""
@@ -244,7 +269,9 @@ class ConnectionHandler:
                         loop = asyncio.new_event_loop()
                         asyncio.set_event_loop(loop)
                         loop.run_until_complete(
-                            self.memory.save_memory(self.dialogue.dialogue)
+                            self.memory.save_memory(
+                                self.dialogue.dialogue, self.session_id
+                            )
                         )
                     except Exception as e:
                         self.logger.bind(tag=TAG).error(f"保存记忆失败: {e}")
@@ -434,6 +461,7 @@ class ConnectionHandler:
                 self.tts.open_audio_channels(self), self.loop
             )
             if self.need_bind:
+                self.bind_completed_event.set()
                 return
             self.selected_module_str = build_module_string(
                 self.config.get("selected_module", {})
@@ -574,16 +602,13 @@ class ConnectionHandler:
             self.bind_completed_event.set()
         except DeviceNotFoundException as e:
             self.need_bind = True
-            self.bind_completed_event.set()  # 状态已确定，设置事件
             private_config = {}
         except DeviceBindException as e:
             self.need_bind = True
             self.bind_code = e.bind_code
-            self.bind_completed_event.set()  # 状态已确定，设置事件
             private_config = {}
         except Exception as e:
             self.need_bind = True
-            self.bind_completed_event.set()  # 状态已确定，设置事件
             self.logger.bind(tag=TAG).error(f"异步获取差异化配置失败: {e}")
             private_config = {}
 
@@ -599,7 +624,8 @@ class ConnectionHandler:
 
         if init_vad:
             self.config["VAD"] = private_config["VAD"]
-            self.config["selected_module"]["VAD"] = private_config["selected_module"][
+            # 新的配置访问方式 - 使用嵌套路径设置
+            self.config["selected_module.VAD"] = private_config["selected_module"][
                 "VAD"
             ]
         if init_asr:
@@ -703,17 +729,16 @@ class ConnectionHandler:
 
         # 获取记忆总结配置
         memory_config = self.config["Memory"]
-        memory_type = self.config["Memory"][self.config["selected_module"]["Memory"]][
+        # 新的配置访问方式 - 使用嵌套get方法
-            "type"
+        memory_module = self.config.get("selected_module.Memory")
-        ]
+        memory_type = self.config.get(f"Memory.{memory_module}.type")
         # 如果使用 nomen，直接返回
         if memory_type == "nomem":
             return
         # 使用 mem_local_short 模式
         elif memory_type == "mem_local_short":
-            memory_llm_name = memory_config[self.config["selected_module"]["Memory"]][
+            # 新的配置访问方式 - 使用嵌套get和f-string
-                "llm"
+            memory_llm_name = self.config.get(f"Memory.{memory_module}.llm")
-            ]
             if memory_llm_name and memory_llm_name in self.config["LLM"]:
                 # 如果配置了专用LLM，则创建独立的LLM实例
                 from core.utils import llm as llm_utils
@@ -1078,6 +1103,7 @@ class ConnectionHandler:
         """处理上报任务"""
         try:
             # 执行异步上报（在事件循环中运行）
+            from core.handle.reportHandle import report
             asyncio.run(report(self, type, text, audio_data, report_time))
         except Exception as e:
             self.logger.bind(tag=TAG).error(f"上报处理异常: {e}")

main/xiaozhi-server/core/context/session_context.py

@@ -0,0 +1,378 @@
+import copy
+import uuid
+import time
+import queue
+import asyncio
+import threading
+from dataclasses import dataclass, field
+from typing import Any, Dict, Optional, List, Callable, Awaitable, Union
+from collections import deque
+from concurrent.futures import ThreadPoolExecutor
+from core.utils.dialogue import Dialogue
+from core.auth import AuthMiddleware
+from core.utils.prompt_manager import PromptManager
+from core.utils.voiceprint_provider import VoiceprintProvider
+from config.logger import setup_logging
+
+
+@dataclass
+class SessionContext:
+    """
+    会话上下文：完全替换ConnectionHandler的所有功能
+    承载单连接生命周期内的状态、组件、资源管理
+    与传输层解耦，支持WebSocket/MQTT/UDP等多协议
+    """
+
+    # === 基础标识 ===
+    session_id: str = field(default_factory=lambda: str(uuid.uuid4()))
+    device_id: Optional[str] = None
+    client_ip: Optional[str] = None
+    headers: Dict[str, str] = field(default_factory=dict)
+
+    # === 配置管理 ===
+    config: Dict[str, Any] = field(default_factory=dict)
+    common_config: Dict[str, Any] = field(default_factory=dict)
+    private_config: Dict[str, Any] = field(default_factory=dict)
+    selected_module_str: str = ""
+    
+    # === 认证与绑定 ===
+    is_authenticated: bool = False
+    need_bind: bool = False
+    bind_code: Optional[str] = None
+    read_config_from_api: bool = False
+    max_output_size: int = 0
+    chat_history_conf: int = 0
+    
+    # === 会话状态 ===
+    is_speaking: bool = False
+    listen_mode: str = "auto"
+    abort_requested: bool = False
+    close_after_chat: bool = False
+    just_woken_up: bool = False
+    load_function_plugin: bool = False
+    intent_type: str = "nointent"
+    
+    # === 音频相关 ===
+    audio_format: str = "opus"
+    client_have_voice: bool = False
+    client_voice_stop: bool = False
+    client_audio_buffer: bytearray = field(default_factory=bytearray)
+    client_voice_window: deque = field(default_factory=lambda: deque(maxlen=5))
+    last_is_voice: bool = False
+    audio_flow_control: Dict[str, Any] = field(default_factory=dict)
+    
+    # === ASR相关 ===
+    asr_audio: List[bytes] = field(default_factory=list)
+    asr_audio_queue: queue.Queue = field(default_factory=queue.Queue)
+    asr_priority_thread: Optional[threading.Thread] = None
+    
+    # === LLM相关 ===
+    llm_finish_task: bool = True
+    dialogue: Optional[Dialogue] = None
+    current_speaker: Optional[str] = None
+    sentence_id: Optional[str] = None
+    
+    # === TTS相关 ===
+    tts_MessageText: str = ""
+    
+    # === IoT相关 ===
+    iot_descriptors: Dict[str, Any] = field(default_factory=dict)
+    func_handler: Optional[Any] = None
+    
+    # === 时间管理 ===
+    last_activity_time_ms: float = field(default_factory=lambda: time.time() * 1000)
+    created_at: float = field(default_factory=lambda: time.time())
+    timeout_seconds: int = 180  # 默认超时时间
+    timeout_task: Optional[asyncio.Task] = None
+
+    # === 组件实例 ===
+    # components属性通过@property方法提供，指向component_manager
+    
+    # === 其他状态 ===
+    welcome_msg: Optional[Dict[str, Any]] = None
+    prompt: Optional[str] = None
+    features: Optional[Dict[str, Any]] = None
+    mcp_client: Optional[Any] = None
+    cmd_exit: List[str] = field(default_factory=list)
+    
+    # === 线程与并发 ===
+    loop: Optional[asyncio.AbstractEventLoop] = None
+    stop_event: Optional[threading.Event] = None
+    executor: Optional[ThreadPoolExecutor] = None
+    
+    # === 队列管理 ===
+    report_queue: queue.Queue = field(default_factory=queue.Queue)
+    report_thread: Optional[threading.Thread] = None
+    report_asr_enable: bool = False
+    report_tts_enable: bool = False
+    
+    # === 组件管理器 ===
+    component_manager: Optional[Any] = None
+    
+    # === 兼容属性（用于向后兼容TTS处理） ===
+    tts: Optional[Any] = None
+    websocket: Optional[Any] = None  # 兼容旧TTS组件
+    transport: Optional[Any] = None  # 新的transport接口
+    
+    # === 工具类 ===
+    auth: Optional[AuthMiddleware] = None
+    prompt_manager: Optional[PromptManager] = None
+    voiceprint_provider: Optional[VoiceprintProvider] = None
+    server: Optional[Any] = None  # WebSocket服务器引用
+    
+    # === 会话级清理回调 ===
+    _cleanup_callbacks: List[Callable[[], Union[None, Awaitable[None]]]] = field(default_factory=list)
+
+    def __post_init__(self):
+        """初始化后处理"""
+        # 深拷贝配置避免污染
+        if self.config:
+            self.common_config = self.config
+            self.config = copy.deepcopy(self.config)
+            
+            # 从配置中读取相关设置
+            self.read_config_from_api = self.config.get("read_config_from_api", False)
+            self.max_output_size = self.config.get("max_output_size", 0)
+            self.chat_history_conf = self.config.get("chat_history_conf", 0)
+            self.cmd_exit = self.config.get("exit_commands", [])
+            self.timeout_seconds = int(self.config.get("close_connection_no_voice_time", 120)) + 60
+            
+            # 初始化认证中间件
+            self.auth = AuthMiddleware(self.config)
+            
+            # 初始化提示词管理器
+            self.prompt_manager = PromptManager(self.config, setup_logging())
+        
+        # 初始化对话管理
+        if not self.dialogue:
+            self.dialogue = Dialogue()
+        
+        # 初始化线程相关
+        if not self.loop:
+            try:
+                self.loop = asyncio.get_event_loop()
+            except RuntimeError:
+                self.loop = asyncio.new_event_loop()
+                
+        if not self.stop_event:
+            self.stop_event = threading.Event()
+            
+        if not self.executor:
+            self.executor = ThreadPoolExecutor(max_workers=5)
+        
+        # 初始化上报设置
+        self.report_asr_enable = self.read_config_from_api
+        self.report_tts_enable = self.read_config_from_api
+
+    def update_activity(self) -> None:
+        """刷新最后活跃时间"""
+        self.last_activity_time_ms = time.time() * 1000
+    
+    def clearSpeakStatus(self) -> None:
+        """清除服务端讲话状态（兼容方法）"""
+        self.is_speaking = False
+        logger = setup_logging()
+        logger.debug("清除服务端讲话状态")
+    
+    def reset_vad_states(self) -> None:
+        """重置VAD状态（兼容方法）"""
+        self.client_audio_buffer = bytearray()
+        self.client_have_voice = False
+        self.client_voice_stop = False
+        logger = setup_logging()
+        logger.debug("VAD states reset.")
+
+    def is_timeout(self, timeout_seconds: int) -> bool:
+        """检查是否超时"""
+        now_ms = time.time() * 1000
+        return (now_ms - self.last_activity_time_ms) > (timeout_seconds * 1000)
+
+    def register_cleanup(self, callback: Callable[[], Union[None, Awaitable[None]]]) -> None:
+        """注册会话结束时需要执行的清理回调"""
+        self._cleanup_callbacks.append(callback)
+
+    async def run_cleanup(self) -> None:
+        """执行所有注册的清理回调"""
+        logger = setup_logging()
+        logger.info(f"Session {self.session_id} 开始执行会话级清理 ({len(self._cleanup_callbacks)} 个回调)")
+        
+        # 停止所有线程
+        if self.stop_event:
+            self.stop_event.set()
+        
+        # 关闭线程池
+        if self.executor:
+            self.executor.shutdown(wait=False)
+        
+        # 取消超时任务
+        if self.timeout_task and not self.timeout_task.done():
+            self.timeout_task.cancel()
+        
+        # 执行清理回调
+        for callback in reversed(self._cleanup_callbacks):
+            try:
+                result = callback()
+                if asyncio.iscoroutine(result):
+                    await result
+            except Exception as e:
+                logger.error(f"Session {self.session_id} 清理回调执行失败: {e}", exc_info=True)
+        
+        self._cleanup_callbacks.clear()
+        logger.info(f"Session {self.session_id} 会话级清理完成")
+
+    # === 兼容旧代码的属性访问 ===
+    @property
+    def client_is_speaking(self) -> bool:
+        """兼容旧代码的属性名"""
+        return self.is_speaking
+    
+    @client_is_speaking.setter
+    def client_is_speaking(self, value: bool):
+        self.is_speaking = value
+    
+    @property
+    def client_listen_mode(self) -> str:
+        """兼容旧代码的属性名"""
+        return self.listen_mode
+    
+    @client_listen_mode.setter
+    def client_listen_mode(self, value: str):
+        self.listen_mode = value
+    
+    @property
+    def client_abort(self) -> bool:
+        """兼容旧代码的属性名"""
+        return self.abort_requested
+    
+    @client_abort.setter
+    def client_abort(self, value: bool):
+        self.abort_requested = value
+    
+    @property
+    def components(self):
+        """组件访问器（兼容属性）"""
+        return self.component_manager
+    
+    @components.setter
+    def components(self, value):
+        """组件设置器（兼容属性）- 实际设置到component_manager"""
+        # 如果尝试设置components，我们忽略它或者给出警告
+        # 因为components应该通过component_manager管理
+        logger = setup_logging()
+        logger.warning("尝试直接设置components属性，请使用component_manager")
+
+    @property
+    def last_activity_time(self) -> float:
+        """兼容旧代码：返回毫秒级时间戳"""
+        return self.last_activity_time_ms
+
+    @last_activity_time.setter
+    def last_activity_time(self, value: float):
+        """兼容旧代码：接受毫秒级时间戳"""
+        self.last_activity_time_ms = value
+
+    # === 日志相关 ===
+    @property
+    def logger(self):
+        """获取日志记录器"""
+        return setup_logging()
+
+    # === 工具方法 ===
+    def get_component(self, component_name: str) -> Optional[Any]:
+        """获取组件实例"""
+        return self.components.get(component_name)
+    
+    def set_component(self, component_name: str, component_instance: Any) -> None:
+        """设置组件实例"""
+        if self.component_manager:
+            self.component_manager._components[component_name] = component_instance
+    
+    def has_component(self, component_name: str) -> bool:
+        """检查是否有指定组件"""
+        return component_name in self.components
+    
+    def clear_audio_buffer(self) -> None:
+        """清空音频缓冲区"""
+        self.client_audio_buffer.clear()
+        self.asr_audio.clear()
+        
+        # 清空队列
+        try:
+            while not self.asr_audio_queue.empty():
+                self.asr_audio_queue.get_nowait()
+        except queue.Empty:
+            pass
+    
+    def reset_voice_state(self) -> None:
+        """重置语音状态"""
+        self.client_have_voice = False
+        self.client_voice_stop = False
+        self.last_is_voice = False
+        self.client_voice_window.clear()
+    
+    def initialize_private_config(self) -> None:
+        """初始化差异化配置（从ConnectionHandler迁移）"""
+        from config.config_loader import get_private_config_from_api
+        from config.manage_api_client import DeviceNotFoundException, DeviceBindException
+        
+        if not self.read_config_from_api:
+            return
+            
+        try:
+            # 获取设备私有配置
+            private_config = get_private_config_from_api(
+                self.config, self.device_id, self.headers.get("client-id")
+            )
+            
+            if private_config:
+                self.private_config = private_config
+                # 合并私有配置到主配置
+                self.config.update(private_config)
+                
+        except DeviceNotFoundException:
+            self.logger.error(f"设备 {self.device_id} 未找到")
+            self.need_bind = True
+        except DeviceBindException as e:
+            self.logger.error(f"设备绑定异常: {e}")
+            self.need_bind = True
+            self.bind_code = str(e)
+        except Exception as e:
+            self.logger.error(f"获取私有配置失败: {e}")
+    
+    async def initialize_components(self) -> None:
+        """异步初始化组件（从ConnectionHandler迁移）"""
+        if not self.component_manager:
+            return
+            
+        try:
+            # 初始化各个组件
+            from core.components.component_registry import ComponentType
+            
+            # 按依赖顺序初始化组件
+            component_types = [
+                ComponentType.VAD,
+                ComponentType.ASR, 
+                ComponentType.LLM,
+                ComponentType.MEMORY,
+                ComponentType.INTENT,
+                ComponentType.TTS
+            ]
+            
+            for component_type in component_types:
+                try:
+                    component = await self.component_manager.get_component(component_type, self)
+                    if component:
+                        self.logger.info(f"组件 {component_type} 初始化成功")
+                except Exception as e:
+                    self.logger.error(f"组件 {component_type} 初始化失败: {e}")
+                    
+        except Exception as e:
+            self.logger.error(f"组件初始化失败: {e}")
+    
+    def __str__(self) -> str:
+        return f"SessionContext(session_id={self.session_id}, device_id={self.device_id})"
+    
+    def __repr__(self) -> str:
+        return self.__str__()
+
+

main/xiaozhi-server/core/handle/sendAudioHandle.py

@@ -7,6 +7,10 @@ from core.providers.tts.dto.dto import SentenceType
 from core.utils.audioRateController import AudioRateController
 
 TAG = __name__
+# 音频帧时长（毫秒）
+AUDIO_FRAME_DURATION = 60
+# 预缓冲包数量，直接发送以减少延迟
+PRE_BUFFER_COUNT = 5
 
 
 async def sendAudioMessage(conn, sentenceType, audios, text):
@@ -45,7 +49,7 @@ async def sendAudioMessage(conn, sentenceType, audios, text):
 
 async def _wait_for_audio_completion(conn):
     """
-    等待音频队列清空
+    等待音频队列清空并等待预缓冲包播放完成
 
     Args:
         conn: 连接对象
@@ -56,6 +60,13 @@ async def _wait_for_audio_completion(conn):
             f"等待音频发送完成，队列中还有 {len(rate_controller.queue)} 个包"
         )
         await rate_controller.queue_empty_event.wait()
+
+        # 等待预缓冲包播放完成
+        # 前N个包直接发送，增加2个网络抖动包，需要额外等待它们在客户端播放完成
+        frame_duration_ms = rate_controller.frame_duration
+        pre_buffer_playback_time = (PRE_BUFFER_COUNT + 2) * frame_duration_ms / 1000.0
+        await asyncio.sleep(pre_buffer_playback_time)
+
         conn.logger.bind(tag=TAG).debug("音频发送完成")
 
 
@@ -81,14 +92,14 @@ async def _send_to_mqtt_gateway(conn, opus_packet, timestamp, sequence):
     await conn.websocket.send(complete_packet)
 
 
-async def sendAudio(conn, audios, frame_duration=60):
+async def sendAudio(conn, audios, frame_duration=AUDIO_FRAME_DURATION):
     """
     发送音频包，使用 AudioRateController 进行精确的流量控制
 
     Args:
         conn: 连接对象
         audios: 单个opus包(bytes) 或 opus包列表
-        frame_duration: 帧时长（毫秒），默认60ms
+        frame_duration: 帧时长（毫秒），默认使用全局常量AUDIO_FRAME_DURATION
     """
     if audios is None or len(audios) == 0:
         return
@@ -187,16 +198,14 @@ async def _send_audio_with_rate_control(
         flow_control: 流控状态
         send_delay: 固定延迟（秒），-1表示使用动态流控
     """
-    pre_buffer_count = 5
-
     for packet in audio_list:
         if conn.client_abort:
             return
 
         conn.last_activity_time = time.time() * 1000
 
-        # 预缓冲：前5个包直接发送
+        # 预缓冲：前N个包直接发送
-        if flow_control["packet_count"] < pre_buffer_count:
+        if flow_control["packet_count"] < PRE_BUFFER_COUNT:
             await _do_send_audio(conn, packet, flow_control)
             conn.client_is_speaking = True
         elif send_delay > 0:

main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py

@@ -69,6 +69,7 @@ class ListenTextMessageHandler(TextMessageHandler):
                     enqueue_asr_report(conn, "嘿，你好呀", [])
                     await startToChat(conn, "嘿，你好呀")
                 else:
+                    conn.just_woken_up = True
                     # 上报纯文字数据（复用ASR上报功能，但不提供音频数据）
                     enqueue_asr_report(conn, original_text, [])
                     # 否则需要LLM对文字内容进行答复

main/xiaozhi-server/core/handle/textHandler/pingMessageHandler.py

@@ -0,0 +1,45 @@
+import json
+import time
+from typing import Dict, Any
+
+from core.handle.textMessageHandler import TextMessageHandler
+from core.handle.textMessageType import TextMessageType
+
+TAG = __name__
+
+
+class PingMessageHandler(TextMessageHandler):
+    """Ping消息处理器，用于保持WebSocket连接"""
+
+    @property
+    def message_type(self) -> TextMessageType:
+        return TextMessageType.PING
+
+    async def handle(self, conn, msg_json: Dict[str, Any]) -> None:
+        """
+        处理PING消息，发送PONG响应
+        消息格式：{"type": "ping"}
+        Args:
+            conn: WebSocket连接对象
+            msg_json: PING消息的JSON数据
+        """
+        # 检查是否启用了WebSocket心跳功能
+        enable_websocket_ping = conn.config.get("enable_websocket_ping", False)
+        if not enable_websocket_ping:
+            conn.logger.debug(f"WebSocket心跳功能未启用，忽略PING消息")
+            return
+
+        try:
+            conn.logger.debug(f"收到PING消息，发送PONG响应")
+            conn.last_activity_time = time.time() * 1000
+            # 构造PONG响应消息
+            pong_message = {
+                "type": "pong",
+                "timestamp": time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()),
+            }
+
+            # 发送PONG响应
+            await conn.websocket.send(json.dumps(pong_message))
+
+        except Exception as e:
+            conn.logger.error(f"处理PING消息时发生错误: {e}")

main/xiaozhi-server/core/handle/textMessageHandlerRegistry.py

@@ -7,6 +7,7 @@ from core.handle.textHandler.listenMessageHandler import ListenTextMessageHandle
 from core.handle.textHandler.mcpMessageHandler import McpTextMessageHandler
 from core.handle.textMessageHandler import TextMessageHandler
 from core.handle.textHandler.serverMessageHandler import ServerTextMessageHandler
+from core.handle.textHandler.pingMessageHandler import PingMessageHandler
 
 TAG = __name__
 
@@ -27,6 +28,7 @@ class TextMessageHandlerRegistry:
             IotTextMessageHandler(),
             McpTextMessageHandler(),
             ServerTextMessageHandler(),
+            PingMessageHandler(),
         ]
 
         for handler in handlers:

main/xiaozhi-server/core/handle/textMessageType.py

@@ -9,3 +9,4 @@ class TextMessageType(Enum):
     IOT = "iot"
     MCP = "mcp"
     SERVER = "server"
+    PING = "ping"

main/xiaozhi-server/core/http_server.py

@@ -33,38 +33,60 @@ class SimpleHttpServer:
             return f"ws://{local_ip}:{port}/xiaozhi/v1/"
 
     async def start(self):
-        server_config = self.config["server"]
+        try:
-        read_config_from_api = self.config.get("read_config_from_api", False)
+            server_config = self.config["server"]
-        host = server_config.get("ip", "0.0.0.0")
+            read_config_from_api = self.config.get("read_config_from_api", False)
-        port = int(server_config.get("http_port", 8003))
+            host = server_config.get("ip", "0.0.0.0")
+            port = int(server_config.get("http_port", 8003))
 
-        if port:
+            if port:
-            app = web.Application()
+                app = web.Application()
 
-            if not read_config_from_api:
+                if not read_config_from_api:
-                # 如果没有开启智控台，只是单模块运行，就需要再添加简单OTA接口，用于下发websocket接口
+                    # 如果没有开启智控台，只是单模块运行，就需要再添加简单OTA接口，用于下发websocket接口
+                    app.add_routes(
+                        [
+                            web.get("/xiaozhi/ota/", self.ota_handler.handle_get),
+                            web.post("/xiaozhi/ota/", self.ota_handler.handle_post),
+                            web.options(
+                                "/xiaozhi/ota/", self.ota_handler.handle_options
+                            ),
+                            # 下载接口，仅提供 data/bin/*.bin 下载
+                            web.get(
+                                "/xiaozhi/ota/download/{filename}",
+                                self.ota_handler.handle_download,
+                            ),
+                            web.options(
+                                "/xiaozhi/ota/download/{filename}",
+                                self.ota_handler.handle_options,
+                            ),
+                        ]
+                    )
+                # 添加路由
                 app.add_routes(
                     [
-                        web.get("/xiaozhi/ota/", self.ota_handler.handle_get),
+                        web.get("/mcp/vision/explain", self.vision_handler.handle_get),
-                        web.post("/xiaozhi/ota/", self.ota_handler.handle_post),
+                        web.post(
-                        web.options("/xiaozhi/ota/", self.ota_handler.handle_post),
+                            "/mcp/vision/explain", self.vision_handler.handle_post
+                        ),
+                        web.options(
+                            "/mcp/vision/explain", self.vision_handler.handle_options
+                        ),
                     ]
                 )
-            # 添加路由
-            app.add_routes(
-                [
-                    web.get("/mcp/vision/explain", self.vision_handler.handle_get),
-                    web.post("/mcp/vision/explain", self.vision_handler.handle_post),
-                    web.options("/mcp/vision/explain", self.vision_handler.handle_post),
-                ]
-            )
 
-            # 运行服务
+                # 运行服务
-            runner = web.AppRunner(app)
+                runner = web.AppRunner(app)
-            await runner.setup()
+                await runner.setup()
-            site = web.TCPSite(runner, host, port)
+                site = web.TCPSite(runner, host, port)
-            await site.start()
+                await site.start()
 
-            # 保持服务运行
+                # 保持服务运行
-            while True:
+                while True:
-                await asyncio.sleep(3600)  # 每隔 1 小时检查一次
+                    await asyncio.sleep(3600)  # 每隔 1 小时检查一次
+        except Exception as e:
+            self.logger.bind(tag=TAG).error(f"HTTP服务器启动失败: {e}")
+            import traceback
+
+            self.logger.bind(tag=TAG).error(f"错误堆栈: {traceback.format_exc()}")
+            raise

main/xiaozhi-server/core/pipeline/message_pipeline.py

@@ -0,0 +1,28 @@
+from abc import ABC, abstractmethod
+from typing import Any, List
+
+
+class MessageProcessor(ABC):
+    """消息处理器接口。返回 True 表示已处理并中止后续处理。"""
+
+    @abstractmethod
+    async def process(self, context: Any, transport: Any, message: Any) -> bool:
+        raise NotImplementedError
+
+
+class MessagePipeline:
+    """责任链式消息处理管道。"""
+
+    def __init__(self) -> None:
+        self._processors: List[MessageProcessor] = []
+
+    def add_processor(self, processor: MessageProcessor) -> None:
+        self._processors.append(processor)
+
+    async def process_message(self, context: Any, transport: Any, message: Any) -> None:
+        for processor in self._processors:
+            handled = await processor.process(context, transport, message)
+            if handled:
+                return
+
+

main/xiaozhi-server/core/processors/abort_processor.py

@@ -0,0 +1,102 @@
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class AbortProcessor(MessageProcessor):
+    """中断消息处理器：完整迁移abortMessageHandler.py和abortHandle.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理abort类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "abort":
+                    await self.handle_abort_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_abort_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理中断消息 - 完整迁移自abortHandle.py的handleAbortMessage"""
+        logger.info("Abort message received")
+        
+        # 设置成打断状态，会自动打断llm、tts任务 - 完整迁移原逻辑
+        context.abort_requested = True
+        
+        # 清理队列 - 完整迁移原逻辑
+        await self._clear_queues(context)
+        
+        # 打断客户端说话状态 - 完整迁移原逻辑
+        await transport.send(json.dumps({
+            "type": "tts", 
+            "state": "stop", 
+            "session_id": context.session_id
+        }))
+        
+        # 清理说话状态 - 完整迁移原逻辑
+        self._clear_speak_status(context)
+        
+        logger.info("Abort message received-end")
+    
+    async def _clear_queues(self, context: SessionContext):
+        """清理所有队列 - 完整迁移原clear_queues逻辑"""
+        try:
+            # 清理TTS音频队列
+            tts_component = context.components.get('tts')
+            if tts_component and hasattr(tts_component, 'tts_instance'):
+                tts_instance = tts_component.tts_instance
+                if hasattr(tts_instance, 'tts_audio_queue'):
+                    try:
+                        while not tts_instance.tts_audio_queue.empty():
+                            tts_instance.tts_audio_queue.get_nowait()
+                    except:
+                        pass
+            
+            # 清理ASR音频队列
+            context.clear_audio_buffer()
+            
+            # 清理其他可能的队列
+            if hasattr(context, 'clear_queues'):
+                context.clear_queues()
+                
+        except Exception as e:
+            logger.error(f"清理队列时出错: {e}")
+    
+    def _clear_speak_status(self, context: SessionContext):
+        """清理说话状态 - 完整迁移原clearSpeakStatus逻辑"""
+        try:
+            # 清理说话状态
+            context.is_speaking = False
+            
+            # 如果有其他说话状态相关的属性，也一并清理
+            if hasattr(context, 'clearSpeakStatus'):
+                context.clearSpeakStatus()
+            
+            # 重置相关状态
+            context.client_have_voice = False
+            context.client_voice_stop = True
+            
+        except Exception as e:
+            logger.error(f"清理说话状态时出错: {e}")
+    
+    async def _send_abort_confirmation(self, transport: TransportInterface, session_id: str):
+        """发送中断确认响应（可选）"""
+        response = {
+            "type": "abort",
+            "status": "success",
+            "message": "中断操作已完成",
+            "session_id": session_id
+        }
+        
+        try:
+            await transport.send(json.dumps(response))
+        except Exception as e:
+            logger.error(f"发送中断确认响应失败: {e}")
+

main/xiaozhi-server/core/processors/audio_receive_processor.py

@@ -0,0 +1,277 @@
+import time
+import json
+import asyncio
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.utils.util import audio_to_data
+from core.utils.output_counter import check_device_output_limit
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class AudioReceiveProcessor(MessageProcessor):
+    """音频接收处理器：完整迁移receiveAudioHandle.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理音频消息"""
+        if isinstance(message, bytes):
+            await self.handle_audio_message(context, transport, message)
+            return True
+        return False
+    
+    async def handle_audio_message(self, context: SessionContext, transport: TransportInterface, audio: bytes):
+        """处理音频消息 - 完整迁移自handleAudioMessage"""
+        # 获取VAD组件
+        vad_component = context.components.get('vad')
+        if not vad_component or not hasattr(vad_component, 'vad_instance'):
+            logger.warning("VAD组件未初始化")
+            return
+            
+        vad_instance = vad_component.vad_instance
+        
+        # 当前片段是否有人说话
+        have_voice = vad_instance.is_vad(context, audio)
+        
+        # 如果设备刚刚被唤醒，短暂忽略VAD检测
+        if have_voice and context.just_woken_up:
+            have_voice = False
+            # 设置一个短暂延迟后恢复VAD检测
+            context.asr_audio.clear()
+            if not hasattr(context, "vad_resume_task") or context.vad_resume_task.done():
+                context.vad_resume_task = asyncio.create_task(self._resume_vad_detection(context))
+            return
+            
+        if have_voice:
+            if context.is_speaking:
+                await self._handle_abort_message(context, transport)
+                
+        # 设备长时间空闲检测，用于say goodbye
+        await self._no_voice_close_connect(context, transport, have_voice)
+        
+        # 接收音频
+        asr_component = context.components.get('asr')
+        if asr_component and hasattr(asr_component, 'asr_instance'):
+            asr_instance = asr_component.asr_instance
+            if hasattr(asr_instance, 'receive_audio'):
+                await asr_instance.receive_audio(context, audio, have_voice)
+    
+    async def _resume_vad_detection(self, context: SessionContext):
+        """恢复VAD检测 - 完整迁移自resume_vad_detection"""
+        # 等待1秒后恢复VAD检测
+        await asyncio.sleep(1)
+        context.just_woken_up = False
+    
+    async def start_to_chat(self, context: SessionContext, transport: TransportInterface, text: str):
+        """开始聊天 - 完整迁移自startToChat"""
+        # 检查输入是否是JSON格式（包含说话人信息）
+        speaker_name = None
+        actual_text = text
+
+        try:
+            # 尝试解析JSON格式的输入
+            if text.strip().startswith('{') and text.strip().endswith('}'):
+                data = json.loads(text)
+                if 'speaker' in data and 'content' in data:
+                    speaker_name = data['speaker']
+                    actual_text = data['content']
+                    logger.info(f"解析到说话人信息: {speaker_name}")
+
+                    # 直接使用JSON格式的文本，不解析
+                    actual_text = text
+        except (json.JSONDecodeError, KeyError):
+            # 如果解析失败，继续使用原始文本
+            pass
+
+        # 保存说话人信息到上下文
+        if speaker_name:
+            context.current_speaker = speaker_name
+        else:
+            context.current_speaker = None
+
+        # 检查设备绑定
+        if context.need_bind:
+            await self._check_bind_device(context, transport)
+            return
+
+        # 如果当日的输出字数大于限定的字数
+        if context.max_output_size > 0:
+            if check_device_output_limit(
+                context.headers.get("device-id"), context.max_output_size
+            ):
+                await self._max_out_size(context, transport)
+                return
+                
+        if context.is_speaking:
+            await self._handle_abort_message(context, transport)
+
+        # 首先进行意图分析，使用实际文本内容
+        from core.processors.chat_processor import ChatProcessor
+        chat_processor = ChatProcessor()
+        intent_handled = await chat_processor.handle_user_intent(context, transport, actual_text)
+
+        if intent_handled:
+            # 如果意图已被处理，不再进行聊天
+            return
+
+        # 意图未被处理，继续常规聊天流程，使用实际文本内容
+        await self._send_stt_message(context, transport, actual_text)
+        
+        # 使用ChatProcessor处理聊天
+        from core.processors.chat_processor import ChatProcessor
+        chat_processor = ChatProcessor()
+        await chat_processor.handle_chat(context, transport, actual_text)
+
+    async def _no_voice_close_connect(self, context: SessionContext, transport: TransportInterface, have_voice: bool):
+        """无声音时关闭连接检测 - 完整迁移自no_voice_close_connect"""
+        if have_voice:
+            context.update_activity()
+            return
+            
+        # 只有在已经初始化过时间戳的情况下才进行超时检查
+        if context.last_activity_time_ms > 0.0:
+            no_voice_time = time.time() * 1000 - context.last_activity_time_ms
+            close_connection_no_voice_time = int(
+                context.config.get("close_connection_no_voice_time", 120)
+            )
+            
+            if (
+                not context.close_after_chat
+                and no_voice_time > 1000 * close_connection_no_voice_time
+            ):
+                context.close_after_chat = True
+                context.abort_requested = False
+                
+                end_prompt = context.config.get("end_prompt", {})
+                if end_prompt and end_prompt.get("enable", True) is False:
+                    logger.info("结束对话，无需发送结束提示语")
+                    await transport.close()
+                    return
+                    
+                prompt = end_prompt.get("prompt")
+                if not prompt:
+                    prompt = "请你以```时间过得真快```未来头，用富有感情、依依不舍的话来结束这场对话吧。！"
+                await self.start_to_chat(context, transport, prompt)
+
+    async def _max_out_size(self, context: SessionContext, transport: TransportInterface):
+        """超出最大输出字数处理 - 完整迁移自max_out_size"""
+        # 播放超出最大输出字数的提示
+        context.abort_requested = False
+        text = "不好意思，我现在有点事情要忙，明天这个时候我们再聊，约好了哦！明天不见不散，拜拜！"
+        await self._send_stt_message(context, transport, text)
+        
+        file_path = "config/assets/max_output_size.wav"
+        opus_packets = audio_to_data(file_path)
+        
+        # 获取TTS组件并添加到队列
+        tts_component = context.components.get('tts')
+        if tts_component and hasattr(tts_component, 'tts_instance'):
+            tts_instance = tts_component.tts_instance
+            if hasattr(tts_instance, 'tts_audio_queue'):
+                from core.providers.tts.dto.dto import SentenceType
+                tts_instance.tts_audio_queue.put((SentenceType.LAST, opus_packets, text))
+                
+        context.close_after_chat = True
+
+    async def _check_bind_device(self, context: SessionContext, transport: TransportInterface):
+        """检查设备绑定 - 完整迁移自check_bind_device"""
+        bind_code = context.bind_code
+        
+        if bind_code:
+            # 确保bind_code是6位数字
+            if len(bind_code) != 6:
+                logger.error(f"无效的绑定码格式: {bind_code}")
+                text = "绑定码格式错误，请检查配置。"
+                await self._send_stt_message(context, transport, text)
+                return
+
+            text = f"请登录控制面板，输入{bind_code}，绑定设备。"
+            await self._send_stt_message(context, transport, text)
+
+            # 获取TTS组件
+            tts_component = context.components.get('tts')
+            if not tts_component or not hasattr(tts_component, 'tts_instance'):
+                return
+                
+            tts_instance = tts_component.tts_instance
+            if not hasattr(tts_instance, 'tts_audio_queue'):
+                return
+
+            # 播放提示音
+            from core.providers.tts.dto.dto import SentenceType
+            music_path = "config/assets/bind_code.wav"
+            opus_packets = audio_to_data(music_path)
+            tts_instance.tts_audio_queue.put((SentenceType.FIRST, opus_packets, text))
+
+            # 逐个播放数字
+            for i in range(6):  # 确保只播放6位数字
+                try:
+                    digit = bind_code[i]
+                    num_path = f"config/assets/bind_code/{digit}.wav"
+                    num_packets = audio_to_data(num_path)
+                    tts_instance.tts_audio_queue.put((SentenceType.MIDDLE, num_packets, None))
+                except Exception as e:
+                    logger.error(f"播放数字音频失败: {e}")
+                    continue
+            tts_instance.tts_audio_queue.put((SentenceType.LAST, [], None))
+        else:
+            # 播放未绑定提示
+            context.abort_requested = False
+            text = f"没有找到该设备的版本信息，请正确配置 OTA地址，然后重新编译固件。"
+            await self._send_stt_message(context, transport, text)
+            
+            # 获取TTS组件
+            tts_component = context.components.get('tts')
+            if tts_component and hasattr(tts_component, 'tts_instance'):
+                tts_instance = tts_component.tts_instance
+                if hasattr(tts_instance, 'tts_audio_queue'):
+                    from core.providers.tts.dto.dto import SentenceType
+                    music_path = "config/assets/bind_not_found.wav"
+                    opus_packets = audio_to_data(music_path)
+                    tts_instance.tts_audio_queue.put((SentenceType.LAST, opus_packets, text))
+
+    async def _handle_abort_message(self, context: SessionContext, transport: TransportInterface):
+        """处理中断消息"""
+        logger.info("Audio processor: Abort message received")
+        context.abort_requested = True
+        
+        # 清理队列
+        await self._clear_queues(context)
+        
+        # 打断客户端说话状态
+        await transport.send(json.dumps({
+            "type": "tts", 
+            "state": "stop", 
+            "session_id": context.session_id
+        }))
+        
+        # 清理说话状态
+        context.is_speaking = False
+
+    async def _clear_queues(self, context: SessionContext):
+        """清理所有队列"""
+        # 清理TTS音频队列
+        tts_component = context.components.get('tts')
+        if tts_component and hasattr(tts_component, 'tts_instance'):
+            tts_instance = tts_component.tts_instance
+            if hasattr(tts_instance, 'tts_audio_queue'):
+                try:
+                    while not tts_instance.tts_audio_queue.empty():
+                        tts_instance.tts_audio_queue.get_nowait()
+                except:
+                    pass
+        
+        # 清理ASR音频队列
+        context.clear_audio_buffer()
+
+    async def _send_stt_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """发送STT消息"""
+        await transport.send(json.dumps({
+            "type": "stt",
+            "text": text,
+            "session_id": context.session_id
+        }))
+
+

main/xiaozhi-server/core/processors/audio_send_processor.py

@@ -0,0 +1,132 @@
+import json
+import time
+import asyncio
+from typing import Any, List
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.providers.tts.dto.dto import SentenceType
+from core.utils import textUtils
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class AudioSendProcessor(MessageProcessor):
+    """音频发送处理器：完整迁移sendAudioHandle.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """这个处理器不直接处理消息，而是被其他处理器调用"""
+        return False
+    
+    async def send_audio_message(self, context: SessionContext, transport: TransportInterface, 
+                                sentence_type: SentenceType, audios: bytes, text: str):
+        """发送音频消息 - 完整迁移自sendAudioMessage"""
+        tts_component = context.components.get('tts')
+        if not tts_component or not hasattr(tts_component, 'tts_instance'):
+            return
+            
+        tts_instance = tts_component.tts_instance
+        
+        if hasattr(tts_instance, 'tts_audio_first_sentence') and tts_instance.tts_audio_first_sentence:
+            logger.info(f"发送第一段语音: {text}")
+            tts_instance.tts_audio_first_sentence = False
+            await self.send_tts_message(context, transport, "start", None)
+
+        if sentence_type == SentenceType.FIRST:
+            await self.send_tts_message(context, transport, "sentence_start", text)
+
+        await self.send_audio(context, transport, audios)
+        
+        # 发送句子开始消息
+        if sentence_type is not SentenceType.MIDDLE:
+            logger.info(f"发送音频消息: {sentence_type}, {text}")
+
+        # 发送结束消息（如果是最后一个文本）
+        if context.llm_finish_task and sentence_type == SentenceType.LAST:
+            await self.send_tts_message(context, transport, "stop", None)
+            context.is_speaking = False
+            if context.close_after_chat:
+                await transport.close()
+    
+    async def send_audio(self, context: SessionContext, transport: TransportInterface, 
+                        audios: bytes, frame_duration: int = 60):
+        """发送单个opus包，支持流控 - 完整迁移自sendAudio"""
+        if audios is None or len(audios) == 0:
+            return
+
+        if isinstance(audios, bytes):
+            if context.abort_requested:
+                return
+
+            context.update_activity()
+            await transport.send(audios)
+            await asyncio.sleep(frame_duration / 1000.0)
+        elif isinstance(audios, list):
+            for audio in audios:
+                if context.abort_requested:
+                    break
+                context.update_activity()
+                await transport.send(audio)
+                await asyncio.sleep(frame_duration / 1000.0)
+    
+    async def send_stt_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """发送STT消息 - 完整迁移自send_stt_message"""
+        await transport.send(json.dumps({
+            "type": "stt",
+            "text": text,
+            "session_id": context.session_id
+        }))
+        logger.info(f"发送STT消息: {text}")
+    
+    async def send_tts_message(self, context: SessionContext, transport: TransportInterface, 
+                              state: str, text: str = None):
+        """发送TTS消息 - 完整迁移自send_tts_message"""
+        message = {
+            "type": "tts",
+            "state": state,
+            "session_id": context.session_id
+        }
+        if text:
+            message["text"] = text
+            
+        await transport.send(json.dumps(message))
+        logger.debug(f"发送TTS消息: state={state}, text={text}")
+    
+    async def send_music_message(self, context: SessionContext, transport: TransportInterface, 
+                                music_path: str, text: str):
+        """发送音乐消息 - 完整迁移自send_music_message"""
+        from core.utils.util import audio_to_data
+        
+        try:
+            # 获取音频数据
+            opus_packets = audio_to_data(music_path)
+            if opus_packets:
+                # 发送音乐开始消息
+                await self.send_tts_message(context, transport, "start", text)
+                
+                # 发送音频数据
+                await self.send_audio(context, transport, opus_packets)
+                
+                # 发送音乐结束消息
+                await self.send_tts_message(context, transport, "stop", None)
+                
+                logger.info(f"发送音乐: {music_path}")
+            else:
+                logger.warning(f"无法加载音乐文件: {music_path}")
+                
+        except Exception as e:
+            logger.error(f"发送音乐失败: {e}")
+    
+    async def send_welcome_audio(self, context: SessionContext, transport: TransportInterface):
+        """发送欢迎音频"""
+        welcome_audio_path = context.config.get("welcome_audio_path")
+        if welcome_audio_path:
+            await self.send_music_message(context, transport, welcome_audio_path, "欢迎使用小智助手")
+    
+    async def send_goodbye_audio(self, context: SessionContext, transport: TransportInterface):
+        """发送告别音频"""
+        goodbye_audio_path = context.config.get("goodbye_audio_path")
+        if goodbye_audio_path:
+            await self.send_music_message(context, transport, goodbye_audio_path, "再见，期待下次相遇")
+

main/xiaozhi-server/core/processors/auth_processor.py

@@ -0,0 +1,44 @@
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.auth import AuthMiddleware, AuthenticationError
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class AuthProcessor(MessageProcessor):
+    """认证处理器：处理连接认证逻辑"""
+    
+    def __init__(self):
+        self.auth_middleware = None
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理认证相关逻辑"""
+        # 如果已经认证，跳过
+        if context.is_authenticated:
+            return False
+        
+        # 初始化认证中间件（延迟初始化）
+        if self.auth_middleware is None:
+            self.auth_middleware = AuthMiddleware(context.config)
+        
+        # 检查是否为认证消息（通过headers进行认证）
+        if context.headers:
+            try:
+                await self.auth_middleware.authenticate(context.headers)
+                context.is_authenticated = True
+                logger.info(f"设备认证成功: {context.device_id}")
+                return False  # 认证成功，继续处理其他消息
+            except AuthenticationError as e:
+                logger.error(f"设备认证失败: {e}")
+                # 发送认证失败消息
+                await transport.send("Authentication failed")
+                await transport.close()
+                return True  # 认证失败，停止处理
+        
+        # 如果没有认证信息，要求认证
+        await transport.send("Authentication required")
+        return True  # 停止后续处理
+

main/xiaozhi-server/core/processors/chat_processor.py

@@ -0,0 +1,516 @@
+import json
+import uuid
+import asyncio
+from typing import Any, Dict
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.utils.dialogue import Message, Dialogue
+from core.utils.util import remove_punctuation_and_length
+from core.providers.tts.dto.dto import ContentType, TTSMessageDTO, SentenceType
+from plugins_func.register import Action, ActionResponse
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class ChatProcessor(MessageProcessor):
+    """聊天处理器：完整迁移intentHandler.py的所有功能"""
+    
+    def __init__(self):
+        # 会话对话历史管理
+        self._dialogues: Dict[str, Dialogue] = {}
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理聊天消息"""
+        # 这个处理器不直接处理原始消息，而是被其他处理器调用
+        return False
+    
+    async def handle_chat(self, context: SessionContext, transport: TransportInterface, text: str):
+        """处理聊天请求 - 完整迁移自handle_user_intent"""
+        try:
+            # 首先进行意图处理
+            intent_handled = await self.handle_user_intent(context, transport, text)
+            if intent_handled:
+                return
+                
+            # 如果意图未处理，进行常规聊天
+            await self._regular_chat(context, transport, text)
+            
+        except Exception as e:
+            logger.error(f"处理聊天失败: {e}")
+            await self._send_error(transport, "聊天处理失败，请重试")
+    
+    async def handle_user_intent(self, context: SessionContext, transport: TransportInterface, text: str):
+        """处理用户意图 - 完整迁移自intentHandler.py"""
+        # 预处理输入文本，处理可能的JSON格式
+        try:
+            if text.strip().startswith('{') and text.strip().endswith('}'):
+                parsed_data = json.loads(text)
+                if isinstance(parsed_data, dict) and "content" in parsed_data:
+                    text = parsed_data["content"]  # 提取content用于意图分析
+                    context.current_speaker = parsed_data.get("speaker")  # 保留说话人信息
+        except (json.JSONDecodeError, TypeError):
+            pass
+
+        # 检查是否有明确的退出命令
+        _, filtered_text = remove_punctuation_and_length(text)
+        if await self._check_direct_exit(context, transport, filtered_text):
+            return True
+
+        # 检查是否是唤醒词
+        if await self._check_wakeup_words(context, transport, filtered_text):
+            return True
+
+        if context.intent_type == "function_call":
+            # 使用支持function calling的聊天方法,不再进行意图分析
+            return False
+            
+        # 使用LLM进行意图分析
+        intent_result = await self._analyze_intent_with_llm(context, text)
+        if not intent_result:
+            return False
+            
+        # 会话开始时生成sentence_id
+        context.sentence_id = str(uuid.uuid4().hex)
+        
+        # 处理各种意图
+        return await self._process_intent_result(context, transport, intent_result, text)
+    
+    def _get_dialogue(self, session_id: str) -> Dialogue:
+        """获取或创建对话历史"""
+        if session_id not in self._dialogues:
+            self._dialogues[session_id] = Dialogue()
+        return self._dialogues[session_id]
+    
+    async def _get_memory_context(self, context: SessionContext, query: str) -> str:
+        """获取记忆上下文"""
+        try:
+            memory_component = context.components.get('memory')
+            if memory_component and hasattr(memory_component, 'memory_instance'):
+                memory_instance = memory_component.memory_instance
+                if hasattr(memory_instance, 'query_memory'):
+                    return await memory_instance.query_memory(query)
+        except Exception as e:
+            logger.warning(f"获取记忆上下文失败: {e}")
+        
+        return None
+    
+    async def _generate_llm_response(self, context: SessionContext, transport: TransportInterface, 
+                                   llm_instance, dialogue_context: list, dialogue: Dialogue):
+        """生成LLM回复"""
+        try:
+            # 初始化sentence_id并发送TTS FIRST标记（模拟原connection.py第692-700行）
+            if not context.sentence_id:
+                context.sentence_id = str(uuid.uuid4().hex)
+            
+            # 发送TTS开始标记
+            await self._send_tts_first_marker(context)
+            
+            # 检查是否支持流式响应
+            if hasattr(llm_instance, 'response'):
+                # 使用流式响应
+                response_generator = llm_instance.response(context.session_id, dialogue_context)
+                
+                response_parts = []
+                async for response_part in self._async_generator_wrapper(response_generator):
+                    if context.abort_requested:
+                        break
+                    
+                    if response_part and len(response_part) > 0:
+                        response_parts.append(response_part)
+                        
+                        # 原架构不发送流式响应给前端，直接进行TTS处理
+                        # 将响应片段放入TTS队列进行语音合成
+                        await self._process_response_part_for_tts(context, response_part)
+                
+                # 完整回复
+                full_response = "".join(response_parts)
+                if full_response:
+                    # 添加助手回复到对话历史
+                    dialogue.put(Message(role="assistant", content=full_response))
+                    
+                    # 原架构不发送response_complete给前端，只进行TTS处理
+                    # 发送TTS结束标记
+                    await self._finalize_tts_response(context, full_response)
+                    
+                    logger.info(f"LLM回复完成: {full_response[:100]}...")
+                
+            else:
+                logger.warning("LLM实例不支持流式响应")
+                
+        except Exception as e:
+            logger.error(f"生成LLM回复失败: {e}")
+            await self._send_error(transport, "生成回复失败")
+    
+    async def _async_generator_wrapper(self, generator):
+        """将同步生成器包装为异步生成器"""
+        try:
+            for item in generator:
+                yield item
+                # 让出控制权，避免阻塞事件循环
+                await asyncio.sleep(0)
+        except Exception as e:
+            logger.error(f"生成器包装失败: {e}")
+    
+    async def _send_tts_first_marker(self, context: SessionContext):
+        """发送TTS开始标记"""
+        try:
+            tts_component = context.components.get('tts')
+            if not tts_component or not hasattr(tts_component, 'tts_instance'):
+                return
+                
+            tts_instance = tts_component.tts_instance
+            if not tts_instance or not hasattr(tts_instance, 'tts_text_queue'):
+                return
+            
+            # 发送TTS开始标记（模拟原connection.py第694-700行）
+            tts_instance.tts_text_queue.put(TTSMessageDTO(
+                sentence_id=context.sentence_id,
+                sentence_type=SentenceType.FIRST,
+                content_type=ContentType.ACTION
+            ))
+                
+        except Exception as e:
+            logger.error(f"发送TTS开始标记失败: {e}")
+    
+    async def _process_response_part_for_tts(self, context: SessionContext, response_part: str):
+        """处理响应片段进行TTS - 模拟原架构逻辑"""
+        try:
+            tts_component = context.components.get('tts')
+            if not tts_component or not hasattr(tts_component, 'tts_instance'):
+                return
+                
+            tts_instance = tts_component.tts_instance
+            if not tts_instance or not hasattr(tts_instance, 'tts_text_queue'):
+                return
+            
+            # 将响应片段放入TTS队列（模拟原connection.py第782-789行逻辑）
+            tts_instance.tts_text_queue.put(TTSMessageDTO(
+                sentence_id=context.sentence_id,
+                sentence_type=SentenceType.MIDDLE,
+                content_type=ContentType.TEXT,
+                content_detail=response_part
+            ))
+                
+        except Exception as e:
+            logger.error(f"处理TTS响应片段失败: {e}")
+    
+    async def _finalize_tts_response(self, context: SessionContext, full_response: str):
+        """完成TTS响应 - 发送结束标记"""
+        try:
+            tts_component = context.components.get('tts')
+            if not tts_component or not hasattr(tts_component, 'tts_instance'):
+                return
+                
+            tts_instance = tts_component.tts_instance
+            if not tts_instance or not hasattr(tts_instance, 'tts_text_queue'):
+                return
+            
+            # 发送TTS结束标记（模拟原speak_txt函数逻辑）
+            tts_instance.tts_text_queue.put(TTSMessageDTO(
+                sentence_id=context.sentence_id,
+                sentence_type=SentenceType.LAST,
+                content_type=ContentType.ACTION
+            ))
+            
+            # 设置LLM完成标记
+            context.llm_finish_task = True
+                
+        except Exception as e:
+            logger.error(f"完成TTS响应失败: {e}")
+    
+    async def _trigger_tts(self, context: SessionContext, transport: TransportInterface, text: str):
+        """触发TTS语音合成 - 完整迁移自原chat方法的TTS处理"""
+        try:
+            tts_component = context.components.get('tts')
+            if not tts_component or not hasattr(tts_component, 'tts_instance'):
+                logger.warning("TTS组件未初始化")
+                return
+                
+            tts_instance = tts_component.tts_instance
+            
+            # 确保有sentence_id
+            if not context.sentence_id:
+                context.sentence_id = str(uuid.uuid4().hex)
+            
+            logger.info(f"触发TTS合成: {text[:50]}...")
+            
+            # 使用原来的TTS处理方式
+            if hasattr(tts_instance, 'tts_text_queue') and hasattr(tts_instance, 'tts_one_sentence'):
+                # 发送FIRST消息到TTS队列
+                tts_instance.tts_text_queue.put(
+                    TTSMessageDTO(
+                        sentence_id=context.sentence_id,
+                        sentence_type=SentenceType.FIRST,
+                        content_type=ContentType.ACTION,
+                    )
+                )
+                
+                # 合成一句话
+                tts_instance.tts_one_sentence(context, ContentType.TEXT, content_detail=text)
+                
+                # 发送LAST消息到TTS队列
+                tts_instance.tts_text_queue.put(
+                    TTSMessageDTO(
+                        sentence_id=context.sentence_id,
+                        sentence_type=SentenceType.LAST,
+                        content_type=ContentType.ACTION,
+                    )
+                )
+                
+                logger.info("TTS合成任务已提交到队列")
+            else:
+                logger.warning("TTS实例不支持队列处理")
+                
+        except Exception as e:
+            logger.error(f"TTS合成失败: {e}")
+    
+    async def _send_error(self, transport: TransportInterface, error_message: str):
+        """发送错误消息"""
+        try:
+            await transport.send(json.dumps({
+                "type": "error",
+                "message": error_message
+            }))
+        except Exception as e:
+            logger.error(f"发送错误消息失败: {e}")
+    
+    # === 意图处理相关方法：完整迁移自intentHandler.py ===
+    
+    async def _check_direct_exit(self, context: SessionContext, transport: TransportInterface, text: str):
+        """检查是否有明确的退出命令 - 完整迁移自check_direct_exit"""
+        _, text = remove_punctuation_and_length(text)
+        cmd_exit = context.cmd_exit
+        for cmd in cmd_exit:
+            if text == cmd:
+                logger.info(f"识别到明确的退出命令: {text}")
+                await self._send_stt_message(context, transport, text)
+                await transport.close()
+                return True
+        return False
+
+    async def _check_wakeup_words(self, context: SessionContext, transport: TransportInterface, text: str):
+        """检查唤醒词 - 调用TextProcessor的方法"""
+        # 这里需要调用TextProcessor的checkWakeupWords方法
+        # 为了避免循环依赖，我们在这里实现简化版本
+        _, filtered_text = remove_punctuation_and_length(text)
+        if filtered_text in context.config.get("wakeup_words", []):
+            return True
+        return False
+
+    async def _analyze_intent_with_llm(self, context: SessionContext, text: str):
+        """使用LLM分析用户意图 - 完整迁移自analyze_intent_with_llm"""
+        intent_component = context.components.get('intent')
+        if not intent_component or not hasattr(intent_component, 'intent_instance'):
+            logger.warning("意图识别服务未初始化")
+            return None
+
+        intent_instance = intent_component.intent_instance
+        
+        # 对话历史记录
+        dialogue = context.dialogue
+        if not dialogue:
+            return None
+            
+        try:
+            intent_result = await intent_instance.detect_intent(context, dialogue.dialogue, text)
+            return intent_result
+        except Exception as e:
+            logger.error(f"意图识别失败: {str(e)}")
+
+        return None
+
+    async def _process_intent_result(self, context: SessionContext, transport: TransportInterface, intent_result: str, original_text: str):
+        """处理意图识别结果 - 完整迁移自process_intent_result"""
+        try:
+            # 尝试将结果解析为JSON
+            intent_data = json.loads(intent_result)
+
+            # 检查是否有function_call
+            if "function_call" in intent_data:
+                # 直接从意图识别获取了function_call
+                logger.debug(f"检测到function_call格式的意图结果: {intent_data['function_call']['name']}")
+                function_name = intent_data["function_call"]["name"]
+                if function_name == "continue_chat":
+                    return False
+
+                function_args = {}
+                if "arguments" in intent_data["function_call"]:
+                    function_args = intent_data["function_call"]["arguments"]
+                    if function_args is None:
+                        function_args = {}
+                        
+                # 确保参数是字符串格式的JSON
+                if isinstance(function_args, dict):
+                    function_args = json.dumps(function_args)
+
+                function_call_data = {
+                    "name": function_name,
+                    "id": str(uuid.uuid4().hex),
+                    "arguments": function_args,
+                }
+
+                await self._send_stt_message(context, transport, original_text)
+                context.abort_requested = False
+
+                # 使用executor执行函数调用和结果处理
+                await self._process_function_call(context, transport, function_call_data, original_text)
+                return True
+            return False
+        except json.JSONDecodeError as e:
+            logger.error(f"处理意图结果时出错: {e}")
+            return False
+
+    async def _process_function_call(self, context: SessionContext, transport: TransportInterface, function_call_data: dict, original_text: str):
+        """处理函数调用 - 完整迁移自process_function_call"""
+        def process_function_call():
+            # 添加用户消息到对话历史
+            dialogue = context.dialogue
+            if dialogue:
+                dialogue.put(Message(role="user", content=original_text))
+
+            # 使用统一工具处理器处理所有工具调用
+            try:
+                func_handler = context.func_handler
+                if not func_handler:
+                    raise Exception("函数处理器未初始化")
+                    
+                loop = context.loop
+                result = asyncio.run_coroutine_threadsafe(
+                    func_handler.handle_llm_function_call(context, function_call_data),
+                    loop,
+                ).result()
+            except Exception as e:
+                logger.error(f"工具调用失败: {e}")
+                result = ActionResponse(
+                    action=Action.ERROR, result=str(e), response=str(e)
+                )
+
+            if result:
+                function_name = function_call_data.get("name", "")
+                if result.action == Action.RESPONSE:  # 直接回复前端
+                    text = result.response
+                    if text is not None:
+                        self._speak_txt(context, text)
+                elif result.action == Action.REQLLM:  # 调用函数后再请求llm生成回复
+                    text = result.result
+                    if dialogue:
+                        dialogue.put(Message(role="tool", content=text))
+                    intent_component = context.components.get('intent')
+                    if intent_component and hasattr(intent_component, 'intent_instance'):
+                        intent_instance = intent_component.intent_instance
+                        if hasattr(intent_instance, 'replyResult'):
+                            llm_result = intent_instance.replyResult(text, original_text)
+                            if llm_result is None:
+                                llm_result = text
+                            self._speak_txt(context, llm_result)
+                elif (
+                    result.action == Action.NOTFOUND
+                    or result.action == Action.ERROR
+                ):
+                    text = result.result
+                    if text is not None:
+                        self._speak_txt(context, text)
+                elif function_name != "play_music":
+                    # For backward compatibility with original code
+                    # 获取当前最新的文本索引
+                    text = result.response
+                    if text is None:
+                        text = result.result
+                    if text is not None:
+                        self._speak_txt(context, text)
+
+        # 将函数执行放在线程池中
+        if context.executor:
+            context.executor.submit(process_function_call)
+        else:
+            # 如果没有executor，直接执行
+            process_function_call()
+
+    def _speak_txt(self, context: SessionContext, text: str):
+        """语音合成文本 - 完整迁移自speak_txt"""
+        tts_component = context.components.get('tts')
+        if not tts_component or not hasattr(tts_component, 'tts_instance'):
+            return
+            
+        tts_instance = tts_component.tts_instance
+        sentence_id = context.sentence_id or str(uuid.uuid4().hex)
+        
+        # 发送TTS消息队列
+        if hasattr(tts_instance, 'tts_text_queue'):
+            tts_instance.tts_text_queue.put(
+                TTSMessageDTO(
+                    sentence_id=sentence_id,
+                    sentence_type=SentenceType.FIRST,
+                    content_type=ContentType.ACTION,
+                )
+            )
+            
+            # 合成一句话
+            if hasattr(tts_instance, 'tts_one_sentence'):
+                tts_instance.tts_one_sentence(context, ContentType.TEXT, content_detail=text)
+                
+            tts_instance.tts_text_queue.put(
+                TTSMessageDTO(
+                    sentence_id=sentence_id,
+                    sentence_type=SentenceType.LAST,
+                    content_type=ContentType.ACTION,
+                )
+            )
+        
+        # 添加到对话历史
+        dialogue = context.dialogue
+        if dialogue:
+            dialogue.put(Message(role="assistant", content=text))
+
+    async def _regular_chat(self, context: SessionContext, transport: TransportInterface, text: str):
+        """常规聊天处理"""
+        # 使用SessionContext的对话历史
+        dialogue = context.dialogue
+        if not dialogue:
+            from core.utils.dialogue import Dialogue
+            dialogue = Dialogue()
+            context.dialogue = dialogue
+        
+        # 获取LLM组件
+        llm_component = context.components.get('llm')
+        if not llm_component:
+            await self._send_error(transport, "LLM组件未初始化")
+            return
+        
+        llm_instance = getattr(llm_component, 'llm_instance', None)
+        if not llm_instance:
+            await self._send_error(transport, "LLM实例未就绪")
+            return
+        
+        # 添加用户消息到对话历史
+        dialogue.put(Message(role="user", content=text))
+        
+        # 原架构不发送thinking状态给前端，直接开始处理
+        
+        # 获取记忆上下文
+        memory_context = await self._get_memory_context(context, text)
+        
+        # 构建对话上下文
+        dialogue_context = dialogue.get_llm_dialogue_with_memory(
+            memory_context, 
+            context.config.get("voiceprint", {})
+        )
+        
+        # 调用LLM生成回复
+        await self._generate_llm_response(context, transport, llm_instance, dialogue_context, dialogue)
+
+    async def _send_stt_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """发送STT消息"""
+        await transport.send(json.dumps({
+            "type": "stt",
+            "text": text,
+            "session_id": context.session_id
+        }))
+
+    def cleanup_session(self, session_id: str):
+        """清理会话对话历史"""
+        if session_id in self._dialogues:
+            del self._dialogues[session_id]
+            logger.info(f"已清理会话对话历史: {session_id}")

main/xiaozhi-server/core/processors/hello_processor.py

@@ -0,0 +1,206 @@
+import time
+import json
+import random
+import asyncio
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.utils.dialogue import Message
+from core.utils.util import audio_to_data, remove_punctuation_and_length, opus_datas_to_wav_bytes
+from core.providers.tts.dto.dto import SentenceType
+from core.utils.wakeup_word import WakeupWordsConfig
+from core.providers.tools.device_mcp import (
+    MCPClient,
+    send_mcp_initialize_message,
+    send_mcp_tools_list_request,
+)
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+# 唤醒词配置
+WAKEUP_CONFIG = {
+    "refresh_time": 5,
+    "words": ["你好", "你好啊", "嘿，你好", "嗨"],
+}
+
+# 创建全局的唤醒词配置管理器
+wakeup_words_config = WakeupWordsConfig()
+
+# 用于防止并发调用wakeupWordsResponse的锁
+_wakeup_response_lock = asyncio.Lock()
+
+
+class HelloProcessor(MessageProcessor):
+    """Hello消息处理器：完整迁移helloHandle.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理hello类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "hello":
+                    await self.handle_hello_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_hello_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理hello消息 - 完整迁移自handleHelloMessage"""
+        # 处理音频参数
+        audio_params = msg_json.get("audio_params")
+        if audio_params:
+            format = audio_params.get("format")
+            logger.info(f"客户端音频格式: {format}")
+            context.audio_format = format
+            if not context.welcome_msg:
+                context.welcome_msg = {}
+            context.welcome_msg["audio_params"] = audio_params
+            
+        # 处理客户端特性
+        features = msg_json.get("features")
+        if features:
+            logger.info(f"客户端特性: {features}")
+            context.features = features
+            if features.get("mcp"):
+                logger.info("客户端支持MCP")
+                context.mcp_client = MCPClient()
+                # 发送初始化 - 传递transport参数
+                asyncio.create_task(send_mcp_initialize_message(context, transport))
+                # 发送mcp消息，获取tools列表 - 传递transport参数
+                asyncio.create_task(send_mcp_tools_list_request(context, transport))
+
+        # 发送欢迎消息
+        if context.welcome_msg:
+            await transport.send(json.dumps(context.welcome_msg))
+        else:
+            # 默认欢迎消息
+            welcome_msg = {
+                "type": "hello",
+                "session_id": context.session_id,
+                "version": 1,
+                "transport": "websocket"
+            }
+            await transport.send(json.dumps(welcome_msg))
+    
+    async def check_wakeup_words(self, context: SessionContext, transport: TransportInterface, text: str) -> bool:
+        """检查唤醒词 - 完整迁移自checkWakeupWords"""
+        enable_wakeup_words_response_cache = context.config.get("enable_wakeup_words_response_cache", False)
+
+        # 等待tts初始化，最多等待3秒
+        tts_component = context.components.get('tts')
+        start_time = time.time()
+        while time.time() - start_time < 3:
+            if tts_component and hasattr(tts_component, 'tts_instance'):
+                break
+            await asyncio.sleep(0.1)
+        else:
+            return False
+
+        if not enable_wakeup_words_response_cache:
+            return False
+
+        _, filtered_text = remove_punctuation_and_length(text)
+        if filtered_text not in context.config.get("wakeup_words", []):
+            return False
+
+        context.just_woken_up = True
+        await self._send_stt_message(context, transport, text)
+
+        # 获取当前音色
+        tts_instance = getattr(tts_component, 'tts_instance', None) if tts_component else None
+        voice = getattr(tts_instance, "voice", "default") if tts_instance else "default"
+        if not voice:
+            voice = "default"
+
+        # 获取唤醒词回复配置
+        response = wakeup_words_config.get_wakeup_response(voice)
+        if not response or not response.get("file_path"):
+            response = {
+                "voice": "default",
+                "file_path": "config/assets/wakeup_words.wav",
+                "time": 0,
+                "text": "哈啰啊，我是小智啦，声音好听的台湾女孩一枚，超开心认识你耶，最近在忙啥，别忘了给我来点有趣的料哦，我超爱听八卦的啦",
+            }
+
+        # 获取音频数据
+        opus_packets = audio_to_data(response.get("file_path"))
+        # 播放唤醒词回复
+        context.abort_requested = False
+
+        logger.info(f"播放唤醒词回复: {response.get('text')}")
+        await self._send_audio_message(context, transport, SentenceType.FIRST, opus_packets, response.get("text"))
+        await self._send_audio_message(context, transport, SentenceType.LAST, [], None)
+
+        # 补充对话
+        if context.dialogue:
+            context.dialogue.put(Message(role="assistant", content=response.get("text")))
+
+        # 检查是否需要更新唤醒词回复
+        if time.time() - response.get("time", 0) > WAKEUP_CONFIG["refresh_time"]:
+            if not _wakeup_response_lock.locked():
+                asyncio.create_task(self._wakeup_words_response(context, transport))
+        return True
+
+    async def _wakeup_words_response(self, context: SessionContext, transport: TransportInterface):
+        """生成唤醒词回复 - 完整迁移自wakeupWordsResponse"""
+        tts_component = context.components.get('tts')
+        llm_component = context.components.get('llm')
+        
+        tts_instance = getattr(tts_component, 'tts_instance', None) if tts_component else None
+        llm_instance = getattr(llm_component, 'llm_instance', None) if llm_component else None
+        
+        if not tts_instance or not llm_instance or not hasattr(llm_instance, 'response_no_stream'):
+            return
+
+        try:
+            # 尝试获取锁，如果获取不到就返回
+            async with _wakeup_response_lock:
+                # 生成唤醒词回复
+                wakeup_word = random.choice(WAKEUP_CONFIG["words"])
+                question = (
+                    "此刻用户正在和你说```"
+                    + wakeup_word
+                    + "```。\n请你根据以上用户的内容进行20-30字回复。要符合系统设置的角色情感和态度，不要像机器人一样说话。\n"
+                    + "请勿对这条内容本身进行任何解释和回应，请勿返回表情符号，仅返回对用户的内容的回复。"
+                )
+
+                result = llm_instance.response_no_stream(context.config.get("prompt", ""), question)
+                if not result or len(result) == 0:
+                    return
+
+                # 生成TTS音频
+                tts_result = await asyncio.to_thread(tts_instance.to_tts, result)
+                if not tts_result:
+                    return
+
+                # 获取当前音色
+                voice = getattr(tts_instance, "voice", "default")
+
+                wav_bytes = opus_datas_to_wav_bytes(tts_result, sample_rate=16000)
+                file_path = wakeup_words_config.generate_file_path(voice)
+                with open(file_path, "wb") as f:
+                    f.write(wav_bytes)
+                # 更新配置
+                wakeup_words_config.update_wakeup_response(voice, file_path, result)
+        except Exception as e:
+            logger.error(f"生成唤醒词回复失败: {e}")
+
+    async def _send_stt_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """发送STT消息"""
+        await transport.send(json.dumps({
+            "type": "stt",
+            "text": text,
+            "session_id": context.session_id
+        }))
+
+    async def _send_audio_message(self, context: SessionContext, transport: TransportInterface, 
+                                 sentence_type: SentenceType, audios: bytes, text: str):
+        """发送音频消息"""
+        # 这里应该调用AudioSendProcessor
+        from core.processors.audio_send_processor import AudioSendProcessor
+        audio_send_processor = AudioSendProcessor()
+        await audio_send_processor.send_audio_message(context, transport, sentence_type, audios, text)
+

main/xiaozhi-server/core/processors/iot_processor.py

@@ -0,0 +1,123 @@
+import asyncio
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.providers.tools.device_iot import handleIotStatus, handleIotDescriptors
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class IotProcessor(MessageProcessor):
+    """IoT消息处理器：完整迁移iotMessageHandler.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理iot类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "iot":
+                    await self.handle_iot_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_iot_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理IoT消息 - 完整迁移自iotMessageHandler.py"""
+        tasks = []
+        
+        # 处理设备描述符 - 完整迁移原逻辑
+        if "descriptors" in msg_json:
+            logger.debug("处理IoT设备描述符")
+            task = asyncio.create_task(
+                self._handle_iot_descriptors(context, transport, msg_json["descriptors"])
+            )
+            tasks.append(task)
+        
+        # 处理设备状态 - 完整迁移原逻辑
+        if "states" in msg_json:
+            logger.debug("处理IoT设备状态")
+            task = asyncio.create_task(
+                self._handle_iot_status(context, transport, msg_json["states"])
+            )
+            tasks.append(task)
+        
+        # 如果没有有效的IoT数据
+        if not tasks:
+            logger.warning("IoT消息缺少descriptors或states字段")
+            await self._send_error_response(
+                transport, 
+                context.session_id, 
+                "IoT消息格式错误：缺少descriptors或states字段"
+            )
+            return
+        
+        # 等待所有任务完成（可选，根据原逻辑决定）
+        # await asyncio.gather(*tasks, return_exceptions=True)
+    
+    async def _handle_iot_descriptors(self, context: SessionContext, transport: TransportInterface, descriptors: Any):
+        """处理IoT设备描述符 - 包装原handleIotDescriptors函数"""
+        try:
+            # 调用原有的handleIotDescriptors函数
+            # 注意：这里需要传入context而不是conn，因为handleIotDescriptors可能需要适配
+            await handleIotDescriptors(context, descriptors)
+            logger.debug("IoT设备描述符处理完成")
+            
+        except Exception as e:
+            logger.error(f"处理IoT设备描述符失败: {e}", exc_info=True)
+            await self._send_error_response(
+                transport, 
+                context.session_id, 
+                f"IoT设备描述符处理失败: {str(e)}"
+            )
+    
+    async def _handle_iot_status(self, context: SessionContext, transport: TransportInterface, states: Any):
+        """处理IoT设备状态 - 包装原handleIotStatus函数"""
+        try:
+            # 调用原有的handleIotStatus函数
+            # 注意：这里需要传入context而不是conn，因为handleIotStatus可能需要适配
+            await handleIotStatus(context, states)
+            logger.debug("IoT设备状态处理完成")
+            
+        except Exception as e:
+            logger.error(f"处理IoT设备状态失败: {e}", exc_info=True)
+            await self._send_error_response(
+                transport, 
+                context.session_id, 
+                f"IoT设备状态处理失败: {str(e)}"
+            )
+    
+    async def _send_error_response(self, transport: TransportInterface, session_id: str, message: str):
+        """发送IoT错误响应"""
+        response = {
+            "type": "iot",
+            "status": "error",
+            "message": message,
+            "session_id": session_id
+        }
+        
+        try:
+            await transport.send(json.dumps(response))
+        except Exception as e:
+            logger.error(f"发送IoT错误响应失败: {e}")
+    
+    async def _send_success_response(self, transport: TransportInterface, session_id: str, 
+                                   message: str, data: dict = None):
+        """发送IoT成功响应"""
+        response = {
+            "type": "iot",
+            "status": "success",
+            "message": message,
+            "session_id": session_id
+        }
+        if data:
+            response["data"] = data
+            
+        try:
+            await transport.send(json.dumps(response))
+        except Exception as e:
+            logger.error(f"发送IoT成功响应失败: {e}")
+

main/xiaozhi-server/core/processors/listen_processor.py

@@ -0,0 +1,129 @@
+import time
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.utils.util import remove_punctuation_and_length
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class ListenProcessor(MessageProcessor):
+    """Listen消息处理器：完整迁移listenMessageHandler.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理listen类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "listen":
+                    await self.handle_listen_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_listen_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理listen消息 - 完整迁移自listenMessageHandler.py"""
+        # 设置拾音模式
+        if "mode" in msg_json:
+            context.listen_mode = msg_json["mode"]
+            logger.debug(f"客户端拾音模式：{context.listen_mode}")
+        
+        # 处理不同的状态
+        state = msg_json.get("state")
+        
+        if state == "start":
+            # 开始监听语音
+            context.client_have_voice = True
+            context.client_voice_stop = False
+            logger.debug("开始语音监听")
+            
+        elif state == "stop":
+            # 停止监听语音
+            context.client_have_voice = True
+            context.client_voice_stop = True
+            # 如果有音频数据，处理最后的音频
+            if len(context.asr_audio) > 0:
+                await self._handle_audio_message(context, transport, b"")
+            logger.debug("停止语音监听")
+            
+        elif state == "detect":
+            # 检测到文本输入
+            context.client_have_voice = False
+            context.asr_audio.clear()
+            
+            if "text" in msg_json:
+                context.update_activity()
+                original_text = msg_json["text"]  # 保留原始文本
+                filtered_len, filtered_text = remove_punctuation_and_length(original_text)
+
+                # 识别是否是唤醒词
+                is_wakeup_words = filtered_text in context.config.get("wakeup_words", [])
+                # 是否开启唤醒词回复
+                enable_greeting = context.config.get("enable_greeting", True)
+
+                if is_wakeup_words and not enable_greeting:
+                    # 如果是唤醒词，且关闭了唤醒词回复，就不用回答
+                    await self._send_stt_message(context, transport, original_text)
+                    await self._send_tts_message(context, transport, "stop", None)
+                    context.is_speaking = False
+                    
+                elif is_wakeup_words:
+                    # 处理唤醒词
+                    context.just_woken_up = True
+                    # 上报纯文字数据（复用ASR上报功能，但不提供音频数据）
+                    await self._enqueue_asr_report(context, "嘿，你好呀", [])
+                    await self._start_to_chat(context, transport, "嘿，你好呀")
+                    
+                else:
+                    # 处理普通文本
+                    # 上报纯文字数据（复用ASR上报功能，但不提供音频数据）
+                    await self._enqueue_asr_report(context, original_text, [])
+                    # 否则需要LLM对文字内容进行答复
+                    await self._start_to_chat(context, transport, original_text)
+    
+    async def _handle_audio_message(self, context: SessionContext, transport: TransportInterface, audio: bytes):
+        """处理音频消息 - 调用AudioReceiveProcessor"""
+        # 这里应该调用AudioReceiveProcessor来处理音频
+        from core.processors.audio_receive_processor import AudioReceiveProcessor
+        audio_processor = AudioReceiveProcessor()
+        await audio_processor.handle_audio_message(context, transport, audio)
+    
+    async def _send_stt_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """发送STT消息"""
+        await transport.send(json.dumps({
+            "type": "stt",
+            "text": text,
+            "session_id": context.session_id
+        }))
+        logger.info(f"发送STT消息: {text}")
+    
+    async def _send_tts_message(self, context: SessionContext, transport: TransportInterface, state: str, text: str = None):
+        """发送TTS消息"""
+        message = {
+            "type": "tts",
+            "state": state,
+            "session_id": context.session_id
+        }
+        if text:
+            message["text"] = text
+            
+        await transport.send(json.dumps(message))
+        logger.debug(f"发送TTS消息: state={state}, text={text}")
+    
+    async def _enqueue_asr_report(self, context: SessionContext, text: str, audio_data: list):
+        """ASR上报队列"""
+        if context.report_asr_enable:
+            from core.processors.report_processor import ReportProcessor
+            report_processor = ReportProcessor()
+            report_processor.enqueue_asr_report(context, text, audio_data)
+    
+    async def _start_to_chat(self, context: SessionContext, transport: TransportInterface, text: str):
+        """开始聊天 - 调用ChatProcessor"""
+        from core.processors.chat_processor import ChatProcessor
+        chat_processor = ChatProcessor()
+        await chat_processor.handle_chat(context, transport, text)
+

main/xiaozhi-server/core/processors/mcp_processor.py

@@ -0,0 +1,90 @@
+import asyncio
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.providers.tools.device_mcp import handle_mcp_message
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class McpProcessor(MessageProcessor):
+    """MCP消息处理器：完整迁移mcpMessageHandler.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理mcp类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "mcp":
+                    await self.handle_mcp_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_mcp_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理MCP消息 - 完整迁移自mcpMessageHandler.py"""
+        if "payload" in msg_json:
+            # 检查MCP客户端是否存在
+            if not context.mcp_client:
+                logger.warning("MCP客户端未初始化，无法处理MCP消息")
+                await self._send_error_response(transport, context.session_id, "MCP客户端未初始化")
+                return
+                
+            # 创建异步任务处理MCP消息 - 完整迁移原逻辑
+            asyncio.create_task(
+                self._handle_mcp_payload(context, transport, msg_json["payload"])
+            )
+        else:
+            logger.warning("MCP消息缺少payload字段")
+            await self._send_error_response(transport, context.session_id, "MCP消息格式错误：缺少payload")
+    
+    async def _handle_mcp_payload(self, context: SessionContext, transport: TransportInterface, payload: dict):
+        """处理MCP payload - 包装原handle_mcp_message函数"""
+        try:
+            # 调用原有的handle_mcp_message函数
+            # 注意：这里需要传入context而不是conn，因为handle_mcp_message可能需要适配
+            await handle_mcp_message(context, context.mcp_client, payload, transport)
+            logger.debug("MCP消息处理完成")
+            
+        except Exception as e:
+            logger.error(f"处理MCP消息失败: {e}", exc_info=True)
+            await self._send_error_response(
+                transport, 
+                context.session_id, 
+                f"MCP消息处理失败: {str(e)}"
+            )
+    
+    async def _send_error_response(self, transport: TransportInterface, session_id: str, message: str):
+        """发送MCP错误响应"""
+        response = {
+            "type": "mcp",
+            "status": "error",
+            "message": message,
+            "session_id": session_id
+        }
+        
+        try:
+            await transport.send(json.dumps(response))
+        except Exception as e:
+            logger.error(f"发送MCP错误响应失败: {e}")
+    
+    async def _send_success_response(self, transport: TransportInterface, session_id: str, 
+                                   message: str, data: dict = None):
+        """发送MCP成功响应"""
+        response = {
+            "type": "mcp",
+            "status": "success", 
+            "message": message,
+            "session_id": session_id
+        }
+        if data:
+            response["data"] = data
+            
+        try:
+            await transport.send(json.dumps(response))
+        except Exception as e:
+            logger.error(f"发送MCP成功响应失败: {e}")

main/xiaozhi-server/core/processors/message_router.py

@@ -0,0 +1,114 @@
+import json
+from typing import Any, List
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from core.processors.hello_processor import HelloProcessor
+from core.processors.listen_processor import ListenProcessor
+from core.processors.audio_receive_processor import AudioReceiveProcessor
+from core.processors.auth_processor import AuthProcessor
+from core.processors.timeout_processor import TimeoutProcessor
+from core.processors.server_processor import ServerProcessor
+from core.processors.mcp_processor import McpProcessor
+from core.processors.iot_processor import IotProcessor
+from core.processors.abort_processor import AbortProcessor
+from core.processors.text_processor import TextProcessor
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class MessageRouter(MessageProcessor):
+    """
+    消息路由器：协调所有独立的processor
+    按功能职责分离，避免耦合，每个processor专注单一职责
+    """
+    
+    def __init__(self):
+        # 初始化所有独立的processor
+        self.auth_processor = AuthProcessor()
+        self.timeout_processor = TimeoutProcessor()
+        self.abort_processor = AbortProcessor()
+        self.hello_processor = HelloProcessor()
+        self.listen_processor = ListenProcessor()
+        self.server_processor = ServerProcessor()
+        self.mcp_processor = McpProcessor()
+        self.iot_processor = IotProcessor()
+        self.audio_receive_processor = AudioReceiveProcessor()
+        self.text_processor = TextProcessor()
+        
+        # 按优先级排序的processor列表
+        self.processors: List[MessageProcessor] = [
+            self.timeout_processor,    # 首先检查超时
+            self.auth_processor,       # 然后检查认证
+            self.abort_processor,      # 中断消息
+            self.hello_processor,      # hello消息
+            self.listen_processor,     # listen消息
+            self.server_processor,     # 服务器消息
+            self.mcp_processor,        # MCP消息
+            self.iot_processor,        # IoT消息
+            self.audio_receive_processor,  # 音频消息
+            self.text_processor,       # 纯文本消息（放在最后，作为兜底处理）
+        ]
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """
+        路由消息到合适的processor
+        每个processor专注处理自己的消息类型，避免耦合
+        """
+        # 更新活动时间
+        context.update_activity()
+        
+        # 按优先级顺序尝试每个processor
+        for processor in self.processors:
+            try:
+                if await processor.process(context, transport, message):
+                    # 消息已被处理，记录日志并返回
+                    logger.debug(f"消息被 {processor.__class__.__name__} 处理")
+                    return True
+            except Exception as e:
+                logger.error(f"{processor.__class__.__name__} 处理消息时出错: {e}", exc_info=True)
+                continue
+        
+        # 如果没有processor处理该消息，记录警告
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                msg_type = msg_json.get("type", "unknown") if isinstance(msg_json, dict) else "non-dict"
+                logger.warning(f"未处理的消息类型: {msg_type}, 内容: {message[:100]}...")
+            except json.JSONDecodeError:
+                logger.warning(f"未处理的非JSON消息: {message[:100]}...")
+        elif isinstance(message, bytes):
+            logger.warning(f"未处理的二进制消息，大小: {len(message)} bytes")
+        else:
+            logger.warning(f"未处理的消息类型: {type(message)}")
+        
+        return False
+    
+    def add_processor(self, processor: MessageProcessor, priority: int = None):
+        """
+        添加新的processor
+        priority: 优先级，数字越小优先级越高，None表示添加到末尾
+        """
+        if priority is None:
+            self.processors.append(processor)
+        else:
+            self.processors.insert(priority, processor)
+        logger.info(f"添加processor: {processor.__class__.__name__}")
+    
+    def remove_processor(self, processor_class):
+        """移除指定类型的processor"""
+        self.processors = [p for p in self.processors if not isinstance(p, processor_class)]
+        logger.info(f"移除processor: {processor_class.__name__}")
+    
+    def get_processor(self, processor_class):
+        """获取指定类型的processor"""
+        for processor in self.processors:
+            if isinstance(processor, processor_class):
+                return processor
+        return None
+    
+    def list_processors(self) -> List[str]:
+        """列出所有processor的名称"""
+        return [processor.__class__.__name__ for processor in self.processors]
+

main/xiaozhi-server/core/processors/report_processor.py

@@ -0,0 +1,176 @@
+import time
+import queue
+import threading
+from typing import Any, List
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from config.manage_api_client import report as manage_report
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class ReportProcessor(MessageProcessor):
+    """上报处理器：完整迁移reportHandle.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """这个处理器不直接处理消息，而是被其他处理器调用"""
+        return False
+    
+    def enqueue_asr_report(self, context: SessionContext, text: str, audio_data: List[bytes]):
+        """ASR上报队列 - 完整迁移自enqueue_asr_report"""
+        if not context.report_asr_enable:
+            return
+            
+        report_time = int(time.time())
+        
+        # 将上报任务放入队列
+        context.report_queue.put({
+            "type": 1,  # 用户类型
+            "text": text,
+            "audio_data": audio_data,
+            "report_time": report_time
+        })
+        
+        # 确保上报线程已启动
+        self._ensure_report_thread(context)
+    
+    def enqueue_tts_report(self, context: SessionContext, text: str, opus_data: bytes):
+        """TTS上报队列 - 完整迁移自enqueue_tts_report"""
+        if not context.report_tts_enable:
+            return
+            
+        report_time = int(time.time())
+        
+        # 将上报任务放入队列
+        context.report_queue.put({
+            "type": 2,  # 智能体类型
+            "text": text,
+            "audio_data": opus_data,
+            "report_time": report_time
+        })
+        
+        # 确保上报线程已启动
+        self._ensure_report_thread(context)
+    
+    def _ensure_report_thread(self, context: SessionContext):
+        """确保上报线程已启动"""
+        if context.report_thread is None or not context.report_thread.is_alive():
+            context.report_thread = threading.Thread(
+                target=self._report_worker,
+                args=(context,),
+                daemon=True
+            )
+            context.report_thread.start()
+            logger.info(f"上报线程已启动: {context.session_id}")
+    
+    def _report_worker(self, context: SessionContext):
+        """上报工作线程 - 完整迁移自ConnectionHandler中的上报逻辑"""
+        logger.info(f"上报工作线程启动: {context.session_id}")
+        
+        while not context.stop_event.is_set():
+            try:
+                # 从队列获取上报任务
+                report_task = context.report_queue.get(timeout=1)
+                
+                # 执行上报
+                self._execute_report(context, report_task)
+                
+            except queue.Empty:
+                continue
+            except Exception as e:
+                logger.error(f"上报工作线程异常: {e}")
+        
+        logger.info(f"上报工作线程退出: {context.session_id}")
+    
+    def _execute_report(self, context: SessionContext, report_task: dict):
+        """执行聊天记录上报操作 - 完整迁移自report函数"""
+        try:
+            report_type = report_task["type"]
+            text = report_task["text"]
+            audio_data = report_task["audio_data"]
+            report_time = report_task["report_time"]
+            
+            # 处理音频数据
+            processed_audio = None
+            if audio_data:
+                if isinstance(audio_data, list):
+                    # ASR音频数据（多个音频片段）
+                    processed_audio = self._process_asr_audio(audio_data)
+                elif isinstance(audio_data, bytes):
+                    # TTS音频数据（opus格式）
+                    processed_audio = self._opus_to_wav(audio_data)
+            
+            # 执行上报
+            manage_report(
+                mac_address=context.device_id,
+                session_id=context.session_id,
+                chat_type=report_type,
+                content=text,
+                audio=processed_audio,
+                report_time=report_time,
+            )
+            
+            logger.debug(f"上报成功: type={report_type}, text={text[:50]}...")
+            
+        except Exception as e:
+            logger.error(f"聊天记录上报失败: {e}")
+    
+    def _process_asr_audio(self, audio_data_list: List[bytes]) -> bytes:
+        """处理ASR音频数据"""
+        try:
+            # 将多个音频片段合并
+            combined_audio = b''.join(audio_data_list)
+            return combined_audio
+        except Exception as e:
+            logger.error(f"处理ASR音频数据失败: {e}")
+            return b''
+    
+    def _opus_to_wav(self, opus_data: bytes) -> bytes:
+        """将Opus数据转换为WAV格式的字节流 - 完整迁移自opus_to_wav"""
+        try:
+            import opuslib_next
+            import io
+            import wave
+            
+            # Opus解码器配置
+            sample_rate = 16000
+            channels = 1
+            
+            # 创建Opus解码器
+            decoder = opuslib_next.Decoder(sample_rate, channels)
+            
+            # 解码Opus数据
+            pcm_data = decoder.decode(opus_data, frame_size=960)
+            
+            # 创建WAV文件
+            wav_buffer = io.BytesIO()
+            with wave.open(wav_buffer, 'wb') as wav_file:
+                wav_file.setnchannels(channels)
+                wav_file.setsampwidth(2)  # 16-bit
+                wav_file.setframerate(sample_rate)
+                wav_file.writeframes(pcm_data)
+            
+            return wav_buffer.getvalue()
+            
+        except Exception as e:
+            logger.error(f"Opus转WAV失败: {e}")
+            return b''
+    
+    def cleanup_session(self, context: SessionContext):
+        """清理会话上报资源"""
+        # 停止上报线程
+        if context.report_thread and context.report_thread.is_alive():
+            context.stop_event.set()
+            context.report_thread.join(timeout=5)
+        
+        # 清理上报队列
+        try:
+            while not context.report_queue.empty():
+                context.report_queue.get_nowait()
+        except queue.Empty:
+            pass
+        
+        logger.info(f"上报资源清理完成: {context.session_id}")
+

main/xiaozhi-server/core/processors/server_processor.py

@@ -0,0 +1,151 @@
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class ServerProcessor(MessageProcessor):
+    """服务器消息处理器：完整迁移serverMessageHandler.py的所有功能"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理server类型的消息"""
+        if isinstance(message, str):
+            try:
+                msg_json = json.loads(message)
+                if isinstance(msg_json, dict) and msg_json.get("type") == "server":
+                    await self.handle_server_message(context, transport, msg_json)
+                    return True
+            except json.JSONDecodeError:
+                pass
+        return False
+    
+    async def handle_server_message(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理server消息 - 完整迁移自serverMessageHandler.py"""
+        # 如果配置是从API读取的，则需要验证secret
+        if not context.read_config_from_api:
+            return
+            
+        # 获取post请求的secret
+        post_secret = msg_json.get("content", {}).get("secret", "")
+        secret = context.config.get("manager-api", {}).get("secret", "")
+        
+        # 如果secret不匹配，则返回
+        if post_secret != secret:
+            await self._send_error_response(
+                transport, 
+                context.session_id,
+                "服务器密钥验证失败"
+            )
+            return
+        
+        # 处理不同的action
+        action = msg_json.get("action")
+        
+        if action == "update_config":
+            await self._handle_update_config(context, transport, msg_json)
+        elif action == "restart":
+            await self._handle_restart(context, transport, msg_json)
+        else:
+            await self._send_error_response(
+                transport,
+                context.session_id,
+                f"未知的服务器操作: {action}"
+            )
+    
+    async def _handle_update_config(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理配置更新 - 完整迁移自update_config逻辑"""
+        try:
+            # 检查是否有服务器实例
+            if not context.server:
+                await self._send_error_response(
+                    transport,
+                    context.session_id,
+                    "无法获取服务器实例",
+                    {"action": "update_config"}
+                )
+                return
+
+            # 更新WebSocketServer的配置
+            if not await context.server.update_config():
+                await self._send_error_response(
+                    transport,
+                    context.session_id,
+                    "更新服务器配置失败",
+                    {"action": "update_config"}
+                )
+                return
+
+            # 发送成功响应
+            await self._send_success_response(
+                transport,
+                context.session_id,
+                "配置更新成功",
+                {"action": "update_config"}
+            )
+            
+        except Exception as e:
+            logger.error(f"更新配置失败: {str(e)}")
+            await self._send_error_response(
+                transport,
+                context.session_id,
+                f"更新配置失败: {str(e)}",
+                {"action": "update_config"}
+            )
+    
+    async def _handle_restart(self, context: SessionContext, transport: TransportInterface, msg_json: dict):
+        """处理服务器重启 - 完整迁移自handle_restart逻辑"""
+        try:
+            # 这里应该调用context的handle_restart方法
+            if hasattr(context, 'handle_restart'):
+                await context.handle_restart(msg_json)
+            else:
+                logger.warning("SessionContext没有handle_restart方法")
+                await self._send_error_response(
+                    transport,
+                    context.session_id,
+                    "重启功能暂不可用",
+                    {"action": "restart"}
+                )
+        except Exception as e:
+            logger.error(f"处理重启请求失败: {str(e)}")
+            await self._send_error_response(
+                transport,
+                context.session_id,
+                f"重启失败: {str(e)}",
+                {"action": "restart"}
+            )
+    
+    async def _send_success_response(self, transport: TransportInterface, session_id: str, 
+                                   message: str, content: dict = None):
+        """发送成功响应"""
+        response = {
+            "type": "server",
+            "status": "success",
+            "message": message,
+            "session_id": session_id
+        }
+        if content:
+            response["content"] = content
+            
+        await transport.send(json.dumps(response))
+        logger.info(f"服务器操作成功: {message}")
+    
+    async def _send_error_response(self, transport: TransportInterface, session_id: str,
+                                 message: str, content: dict = None):
+        """发送错误响应"""
+        response = {
+            "type": "server",
+            "status": "error",
+            "message": message,
+            "session_id": session_id
+        }
+        if content:
+            response["content"] = content
+            
+        await transport.send(json.dumps(response))
+        logger.error(f"服务器操作失败: {message}")
+

main/xiaozhi-server/core/processors/text_processor.py

@@ -0,0 +1,55 @@
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class TextProcessor(MessageProcessor):
+    """
+    纯文本消息处理器：处理非JSON格式的文本消息
+    这是新架构中缺失的重要组件，用于处理直接发送的文本聊天内容
+    """
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """处理纯文本消息"""
+        if isinstance(message, str):
+            try:
+                # 尝试解析为JSON，如果成功则不是纯文本消息
+                json.loads(message)
+                return False  # JSON消息由其他processor处理
+            except json.JSONDecodeError:
+                # 确实是纯文本消息，进行聊天处理
+                await self.handle_text_message(context, transport, message)
+                return True
+        return False
+    
+    async def handle_text_message(self, context: SessionContext, transport: TransportInterface, text: str):
+        """处理纯文本消息 - 直接调用ChatProcessor进行聊天"""
+        try:
+            # 记录收到纯文本消息
+            logger.info(f"收到纯文本消息: {text[:100]}...")
+            
+            # 使用ChatProcessor处理聊天
+            from core.processors.chat_processor import ChatProcessor
+            chat_processor = ChatProcessor()
+            await chat_processor.handle_chat(context, transport, text)
+            
+        except Exception as e:
+            logger.error(f"处理纯文本消息失败: {e}")
+            # 发送错误响应
+            await self._send_error_response(transport, "文本处理失败，请重试")
+    
+    async def _send_error_response(self, transport: TransportInterface, error_message: str):
+        """发送错误响应"""
+        try:
+            await transport.send(json.dumps({
+                "type": "error",
+                "message": error_message
+            }))
+        except Exception as e:
+            logger.error(f"发送错误响应失败: {e}")
+

main/xiaozhi-server/core/processors/timeout_processor.py

@@ -0,0 +1,41 @@
+import json
+from typing import Any
+from core.pipeline.message_pipeline import MessageProcessor
+from core.context.session_context import SessionContext
+from core.transport.transport_interface import TransportInterface
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class TimeoutProcessor(MessageProcessor):
+    """超时检查处理器：检查会话是否超时"""
+    
+    async def process(self, context: SessionContext, transport: TransportInterface, message: Any) -> bool:
+        """检查会话超时"""
+        # 更新活动时间（在其他处理器中已更新，这里只检查）
+        
+        # 获取超时配置
+        timeout_seconds = context.config.get("close_connection_no_voice_time", 120)
+        
+        # 检查是否超时
+        if context.is_timeout(timeout_seconds):
+            logger.info(f"会话超时，准备关闭连接: {context.session_id}")
+            
+            # 发送超时通知
+            timeout_msg = {
+                "type": "timeout",
+                "message": "连接超时，即将关闭",
+                "session_id": context.session_id
+            }
+            
+            try:
+                await transport.send(json.dumps(timeout_msg))
+                await transport.close()
+            except Exception as e:
+                logger.error(f"发送超时消息失败: {e}")
+            
+            return True  # 消息已处理，停止后续处理
+        
+        return False  # 未超时，继续处理
+

main/xiaozhi-server/core/protocols/mqtt_connection.py

@@ -0,0 +1,279 @@
+import asyncio
+import json
+import time
+import uuid
+from typing import Dict, Any, Optional, Callable
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+class MQTTConnection:
+    """
+    MQTT连接处理类：管理单个MQTT客户端连接
+    处理MQTT协议消息和会话管理
+    """
+    
+    def __init__(self, socket, connection_id: int, mqtt_server):
+        self.socket = socket
+        self.connection_id = connection_id
+        self.mqtt_server = mqtt_server
+        
+        # 连接信息
+        self.client_id = None
+        self.device_id = None
+        self.username = None
+        self.password = None
+        self.session_id = None
+        
+        # 协议状态
+        self.is_connected_flag = False
+        self.keep_alive_interval = 0
+        self.last_activity = time.time()
+        
+        # 消息处理
+        self.message_callback = None
+        self.reply_topic = None
+        
+        # UDP相关
+        self.udp_config = None
+        
+        # 任务管理
+        self.keep_alive_task = None
+        self._closed = False
+        
+        # 创建MQTT协议处理器
+        from core.protocols.mqtt_protocol import MQTTProtocol
+        self.protocol = MQTTProtocol(socket)
+        self._setup_protocol_handlers()
+    
+    def _setup_protocol_handlers(self):
+        """设置协议事件处理"""
+        self.protocol.on('connect', self._handle_connect)
+        self.protocol.on('publish', self._handle_publish)
+        self.protocol.on('subscribe', self._handle_subscribe)
+        self.protocol.on('disconnect', self._handle_disconnect)
+        self.protocol.on('close', self._handle_close)
+        self.protocol.on('error', self._handle_error)
+    
+    async def _handle_connect(self, connect_data: Dict[str, Any]):
+        """处理CONNECT消息"""
+        try:
+            self.client_id = connect_data['clientId']
+            self.username = connect_data.get('username')
+            self.password = connect_data.get('password')
+            self.keep_alive_interval = connect_data.get('keepAlive', 0) * 1000  # 转换为毫秒
+            
+            logger.info(f"MQTT客户端连接: {self.client_id}")
+            
+            # 解析客户端ID获取设备信息
+            if not self._parse_client_id():
+                await self.protocol.send_connack(1)  # 连接被拒绝
+                await self.close()
+                return
+            
+            # 生成会话ID
+            self.session_id = str(uuid.uuid4())
+            
+            # 设置回复主题
+            self.reply_topic = f"devices/p2p/{self.device_id.replace(':', '_')}"
+            
+            # 发送连接确认
+            await self.protocol.send_connack(0)  # 连接接受
+            self.is_connected_flag = True
+            
+            # 启动心跳检查
+            if self.keep_alive_interval > 0:
+                self.keep_alive_task = asyncio.create_task(self._keep_alive_check())
+            
+            # 通知服务器新连接
+            await self.mqtt_server.on_client_connected(self)
+            
+        except Exception as e:
+            logger.error(f"处理CONNECT消息失败: {e}")
+            await self.close()
+    
+    def _parse_client_id(self) -> bool:
+        """解析客户端ID获取设备信息"""
+        try:
+            # 支持格式: GID_test@@@mac_address@@@uuid 或 GID_test@@@mac_address
+            parts = self.client_id.split('@@@')
+            
+            if len(parts) >= 2:
+                self.group_id = parts[0]
+                # 将下划线替换为冒号格式的MAC地址
+                self.device_id = parts[1].replace('_', ':')
+                
+                if len(parts) >= 3:
+                    self.uuid = parts[2]
+                
+                return True
+            else:
+                logger.error(f"无效的客户端ID格式: {self.client_id}")
+                return False
+                
+        except Exception as e:
+            logger.error(f"解析客户端ID失败: {e}")
+            return False
+    
+    async def _handle_publish(self, publish_data: Dict[str, Any]):
+        """处理PUBLISH消息"""
+        try:
+            topic = publish_data['topic']
+            payload = publish_data['payload']
+            
+            logger.debug(f"收到MQTT发布消息: topic={topic}, payload={payload}")
+            
+            # 更新活动时间
+            self.last_activity = time.time()
+            
+            # 解析JSON消息
+            try:
+                message_data = json.loads(payload)
+                
+                # 处理不同类型的消息
+                if message_data.get('type') == 'hello':
+                    await self._handle_hello_message(message_data)
+                else:
+                    # 其他消息通过回调处理
+                    if self.message_callback:
+                        self.message_callback(topic, payload)
+                        
+            except json.JSONDecodeError:
+                logger.error(f"MQTT消息JSON解析失败: {payload}")
+                
+        except Exception as e:
+            logger.error(f"处理PUBLISH消息失败: {e}")
+    
+    async def _handle_hello_message(self, message_data: Dict[str, Any]):
+        """处理hello消息，初始化UDP配置"""
+        try:
+            # 生成UDP加密配置
+            import os
+            
+            self.udp_config = {
+                'key': os.urandom(16),
+                'encryption': 'aes-128-ctr',
+                'server': self.mqtt_server.public_ip,
+                'port': self.mqtt_server.udp_port
+            }
+            
+            # 构造hello回复
+            hello_reply = {
+                'type': 'hello',
+                'version': message_data.get('version', 3),
+                'session_id': self.session_id,
+                'transport': 'udp',
+                'udp': {
+                    'server': self.udp_config['server'],
+                    'port': self.udp_config['port'],
+                    'encryption': self.udp_config['encryption'],
+                    'key': self.udp_config['key'].hex(),
+                    'nonce': '00' * 16  # 临时nonce
+                },
+                'audio_params': message_data.get('audio_params', {})
+            }
+            
+            # 发送回复
+            await self.send_message(self.reply_topic, json.dumps(hello_reply))
+            
+            logger.info(f"MQTT Hello消息处理完成: {self.client_id}")
+            
+        except Exception as e:
+            logger.error(f"处理hello消息失败: {e}")
+    
+    async def _handle_subscribe(self, subscribe_data: Dict[str, Any]):
+        """处理SUBSCRIBE消息"""
+        try:
+            topic = subscribe_data['topic']
+            packet_id = subscribe_data['packetId']
+            
+            logger.debug(f"客户端订阅主题: {topic}")
+            
+            # 发送订阅确认
+            await self.protocol.send_suback(packet_id, 0)
+            
+        except Exception as e:
+            logger.error(f"处理SUBSCRIBE消息失败: {e}")
+    
+    async def _handle_disconnect(self):
+        """处理DISCONNECT消息"""
+        logger.info(f"客户端主动断开连接: {self.client_id}")
+        await self.close()
+    
+    async def _handle_close(self):
+        """处理连接关闭"""
+        logger.info(f"MQTT连接关闭: {self.client_id}")
+        await self.close()
+    
+    async def _handle_error(self, error):
+        """处理连接错误"""
+        logger.error(f"MQTT连接错误: {self.client_id}, error: {error}")
+        await self.close()
+    
+    async def _keep_alive_check(self):
+        """心跳检查任务"""
+        try:
+            while self.is_connected_flag and not self._closed:
+                await asyncio.sleep(self.keep_alive_interval / 1000 / 2)  # 检查间隔为心跳间隔的一半
+                
+                current_time = time.time()
+                if current_time - self.last_activity > self.keep_alive_interval / 1000 * 1.5:
+                    logger.info(f"MQTT客户端心跳超时: {self.client_id}")
+                    await self.close()
+                    break
+                    
+        except asyncio.CancelledError:
+            pass
+        except Exception as e:
+            logger.error(f"心跳检查任务出错: {e}")
+    
+    def set_message_callback(self, callback: Callable[[str, str], None]):
+        """设置消息接收回调"""
+        self.message_callback = callback
+    
+    async def send_message(self, topic: str, payload: str):
+        """发送MQTT消息"""
+        if self._closed or not self.is_connected_flag:
+            return
+            
+        try:
+            await self.protocol.send_publish(topic, payload, qos=0)
+            logger.debug(f"发送MQTT消息: topic={topic}, payload={payload}")
+            
+        except Exception as e:
+            logger.error(f"发送MQTT消息失败: {e}")
+    
+    def is_connected(self) -> bool:
+        """检查连接状态"""
+        return self.is_connected_flag and not self._closed
+    
+    async def close(self):
+        """关闭连接"""
+        if self._closed:
+            return
+            
+        self._closed = True
+        self.is_connected_flag = False
+        
+        # 取消心跳检查任务
+        if self.keep_alive_task and not self.keep_alive_task.done():
+            self.keep_alive_task.cancel()
+            try:
+                await self.keep_alive_task
+            except asyncio.CancelledError:
+                pass
+        
+        # 通知服务器连接关闭
+        try:
+            await self.mqtt_server.on_client_disconnected(self)
+        except Exception as e:
+            logger.error(f"通知服务器连接关闭失败: {e}")
+        
+        # 关闭协议处理器
+        try:
+            await self.protocol.close()
+        except Exception as e:
+            logger.error(f"关闭MQTT协议处理器失败: {e}")
+        
+        logger.info(f"MQTT连接已关闭: {self.client_id}")

main/xiaozhi-server/core/protocols/mqtt_protocol.py

@@ -0,0 +1,433 @@
+import asyncio
+from typing import Dict, Any, Callable
+from config.logger import setup_logging
+
+logger = setup_logging()
+
+
+# MQTT 固定头部的类型
+class PacketType:
+    CONNECT = 1
+    CONNACK = 2
+    PUBLISH = 3
+    SUBSCRIBE = 8
+    SUBACK = 9
+    PINGREQ = 12
+    PINGRESP = 13
+    DISCONNECT = 14
+
+
+class MQTTProtocol:
+    """
+    MQTT协议处理器：负责MQTT协议的解析和封装
+    """
+    
+    def __init__(self, socket):
+        self.socket = socket
+        self.buffer = b''
+        self.event_handlers = {}
+        self.is_connected = False
+        self.keep_alive_interval = 0
+        self.last_activity = 0
+        
+        # 启动消息处理任务
+        self._processing_task = asyncio.create_task(self._process_messages())
+    
+    def on(self, event: str, handler: Callable):
+        """注册事件处理器"""
+        self.event_handlers[event] = handler
+    
+    def emit(self, event: str, *args, **kwargs):
+        """触发事件"""
+        handler = self.event_handlers.get(event)
+        if handler:
+            if asyncio.iscoroutinefunction(handler):
+                asyncio.create_task(handler(*args, **kwargs))
+            else:
+                handler(*args, **kwargs)
+    
+    async def _process_messages(self):
+        """处理消息的主循环"""
+        try:
+            while True:
+                # 从socket读取数据
+                data = await self._read_socket()
+                if not data:
+                    break
+                
+                # 添加到缓冲区
+                self.buffer += data
+                
+                # 处理缓冲区中的消息
+                await self._process_buffer()
+                
+        except asyncio.CancelledError:
+            pass
+        except Exception as e:
+            logger.error(f"MQTT消息处理循环出错: {e}")
+            self.emit('error', e)
+        finally:
+            self.emit('close')
+    
+    async def _read_socket(self) -> bytes:
+        """从socket读取数据"""
+        try:
+            # 使用asyncio的socket读取
+            loop = asyncio.get_event_loop()
+            data = await loop.sock_recv(self.socket, 4096)
+            return data
+        except Exception as e:
+            logger.error(f"读取socket数据失败: {e}")
+            return b''
+    
+    async def _process_buffer(self):
+        """处理缓冲区中的消息"""
+        while len(self.buffer) >= 2:  # 至少需要2字节开始解析
+            try:
+                # 解析消息
+                message_length, message = self._parse_message()
+                if message_length == 0:
+                    break  # 消息不完整，等待更多数据
+                
+                # 从缓冲区移除已处理的消息
+                self.buffer = self.buffer[message_length:]
+                
+                # 处理消息
+                await self._handle_message(message)
+                
+            except Exception as e:
+                logger.error(f"处理MQTT消息失败: {e}")
+                self.emit('protocolError', e)
+                break
+    
+    def _parse_message(self) -> tuple[int, Dict[str, Any]]:
+        """解析MQTT消息"""
+        if len(self.buffer) < 2:
+            return 0, {}
+        
+        # 获取消息类型
+        first_byte = self.buffer[0]
+        packet_type = (first_byte >> 4)
+        
+        # 解析剩余长度
+        remaining_length, bytes_read = self._decode_remaining_length()
+        if remaining_length == -1:
+            return 0, {}  # 长度解析失败，等待更多数据
+        
+        # 计算完整消息长度
+        total_length = 1 + bytes_read + remaining_length
+        
+        if len(self.buffer) < total_length:
+            return 0, {}  # 消息不完整
+        
+        # 提取消息数据
+        message_data = self.buffer[:total_length]
+        
+        # 根据消息类型解析
+        if packet_type == PacketType.CONNECT:
+            message = self._parse_connect(message_data)
+        elif packet_type == PacketType.PUBLISH:
+            message = self._parse_publish(message_data)
+        elif packet_type == PacketType.SUBSCRIBE:
+            message = self._parse_subscribe(message_data)
+        elif packet_type == PacketType.PINGREQ:
+            message = {'type': 'pingreq'}
+        elif packet_type == PacketType.DISCONNECT:
+            message = {'type': 'disconnect'}
+        else:
+            logger.warning(f"未处理的MQTT消息类型: {packet_type}")
+            message = {'type': 'unknown', 'packet_type': packet_type}
+        
+        return total_length, message
+    
+    def _decode_remaining_length(self) -> tuple[int, int]:
+        """解码剩余长度字段"""
+        multiplier = 1
+        value = 0
+        bytes_read = 0
+        
+        while bytes_read < 4 and bytes_read + 1 < len(self.buffer):
+            digit = self.buffer[bytes_read + 1]
+            bytes_read += 1
+            
+            value += (digit & 127) * multiplier
+            multiplier *= 128
+            
+            if (digit & 128) == 0:
+                break
+        else:
+            if bytes_read >= 4:
+                return -1, 0  # 长度字段过长
+            return -1, 0  # 数据不完整
+        
+        return value, bytes_read
+    
+    def _encode_remaining_length(self, length: int) -> bytes:
+        """编码剩余长度字段"""
+        result = bytearray()
+        
+        while True:
+            digit = length % 128
+            length = length // 128
+            
+            if length > 0:
+                digit |= 0x80
+            
+            result.append(digit)
+            
+            if length == 0:
+                break
+        
+        return bytes(result)
+    
+    def _parse_connect(self, message_data: bytes) -> Dict[str, Any]:
+        """解析CONNECT消息"""
+        try:
+            # 跳过固定头部和剩余长度
+            _, bytes_read = self._decode_remaining_length()
+            pos = 1 + bytes_read
+            
+            # 协议名长度
+            protocol_length = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            
+            # 协议名
+            protocol = message_data[pos:pos+protocol_length].decode('utf-8')
+            pos += protocol_length
+            
+            # 协议级别
+            protocol_level = message_data[pos]
+            pos += 1
+            
+            # 连接标志
+            connect_flags = message_data[pos]
+            has_username = (connect_flags & 0x80) != 0
+            has_password = (connect_flags & 0x40) != 0
+            pos += 1
+            
+            # 保持连接时间
+            keep_alive = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            
+            # 客户端ID
+            client_id_length = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            client_id = message_data[pos:pos+client_id_length].decode('utf-8')
+            pos += client_id_length
+            
+            # 用户名（如果存在）
+            username = ''
+            if has_username:
+                username_length = int.from_bytes(message_data[pos:pos+2], 'big')
+                pos += 2
+                username = message_data[pos:pos+username_length].decode('utf-8')
+                pos += username_length
+            
+            # 密码（如果存在）
+            password = ''
+            if has_password:
+                password_length = int.from_bytes(message_data[pos:pos+2], 'big')
+                pos += 2
+                password = message_data[pos:pos+password_length].decode('utf-8')
+                pos += password_length
+            
+            return {
+                'type': 'connect',
+                'protocol': protocol,
+                'protocolLevel': protocol_level,
+                'clientId': client_id,
+                'keepAlive': keep_alive,
+                'username': username,
+                'password': password
+            }
+            
+        except Exception as e:
+            logger.error(f"解析CONNECT消息失败: {e}")
+            raise
+    
+    def _parse_publish(self, message_data: bytes) -> Dict[str, Any]:
+        """解析PUBLISH消息"""
+        try:
+            # 获取QoS等标志
+            first_byte = message_data[0]
+            qos = (first_byte & 0x06) >> 1
+            dup = (first_byte & 0x08) != 0
+            retain = (first_byte & 0x01) != 0
+            
+            # 跳过固定头部和剩余长度
+            _, bytes_read = self._decode_remaining_length()
+            pos = 1 + bytes_read
+            
+            # 主题长度
+            topic_length = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            
+            # 主题
+            topic = message_data[pos:pos+topic_length].decode('utf-8')
+            pos += topic_length
+            
+            # 消息ID（QoS > 0时存在）
+            packet_id = None
+            if qos > 0:
+                packet_id = int.from_bytes(message_data[pos:pos+2], 'big')
+                pos += 2
+            
+            # 有效载荷
+            payload = message_data[pos:].decode('utf-8')
+            
+            return {
+                'type': 'publish',
+                'topic': topic,
+                'payload': payload,
+                'qos': qos,
+                'dup': dup,
+                'retain': retain,
+                'packetId': packet_id
+            }
+            
+        except Exception as e:
+            logger.error(f"解析PUBLISH消息失败: {e}")
+            raise
+    
+    def _parse_subscribe(self, message_data: bytes) -> Dict[str, Any]:
+        """解析SUBSCRIBE消息"""
+        try:
+            # 跳过固定头部和剩余长度
+            _, bytes_read = self._decode_remaining_length()
+            pos = 1 + bytes_read
+            
+            # 消息ID
+            packet_id = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            
+            # 主题长度
+            topic_length = int.from_bytes(message_data[pos:pos+2], 'big')
+            pos += 2
+            
+            # 主题
+            topic = message_data[pos:pos+topic_length].decode('utf-8')
+            pos += topic_length
+            
+            # QoS
+            qos = message_data[pos]
+            
+            return {
+                'type': 'subscribe',
+                'packetId': packet_id,
+                'topic': topic,
+                'qos': qos
+            }
+            
+        except Exception as e:
+            logger.error(f"解析SUBSCRIBE消息失败: {e}")
+            raise
+    
+    async def _handle_message(self, message: Dict[str, Any]):
+        """处理解析后的消息"""
+        message_type = message.get('type')
+        
+        if message_type == 'connect':
+            self.keep_alive_interval = message.get('keepAlive', 0)
+            self.is_connected = True
+            self.emit('connect', message)
+        elif message_type == 'publish':
+            self.emit('publish', message)
+        elif message_type == 'subscribe':
+            self.emit('subscribe', message)
+        elif message_type == 'pingreq':
+            await self.send_pingresp()
+        elif message_type == 'disconnect':
+            self.emit('disconnect')
+        else:
+            logger.warning(f"未处理的消息类型: {message_type}")
+    
+    async def send_connack(self, return_code: int = 0, session_present: bool = False):
+        """发送CONNACK消息"""
+        packet = bytearray([
+            PacketType.CONNACK << 4,  # 固定头部
+            2,  # 剩余长度
+            1 if session_present else 0,  # 连接确认标志
+            return_code  # 返回码
+        ])
+        
+        await self._send_packet(packet)
+    
+    async def send_publish(self, topic: str, payload: str, qos: int = 0, 
+                          dup: bool = False, retain: bool = False, packet_id: int = None):
+        """发送PUBLISH消息"""
+        # 构造固定头部
+        first_byte = PacketType.PUBLISH << 4
+        if dup:
+            first_byte |= 0x08
+        if qos > 0:
+            first_byte |= (qos << 1)
+        if retain:
+            first_byte |= 0x01
+        
+        # 构造可变头部和载荷
+        topic_bytes = topic.encode('utf-8')
+        payload_bytes = payload.encode('utf-8')
+        
+        variable_header = bytearray()
+        variable_header.extend(len(topic_bytes).to_bytes(2, 'big'))
+        variable_header.extend(topic_bytes)
+        
+        if qos > 0 and packet_id is not None:
+            variable_header.extend(packet_id.to_bytes(2, 'big'))
+        
+        # 计算剩余长度
+        remaining_length = len(variable_header) + len(payload_bytes)
+        remaining_length_bytes = self._encode_remaining_length(remaining_length)
+        
+        # 构造完整消息
+        packet = bytearray([first_byte])
+        packet.extend(remaining_length_bytes)
+        packet.extend(variable_header)
+        packet.extend(payload_bytes)
+        
+        await self._send_packet(packet)
+    
+    async def send_suback(self, packet_id: int, return_code: int = 0):
+        """发送SUBACK消息"""
+        packet = bytearray([
+            PacketType.SUBACK << 4,  # 固定头部
+            3,  # 剩余长度
+            packet_id >> 8,  # 消息ID高字节
+            packet_id & 0xFF,  # 消息ID低字节
+            return_code  # 返回码
+        ])
+        
+        await self._send_packet(packet)
+    
+    async def send_pingresp(self):
+        """发送PINGRESP消息"""
+        packet = bytearray([
+            PacketType.PINGRESP << 4,  # 固定头部
+            0  # 剩余长度
+        ])
+        
+        await self._send_packet(packet)
+    
+    async def _send_packet(self, packet: bytearray):
+        """发送数据包"""
+        try:
+            loop = asyncio.get_event_loop()
+            await loop.sock_sendall(self.socket, bytes(packet))
+        except Exception as e:
+            logger.error(f"发送MQTT数据包失败: {e}")
+            raise
+    
+    async def close(self):
+        """关闭协议处理器"""
+        if hasattr(self, '_processing_task') and not self._processing_task.done():
+            self._processing_task.cancel()
+            try:
+                await self._processing_task
+            except asyncio.CancelledError:
+                pass
+        
+        try:
+            self.socket.close()
+        except Exception as e:
+            logger.error(f"关闭socket失败: {e}")
+

main/xiaozhi-server/core/providers/asr/base.py

@@ -12,15 +12,80 @@ import opuslib_next
 from abc import ABC, abstractmethod
 from config.logger import setup_logging
 from typing import Optional, Tuple, List
-from core.handle.receiveAudioHandle import startToChat
+# from core.handle.receiveAudioHandle import startToChat  # 旧的handler
-from core.handle.reportHandle import enqueue_asr_report
+# from core.handle.reportHandle import enqueue_asr_report  # 旧的handler
+# from core.handle.receiveAudioHandle import handleAudioMessage  # 旧的handler
+# 使用新的processor替代
 from core.utils.util import remove_punctuation_and_length
-from core.handle.receiveAudioHandle import handleAudioMessage
 
 TAG = __name__
 logger = setup_logging()
 
 
+async def handleAudioMessage(conn, message):
+    """兼容函数：使用新的processor处理音频消息"""
+    try:
+        # 获取transport接口
+        transport = getattr(conn, 'transport', None)
+        if not transport:
+            logger.error("SessionContext中没有transport接口")
+            return
+            
+        # 使用AudioReceiveProcessor处理音频消息
+        from core.processors.audio_receive_processor import AudioReceiveProcessor
+        processor = AudioReceiveProcessor()
+        
+        # 处理音频消息
+        await processor.handle_audio_message(conn, transport, message)
+        
+    except Exception as e:
+        logger.error(f"处理音频消息失败: {e}")
+        import traceback
+        traceback.print_exc()
+
+
+async def startToChat(conn, text):
+    """兼容函数：使用新的processor开始聊天"""
+    try:
+        # 获取transport接口
+        transport = getattr(conn, 'transport', None)
+        if not transport:
+            logger.error("SessionContext中没有transport接口")
+            return
+            
+        # 使用ChatProcessor处理聊天
+        from core.processors.chat_processor import ChatProcessor
+        processor = ChatProcessor()
+        
+        # 开始聊天
+        await processor.handle_chat(conn, transport, text)
+        
+    except Exception as e:
+        logger.error(f"开始聊天失败: {e}")
+        import traceback
+        traceback.print_exc()
+
+
+def enqueue_asr_report(conn, text, audio_data):
+    """兼容函数：使用新的processor处理ASR报告"""
+    try:
+        # 获取transport接口
+        transport = getattr(conn, 'transport', None)
+        if not transport:
+            logger.error("SessionContext中没有transport接口")
+            return
+            
+        # 使用ReportProcessor处理报告
+        from core.processors.report_processor import ReportProcessor
+        processor = ReportProcessor()
+        
+        # 处理ASR报告
+        processor.enqueue_asr_report(conn, text, audio_data)
+        
+    except Exception as e:
+        logger.error(f"ASR报告处理失败: {e}")
+
+
 class ASRProviderBase(ABC):
     def __init__(self):
         pass

main/xiaozhi-server/core/providers/asr/doubao_stream.py

@@ -47,6 +47,8 @@ class ASRProvider(ASRProviderBase):
         self.channel = config.get("channel", 1)
         self.auth_method = config.get("auth_method", "token")
         self.secret = config.get("secret", "access_secret")
+        end_window_size = config.get("end_window_size")
+        self.end_window_size = int(end_window_size) if end_window_size else 200
 
     async def open_audio_channels(self, conn):
         await super().open_audio_channels(conn)
@@ -291,7 +293,7 @@ class ASRProvider(ASRProviderBase):
                 "sequence": 1,
                 "boosting_table_name": self.boosting_table_name,
                 "correct_table_name": self.correct_table_name,
-                "end_window_size": 200,
+                "end_window_size": self.end_window_size,
             },
             "audio": {
                 "format": self.format,

main/xiaozhi-server/core/providers/asr/shared_asr_manager.py

@@ -0,0 +1,407 @@
+"""
+SharedASRManager: 全局 ASR 管理器
+实现单例模型 + 单推理执行器 + 队列限流。
+单例的原因是：推理是 CPU/GPU-bound，不是 I/O-bound，多实例不仅会占用内存，还会降低吞吐能力
+"""
+
+import asyncio
+from concurrent.futures import ThreadPoolExecutor
+from typing import Dict, Any, Optional, Tuple, List
+from config.logger import setup_logging
+
+logger = setup_logging()
+TAG = __name__
+
+
+class SharedASRManager:
+    """
+    全局共享 ASR 管理器
+    """
+    
+    # 支持预加载的本地模型类型
+    LOCAL_MODEL_TYPES = [
+        "fun_local",           # FunASR 本地
+        "sherpa_onnx_local",   # Sherpa ONNX
+        "sense_voice"          # SenseVoice
+    ]
+    
+    def __init__(self, config: Dict[str, Any], asr_type: str = None):
+        """
+        初始化 ASR 管理器
+        Args:
+            config: 服务器配置
+            asr_type: ASR 类型（Optional，用于显式指定）
+        """
+        self.config = config
+        self.asr_type = asr_type
+        
+        # 模型实例（全局单例）
+        self.model_instance = None
+        
+        # 任务队列（限流）
+        queue_max_size = self._get_queue_max_size()
+        self.task_queue: asyncio.Queue = asyncio.Queue(maxsize=queue_max_size)
+        
+        # 推理锁（使得推理串行化）
+        self.inference_lock = asyncio.Lock()
+        # 线程池执行器，用于阻塞调用
+        self.executor: Optional[ThreadPoolExecutor] = None
+        
+        # 运行状态
+        self.running = False
+        self._inference_task: Optional[asyncio.Task] = None
+        self.is_local_model = self._check_local_model()
+        
+        logger.bind(tag=TAG).info(
+            f"SharedASRManager 初始化完成, "
+            f"类型: {self.asr_type}, "
+            f"本地模型: {self.is_local_model}, "
+            f"队列大小: {queue_max_size}"
+        )
+    
+    def _get_queue_max_size(self) -> int:
+        """获取队列最大大小"""
+        # 尝试从配置获取
+        selected_asr = self.config.get("selected_module", {}).get("ASR")
+        if selected_asr:
+            asr_config = self.config.get("ASR", {}).get(selected_asr, {})
+            return asr_config.get("queue_max_size", 100)
+        return 100
+    
+    def _check_local_model(self) -> bool:
+        """检查是否为本地模型"""
+        if self.asr_type:
+            return self.asr_type in self.LOCAL_MODEL_TYPES
+        
+        # 从配置推断
+        selected_asr = self.config.get("selected_module", {}).get("ASR")
+        if not selected_asr:
+            return False
+        
+        asr_config = self.config.get("ASR", {}).get(selected_asr, {})
+        asr_type = asr_config.get("type", selected_asr)
+        self.asr_type = asr_type
+        
+        return asr_type in self.LOCAL_MODEL_TYPES
+    
+    async def initialize(self):
+        """
+        初始化管理器
+        - 预加载模型
+        - 启动推理执行器
+        """
+        if not self.is_local_model:
+            logger.bind(tag=TAG).info("非本地模型，跳过预加载")
+            return
+        
+        if self.running:
+            logger.bind(tag=TAG).warning("管理器已在运行中")
+            return
+        
+        try:
+            logger.bind(tag=TAG).info(f"开始预加载 ASR 模型: {self.asr_type}")
+            
+            # 预加载模型
+            await self._preload_model()
+            
+            # 启动推理执行器
+            self.running = True
+            self._inference_task = asyncio.create_task(self._inference_loop())
+            
+            logger.bind(tag=TAG).info("ASR 模型预加载完成，推理执行器已启动")
+            
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"ASR 模型预加载失败: {e}")
+            raise
+    
+    async def _preload_model(self):
+        """在线程池中预加载模型"""
+        # 创建线程池
+        self.executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="asr_worker")
+        
+        loop = asyncio.get_event_loop()
+        self.model_instance = await loop.run_in_executor(
+            self.executor,
+            self._create_model_instance
+        )
+        
+        logger.bind(tag=TAG).info("模型实例创建完成")
+    
+    def _create_model_instance(self):
+        """
+        实际创建模型实例（在线程中执行）
+        
+        Returns:
+            ASR Provider 实例
+        """
+        from core.utils.modules_initialize import initialize_asr
+        
+        logger.bind(tag=TAG).info("正在创建 ASR 模型实例...")
+        instance = initialize_asr(self.config)
+        logger.bind(tag=TAG).info("ASR 模型实例创建成功")
+        
+        return instance
+    
+    async def submit_task(
+        self, 
+        opus_data: List[bytes], 
+        session_id: str, 
+        audio_format: str = "opus"
+    ) -> Tuple[Optional[str], Optional[str]]:
+        """
+        提交推理任务
+        
+        Args:
+            opus_data: 音频数据
+            session_id: 会话 ID
+            audio_format: 音频格式
+            
+        Returns:
+            (识别文本, 文件路径)
+            
+        Raises:
+            RuntimeError: 队列满或服务未运行
+        """
+        if not self.running:
+            raise RuntimeError("ASR 服务未运行")
+        
+        # 检查队列是否满（限流）
+        if self.task_queue.full():
+            queue_status = self.get_queue_status()
+            logger.bind(tag=TAG).warning(
+                f"ASR 队列已满: {queue_status}"
+            )
+            raise RuntimeError("ASR 服务繁忙，请稍后重试")
+        
+        # 创建 Future 用于返回结果
+        result_future: asyncio.Future = asyncio.Future()
+        
+        # 构造任务
+        task = {
+            'opus_data': opus_data,
+            'session_id': session_id,
+            'audio_format': audio_format,
+            'future': result_future
+        }
+        
+        # 放入队列
+        await self.task_queue.put(task)
+        
+        logger.bind(tag=TAG).debug(
+            f"任务已提交, session: {session_id}, "
+            f"队列大小: {self.task_queue.qsize()}"
+        )
+        
+        # 等待结果
+        return await result_future
+    
+    async def _inference_loop(self):
+        """
+        单个推理执行器循环
+        
+        核心原则：
+        - 只有一个执行器
+        - 串行处理任务
+        - 带超时的队列获取，支持优雅退出
+        """
+        logger.bind(tag=TAG).info("推理执行器启动")
+        
+        while self.running:
+            task = None
+            try:
+                # 带超时的队列获取，避免关闭时卡住
+                try:
+                    task = await asyncio.wait_for(
+                        self.task_queue.get(),
+                        timeout=1.0
+                    )
+                except asyncio.TimeoutError:
+                    # 超时后检查 running 状态，继续循环
+                    continue
+                
+                # 执行推理（加锁保证串行）
+                async with self.inference_lock:
+                    result = await self._run_inference(
+                        task['opus_data'],
+                        task['session_id'],
+                        task['audio_format']
+                    )
+                
+                # 返回结果
+                if not task['future'].done():
+                    task['future'].set_result(result)
+                
+                logger.bind(tag=TAG).debug(
+                    f"推理完成, session: {task['session_id']}"
+                )
+                
+            except asyncio.CancelledError:
+                logger.bind(tag=TAG).info("推理执行器被取消")
+                break
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"推理执行失败: {e}")
+                if task and 'future' in task and not task['future'].done():
+                    task['future'].set_exception(e)
+        
+        logger.bind(tag=TAG).info("推理执行器已停止")
+    
+    async def _run_inference(
+        self, 
+        opus_data: List[bytes], 
+        session_id: str, 
+        audio_format: str
+    ) -> Tuple[Optional[str], Optional[str]]:
+        """
+        执行实际推理（在线程池中）
+        
+        Args:
+            opus_data: 音频数据
+            session_id: 会话 ID
+            audio_format: 音频格式
+            
+        Returns:
+            (识别文本, 文件路径)
+        """
+        loop = asyncio.get_event_loop()
+        
+        # 在线程池中执行推理
+        result = await loop.run_in_executor(
+            self.executor,
+            lambda: self.model_instance.speech_to_text_sync(
+                opus_data, session_id, audio_format
+            ) if hasattr(self.model_instance, 'speech_to_text_sync')
+            else self._sync_wrapper(opus_data, session_id, audio_format)
+        )
+        
+        return result
+    
+    def _sync_wrapper(
+        self, 
+        opus_data: List[bytes], 
+        session_id: str, 
+        audio_format: str
+    ) -> Tuple[Optional[str], Optional[str]]:
+        """
+        同步包装器
+        处理 async speech_to_text 方法
+        """
+        import asyncio
+        
+        async def _call():
+            return await self.model_instance.speech_to_text(
+                opus_data, session_id, audio_format
+            )
+        
+        # 创建新的事件循环执行
+        loop = None
+        try:
+            loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
+            result = loop.run_until_complete(_call())
+            return result
+        finally:
+            if loop:
+                loop.close()
+    
+    async def shutdown(self):
+        """
+        优雅停机
+        
+        步骤：
+        1. 停止接收新任务
+        2. 等待当前任务完成（带超时）
+        3. 取消未完成的任务
+        4. 关闭线程池
+        """
+        if not self.running:
+            return
+        
+        logger.bind(tag=TAG).info("开始关闭 ASR 管理器...")
+        
+        # 停止接收新任务
+        self.running = False
+        
+        # 等待推理任务完成
+        if self._inference_task and not self._inference_task.done():
+            try:
+                # 最多等待 5 秒
+                await asyncio.wait_for(
+                    self._inference_task,
+                    timeout=5.0
+                )
+            except asyncio.TimeoutError:
+                logger.bind(tag=TAG).warning("推理任务超时，强制取消")
+                self._inference_task.cancel()
+                try:
+                    await self._inference_task
+                except asyncio.CancelledError:
+                    pass
+            except asyncio.CancelledError:
+                pass
+        
+        # 取消所有队列中未完成的任务
+        cancelled_count = 0
+        while not self.task_queue.empty():
+            try:
+                task = self.task_queue.get_nowait()
+                if not task['future'].done():
+                    task['future'].set_exception(
+                        RuntimeError("ASR 服务正在关闭")
+                    )
+                    cancelled_count += 1
+            except asyncio.QueueEmpty:
+                break
+        
+        if cancelled_count > 0:
+            logger.bind(tag=TAG).info(f"已取消 {cancelled_count} 个待处理任务")
+        
+        # 关闭线程池
+        if self.executor:
+            self.executor.shutdown(wait=False)
+            self.executor = None
+            logger.bind(tag=TAG).info("线程池已关闭")
+        
+        # 清理模型实例
+        self.model_instance = None
+        
+        logger.bind(tag=TAG).info("ASR 管理器已关闭")
+    
+    def get_queue_status(self) -> Dict[str, Any]:
+        """
+        获取队列状态（用于监控）
+        
+        Returns:
+            队列状态字典
+        """
+        queue_size = self.task_queue.qsize()
+        queue_max = self.task_queue.maxsize
+        
+        return {
+            'queue_size': queue_size,
+            'queue_max': queue_max,
+            'is_busy': queue_size > queue_max * 0.8,
+            'utilization': queue_size / queue_max if queue_max > 0 else 0,
+            'running': self.running
+        }
+    
+    def is_ready(self) -> bool:
+        """检查管理器是否就绪"""
+        return (
+            self.running and 
+            self.model_instance is not None and 
+            self.executor is not None
+        )
+    
+    @classmethod
+    def is_local_model_type(cls, asr_type: str) -> bool:
+        """
+        检查 ASR 类型是否为本地模型
+        
+        Args:
+            asr_type: ASR 类型
+            
+        Returns:
+            是否为本地模型
+        """
+        return asr_type in cls.LOCAL_MODEL_TYPES
+

main/xiaozhi-server/core/providers/asr/shared_asr_proxy.py

@@ -0,0 +1,114 @@
+"""
+SharedASRProxy: 共享 ASR 管理器的代理类
+
+功能：
+- 包装 SharedASRManager
+- 提供与原 ASR Provider 相同的接口
+- 处理队列满等异常情况
+"""
+
+from typing import List, Tuple, Optional, Dict, Any
+from core.providers.asr.base import ASRProviderBase
+from core.providers.asr.dto.dto import InterfaceType
+from config.logger import setup_logging
+
+logger = setup_logging()
+TAG = __name__
+
+
+class SharedASRProxy(ASRProviderBase):
+    """
+    共享 ASR 管理器的代理类
+    
+    该类提供与原 ASR Provider 相同的接口，
+    但实际推理工作由 SharedASRManager 完成。
+    """
+    
+    def __init__(self, manager):
+        """
+        初始化代理
+        
+        Args:
+            manager: SharedASRManager 实例
+        """
+        super().__init__()
+        self.manager = manager
+        
+        # 从共享管理器获取接口类型
+        if manager.model_instance and hasattr(manager.model_instance, 'interface_type'):
+            self.interface_type = manager.model_instance.interface_type
+        else:
+            self.interface_type = InterfaceType.LOCAL
+        
+        logger.bind(tag=TAG).info("SharedASRProxy 初始化完成")
+    
+    async def speech_to_text(
+        self, 
+        opus_data: List[bytes], 
+        session_id: str, 
+        audio_format: str = "opus"
+    ) -> Tuple[Optional[str], Optional[str]]:
+        """
+        语音转文本（通过共享管理器）
+        
+        Args:
+            opus_data: 音频数据（Opus 编码的字节列表）
+            session_id: 会话 ID
+            audio_format: 音频格式，默认 "opus"
+            
+        Returns:
+            Tuple[str, str]: (识别的文本, 音频文件路径)
+        """
+        try:
+            # 检查管理器状态
+            if not self.manager.is_ready():
+                logger.bind(tag=TAG).error("ASR 管理器未就绪")
+                return "", None
+            
+            # 提交任务到共享管理器
+            result = await self.manager.submit_task(
+                opus_data, 
+                session_id, 
+                audio_format
+            )
+            
+            return result
+            
+        except RuntimeError as e:
+            # 队列满或服务未运行
+            logger.bind(tag=TAG).warning(f"ASR 服务繁忙: {e}")
+            # 返回友好提示，而不是空字符串
+            return "服务繁忙，请稍后重试", None
+            
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"ASR 推理失败: {e}")
+            return "", None
+    
+    def get_queue_status(self) -> Dict[str, Any]:
+        """
+        获取队列状态
+        
+        Returns:
+            队列状态字典
+        """
+        return self.manager.get_queue_status()
+    
+    def is_ready(self) -> bool:
+        """
+        检查代理是否就绪
+        
+        Returns:
+            是否就绪
+        """
+        return self.manager.is_ready()
+    
+    async def close(self):
+        """
+        关闭代理
+        
+        注意：不关闭共享管理器，由服务器统一管理
+        """
+        logger.bind(tag=TAG).debug("SharedASRProxy 关闭")
+        # 代理不负责关闭共享管理器
+        pass
+

main/xiaozhi-server/core/providers/memory/base.py

@@ -14,7 +14,7 @@ class MemoryProviderBase(ABC):
         self.llm = llm
 
     @abstractmethod
-    async def save_memory(self, msgs):
+    async def save_memory(self, msgs, session_id=None):
         """Save a new memory for specific role and return memory ID"""
         print("this is base func", msgs)
 

main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py

@@ -28,7 +28,7 @@ class MemoryProvider(MemoryProviderBase):
             logger.bind(tag=TAG).error(f"详细错误: {traceback.format_exc()}")
             self.use_mem0 = False
 
-    async def save_memory(self, msgs):
+    async def save_memory(self, msgs, session_id=None):
         if not self.use_mem0:
             return None
         if len(msgs) < 2:
@@ -41,9 +41,7 @@ class MemoryProvider(MemoryProviderBase):
                 for message in msgs
                 if message.role != "system"
             ]
-            result = self.client.add(
+            result = self.client.add(messages, user_id=self.role_id)
-                messages, user_id=self.role_id
-            )
             logger.bind(tag=TAG).debug(f"Save memory result: {result}")
         except Exception as e:
             logger.bind(tag=TAG).error(f"保存记忆失败: {str(e)}")

main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py

@@ -4,7 +4,7 @@ import json
 import os
 import yaml
 from config.config_loader import get_project_dir
-from config.manage_api_client import save_mem_local_short
+from config.manage_api_client import generate_and_save_chat_summary
 import asyncio
 from core.utils.util import check_model_key
 
@@ -75,18 +75,6 @@ short_term_memory_prompt = """
 ```
 """
 
-short_term_memory_prompt_only_content = """
-你是一个经验丰富的记忆总结者，擅长将对话内容进行总结摘要，遵循以下规则：
-1、总结user的重要信息，以便在未来的对话中提供更个性化的服务
-2、不要重复总结，不要遗忘之前记忆，除非原来的记忆超过了1800字内，否则不要遗忘、不要压缩用户的历史记忆
-3、用户操控的设备音量、播放音乐、天气、退出、不想对话等和用户本身无关的内容，这些信息不需要加入到总结中
-4、聊天内容中的今天的日期时间、今天的天气情况与用户事件无关的数据，这些信息如果当成记忆存储会影响后序对话，这些信息不需要加入到总结中
-5、不要把设备操控的成果结果和失败结果加入到总结中，也不要把用户的一些废话加入到总结中
-6、不要为了总结而总结，如果用户的聊天没有意义，请返回原来的历史记录也是可以的
-7、只需要返回总结摘要，严格控制在1800字内
-8、不要包含代码、xml，不需要解释、注释和说明，保存记忆时仅从对话提取信息，不要混入示例内容
-"""
-
 
 def extract_json_data(json_code):
     start = json_code.find("```json")
@@ -144,7 +132,7 @@ class MemoryProvider(MemoryProviderBase):
         with open(self.memory_path, "w", encoding="utf-8") as f:
             yaml.dump(all_memory, f, allow_unicode=True)
 
-    async def save_memory(self, msgs):
+    async def save_memory(self, msgs, session_id=None):
         # 打印使用的模型信息
         model_info = getattr(self.llm, "model_name", str(self.llm.__class__.__name__))
         logger.bind(tag=TAG).debug(f"使用记忆保存模型: {model_info}")
@@ -188,20 +176,12 @@ class MemoryProvider(MemoryProviderBase):
             except Exception as e:
                 print("Error:", e)
         else:
-            result = self.llm.response_no_stream(
+            # 当save_to_file为False时，调用Java端的聊天记录总结接口
-                short_term_memory_prompt_only_content,
+            summary_id = session_id if session_id else self.role_id
-                msgStr,
+            await generate_and_save_chat_summary(summary_id)
-                max_tokens=2000,
+        logger.bind(tag=TAG).info(
-                temperature=0.2,
+            f"Save memory successful - Role: {self.role_id}, Session: {session_id}"
-            )
+        )
-            # 使用异步版本，需要在事件循环中运行
-            try:
-                loop = asyncio.get_running_loop()
-                loop.create_task(save_mem_local_short(self.role_id, result))
-            except RuntimeError:
-                # 如果没有运行中的事件循环，创建一个新的
-                asyncio.run(save_mem_local_short(self.role_id, result))
-        logger.bind(tag=TAG).info(f"Save memory successful - Role: {self.role_id}")
 
         return self.short_memory
 

main/xiaozhi-server/core/providers/memory/nomem/nomem.py

@@ -11,7 +11,7 @@ class MemoryProvider(MemoryProviderBase):
     def __init__(self, config, summary_memory=None):
         super().__init__(config)
 
-    async def save_memory(self, msgs):
+    async def save_memory(self, msgs, session_id=None):
         logger.bind(tag=TAG).debug("nomem mode: No memory saving is performed.")
         return None
 

main/xiaozhi-server/core/providers/tools/device_iot/iot_executor.py

@@ -127,7 +127,16 @@ class DeviceIoTExecutor(ToolExecutor):
                         send_message = json.dumps(
                             {"type": "iot", "commands": [command]}
                         )
-                        await self.conn.websocket.send(send_message)
+                        
+                        # 使用transport接口发送消息
+                        if hasattr(self.conn, 'transport') and self.conn.transport:
+                            await self.conn.transport.send(send_message)
+                        elif hasattr(self.conn, 'websocket') and self.conn.websocket:
+                            # 兼容旧版本
+                            logger.warning("未找到SessionContext的传输层接口, 回退使用旧版conn.websocket发送消息")
+                            await self.conn.websocket.send(send_message)
+                        else:
+                            raise AttributeError("无法找到可用的传输层接口")
                         return
 
         raise Exception(f"未找到设备{device_name}的方法{method_name}")

main/xiaozhi-server/core/providers/tools/device_mcp/mcp_handler.py

@@ -96,7 +96,7 @@ class MCPClient:
                 self.call_results.pop(id)
 
 
-async def send_mcp_message(conn, payload: dict):
+async def send_mcp_message(conn, payload: dict, transport=None):
     """Helper to send MCP messages, encapsulating common logic."""
     if not conn.features.get("mcp"):
         logger.bind(tag=TAG).warning("客户端不支持MCP，无法发送MCP消息")
@@ -105,13 +105,23 @@ async def send_mcp_message(conn, payload: dict):
     message = json.dumps({"type": "mcp", "payload": payload})
 
     try:
-        await conn.websocket.send(message)
+        # 优先使用传入的transport，否则尝试从conn获取
+        if transport:
+            await transport.send(message)
+        elif hasattr(conn, 'websocket'):
+            # 兼容旧版本
+            await conn.websocket.send(message)
+        elif hasattr(conn, 'transport'):
+            # 新架构
+            await conn.transport.send(message)
+        else:
+            raise AttributeError("无法找到可用的传输层接口")
         logger.bind(tag=TAG).debug(f"成功发送MCP消息: {message}")
     except Exception as e:
         logger.bind(tag=TAG).error(f"发送MCP消息失败: {e}")
 
 
-async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
+async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict, transport=None):
     """处理MCP消息,包括初始化、工具列表和工具调用响应等"""
     logger.bind(tag=TAG).debug(f"处理MCP消息: {str(payload)[:100]}")
 
@@ -196,7 +206,7 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
                 next_cursor = result.get("nextCursor", "")
                 if next_cursor:
                     logger.bind(tag=TAG).debug(f"有更多工具，nextCursor: {next_cursor}")
-                    await send_mcp_tools_list_continue_request(conn, next_cursor)
+                    await send_mcp_tools_list_continue_request(conn, next_cursor, transport)
                 else:
                     await mcp_client.set_ready(True)
                     logger.bind(tag=TAG).debug("所有工具已获取，MCP客户端准备就绪")
@@ -224,7 +234,7 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
             )
 
 
-async def send_mcp_initialize_message(conn):
+async def send_mcp_initialize_message(conn, transport=None):
     """发送MCP初始化消息"""
 
     vision_url = get_vision_url(conn.config)
@@ -256,10 +266,10 @@ async def send_mcp_initialize_message(conn):
         },
     }
     logger.bind(tag=TAG).debug("发送MCP初始化消息")
-    await send_mcp_message(conn, payload)
+    await send_mcp_message(conn, payload, transport)
 
 
-async def send_mcp_tools_list_request(conn):
+async def send_mcp_tools_list_request(conn, transport=None):
     """发送MCP工具列表请求"""
     payload = {
         "jsonrpc": "2.0",
@@ -267,10 +277,10 @@ async def send_mcp_tools_list_request(conn):
         "method": "tools/list",
     }
     logger.bind(tag=TAG).debug("发送MCP工具列表请求")
-    await send_mcp_message(conn, payload)
+    await send_mcp_message(conn, payload, transport)
 
 
-async def send_mcp_tools_list_continue_request(conn, cursor: str):
+async def send_mcp_tools_list_continue_request(conn, cursor: str, transport=None):
     """发送带有cursor的MCP工具列表请求"""
     payload = {
         "jsonrpc": "2.0",
@@ -279,7 +289,7 @@ async def send_mcp_tools_list_continue_request(conn, cursor: str):
         "params": {"cursor": cursor},
     }
     logger.bind(tag=TAG).info(f"发送带cursor的MCP工具列表请求: {cursor}")
-    await send_mcp_message(conn, payload)
+    await send_mcp_message(conn, payload, transport)
 
 
 async def call_mcp_tool(

main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py

@@ -3,12 +3,8 @@
 import asyncio
 import os
 import json
-from datetime import timedelta
 from typing import Dict, Any, List
 
-from mcp import Implementation
-from mcp.client.session import SamplingFnT, ElicitationFnT, ListRootsFnT, LoggingFnT, MessageHandlerFnT
-from mcp.shared.session import ProgressFnT
 from mcp.types import LoggingMessageNotificationParams
 
 from config.config_loader import get_project_dir
@@ -33,6 +29,7 @@ class ServerMCPManager:
             )
         self.clients: Dict[str, ServerMCPClient] = {}
         self.tools = []
+        self._init_lock = asyncio.Lock()
 
     def load_config(self) -> Dict[str, Any]:
         """加载MCP服务配置"""
@@ -49,29 +46,50 @@ class ServerMCPManager:
             )
             return {}
 
+    async def _init_server(self, name: str, srv_config: Dict[str, Any]):
+        """初始化单个MCP服务"""
+        client = None
+        try:
+            # 初始化服务端MCP客户端
+            logger.bind(tag=TAG).info(f"初始化服务端MCP客户端: {name}")
+            client = ServerMCPClient(srv_config)
+            # 设置超时时间10秒
+            await asyncio.wait_for(client.initialize(logging_callback=self.logging_callback), timeout=10)
+
+            # 使用锁保护共享状态的修改
+            async with self._init_lock:
+                self.clients[name] = client
+                client_tools = client.get_available_tools()
+                self.tools.extend(client_tools)
+
+        except asyncio.TimeoutError:
+            logger.bind(tag=TAG).error(
+                f"Failed to initialize MCP server {name}: Timeout"
+            )
+            if client:
+                await client.cleanup()
+        except Exception as e:
+            logger.bind(tag=TAG).error(
+                f"Failed to initialize MCP server {name}: {e}"
+            )
+            if client:
+                await client.cleanup()
+
     async def initialize_servers(self) -> None:
         """初始化所有MCP服务"""
         config = self.load_config()
+        tasks = []
         for name, srv_config in config.items():
             if not srv_config.get("command") and not srv_config.get("url"):
                 logger.bind(tag=TAG).warning(
                     f"Skipping server {name}: neither command nor url specified"
                 )
                 continue
-
+            
-            try:
+            tasks.append(self._init_server(name, srv_config))
-                # 初始化服务端MCP客户端
+        
-                logger.bind(tag=TAG).info(f"初始化服务端MCP客户端: {name}")
+        if tasks:
-                client = ServerMCPClient(srv_config)
+            await asyncio.gather(*tasks)
-                await client.initialize(logging_callback=self.logging_callback)
-                self.clients[name] = client
-                client_tools = client.get_available_tools()
-                self.tools.extend(client_tools)
-
-            except Exception as e:
-                logger.bind(tag=TAG).error(
-                    f"Failed to initialize MCP server {name}: {e}"
-                )
 
         # 输出当前支持的服务端MCP工具列表
         if hasattr(self.conn, "func_handler") and self.conn.func_handler: