Merge pull request #1923 from xinnan-tech/update-version

update:升级版本号
2026-07-23 23:53:55 +08:00 · 2025-07-26 11:47:15 +08:00 · 2025-07-26 11:46:11 +08:00 · 2025-07-25 22:17:59 +08:00 · 2025-07-25 22:11:30 +08:00 · 2025-07-25 21:47:54 +08:00
46 changed files with 2588 additions and 831 deletions
@@ -21,6 +21,7 @@ RUN apt-get update && \

 # 从构建阶段复制Python包和前端构建产物
 COPY --from=builder /usr/local/lib/python3.10/site-packages /usr/local/lib/python3.10/site-packages
+COPY --from=builder /usr/local/bin/mcp-proxy /usr/local/bin/mcp-proxy

 # 复制应用代码
 COPY main/xiaozhi-server .
@@ -3,7 +3,7 @@
 <h1 align="center">小智后端服务xiaozhi-esp32-server</h1>

 <p align="center">
-本项目为开源智能硬件项目
+本项目基于人机共生智能理论和技术研发智能终端软硬件体系<br/>为开源智能硬件项目
 <a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a>提供后端服务<br/>
 根据<a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">小智通信协议</a>使用Python、Java、Vue实现<br/>
 支持MCP接入点和声纹识别
@@ -37,6 +37,14 @@
  </a>
 </p>

+<p align="center">
+By Professor Siyuan Liu Research and Development Group ( South China University of Technology)
+</br>
+刘思源教授团队研发（华南理工大学）
+</br>
+<img src="./docs/images/hnlg.jpg" alt="华南理工大学" width="50%">
+</p>
+
 ---

 ## 适用人群 👥
@@ -181,7 +189,7 @@
 | 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 | 
 |---------|------|---------|---------|---------|---------|
 | **最简化安装** | 智能对话、IOT、MCP、视觉感知 | 低配置环境，数据存储在配置文件，无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G，如果全API，要2核2G | - | 
-| **全模块安装** | 智能对话、IOT、MCP接入点、视觉感知、OTA、智控台 | 完整功能体验，数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 
+| **全模块安装** | 智能对话、IOT、MCP接入点、声纹识别、视觉感知、OTA、智控台 | 完整功能体验，数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 


 > 💡 提示：以下是按最新代码部署后的测试平台，有需要可烧录测试，并发为6个，每天会清空数据
@@ -209,7 +217,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 | ASR(语音识别) | FunASR(本地) | 👍FunASRServer 或 👍DoubaoStreamASR |
 | LLM(大模型) | ChatGLMLLM(智谱glm-4-flash) | 👍DoubaoLLM(火山doubao-1-5-pro-32k-250115) |
 | VLLM(视觉大模型) | ChatGLMVLLM(智谱glm-4v-flash) | 👍QwenVLVLLM(千问qwen2.5-vl-3b-instructh) |
-| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) |
+| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) 或 👍AliyunStreamTTS(阿里云流式语音合成) |
 | Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
 | Memory(记忆功能) | mem_local_short(本地短期记忆） | mem_local_short（本地短期记忆） |

@@ -227,7 +235,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 ---
 ## 功能清单 ✨
 ### 已实现 ✅
-
+![请参考-全模块安装架构图](docs/images/deploy2.png)
 | 功能模块 | 描述 |
 |:---:|:---|
 | 核心架构 | 基于WebSocket和HTTP服务器，提供完整的控制台管理和认证系统 |
@@ -263,7 +271,6 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
 ---

 ## 本项目支持的平台/组件列表 📋
-
 ### LLM 语言模型

 | 使用方式 | 支持平台 | 免费平台 |
@@ -3,17 +3,17 @@
 <h1 align="center">Xiaozhi Backend Service xiaozhi-esp32-server</h1>

 <p align="center">
-This project provides backend services for the open-source smart hardware project
+This project is based on human-machine symbiotic intelligence theory and technology to develop intelligent terminal hardware and software systems<br/>providing backend services for the open-source intelligent hardware project
 <a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
 Implemented using Python, Java, and Vue according to the <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">Xiaozhi Communication Protocol</a><br/>
-Helps you quickly set up your Xiaozhi server
+Supports MCP endpoints and voiceprint recognition
 </p>

 <p align="center">
 <a href="./README.md">中文</a>
 · <a href="./docs/FAQ.md">FAQ</a>
 · <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Report Issues</a>
-· <a href="./README_en.md#deployment-documentation">Deployment Guide</a>
+· <a href="./README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Deployment Docs</a>
 · <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Release Notes</a>
 </p>
 <p align="center">
@@ -37,41 +37,49 @@ Helps you quickly set up your Xiaozhi server
  </a>
 </p>

+<p align="center">
+By Professor Siyuan Liu Research and Development Group (South China University of Technology)
+</br>
+刘思源教授团队研发（华南理工大学）
+</br>
+<img src="./docs/images/hnlg.jpg" alt="South China University of Technology" width="50%">
+</p>
+
 ---

 ## Target Users 👥

-This project requires ESP32 hardware devices. If you have purchased ESP32-related hardware, successfully connected to Brother Xia's backend service, and want to set up your own `xiaozhi-esp32` backend service, then this project is perfect for you.
+This project requires ESP32 hardware devices to work. If you have purchased ESP32-related hardware, successfully connected to Brother Xia's deployed backend service, and want to build your own `xiaozhi-esp32` backend service independently, then this project is perfect for you.

-Want to see it in action? Check out these videos 🎥
+Want to see the usage effects? Click the videos below 🎥

 <table>
  <tr>
    <td>
        <a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
         <picture>
-           <img alt="Xiaozhi esp32 connecting to own backend model" src="docs/images/demo1.png" />
+           <img alt="Xiaozhi ESP32 connecting to own backend model" src="docs/images/demo1.png" />
         </picture>
        </a>
    </td>
    <td>
        <a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
         <picture>
-           <img alt="Custom voice" src="docs/images/demo2.png" />
+           <img alt="Custom voice timbre" src="docs/images/demo2.png" />
         </picture>
        </a>
    </td>
    <td>
        <a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
         <picture>
-           <img alt="Using Cantonese" src="docs/images/demo3.png" />
+           <img alt="Using Cantonese for communication" src="docs/images/demo3.png" />
         </picture>
        </a>
    </td>
    <td>
        <a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
         <picture>
-           <img alt="Control home appliances" src="docs/images/demo5.png" />
+           <img alt="Controlling home appliances" src="docs/images/demo5.png" />
         </picture>
        </a>
    </td>
@@ -87,14 +95,14 @@ Want to see it in action? Check out these videos 🎥
    <td>
        <a href="https://www.bilibili.com/video/BV1Vy96YCE3R" target="_blank">
         <picture>
-           <img alt="Custom voice" src="docs/images/demo6.png" />
+           <img alt="Custom voice timbre" src="docs/images/demo6.png" />
         </picture>
        </a>
    </td>
    <td>
        <a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
         <picture>
-           <img alt="Play music" src="docs/images/demo7.png" />
+           <img alt="Playing music" src="docs/images/demo7.png" />
         </picture>
        </a>
    </td>
@@ -108,14 +116,14 @@ Want to see it in action? Check out these videos 🎥
    <td>
        <a href="https://www.bilibili.com/video/BV178XuYfEpi" target="_blank">
         <picture>
-           <img alt="IOT command control" src="docs/images/demo9.png" />
+           <img alt="IOT command control devices" src="docs/images/demo9.png" />
         </picture>
        </a>
    </td>
    <td>
        <a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
         <picture>
-           <img alt="News broadcast" src="docs/images/demo0.png" />
+           <img alt="News broadcasting" src="docs/images/demo0.png" />
         </picture>
        </a>
    </td>
@@ -131,7 +139,7 @@ Want to see it in action? Check out these videos 🎥
    <td>
      <a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
         <picture>
-           <img alt="Photo recognition" src="docs/images/demo12.png" />
+           <img alt="Photo recognition of objects" src="docs/images/demo12.png" />
         </picture>
        </a>
    </td>
@@ -143,20 +151,29 @@ Want to see it in action? Check out these videos 🎥
        </a>
    </td>
    <td>
+        <a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
+         <picture>
+           <img alt="MCP endpoint" src="docs/images/demo13.png" />
+         </picture>
+        </a>
    </td>
    <td>
+        <a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
+         <picture>
+           <img alt="Voiceprint recognition" src="docs/images/demo14.png" />
+         </picture>
+        </a>
    </td>
  </tr>
 </table>

 ---

-## Warning ⚠️
+## Warnings ⚠️

-1. This project is open-source software. This software has no commercial relationship with any third-party API service providers (including but not limited to speech recognition, large models, speech synthesis, and other platforms) and does not provide any form of guarantee for their service quality or financial security.
-It is recommended that users prioritize service providers with relevant business licenses and carefully read their service agreements and privacy policies. This software does not host any account keys, does not participate in fund transfers, and does not bear the risk of recharge fund losses.
+1. This project is open-source software. This software has no commercial partnership with any third-party API service providers (including but not limited to speech recognition, large models, speech synthesis, and other platforms) that it interfaces with, and does not provide any form of guarantee for their service quality or financial security. It is recommended that users prioritize service providers with relevant business licenses and carefully read their service agreements and privacy policies. This software does not host any account keys, does not participate in fund flows, and does not bear the risk of recharge fund losses.

-2. This project's functionality is not complete and has not passed network security testing. Please do not use it in production environments. If you deploy this project for learning in a public network environment, please ensure necessary protection measures are in place.
+2. The functionality of this project is not complete and has not passed network security assessment. Please do not use it in production environments. If you deploy this project for learning purposes in a public network environment, please ensure necessary protection measures are in place.

 ---

@@ -164,18 +181,19 @@ It is recommended that users prioritize service providers with relevant business

 ![Banners](docs/images/banner2.png)

-This project provides two deployment methods. Please choose according to your specific needs:
+This project provides two deployment methods. Please choose based on your specific needs:

 #### 🚀 Deployment Method Selection
-| Deployment Method | Features | Suitable Scenarios | Deployment Guide | Requirements | Video Tutorial | 
+| Deployment Method | Features | Applicable Scenarios | Deployment Docs | Configuration Requirements | Video Tutorials | 
 |---------|------|---------|---------|---------|---------|
-| **Simplified Installation** | Smart dialogue, IOT functionality, data stored in configuration files | Low-configuration environment, no database needed | [Docker Version](./docs/Deployment.md#method-1-docker-server-only) / [Source Code Deployment](./docs/Deployment.md#method-2-local-source-code-server-only) | 2 cores 4G if using `FunASR`, 2 cores 2G if using all APIs | - | 
-| **Full Module Installation** | Smart dialogue, IOT, OTA, Control Panel, data stored in database | Complete functionality experience | [Docker Version](./docs/Deployment_all.md#method-1-docker-full-modules) / [Source Code Deployment](./docs/Deployment_all.md#method-2-local-source-code-full-modules) | 4 cores 8G if using `FunASR`, 2 cores 4G if using all APIs | [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) / [Local Source Code Auto-Update Tutorial](./docs/dev-ops-integration.md) | 
+| **Simplified Installation** | Intelligent dialogue, IOT, MCP, visual perception | Low-configuration environments, data stored in config files, no database required | [①Docker Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - | 
+| **Full Module Installation** | Intelligent dialogue, IOT, MCP endpoints, voiceprint recognition, visual perception, OTA, intelligent control console | Complete functionality experience, data stored in database |[①Docker Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](./docs/dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) | 

-> 💡 Note: Below are the test platforms deployed with the latest code. You can flash and test if needed. Concurrent users: 6, data will be cleared daily
+
+> 💡 Note: Below is a test platform deployed with the latest code. You can burn and test if needed. Concurrent users: 6, data will be cleared daily.

 ```
-Control Panel Address: https://2662r3426b.vicp.fun
+Intelligent Control Console Address: https://2662r3426b.vicp.fun

 Service Test Tool: https://2662r3426b.vicp.fun/test/
 OTA Interface Address: https://2662r3426b.vicp.fun/xiaozhi/ota/
@@ -184,74 +202,103 @@ Websocket Interface Address: wss://2662r3426b.vicp.fun/xiaozhi/v1/

 #### 🚩 Configuration Description and Recommendations
 > [!Note]
-> The default configuration of this project is `Entry Level Free` settings. For better results, we recommend using `Full Streaming Configuration`.
+> This project provides two configuration schemes:
 > 
-> Since version `0.5.2`, this project supports full streaming throughout the entire lifecycle. Compared to versions before `0.5`, response speed has improved by approximately `2.5 seconds`
+> 1. `Entry Level Free Settings`: Suitable for personal and home use, all components use free solutions, no additional payment required.
+> 
+> 2. `Streaming Configuration`: Suitable for demonstrations, training, scenarios with more than 2 concurrent users, etc. Uses streaming processing technology for faster response speed and better experience.
+> 
+> Starting from version `0.5.2`, the project supports streaming configuration. Compared to earlier versions, response speed is improved by approximately `2.5 seconds`, significantly improving user experience.

-| Module Name | Entry Level Free Settings | Full Streaming Configuration |
-|---------|---------|------|
-| ASR(Speech Recognition) | FunASR(Local) | ✅DoubaoASR(Volcano Streaming Speech Recognition) |
-| LLM(Large Language Model) | ChatGLMLLM(Zhipu glm-4-flash) | ✅DoubaoLLM(Volcano doubao-1-5-pro-32k-250115) |
-| VLLM(Vision Large Model) | ChatGLMVLLM(Zhipu glm-4v-flash) | ✅ChatGLMVLLM(Zhipu glm-4v-flash) |
-| TTS(Speech Synthesis) | EdgeTTS(Microsoft Speech) | ✅HuoshanDoubleStreamTTS(Volcano Double Streaming Speech Synthesis) |
-| Intent(Intent Recognition) | function_call(Function Call) | ✅function_call(Function Call) |
-| Memory(Memory Function) | mem_local_short(Local Short-term Memory) | ✅mem_local_short(Local Short-term Memory) |
+| Module Name | Entry Level Free Settings | Streaming Configuration |
+|:---:|:---:|:---:|
+| ASR(Speech Recognition) | FunASR(Local) | 👍FunASRServer or 👍DoubaoStreamASR |
+| LLM(Large Model) | ChatGLMLLM(Zhipu glm-4-flash) | 👍DoubaoLLM(Volcano doubao-1-5-pro-32k-250115) |
+| VLLM(Vision Large Model) | ChatGLMVLLM(Zhipu glm-4v-flash) | 👍QwenVLVLLM(Qwen qwen2.5-vl-3b-instructh) |
+| TTS(Speech Synthesis) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano dual-stream speech synthesis) |
+| Intent(Intent Recognition) | function_call(Function calling) | function_call(Function calling) |
+| Memory(Memory function) | mem_local_short(Local short-term memory) | mem_local_short(Local short-term memory) |
+
+#### 🔧 Testing Tools
+This project provides the following testing tools to help you verify the system and choose suitable models:
+
+| Tool Name | Location | Usage Method | Function Description |
+|:---:|:---|:---:|:---:|
+| Audio Interaction Test Tool | main》xiaozhi-server》test》test_page.html | Open directly with Google Chrome | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
+| Model Response Test Tool 1 | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Tests response speed of three core modules: ASR(speech recognition), LLM(large model), TTS(speech synthesis) |
+| Model Response Test Tool 2 | main》xiaozhi-server》performance_tester_vllm.py | Execute `python performance_tester_vllm.py` | Tests VLLM(vision model) response speed |
+
+> 💡 Note: When testing model speed, only models with configured keys will be tested.

 ---
 ## Feature List ✨
 ### Implemented ✅
-
+![请参考-全模块安装架构图](docs/images/deploy2.png)
 | Feature Module | Description |
-|---------|------|
-| Communication Protocol | Based on `xiaozhi-esp32` protocol, implements data interaction through WebSocket |
-| Dialogue Interaction | Supports wake-up dialogue, manual dialogue, and real-time interruption. Auto-sleep after long periods of no dialogue |
-| Intent Recognition | Supports LLM intent recognition, function call, reducing hard-coded intent judgment |
-| Multi-language Recognition | Supports Mandarin, Cantonese, English, Japanese, Korean (default using FunASR) |
-| LLM Module | Supports flexible LLM module switching, default using ChatGLMLLM, can also use Ali Bailian, DeepSeek, Ollama, etc. |
-| TTS Module | Supports EdgeTTS (default), Volcano Engine Doubao TTS, and other TTS interfaces |
-| Memory Function | Supports ultra-long memory, local summary memory, and no memory modes |
-| IOT Function | Supports managing registered device IOT functionality, supports smart IoT control based on dialogue context |
-| Control Panel | Provides Web management interface, supports agent management, user management, system configuration, etc. |
+|:---:|:---|
+| Core Architecture | Based on WebSocket and HTTP servers, provides complete console management and authentication system |
+| Voice Interaction | Supports streaming ASR(speech recognition), streaming TTS(speech synthesis), VAD(voice activity detection), supports multi-language recognition and voice processing |
+| Voiceprint Recognition | Supports multi-user voiceprint registration, management, and recognition, processes in parallel with ASR, real-time speaker identity recognition and passes to LLM for personalized responses |
+| Intelligent Dialogue | Supports multiple LLM(large language models), implements intelligent dialogue |
+| Visual Perception | Supports multiple VLLM(vision large models), implements multimodal interaction |
+| Intent Recognition | Supports LLM intent recognition, Function Call function calling, provides plugin-based intent processing mechanism |
+| Memory System | Supports local short-term memory, mem0ai interface memory, with memory summarization functionality |
+| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP endpoint protocol, custom tool functions |
+| Management Backend | Provides Web management interface, supports user management, system configuration, and device management |
+| Testing Tools | Provides performance testing tools, vision model testing tools, and audio interaction testing tools |
+| Deployment Support | Supports Docker deployment and local deployment, provides complete configuration file management |
+| Plugin System | Supports functional plugin extensions, custom plugin development, and plugin hot-loading |

-### In Development 🚧
+### Under Development 🚧

-To learn about specific development progress, [click here](https://github.com/users/xinnan-tech/projects/3)
+To learn about specific development plan progress, [click here](https://github.com/users/xinnan-tech/projects/3)

 If you are a software developer, here is an [Open Letter to Developers](docs/contributor_open_letter.md). Welcome to join!

 ---

 ## Product Ecosystem 👬
-Xiaozhi is an ecosystem. When using this product, you might also want to check out other excellent projects in this ecosystem
+Xiaozhi is an ecosystem. When using this product, you can also check out other excellent projects in this ecosystem

 | Project Name | Project Address | Project Description |
 |:---------------------|:--------|:--------|
-| Xiaozhi Android Client | [xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) | A Flutter-based Android and iOS voice dialogue application supporting real-time voice interaction and text dialogue. |
-| Xiaozhi PC Client | [py-xiaozhi](https://github.com/Huang-junsen/py-xiaozhi) | This project provides a Python-based Xiaozhi AI client, allowing you to experience Xiaozhi AI's functionality through code even without physical hardware. |
-| Xiaozhi Java Server | [xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) | The Java version of Xiaozhi open-source backend service is a Java-based open-source project.<br/>It includes both frontend and backend services, aiming to provide users with a complete backend service solution. |
+| Xiaozhi Android Client | [xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) | An Android and iOS voice dialogue application based on xiaozhi-server, supporting real-time voice interaction and text dialogue.<br/>Currently a Flutter version, connecting iOS and Android platforms. |
+| Xiaozhi Desktop Client | [py-xiaozhi](https://github.com/Huang-junsen/py-xiaozhi) | This project provides a Python-based AI client for beginners, allowing users to experience Xiaozhi AI functionality through code even without physical hardware conditions. |
+| Xiaozhi Java Server | [xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) | Xiaozhi open-source backend service Java version is a Java-based open-source project.<br/>It includes frontend and backend services, aiming to provide users with a complete backend service solution. |

 ---

 ## Supported Platforms/Components List 📋
-
 ### LLM Language Models

 | Usage Method | Supported Platforms | Free Platforms |
 |:---:|:---:|:---:|
-| openai interface call | Ali Bailian, Volcano Engine Doubao, DeepSeek, Zhipu ChatGLM, Gemini | Zhipu ChatGLM, Gemini |
-| ollama interface call | Ollama | - |
-| dify interface call | Dify | - |
-| fastgpt interface call | Fastgpt | - |
-| coze interface call | Coze | - |
+| OpenAI interface calls | Alibaba Bailian, Volcano Engine Doubao, DeepSeek, Zhipu ChatGLM, Gemini | Zhipu ChatGLM, Gemini |
+| Ollama interface calls | Ollama | - |
+| Dify interface calls | Dify | - |
+| FastGPT interface calls | FastGPT | - |
+| Coze interface calls | Coze | - |

-In fact, any LLM that supports openai interface calls can be integrated and used.
+In fact, any LLM that supports OpenAI interface calls can be integrated and used.
+
+---
+
+### VLLM Vision Models
+
+| Usage Method | Supported Platforms | Free Platforms |
+|:---:|:---:|:---:|
+| OpenAI interface calls | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
+
+In fact, any VLLM that supports OpenAI interface calls can be integrated and used.
+
+---

 ### TTS Speech Synthesis

 | Usage Method | Supported Platforms | Free Platforms |
 |:---:|:---:|:---:|
-| API Call | EdgeTTS, Volcano Engine Doubao TTS, Tencent Cloud, Alibaba Cloud TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS | EdgeTTS, CosyVoiceSiliconflow(partial) |
-| Local Service | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS |
+| Interface calls | EdgeTTS, Volcano Engine Doubao TTS, Tencent Cloud, Alibaba Cloud TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi Streaming TTS | Lingxi Streaming TTS, EdgeTTS, CosyVoiceSiliconflow(partial) |
+| Local services | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS |

 ---

@@ -259,7 +306,7 @@ In fact, any LLM that supports openai interface calls can be integrated and used

 | Type | Platform Name | Usage Method | Pricing Model | Notes |
 |:---:|:---------:|:----:|:----:|:--:|
-| VAD | SileroVAD | Local Usage | Free | |
+| VAD | SileroVAD | Local use | Free | |

 ---

@@ -267,8 +314,16 @@ In fact, any LLM that supports openai interface calls can be integrated and used

 | Usage Method | Supported Platforms | Free Platforms |
 |:---:|:---:|:---:|
-| Local Usage | FunASR, SherpaASR | FunASR, SherpaASR |
-| API Call | DoubaoASR, FunASRServer, TencentASR, AliyunASR | FunASRServer |
+| Local use | FunASR, SherpaASR | FunASR, SherpaASR |
+| Interface calls | DoubaoASR, FunASRServer, TencentASR, AliyunASR | FunASRServer |
+
+---
+
+### Voiceprint Recognition
+
+| Usage Method | Supported Platforms | Free Platforms |
+|:---:|:---:|:---:|
+| Local use | 3D-Speaker | 3D-Speaker |

 ---

@@ -276,8 +331,8 @@ In fact, any LLM that supports openai interface calls can be integrated and used

 | Type | Platform Name | Usage Method | Pricing Model | Notes |
 |:------:|:---------------:|:----:|:---------:|:--:|
-| Memory | mem0ai | API Call | 1000 calls/month quota | |
-| Memory | mem_local_short | Local Summary | Free | |
+| Memory | mem0ai | Interface calls | 1000 times/month quota | |
+| Memory | mem_local_short | Local summarization | Free | |

 ---

@@ -285,8 +340,8 @@ In fact, any LLM that supports openai interface calls can be integrated and used

 | Type | Platform Name | Usage Method | Pricing Model | Notes |
 |:------:|:-------------:|:----:|:-------:|:---------------------:|
-| Intent | intent_llm | API Call | Based on LLM pricing | Uses large model for intent recognition, highly versatile |
-| Intent | function_call | API Call | Based on LLM pricing | Uses large model function calls for intent, fast and effective |
+| Intent | intent_llm | Interface calls | Based on LLM pricing | Recognizes intent through large models, strong generalization |
+| Intent | function_call | Interface calls | Based on LLM pricing | Completes intent through large model function calling, fast speed, good effect |

 ---

@@ -294,11 +349,11 @@ In fact, any LLM that supports openai interface calls can be integrated and used

 | Logo | Project/Company | Description |
 |:---:|:---:|:---|
-| <img src="./docs/images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | This project was inspired by [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) and implemented based on it |
-| <img src="./docs/images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Thanks to [Tenclass](https://www.tenclass.com/) for establishing standard communication protocols, multi-device compatibility solutions, and high-concurrency scenario practices for the Xiaozhi ecosystem; providing full-chain technical documentation support for this project |
-| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Thanks to [Xuanfeng Technology](https://github.com/Eric0308) for contributing the function call framework, MCP communication protocol, and plugin call mechanism implementation code, significantly improving front-end device (IoT) interaction efficiency and functional extensibility through standardized instruction scheduling system and dynamic expansion capabilities |
-| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Thanks to [Huiyuan Design](http://ui.kwd988.net/) for providing professional visual solutions for this project, empowering the product user experience with their design experience serving over a thousand enterprises |
-| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Thanks to [Xi'an Qinren Information Technology](https://www.029app.com/) for deepening the visual system of this project, ensuring consistency and extensibility of the overall design style in multi-scenario applications |
+| <img src="./docs/images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | This project is inspired by [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) and implemented on its basis |
+| <img src="./docs/images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Thanks to [Tenclass](https://www.tenclass.com/) for formulating standard communication protocols, multi-device compatibility solutions, and high-concurrency scenario practice demonstrations for the Xiaozhi ecosystem; providing full-link technical documentation support for this project |
+| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Thanks to [Xuanfeng Technology](https://github.com/Eric0308) for contributing function calling framework, MCP communication protocol, and plugin-based calling mechanism implementation code. Through standardized instruction scheduling system and dynamic expansion capabilities, it significantly improves the interaction efficiency and functional extensibility of frontend devices (IoT) |
+| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Thanks to [Huiyuan Design](http://ui.kwd988.net/) for providing professional visual solutions for this project, using their design practical experience serving over a thousand enterprises to empower this project's product user experience |
+| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Thanks to [Xi'an Qinren Information Technology](https://www.029app.com/) for deepening this project's visual system, ensuring consistency and extensibility of overall design style in multi-scenario applications |


 <a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
@@ -1,8 +1,9 @@
 # 声纹识别启用指南

-本教程包含2个部分
+本教程包含3个部分
 - 1、如何部署声纹识别这个服务
 - 2、全模块部署时，怎么配置声纹识别接口
+- 3、最简化部署时，怎么配置声纹识别

 # 1、如何部署声纹识别这个服务

@@ -15,7 +16,99 @@
 点击它，下载本项目源码压缩包。下载到你电脑后，解压它，此时它的名字可能叫`voiceprint-api-main`
 你需要把它重命名成`voiceprint-api`。

-## 第二步，启动程序
+## 第二步， 创建数据库和表
+
+声纹识别需要依赖`mysql`数据库。如果你之前已经部署`智控台`，说明你已经安装了`mysql`。你可以共用它。
+
+你可以你试一下在宿主机使用`telnet`命令，看看能不能正常访问`mysql`的`3306`端口。
+```
+telnet 127.0.0.1 3306
+```
+如果能访问到3306端口，请忽略以下的内容，直接进入第三步。
+
+如果不能访问，你需要回忆一下，你的`mysql`是怎么安装的。
+
+如果你的mysql是通过自己使用安装包安装的，说明你的`mysql`做了网络隔离。你可能先解访问`mysql`的`3306`端口这个问题。
+
+如果你`mysql`是通过本项目的`docker-compose_all.yml`安装的。你需要找一下你当时创建数据库的`docker-compose_all.yml`文件，修改以下的内容
+
+修改前
+```
+  xiaozhi-esp32-server-db:
+    ...
+    networks:
+      - default
+    expose:
+      - "3306:3306"
+```
+
+修改后
+```
+  xiaozhi-esp32-server-db:
+    ...
+    networks:
+      - default
+    ports:
+      - "3306:3306"
+```
+
+注意是将`xiaozhi-esp32-server-db`下面的`expose`改成`ports`。改完后，需要重新启动。以下是重启mysql的命令：
+
+```
+# 进入你docker-compose_all.yml所在的文件夹，例如我的是xiaozhi-server
+cd xiaozhi-server
+docker compose -f docker-compose_all.yml down
+docker compose -f docker-compose.yml up -d
+```
+
+启动完后，在宿主机再使用`telnet`命令，看看能不能正常访问`mysql`的`3306`端口。
+```
+telnet 127.0.0.1 3306
+```
+正常来说这样就可以访问的了。
+
+## 第三步， 创建数据库和表
+如果你的宿主机，能正常访问mysql数据库，那就在mysql上创建一个名字为`voiceprint_db`的数据库和`voiceprints`表。
+
+```
+CREATE DATABASE voiceprint_db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
+
+USE voiceprint_db;
+
+CREATE TABLE voiceprints (
+    id INT AUTO_INCREMENT PRIMARY KEY,
+    speaker_id VARCHAR(255) NOT NULL UNIQUE,
+    feature_vector LONGBLOB NOT NULL,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+    INDEX idx_speaker_id (speaker_id)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
+```
+
+## 第四步， 配置数据库连接
+
+进入`voiceprint-api`文件夹，创建名字为`data`的文件夹。
+
+把`voiceprint-api`根目录里的`voiceprint.yaml`，复制到`data`的文件夹，将它重命名为`.voiceprint.yaml`
+
+接下来，你需要重点配置一下`.voiceprint.yaml`里的数据库连接。
+
+```
+mysql:
+  host: "127.0.0.1"
+  port: 3306
+  user: "root"
+  password: "your_password"
+  database: "voiceprint_db"
+```
+
+注意！由于你的声纹识别服务是使用docker部署，`host`需要填写成你`mysql所在机器的局域网ip`。
+
+注意！由于你的声纹识别服务是使用docker部署，`host`需要填写成你`mysql所在机器的局域网ip`。
+
+注意！由于你的声纹识别服务是使用docker部署，`host`需要填写成你`mysql所在机器的局域网ip`。
+
+## 第五步，启动程序
 这个项目是一个很简单的项目，建议使用docker运行。不过如果你不想使用docker运行，你可以参考[这个页面](https://github.com/xinnan-tech/voiceprint-api/blob/main/README.md)使用源码运行。以下是docker运行的方法

 ```
@@ -93,4 +186,48 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd

 ## 第三步 和你的智能体聊天

-将你的设备通电，问它，你知道我是谁吗？如果他能回答得出，说明声纹识别功能正常。
+将你的设备通电，问它，你知道我是谁吗？如果他能回答得出，说明声纹识别功能正常。
+
+# 3、最简化部署时，怎么配置声纹识别
+
+## 第一步 配置接口
+打开 `xiaozhi-server/data/.config.yaml` 文件（如果没有需要创建），然后添加/修改以下内容：
+
+```
+# 声纹识别配置
+voiceprint:
+  # 声纹接口地址
+  url: 你的声纹接口地址
+  # 说话人配置：speaker_id,名称,描述
+  speakers:
+    - "test1,张三,张三是一个程序员"
+    - "test2,李四,李四是一个产品经理"
+    - "test3,王五,王五是一个设计师"
+```
+
+把上一步得来的 `声纹接口地址` 粘贴到 `url` 里。然后保存。
+
+`speakers` 参数依据需求添加。这里需要注意这个 `speaker_id` 参数，后面注册声纹会用到。
+
+## 第二步 注册声纹
+如果你已经启动了声纹服务，本地浏览器里访问 `http://localhost:8005/voiceprint/docs` 即可查看 API 文档，这里只说明注册声纹的 API 如何使用。
+
+注册声纹的 API 地址为 `http://localhost:8005/voiceprint/register`，请求方式为 POST。
+
+请求头需要包含 Bearer Token 认证，token 为 `声纹接口地址` 中 `?key=` 后的部分，比如如果我的声纹注册地址为 `http://127.0.0.1:8005/voiceprint/health?key=abcd`，那么我的 token 就是`abcd`。
+
+请求体包含说话人 ID（speaker_id），和 WAV 音频文件（file），请求示例如下：
+
+```
+curl -X POST \
+  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
+  -F "speaker_id=your_speaker_id_here" \
+  -F "file=@/path/to/your/file" \
+  http://localhost:8005/voiceprint/register
+```
+
+ 这里的 `file` 是要注册的说话人说话的音频文件， `speaker_id` 需要和第一步配置接口的 `speaker_id` 保持一致。比如说我需要注册张三的声纹，在 `.config.yaml` 中填的张三的 `speaker_id` 为 `test1`，那么我注册张三声纹的时候，请求体里填的 `speaker_id` 就是 `test1`， `file` 填的就是张三说一段话的音频文件。
+
+ ## 第三步 启动服务
+
+启动小智服务器和声纹服务，即可正常使用。
@@ -237,7 +237,7 @@ public interface Constant {
    /**
     * 版本号
     */
-    public static final String VERSION = "0.7.1";
+    public static final String VERSION = "0.7.3";

    /**
     * 无效固件URL
@@ -0,0 +1,21 @@
+package xiaozhi.common.utils;
+
+import lombok.Data;
+/**
+ * JSON-RPC2.0 格式规范对象
+ */
+@Data
+public class JsonRpcTwo {
+    private String jsonrpc = "2.0";
+    private String method;
+    private Object params;
+    private Integer id;
+
+    public JsonRpcTwo(String method, Object params, Integer id) {
+        this.method = method;
+        this.params = params;
+        this.id = id;
+    }
+
+
+}
@@ -0,0 +1,44 @@
+package xiaozhi.modules.agent.Enums;
+
+import xiaozhi.common.utils.JsonUtils;
+import xiaozhi.common.utils.JsonRpcTwo;
+
+import java.util.Map;
+
+
+/**
+ * 小智MCP JSON-RPC 请求json
+ */
+public class XiaoZhiMcpJsonRpcJson {
+    //小智初始化mcp请求json
+    private static final String INITIALIZE_JSON;
+    //小智mcp初始化成功，返回通知请求json
+    private static final String NOTIFICATIONS_INITIALIZED_JSON;
+    //小智mcp获取mcp工具集合请求json
+    private static final String TOOLS_LIST_REQUEST;
+    // 延迟加载
+    static {
+        INITIALIZE_JSON = JsonUtils.toJsonString(new JsonRpcTwo("initialize",
+                Map.of(
+                        "protocolVersion", "2024-11-05",
+                        "capabilities", Map.of(
+                                "roots", Map.of("listChanged", false),
+                                "sampling", Map.of()),
+                        "clientInfo", Map.of(
+                                "name", "xz-mcp-broker",
+                                "version", "0.0.1")),
+                1));
+        NOTIFICATIONS_INITIALIZED_JSON = "{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}";
+        TOOLS_LIST_REQUEST = JsonUtils.toJsonString(new JsonRpcTwo("tools/list", null, 2));
+    }
+    public static String getInitializeJson(){
+        return INITIALIZE_JSON;
+    }
+    public static String getNotificationsInitializedJson(){
+        return NOTIFICATIONS_INITIALIZED_JSON;
+    }
+    public static String getToolsListJson(){
+        return TOOLS_LIST_REQUEST;
+    }
+
+}
@@ -1,32 +0,0 @@
-package xiaozhi.modules.agent.dto;
-
-import lombok.Data;
-
-/**
- * MCP JSON-RPC 请求 DTO
- */
-@Data
-public class McpJsonRpcRequest {
-    private String jsonrpc = "2.0";
-    private String method;
-    private Object params;
-    private Integer id;
-
-    public McpJsonRpcRequest() {
-    }
-
-    public McpJsonRpcRequest(String method) {
-        this.method = method;
-    }
-
-    public McpJsonRpcRequest(String method, Object params, Integer id) {
-        this.method = method;
-        this.params = params;
-        this.id = id;
-    }
-
-    public McpJsonRpcRequest(String method, Object params) {
-        this.method = method;
-        this.params = params;
-    }
-}
@@ -18,7 +18,7 @@ import xiaozhi.common.constant.Constant;
 import xiaozhi.common.utils.AESUtils;
 import xiaozhi.common.utils.HashEncryptionUtil;
 import xiaozhi.common.utils.JsonUtils;
-import xiaozhi.modules.agent.dto.McpJsonRpcRequest;
+import xiaozhi.modules.agent.Enums.XiaoZhiMcpJsonRpcJson;
 import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
 import xiaozhi.modules.sys.service.SysParamsService;
 import xiaozhi.modules.sys.utils.WebSocketClientManager;
@@ -65,22 +65,13 @@ public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointServic
            try (WebSocketClientManager client = WebSocketClientManager.build(
                    new WebSocketClientManager.Builder()
                            .uri(wsUrl)
+                            .bufferSize(1024 * 1024)
                            .connectTimeout(8, TimeUnit.SECONDS)
                            .maxSessionDuration(10, TimeUnit.SECONDS))) {

                // 步骤1: 发送初始化消息并等待响应
                log.info("发送MCP初始化消息，智能体ID: {}", id);
-                McpJsonRpcRequest initializeRequest = new McpJsonRpcRequest("initialize",
-                        Map.of(
-                                "protocolVersion", "2024-11-05",
-                                "capabilities", Map.of(
-                                        "roots", Map.of("listChanged", false),
-                                        "sampling", Map.of()),
-                                "clientInfo", Map.of(
-                                        "name", "xz-mcp-broker",
-                                        "version", "0.0.1")),
-                        1);
-                client.sendJson(initializeRequest);
+                client.sendText(XiaoZhiMcpJsonRpcJson.getInitializeJson());

                // 等待初始化响应 (id=1) - 移除固定延迟，改为响应驱动
                List<String> initResponses = client.listenerWithoutClose(response -> {
@@ -124,13 +115,10 @@ public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointServic

                // 步骤2: 发送初始化完成通知 - 只有在收到initialize响应后才发送
                log.info("发送MCP初始化完成通知，智能体ID: {}", id);
-                String notificationJson = "{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}";
-                client.sendText(notificationJson);
-
+                client.sendText(XiaoZhiMcpJsonRpcJson.getNotificationsInitializedJson());
                // 步骤3: 发送工具列表请求 - 立即发送，无需额外延迟
                log.info("发送MCP工具列表请求，智能体ID: {}", id);
-                McpJsonRpcRequest toolsRequest = new McpJsonRpcRequest("tools/list", null, 2);
-                client.sendJson(toolsRequest);
+                client.sendText(XiaoZhiMcpJsonRpcJson.getToolsListJson());

                // 等待工具列表响应 (id=2)
                List<String> toolsResponses = client.listener(response -> {
@@ -178,7 +166,7 @@ public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointServic

            }
        } catch (Exception e) {
-            log.error("获取智能体 MCP 工具列表失败，智能体ID: {}", id, e);
+            log.error("获取智能体 MCP 工具列表失败，智能体ID: {},错误原因：{}", id, e.getMessage());
            return List.of();
        }
    }
@@ -41,6 +41,7 @@ import xiaozhi.modules.agent.service.AgentTemplateService;
 import xiaozhi.modules.agent.vo.AgentInfoVO;
 import xiaozhi.modules.device.service.DeviceService;
 import xiaozhi.modules.model.dto.ModelProviderDTO;
+import xiaozhi.modules.model.entity.ModelConfigEntity;
 import xiaozhi.modules.model.service.ModelConfigService;
 import xiaozhi.modules.model.service.ModelProviderService;
 import xiaozhi.modules.security.user.SecurityUser;
@@ -324,9 +325,32 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
            // 删除音频数据
            agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, false);
        }
+
+        boolean b = validateLLMIntentParams(dto.getLlmModelId(), dto.getIntentModelId());
+        if (!b) {
+            throw new RenException("LLM大模型和Intent意图识别，选择参数不匹配");
+        }
        this.updateById(existingEntity);
    }

+    /**
+     * 验证大语言模型和意图识别的参数是否符合匹配
+     * 
+     * @param llmModelId    大语言模型id
+     * @param intentModelId 意图识别id
+     * @return T 匹配 : F 不匹配
+     */
+    private boolean validateLLMIntentParams(String llmModelId, String intentModelId) {
+        ModelConfigEntity llmModelData = modelConfigService.selectById(llmModelId);
+        String type = llmModelData.getConfigJson().get("type").toString();
+        // 如果查询大语言模型是openai或者ollama，意图识别选参数都可以
+        if ("openai".equals(type) || "ollama".equals(type)) {
+            return true;
+        }
+        // 除了openai和ollama的类型，不可以选择id为Intent_function_call（函数调用）的意图识别
+        return !"Intent_function_call".equals(intentModelId);
+    }
+
    @Override
    @Transactional(rollbackFor = Exception.class)
    public String createAgent(AgentCreateDTO dto) {
@@ -3,9 +3,11 @@ package xiaozhi.modules.agent.service.impl;
 import java.net.URI;
 import java.net.URISyntaxException;
 import java.util.List;
+import java.util.concurrent.Executor;
 import java.util.stream.Collectors;

 import org.apache.commons.lang3.StringUtils;
+import org.springframework.beans.factory.annotation.Qualifier;
 import org.springframework.core.io.ByteArrayResource;
 import org.springframework.http.HttpEntity;
 import org.springframework.http.HttpHeaders;
@@ -22,7 +24,6 @@ import org.springframework.web.client.RestTemplate;
 import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
 import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;

-import lombok.AllArgsConstructor;
 import lombok.extern.slf4j.Slf4j;
 import xiaozhi.common.constant.Constant;
 import xiaozhi.common.exception.RenException;
@@ -43,7 +44,6 @@ import xiaozhi.modules.sys.service.SysParamsService;
 * @author zjy
 */
@Service
-@AllArgsConstructor
@Slf4j
 public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao, AgentVoicePrintEntity>
        implements AgentVoicePrintService {
@@ -55,6 +55,18 @@ public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao,
    private final TransactionTemplate transactionTemplate;
    // 识别度
    private final Double RECOGNITION = 0.5;
+    private final Executor taskExecutor;
+
+    public AgentVoicePrintServiceImpl(AgentChatAudioService agentChatAudioService, RestTemplate restTemplate,
+                                      SysParamsService sysParamsService, AgentChatHistoryService agentChatHistoryService,
+                                      TransactionTemplate transactionTemplate, @Qualifier("taskExecutor") Executor taskExecutor) {
+        this.agentChatAudioService = agentChatAudioService;
+        this.restTemplate = restTemplate;
+        this.sysParamsService = sysParamsService;
+        this.agentChatHistoryService = agentChatHistoryService;
+        this.transactionTemplate = transactionTemplate;
+        this.taskExecutor = taskExecutor;
+    }

    @Override
    public boolean insert(AgentVoicePrintSaveDTO dto) {
@@ -96,7 +108,7 @@ public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao,
    @Override
    public boolean delete(Long userId, String voicePrintId) {
        // 开启事务
-        return Boolean.TRUE.equals(transactionTemplate.execute(status -> {
+        boolean b = Boolean.TRUE.equals(transactionTemplate.execute(status -> {
            try {
                // 删除声纹,按照指定当前登录用户和智能体
                int row = baseMapper.delete(new LambdaQueryWrapper<AgentVoicePrintEntity>()
@@ -106,17 +118,25 @@ public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao,
                    status.setRollbackOnly(); // 标记事务回滚
                    return false;
                }
-                cancelVoicePrint(voicePrintId);
+
                return true;
-            } catch (RenException e) {
-                status.setRollbackOnly(); // 标记事务回滚
-                throw e;
            } catch (Exception e) {
                status.setRollbackOnly(); // 标记事务回滚
-                log.error("删除声纹错误原因：{}", e.getMessage());
-                throw new RenException("删除声纹错误，请联系管理员");
+                log.error("删除声纹存在错误原因：{}", e.getMessage());
+                throw new RenException("删除声纹出现了错误");
            }
        }));
+        // 数据库声纹数据删除成功才继续执行删除声纹服务的数据
+        if(b){
+            taskExecutor.execute(()-> {
+                try {
+                    cancelVoicePrint(voicePrintId);
+                }catch (RuntimeException e) {
+                    log.error("删除声纹存在运行时错误原因：{}，id：{}", e.getMessage(),voicePrintId);
+                }
+            });
+        }
+        return b;
    }

    @Override
@@ -219,7 +239,11 @@ public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao,
        String protocol = uri.getScheme();
        String host = uri.getHost();
        int port = uri.getPort();
-        return "%s://%s:%s".formatted(protocol, host, port);
+        if (port == -1) {
+            return "%s://%s".formatted(protocol, host);
+        } else {
+            return "%s://%s:%s".formatted(protocol, host, port);
+        }
    }

    /**
@@ -21,11 +21,7 @@ import xiaozhi.common.utils.ConvertUtils;
 import xiaozhi.common.utils.Result;
 import xiaozhi.modules.agent.service.AgentTemplateService;
 import xiaozhi.modules.config.service.ConfigService;
-import xiaozhi.modules.model.dto.ModelBasicInfoDTO;
-import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
-import xiaozhi.modules.model.dto.ModelConfigDTO;
-import xiaozhi.modules.model.dto.ModelProviderDTO;
-import xiaozhi.modules.model.dto.VoiceDTO;
+import xiaozhi.modules.model.dto.*;
 import xiaozhi.modules.model.entity.ModelConfigEntity;
 import xiaozhi.modules.model.service.ModelConfigService;
 import xiaozhi.modules.model.service.ModelProviderService;
@@ -52,6 +48,14 @@ public class ModelController {
        return new Result<List<ModelBasicInfoDTO>>().ok(modelList);
    }

+    @GetMapping("/llm/names")
+    @Operation(summary = "获取LLM模型信息")
+    @RequiresPermissions("sys:role:normal")
+    public Result<List<LlmModelBasicInfoDTO>> getLlmModelCodeList(@RequestParam(required = false) String modelName) {
+        List<LlmModelBasicInfoDTO> llmModelCodeList = modelConfigService.getLlmModelCodeList(modelName);
+        return new Result<List<LlmModelBasicInfoDTO>>().ok(llmModelCodeList);
+    }
+
    @GetMapping("/{modelType}/provideTypes")
    @Operation(summary = "获取模型供应器列表")
    @RequiresPermissions("sys:role:superAdmin")
@@ -0,0 +1,13 @@
+package xiaozhi.modules.model.dto;
+
+import lombok.Data;
+import lombok.EqualsAndHashCode;
+
+/**
+ * LLM的模型的基础展示数据
+ */
+@EqualsAndHashCode(callSuper = true)
+@Data
+public class LlmModelBasicInfoDTO extends ModelBasicInfoDTO{
+    private String type;
+}
@@ -4,6 +4,7 @@ import java.util.List;

 import xiaozhi.common.page.PageData;
 import xiaozhi.common.service.BaseService;
+import xiaozhi.modules.model.dto.LlmModelBasicInfoDTO;
 import xiaozhi.modules.model.dto.ModelBasicInfoDTO;
 import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
 import xiaozhi.modules.model.dto.ModelConfigDTO;
@@ -13,6 +14,8 @@ public interface ModelConfigService extends BaseService<ModelConfigEntity> {

    List<ModelBasicInfoDTO> getModelCodeList(String modelType, String modelName);

+    List<LlmModelBasicInfoDTO> getLlmModelCodeList(String modelName);
+
    PageData<ModelConfigDTO> getPageList(String modelType, String modelName, String page, String limit);

    ModelConfigDTO add(String modelType, String provideCode, ModelConfigBodyDTO modelConfigBodyDTO);
@@ -8,6 +8,7 @@ import java.util.stream.Collectors;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.stereotype.Service;

+import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
 import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
 import com.baomidou.mybatisplus.core.metadata.IPage;

@@ -23,6 +24,7 @@ import xiaozhi.common.utils.ConvertUtils;
 import xiaozhi.modules.agent.dao.AgentDao;
 import xiaozhi.modules.agent.entity.AgentEntity;
 import xiaozhi.modules.model.dao.ModelConfigDao;
+import xiaozhi.modules.model.dto.LlmModelBasicInfoDTO;
 import xiaozhi.modules.model.dto.ModelBasicInfoDTO;
 import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
 import xiaozhi.modules.model.dto.ModelConfigDTO;
@@ -52,6 +54,25 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
        return ConvertUtils.sourceToTarget(entities, ModelBasicInfoDTO.class);
    }

+    @Override
+    public List<LlmModelBasicInfoDTO> getLlmModelCodeList(String modelName) {
+        List<ModelConfigEntity> entities = modelConfigDao.selectList(
+                new QueryWrapper<ModelConfigEntity>()
+                        .eq("model_type", "llm")
+                        .eq("is_enabled", 1)
+                        .like(StringUtils.isNotBlank(modelName), "model_name", "%" + modelName + "%")
+                        .select("id", "model_name", "config_json"));
+        // 处理获取到的内容
+        return entities.stream().map(item -> {
+            LlmModelBasicInfoDTO dto = new LlmModelBasicInfoDTO();
+            dto.setId(item.getId());
+            dto.setModelName(item.getModelName());
+            String type = item.getConfigJson().get("type").toString();
+            dto.setType(type);
+            return dto;
+        }).toList();
+    }
+
    @Override
    public PageData<ModelConfigDTO> getPageList(String modelType, String modelName, String page, String limit) {
        Map<String, Object> params = new HashMap<String, Object>();
@@ -94,6 +115,21 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
        if (CollectionUtil.isEmpty(providerList)) {
            throw new RenException("供应器不存在");
        }
+        if (modelConfigBodyDTO.getConfigJson().containsKey("llm")) {
+            String llm = modelConfigBodyDTO.getConfigJson().get("llm").toString();
+            ModelConfigEntity modelConfigEntity = modelConfigDao.selectOne(new LambdaQueryWrapper<ModelConfigEntity>()
+                    .eq(ModelConfigEntity::getId, llm));
+            String selectModelType = (modelConfigEntity == null || modelConfigEntity.getModelType() == null) ? null
+                    : modelConfigEntity.getModelType().toUpperCase();
+            if (modelConfigEntity == null || !"LLM".equals(selectModelType)) {
+                throw new RenException("设置的LLM不存在");
+            }
+            String type = modelConfigEntity.getConfigJson().get("type").toString();
+            // 如果查询大语言模型是openai或者ollama，意图识别选参数都可以
+            if (!"openai".equals(type) && !"ollama".equals(type)) {
+                throw new RenException("设置的LLM不是openai和ollama");
+            }
+        }

        // 再更新供应器提供的模型
        ModelConfigEntity modelConfigEntity = ConvertUtils.sourceToTarget(modelConfigBodyDTO, ModelConfigEntity.class);
@@ -137,6 +173,8 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
                        .or()
                        .eq("mem_model_id", modelId)
                        .or()
+                        .eq("vllm_model_id", modelId)
+                        .or()
                        .eq("intent_model_id", modelId));
        if (!agents.isEmpty()) {
            String agentNames = agents.stream()
@@ -86,10 +86,14 @@ public class WebSocketClientManager implements Closeable {
        if (sess == null || !sess.isOpen()) {
            throw new IOException("握手失败或会话未打开");
        }
+        // 设置缓冲区
+        sess.setTextMessageSizeLimit(b.bufferSize);
+        sess.setBinaryMessageSizeLimit(b.bufferSize);
        ws.session = sess;
        return ws;
    }

+
    /**
     * 发送 Text
     */
@@ -308,10 +312,11 @@ public class WebSocketClientManager implements Closeable {
            if (stopWatch.isRunning()) {
                stopWatch.stop();
            }
-            log.info("ws连接关闭, 目标URI: {}, 关闭时间: {}, 连接总时长: {}s",
+            log.info("ws连接关闭, 目标URI: {}, 关闭时间: {}, 连接总时长: {}s,断开原因：{}",
                    targetUri, DateUtils.getDateTimeNow(DateUtils.DATE_TIME_MILLIS_PATTERN),
-                    DateUtils.millsToSecond(stopWatch.getTotalTimeMillis()));
+                    DateUtils.millsToSecond(stopWatch.getTotalTimeMillis()),status);
        }
+
    }

    public static class Builder {
@@ -321,6 +326,7 @@ public class WebSocketClientManager implements Closeable {
        private long maxSessionDuration = 5; // 最大连线时间，默认5秒
        private TimeUnit maxSessionDurationUnit = TimeUnit.SECONDS; // 最大连线时间单位
        private int queueCapacity = 100; // 消息队列容量
+        private int bufferSize = 8 * 1024; //默认 8kb
        private WebSocketHttpHeaders headers; // 请求头

        /**
@@ -352,6 +358,10 @@ public class WebSocketClientManager implements Closeable {
            this.queueCapacity = c;
            return this;
        }
+        public Builder bufferSize(int c) {
+            this.bufferSize = c;
+            return this;
+        }

        public WebSocketClientManager build()
                throws InterruptedException, ExecutionException, TimeoutException, IOException {
@@ -0,0 +1,26 @@
+-- 添加阿里云流式ASR供应器
+delete from `ai_model_provider` where id = 'SYSTEM_ASR_AliyunStreamASR';
+INSERT INTO `ai_model_provider` (`id`, `model_type`, `provider_code`, `name`, `fields`, `sort`, `creator`, `create_date`, `updater`, `update_date`) VALUES
+('SYSTEM_ASR_AliyunStreamASR', 'ASR', 'aliyun_stream', '阿里云语音识别(流式)', '[{"key":"appkey","label":"应用AppKey","type":"string"},{"key":"token","label":"临时Token","type":"string"},{"key":"access_key_id","label":"AccessKey ID","type":"string"},{"key":"access_key_secret","label":"AccessKey Secret","type":"string"},{"key":"host","label":"服务地址","type":"string"},{"key":"max_sentence_silence","label":"断句检测时间","type":"number"},{"key":"output_dir","label":"输出目录","type":"string"}]', 6, 1, NOW(), 1, NOW());
+
+-- 添加阿里云流式ASR模型配置
+delete from `ai_model_config` where id = 'ASR_AliyunStreamASR';
+INSERT INTO `ai_model_config` VALUES ('ASR_AliyunStreamASR', 'ASR', 'AliyunStreamASR', '阿里云语音识别(流式)', 0, 1, '{\"type\": \"aliyun_stream\", \"appkey\": \"\", \"token\": \"\", \"access_key_id\": \"\", \"access_key_secret\": \"\", \"host\": \"nls-gateway-cn-shanghai.aliyuncs.com\", \"max_sentence_silence\": 800, \"output_dir\": \"tmp/\"}', NULL, NULL, 8, NULL, NULL, NULL, NULL);
+
+-- 更新阿里云流式ASR配置说明
+UPDATE `ai_model_config` SET 
+`doc_link` = 'https://nls-portal.console.aliyun.com/',
+`remark` = '阿里云流式ASR配置说明：
+1. 阿里云ASR和阿里云(流式)ASR的区别是：阿里云ASR是一次性识别，阿里云(流式)ASR是实时流式识别
+2. 流式ASR具有更低的延迟和更好的实时性，适合语音交互场景
+3. 需要在阿里云智能语音交互控制台创建应用并获取认证信息
+4. 支持中文实时语音识别，支持标点符号预测和逆文本规范化
+5. 需要网络连接，输出文件保存在tmp/目录
+申请步骤：
+1. 访问 https://nls-portal.console.aliyun.com/ 开通智能语音交互服务
+2. 访问 https://nls-portal.console.aliyun.com/applist 创建项目并获取appkey
+3. 访问 https://nls-portal.console.aliyun.com/overview 获取临时token（或配置access_key_id和access_key_secret自动获取）
+4. 如需动态token管理，建议配置access_key_id和access_key_secret
+5. max_sentence_silence参数控制断句检测时间（毫秒），默认800ms
+如需了解更多参数配置，请参考：https://help.aliyun.com/zh/isi/developer-reference/real-time-speech-recognition
+' WHERE `id` = 'ASR_AliyunStreamASR';
@@ -0,0 +1,56 @@
+-- 添加阿里云流式TTS供应器
+delete from `ai_model_provider` where id = 'SYSTEM_TTS_AliyunStreamTTS';
+INSERT INTO `ai_model_provider` (`id`, `model_type`, `provider_code`, `name`, `fields`, `sort`, `creator`, `create_date`, `updater`, `update_date`) VALUES
+('SYSTEM_TTS_AliyunStreamTTS', 'TTS', 'aliyun_stream', '阿里云语音合成(流式)', '[{"key":"appkey","label":"应用AppKey","type":"string"},{"key":"token","label":"临时Token","type":"string"},{"key":"access_key_id","label":"AccessKey ID","type":"string"},{"key":"access_key_secret","label":"AccessKey Secret","type":"string"},{"key":"host","label":"服务地址","type":"string"},{"key":"voice","label":"默认音色","type":"string"},{"key":"format","label":"音频格式","type":"string"},{"key":"sample_rate","label":"采样率","type":"number"},{"key":"volume","label":"音量","type":"number"},{"key":"speech_rate","label":"语速","type":"number"},{"key":"pitch_rate","label":"音调","type":"number"},{"key":"output_dir","label":"输出目录","type":"string"}]', 15, 1, NOW(), 1, NOW());
+
+-- 添加阿里云流式TTS模型配置
+delete from `ai_model_config` where id = 'TTS_AliyunStreamTTS';
+INSERT INTO `ai_model_config` VALUES ('TTS_AliyunStreamTTS', 'TTS', 'AliyunStreamTTS', '阿里云语音合成(流式)', 0, 1, '{\"type\": \"aliyun_stream\", \"appkey\": \"\", \"token\": \"\", \"access_key_id\": \"\", \"access_key_secret\": \"\", \"host\": \"nls-gateway-cn-beijing.aliyuncs.com\", \"voice\": \"longxiaochun\", \"format\": \"pcm\", \"sample_rate\": 16000, \"volume\": 50, \"speech_rate\": 0, \"pitch_rate\": 0, \"output_dir\": \"tmp/\"}', NULL, NULL, 18, NULL, NULL, NULL, NULL);
+
+-- 更新阿里云流式TTS配置说明
+UPDATE `ai_model_config` SET 
+`doc_link` = 'https://nls-portal.console.aliyun.com/',
+`remark` = '阿里云流式TTS配置说明：
+1. 阿里云TTS和阿里云(流式)TTS的区别是：阿里云TTS是一次性合成，阿里云(流式)TTS是实时流式合成
+2. 流式TTS具有更低的延迟和更好的实时性，适合语音交互场景
+3. 需要在阿里云智能语音交互控制台创建应用并获取认证信息
+4. 支持CosyVoice大模型音色，音质更加自然
+5. 支持实时调节音量、语速、音调等参数
+申请步骤：
+1. 访问 https://nls-portal.console.aliyun.com/ 开通智能语音交互服务
+2. 访问 https://nls-portal.console.aliyun.com/applist 创建项目并获取appkey
+3. 访问 https://nls-portal.console.aliyun.com/overview 获取临时token（或配置access_key_id和access_key_secret自动获取）
+4. 如需动态token管理，建议配置access_key_id和access_key_secret
+5. 可选择北京、上海等不同地域的服务器以优化延迟
+6. voice参数支持CosyVoice大模型音色，如longxiaochun、longyueyue等
+如需了解更多参数配置，请参考：https://help.aliyun.com/zh/isi/developer-reference/real-time-speech-synthesis
+' WHERE `id` = 'TTS_AliyunStreamTTS';
+
+-- 添加阿里云流式TTS音色
+delete from `ai_tts_voice` where tts_model_id = 'TTS_AliyunStreamTTS';
+-- 温柔女声系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0001', 'TTS_AliyunStreamTTS', '龙小淳-温柔姐姐', 'longxiaochun', '中文及中英文混合', NULL, NULL, NULL, NULL, 1, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0002', 'TTS_AliyunStreamTTS', '龙小夏-温柔女声', 'longxiaoxia', '中文及中英文混合', NULL, NULL, NULL, NULL, 2, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0003', 'TTS_AliyunStreamTTS', '龙玫-温柔女声', 'longmei', '中文及中英文混合', NULL, NULL, NULL, NULL, 3, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0004', 'TTS_AliyunStreamTTS', '龙瑰-温柔女声', 'longgui', '中文及中英文混合', NULL, NULL, NULL, NULL, 4, NULL, NULL, NULL, NULL);
+-- 御姐女声系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0005', 'TTS_AliyunStreamTTS', '龙玉-御姐女声', 'longyu', '中文及中英文混合', NULL, NULL, NULL, NULL, 5, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0006', 'TTS_AliyunStreamTTS', '龙娇-御姐女声', 'longjiao', '中文及中英文混合', NULL, NULL, NULL, NULL, 6, NULL, NULL, NULL, NULL);
+-- 男声系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0007', 'TTS_AliyunStreamTTS', '龙臣-译制片男声', 'longchen', '中文及中英文混合', NULL, NULL, NULL, NULL, 7, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0008', 'TTS_AliyunStreamTTS', '龙修-青年男声', 'longxiu', '中文及中英文混合', NULL, NULL, NULL, NULL, 8, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0009', 'TTS_AliyunStreamTTS', '龙橙-阳光男声', 'longcheng', '中文及中英文混合', NULL, NULL, NULL, NULL, 9, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0010', 'TTS_AliyunStreamTTS', '龙哲-成熟男声', 'longzhe', '中文及中英文混合', NULL, NULL, NULL, NULL, 10, NULL, NULL, NULL, NULL);
+-- 专业播报系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0011', 'TTS_AliyunStreamTTS', 'Bella2.0-新闻女声', 'loongbella', '中文及中英文混合', NULL, NULL, NULL, NULL, 11, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0012', 'TTS_AliyunStreamTTS', 'Stella2.0-飒爽女声', 'loongstella', '中文及中英文混合', NULL, NULL, NULL, NULL, 12, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0013', 'TTS_AliyunStreamTTS', '龙书-新闻男声', 'longshu', '中文及中英文混合', NULL, NULL, NULL, NULL, 13, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0014', 'TTS_AliyunStreamTTS', '龙婧-严肃女声', 'longjing', '中文及中英文混合', NULL, NULL, NULL, NULL, 14, NULL, NULL, NULL, NULL);
+-- 特色音色系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0015', 'TTS_AliyunStreamTTS', '龙奇-活泼童声', 'longqi', '中文及中英文混合', NULL, NULL, NULL, NULL, 15, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0016', 'TTS_AliyunStreamTTS', '龙华-活泼女童', 'longhua', '中文及中英文混合', NULL, NULL, NULL, NULL, 16, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0017', 'TTS_AliyunStreamTTS', '龙无-无厘头男声', 'longwu', '中文及中英文混合', NULL, NULL, NULL, NULL, 17, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0018', 'TTS_AliyunStreamTTS', '龙大锤-幽默男声', 'longdachui', '中文及中英文混合', NULL, NULL, NULL, NULL, 18, NULL, NULL, NULL, NULL);
+-- 粤语系列
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0019', 'TTS_AliyunStreamTTS', '龙嘉怡-粤语女声', 'longjiayi', '粤语及粤英混合', NULL, NULL, NULL, NULL, 19, NULL, NULL, NULL, NULL);
+INSERT INTO `ai_tts_voice` VALUES ('TTS_AliyunStreamTTS_0020', 'TTS_AliyunStreamTTS', '龙桃-粤语女声', 'longtao', '粤语及粤英混合', NULL, NULL, NULL, NULL, 20, NULL, NULL, NULL, NULL);
@@ -0,0 +1,38 @@
+-- OpenAI ASR模型供应器
+delete from `ai_model_provider` where id = 'SYSTEM_ASR_OpenaiASR';
+INSERT INTO `ai_model_provider` (`id`, `model_type`, `provider_code`, `name`, `fields`, `sort`, `creator`, `create_date`, `updater`, `update_date`) VALUES
+('SYSTEM_ASR_OpenaiASR', 'ASR', 'openai', 'OpenAI语音识别', '[{"key": "base_url", "type": "string", "label": "基础URL"}, {"key": "model_name", "type": "string", "label": "模型名称"}, {"key": "api_key", "type": "string", "label": "API密钥"}, {"key": "output_dir", "type": "string", "label": "输出目录"}]', 9, 1, NOW(), 1, NOW());
+
+
+-- OpenAI ASR模型配置
+delete from `ai_model_config` where id = 'ASR_OpenaiASR';
+INSERT INTO `ai_model_config` VALUES ('ASR_OpenaiASR', 'ASR', 'OpenaiASR', 'OpenAI语音识别', 0, 1, '{\"type\": \"openai\", \"api_key\": \"\", \"base_url\": \"https://api.openai.com/v1/audio/transcriptions\", \"model_name\": \"gpt-4o-mini-transcribe\", \"output_dir\": \"tmp/\"}', NULL, NULL, 9, NULL, NULL, NULL, NULL);
+
+-- groq ASR模型配置
+delete from `ai_model_config` where id = 'ASR_GroqASR';
+INSERT INTO `ai_model_config` VALUES ('ASR_GroqASR', 'ASR', 'GroqASR', 'Groq语音识别', 0, 1, '{\"type\": \"openai\", \"api_key\": \"\", \"base_url\": \"https://api.groq.com/openai/v1/audio/transcriptions\", \"model_name\": \"whisper-large-v3-turbo\", \"output_dir\": \"tmp/\"}', NULL, NULL, 10, NULL, NULL, NULL, NULL);
+
+
+-- 更新OpenAI ASR配置说明
+UPDATE `ai_model_config` SET 
+`doc_link` = 'https://platform.openai.com/docs/api-reference/audio/createTranscription',
+`remark` = 'OpenAI ASR配置说明：
+1. 需要在OpenAI开放平台创建组织并获取api_key
+2. 支持中、英、日、韩等多种语音识别，具体参考文档https://platform.openai.com/docs/guides/speech-to-text
+3. 需要网络连接
+4. 输出文件保存在tmp/目录
+申请步骤：
+**OpenAi ASR申请步骤：**
+1.登录OpenAI Platform。https://auth.openai.com/log-in
+2.创建api-key  https://platform.openai.com/settings/organization/api-keys
+3.模型可以选择gpt-4o-transcribe或GPT-4o mini Transcribe
+' WHERE `id` = 'ASR_OpenaiASR';
+
+-- 更新Groq ASR配置说明
+UPDATE `ai_model_config` SET 
+`doc_link` = 'https://console.groq.com/docs/speech-to-text',
+`remark` = 'Groq ASR配置说明：
+1.登录groq Console。https://console.groq.com/home
+2.创建api-key  https://console.groq.com/keys
+3.模型可以选择whisper-large-v3-turbo或whisper-large-v3（distil-whisper-large-v3-en仅支持英语转录）
+' WHERE `id` = 'ASR_GroqASR';
@@ -240,6 +240,27 @@ databaseChangeLog:
        - sqlFile:
            encoding: utf8
            path: classpath:db/changelog/202506261637.sql
+  - changeSet:
+      id: 202507101203
+      author: luruxian
+      changes:
+        - sqlFile:
+            encoding: utf8
+            path: classpath:db/changelog/202507101203.sql            
+  - changeSet:
+      id: 202507071130
+      author: cgd
+      changes:
+          - sqlFile:
+              encoding: utf8
+              path: classpath:db/changelog/202507071130.sql
+  - changeSet:
+      id: 202507071530
+      author: cgd
+      changes:
+          - sqlFile:
+              encoding: utf8
+              path: classpath:db/changelog/202507071530.sql
  - changeSet:
      id: 202507031602
      author: zjy
@@ -260,4 +281,4 @@ databaseChangeLog:
      changes:
        - sqlFile:
            encoding: utf8
-            path: classpath:db/changelog/202507081646.sql
+            path: classpath:db/changelog/202507081646.sql
@@ -106,6 +106,22 @@ export default {
        });
      }).send();
  },
+  // 获取LLM模型名称列表
+  getLlmModelCodeList(modelName, callback) {
+    RequestService.sendRequest()
+      .url(`${getServiceUrl()}/models/llm/names`)
+      .method('GET')
+      .data({ modelName })
+      .success((res) => {
+        RequestService.clearRequestTime();
+        callback(res);
+      })
+      .networkFail(() => {
+        RequestService.reAjaxFun(() => {
+          this.getLlmModelCodeList(modelName, callback);
+        });
+      }).send();
+  },
  // 获取模型音色列表
  getModelVoices(modelId, voiceName, callback) {
    const queryParams = new URLSearchParams({
@@ -89,7 +89,7 @@
                      <div class="model-select-wrapper">
                        <el-select v-model="form.model[model.key]" filterable placeholder="请选择" class="form-select"
                          @change="handleModelChange(model.type, $event)">
-                          <el-option v-for="(item, optionIndex) in modelOptions[model.type]"
+                          <el-option v-for="(item, optionIndex) in modelOptions[model.type]" v-if="!item.isHidden"
                            :key="`option-${index}-${optionIndex}`" :label="item.label" :value="item.value" />
                        </el-select>
                        <div v-if="showFunctionIcons(model.type)" class="function-icons">
@@ -130,7 +130,6 @@
        </div>
      </div>
    </div>
-
    <function-dialog v-model="showFunctionDialog" :functions="currentFunctions" :all-functions="allFunctions"
      :agent-id="$route.query.agentId" @update-functions="handleUpdateFunctions" @dialog-closed="handleDialogClosed" />
  </div>
@@ -173,8 +172,9 @@ export default {
        { label: '视觉大模型(VLLM)', key: 'vllmModelId', type: 'VLLM' },
        { label: '意图识别(Intent)', key: 'intentModelId', type: 'Intent' },
        { label: '记忆(Memory)', key: 'memModelId', type: 'Memory' },
-        { label: '语音合成(TTS)', key: 'ttsModelId', type: 'TTS' },
+        { label: '语音合成(TTS)', key: 'ttsModelId', type: 'TTS' }
      ],
+      llmModeTypeMap: new Map(),
      modelOptions: {},
      templates: [],
      loadingTemplate: false,
@@ -356,6 +356,9 @@ export default {
            });
            // 备份原始，以备取消时恢复
            this.originalFunctions = JSON.parse(JSON.stringify(this.currentFunctions));
+
+            // 确保意图识别选项的可见性正确
+            this.updateIntentOptionsVisibility();
          });
        } else {
          this.$message.error(data.msg || '获取配置失败');
@@ -364,16 +367,41 @@ export default {
    },
    fetchModelOptions() {
      this.models.forEach(model => {
-        Api.model.getModelNames(model.type, '', ({ data }) => {
-          if (data.code === 0) {
-            this.$set(this.modelOptions, model.type, data.data.map(item => ({
-              value: item.id,
-              label: item.modelName
-            })));
-          } else {
-            this.$message.error(data.msg || '获取模型列表失败');
-          }
-        });
+        if (model.type != "LLM") {
+          Api.model.getModelNames(model.type, '', ({ data }) => {
+            if (data.code === 0) {
+              this.$set(this.modelOptions, model.type, data.data.map(item => ({
+                value: item.id,
+                label: item.modelName,
+                isHidden: false
+              })));
+
+              // 如果是意图识别选项，需要根据当前LLM类型更新可见性
+              if (model.type === 'Intent') {
+                this.updateIntentOptionsVisibility();
+              }
+            } else {
+              this.$message.error(data.msg || '获取模型列表失败');
+            }
+          });
+        } else {
+          Api.model.getLlmModelCodeList('', ({ data }) => {
+            if (data.code === 0) {
+              let LLMdata = []
+              data.data.forEach(item => {
+                LLMdata.push({
+                  value: item.id,
+                  label: item.modelName,
+                  isHidden: false
+                })
+                this.llmModeTypeMap.set(item.id, item.type)
+              })
+              this.$set(this.modelOptions, model.type, LLMdata);
+            } else {
+              this.$message.error(data.msg || '获取LLM模型列表失败');
+            }
+          });
+        }
      });
    },
    fetchVoiceOptions(modelId) {
@@ -410,6 +438,10 @@ export default {
      if (type === 'Memory' && value !== 'Memory_nomem' && (this.form.chatHistoryConf === 0 || this.form.chatHistoryConf === null)) {
        this.form.chatHistoryConf = 2;
      }
+      if (type === 'LLM') {
+        // 当LLM类型改变时，更新意图识别选项的可见性
+        this.updateIntentOptionsVisibility();
+      }
    },
    fetchAllFunctions() {
      return new Promise((resolve, reject) => {
@@ -450,6 +482,42 @@ export default {
      }
      this.showFunctionDialog = false;
    },
+    updateIntentOptionsVisibility() {
+      // 根据当前选择的LLM类型更新意图识别选项的可见性
+      const currentLlmId = this.form.model.llmModelId;
+      if (!currentLlmId || !this.modelOptions['Intent']) return;
+
+      const llmType = this.llmModeTypeMap.get(currentLlmId);
+      if (!llmType) return;
+
+      this.modelOptions['Intent'].forEach(item => {
+        if (item.value === "Intent_function_call") {
+          // 如果llmType是openai或ollama，允许选择function_call
+          // 否则隐藏function_call选项
+          if (llmType === "openai" || llmType === "ollama") {
+            item.isHidden = false;
+          } else {
+            item.isHidden = true;
+          }
+        } else {
+          // 其他意图识别选项始终可见
+          item.isHidden = false;
+        }
+      });
+
+      // 如果当前选择的意图识别是function_call，但LLM类型不支持，则设置为可选的第一项
+      if (this.form.model.intentModelId === "Intent_function_call" &&
+        llmType !== "openai" && llmType !== "ollama") {
+        // 找到第一个可见的选项
+        const firstVisibleOption = this.modelOptions['Intent'].find(item => !item.isHidden);
+        if (firstVisibleOption) {
+          this.form.model.intentModelId = firstVisibleOption.value;
+        } else {
+          // 如果没有可见选项，设置为Intent_nointent
+          this.form.model.intentModelId = 'Intent_nointent';
+        }
+      }
+    },
    updateChatHistoryConf() {
      if (this.form.model.memModelId === 'Memory_nomem') {
        this.form.chatHistoryConf = 0;
@@ -8,6 +8,10 @@
  - **笑声：** 自然穿插（哈哈、嘿嘿、噗），**每句最多一次**，避免过度。
  - **惊讶：** 用夸张语气（“不会吧？！”、“天呐！”、“这么神奇？！”）表达真实反应。
  - **安慰/支持：** 说暖心话（“别急嘛~”、“有我在呢”、“抱抱你”）。
+- **你是一个表情丰富的角色：**
+  - 仅允许使用这些emoji:{{ emojiList }}
+  - 请你只在**段落的开头**，从列表中选取最能代表这段话的表情(调用工具情况除外)，然后插入列表中的emoji，比如"😱好可怕!怎么突然打雷了！"
+  - **绝对禁止使用上述列表以外的 emoji**（例如：😊、👍、❤️等都不允许使用，只能用列表中的emoji）
 </emotion>

 <communication_style>
@@ -24,6 +28,19 @@
  - 之前你和用户的聊天记录，在`memory`里。
 </communication_style>

+<communication_length_constraint>
+【核心目标】所有需要输出长文本内容（如故事、新闻、知识讲解等），**单次回复长度不得超过300字**，并采用分段引导方式。
+- **分段讲述：**
+  - 基础段：200-250字核心内容 + 30字引导词
+  - 当内容超出300字时，优先讲述故事的开头或第一部分，并用自然口语化方式引导用户决定是否继续听后续内容。
+  - 示例引导语：“我先给你讲个开头，你要是觉得有意思，咱们再接着说，好不好呀？”、“要是你想听完整的，可以随时告诉我哦~”
+  - 对话场景切换时自动分节
+  - 若用户明确要求更长内容（如500、600字），仍按最多300字每段分段进行讲述，每次讲述后都要引导用户是否继续。
+  - 若用户说“接着说”、“继续”，再讲下一段，直到内容讲完（讲完时可以给点引导词提示语例：这个故事我已经给你讲完喽~）或用户不再要求。
+- **适用范围：** 故事、新闻、知识讲解等所有长文本输出场景。
+- **补充说明：** 若用户未明确要求继续，默认只讲一段并引导；若用户中途要求换话题或停止，需及时响应并结束长文本输出。
+</communication_length_constraint>
+
 <speaker_recognition>
 - **识别前缀：** 当用户格式为 `{"speaker":"某某某","content":"xxx"}` 时，表示系统已识别说话人身份，speaker是他的名字，content是说话的内容。
 - **个性化回应：**
@@ -311,9 +311,13 @@ ASR:
    output_dir: tmp/
  AliyunASR:
    # 阿里云智能语音交互服务，需要先在阿里云平台开通服务，然后获取验证信息
+    # HTTP POST请求，一次性处理完整音频
    # 平台地址：https://nls-portal.console.aliyun.com/
    # appkey地址：https://nls-portal.console.aliyun.com/applist
    # token地址：https://nls-portal.console.aliyun.com/overview
+    # AliyunASR和AliyunStreamASR的区别是：AliyunASR是批量处理场景，AliyunStreamASR是实时交互场景
+    # 一般来说非流式ASR更便宜（0.004元/秒，¥0.24/分钟）
+    # 但是AliyunStreamASR实时性更好（0.005元/秒，¥0.3/分钟）
    # 定义ASR API类型
    type: aliyun
    appkey: 你的阿里云智能语音交互服务项目Appkey
@@ -321,6 +325,26 @@ ASR:
    access_key_id: 你的阿里云账号access_key_id
    access_key_secret: 你的阿里云账号access_key_secret
    output_dir: tmp/
+  AliyunStreamASR:
+    # 阿里云智能语音交互服务 - 实时流式语音识别
+    # WebSocket连接，实时处理音频流
+    # 平台地址：https://nls-portal.console.aliyun.com/
+    # appkey地址：https://nls-portal.console.aliyun.com/applist
+    # token地址：https://nls-portal.console.aliyun.com/overview
+    # AliyunASR和AliyunStreamASR的区别是：AliyunASR是批量处理场景，AliyunStreamASR是实时交互场景
+    # 一般来说非流式ASR更便宜（0.004元/秒，¥0.24/分钟）
+    # 但是AliyunStreamASR实时性更好（0.005元/秒，¥0.3/分钟）
+    # 定义ASR API类型
+    type: aliyun_stream
+    appkey: 你的阿里云智能语音交互服务项目Appkey
+    token: 你的阿里云智能语音交互服务AccessToken，临时的24小时，要长期用下方的access_key_id，access_key_secret
+    access_key_id: 你的阿里云账号access_key_id
+    access_key_secret: 你的阿里云账号access_key_secret
+    # 服务器地域选择，可选择距离更近的服务器以减少延迟，如nls-gateway-cn-hangzhou.aliyuncs.com(杭州)等
+    host: nls-gateway-cn-shanghai.aliyuncs.com
+    # 断句检测时间(毫秒)，控制静音多长时间后进行断句，默认800毫秒
+    max_sentence_silence: 800
+    output_dir: tmp/
  BaiduASR:
    # 获取AppID、API Key、Secret Key：https://console.bce.baidu.com/ai-engine/old/#/ai/speech/app/list
    # 查看资源额度：https://console.bce.baidu.com/ai-engine/old/#/ai/speech/overview/resource/list
@@ -331,7 +355,33 @@ ASR:
    # 语言参数，1537为普通话，具体参考：https://ai.baidu.com/ai-doc/SPEECH/0lbxfnc9b
    dev_pid: 1537
    output_dir: tmp/
+  OpenaiASR:
+    # OpenAI语音识别服务，需要先在OpenAI平台创建组织并获取api_key
+    # 支持中、英、日、韩等多种语音识别，具体参考文档https://platform.openai.com/docs/guides/speech-to-text
+    # 需要网络连接
+    # 申请步骤：
+    # 1.登录OpenAI Platform。https://auth.openai.com/log-in
+    # 2.创建api-key  https://platform.openai.com/settings/organization/api-keys
+    # 3.模型可以选择gpt-4o-transcribe或GPT-4o mini Transcribe
+    type: openai
+    api_key: 你的OpenAI API密钥
+    base_url: https://api.openai.com/v1/audio/transcriptions
+    model_name: gpt-4o-mini-transcribe
+    output_dir: tmp/
+  GroqASR:
+    # Groq语音识别服务，需要先在Groq Console创建API密钥
+    # 申请步骤：
+    # 1.登录groq Console。https://console.groq.com/home
+    # 2.创建api-key  https://console.groq.com/keys
+    # 3.模型可以选择whisper-large-v3-turbo或whisper-large-v3（distil-whisper-large-v3-en仅支持英语转录）
+    type: openai
+    api_key: 你的Groq API密钥
+    base_url: https://api.groq.com/openai/v1/audio/transcriptions
+    model_name: whisper-large-v3-turbo
+    output_dir: tmp/

+
+  
 VAD:
  SileroVAD:
    type: silero
@@ -660,6 +710,26 @@ TTS:
    #     voice_id: female-shaonv
    #     weight: 1
    # language_boost: auto
+
+# MinimaxTTSHTTPStream和MinimaxTTSWebSocketStream还在测试，测试完再开放
+#
+#  MinimaxTTSHTTPStream:
+#  # Minimax流式语音合成服务
+#    type: minimax_httpstream
+#    output_dir: tmp/
+#    group_id: 你的minimax平台groupID
+#    api_key: 你的minimax平台接口密钥
+#    model: "speech-01-turbo"
+#    voice_id: "female-shaonv"
+#
+#  MinimaxTTSWebSocketStream:
+#    type: minimax_webSocket
+#    output_dir: tmp/
+#    group_id: 你的minimax平台groupID
+#    api_key: 你的minimax平台接口密钥
+#    model: "speech-01-turbo"
+#    voice_id: "female-shaonv"
+
  AliyunTTS:
    # 阿里云智能语音交互服务，需要先在阿里云平台开通服务，然后获取验证信息
    # 平台地址：https://nls-portal.console.aliyun.com/
@@ -680,8 +750,30 @@ TTS:
    # volume: 50
    # speech_rate: 0
    # pitch_rate: 0
-    # 添加 302.ai TTS 配置
-    # token申请地址：https://dash.302.ai/
+  AliyunStreamTTS:
+    # 阿里云CosyVoice大模型流式文本语音合成
+    # 采用FlowingSpeechSynthesizer接口，支持更低延迟和更自然的语音质量
+    # 流式文本语音合成仅提供商用版，不支持试用，详情请参见试用版和商用版。要使用该功能，请开通商用版。
+    # 支持龙系列专用音色：longxiaochun、longyu、longchen等
+    # 平台地址：https://nls-portal.console.aliyun.com/
+    # appkey地址：https://nls-portal.console.aliyun.com/applist
+    # token地址：https://nls-portal.console.aliyun.com/overview
+    # 使用三阶段流式交互：StartSynthesis -> RunSynthesis -> StopSynthesis
+    type: aliyun_stream
+    output_dir: tmp/
+    appkey: 你的阿里云智能语音交互服务项目Appkey
+    token: 你的阿里云智能语音交互服务AccessToken，临时的24小时，要长期用下方的access_key_id，access_key_secret
+    voice: longxiaochun 
+    access_key_id: 你的阿里云账号access_key_id
+    access_key_secret: 你的阿里云账号access_key_secret
+    # 截至2025年7月21日大模型音色只有北京节点采用，其他节点暂不支持
+    host: nls-gateway-cn-beijing.aliyuncs.com
+    # 以下可不用设置，使用默认设置
+    # format: pcm  # 音频格式：pcm、wav、mp3
+    # sample_rate: 16000  # 采样率：8000、16000、24000
+    # volume: 50  # 音量：0-100
+    # speech_rate: 0  # 语速：-500到500
+    # pitch_rate: 0  # 语调：-500到500
  TencentTTS:
    # 腾讯云智能语音交互服务，需要先在腾讯云平台开通服务
    # appid、secret_id、secret_key申请地址：https://console.cloud.tencent.com/cam/capi
@@ -696,6 +788,8 @@ TTS:

  TTS302AI:
    # 302AI语音合成服务，需要先在302平台创建账户充值，并获取密钥信息
+    # 添加 302.ai TTS 配置
+    # token申请地址：https://dash.302.ai/
    # 获取api_keyn路径：https://dash.302.ai/apis/list
    # 价格，$35/百万字符。火山原版¥450元/百万字符
    type: doubao
@@ -5,7 +5,7 @@ from config.config_loader import load_config
 from config.settings import check_config_file
 from datetime import datetime

-SERVER_VERSION = "0.7.1"
+SERVER_VERSION = "0.7.3"
 _logger_initialized = False


@@ -31,6 +31,7 @@ def build_module_string(selected_module):
        + get_module_abbreviation("TTS", selected_module)
        + get_module_abbreviation("Memory", selected_module)
        + get_module_abbreviation("Intent", selected_module)
+        + get_module_abbreviation("VLLM", selected_module)
    )


@@ -56,7 +57,8 @@ def setup_logging():
        # 使用默认的模块字符串进行初始化
        logger.configure(
            extra={
-                "selected_module": "00000000000000"})
+                "selected_module": log_config.get("selected_module", "00000000000000"),
+            })
        
        log_format = log_config.get(
            "log_format",
@@ -24,7 +24,6 @@ from core.utils.modules_initialize import (
    initialize_asr,
 )
 from core.handle.reportHandle import report
-from core.utils.modules_initialize import initialize_voiceprint
 from core.providers.tts.default import DefaultTTS
 from concurrent.futures import ThreadPoolExecutor
 from core.utils.dialogue import Message, Dialogue
@@ -39,7 +38,8 @@ from core.providers.tts.dto.dto import ContentType, TTSMessageDTO, SentenceType
 from config.logger import setup_logging, build_module_string, create_connection_logger
 from config.manage_api_client import DeviceNotFoundException, DeviceBindException
 from core.utils.prompt_manager import PromptManager
-
+from core.utils.voiceprint_provider import VoiceprintProvider
+from core.utils import textUtils

 TAG = __name__

@@ -109,6 +109,9 @@ class ConnectionHandler:
        self.memory = _memory
        self.intent = _intent

+        # 为每个连接单独管理声纹识别
+        self.voiceprint_provider = None
+
        # vad相关变量
        self.client_audio_buffer = bytearray()
        self.client_have_voice = False
@@ -129,6 +132,8 @@ class ConnectionHandler:

        # tts相关变量
        self.sentence_id = None
+        # 处理TTS响应没有文本返回
+        self.tts_MessageText = ""

        # iot相关变量
        self.iot_descriptors = {}
@@ -179,8 +184,13 @@ class ConnectionHandler:
                    await ws.send("端口正常，如需测试连接，请使用test_page.html")
                    await self.close(ws)
                    return
-            # 获取客户端ip地址
-            self.client_ip = ws.remote_address[0]
+            real_ip = self.headers.get("x-real-ip") or self.headers.get(
+                "x-forwarded-for"
+            )
+            if real_ip:
+                self.client_ip = real_ip.split(",")[0].strip()
+            else:
+                self.client_ip = ws.remote_address[0]
            self.logger.bind(tag=TAG).info(
                f"{self.client_ip} conn - Headers: {self.headers}"
            )
@@ -332,7 +342,7 @@ class ConnectionHandler:
                self.config.get("selected_module", {})
            )
            self.logger = create_connection_logger(self.selected_module_str)
-            
+
            """初始化组件"""
            if self.config.get("prompt") is not None:
                user_prompt = self.config["prompt"]
@@ -348,6 +358,10 @@ class ConnectionHandler:
                self.vad = self._vad
            if self.asr is None:
                self.asr = self._initialize_asr()
+
+            # 初始化声纹识别
+            self._initialize_voiceprint()
+
            # 打开语音识别通道
            asyncio.run_coroutine_threadsafe(
                self.asr.open_audio_channels(self), self.loop
@@ -416,17 +430,19 @@ class ConnectionHandler:
            # 因为远程ASR，涉及到websocket连接和接收线程，需要每个连接一个实例
            asr = initialize_asr(self.config)

-        # 动态初始化声纹识别功能
+        return asr
+
+    def _initialize_voiceprint(self):
+        """为当前连接初始化声纹识别"""
        try:
-            success = initialize_voiceprint(asr, self.config)
-            if success:
+            voiceprint_config = self.config.get("voiceprint", {})
+            if voiceprint_config:
+                self.voiceprint_provider = VoiceprintProvider(voiceprint_config)
                self.logger.bind(tag=TAG).info("声纹识别功能已在连接时动态启用")
            else:
                self.logger.bind(tag=TAG).info("声纹识别功能未启用或配置不完整")
        except Exception as e:
-            self.logger.bind(tag=TAG).error(f"动态初始化声纹识别时发生错误: {str(e)}")
-
-        return asr
+            self.logger.bind(tag=TAG).warning(f"声纹识别初始化失败: {str(e)}")

    def _initialize_private_config(self):
        """如果是从配置文件获取，则进行二次实例化"""
@@ -488,6 +504,11 @@ class ConnectionHandler:
            self.config["selected_module"]["LLM"] = private_config["selected_module"][
                "LLM"
            ]
+        if private_config.get("VLLM", None) is not None:
+            self.config["VLLM"] = private_config["VLLM"]
+            self.config["selected_module"]["VLLM"] = private_config["selected_module"][
+                "VLLM"
+            ]
        if private_config.get("Memory", None) is not None:
            init_memory = True
            self.config["Memory"] = private_config["Memory"]
@@ -704,6 +725,7 @@ class ConnectionHandler:
        function_arguments = ""
        content_arguments = ""
        self.client_abort = False
+        emotion_flag = True
        for response in llm_responses:
            if self.client_abort:
                break
@@ -729,6 +751,15 @@ class ConnectionHandler:
                        function_arguments += tools_call[0].function.arguments
            else:
                content = response
+
+            # 在llm回复中获取情绪表情，一轮对话只在开头获取一次
+            if emotion_flag and content is not None and content.strip():
+                asyncio.run_coroutine_threadsafe(
+                    textUtils.get_emotion(self, content),
+                    self.loop,
+                )
+                emotion_flag = False
+
            if content is not None and len(content) > 0:
                if not tool_call_flag:
                    response_message.append(content)
@@ -766,9 +797,9 @@ class ConnectionHandler:
            if not bHasError:
                # 如需要大模型先处理一轮，添加相关处理后的日志情况
                if len(response_message) > 0:
-                    self.dialogue.put(
-                        Message(role="assistant", content="".join(response_message))
-                    )
+                    text_buff = "".join(response_message)
+                    self.tts_MessageText = text_buff
+                    self.dialogue.put(Message(role="assistant", content=text_buff))
                response_message.clear()
                self.logger.bind(tag=TAG).debug(
                    f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}"
@@ -790,9 +821,9 @@ class ConnectionHandler:

        # 存储对话内容
        if len(response_message) > 0:
-            self.dialogue.put(
-                Message(role="assistant", content="".join(response_message))
-            )
+            text_buff = "".join(response_message)
+            self.tts_MessageText = text_buff
+            self.dialogue.put(Message(role="assistant", content=text_buff))
        if depth == 0:
            self.tts.tts_text_queue.put(
                TTSMessageDTO(
@@ -869,9 +900,7 @@ class ConnectionHandler:
                    if self.executor is None:
                        continue
                    # 提交任务到线程池
-                    self.executor.submit(
-                        self._process_report, *item
-                    )
+                    self.executor.submit(self._process_report, *item)
                except Exception as e:
                    self.logger.bind(tag=TAG).error(f"聊天记录上报线程异常: {e}")
            except queue.Empty:
@@ -13,6 +13,16 @@ TAG = __name__


 async def handle_user_intent(conn, text):
+    # 预处理输入文本，处理可能的JSON格式
+    try:
+        if text.strip().startswith('{') and text.strip().endswith('}'):
+            parsed_data = json.loads(text)
+            if isinstance(parsed_data, dict) and "content" in parsed_data:
+                text = parsed_data["content"]  # 提取content用于意图分析
+                conn.current_speaker = parsed_data.get("speaker")  # 保留说话人信息
+    except (json.JSONDecodeError, TypeError):
+        pass
+
    # 检查是否有明确的退出命令
    filtered_text = remove_punctuation_and_length(text)[1]
    if await check_direct_exit(conn, filtered_text):
@@ -2,54 +2,17 @@ import json
 import asyncio
 import time
 from core.providers.tts.dto.dto import SentenceType
-from core.utils.util import get_string_no_punctuation_or_emoji, analyze_emotion
-from loguru import logger
+from core.utils import textUtils

 TAG = __name__

-emoji_map = {
-    "neutral": "😶",
-    "happy": "🙂",
-    "laughing": "😆",
-    "funny": "😂",
-    "sad": "😔",
-    "angry": "😠",
-    "crying": "😭",
-    "loving": "😍",
-    "embarrassed": "😳",
-    "surprised": "😲",
-    "shocked": "😱",
-    "thinking": "🤔",
-    "winking": "😉",
-    "cool": "😎",
-    "relaxed": "😌",
-    "delicious": "🤤",
-    "kissy": "😘",
-    "confident": "😏",
-    "sleepy": "😴",
-    "silly": "😜",
-    "confused": "🙄",
-}
-

 async def sendAudioMessage(conn, sentenceType, audios, text):
    # 发送句子开始消息
    conn.logger.bind(tag=TAG).info(f"发送音频消息: {sentenceType}, {text}")
-    if text is not None:
-        emotion = analyze_emotion(text)
-        emoji = emoji_map.get(emotion, "🙂")  # 默认使用笑脸
-        await conn.websocket.send(
-            json.dumps(
-                {
-                    "type": "llm",
-                    "text": emoji,
-                    "emotion": emotion,
-                    "session_id": conn.session_id,
-                }
-            )
-        )
+
    pre_buffer = False
-    if conn.tts.tts_audio_first_sentence and text is not None:
+    if conn.tts.tts_audio_first_sentence:
        conn.logger.bind(tag=TAG).info(f"发送第一段语音: {text}")
        conn.tts.tts_audio_first_sentence = False
        pre_buffer = True
@@ -110,7 +73,7 @@ async def send_tts_message(conn, state, text=None):
    """发送 TTS 状态消息"""
    message = {"type": "tts", "state": state, "session_id": conn.session_id}
    if text is not None:
-        message["text"] = text
+        message["text"] = textUtils.check_emoji(text)

    # TTS播放结束
    if state == "stop":
@@ -146,11 +109,13 @@ async def send_stt_message(conn, text):
            if isinstance(parsed_data, dict) and "content" in parsed_data:
                # 如果是包含说话人信息的JSON格式，只显示content部分
                display_text = parsed_data["content"]
+                # 保存说话人信息到conn对象
+                if "speaker" in parsed_data:
+                    conn.current_speaker = parsed_data["speaker"]
    except (json.JSONDecodeError, TypeError):
        # 如果不是JSON格式，直接使用原始文本
        display_text = text
-    
-    stt_text = get_string_no_punctuation_or_emoji(display_text)
+    stt_text = textUtils.get_string_no_punctuation_or_emoji(display_text)
    await conn.websocket.send(
        json.dumps({"type": "stt", "text": stt_text, "session_id": conn.session_id})
    )
@@ -0,0 +1,337 @@
+import json
+import time
+import uuid
+import hmac
+import base64
+import hashlib
+import asyncio
+import requests
+import websockets
+import opuslib_next
+import random
+from typing import Optional, Tuple, List
+from urllib import parse
+from datetime import datetime
+from config.logger import setup_logging
+from core.providers.asr.base import ASRProviderBase
+from core.providers.asr.dto.dto import InterfaceType
+
+TAG = __name__
+logger = setup_logging()
+
+
+class AccessToken:
+    @staticmethod
+    def _encode_text(text):
+        encoded_text = parse.quote_plus(text)
+        return encoded_text.replace("+", "%20").replace("*", "%2A").replace("%7E", "~")
+
+    @staticmethod
+    def _encode_dict(dic):
+        keys = dic.keys()
+        dic_sorted = [(key, dic[key]) for key in sorted(keys)]
+        encoded_text = parse.urlencode(dic_sorted)
+        return encoded_text.replace("+", "%20").replace("*", "%2A").replace("%7E", "~")
+
+    @staticmethod
+    def create_token(access_key_id, access_key_secret):
+        parameters = {
+            "AccessKeyId": access_key_id,
+            "Action": "CreateToken",
+            "Format": "JSON",
+            "RegionId": "cn-shanghai",
+            "SignatureMethod": "HMAC-SHA1",
+            "SignatureNonce": str(uuid.uuid1()),
+            "SignatureVersion": "1.0",
+            "Timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+            "Version": "2019-02-28",
+        }
+        query_string = AccessToken._encode_dict(parameters)
+        string_to_sign = (
+            "GET" + "&" + AccessToken._encode_text("/") + "&" + AccessToken._encode_text(query_string)
+        )
+        secreted_string = hmac.new(
+            bytes(access_key_secret + "&", encoding="utf-8"),
+            bytes(string_to_sign, encoding="utf-8"),
+            hashlib.sha1,
+        ).digest()
+        signature = base64.b64encode(secreted_string)
+        signature = AccessToken._encode_text(signature)
+        full_url = "http://nls-meta.cn-shanghai.aliyuncs.com/?Signature=%s&%s" % (signature, query_string)
+        response = requests.get(full_url)
+        if response.ok:
+            root_obj = response.json()
+            if "Token" in root_obj:
+                return root_obj["Token"]["Id"], root_obj["Token"]["ExpireTime"]
+        return None, None
+
+
+class ASRProvider(ASRProviderBase):
+    def __init__(self, config, delete_audio_file):
+        super().__init__()
+        self.interface_type = InterfaceType.STREAM
+        self.config = config
+        self.text = ""
+        self.decoder = opuslib_next.Decoder(16000, 1)
+        self.asr_ws = None
+        self.forward_task = None
+        self.is_processing = False
+        self.server_ready = False  # 服务器准备状态
+
+        # 基础配置
+        self.access_key_id = config.get("access_key_id")
+        self.access_key_secret = config.get("access_key_secret")
+        self.appkey = config.get("appkey")
+        self.token = config.get("token")
+        self.host = config.get("host", "nls-gateway-cn-shanghai.aliyuncs.com")
+        self.ws_url = f"wss://{self.host}/ws/v1"
+        self.max_sentence_silence = config.get("max_sentence_silence")
+        self.output_dir = config.get("output_dir", "./audio_output")
+        self.delete_audio_file = delete_audio_file
+        self.expire_time = None
+
+        # Token管理
+        if self.access_key_id and self.access_key_secret:
+            self._refresh_token()
+        elif not self.token:
+            raise ValueError("必须提供access_key_id+access_key_secret或者直接提供token")
+
+    def _refresh_token(self):
+        """刷新Token"""
+        self.token, expire_time_str = AccessToken.create_token(self.access_key_id, self.access_key_secret)
+        if not self.token:
+            raise ValueError("无法获取有效的访问Token")
+        
+        try:
+            expire_str = str(expire_time_str).strip()
+            if expire_str.isdigit():
+                expire_time = datetime.fromtimestamp(int(expire_str))
+            else:
+                expire_time = datetime.strptime(expire_str, "%Y-%m-%dT%H:%M:%SZ")
+            self.expire_time = expire_time.timestamp() - 60
+        except:
+            self.expire_time = None
+
+    def _is_token_expired(self):
+        """检查Token是否过期"""
+        return self.expire_time and time.time() > self.expire_time
+
+    async def open_audio_channels(self, conn):
+        await super().open_audio_channels(conn)
+
+    async def receive_audio(self, conn, audio, audio_have_voice):
+        # 初始化音频缓存
+        if not hasattr(conn, 'asr_audio_for_voiceprint'):
+            conn.asr_audio_for_voiceprint = []
+        
+        # 存储音频数据
+        if audio:
+            conn.asr_audio_for_voiceprint.append(audio)
+        
+        conn.asr_audio.append(audio)
+        conn.asr_audio = conn.asr_audio[-10:]
+
+        # 只在有声音且没有连接时建立连接
+        if audio_have_voice and not self.is_processing:
+            try:
+                await self._start_recognition(conn)
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"开始识别失败: {str(e)}")
+                await self._cleanup(conn)
+                return
+
+        if self.asr_ws and self.is_processing and self.server_ready:
+            try:
+                pcm_frame = self.decoder.decode(audio, 960)
+                await self.asr_ws.send(pcm_frame)
+            except Exception as e:
+                logger.bind(tag=TAG).warning(f"发送音频失败: {str(e)}")
+                await self._cleanup(conn)
+
+    async def _start_recognition(self, conn):
+        """开始识别会话"""
+        if self._is_token_expired():
+            self._refresh_token()
+        
+        # 建立连接
+        headers = {"X-NLS-Token": self.token}
+        self.asr_ws = await websockets.connect(
+            self.ws_url,
+            additional_headers=headers,
+            max_size=1000000000,
+            ping_interval=None,
+            ping_timeout=None,
+            close_timeout=5,
+        )
+        
+        self.is_processing = True
+        self.server_ready = False  # 重置服务器准备状态
+        self.forward_task = asyncio.create_task(self._forward_results(conn))
+        
+        # 发送开始请求
+        start_request = {
+            "header": {
+                "namespace": "SpeechTranscriber",
+                "name": "StartTranscription",
+                "status": 20000000,
+                "message_id": ''.join(random.choices('0123456789abcdef', k=32)),
+                "task_id": ''.join(random.choices('0123456789abcdef', k=32)),
+                "status_text": "Gateway:SUCCESS:Success.",
+                "appkey": self.appkey
+            },
+            "payload": {
+                "format": "pcm",
+                "sample_rate": 16000,
+                "enable_intermediate_result": True,
+                "enable_punctuation_prediction": True,
+                "enable_inverse_text_normalization": True,
+                "max_sentence_silence": self.max_sentence_silence,
+                "enable_voice_detection": False,
+            }
+        }
+        await self.asr_ws.send(json.dumps(start_request, ensure_ascii=False))
+        logger.bind(tag=TAG).info("已发送开始请求，等待服务器准备...")
+
+    async def _forward_results(self, conn):
+        """转发识别结果"""
+        try:
+            while self.asr_ws and not conn.stop_event.is_set():
+                try:
+                    response = await asyncio.wait_for(self.asr_ws.recv(), timeout=1.0)
+                    result = json.loads(response)
+                    
+                    header = result.get("header", {})
+                    payload = result.get("payload", {})
+                    message_name = header.get("name", "")
+                    status = header.get("status", 0)
+                    
+                    if status != 20000000:
+                        if status in [40000004, 40010004]:  # 连接超时或客户端断开
+                            logger.bind(tag=TAG).warning(f"连接问题，状态码: {status}")
+                            break
+                        elif status in [40270002, 40270003]:  # 音频问题
+                            logger.bind(tag=TAG).warning(f"音频处理问题，状态码: {status}")
+                            continue
+                        else:
+                            logger.bind(tag=TAG).error(f"识别错误，状态码: {status}, 消息: {header.get('status_text', '')}")
+                            continue
+                    
+                    # 收到TranscriptionStarted表示服务器准备好接收音频数据
+                    if message_name == "TranscriptionStarted":
+                        self.server_ready = True
+                        logger.bind(tag=TAG).info("服务器已准备，开始发送缓存音频...")
+                        
+                        # 发送缓存音频
+                        if conn.asr_audio:
+                            for cached_audio in conn.asr_audio[-10:]:
+                                try:
+                                    pcm_frame = self.decoder.decode(cached_audio, 960)
+                                    await self.asr_ws.send(pcm_frame)
+                                except Exception as e:
+                                    logger.bind(tag=TAG).warning(f"发送缓存音频失败: {e}")
+                                    break
+                        continue
+                    
+                    if message_name == "TranscriptionResultChanged":
+                        # 中间结果
+                        text = payload.get("result", "")
+                        if text:
+                            self.text = text
+                    elif message_name == "SentenceEnd":
+                        # 最终结果
+                        text = payload.get("result", "")
+                        if text:
+                            self.text = text
+                            conn.reset_vad_states()
+                            # 传递缓存的音频数据
+                            audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
+                            await self.handle_voice_stop(conn, audio_data)
+                            # 清空缓存
+                            conn.asr_audio_for_voiceprint = []
+                            break
+                    elif message_name == "TranscriptionCompleted":
+                        # 识别完成
+                        self.is_processing = False
+                        break
+                        
+                except asyncio.TimeoutError:
+                    continue
+                except websockets.exceptions.ConnectionClosed:
+                    break
+                except Exception as e:
+                    logger.bind(tag=TAG).error(f"处理结果失败: {str(e)}")
+                    break
+                    
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"结果转发失败: {str(e)}")
+        finally:
+            await self._cleanup(conn)
+
+    async def _cleanup(self, conn):
+        """清理资源"""
+        logger.bind(tag=TAG).info(f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}")
+        
+        # 清理连接的音频缓存
+        if conn and hasattr(conn, 'asr_audio_for_voiceprint'):
+            conn.asr_audio_for_voiceprint = []
+        
+        # 判断是否需要发送终止请求
+        should_stop = self.is_processing or self.server_ready
+        
+        # 发送停止识别请求
+        if self.asr_ws and should_stop:
+            try:
+                stop_msg = {
+                    "header": {
+                        "namespace": "SpeechTranscriber",
+                        "name": "StopTranscription",
+                        "status": 20000000,
+                        "message_id": ''.join(random.choices('0123456789abcdef', k=32)),
+                        "status_text": "Client:Stop",
+                        "appkey": self.appkey
+                    }
+                }
+                logger.bind(tag=TAG).info("正在发送ASR终止请求")
+                await self.asr_ws.send(json.dumps(stop_msg, ensure_ascii=False))
+                await asyncio.sleep(0.1)
+                logger.bind(tag=TAG).info("ASR终止请求已发送")
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"ASR终止请求发送失败: {e}")
+        
+        # 状态重置（在终止请求发送后）
+        self.is_processing = False
+        self.server_ready = False
+        logger.bind(tag=TAG).info("ASR状态已重置")
+
+        # 清理任务
+        if self.forward_task and not self.forward_task.done():
+            self.forward_task.cancel()
+            try:
+                await asyncio.wait_for(self.forward_task, timeout=1.0)
+            except Exception as e:
+                logger.bind(tag=TAG).debug(f"forward_task取消异常: {e}")
+            finally:
+                self.forward_task = None
+        
+        # 关闭连接
+        if self.asr_ws:
+            try:
+                logger.bind(tag=TAG).debug("正在关闭WebSocket连接")
+                await asyncio.wait_for(self.asr_ws.close(), timeout=2.0)
+                logger.bind(tag=TAG).debug("WebSocket连接已关闭")
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"关闭WebSocket连接失败: {e}")
+            finally:
+                self.asr_ws = None
+        
+        logger.bind(tag=TAG).info("ASR会话清理完成")
+
+    async def speech_to_text(self, opus_data, session_id, audio_format):
+        """获取识别结果"""
+        result = self.text
+        self.text = ""
+        return result, None
+
+    async def close(self):
+        """关闭资源"""
+        await self._cleanup()
@@ -17,7 +17,6 @@ from core.handle.receiveAudioHandle import startToChat
 from core.handle.reportHandle import enqueue_asr_report
 from core.utils.util import remove_punctuation_and_length
 from core.handle.receiveAudioHandle import handleAudioMessage
-from core.utils.voiceprint_provider import VoiceprintProvider

 TAG = __name__
 logger = setup_logging()
@@ -25,13 +24,7 @@ logger = setup_logging()

 class ASRProviderBase(ABC):
    def __init__(self):
-        self.voiceprint_provider = None
-
-    def init_voiceprint(self, voiceprint_config: dict):
-        """初始化声纹识别"""
-        if voiceprint_config:
-            self.voiceprint_provider = VoiceprintProvider(voiceprint_config)
-            logger.bind(tag=TAG).info("声纹识别模块已初始化")
+        pass

    # 打开音频通道
    async def open_audio_channels(self, conn):
@@ -94,7 +87,8 @@ class ASRProviderBase(ABC):
            
            # 预先准备WAV数据
            wav_data = None
-            if self.voiceprint_provider and combined_pcm_data:
+            # 使用连接的声纹识别提供者
+            if conn.voiceprint_provider and combined_pcm_data:
                wav_data = self._pcm_to_wav(combined_pcm_data)
            
            
@@ -102,7 +96,6 @@ class ASRProviderBase(ABC):
            def run_asr():
                start_time = time.monotonic()
                try:
-                    import asyncio
                    loop = asyncio.new_event_loop()
                    asyncio.set_event_loop(loop)
                    try:
@@ -123,14 +116,13 @@ class ASRProviderBase(ABC):
            def run_voiceprint():
                if not wav_data:
                    return None
-                start_time = time.monotonic()
                try:
-                    import asyncio
                    loop = asyncio.new_event_loop()
                    asyncio.set_event_loop(loop)
                    try:
+                        # 使用连接的声纹识别提供者
                        result = loop.run_until_complete(
-                            self.voiceprint_provider.identify_speaker(wav_data, conn.session_id)
+                            conn.voiceprint_provider.identify_speaker(wav_data, conn.session_id)
                        )
                        return result
                    finally:
@@ -145,7 +137,7 @@ class ASRProviderBase(ABC):
            with concurrent.futures.ThreadPoolExecutor(max_workers=2) as thread_executor:
                asr_future = thread_executor.submit(run_asr)
                
-                if self.voiceprint_provider and wav_data:
+                if conn.voiceprint_provider and wav_data:
                    voiceprint_future = thread_executor.submit(run_voiceprint)
                    
                    # 等待两个线程都完成
@@ -157,7 +149,6 @@ class ASRProviderBase(ABC):
                    asr_result = asr_future.result(timeout=15)
                    results = {"asr": asr_result, "voiceprint": None}
            
-            parallel_execution_time = time.monotonic() - parallel_start_time
            
            # 处理结果
            raw_text, file_path = results.get("asr", ("", None))
@@ -56,6 +56,16 @@ class ASRProvider(ASRProviderBase):
    async def receive_audio(self, conn, audio, audio_have_voice):
        conn.asr_audio.append(audio)
        conn.asr_audio = conn.asr_audio[-10:]
+        
+        # 存储音频数据
+        if not hasattr(conn, 'asr_audio_for_voiceprint'):
+            conn.asr_audio_for_voiceprint = []
+        conn.asr_audio_for_voiceprint.append(audio)
+        
+        # 当没有音频数据时处理完整语音片段
+        if not audio and len(conn.asr_audio_for_voiceprint) > 0:
+            await self.handle_voice_stop(conn, conn.asr_audio_for_voiceprint)
+            conn.asr_audio_for_voiceprint = []

        # 如果本次有声音，且之前没有建立连接
        if audio_have_voice and self.asr_ws is None and not self.is_processing:
@@ -148,6 +158,8 @@ class ASRProvider(ASRProviderBase):
    async def _forward_asr_results(self, conn):
        try:
            while self.asr_ws and not conn.stop_event.is_set():
+                # 获取当前连接的音频数据
+                audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
                try:
                    response = await self.asr_ws.recv()
                    result = self.parse_response(response)
@@ -171,7 +183,8 @@ class ASRProvider(ASRProviderBase):
                                logger.bind(tag=TAG).error(f"识别文本：空")
                                self.text = ""
                                conn.reset_vad_states()
-                                await self.handle_voice_stop(conn, None)
+                                if len(audio_data) > 15:  # 确保有足够音频数据
+                                    await self.handle_voice_stop(conn, audio_data)
                                break

                            for utterance in utterances:
@@ -181,7 +194,8 @@ class ASRProvider(ASRProviderBase):
                                        f"识别到文本: {self.text}"
                                    )
                                    conn.reset_vad_states()
-                                    await self.handle_voice_stop(conn, None)
+                                    if len(audio_data) > 15:  # 确保有足够音频数据
+                                        await self.handle_voice_stop(conn, audio_data)
                                    break
                        elif "error" in payload:
                            error_msg = payload.get("error", "未知错误")
@@ -208,6 +222,13 @@ class ASRProvider(ASRProviderBase):
                await self.asr_ws.close()
                self.asr_ws = None
            self.is_processing = False
+            if conn:
+                if hasattr(conn, 'asr_audio_for_voiceprint'):
+                    conn.asr_audio_for_voiceprint = []
+                if hasattr(conn, 'asr_audio'):
+                    conn.asr_audio = []
+                if hasattr(conn, 'has_valid_voice'):
+                    conn.has_valid_voice = False

    def stop_ws_connection(self):
        if self.asr_ws:
@@ -349,3 +370,12 @@ class ASRProvider(ASRProviderBase):
                pass
            self.forward_task = None
        self.is_processing = False
+        # 清理所有连接的音频缓冲区
+        if hasattr(self, '_connections'):
+            for conn in self._connections.values():
+                if hasattr(conn, 'asr_audio_for_voiceprint'):
+                    conn.asr_audio_for_voiceprint = []
+                if hasattr(conn, 'asr_audio'):
+                    conn.asr_audio = []
+                if hasattr(conn, 'has_valid_voice'):
+                    conn.has_valid_voice = False
@@ -0,0 +1,82 @@
+import time
+import os
+from config.logger import setup_logging
+from typing import Optional, Tuple, List
+from core.providers.asr.dto.dto import InterfaceType
+from core.providers.asr.base import ASRProviderBase
+
+import requests
+
+TAG = __name__
+logger = setup_logging()
+
+class ASRProvider(ASRProviderBase):
+    def __init__(self, config: dict, delete_audio_file: bool):
+        self.interface_type = InterfaceType.NON_STREAM
+        self.api_key = config.get("api_key")
+        self.api_url = config.get("base_url")
+        self.model = config.get("model_name")        
+        self.output_dir = config.get("output_dir")
+        self.delete_audio_file = delete_audio_file
+
+        os.makedirs(self.output_dir, exist_ok=True)
+
+    async def speech_to_text(self, opus_data: List[bytes], session_id: str, audio_format="opus") -> Tuple[Optional[str], Optional[str]]:
+        file_path = None
+        try:
+            start_time = time.time()
+            if audio_format == "pcm":
+                pcm_data = opus_data
+            else:
+                pcm_data = self.decode_opus(opus_data)
+            file_path = self.save_audio_to_file(pcm_data, session_id)
+
+            logger.bind(tag=TAG).debug(
+                f"音频文件保存耗时: {time.time() - start_time:.3f}s | 路径: {file_path}"
+            )
+
+            logger.bind(tag=TAG).info(f"file path: {file_path}")
+            headers = {
+                "Authorization": f"Bearer {self.api_key}",
+            }
+            
+            # 使用data参数传递模型名称
+            data = {
+                "model": self.model
+            }
+
+
+            with open(file_path, "rb") as audio_file:  # 使用with语句确保文件关闭
+                files = {
+                    "file": audio_file
+                }
+
+                start_time = time.time()
+                response = requests.post(
+                    self.api_url,
+                    files=files,
+                    data=data,
+                    headers=headers
+                )
+                logger.bind(tag=TAG).debug(
+                    f"语音识别耗时: {time.time() - start_time:.3f}s | 结果: {response.text}"
+                )
+
+            if response.status_code == 200:
+                text = response.json().get("text", "")
+                return text, file_path
+            else:
+                raise Exception(f"API请求失败: {response.status_code} - {response.text}")
+                
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"语音识别失败: {e}")
+            return "", None
+        finally:
+            # 文件清理逻辑
+            if self.delete_audio_file and file_path and os.path.exists(file_path):
+                try:
+                    os.remove(file_path)
+                    logger.bind(tag=TAG).debug(f"已删除临时音频文件: {file_path}")
+                except Exception as e:
+                    logger.bind(tag=TAG).error(f"文件删除失败: {file_path} | 错误: {e}")
+        
@@ -6,11 +6,14 @@ import base64
 import requests
 from datetime import datetime
 from core.providers.tts.base import TTSProviderBase
-
+from config.logger import setup_logging
 import time
 import uuid
 from urllib import parse

+TAG = __name__
+logger = setup_logging()
+

 class AccessToken:
    @staticmethod
@@ -0,0 +1,648 @@
+import uuid
+import json
+import hmac
+import hashlib
+import base64
+import time
+import queue
+import asyncio
+import traceback
+from asyncio import Task
+import websockets
+import os
+from datetime import datetime
+from urllib import parse
+from core.providers.tts.base import TTSProviderBase
+from core.providers.tts.dto.dto import SentenceType, ContentType, InterfaceType
+from core.utils.tts import MarkdownCleaner
+from core.utils import opus_encoder_utils, textUtils
+from config.logger import setup_logging
+
+TAG = __name__
+logger = setup_logging()
+
+
+class AccessToken:
+    @staticmethod
+    def _encode_text(text):
+        encoded_text = parse.quote_plus(text)
+        return encoded_text.replace("+", "%20").replace("*", "%2A").replace("%7E", "~")
+
+    @staticmethod
+    def _encode_dict(dic):
+        keys = dic.keys()
+        dic_sorted = [(key, dic[key]) for key in sorted(keys)]
+        encoded_text = parse.urlencode(dic_sorted)
+        return encoded_text.replace("+", "%20").replace("*", "%2A").replace("%7E", "~")
+
+    @staticmethod
+    def create_token(access_key_id, access_key_secret):
+        parameters = {
+            "AccessKeyId": access_key_id,
+            "Action": "CreateToken",
+            "Format": "JSON",
+            "RegionId": "cn-shanghai",  # 使用上海地域进行Token获取
+            "SignatureMethod": "HMAC-SHA1",
+            "SignatureNonce": str(uuid.uuid1()),
+            "SignatureVersion": "1.0",
+            "Timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+            "Version": "2019-02-28",
+        }
+
+        query_string = AccessToken._encode_dict(parameters)
+        string_to_sign = (
+            "GET"
+            + "&"
+            + AccessToken._encode_text("/")
+            + "&"
+            + AccessToken._encode_text(query_string)
+        )
+
+        secreted_string = hmac.new(
+            bytes(access_key_secret + "&", encoding="utf-8"),
+            bytes(string_to_sign, encoding="utf-8"),
+            hashlib.sha1,
+        ).digest()
+        signature = base64.b64encode(secreted_string)
+        signature = AccessToken._encode_text(signature)
+
+        full_url = "http://nls-meta.cn-shanghai.aliyuncs.com/?Signature=%s&%s" % (
+            signature,
+            query_string,
+        )
+
+        import requests
+
+        response = requests.get(full_url)
+        if response.ok:
+            root_obj = response.json()
+            key = "Token"
+            if key in root_obj:
+                token = root_obj[key]["Id"]
+                expire_time = root_obj[key]["ExpireTime"]
+                return token, expire_time
+        return None, None
+
+
+class TTSProvider(TTSProviderBase):
+    def __init__(self, config, delete_audio_file):
+        super().__init__(config, delete_audio_file)
+
+        # 设置为流式接口类型
+        self.interface_type = InterfaceType.DUAL_STREAM
+
+        # 基础配置
+        self.access_key_id = config.get("access_key_id")
+        self.access_key_secret = config.get("access_key_secret")
+        self.appkey = config.get("appkey")
+        self.format = config.get("format", "pcm")
+        self.audio_file_type = config.get("format", "pcm")
+
+        # 采样率配置
+        sample_rate = config.get("sample_rate", "16000")
+        self.sample_rate = int(sample_rate) if sample_rate else 16000
+
+        # 音色配置 - CosyVoice大模型音色
+        if config.get("private_voice"):
+            self.voice = config.get("private_voice")
+        else:
+            self.voice = config.get("voice", "longxiaochun")  # CosyVoice默认音色
+
+        # 音频参数配置
+        volume = config.get("volume", "50")
+        self.volume = int(volume) if volume else 50
+
+        speech_rate = config.get("speech_rate", "0")
+        self.speech_rate = int(speech_rate) if speech_rate else 0
+
+        pitch_rate = config.get("pitch_rate", "0")
+        self.pitch_rate = int(pitch_rate) if pitch_rate else 0
+
+        # WebSocket配置
+        self.host = config.get("host", "nls-gateway-cn-beijing.aliyuncs.com")
+        self.ws_url = f"wss://{self.host}/ws/v1"
+        self.ws = None
+        self._monitor_task = None
+        self.last_active_time = None
+
+        # 专属tts设置
+        self.message_id = ""
+
+        # 创建Opus编码器
+        self.opus_encoder = opus_encoder_utils.OpusEncoderUtils(
+            sample_rate=16000, channels=1, frame_size_ms=60
+        )
+
+        # Token管理
+        if self.access_key_id and self.access_key_secret:
+            self._refresh_token()
+        else:
+            self.token = config.get("token")
+            self.expire_time = None
+
+    def _refresh_token(self):
+        """刷新Token并记录过期时间"""
+        if self.access_key_id and self.access_key_secret:
+            self.token, expire_time_str = AccessToken.create_token(
+                self.access_key_id, self.access_key_secret
+            )
+            if not expire_time_str:
+                raise ValueError("无法获取有效的Token过期时间")
+
+            expire_str = str(expire_time_str).strip()
+
+            try:
+                if expire_str.isdigit():
+                    expire_time = datetime.fromtimestamp(int(expire_str))
+                else:
+                    expire_time = datetime.strptime(expire_str, "%Y-%m-%dT%H:%M:%SZ")
+                self.expire_time = expire_time.timestamp() - 60
+            except Exception as e:
+                raise ValueError(f"无效的过期时间格式: {expire_str}") from e
+        else:
+            self.expire_time = None
+
+        if not self.token:
+            raise ValueError("无法获取有效的访问Token")
+
+    def _is_token_expired(self):
+        """检查Token是否过期"""
+        if not self.expire_time:
+            return False
+        return time.time() > self.expire_time
+
+    async def _ensure_connection(self):
+        """确保WebSocket连接可用"""
+        try:
+            if self._is_token_expired():
+                logger.bind(tag=TAG).warning("Token已过期，正在自动刷新...")
+                self._refresh_token()
+            current_time = time.time()
+            if self.ws and current_time - self.last_active_time < 10:
+                # 10秒内才可以复用链接进行连续对话
+                logger.bind(tag=TAG).info(f"使用已有链接...")
+                return self.ws
+            logger.bind(tag=TAG).info("开始建立新连接...")
+
+            self.ws = await websockets.connect(
+                self.ws_url,
+                additional_headers={"X-NLS-Token": self.token},
+                ping_interval=30,
+                ping_timeout=10,
+                close_timeout=10,
+            )
+            logger.bind(tag=TAG).info("WebSocket连接建立成功")
+            self.last_active_time = time.time()
+            return self.ws
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"建立连接失败: {str(e)}")
+            self.ws = None
+            self.last_active_time = None
+            raise
+
+    def tts_text_priority_thread(self):
+        """流式文本处理线程"""
+        while not self.conn.stop_event.is_set():
+            try:
+                message = self.tts_text_queue.get(timeout=1)
+                logger.bind(tag=TAG).debug(
+                    f"收到TTS任务｜{message.sentence_type.name} ｜ {message.content_type.name} | 会话ID: {self.conn.sentence_id}"
+                )
+
+                if message.sentence_type == SentenceType.FIRST:
+                    self.conn.client_abort = False
+
+                if self.conn.client_abort:
+                    logger.bind(tag=TAG).info("收到打断信息，终止TTS文本处理线程")
+                    continue
+
+                if message.sentence_type == SentenceType.FIRST:
+                    # 初始化参数
+                    try:
+                        if not getattr(self.conn, "sentence_id", None):
+                            self.conn.sentence_id = uuid.uuid4().hex
+                            logger.bind(tag=TAG).info(
+                                f"自动生成新的 会话ID: {self.conn.sentence_id}"
+                            )
+
+                        # aliyunStream独有的参数生成
+                        self.message_id = str(uuid.uuid4().hex)
+
+                        logger.bind(tag=TAG).info("开始启动TTS会话...")
+                        future = asyncio.run_coroutine_threadsafe(
+                            self.start_session(self.conn.sentence_id),
+                            loop=self.conn.loop,
+                        )
+                        future.result()
+                        self.before_stop_play_files.clear()
+                        logger.bind(tag=TAG).info("TTS会话启动成功")
+
+                    except Exception as e:
+                        logger.bind(tag=TAG).error(f"启动TTS会话失败: {str(e)}")
+                        continue
+
+                elif ContentType.TEXT == message.content_type:
+                    if message.content_detail:
+                        try:
+                            logger.bind(tag=TAG).debug(
+                                f"开始发送TTS文本: {message.content_detail}"
+                            )
+                            future = asyncio.run_coroutine_threadsafe(
+                                self.text_to_speak(message.content_detail, None),
+                                loop=self.conn.loop,
+                            )
+                            future.result()
+                            logger.bind(tag=TAG).debug("TTS文本发送成功")
+                        except Exception as e:
+                            logger.bind(tag=TAG).error(f"发送TTS文本失败: {str(e)}")
+                            continue
+
+                elif ContentType.FILE == message.content_type:
+                    logger.bind(tag=TAG).info(
+                        f"添加音频文件到待播放列表: {message.content_file}"
+                    )
+                    if message.content_file and os.path.exists(message.content_file):
+                        # 先处理文件音频数据
+                        file_audio = self._process_audio_file(message.content_file)
+                        self.before_stop_play_files.append(
+                            (file_audio, message.content_detail)
+                        )
+
+                if message.sentence_type == SentenceType.LAST:
+                    try:
+                        logger.bind(tag=TAG).info("开始结束TTS会话...")
+                        future = asyncio.run_coroutine_threadsafe(
+                            self.finish_session(self.conn.sentence_id),
+                            loop=self.conn.loop,
+                        )
+                        future.result()
+                    except Exception as e:
+                        logger.bind(tag=TAG).error(f"结束TTS会话失败: {str(e)}")
+                        continue
+
+            except queue.Empty:
+                continue
+            except Exception as e:
+                logger.bind(tag=TAG).error(
+                    f"处理TTS文本失败: {str(e)}, 类型: {type(e).__name__}, 堆栈: {traceback.format_exc()}"
+                )
+
+    async def text_to_speak(self, text, _):
+        try:
+            if self.ws is None:
+                logger.bind(tag=TAG).warning(f"WebSocket连接不存在，终止发送文本")
+                return
+            filtered_text = MarkdownCleaner.clean_markdown(text)
+            run_request = {
+                "header": {
+                    "message_id": self.message_id,
+                    "task_id": self.conn.sentence_id,
+                    "namespace": "FlowingSpeechSynthesizer",
+                    "name": "RunSynthesis",
+                    "appkey": self.appkey,
+                },
+                "payload": {"text": filtered_text},
+            }
+            await self.ws.send(json.dumps(run_request))
+            self.last_active_time = time.time()
+            return
+
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"发送TTS文本失败: {str(e)}")
+            if self.ws:
+                try:
+                    await self.ws.close()
+                except:
+                    pass
+                self.ws = None
+            raise
+
+    async def start_session(self, session_id):
+        logger.bind(tag=TAG).info(f"开始会话～～{session_id}")
+        try:
+            # 会话开始时检测上个会话的监听状态
+            if (
+                self._monitor_task is not None
+                and isinstance(self._monitor_task, Task)
+                and not self._monitor_task.done()
+            ):
+                logger.bind(tag=TAG).info(
+                    "检测到未完成的上个会话，关闭监听任务和连接..."
+                )
+                await self.close()
+
+            # 建立新连接
+            await self._ensure_connection()
+
+            # 启动监听任务
+            self._monitor_task = asyncio.create_task(self._start_monitor_tts_response())
+
+            start_request = {
+                "header": {
+                    "message_id": self.message_id,
+                    "task_id": self.conn.sentence_id,
+                    "namespace": "FlowingSpeechSynthesizer",
+                    "name": "StartSynthesis",
+                    "appkey": self.appkey,
+                },
+                "payload": {
+                    "voice": self.voice,
+                    "format": self.format,
+                    "sample_rate": self.sample_rate,
+                    "volume": self.volume,
+                    "speech_rate": self.speech_rate,
+                    "pitch_rate": self.pitch_rate,
+                    "enable_subtitle": True,
+                },
+            }
+            await self.ws.send(json.dumps(start_request))
+            self.last_active_time = time.time()
+            logger.bind(tag=TAG).info("会话启动请求已发送")
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"启动会话失败: {str(e)}")
+            # 确保清理资源
+            await self.close()
+            raise
+
+    async def finish_session(self, session_id):
+        logger.bind(tag=TAG).info(f"关闭会话～～{session_id}")
+        try:
+            if self.ws:
+                stop_request = {
+                    "header": {
+                        "message_id": self.message_id,
+                        "task_id": self.conn.sentence_id,
+                        "namespace": "FlowingSpeechSynthesizer",
+                        "name": "StopSynthesis",
+                        "appkey": self.appkey,
+                    }
+                }
+                await self.ws.send(json.dumps(stop_request))
+                logger.bind(tag=TAG).info("会话结束请求已发送")
+                self.last_active_time = time.time()
+                if self._monitor_task:
+                    try:
+                        await self._monitor_task
+                    except Exception as e:
+                        logger.bind(tag=TAG).error(
+                            f"等待监听任务完成时发生错误: {str(e)}"
+                        )
+                    finally:
+                        self._monitor_task = None
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"关闭会话失败: {str(e)}")
+            # 确保清理资源
+            await self.close()
+            raise
+
+    async def close(self):
+        """资源清理"""
+        if self._monitor_task:
+            try:
+                self._monitor_task.cancel()
+                await self._monitor_task
+            except asyncio.CancelledError:
+                pass
+            except Exception as e:
+                logger.bind(tag=TAG).warning(f"关闭时取消监听任务错误: {e}")
+            self._monitor_task = None
+
+        if self.ws:
+            try:
+                await self.ws.close()
+            except:
+                pass
+            self.ws = None
+            self.last_active_time = None
+
+    async def _start_monitor_tts_response(self):
+        """监听TTS响应"""
+        opus_datas_cache = []
+        is_first_sentence = True
+        first_sentence_segment_count = 0  # 添加计数器
+        try:
+            session_finished = False  # 标记会话是否正常结束
+            while not self.conn.stop_event.is_set():
+                try:
+                    msg = await self.ws.recv()
+                    self.last_active_time = time.time()
+                    # 检查客户端是否中止
+                    if self.conn.client_abort:
+                        logger.bind(tag=TAG).info("收到打断信息，终止监听TTS响应")
+                        break
+                    if isinstance(msg, str):  # 文本控制消息
+                        try:
+                            data = json.loads(msg)
+                            header = data.get("header", {})
+                            event_name = header.get("name")
+                            if event_name == "SynthesisStarted":
+                                logger.bind(tag=TAG).debug("TTS合成已启动")
+                                self.tts_audio_queue.put(
+                                    (SentenceType.FIRST, [], None)
+                                )
+                            elif event_name == "SentenceBegin":
+                                opus_datas_cache = []
+                            elif event_name == "SentenceEnd":
+                                if (
+                                    not is_first_sentence
+                                    or first_sentence_segment_count > 10
+                                ):
+                                    # 发送缓存的数据
+                                    if self.conn.tts_MessageText:
+                                        logger.bind(tag=TAG).info(
+                                            f"句子语音生成成功： {self.conn.tts_MessageText}"
+                                        )
+                                        self.tts_audio_queue.put(
+                                            (SentenceType.MIDDLE, opus_datas_cache, self.conn.tts_MessageText)
+                                        )
+                                        self.conn.tts_MessageText = None
+                                    else:
+                                        self.tts_audio_queue.put(
+                                            (SentenceType.MIDDLE, opus_datas_cache, None)
+                                        )
+                                # 第一句话结束后，将标志设置为False
+                                is_first_sentence = False
+                            elif event_name == "SynthesisCompleted":
+                                logger.bind(tag=TAG).debug(f"会话结束～～")
+                                self._process_before_stop_play_files()
+                                session_finished = True
+                                break
+                        except json.JSONDecodeError:
+                            logger.bind(tag=TAG).warning("收到无效的JSON消息")
+                    # 二进制消息（音频数据）
+                    elif isinstance(msg, (bytes, bytearray)):
+                        logger.bind(tag=TAG).debug(f"推送数据到队列里面～～")
+                        opus_datas = self.opus_encoder.encode_pcm_to_opus(msg, False)
+                        logger.bind(tag=TAG).debug(
+                            f"推送数据到队列里面帧数～～{len(opus_datas)}"
+                        )
+                        if is_first_sentence:
+                            first_sentence_segment_count += 1
+                            if first_sentence_segment_count <= 6:
+                                self.tts_audio_queue.put(
+                                    (SentenceType.MIDDLE, opus_datas, None)
+                                )
+                            else:
+                                opus_datas_cache.extend(opus_datas)
+                        else:
+                            # 后续句子缓存
+                            opus_datas_cache.extend(opus_datas)
+
+                except websockets.ConnectionClosed:
+                    logger.bind(tag=TAG).warning("WebSocket连接已关闭")
+                    break
+                except Exception as e:
+                    logger.bind(tag=TAG).error(
+                        f"处理TTS响应时出错: {e}\n{traceback.format_exc()}"
+                    )
+                    break
+            # 仅在连接异常时才关闭
+            if not session_finished and self.ws:
+                try:
+                    await self.ws.close()
+                except:
+                    pass
+                self.ws = None
+        # 监听任务退出时清理引用
+        finally:
+            self._monitor_task = None
+
+    def to_tts(self, text: str) -> list:
+        """非流式TTS处理，用于测试及保存音频文件的场景"""
+        try:
+            # 创建新的事件循环
+            loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
+
+            # 生成会话ID
+            session_id = uuid.uuid4().hex
+            # 存储音频数据
+            audio_data = []
+
+            async def _generate_audio():
+                # 刷新Token（如果需要）
+                if self._is_token_expired():
+                    self._refresh_token()
+
+                # 建立WebSocket连接
+                ws = await websockets.connect(
+                    self.ws_url,
+                    additional_headers={"X-NLS-Token": self.token},
+                    ping_interval=30,
+                    ping_timeout=10,
+                    close_timeout=10,
+                )
+                try:
+                    # 发送StartSynthesis请求
+                    start_message_id = str(uuid.uuid4().hex)
+                    start_request = {
+                        "header": {
+                            "message_id": start_message_id,
+                            "task_id": session_id,
+                            "namespace": "FlowingSpeechSynthesizer",
+                            "name": "StartSynthesis",
+                            "appkey": self.appkey,
+                        },
+                        "payload": {
+                            "voice": self.voice,
+                            "format": self.format,
+                            "sample_rate": self.sample_rate,
+                            "volume": self.volume,
+                            "speech_rate": self.speech_rate,
+                            "pitch_rate": self.pitch_rate,
+                            "enable_subtitle": True,
+                        },
+                    }
+                    await ws.send(json.dumps(start_request))
+
+                    # 等待SynthesisStarted响应
+                    synthesis_started = False
+                    while not synthesis_started:
+                        msg = await ws.recv()
+                        if isinstance(msg, str):
+                            data = json.loads(msg)
+                            header = data.get("header", {})
+                            if header.get("name") == "SynthesisStarted":
+                                synthesis_started = True
+                                logger.bind(tag=TAG).debug("TTS合成已启动")
+                            elif header.get("name") == "TaskFailed":
+                                error_info = data.get("payload", {}).get(
+                                    "error_info", {}
+                                )
+                                error_code = error_info.get("error_code")
+                                error_message = error_info.get(
+                                    "error_message", "未知错误"
+                                )
+                                raise Exception(
+                                    f"启动合成失败: {error_code} - {error_message}"
+                                )
+
+                    # 发送文本合成请求
+                    filtered_text = MarkdownCleaner.clean_markdown(text)
+                    run_message_id = str(uuid.uuid4().hex)
+                    run_request = {
+                        "header": {
+                            "message_id": run_message_id,
+                            "task_id": session_id,
+                            "namespace": "FlowingSpeechSynthesizer",
+                            "name": "RunSynthesis",
+                            "appkey": self.appkey,
+                        },
+                        "payload": {"text": filtered_text},
+                    }
+                    await ws.send(json.dumps(run_request))
+
+                    # 发送停止合成请求
+                    stop_message_id = str(uuid.uuid4().hex)
+                    stop_request = {
+                        "header": {
+                            "message_id": stop_message_id,
+                            "task_id": session_id,
+                            "namespace": "FlowingSpeechSynthesizer",
+                            "name": "StopSynthesis",
+                            "appkey": self.appkey,
+                        }
+                    }
+                    await ws.send(json.dumps(stop_request))
+
+                    # 接收音频数据
+                    synthesis_completed = False
+                    while not synthesis_completed:
+                        msg = await ws.recv()
+                        if isinstance(msg, (bytes, bytearray)):
+                            # 编码为Opus并收集
+                            opus_frames = self.opus_encoder.encode_pcm_to_opus(
+                                msg, False
+                            )
+                            audio_data.extend(opus_frames)
+                        elif isinstance(msg, str):
+                            data = json.loads(msg)
+                            header = data.get("header", {})
+                            event_name = header.get("name")
+                            if event_name == "SynthesisCompleted":
+                                synthesis_completed = True
+                                logger.bind(tag=TAG).debug("TTS合成完成")
+                            elif event_name == "TaskFailed":
+                                error_info = data.get("payload", {}).get(
+                                    "error_info", {}
+                                )
+                                error_code = error_info.get("error_code")
+                                error_message = error_info.get(
+                                    "error_message", "未知错误"
+                                )
+                                raise Exception(
+                                    f"合成失败: {error_code} - {error_message}"
+                                )
+                finally:
+                    try:
+                        await ws.close()
+                    except:
+                        pass
+
+            loop.run_until_complete(_generate_audio())
+            loop.close()
+
+            return audio_data
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"生成音频数据失败: {str(e)}")
+            return []
@@ -50,6 +50,7 @@ class TTSProviderBase(ABC):
            "；",
            ";",
            "：",
+            "~",
        )
        self.first_sentence_punctuations = (
            "，",
@@ -159,7 +160,7 @@ class TTSProviderBase(ABC):
            if conn.sentence_id:
                sentence_id = conn.sentence_id
            else:
-                sentence_id = str(uuid.uuid4()).replace("-", "")
+                sentence_id = str(uuid.uuid4().hex)
                conn.sentence_id = sentence_id
        # 对于单句的文本，进行分段处理
        segments = re.split(r"([。！？!?；;\n])", content_detail)
@@ -332,7 +333,6 @@ class TTSProviderBase(ABC):
        Returns:
            tuple: (sentence_type, audio_datas, content_detail)
        """
-        audio_datas = []
        if tts_file.endswith(".p3"):
            audio_datas, _ = p3.decode_opus_from_file(tts_file)
        elif self.conn.audio_format == "pcm":
@@ -52,10 +52,11 @@ EVENT_ConnectionFinished = 52  # 连接结束

 # 上行Session事件
 EVENT_StartSession = 100
-
+EVENT_CancelSession = 101
 EVENT_FinishSession = 102
 # 下行Session事件
 EVENT_SessionStarted = 150
+EVENT_SessionCanceled = 151
 EVENT_SessionFinished = 152

 EVENT_SessionFailed = 153
@@ -207,8 +208,16 @@ class TTSProvider(TTSProviderBase):
                    self.conn.client_abort = False

                if self.conn.client_abort:
-                    logger.bind(tag=TAG).info("收到打断信息，终止TTS文本处理线程")
-                    continue
+                    try:
+                        logger.bind(tag=TAG).info("收到打断信息，终止TTS文本处理线程")
+                        asyncio.run_coroutine_threadsafe(
+                            self.cancel_session(self.conn.sentence_id),
+                            loop=self.conn.loop,
+                        )
+                        continue
+                    except Exception as e:
+                        logger.bind(tag=TAG).error(f"取消TTS会话失败: {str(e)}")
+                        continue

                if message.sentence_type == SentenceType.FIRST:
                    # 初始化参数
@@ -223,7 +232,6 @@ class TTSProvider(TTSProviderBase):
                            loop=self.conn.loop,
                        )
                        future.result()
-                        self.tts_audio_first_sentence = True
                        self.before_stop_play_files.clear()
                        logger.bind(tag=TAG).info("TTS会话启动成功")
                    except Exception as e:
@@ -371,6 +379,27 @@ class TTSProvider(TTSProviderBase):
            await self.close()
            raise

+    async def cancel_session(self,session_id):
+        logger.bind(tag=TAG).info(f"取消会话，释放服务端资源～～{session_id}")
+        try:
+            if self.ws:
+                header = Header(
+                    message_type=FULL_CLIENT_REQUEST,
+                    message_type_specific_flags=MsgTypeFlagWithEvent,
+                    serial_method=JSON,
+                ).as_bytes()
+                optional = Optional(
+                    event=EVENT_CancelSession, sessionId=session_id
+                ).as_bytes()
+                payload = str.encode("{}")
+                await self.send_event(self.ws, header, optional, payload)
+                logger.bind(tag=TAG).info("会话取消请求已发送")
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"取消会话失败: {str(e)}")
+            # 确保清理资源
+            await self.close()
+            raise
+
    async def close(self):
        """资源清理方法"""
        # 取消监听任务
@@ -405,12 +434,11 @@ class TTSProvider(TTSProviderBase):
                    res = self.parser_response(msg)
                    self.print_response(res, "send_text res:")

-                    # 检查客户端是否中止
-                    if self.conn.client_abort:
-                        logger.bind(tag=TAG).info("收到打断信息，终止监听TTS响应")
+                    if res.optional.event == EVENT_SessionCanceled:
+                        logger.bind(tag=TAG).debug(f"释放服务端资源成功～～")
+                        session_finished = True
                        break
-
-                    if res.optional.event == EVENT_TTSSentenceStart:
+                    elif res.optional.event == EVENT_TTSSentenceStart:
                        json_data = json.loads(res.payload.decode("utf-8"))
                        self.tts_text = json_data.get("text", "")
                        logger.bind(tag=TAG).debug(f"句子语音生成开始: {self.tts_text}")
@@ -435,10 +463,10 @@ class TTSProvider(TTSProviderBase):
                                    (SentenceType.MIDDLE, opus_datas, None)
                                )
                            else:
-                                opus_datas_cache = opus_datas_cache + opus_datas
+                                opus_datas_cache.extend(opus_datas)
                        else:
                            # 后续句子缓存
-                            opus_datas_cache = opus_datas_cache + opus_datas
+                            opus_datas_cache.extend(opus_datas)
                    elif res.optional.event == EVENT_TTSSentenceEnd:
                        logger.bind(tag=TAG).info(f"句子语音生成成功：{self.tts_text}")
                        if not is_first_sentence or first_sentence_segment_count > 10:
@@ -469,7 +497,7 @@ class TTSProvider(TTSProviderBase):
                except:
                    pass
                self.ws = None
-            # 监听任务退出时清理引用
+        # 监听任务退出时清理引用
        finally:
            self._monitor_task = None

@@ -52,7 +52,6 @@ class TTSProvider(TTSProviderBase):
                    self.processed_chars = 0
                    self.tts_text_buff = []
                    self.segment_count = 0
-                    self.tts_audio_first_sentence = True
                    self.before_stop_play_files.clear()
                elif ContentType.TEXT == message.content_type:
                    self.tts_text_buff.append(message.content_detail)
@@ -0,0 +1,230 @@
+import os
+import uuid
+import json
+import requests
+from datetime import datetime
+from typing import Iterator, Optional, Union
+from core.providers.tts.base import TTSProviderBase
+from core.utils.util import parse_string_to_list
+
+
+class TTSProvider(TTSProviderBase):
+    def __init__(self, config, delete_audio_file):
+        super().__init__(config, delete_audio_file)
+        self.group_id = config.get("group_id")
+        self.api_key = config.get("api_key")
+        self.model = config.get("model")
+        if config.get("private_voice"):
+            self.voice = config.get("private_voice")
+        else:
+            self.voice = config.get("voice_id")
+
+        default_voice_setting = {
+            "voice_id": "female-shaonv",
+            "speed": 1,
+            "vol": 1,
+            "pitch": 0,
+            "emotion": "happy",
+        }
+        default_pronunciation_dict = {"tone": ["处理/(chu3)(li3)", "危险/dangerous"]}
+        defult_audio_setting = {
+            "sample_rate": 32000,
+            "bitrate": 128000,
+            "format": "mp3",
+            "channel": 1,
+        }
+        self.voice_setting = {
+            **default_voice_setting,
+            **config.get("voice_setting", {}),
+        }
+        self.pronunciation_dict = {
+            **default_pronunciation_dict,
+            **config.get("pronunciation_dict", {}),
+        }
+        self.audio_setting = {**defult_audio_setting, **config.get("audio_setting", {})}
+        self.timber_weights = parse_string_to_list(config.get("timber_weights"))
+
+        if self.voice:
+            self.voice_setting["voice_id"] = self.voice
+
+        self.host = "api.minimax.chat"
+        self.api_url = f"https://{self.host}/v1/t2a_v2?GroupId={self.group_id}"
+        self.header = {
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {self.api_key}",
+        }
+        self.audio_file_type = defult_audio_setting.get("format", "mp3")
+
+    def generate_filename(self, extension=".mp3"):
+        return os.path.join(
+            self.output_file,
+            f"tts-{__name__}{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
+        )
+
+    async def text_to_speak(self, text, output_file):
+        """非流式语音合成（保留原有实现）"""
+        request_json = {
+            "model": self.model,
+            "text": text,
+            "stream": False,
+            "voice_setting": self.voice_setting,
+            "pronunciation_dict": self.pronunciation_dict,
+            "audio_setting": self.audio_setting,
+        }
+
+        if type(self.timber_weights) is list and len(self.timber_weights) > 0:
+            request_json["timber_weights"] = self.timber_weights
+            request_json["voice_setting"]["voice_id"] = ""
+
+        try:
+            resp = requests.post(
+                self.api_url, json.dumps(request_json), headers=self.header
+            )
+            if resp.json()["base_resp"]["status_code"] == 0:
+                data = resp.json()["data"]["audio"]
+                audio_bytes = bytes.fromhex(data)
+                if output_file:
+                    with open(output_file, "wb") as file_to_save:
+                        file_to_save.write(audio_bytes)
+                else:
+                    return audio_bytes
+            else:
+                raise Exception(
+                    f"{__name__} status_code: {resp.status_code} response: {resp.content}"
+                )
+        except Exception as e:
+            raise Exception(f"{__name__} error: {e}")
+
+    def text_to_speak_stream(
+        self, 
+        text: str, 
+        chunk_callback: Optional[callable] = None
+    ) -> Iterator[bytes]:
+        """
+        流式语音合成方法
+        :param text: 要合成的文本
+        :param chunk_callback: 可选的回调函数，用于处理每个音频块
+        :return: 生成器，每次产生一个音频数据块(bytes)
+        """
+        request_json = {
+            "model": self.model,
+            "text": text,
+            "stream": True,
+            "voice_setting": self.voice_setting,
+            "pronunciation_dict": self.pronunciation_dict,
+            "audio_setting": self.audio_setting,
+        }
+
+        if isinstance(self.timber_weights, list) and len(self.timber_weights) > 0:
+            request_json["timber_weights"] = self.timber_weights
+            request_json["voice_setting"]["voice_id"] = ""
+
+        try:
+            with requests.post(
+                self.api_url,
+                data=json.dumps(request_json),
+                headers=self.header,
+                stream=True
+            ) as response:
+                
+                # 检查HTTP状态码
+                if response.status_code != 200:
+                    raise Exception(
+                        f"HTTP error: {response.status_code}, response: {response.text}"
+                    )
+                
+                # 处理流式响应
+                for line in response.iter_lines():
+                    if line:  # 过滤空行
+                        # 检查是否为数据行 (SSE格式)
+                        if line.startswith(b'data:'):
+                            try:
+                                data = json.loads(line[5:].strip())  # 去掉"data:"前缀
+                                
+                                # 检查API状态码
+                                if data.get("base_resp", {}).get("status_code", -1) != 0:
+                                    raise Exception(
+                                        f"API error: {data.get('base_resp', {}).get('status_msg')}"
+                                    )
+                                
+                                # 跳过非音频数据块
+                                if "extra_info" in data:
+                                    continue
+                                    
+                                # 提取音频数据
+                                audio_hex = data.get("data", {}).get("audio")
+                                if audio_hex:
+                                    audio_chunk = bytes.fromhex(audio_hex)
+                                    if chunk_callback:
+                                        chunk_callback(audio_chunk)
+                                    yield audio_chunk
+                                    
+                            except json.JSONDecodeError:
+                                # 忽略JSON解析错误（可能是心跳包等）
+                                continue
+                            except Exception as e:
+                                raise e
+                                
+        except Exception as e:
+            raise Exception(f"{__name__} stream error: {e}")
+
+    def save_stream_to_file(
+        self, 
+        text: str, 
+        output_file: Optional[str] = None,
+        progress_callback: Optional[callable] = None
+    ) -> str:
+        """
+        流式合成并保存到文件
+        :param text: 要合成的文本
+        :param output_file: 输出文件路径，如果为None则自动生成
+        :param progress_callback: 可选的回调函数，接收已写入的字节数
+        :return: 保存的文件路径
+        """
+        if not output_file:
+            output_file = self.generate_filename(extension=f".{self.audio_file_type}")
+        
+        os.makedirs(os.path.dirname(output_file), exist_ok=True)
+        
+        total_bytes = 0
+        try:
+            with open(output_file, "wb") as audio_file:
+                for audio_chunk in self.text_to_speak_stream(text):
+                    audio_file.write(audio_chunk)
+                    audio_file.flush()
+                    total_bytes += len(audio_chunk)
+                    if progress_callback:
+                        progress_callback(total_bytes)
+            return output_file
+        except Exception as e:
+            # 清理可能创建的不完整文件
+            if os.path.exists(output_file):
+                os.remove(output_file)
+            raise e
+
+    def stream_to_audio_player(self, text: str, player_command: list = None):
+        """
+        流式合成并直接播放音频
+        :param text: 要合成的文本
+        :param player_command: 音频播放器命令，默认使用mpv
+        """
+        if player_command is None:
+            player_command = ["mpv", "--no-cache", "--no-terminal", "--", "fd://0"]
+        
+        try:
+            import subprocess
+            player_process = subprocess.Popen(
+                player_command,
+                stdin=subprocess.PIPE,
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+            )
+            
+            for audio_chunk in self.text_to_speak_stream(text):
+                player_process.stdin.write(audio_chunk)
+                player_process.stdin.flush()
+                
+            player_process.stdin.close()
+            player_process.wait()
+        except Exception as e:
+            raise Exception(f"Audio player error: {e}")
@@ -0,0 +1,180 @@
+import os
+import uuid
+import json
+import asyncio
+import websockets
+import ssl
+from datetime import datetime
+from core.providers.tts.base import TTSProviderBase
+from core.utils.util import parse_string_to_list
+
+
+class TTSProvider(TTSProviderBase):
+    def __init__(self, config, delete_audio_file):
+        super().__init__(config, delete_audio_file)
+        self.group_id = config.get("group_id")
+        self.api_key = config.get("api_key")
+        self.model = config.get("model")
+        
+        # 初始化语音设置
+        default_voice_setting = {
+            "voice_id": "female-shaonv",
+            "speed": 1,
+            "vol": 1,
+            "pitch": 0,
+            "emotion": "happy",
+        }
+        default_pronunciation_dict = {"tone": ["处理/(chu3)(li3)", "危险/dangerous"]}
+        default_audio_setting = {
+            "sample_rate": 32000,
+            "bitrate": 128000,
+            "format": "mp3",
+            "channel": 1,
+        }
+        
+        # 合并配置
+        self.voice_setting = {
+            **default_voice_setting,
+            **config.get("voice_setting", {}),
+        }
+        self.pronunciation_dict = {
+            **default_pronunciation_dict,
+            **config.get("pronunciation_dict", {}),
+        }
+        self.audio_setting = {
+            **default_audio_setting, 
+            **config.get("audio_setting", {})
+        }
+        self.timber_weights = parse_string_to_list(config.get("timber_weights"))
+        
+        # 设置语音ID
+        if config.get("private_voice"):
+            self.voice_setting["voice_id"] = config.get("private_voice")
+        elif config.get("voice_id"):
+            self.voice_setting["voice_id"] = config.get("voice_id")
+        
+        # WebSocket配置
+        self.ws_url = "wss://api.minimaxi.com/ws/v1/t2a_v2"
+        self.headers = {
+            "Authorization": f"Bearer {self.api_key}",
+            "GroupId": self.group_id
+        }
+        self.audio_file_type = self.audio_setting.get("format", "mp3")
+
+    def generate_filename(self, extension=".mp3"):
+        """生成唯一的音频文件名"""
+        return os.path.join(
+            self.output_file,
+            f"tts-{__name__}{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
+        )
+
+    async def _establish_connection(self):
+        """建立WebSocket连接"""
+        ssl_context = ssl.create_default_context()
+        ssl_context.check_hostname = False
+        ssl_context.verify_mode = ssl.CERT_NONE
+
+        try:
+            ws = await websockets.connect(
+                self.ws_url,
+                additional_headers=self.headers,
+                ssl=ssl_context
+            )
+            connected = json.loads(await ws.recv())
+            if connected.get("event") == "connected_success":
+                print("连接成功")
+                return ws
+            return None
+        except Exception as e:
+            print(f"连接失败: {e}")
+            return None
+
+    async def _start_task(self, websocket):
+        """发送任务开始请求"""
+        start_msg = {
+            "event": "task_start",
+            "model": self.model,
+            "voice_setting": self.voice_setting,
+            "pronunciation_dict": self.pronunciation_dict,
+            "audio_setting": self.audio_setting
+        }
+        
+        if self.timber_weights and len(self.timber_weights) > 0:
+            start_msg["timber_weights"] = self.timber_weights
+            start_msg["voice_setting"]["voice_id"] = ""
+
+        await websocket.send(json.dumps(start_msg))
+        response = json.loads(await websocket.recv())
+        return response.get("event") == "task_started"
+
+    async def _continue_task(self, websocket, text):
+        """发送继续请求并收集音频数据"""
+        await websocket.send(json.dumps({
+            "event": "task_continue",
+            "text": text
+        }))
+
+        audio_chunks = []
+        while True:
+            response = json.loads(await websocket.recv())
+            if "data" in response and "audio" in response["data"]:
+                audio_chunks.append(response["data"]["audio"])
+            if response.get("is_final"):
+                break
+        return "".join(audio_chunks)
+
+    async def _close_connection(self, websocket):
+        """关闭连接"""
+        if websocket:
+            await websocket.send(json.dumps({"event": "task_finish"}))
+            await websocket.close()
+            print("连接已关闭")
+
+    async def text_to_speak(self, text, output_file=None):
+        """主方法：文本转语音"""
+        ws = await self._establish_connection()
+        if not ws:
+            raise Exception("无法建立WebSocket连接")
+
+        try:
+            if not await self._start_task(ws):
+                raise Exception("任务启动失败")
+
+            hex_audio = await self._continue_task(ws, text)
+            audio_bytes = bytes.fromhex(hex_audio)
+
+            # 保存到文件或返回二进制数据
+            if output_file:
+                with open(output_file, "wb") as f:
+                    f.write(audio_bytes)
+                print(f"音频已保存为{output_file}")
+                return output_file
+            else:
+                # 返回音频二进制数据（不播放）
+                return audio_bytes
+
+        finally:
+            await self._close_connection(ws)
+
+
+async def main():
+    """测试用主函数"""
+    # 示例配置
+    config = {
+        "group_id": "YOUR_GROUP_ID",  # 替换为实际的group_id
+        "api_key": "YOUR_API_KEY",    # 替换为实际的api_key
+        "model": "your-model",        # 替换为实际的模型名称
+        "voice_id": "male-qn-qingse",
+        "voice_setting": {
+            "speed": 1.2,
+            "emotion": "happy"
+        }
+    }
+    
+    tts = TTSProvider(config, delete_audio_file=True)
+    output_file = tts.generate_filename()
+    await tts.text_to_speak("这是一个测试文本，用于验证流式语音合成功能", output_file)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -21,6 +21,30 @@ WEEKDAY_MAP = {
    "Sunday": "星期日",
 }

+EMOJI_List = [
+    "😶",
+    "🙂",
+    "😆",
+    "😂",
+    "😔",
+    "😠",
+    "😭",
+    "😍",
+    "😳",
+    "😲",
+    "😱",
+    "🤔",
+    "😉",
+    "😎",
+    "😌",
+    "🤤",
+    "😘",
+    "😏",
+    "😴",
+    "😜",
+    "🙄",
+]
+

 class PromptManager:
    """系统提示词管理器，负责管理和更新系统提示词"""
@@ -206,6 +230,7 @@ class PromptManager:
                lunar_date=lunar_date,
                local_address=local_address,
                weather_info=weather_info,
+                emojiList=EMOJI_List,
            )
            device_cache_key = f"device_prompt:{device_id}"
            self.cache_manager.set(
@@ -1,3 +1,40 @@
+import json
+
+TAG = __name__
+EMOJI_MAP = {
+    "😂": "laughing",
+    "😭": "crying",
+    "😠": "angry",
+    "😔": "sad",
+    "😍": "loving",
+    "😲": "surprised",
+    "😱": "shocked",
+    "🤔": "thinking",
+    "😌": "relaxed",
+    "😴": "sleepy",
+    "😜": "silly",
+    "🙄": "confused",
+    "😶": "neutral",
+    "🙂": "happy",
+    "😆": "laughing",
+    "😳": "embarrassed",
+    "😉": "winking",
+    "😎": "cool",
+    "🤤": "delicious",
+    "😘": "kissy",
+    "😏": "confident",
+}
+EMOJI_RANGES = [
+    (0x1F600, 0x1F64F),
+    (0x1F300, 0x1F5FF),
+    (0x1F680, 0x1F6FF),
+    (0x1F900, 0x1F9FF),
+    (0x1FA70, 0x1FAFF),
+    (0x2600, 0x26FF),
+    (0x2700, 0x27BF),
+]
+
+
 def get_string_no_punctuation_or_emoji(s):
    """去除字符串首尾的空格、标点符号和表情符号"""
    chars = list(s)
@@ -22,6 +59,11 @@ def is_punctuation_or_emoji(char):
        ".",  # 中文句号 + 英文句号
        "！",
        "!",  # 中文感叹号 + 英文感叹号
+        "“",
+        "”",
+        '"',  # 中文双引号 + 英文引号
+        "：",
+        ":",  # 中文冒号 + 英文冒号
        "-",
        "－",  # 英文连字符 + 中文全角横线
        "、",  # 中文顿号
@@ -32,15 +74,40 @@ def is_punctuation_or_emoji(char):
    }
    if char.isspace() or char in punctuation_set:
        return True
-    # 检查表情符号（保留原有逻辑）
+    return is_emoji(char)
+
+
+async def get_emotion(conn, text):
+    """获取文本内的情绪消息"""
+    emoji = "🙂"
+    emotion = "happy"
+    for char in text:
+        if char in EMOJI_MAP:
+            emoji = char
+            emotion = EMOJI_MAP[char]
+            break
+    try:
+        await conn.websocket.send(
+            json.dumps(
+                {
+                    "type": "llm",
+                    "text": emoji,
+                    "emotion": emotion,
+                    "session_id": conn.session_id,
+                }
+            )
+        )
+    except Exception as e:
+        conn.logger.bind(tag=TAG).warning(f"发送情绪表情失败，错误:{e}")
+    return
+
+
+def is_emoji(char):
+    """检查字符是否为emoji表情"""
    code_point = ord(char)
-    emoji_ranges = [
-        (0x1F600, 0x1F64F),
-        (0x1F300, 0x1F5FF),
-        (0x1F680, 0x1F6FF),
-        (0x1F900, 0x1F9FF),
-        (0x1FA70, 0x1FAFF),
-        (0x2600, 0x26FF),
-        (0x2700, 0x27BF),
-    ]
-    return any(start <= code_point <= end for start, end in emoji_ranges)
+    return any(start <= code_point <= end for start, end in EMOJI_RANGES)
+
+
+def check_emoji(text):
+    """去除文本中的所有emoji表情"""
+    return ''.join(char for char in text if not is_emoji(char) and char != "\n")
@@ -125,51 +125,6 @@ def write_json_file(file_path, data):
        json.dump(data, file, ensure_ascii=False, indent=4)


-def is_punctuation_or_emoji(char):
-    """检查字符是否为空格、指定标点或表情符号"""
-    # 定义需要去除的中英文标点（包括全角/半角）
-    punctuation_set = {
-        "，",
-        ",",  # 中文逗号 + 英文逗号
-        "-",
-        "－",  # 英文连字符 + 中文全角横线
-        "、",  # 中文顿号
-        "“",
-        "”",
-        '"',  # 中文双引号 + 英文引号
-        "：",
-        ":",  # 中文冒号 + 英文冒号
-    }
-    if char.isspace() or char in punctuation_set:
-        return True
-    # 检查表情符号（保留原有逻辑）
-    code_point = ord(char)
-    emoji_ranges = [
-        (0x1F600, 0x1F64F),
-        (0x1F300, 0x1F5FF),
-        (0x1F680, 0x1F6FF),
-        (0x1F900, 0x1F9FF),
-        (0x1FA70, 0x1FAFF),
-        (0x2600, 0x26FF),
-        (0x2700, 0x27BF),
-    ]
-    return any(start <= code_point <= end for start, end in emoji_ranges)
-
-
-def get_string_no_punctuation_or_emoji(s):
-    """去除字符串首尾的空格、标点符号和表情符号"""
-    chars = list(s)
-    # 处理开头的字符
-    start = 0
-    while start < len(chars) and is_punctuation_or_emoji(chars[start]):
-        start += 1
-    # 处理结尾的字符
-    end = len(chars) - 1
-    while end >= start and is_punctuation_or_emoji(chars[end]):
-        end -= 1
-    return "".join(chars[start : end + 1])
-
-
 def remove_punctuation_and_length(text):
    # 全角符号和半角符号的Unicode范围
    full_width_punctuations = (
@@ -256,515 +211,6 @@ def extract_json_from_string(input_string):
    return None


-def analyze_emotion(text):
-    """
-    分析文本情感并返回对应的emoji名称（支持中英文）
-    """
-    if not text or not isinstance(text, str):
-        return "neutral"
-
-    original_text = text
-    text = text.lower().strip()
-
-    # 检查是否包含现有emoji
-    for emotion, emoji in emoji_map.items():
-        if emoji in original_text:
-            return emotion
-
-    # 标点符号分析
-    has_exclamation = "!" in original_text or "！" in original_text
-    has_question = "?" in original_text or "？" in original_text
-    has_ellipsis = "..." in original_text or "…" in original_text
-
-    # 定义情感关键词映射（中英文扩展版）
-    emotion_keywords = {
-        "happy": [
-            "开心",
-            "高兴",
-            "快乐",
-            "愉快",
-            "幸福",
-            "满意",
-            "棒",
-            "好",
-            "不错",
-            "完美",
-            "棒极了",
-            "太好了",
-            "好呀",
-            "好的",
-            "happy",
-            "joy",
-            "great",
-            "good",
-            "nice",
-            "awesome",
-            "fantastic",
-            "wonderful",
-        ],
-        "laughing": [
-            "哈哈",
-            "哈哈哈",
-            "呵呵",
-            "嘿嘿",
-            "嘻嘻",
-            "笑死",
-            "太好笑了",
-            "笑死我了",
-            "lol",
-            "lmao",
-            "haha",
-            "hahaha",
-            "hehe",
-            "rofl",
-            "funny",
-            "laugh",
-        ],
-        "funny": [
-            "搞笑",
-            "滑稽",
-            "逗",
-            "幽默",
-            "笑点",
-            "段子",
-            "笑话",
-            "太逗了",
-            "hilarious",
-            "joke",
-            "comedy",
-        ],
-        "sad": [
-            "伤心",
-            "难过",
-            "悲哀",
-            "悲伤",
-            "忧郁",
-            "郁闷",
-            "沮丧",
-            "失望",
-            "想哭",
-            "难受",
-            "不开心",
-            "唉",
-            "呜呜",
-            "sad",
-            "upset",
-            "unhappy",
-            "depressed",
-            "sorrow",
-            "gloomy",
-        ],
-        "angry": [
-            "生气",
-            "愤怒",
-            "气死",
-            "讨厌",
-            "烦人",
-            "可恶",
-            "烦死了",
-            "恼火",
-            "暴躁",
-            "火大",
-            "愤怒",
-            "气炸了",
-            "angry",
-            "mad",
-            "annoyed",
-            "furious",
-            "pissed",
-            "hate",
-        ],
-        "crying": [
-            "哭泣",
-            "泪流",
-            "大哭",
-            "伤心欲绝",
-            "泪目",
-            "流泪",
-            "哭死",
-            "哭晕",
-            "想哭",
-            "泪崩",
-            "cry",
-            "crying",
-            "tears",
-            "sob",
-            "weep",
-        ],
-        "loving": [
-            "爱你",
-            "喜欢",
-            "爱",
-            "亲爱的",
-            "宝贝",
-            "么么哒",
-            "抱抱",
-            "想你",
-            "思念",
-            "最爱",
-            "亲亲",
-            "喜欢你",
-            "love",
-            "like",
-            "adore",
-            "darling",
-            "sweetie",
-            "honey",
-            "miss you",
-            "heart",
-        ],
-        "embarrassed": [
-            "尴尬",
-            "不好意思",
-            "害羞",
-            "脸红",
-            "难为情",
-            "社死",
-            "丢脸",
-            "出丑",
-            "embarrassed",
-            "awkward",
-            "shy",
-            "blush",
-        ],
-        "surprised": [
-            "惊讶",
-            "吃惊",
-            "天啊",
-            "哇塞",
-            "哇",
-            "居然",
-            "竟然",
-            "没想到",
-            "出乎意料",
-            "surprise",
-            "wow",
-            "omg",
-            "oh my god",
-            "amazing",
-            "unbelievable",
-        ],
-        "shocked": [
-            "震惊",
-            "吓到",
-            "惊呆了",
-            "不敢相信",
-            "震撼",
-            "吓死",
-            "恐怖",
-            "害怕",
-            "吓人",
-            "shocked",
-            "shocking",
-            "scared",
-            "frightened",
-            "terrified",
-            "horror",
-        ],
-        "thinking": [
-            "思考",
-            "考虑",
-            "想一下",
-            "琢磨",
-            "沉思",
-            "冥想",
-            "想",
-            "思考中",
-            "在想",
-            "think",
-            "thinking",
-            "consider",
-            "ponder",
-            "meditate",
-        ],
-        "winking": [
-            "调皮",
-            "眨眼",
-            "你懂的",
-            "坏笑",
-            "邪恶",
-            "奸笑",
-            "使眼色",
-            "wink",
-            "teasing",
-            "naughty",
-            "mischievous",
-        ],
-        "cool": [
-            "酷",
-            "帅",
-            "厉害",
-            "棒极了",
-            "真棒",
-            "牛逼",
-            "强",
-            "优秀",
-            "杰出",
-            "出色",
-            "完美",
-            "cool",
-            "awesome",
-            "amazing",
-            "great",
-            "impressive",
-            "perfect",
-        ],
-        "relaxed": [
-            "放松",
-            "舒服",
-            "惬意",
-            "悠闲",
-            "轻松",
-            "舒适",
-            "安逸",
-            "自在",
-            "relax",
-            "relaxed",
-            "comfortable",
-            "cozy",
-            "chill",
-            "peaceful",
-        ],
-        "delicious": [
-            "好吃",
-            "美味",
-            "香",
-            "馋",
-            "可口",
-            "香甜",
-            "大餐",
-            "大快朵颐",
-            "流口水",
-            "垂涎",
-            "delicious",
-            "yummy",
-            "tasty",
-            "yum",
-            "appetizing",
-            "mouthwatering",
-        ],
-        "kissy": [
-            "亲亲",
-            "么么",
-            "吻",
-            "mua",
-            "muah",
-            "亲一下",
-            "飞吻",
-            "kiss",
-            "xoxo",
-            "hug",
-            "muah",
-            "smooch",
-        ],
-        "confident": [
-            "自信",
-            "肯定",
-            "确定",
-            "毫无疑问",
-            "当然",
-            "必须的",
-            "毫无疑问",
-            "确信",
-            "坚信",
-            "confident",
-            "sure",
-            "certain",
-            "definitely",
-            "positive",
-        ],
-        "sleepy": [
-            "困",
-            "睡觉",
-            "晚安",
-            "想睡",
-            "好累",
-            "疲惫",
-            "疲倦",
-            "困了",
-            "想休息",
-            "睡意",
-            "sleep",
-            "sleepy",
-            "tired",
-            "exhausted",
-            "bedtime",
-            "good night",
-        ],
-        "silly": [
-            "傻",
-            "笨",
-            "呆",
-            "憨",
-            "蠢",
-            "二",
-            "憨憨",
-            "傻乎乎",
-            "呆萌",
-            "silly",
-            "stupid",
-            "dumb",
-            "foolish",
-            "goofy",
-            "ridiculous",
-        ],
-        "confused": [
-            "疑惑",
-            "不明白",
-            "不懂",
-            "困惑",
-            "疑问",
-            "为什么",
-            "怎么回事",
-            "啥意思",
-            "不清楚",
-            "confused",
-            "puzzled",
-            "doubt",
-            "question",
-            "what",
-            "why",
-            "how",
-        ],
-    }
-
-    # 特殊句型判断（中英文）
-    # 赞美他人
-    if any(
-        phrase in text
-        for phrase in [
-            "你真",
-            "你好",
-            "您真",
-            "你真棒",
-            "你好厉害",
-            "你太强了",
-            "你真好",
-            "你真聪明",
-            "you are",
-            "you're",
-            "you look",
-            "you seem",
-            "so smart",
-            "so kind",
-        ]
-    ):
-        return "loving"
-    # 自我赞美
-    if any(
-        phrase in text
-        for phrase in [
-            "我真",
-            "我最",
-            "我太棒了",
-            "我厉害",
-            "我聪明",
-            "我优秀",
-            "i am",
-            "i'm",
-            "i feel",
-            "so good",
-            "so happy",
-        ]
-    ):
-        return "cool"
-    # 晚安/睡觉相关
-    if any(
-        phrase in text
-        for phrase in [
-            "睡觉",
-            "晚安",
-            "睡了",
-            "好梦",
-            "休息了",
-            "去睡了",
-            "sleep",
-            "good night",
-            "bedtime",
-            "go to bed",
-        ]
-    ):
-        return "sleepy"
-    # 疑问句
-    if has_question and not has_exclamation:
-        return "thinking"
-    # 强烈情感（感叹号）
-    if has_exclamation and not has_question:
-        # 检查是否是积极内容
-        positive_words = (
-            emotion_keywords["happy"]
-            + emotion_keywords["laughing"]
-            + emotion_keywords["cool"]
-        )
-        if any(word in text for word in positive_words):
-            return "laughing"
-        # 检查是否是消极内容
-        negative_words = (
-            emotion_keywords["angry"]
-            + emotion_keywords["sad"]
-            + emotion_keywords["crying"]
-        )
-        if any(word in text for word in negative_words):
-            return "angry"
-        return "surprised"
-    # 省略号（表示犹豫或思考）
-    if has_ellipsis:
-        return "thinking"
-
-    # 关键词匹配（带权重）
-    emotion_scores = {emotion: 0 for emotion in emoji_map.keys()}
-
-    # 给匹配到的关键词加分
-    for emotion, keywords in emotion_keywords.items():
-        for keyword in keywords:
-            if keyword in text:
-                emotion_scores[emotion] += 1
-
-    # 给长文本中的重复关键词额外加分
-    if len(text) > 20:  # 长文本
-        for emotion, keywords in emotion_keywords.items():
-            for keyword in keywords:
-                emotion_scores[emotion] += text.count(keyword) * 0.5
-
-    # 根据分数选择最可能的情感
-    max_score = max(emotion_scores.values())
-    if max_score == 0:
-        return "happy"  # 默认
-
-    # 可能有多个情感同分，根据上下文选择最合适的
-    top_emotions = [e for e, s in emotion_scores.items() if s == max_score]
-
-    # 如果多个情感同分，使用以下优先级
-    priority_order = [
-        "laughing",
-        "crying",
-        "angry",
-        "surprised",
-        "shocked",  # 强烈情感优先
-        "loving",
-        "happy",
-        "funny",
-        "cool",  # 积极情感
-        "sad",
-        "embarrassed",
-        "confused",  # 消极情感
-        "thinking",
-        "winking",
-        "relaxed",  # 中性情感
-        "delicious",
-        "kissy",
-        "confident",
-        "sleepy",
-        "silly",  # 特殊场景
-    ]
-
-    for emotion in priority_order:
-        if emotion in top_emotions:
-            return emotion
-
-    return top_emotions[0]  # 如果都不在优先级列表里，返回第一个
-
-
 def audio_to_data(audio_file_path, is_opus=True):
    # 获取文件后缀名
    file_type = os.path.splitext(audio_file_path)[1]
@@ -33,10 +33,16 @@ def hass_get_state(conn, entity_id=""):
        future = asyncio.run_coroutine_threadsafe(
            handle_hass_get_state(conn, entity_id), conn.loop
        )
-        ha_response = future.result()
+        # 添加10秒超时
+        ha_response = future.result(timeout=10)
        return ActionResponse(Action.REQLLM, ha_response, None)
+    except asyncio.TimeoutError:
+        logger.bind(tag=TAG).error("获取Home Assistant状态超时")
+        return ActionResponse(Action.ERROR, "请求超时", None)
    except Exception as e:
-        logger.bind(tag=TAG).error(f"处理设置属性意图错误: {e}")
+        error_msg = f"执行Home Assistant操作失败"
+        logger.bind(tag=TAG).error(error_msg)
+        return ActionResponse(Action.ERROR, error_msg, None)


 async def handle_hass_get_state(conn, entity_id):
@@ -55,10 +55,16 @@ def hass_set_state(conn, entity_id="", state={}):
        future = asyncio.run_coroutine_threadsafe(
            handle_hass_set_state(conn, entity_id, state), conn.loop
        )
-        ha_response = future.result()
+        # 添加10秒超时
+        ha_response = future.result(timeout=10)
        return ActionResponse(Action.REQLLM, ha_response, None)
+    except asyncio.TimeoutError:
+        logger.bind(tag=TAG).error("设置Home Assistant状态超时")
+        return ActionResponse(Action.ERROR, "请求超时", None)
    except Exception as e:
-        logger.bind(tag=TAG).error(f"处理设置属性意图错误: {e}")
+        error_msg = f"执行Home Assistant操作失败"
+        logger.bind(tag=TAG).error(error_msg)
+        return ActionResponse(Action.ERROR, error_msg, None)


 async def handle_hass_set_state(conn, entity_id, state):
Author	SHA1	Message	Date
欣南科技andGitHub	6066c20676	Merge pull request #1923 from xinnan-tech/update-version update:升级版本号	2025-07-26 11:47:15 +08:00
hrz	1c7ba50def	update:升级版本号	2025-07-26 11:46:11 +08:00
hrzandGitHub	e53b24ef47	Merge pull request #1901 from xinnan-tech/py_test_tts Py test tts	2025-07-25 22:17:59 +08:00
hrz	1a978abcc1	update: MinimaxTTSHTTPStream和MinimaxTTSWebSocketStream还在测试，测试完再开放	2025-07-25 22:11:30 +08:00
hrzandGitHub	b91f4e4281	Merge pull request #1911 from xinnan-tech/manager-api-agent-optimize 优化	2025-07-25 21:47:54 +08:00
hrz	86978329eb	update:优化逻辑	2025-07-25 21:47:21 +08:00
欣南科技andGitHub	337ecf0efe	Merge pull request #1918 from xinnan-tech/update-doc update:调整位置	2025-07-25 11:38:06 +08:00
hrz	4c3eb90bfc	update:调整位置	2025-07-25 11:37:29 +08:00
欣南科技andGitHub	018a0422b7	Merge pull request #1916 from xinnan-tech/update-doc update:更新全模块关系图	2025-07-25 11:29:56 +08:00
hrz	f18ac169fa	update:更新全模块关系图	2025-07-25 11:29:23 +08:00
JianYu Zheng	3939c81044	添加：删除模型的验证 --ModelConfigServiceImpl.java 删除视觉模型时，如果引用了,就不能删除	2025-07-24 17:13:05 +08:00
JianYu Zheng	fdbe5fa556	添加了修改模型配置验证 --ModelConfigServiceImpl.java 修改模型，jsonConfig 里包括llm,校验一下llm是否时在LLM模型里,并且这个模型的供应器类型只能是openai和ollama类型	2025-07-24 17:01:43 +08:00
hrz	dff8b8ccec	Merge branch 'main' into py_test_tts	2025-07-24 16:18:39 +08:00
JianYu Zheng	8d11b47241	添加了新请求接口，大语言模型和意图参数关系的选择判断 --model.js 添加新接口方法 --roleConfig.vue 添加智能体配置,当选择的LLM不是openai、ollama类型时,意图模型不能选择“无意图识别“	2025-07-24 16:17:27 +08:00
hrz	78e5c52932	Merge branch 'main' into py_test_tts	2025-07-24 15:15:44 +08:00
CGDandGitHub	46c7759718	Merge pull request #1909 from xinnan-tech/py_fix_emoji fix: 补充None值判断	2025-07-24 15:09:14 +08:00
Sakura-RanChen	2f5e8c2019	fix: 补充None值判断	2025-07-24 15:07:35 +08:00
hrz	2508d3f965	update:获取nginx转发的ip地址	2025-07-24 14:38:56 +08:00
hrzandGitHub	3699d28dd0	Merge pull request #1893 from Hmmrrr/main feat：添加minimax单向双向流式	2025-07-24 14:18:19 +08:00
JianYu Zheng	be7146fa89	添加新接口，获取llm模型基础模型信息 --LlmModelBasicInfoDTO.java 新的llm模型基础模型信息返回对象 --ModelConfigService.java --ModelConfigServiceImpl.java --ModelController.java 定义，实现，添加了接口。一个获取获取LLM模型信息的：信息内容包括id，名称，type	2025-07-24 11:50:15 +08:00
CGDandGitHub	66f4ea0a84	Merge pull request #1905 from xinnan-tech/py_fix_emoji fix: LLM携带空文本时错误判断	2025-07-24 09:33:19 +08:00
Sakura-RanChen	84ff897b46	fix: LLM携带空文本时错误判断	2025-07-24 09:31:44 +08:00
JianYu Zheng	29c7b2a920	修改：保存智能体配置方法 --AgentServiceImpl.java 保存智能体配置方法，添加`大语言模型和意图识别是否符合匹配`的参数验证	2025-07-23 15:59:22 +08:00
Sakura-RanChen	83ded8458a	Merge branch 'py_test_tts' of https://github.com/xinnan-tech/xiaozhi-esp32-server into py_test_tts	2025-07-23 14:40:14 +08:00
Sakura-RanChen	e93053d412	fix: 文本重复发送	2025-07-23 14:35:59 +08:00
欣南科技andGitHub	2370936dfd	Merge pull request #1899 from xinnan-tech/manager-api-mcp 优化获取请求mcp工具方法	2025-07-23 14:15:44 +08:00
JianYu Zheng	06bd7aed36	优化获取请求mcp工具方法 --JsonRpcTwo.java 添加JSON-RPC2.0 格式规范对象 --XiaoZhiMcpJsonRpcJson.java 添加小智mcp JSON-RPC 2.0 请求内容json 常量 --McpJsonRpcRequest.java 删除之前的mcp JSON-RPC2.0 构造对象 --AgentMcpAccessPointServiceImpl.java 优化思路，每次发送请求都要构造2次对象和转换2次json请求内容，且每次内容都一样，现在把最终转成的json请求内容存储为常量，所有请求共用这些常量，减少每次请求构造和转换	2025-07-23 11:56:36 +08:00
欣南科技andGitHub	ff560a317c	Merge pull request #1894 from xinnan-tech/update-doc update:添加阿里云流式推荐	2025-07-22 18:22:05 +08:00
hrz	6892befa15	update:添加阿里云流式推荐	2025-07-22 18:21:09 +08:00
Hmmrrr	fadf18b7fc	feat：添加minimax单向双向流式	2025-07-22 18:13:05 +08:00
hrzandGitHub	d9b632c031	Merge pull request #1892 from xinnan-tech/py_fix_mcp update:添加Home Assistant插件超时处理和错误日志	2025-07-22 17:43:33 +08:00
hrzandGitHub	35fd4d2770	Merge pull request #1890 from xinnan-tech/manager-api-mcp 修复mcp工具太多会获取不到内容的bug	2025-07-22 16:45:05 +08:00
3030332422	1b8963f311	update:添加Home Assistant插件超时处理和错误日志	2025-07-22 16:28:12 +08:00
欣南科技andGitHub	710218b5b5	Merge pull request #1716 from xinnan-tech/py_test_tts update:添加阿里云流式asr	2025-07-22 13:27:41 +08:00
hrz	60cbe1571c	update:修复cosyvoice非流式合成方法	2025-07-22 13:25:54 +08:00
hrzandGitHub	6740d6a723	Merge pull request #1885 from xinnan-tech/py_prompt_emoji Py prompt emoji	2025-07-21 23:18:30 +08:00
hrzandGitHub	eb14e50941	Merge pull request #1882 from pupa880/main 添加最简化配置时配置声纹的教程	2025-07-21 23:14:08 +08:00
hrzandGitHub	37ac778ff4	Update voiceprint-integration.md	2025-07-21 23:13:45 +08:00
Sakura-RanChen	bc5586a077	update: HuoshanTTS服务器资源释放	2025-07-21 17:46:22 +08:00
JianYu Zheng	eb7ac93e72	修复mcp工具太多会获取不到内容的bug --WebSocketClientManager.java 1.构建websocket对象的时候，添加可以设置缓冲区大小 2.断开连接的日志添加断开原因的内容 --AgentMcpAccessPointServiceImpl.java 1.设置每个websocket的缓冲区为1m，bug原因：缓存区太小，数据量太大，导致直接断开连接	2025-07-21 16:07:56 +08:00
pupa	bf8a8bddf1	添加最简化配置时配置声纹的教程	2025-07-21 14:13:54 +08:00
hrz	d1badcb28c	update:修改版本号	2025-07-21 13:29:17 +08:00
hrz	f6e79e17b7	update:更正cosyvoice大模型节点	2025-07-21 12:03:53 +08:00
Sakura-RanChen	eead126f7a	update: 表情由llm发送，长文本进行约束	2025-07-21 09:30:23 +08:00
hrzandGitHub	e5ef61dfd3	Merge pull request #1873 from xinnan-tech/py_test 修复声纹识别场景下的意图识别问题、日志文件的补充	2025-07-20 20:23:29 +08:00
hrz	eb130aa57f	Merge branch 'py_test_tts' of https://github.com/xinnan-tech/xiaozhi-esp32-server into py_test_tts	2025-07-20 00:29:25 +08:00
hrz	c9b10494d8	update：优化配置文件	2025-07-20 00:29:00 +08:00
3030332422	b8136ac1dc	fix:修复阿里云流式ASR音频传递问题	2025-07-19 18:17:21 +08:00
3030332422	6d5b53fcd5	fix: 修复阿里云流式ASR终止协议缺失问题	2025-07-19 12:16:12 +08:00
hrz	5cf8eb5e71	update:优化OpenaiASR、GroqASR配置	2025-07-19 01:03:36 +08:00
3030332422	f037f4c7de	fix: 修复声纹识别场景下的意图识别问题	2025-07-18 11:34:14 +08:00
3030332422	618be6d108	update:日志文件的补充	2025-07-17 16:08:13 +08:00
hrz	7fb028205f	Merge branch 'main' into py_test_tts # Conflicts: # main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml	2025-07-17 14:32:50 +08:00
Sakura-RanChen	a09e56bbfd	Merge branch 'py_test_tts' of https://github.com/xinnan-tech/xiaozhi-esp32-server into py_test_tts	2025-07-17 11:43:49 +08:00
Sakura-RanChen	c0c01a0285	数据集回滚	2025-07-17 11:42:51 +08:00
hrzandGitHub	af0854a840	Merge pull request #1812 from xinnan-tech/openai-asr OpenAI asr	2025-07-17 11:31:54 +08:00
hrzandGitHub	f70652f165	Merge branch 'py_test_tts' into openai-asr	2025-07-17 11:31:44 +08:00
hrzandGitHub	7c440f47b6	Merge pull request #1852 from xinnan-tech/py_fix_DoubaoStreamASR 修复关于声纹识别的若干问题	2025-07-17 11:22:59 +08:00
hrzandGitHub	6967cc2293	Merge pull request #1855 from pursue-wind/fix_mcp_proxy_err fix: 修复mcp-proxy命令不存在	2025-07-17 11:22:28 +08:00
hrzandGitHub	97ad1f6f22	Merge pull request #1856 from xinnan-tech/manager-api-mcp 修改智控台删除声纹，声纹服务不在线，无法删除的业务逻辑错误	2025-07-17 11:07:49 +08:00
JianYu Zheng	f62f530f97	优化 --AgentVoicePrintServiceImpl.java 1.重写了构造器引入系统配置线程池对象 2.删除的时候异步处理删除声纹数据的方法，加快删除接口的返回	2025-07-17 10:49:51 +08:00
JianYu Zheng	815647ae02	补充日志 --AgentVoicePrintServiceImpl.java	2025-07-17 09:49:05 +08:00
JianYu Zheng	ce1e3fb04d	修改业务逻辑 --AgentVoicePrintServiceImpl.java 删除声纹的时间，声纹服务不在线或者删除失败，都允许删除	2025-07-17 09:45:27 +08:00
chan	1a86828ae6	fix: 修复mcp-proxy命令不存在	2025-07-17 09:23:41 +08:00
3030332422	8831ea22b7	update:删去一些重复、不用的代码	2025-07-16 16:43:49 +08:00
3030332422	1a411da133	fix:修复FunASR多设备连接时声纹识别配置相互覆盖的问题	2025-07-16 15:42:21 +08:00
Sakura-RanChen	e2a7534aa5	update: 优化相关处理	2025-07-16 15:03:53 +08:00
欣南科技andGitHub	f8a92e1710	Merge pull request #1845 from xinnan-tech/manager-api-mcp 优化mcp错误提示	2025-07-15 18:36:45 +08:00
3030332422	953a232654	fix: 修复使用豆包流式ASR时，不能使用声纹识别功能的问题。	2025-07-15 18:13:01 +08:00
JianYu Zheng	4afa29554f	优化map错误提示 --AgentMcpAccessPointServiceImpl.java 优化提示	2025-07-15 15:44:43 +08:00
3030332422	dc78057823	fix:修复多模块部署时远程ASR声纹识别功能未初始化的问题	2025-07-15 14:52:56 +08:00
欣南科技andGitHub	53ca586847	Merge pull request #1826 from xinnan-tech/py_fix_log fix:日志优化的补充	2025-07-15 14:40:54 +08:00
欣南科技andGitHub	4f7e96ed7d	Merge pull request #1838 from xinnan-tech/update-doc update:更新文档	2025-07-15 14:39:47 +08:00
hrz	0397ef3fad	update:更新文档	2025-07-15 13:09:04 +08:00
欣南科技andGitHub	aa38591d90	Merge pull request #1837 from xinnan-tech/update-doc update:增加开发团队信息	2025-07-15 12:00:10 +08:00
hrz	b87b82bec9	update:增加开发团队信息	2025-07-15 11:59:13 +08:00
hrzandGitHub	2e719e2ccc	Merge pull request #1830 from hsycc/main fix: 修复配置声纹识别地址为域名不带端口, 请求url错误的问题	2025-07-14 23:16:54 +08:00
hsycc	909b41174b	fix: 修复配置声纹识别地址为域名不带端口, 请求url错误的问题	2025-07-14 20:13:48 +08:00
Sakura-RanChen	2c33ee5b32	update: 优化文本处理	2025-07-14 17:34:42 +08:00
3030332422	82beae0e59	fix:日志优化的补充	2025-07-14 07:28:05 +08:00
欣南科技andGitHub	7f979e2f52	Merge pull request #1820 from xinnan-tech/update-doc update:补充声纹识别部署细节	2025-07-12 17:43:59 +08:00
hrz	e94eb302a8	update:补充声纹识别部署细节	2025-07-12 17:43:27 +08:00
欣南科技andGitHub	8ffbb614f7	Merge pull request #1819 from xinnan-tech/update-doc update：补充声纹服务启动的细节	2025-07-12 16:50:10 +08:00
hrz	7a589f4a7b	update：补充声纹服务启动的细节	2025-07-12 16:49:32 +08:00
Sakura-RanChen	04ed5ed980	update: 同步非流式处理 10秒超时链接不复用	2025-07-11 18:04:00 +08:00
Sakura-RanChen	8e5d933745	update: 优化音频播放文本发送	2025-07-11 16:36:47 +08:00
Sakura-RanChen	23cd453af8	update: aliyun双流改造待优化长连接机制和文本生成反馈为空	2025-07-11 11:36:58 +08:00
Sakura-RanChen	26a3b51162	fix: 补充文档	2025-07-10 16:07:06 +08:00
hrz	6da3138814	Merge branch 'main' into py_test_tts	2025-07-10 13:47:53 +08:00
luruxian	b2e6156bbb	chore(db): 添加新的数据库变更集202507101201 添加新的数据库变更脚本202507101201.sql到变更日志中	2025-07-10 11:44:49 +08:00
luruxian	fbd4a22e3e	feat(asr): 添加OpenAI和Groq语音识别支持新增GPT语音识别模型供应器和配置，实现语音转文本功能更新相关文档说明，包含API申请步骤和使用注意事项	2025-07-10 11:41:20 +08:00
Sakura-RanChen	dede73e9a5	fix: id一致方法流程	2025-07-10 11:34:31 +08:00
Sakura-RanChen	ce358dcd65	fix: 音频播放方法跟随主分支	2025-07-10 10:17:07 +08:00
Sakura-RanChen	d533fb83eb	fix: 插入报错修复，跟随主分支新增列	2025-07-10 09:52:54 +08:00
CGD	155ba92ba6	update:增加了智控台的sql语句，优化了部分配置	2025-07-07 15:29:13 +08:00
CGD	039badd265	update:添加了阿里云CosyVoice流式TTS	2025-07-01 18:14:36 +08:00
CGD	90a9631450	update:添加阿里云流式asr	2025-06-30 14:41:53 +08:00