mirror of
https://github.com/xinnan-tech/xiaozhi-esp32-server.git
synced 2026-07-22 07:03:53 +08:00
Merge branch 'main' into py_test_tts
This commit is contained in:
@@ -174,3 +174,4 @@ main/xiaozhi-server/mysql
|
||||
uploadfile
|
||||
*.json
|
||||
.vscode
|
||||
.cursor
|
||||
|
||||
@@ -144,6 +144,11 @@
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
|
||||
<picture>
|
||||
<img alt="MCP接入点" src="docs/images/demo13.png" />
|
||||
</picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
@@ -170,8 +175,8 @@
|
||||
#### 🚀 部署方式选择
|
||||
| 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 |
|
||||
|---------|------|---------|---------|---------|---------|
|
||||
| **最简化安装** | 智能对话、IOT、MCP、视觉感知,数据存储在配置文件 | 低配置环境,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G,如果全API,要2核2G | - |
|
||||
| **全模块安装** | 智能对话、IOT、MCP、视觉感知、OTA、智控台,数据存储在数据库 | 完整功能体验 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G,如果全API,要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
| **最简化安装** | 智能对话、IOT、MCP、视觉感知 | 低配置环境,数据存储在配置文件,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G,如果全API,要2核2G | - |
|
||||
| **全模块安装** | 智能对话、IOT、MCP接入点、视觉感知、OTA、智控台 | 完整功能体验,数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G,如果全API,要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
|
||||
|
||||
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据
|
||||
@@ -226,7 +231,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|
||||
| 视觉感知系统 | 支持多种VLLM(视觉大模型),实现多模态交互 |
|
||||
| 意图识别系统 | 支持LLM意图识别、Function Call函数调用,提供插件化意图处理机制 |
|
||||
| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆,具备记忆总结功能 |
|
||||
| IOT/MCP控制协议 | 支持设备注册管理、智能控制接口,同时支持IOT、MCP控制协议 |
|
||||
| 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
|
||||
| 管理后台 | 提供Web管理界面,支持用户管理、系统配置和设备管理 |
|
||||
| 测试工具 | 提供性能测试工具、视觉模型测试工具和音频交互测试工具 |
|
||||
| 部署支持 | 支持Docker部署和本地部署,提供完整的配置文件管理 |
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
本项目的测试平台`https://2662r3426b.vicp.fun`,从开放以来就使用了该方法,效果良好。
|
||||
|
||||
教程可参考B站博主`毕乐labs`发布的视频教程:[《开源小智服务器xiaozhi-server自动更新以及最新版本MCP接入点配置保姆教程》](https://www.bilibili.com/video/BV15H37zHE7Q)
|
||||
|
||||
# 开始条件
|
||||
- 你的电脑/服务器是linux操作系统
|
||||
- 你已经跑通了整个流程
|
||||
@@ -40,6 +42,9 @@ git clone https://ghproxy.net/https://github.com/xinnan-tech/xiaozhi-esp32-serve
|
||||
|
||||
此刻你需要把`model.pt`文件复制到新的目录去,你可以这样
|
||||
```
|
||||
# 创建需要的目录
|
||||
mkdir -p /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/data/
|
||||
|
||||
cp 你原来的.config.yaml完整路径 /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/data/.config.yaml
|
||||
cp 你原来的model.pt完整路径 /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/models/SenseVoiceSmall/model.pt
|
||||
```
|
||||
@@ -161,3 +166,11 @@ tail -f /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/tmp/server
|
||||
|
||||
# 注意事项
|
||||
测试平台`https://2662r3426b.vicp.fun`,是使用nginx做了反向代理。nginx.conf详细配置可以[参考这里](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)
|
||||
|
||||
## 常见问题
|
||||
|
||||
### 1、为什么没有见到8001端口?
|
||||
回答:8001是开发环境使用的,用于运行前端的端口。如果你是服务器部署,不建议使用`npm run serve`启动8001端口运行前端,而是像本教程一样编译成html文件,然后使用nginx来管理访问。
|
||||
|
||||
### 2、每次更新需要更新手动SQL语句吗?
|
||||
回答:不需要,因为项目使用**Liquibase**管理数据库版本,会自动执行新的sql脚本。
|
||||
@@ -42,9 +42,9 @@ http {
|
||||
proxy_set_header Referer $http_referer;
|
||||
proxy_set_header Cookie $http_cookie;
|
||||
|
||||
proxy_connect_timeout 10;
|
||||
proxy_send_timeout 10;
|
||||
proxy_read_timeout 10;
|
||||
proxy_connect_timeout 15;
|
||||
proxy_send_timeout 15;
|
||||
proxy_read_timeout 15;
|
||||
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
|
||||
@@ -6,6 +6,7 @@ java -jar /app/xiaozhi-esp32-api.jar \
|
||||
--spring.datasource.druid.username=${SPRING_DATASOURCE_DRUID_USERNAME} \
|
||||
--spring.datasource.druid.password=${SPRING_DATASOURCE_DRUID_PASSWORD} \
|
||||
--spring.data.redis.host=${SPRING_DATA_REDIS_HOST} \
|
||||
--spring.data.redis.password=${SPRING_DATA_REDIS_PASSWORD} \
|
||||
--spring.data.redis.port=${SPRING_DATA_REDIS_PORT} &
|
||||
|
||||
# 启动Nginx(前台运行保持容器存活)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 359 KiB |
@@ -0,0 +1,125 @@
|
||||
# MCP 接入点部署使用指南
|
||||
|
||||
本教程包含2个部分
|
||||
- 1、如何部署MCP接入点这个服务
|
||||
- 1、全模块部署时,怎么配置MCP接入点
|
||||
- 2、单模块部署时,怎么配置MCP接入点
|
||||
|
||||
# 1、如何部署MCP接入点这个服务
|
||||
|
||||
## 第一步,下载mcp接入点项目源码
|
||||
|
||||
浏览器打开[mcp接入点项目地址](https://github.com/xinnan-tech/mcp-endpoint-server)
|
||||
|
||||
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
|
||||
|
||||
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-endpoint-server-main`
|
||||
你需要把它重命名成`mcp-endpoint-server`。
|
||||
|
||||
## 第二步,启动程序
|
||||
这个项目是一个很简单的项目,建议使用docker运行。不过如果你不想使用docker运行,你可以参考[这个页面](https://github.com/xinnan-tech/mcp-endpoint-server/blob/main/README_dev.md)使用源码运行。以下是docker运行的方法
|
||||
|
||||
```
|
||||
# 进入本项目源码根目录
|
||||
cd mcp-endpoint-server
|
||||
|
||||
# 清除缓存
|
||||
docker compose -f docker-compose.yml down
|
||||
docker stop mcp-endpoint-server
|
||||
docker rm mcp-endpoint-server
|
||||
docker rmi ghcr.nju.edu.cn/xinnan-tech/mcp-endpoint-server:latest
|
||||
|
||||
# 启动docker容器
|
||||
docker compose -f docker-compose.yml up -d
|
||||
# 查看日志
|
||||
docker logs -f mcp-endpoint-server
|
||||
```
|
||||
|
||||
此时,日志里会输出类似以下的日志
|
||||
```
|
||||
250705 INFO-=====下面的地址分别是智控台/单模块MCP接入点地址====
|
||||
250705 INFO-智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
|
||||
250705 INFO-单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
|
||||
250705 INFO-=====请根据具体部署选择使用,请勿泄露给任何人======
|
||||
```
|
||||
|
||||
请你把两个接口地址复制出来:
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
你先把地址复制出来,放在一个草稿里,你要知道你的电脑的局域网ip是什么,例如我的电脑局域网ip是`192.168.1.25`,那么
|
||||
原来我的接口地址
|
||||
```
|
||||
智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
|
||||
单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
|
||||
```
|
||||
就要改成
|
||||
```
|
||||
智控台MCP参数配置: http://192.168.1.25:8004/mcp_endpoint/health?key=abc
|
||||
单模块部署MCP接入点: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
|
||||
```
|
||||
|
||||
改好后,请使用浏览器直接访问`智控台MCP参数配置`。当浏览器出现类似这样的代码,说明是成功了。
|
||||
```
|
||||
{"result":{"status":"success","connections":{"tool_connections":0,"robot_connections":0,"total_connections":0}},"error":null,"id":null,"jsonrpc":"2.0"}
|
||||
```
|
||||
|
||||
请你保留好上面两个`接口地址`,下一步要用到。
|
||||
|
||||
# 2、全模块部署时,怎么配置MCP接入点
|
||||
|
||||
如果你是全模块部署,使用管理员账号,登录智控台,点击顶部`参数字典`,选择`参数管理`功能。
|
||||
|
||||
然后搜索参数`server.mcp_endpoint`,此时,它的值应该是`null`值。
|
||||
点击修改按钮,把上一步得来的`智控台MCP参数配置`粘贴到`参数值`里。然后保存。
|
||||
|
||||
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问mcp接入点,很大概率是网络防火墙,或者没有填写正确的局域网ip。
|
||||
|
||||
# 3、单模块部署时,怎么配置MCP接入点
|
||||
|
||||
如果你是单模块部署,找到你的配置文件`data/.config.yaml`。
|
||||
在配置文件搜索`mcp_endpoint`,如果没有找到,你就增加`mcp_endpoint`配置。类似我是就是这样
|
||||
```
|
||||
server:
|
||||
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
|
||||
http_port: 8002
|
||||
log:
|
||||
log_level: INFO
|
||||
|
||||
# 此处可能还更多配置..
|
||||
|
||||
mcp_endpoint: 你的接入点websocket地址
|
||||
```
|
||||
这时,请你把`如何部署MCP接入点这个服务`中得到的`单模块部署MCP接入点` 粘贴到 `mcp_endpoint`中。类似这样
|
||||
|
||||
```
|
||||
server:
|
||||
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
|
||||
http_port: 8002
|
||||
log:
|
||||
log_level: INFO
|
||||
|
||||
# 此处可能还更多配置
|
||||
|
||||
mcp_endpoint: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
|
||||
```
|
||||
|
||||
配置好后,启动单模块会输出如下的日志。
|
||||
```
|
||||
250705[__main__]-INFO-初始化组件: vad成功 SileroVAD
|
||||
250705[__main__]-INFO-初始化组件: asr成功 FunASRServer
|
||||
250705[__main__]-INFO-OTA接口是 http://192.168.1.25:8002/xiaozhi/ota/
|
||||
250705[__main__]-INFO-视觉分析接口是 http://192.168.1.25:8002/mcp/vision/explain
|
||||
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
|
||||
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
|
||||
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
|
||||
250705[__main__]-INFO-=============================================================
|
||||
```
|
||||
|
||||
如上,如果能输出类似的`mcp接入点是`中`ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc`说明配置成功了。
|
||||
|
||||
@@ -1,92 +1,36 @@
|
||||
# MCP 接入点部署使用指南
|
||||
# MCP 接入点使用指南
|
||||
|
||||
本教程包含2个部分
|
||||
- 1、如何开启mcp接入点
|
||||
- 2、如何为智能体接入一个简单的mcp功能,如计算器功能
|
||||
本教程以虾哥开源的mcp计算器功能为示例,介绍如何将自己自定义的mcp服务接入到自己的接入点里。
|
||||
|
||||
部署的前提条件:
|
||||
- 1、你已经部署了全模块,因为mcp接入点需要全模块中的智控台功能
|
||||
- 2、你想在不修改xiaozhi-server项目的前提下,扩展小智的功能
|
||||
|
||||
# 如何开启mcp接入点
|
||||
|
||||
## 第一步,下载mcp接入点项目源码
|
||||
|
||||
浏览器打开[mcp接入点项目地址](https://github.com/xinnan-tech/mcp-endpoint-server)
|
||||
|
||||
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
|
||||
|
||||
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-endpoint-server-main`
|
||||
你需要把它重命名成`mcp-endpoint-server`。
|
||||
|
||||
## 第二步,启动程序
|
||||
这个项目是一个很简单的项目,建议使用docker运行。不过如果你不想使用docker运行,你可以参考[这个页面](https://github.com/xinnan-tech/mcp-endpoint-server/blob/main/README_dev.md)使用源码运行。以下是docker运行的方法
|
||||
|
||||
```
|
||||
# 进入本项目源码根目录
|
||||
cd mcp-endpoint-server
|
||||
|
||||
# 清除缓存
|
||||
docker compose -f docker-compose.yml down
|
||||
docker stop mcp-endpoint-server
|
||||
docker rm mcp-endpoint-server
|
||||
docker rmi ghcr.nju.edu.cn/xinnan-tech/mcp-endpoint-server:latest
|
||||
|
||||
# 启动docker容器
|
||||
docker compose -f docker-compose.yml up -d
|
||||
# 查看日志
|
||||
docker logs -f mcp-endpoint-server
|
||||
```
|
||||
|
||||
此时,日志里会输出类似以下的日志
|
||||
```
|
||||
======================================================
|
||||
接口地址: http://172.1.1.1:8004/mcp_endpoint/health?key=xxxx
|
||||
=======上面的地址是MCP接入点地址,请勿泄露给任何人============
|
||||
```
|
||||
|
||||
请你把接口地址复制出来:
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
你先把地址复制出来,放在一个草稿里,你要知道你的电脑的局域网ip是什么,例如我的电脑局域网ip是`192.168.1.25`,那么
|
||||
原来我的接口地址
|
||||
```
|
||||
http://172.1.1.1:8004/mcp_endpoint/health?key=xxxx
|
||||
```
|
||||
就要改成
|
||||
```
|
||||
http://192.168.1.25:8004/mcp_endpoint/health?key=xxxx
|
||||
```
|
||||
|
||||
改好后,请使用浏览器直接访问这个接口。当浏览器出现类似这样的代码,说明是成功了。
|
||||
```
|
||||
{"result":{"status":"success","connections":{"tool_connections":0,"robot_connections":0,"total_connections":0}},"error":null,"id":null,"jsonrpc":"2.0"}
|
||||
```
|
||||
|
||||
请你保留好这个`接口地址`,下一步要用到。
|
||||
|
||||
## 第三步,配置智控台
|
||||
|
||||
使用管理员账号,登录智控台,点击顶部`参数字典`,选择`参数管理`功能。
|
||||
|
||||
然后搜索参数`server.mcp_endpoint`,此时,它的值应该是`null`值。
|
||||
点击修改按钮,把上一步得来的`接口地址`粘贴到`参数值`里。然后保存。
|
||||
|
||||
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问mcp接入点,很大概率是网络防火墙,或者没有填写正确的局域网ip。
|
||||
本教程的前提是,你的`xiaozhi-server`已经启用了mcp接入点功能,如果你还没启用,可以先根据[这个教程](./mcp-endpoint-enable.md)启用。
|
||||
|
||||
# 如何为智能体接入一个简单的mcp功能,如计算器功能
|
||||
|
||||
如果以上步骤顺利,你可以进入智能体管理,点击`配置角色`,在`意图识别`的右边,有一个`编辑功能`的按钮。
|
||||
### 如果你是全模块部署
|
||||
如果你是全模块部署,你可以进入智控台,智能体管理,点击`配置角色`,在`意图识别`的右边,有一个`编辑功能`的按钮。
|
||||
|
||||
点击这个按钮。在弹出的页面里,位于底部,会有`MCP接入点`,正常来说,会显示这个智能体的`MCP接入点地址`,接下来,我们来给这个智能体扩展一个基于MCP技术的计算器的功能。
|
||||
|
||||
这个`MCP接入点地址`很重要,你等一下会用到。
|
||||
|
||||
### 如果你是单模块部署
|
||||
如果你是单模块部署,且你已经在配置文件里配置了MCP接入点地址,那么正常来说,单模块部署启动的时候,会输出如下的日志。
|
||||
```
|
||||
250705[__main__]-INFO-初始化组件: vad成功 SileroVAD
|
||||
250705[__main__]-INFO-初始化组件: asr成功 FunASRServer
|
||||
250705[__main__]-INFO-OTA接口是 http://192.168.1.25:8002/xiaozhi/ota/
|
||||
250705[__main__]-INFO-视觉分析接口是 http://192.168.1.25:8002/mcp/vision/explain
|
||||
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
|
||||
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
|
||||
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
|
||||
250705[__main__]-INFO-=============================================================
|
||||
```
|
||||
|
||||
如上,输出`mcp接入点是`中`ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc`就是你的`MCP接入点地址`。
|
||||
|
||||
这个`MCP接入点地址`很重要,你等一下会用到。
|
||||
|
||||
## 第一步 下载虾哥MCP计算器项目代码
|
||||
|
||||
浏览器打开虾哥写的[计算器项目](https://github.com/78/mcp-calculator),
|
||||
@@ -114,13 +58,13 @@ pip install -r requirements.txt
|
||||
|
||||
例如我的智能体的mcp地址是
|
||||
```
|
||||
ws://192.168.4.7:8004/mcp_endpoint/mcp/?token=abc
|
||||
ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
```
|
||||
|
||||
开始输入命令
|
||||
|
||||
```bash
|
||||
export MCP_ENDPOINT=ws://192.168.4.7:8004/mcp_endpoint/mcp/?token=abc
|
||||
export MCP_ENDPOINT=ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
```
|
||||
|
||||
输入完后,启动程序
|
||||
@@ -129,6 +73,22 @@ export MCP_ENDPOINT=ws://192.168.4.7:8004/mcp_endpoint/mcp/?token=abc
|
||||
python mcp_pipe.py calculator.py
|
||||
```
|
||||
|
||||
### 如果你是智控台部署
|
||||
如果你是智控台部署,启动完后,你再进入智控台,点击刷新MCP的接入状态,就会看到你扩展的功能列表了。
|
||||
|
||||
启动完后,你再进入智控台,点击刷新MCP的接入状态,就会看到你扩展的功能列表了。
|
||||
### 如果你是单模块部署
|
||||
如果你是单模块部署,当设备连接后,会输出类似的日志,说明成功了
|
||||
|
||||
```
|
||||
250705 -INFO-正在初始化MCP接入点: wss://2662r3426b.vicp.fun/mcp_e
|
||||
250705 -INFO-发送MCP接入点初始化消息
|
||||
250705 -INFO-MCP接入点连接成功
|
||||
250705 -INFO-MCP接入点初始化成功
|
||||
250705 -INFO-统一工具处理器初始化完成
|
||||
250705 -INFO-MCP接入点服务器信息: name=Calculator, version=1.9.4
|
||||
250705 -INFO-MCP接入点支持的工具数量: 1
|
||||
250705 -INFO-所有MCP接入点工具已获取,客户端准备就绪
|
||||
250705 -INFO-工具缓存已刷新
|
||||
250705 -INFO-当前支持的函数列表: [ 'get_time', 'get_lunar', 'play_music', 'get_weather', 'handle_exit_intent', 'calculator']
|
||||
```
|
||||
如果包含了 `'calculator'`,说明设备将可以根据意图识别,调用计算器这个工具。
|
||||
@@ -0,0 +1,105 @@
|
||||
# get_news_from_newsnow 插件新闻源配置指南
|
||||
|
||||
## 概述
|
||||
|
||||
`get_news_from_newsnow` 插件现在支持通过Web管理界面动态配置新闻源,不再需要修改代码。用户可以在智控台中为每个智能体配置不同的新闻源。
|
||||
|
||||
## 配置方式
|
||||
|
||||
### 1. 通过Web管理界面配置(推荐)
|
||||
|
||||
1. 登录智控台
|
||||
2. 进入"角色配置"页面
|
||||
3. 选择要配置的智能体
|
||||
4. 点击"编辑功能"按钮
|
||||
5. 在右侧参数配置区域找到"newsnow新闻聚合"插件
|
||||
6. 在"新闻源配置"字段中输入分号分隔的中文名称
|
||||
|
||||
### 2. 配置文件方式
|
||||
|
||||
在 `config.yaml` 中配置:
|
||||
|
||||
```yaml
|
||||
plugins:
|
||||
get_news_from_newsnow:
|
||||
url: "https://newsnow.busiyi.world/api/s?id="
|
||||
news_sources: "澎湃新闻;百度热搜;财联社;微博;抖音"
|
||||
```
|
||||
|
||||
## 新闻源配置格式
|
||||
|
||||
新闻源配置使用分号分隔的中文名称,格式为:
|
||||
|
||||
```
|
||||
中文名称1;中文名称2;中文名称3
|
||||
```
|
||||
|
||||
### 配置示例
|
||||
|
||||
```
|
||||
澎湃新闻;百度热搜;财联社;微博;抖音;知乎;36氪
|
||||
```
|
||||
|
||||
## 支持的新闻源
|
||||
|
||||
插件支持以下新闻源的中文名称:
|
||||
|
||||
- 澎湃新闻
|
||||
- 百度热搜
|
||||
- 财联社
|
||||
- 微博
|
||||
- 抖音
|
||||
- 知乎
|
||||
- 36氪
|
||||
- 华尔街见闻
|
||||
- IT之家
|
||||
- 今日头条
|
||||
- 虎扑
|
||||
- 哔哩哔哩
|
||||
- 快手
|
||||
- 雪球
|
||||
- 格隆汇
|
||||
- 法布财经
|
||||
- 金十数据
|
||||
- 牛客
|
||||
- 少数派
|
||||
- 稀土掘金
|
||||
- 凤凰网
|
||||
- 虫部落
|
||||
- 联合早报
|
||||
- 酷安
|
||||
- 远景论坛
|
||||
- 参考消息
|
||||
- 卫星通讯社
|
||||
- 百度贴吧
|
||||
- 靠谱新闻
|
||||
- 以及更多...
|
||||
|
||||
## 默认配置
|
||||
|
||||
如果未配置新闻源,插件将使用以下默认配置:
|
||||
|
||||
```
|
||||
澎湃新闻;百度热搜;财联社
|
||||
```
|
||||
|
||||
## 使用说明
|
||||
|
||||
1. **配置新闻源**:在Web界面或配置文件中设置新闻源的中文名称,用分号分隔
|
||||
2. **调用插件**:用户可以说"播报新闻"或"获取新闻"
|
||||
3. **指定新闻源**:用户可以说"播报澎湃新闻"或"获取百度热搜"
|
||||
4. **获取详情**:用户可以说"详细介绍这条新闻"
|
||||
|
||||
## 工作原理
|
||||
|
||||
1. 插件接受中文名称作为参数(如"澎湃新闻")
|
||||
2. 根据配置的新闻源列表,将中文名称转换为对应的英文ID(如"thepaper")
|
||||
3. 使用英文ID调用API获取新闻数据
|
||||
4. 返回新闻内容给用户
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. 配置的中文名称必须与 CHANNEL_MAP 中定义的名称完全一致
|
||||
2. 配置更改后需要重启服务或重新加载配置
|
||||
3. 如果配置的新闻源无效,插件会自动使用默认新闻源
|
||||
4. 多个新闻源之间使用英文分号(;)分隔,不要使用中文分号(;)
|
||||
@@ -232,7 +232,7 @@ public interface Constant {
|
||||
/**
|
||||
* 版本号
|
||||
*/
|
||||
public static final String VERSION = "0.6.1";
|
||||
public static final String VERSION = "0.6.3";
|
||||
|
||||
/**
|
||||
* 无效固件URL
|
||||
|
||||
+12
@@ -19,6 +19,8 @@ import xiaozhi.modules.agent.service.AgentChatAudioService;
|
||||
import xiaozhi.modules.agent.service.AgentChatHistoryService;
|
||||
import xiaozhi.modules.agent.service.AgentService;
|
||||
import xiaozhi.modules.agent.service.biz.AgentChatHistoryBizService;
|
||||
import xiaozhi.modules.device.entity.DeviceEntity;
|
||||
import xiaozhi.modules.device.service.DeviceService;
|
||||
|
||||
/**
|
||||
* {@link AgentChatHistoryBizService} impl
|
||||
@@ -35,6 +37,7 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
|
||||
private final AgentChatHistoryService agentChatHistoryService;
|
||||
private final AgentChatAudioService agentChatAudioService;
|
||||
private final RedisUtils redisUtils;
|
||||
private final DeviceService deviceService;
|
||||
|
||||
/**
|
||||
* 处理聊天记录上报,包括文件上传和相关信息记录
|
||||
@@ -68,6 +71,15 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
|
||||
|
||||
// 更新设备最后对话时间
|
||||
redisUtils.set(RedisKeys.getAgentDeviceLastConnectedAtById(agentId), new Date());
|
||||
|
||||
// 更新设备最后连接时间
|
||||
DeviceEntity device = deviceService.getDeviceByMacAddress(macAddress);
|
||||
if (device != null) {
|
||||
deviceService.updateDeviceConnectionInfo(agentId, device.getId(), null);
|
||||
} else {
|
||||
log.warn("聊天记录上报时,未找到mac地址为 {} 的设备", macAddress);
|
||||
}
|
||||
|
||||
return Boolean.TRUE;
|
||||
}
|
||||
|
||||
|
||||
+96
-30
@@ -5,6 +5,7 @@ import java.net.URISyntaxException;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
@@ -18,7 +19,6 @@ import xiaozhi.common.utils.AESUtils;
|
||||
import xiaozhi.common.utils.HashEncryptionUtil;
|
||||
import xiaozhi.common.utils.JsonUtils;
|
||||
import xiaozhi.modules.agent.dto.McpJsonRpcRequest;
|
||||
import xiaozhi.modules.agent.dto.McpJsonRpcResponse;
|
||||
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
|
||||
import xiaozhi.modules.sys.service.SysParamsService;
|
||||
import xiaozhi.modules.sys.utils.WebSocketClientManager;
|
||||
@@ -61,53 +61,119 @@ public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointServic
|
||||
wsUrl = wsUrl.replace("/mcp/", "/call/");
|
||||
|
||||
try {
|
||||
// 创建 WebSocket 连接
|
||||
// 创建 WebSocket 连接,增加超时时间到15秒
|
||||
try (WebSocketClientManager client = WebSocketClientManager.build(
|
||||
new WebSocketClientManager.Builder()
|
||||
.uri(wsUrl)
|
||||
.connectTimeout(5, TimeUnit.SECONDS)
|
||||
.maxSessionDuration(20, TimeUnit.SECONDS))) {
|
||||
.connectTimeout(8, TimeUnit.SECONDS)
|
||||
.maxSessionDuration(10, TimeUnit.SECONDS))) {
|
||||
|
||||
// 发送初始化通知
|
||||
McpJsonRpcRequest initRequest = new McpJsonRpcRequest("notifications/initialized");
|
||||
client.sendJson(initRequest);
|
||||
// 步骤1: 发送初始化消息并等待响应
|
||||
log.info("发送MCP初始化消息,智能体ID: {}", id);
|
||||
McpJsonRpcRequest initializeRequest = new McpJsonRpcRequest("initialize",
|
||||
Map.of(
|
||||
"protocolVersion", "2024-11-05",
|
||||
"capabilities", Map.of(
|
||||
"roots", Map.of("listChanged", false),
|
||||
"sampling", Map.of()),
|
||||
"clientInfo", Map.of(
|
||||
"name", "xz-mcp-broker",
|
||||
"version", "0.0.1")),
|
||||
1);
|
||||
client.sendJson(initializeRequest);
|
||||
|
||||
// 等待 0.2 秒
|
||||
Thread.sleep(200);
|
||||
|
||||
// 发送工具列表请求
|
||||
McpJsonRpcRequest toolsRequest = new McpJsonRpcRequest("tools/list", null, 1);
|
||||
client.sendJson(toolsRequest);
|
||||
|
||||
// 监听响应,直到收到包含 id=1 的响应
|
||||
List<String> responses = client.listener(response -> {
|
||||
// 等待初始化响应 (id=1) - 移除固定延迟,改为响应驱动
|
||||
List<String> initResponses = client.listenerWithoutClose(response -> {
|
||||
try {
|
||||
McpJsonRpcResponse jsonResponse = JsonUtils.parseObject(response, McpJsonRpcResponse.class);
|
||||
return jsonResponse != null && Integer.valueOf(1).equals(jsonResponse.getId());
|
||||
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
|
||||
if (jsonMap != null && Integer.valueOf(1).equals(jsonMap.get("id"))) {
|
||||
// 检查是否有result字段,表示初始化成功
|
||||
return jsonMap.containsKey("result") && !jsonMap.containsKey("error");
|
||||
}
|
||||
return false;
|
||||
} catch (Exception e) {
|
||||
log.warn("解析响应失败: {}", response, e);
|
||||
log.warn("解析初始化响应失败: {}", response, e);
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
// 处理响应
|
||||
for (String response : responses) {
|
||||
// 验证初始化响应
|
||||
boolean initSucceeded = false;
|
||||
for (String response : initResponses) {
|
||||
try {
|
||||
McpJsonRpcResponse jsonResponse = JsonUtils.parseObject(response, McpJsonRpcResponse.class);
|
||||
if (jsonResponse != null && Integer.valueOf(1).equals(jsonResponse.getId())
|
||||
&& jsonResponse.getResult() != null && jsonResponse.getResult().getTools() != null) {
|
||||
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
|
||||
if (jsonMap != null && Integer.valueOf(1).equals(jsonMap.get("id"))) {
|
||||
if (jsonMap.containsKey("result")) {
|
||||
log.info("MCP初始化成功,智能体ID: {}", id);
|
||||
initSucceeded = true;
|
||||
break;
|
||||
} else if (jsonMap.containsKey("error")) {
|
||||
log.error("MCP初始化失败,智能体ID: {}, 错误: {}", id, jsonMap.get("error"));
|
||||
return List.of();
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("处理初始化响应失败: {}", response, e);
|
||||
}
|
||||
}
|
||||
|
||||
// 提取工具名称列表
|
||||
return java.util.Arrays.stream(jsonResponse.getResult().getTools())
|
||||
.map(McpJsonRpcResponse.McpTool::getName)
|
||||
.collect(Collectors.toList());
|
||||
if (!initSucceeded) {
|
||||
log.error("未收到有效的MCP初始化响应,智能体ID: {}", id);
|
||||
return List.of();
|
||||
}
|
||||
|
||||
// 步骤2: 发送初始化完成通知 - 只有在收到initialize响应后才发送
|
||||
log.info("发送MCP初始化完成通知,智能体ID: {}", id);
|
||||
String notificationJson = "{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}";
|
||||
client.sendText(notificationJson);
|
||||
|
||||
// 步骤3: 发送工具列表请求 - 立即发送,无需额外延迟
|
||||
log.info("发送MCP工具列表请求,智能体ID: {}", id);
|
||||
McpJsonRpcRequest toolsRequest = new McpJsonRpcRequest("tools/list", null, 2);
|
||||
client.sendJson(toolsRequest);
|
||||
|
||||
// 等待工具列表响应 (id=2)
|
||||
List<String> toolsResponses = client.listener(response -> {
|
||||
try {
|
||||
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
|
||||
return jsonMap != null && Integer.valueOf(2).equals(jsonMap.get("id"));
|
||||
} catch (Exception e) {
|
||||
log.warn("解析工具列表响应失败: {}", response, e);
|
||||
return false;
|
||||
}
|
||||
});
|
||||
|
||||
// 处理工具列表响应
|
||||
for (String response : toolsResponses) {
|
||||
try {
|
||||
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
|
||||
if (jsonMap != null && Integer.valueOf(2).equals(jsonMap.get("id"))) {
|
||||
// 检查是否有result字段
|
||||
Object resultObj = jsonMap.get("result");
|
||||
if (resultObj instanceof Map) {
|
||||
Map<String, Object> resultMap = (Map<String, Object>) resultObj;
|
||||
Object toolsObj = resultMap.get("tools");
|
||||
if (toolsObj instanceof List) {
|
||||
List<Map<String, Object>> toolsList = (List<Map<String, Object>>) toolsObj;
|
||||
// 提取工具名称列表
|
||||
List<String> result = toolsList.stream()
|
||||
.map(tool -> (String) tool.get("name"))
|
||||
.filter(name -> name != null)
|
||||
.collect(Collectors.toList());
|
||||
log.info("成功获取MCP工具列表,智能体ID: {}, 工具数量: {}", id, result.size());
|
||||
return result;
|
||||
}
|
||||
} else if (jsonMap.containsKey("error")) {
|
||||
log.error("获取工具列表失败,智能体ID: {}, 错误: {}", id, jsonMap.get("error"));
|
||||
return List.of();
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("处理工具列表响应失败: {}", response, e);
|
||||
}
|
||||
}
|
||||
|
||||
log.warn("未找到有效的工具列表响应");
|
||||
log.warn("未找到有效的工具列表响应,智能体ID: {}", id);
|
||||
return List.of();
|
||||
|
||||
}
|
||||
@@ -177,4 +243,4 @@ public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointServic
|
||||
// 加密后成token值
|
||||
return AESUtils.encrypt(key, json);
|
||||
}
|
||||
}
|
||||
}
|
||||
+32
-12
@@ -72,6 +72,8 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
agent.getVadModelId(),
|
||||
agent.getAsrModelId(),
|
||||
null,
|
||||
@@ -108,9 +110,13 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
}
|
||||
// 获取音色信息
|
||||
String voice = null;
|
||||
String referenceAudio = null;
|
||||
String referenceText = null;
|
||||
TimbreDetailsVO timbre = timbreService.get(agent.getTtsVoiceId());
|
||||
if (timbre != null) {
|
||||
voice = timbre.getTtsVoice();
|
||||
referenceAudio = timbre.getReferenceAudio();
|
||||
referenceText = timbre.getReferenceText();
|
||||
}
|
||||
// 构建返回数据
|
||||
Map<String, Object> result = new HashMap<>();
|
||||
@@ -163,6 +169,8 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
agent.getSystemPrompt(),
|
||||
agent.getSummaryMemory(),
|
||||
voice,
|
||||
referenceAudio,
|
||||
referenceText,
|
||||
agent.getVadModelId(),
|
||||
agent.getAsrModelId(),
|
||||
agent.getLlmModelId(),
|
||||
@@ -179,7 +187,7 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
/**
|
||||
* 构建配置信息
|
||||
*
|
||||
* @param paramsList 系统参数列表
|
||||
* @param config 系统参数列表
|
||||
* @return 配置信息
|
||||
*/
|
||||
private Object buildConfig(Map<String, Object> config) {
|
||||
@@ -250,21 +258,25 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
/**
|
||||
* 构建模块配置
|
||||
*
|
||||
* @param prompt 提示词
|
||||
* @param voice 音色
|
||||
* @param vadModelId VAD模型ID
|
||||
* @param asrModelId ASR模型ID
|
||||
* @param llmModelId LLM模型ID
|
||||
* @param ttsModelId TTS模型ID
|
||||
* @param memModelId 记忆模型ID
|
||||
* @param intentModelId 意图模型ID
|
||||
* @param result 结果Map
|
||||
* @param prompt 提示词
|
||||
* @param voice 音色
|
||||
* @param referenceAudio 参考音频路径
|
||||
* @param referenceText 参考文本
|
||||
* @param vadModelId VAD模型ID
|
||||
* @param asrModelId ASR模型ID
|
||||
* @param llmModelId LLM模型ID
|
||||
* @param ttsModelId TTS模型ID
|
||||
* @param memModelId 记忆模型ID
|
||||
* @param intentModelId 意图模型ID
|
||||
* @param result 结果Map
|
||||
*/
|
||||
private void buildModuleConfig(
|
||||
String assistantName,
|
||||
String prompt,
|
||||
String summaryMemory,
|
||||
String voice,
|
||||
String referenceAudio,
|
||||
String referenceText,
|
||||
String vadModelId,
|
||||
String asrModelId,
|
||||
String llmModelId,
|
||||
@@ -286,12 +298,20 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
continue;
|
||||
}
|
||||
ModelConfigEntity model = modelConfigService.getModelById(modelIds[i], isCache);
|
||||
if (model == null) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> typeConfig = new HashMap<>();
|
||||
if (model.getConfigJson() != null) {
|
||||
typeConfig.put(model.getId(), model.getConfigJson());
|
||||
// 如果是TTS类型,添加private_voice属性
|
||||
if ("TTS".equals(modelTypes[i]) && voice != null) {
|
||||
((Map<String, Object>) model.getConfigJson()).put("private_voice", voice);
|
||||
if ("TTS".equals(modelTypes[i])) {
|
||||
if (voice != null)
|
||||
((Map<String, Object>) model.getConfigJson()).put("private_voice", voice);
|
||||
if (referenceAudio != null)
|
||||
((Map<String, Object>) model.getConfigJson()).put("ref_audio", referenceAudio);
|
||||
if (referenceText != null)
|
||||
((Map<String, Object>) model.getConfigJson()).put("ref_text", referenceText);
|
||||
}
|
||||
// 如果是Intent类型,且type=intent_llm,则给他添加附加模型
|
||||
if ("Intent".equals(modelTypes[i])) {
|
||||
|
||||
+10
@@ -25,6 +25,7 @@ import xiaozhi.common.utils.Result;
|
||||
import xiaozhi.modules.device.dto.DeviceRegisterDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceUnBindDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceUpdateDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
|
||||
import xiaozhi.modules.device.entity.DeviceEntity;
|
||||
import xiaozhi.modules.device.service.DeviceService;
|
||||
import xiaozhi.modules.security.user.SecurityUser;
|
||||
@@ -99,4 +100,13 @@ public class DeviceController {
|
||||
deviceService.updateById(entity);
|
||||
return new Result<Void>();
|
||||
}
|
||||
|
||||
@PostMapping("/manual-add")
|
||||
@Operation(summary = "手动添加设备")
|
||||
@RequiresPermissions("sys:role:normal")
|
||||
public Result<Void> manualAddDevice(@RequestBody @Valid DeviceManualAddDTO dto) {
|
||||
UserDetail user = SecurityUser.getUser();
|
||||
deviceService.manualAddDevice(user.getId(), dto);
|
||||
return new Result<>();
|
||||
}
|
||||
}
|
||||
@@ -51,13 +51,12 @@ public class OTAController {
|
||||
if (StringUtils.isBlank(clientId)) {
|
||||
clientId = deviceId;
|
||||
}
|
||||
String macAddress = deviceReportReqDTO.getMacAddress();
|
||||
boolean macAddressValid = isMacAddressValid(macAddress);
|
||||
boolean macAddressValid = isMacAddressValid(deviceId);
|
||||
// 设备Id和Mac地址应是一致的, 并且必须需要application字段
|
||||
if (!deviceId.equals(macAddress) || !macAddressValid || deviceReportReqDTO.getApplication() == null) {
|
||||
return createResponse(DeviceReportRespDTO.createError("Invalid OTA request"));
|
||||
if (!macAddressValid) {
|
||||
return createResponse(DeviceReportRespDTO.createError("Invalid device ID"));
|
||||
}
|
||||
return createResponse(deviceService.checkDeviceActive(macAddress, clientId, deviceReportReqDTO));
|
||||
return createResponse(deviceService.checkDeviceActive(deviceId, clientId, deviceReportReqDTO));
|
||||
}
|
||||
|
||||
@Operation(summary = "设备快速检查激活状态")
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
package xiaozhi.modules.device.dto;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
public class DeviceManualAddDTO {
|
||||
private String agentId;
|
||||
private String board; // 设备型号
|
||||
private String appVersion; // 固件版本
|
||||
private String macAddress; // Mac地址
|
||||
}
|
||||
@@ -8,6 +8,7 @@ import xiaozhi.common.service.BaseService;
|
||||
import xiaozhi.modules.device.dto.DevicePageUserDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceReportReqDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceReportRespDTO;
|
||||
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
|
||||
import xiaozhi.modules.device.entity.DeviceEntity;
|
||||
import xiaozhi.modules.device.vo.UserShowDeviceListVO;
|
||||
|
||||
@@ -87,5 +88,14 @@ public interface DeviceService extends BaseService<DeviceEntity> {
|
||||
*/
|
||||
Date getLatestLastConnectionTime(String agentId);
|
||||
|
||||
/**
|
||||
* 手动添加设备
|
||||
*/
|
||||
void manualAddDevice(Long userId, DeviceManualAddDTO dto);
|
||||
|
||||
/**
|
||||
* 更新设备连接信息
|
||||
*/
|
||||
void updateDeviceConnectionInfo(String agentId, String deviceId, String appVersion);
|
||||
|
||||
}
|
||||
+27
@@ -44,6 +44,7 @@ import xiaozhi.modules.device.vo.UserShowDeviceListVO;
|
||||
import xiaozhi.modules.security.user.SecurityUser;
|
||||
import xiaozhi.modules.sys.service.SysParamsService;
|
||||
import xiaozhi.modules.sys.service.SysUserUtilService;
|
||||
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
|
||||
|
||||
@Slf4j
|
||||
@Service
|
||||
@@ -410,4 +411,30 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void manualAddDevice(Long userId, DeviceManualAddDTO dto) {
|
||||
// 检查mac是否已存在
|
||||
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
|
||||
wrapper.eq("mac_address", dto.getMacAddress());
|
||||
DeviceEntity exist = baseDao.selectOne(wrapper);
|
||||
if (exist != null) {
|
||||
throw new RenException("该Mac地址已存在");
|
||||
}
|
||||
Date now = new Date();
|
||||
DeviceEntity entity = new DeviceEntity();
|
||||
entity.setId(dto.getMacAddress());
|
||||
entity.setUserId(userId);
|
||||
entity.setAgentId(dto.getAgentId());
|
||||
entity.setBoard(dto.getBoard());
|
||||
entity.setAppVersion(dto.getAppVersion());
|
||||
entity.setMacAddress(dto.getMacAddress());
|
||||
entity.setCreateDate(now);
|
||||
entity.setUpdateDate(now);
|
||||
entity.setLastConnectedAt(now);
|
||||
entity.setCreator(userId);
|
||||
entity.setUpdater(userId);
|
||||
entity.setAutoUpdate(1);
|
||||
baseDao.insert(entity);
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -218,7 +218,7 @@ public class SysParamsController {
|
||||
if (response.getStatusCode() != HttpStatus.OK) {
|
||||
throw new RenException("MCP接口访问失败,状态码:" + response.getStatusCode());
|
||||
}
|
||||
// 检查响应内容是否包含OTA相关信息
|
||||
// 检查响应内容是否包含mcp相关信息
|
||||
String body = response.getBody();
|
||||
if (body == null || !body.contains("success")) {
|
||||
throw new RenException("MCP接口返回内容格式不正确,可能不是一个真实的MCP接口");
|
||||
|
||||
@@ -137,6 +137,37 @@ public class WebSocketClientManager implements Closeable {
|
||||
return collected;
|
||||
}
|
||||
|
||||
private <T> List<T> listenerCustomWithoutClose(
|
||||
BlockingQueue<T> queue,
|
||||
Predicate<T> predicate)
|
||||
throws InterruptedException, TimeoutException, ExecutionException {
|
||||
List<T> collected = new ArrayList<>();
|
||||
long deadline = System.currentTimeMillis() + maxSessionDurationUnit.toMillis(maxSessionDuration);
|
||||
|
||||
while (true) {
|
||||
if (errorFuture.isDone()) {
|
||||
errorFuture.get();
|
||||
}
|
||||
|
||||
long remaining = deadline - System.currentTimeMillis();
|
||||
if (remaining <= 0) {
|
||||
throw new TimeoutException("等待批量消息超时");
|
||||
}
|
||||
|
||||
T msg = queue.poll(remaining, TimeUnit.MILLISECONDS);
|
||||
if (msg == null) {
|
||||
throw new TimeoutException("等待批量消息超时");
|
||||
}
|
||||
|
||||
collected.add(msg);
|
||||
if (predicate.test(msg)) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
// 不调用 close(),保持连接开放
|
||||
return collected;
|
||||
}
|
||||
|
||||
/**
|
||||
* 同步接收多条消息,直到 predicate 为 true 或超时抛异常;
|
||||
*
|
||||
@@ -147,6 +178,17 @@ public class WebSocketClientManager implements Closeable {
|
||||
return listenerCustom(textMessageQueue, predicate);
|
||||
}
|
||||
|
||||
/**
|
||||
* 同步接收多条消息,直到 predicate 为 true 或超时抛异常;
|
||||
* 不自动关闭连接,适用于需要在同一连接上发送多个消息的场景
|
||||
*
|
||||
* @return 返回监听期间的所有消息列表
|
||||
*/
|
||||
public List<String> listenerWithoutClose(Predicate<String> predicate)
|
||||
throws InterruptedException, TimeoutException, ExecutionException {
|
||||
return listenerCustomWithoutClose(textMessageQueue, predicate);
|
||||
}
|
||||
|
||||
public List<byte[]> listenerBinary(Predicate<byte[]> predicate)
|
||||
throws InterruptedException, TimeoutException, ExecutionException {
|
||||
return listenerCustom(binaryMessageQueue, predicate);
|
||||
|
||||
@@ -26,6 +26,12 @@ public class TimbreDataDTO {
|
||||
@Schema(description = "备注")
|
||||
private String remark;
|
||||
|
||||
@Schema(description = "参考音频路径")
|
||||
private String referenceAudio;
|
||||
|
||||
@Schema(description = "參考文本")
|
||||
private String referenceText;
|
||||
|
||||
@Schema(description = "排序")
|
||||
@Min(value = 0, message = "{sort.number}")
|
||||
private long sort;
|
||||
|
||||
@@ -34,6 +34,12 @@ public class TimbreEntity {
|
||||
@Schema(description = "备注")
|
||||
private String remark;
|
||||
|
||||
@Schema(description = "参考音频路径")
|
||||
private String referenceAudio;
|
||||
|
||||
@Schema(description = "參考文本")
|
||||
private String referenceText;
|
||||
|
||||
@Schema(description = "排序")
|
||||
private long sort;
|
||||
|
||||
|
||||
@@ -25,6 +25,12 @@ public class TimbreDetailsVO implements Serializable {
|
||||
@Schema(description = "备注")
|
||||
private String remark;
|
||||
|
||||
@Schema(description = "参考音频路径")
|
||||
private String referenceAudio;
|
||||
|
||||
@Schema(description = "參考文本")
|
||||
private String referenceText;
|
||||
|
||||
@Schema(description = "排序")
|
||||
private long sort;
|
||||
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
ALTER TABLE `ai_tts_voice`
|
||||
ADD COLUMN `reference_audio` VARCHAR(500) DEFAULT NULL COMMENT '参考音频路径' AFTER `remark`,
|
||||
ADD COLUMN `reference_text` VARCHAR(500) DEFAULT NULL COMMENT '参考文本' AFTER `reference_audio`;
|
||||
@@ -0,0 +1,18 @@
|
||||
-- 更新现有的 get_news_from_newsnow 插件配置
|
||||
UPDATE ai_model_provider
|
||||
SET fields = JSON_ARRAY(
|
||||
JSON_OBJECT(
|
||||
'key', 'url',
|
||||
'type', 'string',
|
||||
'label', '接口地址',
|
||||
'default', 'https://newsnow.busiyi.world/api/s?id='
|
||||
),
|
||||
JSON_OBJECT(
|
||||
'key', 'news_sources',
|
||||
'type', 'string',
|
||||
'label', '新闻源配置',
|
||||
'default', '澎湃新闻;百度热搜;财联社'
|
||||
)
|
||||
)
|
||||
WHERE provider_code = 'get_news_from_newsnow'
|
||||
AND model_type = 'Plugin';
|
||||
@@ -205,6 +205,13 @@ databaseChangeLog:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202506080955.sql
|
||||
- changeSet:
|
||||
id: 202506091720
|
||||
author: shane0411
|
||||
changes:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202506091720.sql
|
||||
- changeSet:
|
||||
id: 202506161101
|
||||
author: hrz
|
||||
@@ -219,6 +226,13 @@ databaseChangeLog:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202506191643.sql
|
||||
- changeSet:
|
||||
id: 202506251620
|
||||
author: Tink
|
||||
changes:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202506251620.sql
|
||||
- changeSet:
|
||||
id: 202506261637
|
||||
author: hrz
|
||||
|
||||
@@ -1 +1 @@
|
||||
redis.call('FLUSHALL')
|
||||
redis.call('FLUSHDB')
|
||||
@@ -7,6 +7,7 @@ import model from './module/model.js'
|
||||
import ota from './module/ota.js'
|
||||
import timbre from "./module/timbre.js"
|
||||
import user from './module/user.js'
|
||||
|
||||
/**
|
||||
* 接口地址
|
||||
* 开发时自动读取使用.env.development文件
|
||||
@@ -22,7 +23,6 @@ export function getServiceUrl() {
|
||||
return DEV_API_SERVICE
|
||||
}
|
||||
|
||||
|
||||
/** request服务封装 */
|
||||
export default {
|
||||
getServiceUrl,
|
||||
|
||||
@@ -68,4 +68,21 @@ export default {
|
||||
})
|
||||
}).send()
|
||||
},
|
||||
// 手动添加设备
|
||||
manualAddDevice(params, callback) {
|
||||
RequestService.sendRequest()
|
||||
.url(`${getServiceUrl()}/device/manual-add`)
|
||||
.method('POST')
|
||||
.data(params)
|
||||
.success((res) => {
|
||||
RequestService.clearRequestTime();
|
||||
callback(res);
|
||||
})
|
||||
.networkFail((err) => {
|
||||
console.error('手动添加设备失败:', err);
|
||||
RequestService.reAjaxFun(() => {
|
||||
this.manualAddDevice(params, callback);
|
||||
});
|
||||
}).send();
|
||||
},
|
||||
}
|
||||
@@ -34,6 +34,8 @@ export default {
|
||||
languages: params.languageType,
|
||||
name: params.voiceName,
|
||||
remark: params.remark,
|
||||
referenceAudio: params.referenceAudio,
|
||||
referenceText: params.referenceText,
|
||||
sort: params.sort,
|
||||
ttsModelId: params.ttsModelId,
|
||||
ttsVoice: params.voiceCode,
|
||||
@@ -75,6 +77,8 @@ export default {
|
||||
languages: params.languageType,
|
||||
name: params.voiceName,
|
||||
remark: params.remark,
|
||||
referenceAudio: params.referenceAudio,
|
||||
referenceText: params.referenceText,
|
||||
ttsModelId: params.ttsModelId,
|
||||
ttsVoice: params.voiceCode,
|
||||
voiceDemo: params.voiceDemo || ''
|
||||
|
||||
@@ -110,8 +110,10 @@
|
||||
<div class="url-header">
|
||||
<div class="address-desc">
|
||||
<span>以下是智能体的MCP接入点地址。</span>
|
||||
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-enable.md"
|
||||
target="_blank" class="doc-link">如何部署MCP接入点</a> |
|
||||
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md"
|
||||
target="_blank" class="doc-link">查看接入点使用文档</a>
|
||||
target="_blank" class="doc-link">如何接入MCP功能</a>
|
||||
</div>
|
||||
</div>
|
||||
<el-input v-model="mcpUrl" readonly class="url-input">
|
||||
@@ -660,6 +662,7 @@ export default {
|
||||
.mcp-left,
|
||||
.mcp-right {
|
||||
flex: 1;
|
||||
padding-bottom: 50px;
|
||||
}
|
||||
|
||||
.url-header {
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
<template>
|
||||
<el-dialog title="手动添加设备" :visible="visible" @close="handleClose" width="30%" center>
|
||||
<div class="dialog-content">
|
||||
<el-form :model="deviceForm" :rules="rules" ref="deviceForm" label-width="100px">
|
||||
<el-form-item label="设备型号" prop="board">
|
||||
<el-select v-model="deviceForm.board" placeholder="请选择设备型号" style="width: 100%">
|
||||
<el-option
|
||||
v-for="item in firmwareTypes"
|
||||
:key="item.key"
|
||||
:label="item.name"
|
||||
:value="item.key">
|
||||
</el-option>
|
||||
</el-select>
|
||||
</el-form-item>
|
||||
<el-form-item label="固件版本" prop="appVersion">
|
||||
<el-input v-model="deviceForm.appVersion" placeholder="请输入固件版本"></el-input>
|
||||
</el-form-item>
|
||||
<el-form-item label="Mac地址" prop="macAddress">
|
||||
<el-input v-model="deviceForm.macAddress" placeholder="请输入Mac地址"></el-input>
|
||||
</el-form-item>
|
||||
</el-form>
|
||||
</div>
|
||||
<div style="display: flex;margin: 15px 15px;gap: 7px;">
|
||||
<div class="dialog-btn" @click="submitForm">确定</div>
|
||||
<div class="dialog-btn cancel-btn" @click="cancel">取消</div>
|
||||
</div>
|
||||
</el-dialog>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
import Api from '@/apis/api';
|
||||
|
||||
export default {
|
||||
name: 'ManualAddDeviceDialog',
|
||||
props: {
|
||||
visible: { type: Boolean, required: true },
|
||||
agentId: { type: String, required: true }
|
||||
},
|
||||
data() {
|
||||
// MAC地址验证规则
|
||||
const validateMac = (rule, value, callback) => {
|
||||
const macRegex = /^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$/;
|
||||
if (!value) {
|
||||
callback(new Error('请输入Mac地址'));
|
||||
} else if (!macRegex.test(value)) {
|
||||
callback(new Error('请输入正确的Mac地址格式,例如:00:1A:2B:3C:4D:5E'));
|
||||
} else {
|
||||
callback();
|
||||
}
|
||||
};
|
||||
|
||||
return {
|
||||
deviceForm: {
|
||||
board: '',
|
||||
appVersion: '',
|
||||
macAddress: ''
|
||||
},
|
||||
firmwareTypes: [],
|
||||
rules: {
|
||||
board: [
|
||||
{ required: true, message: '请选择设备型号', trigger: 'change' }
|
||||
],
|
||||
appVersion: [
|
||||
{ required: true, message: '请输入固件版本', trigger: 'blur' }
|
||||
],
|
||||
macAddress: [
|
||||
{ required: true, validator: validateMac, trigger: 'blur' }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
created() {
|
||||
this.getFirmwareTypes();
|
||||
},
|
||||
methods: {
|
||||
async getFirmwareTypes() {
|
||||
try {
|
||||
const res = await Api.dict.getDictDataByType('FIRMWARE_TYPE');
|
||||
this.firmwareTypes = res.data;
|
||||
} catch (error) {
|
||||
console.error('获取固件类型失败:', error);
|
||||
this.$message.error(error.message || '获取固件类型失败');
|
||||
}
|
||||
},
|
||||
submitForm() {
|
||||
this.$refs.deviceForm.validate((valid) => {
|
||||
if (valid) {
|
||||
this.addDevice();
|
||||
}
|
||||
});
|
||||
},
|
||||
addDevice() {
|
||||
const params = {
|
||||
agentId: this.agentId,
|
||||
...this.deviceForm
|
||||
};
|
||||
|
||||
Api.device.manualAddDevice(params, ({ data }) => {
|
||||
if (data.code === 0) {
|
||||
this.$message.success('设备添加成功');
|
||||
this.$emit('refresh');
|
||||
this.closeDialog();
|
||||
} else {
|
||||
this.$message.error(data.msg || '添加失败');
|
||||
}
|
||||
});
|
||||
},
|
||||
closeDialog() {
|
||||
this.$emit('update:visible', false);
|
||||
this.$refs.deviceForm.resetFields();
|
||||
},
|
||||
cancel() {
|
||||
this.closeDialog();
|
||||
},
|
||||
handleClose() {
|
||||
this.closeDialog();
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.dialog-content {
|
||||
padding: 0 20px;
|
||||
}
|
||||
|
||||
.dialog-btn {
|
||||
cursor: pointer;
|
||||
flex: 1;
|
||||
border-radius: 23px;
|
||||
background: #5778ff;
|
||||
height: 40px;
|
||||
font-weight: 500;
|
||||
font-size: 12px;
|
||||
color: #fff;
|
||||
line-height: 40px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.cancel-btn {
|
||||
background: #e6ebff;
|
||||
border: 1px solid #adbdff;
|
||||
color: #5778ff;
|
||||
}
|
||||
|
||||
::v-deep .el-dialog {
|
||||
border-radius: 15px;
|
||||
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
|
||||
}
|
||||
|
||||
::v-deep .el-dialog__body {
|
||||
padding: 20px 6px;
|
||||
}
|
||||
|
||||
::v-deep .el-form-item {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
||||
@@ -1,79 +1,66 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
:visible.sync="localVisible"
|
||||
width="85%"
|
||||
@close="handleClose"
|
||||
:show-close="false"
|
||||
:append-to-body="true"
|
||||
:close-on-click-modal="true">
|
||||
<el-dialog :visible.sync="localVisible" width="90%" @close="handleClose" :show-close="false" :append-to-body="true"
|
||||
:close-on-click-modal="true">
|
||||
<button class="custom-close-btn" @click="handleClose">
|
||||
×
|
||||
</button>
|
||||
<div class="scroll-wrapper">
|
||||
<div
|
||||
class="table-container"
|
||||
ref="tableContainer"
|
||||
@scroll="handleScroll">
|
||||
<el-table
|
||||
v-loading="loading"
|
||||
:data="filteredTtsModels"
|
||||
style="width: 100%;"
|
||||
class="data-table"
|
||||
header-row-class-name="table-header"
|
||||
:fit="true"
|
||||
element-loading-text="拼命加载中"
|
||||
element-loading-spinner="el-icon-loading"
|
||||
element-loading-background="rgba(0, 0, 0, 0.8)">
|
||||
<div class="table-container" ref="tableContainer" @scroll="handleScroll">
|
||||
<el-table v-loading="loading" :data="filteredTtsModels" style="width: 100%;" class="data-table"
|
||||
header-row-class-name="table-header" :fit="true" element-loading-text="拼命加载中"
|
||||
element-loading-spinner="el-icon-loading" element-loading-background="rgba(0, 0, 0, 0.8)">
|
||||
<el-table-column label="选择" width="50" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-checkbox v-model="scope.row.selected"></el-checkbox>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="音色编码" align="center" min-width="50">
|
||||
<el-table-column label="音色编码" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.voiceCode"></el-input>
|
||||
<span v-else>{{ scope.row.voiceCode }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="音色名称" align="center" min-width="50">
|
||||
<el-table-column label="音色名称" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.voiceName"></el-input>
|
||||
<span v-else>{{ scope.row.voiceName }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="语言类型" align="center" min-width="50">
|
||||
<el-table-column label="语言类型" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.languageType"></el-input>
|
||||
<span v-else>{{ scope.row.languageType }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="试听" align="center" min-width="100px" class-name="audio-column">
|
||||
<el-table-column v-if="!showReferenceColumns" label="试听" align="center" class-name="audio-column">
|
||||
<template slot-scope="scope">
|
||||
<div class="custom-audio-container">
|
||||
<el-input
|
||||
v-if="scope.row.editing"
|
||||
v-model="scope.row.voiceDemo"
|
||||
placeholder="请输入MP3地址"
|
||||
size="mini"
|
||||
class="audio-input">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.voiceDemo" placeholder="请输入MP3地址"
|
||||
class="audio-input">
|
||||
</el-input>
|
||||
<AudioPlayer v-else-if="isValidAudioUrl(scope.row.voiceDemo)" :audioUrl="scope.row.voiceDemo"/>
|
||||
<AudioPlayer v-else-if="isValidAudioUrl(scope.row.voiceDemo)" :audioUrl="scope.row.voiceDemo" />
|
||||
</div>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="备注" align="center">
|
||||
<el-table-column v-if="!showReferenceColumns" label="备注" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input
|
||||
v-if="scope.row.editing"
|
||||
type="textarea"
|
||||
:rows="1"
|
||||
autosize
|
||||
v-model="scope.row.remark"
|
||||
placeholder="这里是备注"
|
||||
class="remark-input"></el-input>
|
||||
<el-input v-if="scope.row.editing" type="textarea" :rows="1" autosize v-model="scope.row.remark"
|
||||
placeholder="这里是备注" class="remark-input"></el-input>
|
||||
<span v-else>{{ scope.row.remark }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column v-if="showReferenceColumns" label="克隆音频路径" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.referenceAudio" placeholder="这里是克隆音频路径"></el-input>
|
||||
<span v-else>{{ scope.row.referenceAudio }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column v-if="showReferenceColumns" label="克隆音频文本" align="center">
|
||||
<template slot-scope="scope">
|
||||
<el-input v-if="scope.row.editing" v-model="scope.row.referenceText" placeholder="这里是克隆音频对应文本"></el-input>
|
||||
<span v-else>{{ scope.row.referenceText }}</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
<el-table-column label="操作" align="center" width="150">
|
||||
<template slot-scope="scope">
|
||||
<template v-if="!scope.row.editing">
|
||||
@@ -84,12 +71,7 @@
|
||||
删除
|
||||
</el-button>
|
||||
</template>
|
||||
<el-button
|
||||
v-else
|
||||
type="success"
|
||||
size="mini"
|
||||
@click="saveEdit(scope.row)"
|
||||
class="save-Tts">保存
|
||||
<el-button v-else type="success" size="mini" @click="saveEdit(scope.row)" class="save-Tts">保存
|
||||
</el-button>
|
||||
</template>
|
||||
</el-table-column>
|
||||
@@ -110,21 +92,19 @@
|
||||
<el-button type="primary" size="mini" @click="addNew" style="background: #5bc98c;border: None;">
|
||||
新增
|
||||
</el-button>
|
||||
<el-button type="primary"
|
||||
size="mini"
|
||||
@click="deleteRow(filteredTtsModels.filter(row => row.selected))"
|
||||
style="background: red;border:None">删除
|
||||
<el-button type="primary" size="mini" @click="deleteRow(filteredTtsModels.filter(row => row.selected))"
|
||||
style="background: red;border:None">删除
|
||||
</el-button>
|
||||
</div>
|
||||
</el-dialog>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
import AudioPlayer from './AudioPlayer.vue'
|
||||
import Api from "@/apis/api";
|
||||
import AudioPlayer from './AudioPlayer.vue';
|
||||
|
||||
export default {
|
||||
components: {AudioPlayer},
|
||||
components: { AudioPlayer },
|
||||
props: {
|
||||
visible: {
|
||||
type: Boolean,
|
||||
@@ -133,6 +113,10 @@ export default {
|
||||
ttsModelId: {
|
||||
type: String,
|
||||
required: true
|
||||
},
|
||||
modelConfig: {
|
||||
type: Object,
|
||||
default: null
|
||||
}
|
||||
},
|
||||
data() {
|
||||
@@ -151,6 +135,7 @@ export default {
|
||||
selectAll: false,
|
||||
selectedRows: [],
|
||||
loading: false,
|
||||
showReferenceColumns: false, // 控制是否显示参考列
|
||||
};
|
||||
},
|
||||
watch: {
|
||||
@@ -158,12 +143,19 @@ export default {
|
||||
this.localVisible = newVal;
|
||||
if (newVal) {
|
||||
this.currentPage = 1;
|
||||
this.updateShowReferenceColumns(); // 更新显示状态
|
||||
this.loadData(); // 对话框显示时加载数据
|
||||
this.$nextTick(() => {
|
||||
this.updateScrollbar();
|
||||
});
|
||||
}
|
||||
},
|
||||
modelConfig: {
|
||||
handler(newVal) {
|
||||
this.updateShowReferenceColumns();
|
||||
},
|
||||
immediate: true
|
||||
},
|
||||
filteredTtsModels() {
|
||||
this.$nextTick(() => {
|
||||
this.updateScrollbar();
|
||||
@@ -173,7 +165,7 @@ export default {
|
||||
computed: {
|
||||
filteredTtsModels() {
|
||||
return this.ttsModels.filter(model =>
|
||||
model.voiceName.toLowerCase().includes(this.searchQuery.toLowerCase())
|
||||
model.voiceName.toLowerCase().includes(this.searchQuery.toLowerCase())
|
||||
);
|
||||
}
|
||||
},
|
||||
@@ -189,6 +181,16 @@ export default {
|
||||
window.removeEventListener('mousemove', this.handleDrag);
|
||||
},
|
||||
methods: {
|
||||
// 更新是否显示参考列
|
||||
updateShowReferenceColumns() {
|
||||
if (this.modelConfig && this.modelConfig.configJson) {
|
||||
const providerType = this.modelConfig.configJson.type;
|
||||
this.showReferenceColumns = ['fishspeech', 'gpt_sovits_v2', 'gpt_sovits_v3'].includes(providerType);
|
||||
} else {
|
||||
this.showReferenceColumns = false;
|
||||
}
|
||||
},
|
||||
|
||||
loadData() {
|
||||
this.loading = true;
|
||||
const params = {
|
||||
@@ -206,6 +208,8 @@ export default {
|
||||
voiceName: item.name || '未命名音色',
|
||||
languageType: item.languages || '',
|
||||
remark: item.remark || '',
|
||||
referenceAudio: item.referenceAudio || '',
|
||||
referenceText: item.referenceText || '',
|
||||
voiceDemo: item.voiceDemo || '',
|
||||
selected: false,
|
||||
editing: false,
|
||||
@@ -237,6 +241,7 @@ export default {
|
||||
this.total = 0;
|
||||
this.selectAll = false;
|
||||
this.searchQuery = '';
|
||||
this.showReferenceColumns = false;
|
||||
|
||||
this.localVisible = false;
|
||||
this.$emit('update:visible', false);
|
||||
@@ -249,7 +254,7 @@ export default {
|
||||
|
||||
if (!container || !scrollbarThumb || !scrollbarTrack) return;
|
||||
|
||||
const {scrollHeight, clientHeight} = container;
|
||||
const { scrollHeight, clientHeight } = container;
|
||||
const trackHeight = scrollbarTrack.clientHeight;
|
||||
const thumbHeight = Math.max((clientHeight / scrollHeight) * trackHeight, 20);
|
||||
|
||||
@@ -264,7 +269,7 @@ export default {
|
||||
|
||||
if (!container || !scrollbarThumb || !scrollbarTrack) return;
|
||||
|
||||
const {scrollHeight, clientHeight, scrollTop} = container;
|
||||
const { scrollHeight, clientHeight, scrollTop } = container;
|
||||
const trackHeight = scrollbarTrack.clientHeight;
|
||||
const thumbHeight = scrollbarThumb.clientHeight;
|
||||
const maxTop = trackHeight - thumbHeight;
|
||||
@@ -332,7 +337,7 @@ export default {
|
||||
|
||||
startEdit(row) {
|
||||
row.editing = true;
|
||||
this.$set(row, 'originalData', {...row});
|
||||
this.$set(row, 'originalData', { ...row });
|
||||
},
|
||||
|
||||
saveEdit(row) {
|
||||
@@ -356,6 +361,12 @@ export default {
|
||||
sort: row.sort
|
||||
};
|
||||
|
||||
// 只有在显示参考列的情况下才添加参考字段
|
||||
if (this.showReferenceColumns) {
|
||||
params.referenceAudio = row.referenceAudio;
|
||||
params.referenceText = row.referenceText;
|
||||
}
|
||||
|
||||
let res;
|
||||
if (row.id) {
|
||||
// 已有ID,执行更新操作
|
||||
@@ -423,8 +434,8 @@ export default {
|
||||
}
|
||||
|
||||
const maxSort = this.ttsModels.length > 0
|
||||
? Math.max(...this.ttsModels.map(item => Number(item.sort) || 0))
|
||||
: 0;
|
||||
? Math.max(...this.ttsModels.map(item => Number(item.sort) || 0))
|
||||
: 0;
|
||||
|
||||
const newRow = {
|
||||
voiceCode: '',
|
||||
@@ -432,6 +443,8 @@ export default {
|
||||
languageType: '中文',
|
||||
voiceDemo: '',
|
||||
remark: '',
|
||||
referenceAudio: '',
|
||||
referenceText: '',
|
||||
selected: false,
|
||||
editing: true,
|
||||
sort: maxSort + 1
|
||||
@@ -441,56 +454,56 @@ export default {
|
||||
},
|
||||
|
||||
deleteRow(row) {
|
||||
// 处理单个音色或音色数组
|
||||
const voices = Array.isArray(row) ? row : [row];
|
||||
// 处理单个音色或音色数组
|
||||
const voices = Array.isArray(row) ? row : [row];
|
||||
|
||||
if (Array.isArray(row) && row.length === 0) {
|
||||
this.$message.warning("请先选择需要删除的音色");
|
||||
return;
|
||||
if (Array.isArray(row) && row.length === 0) {
|
||||
this.$message.warning("请先选择需要删除的音色");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const voiceCount = voices.length;
|
||||
this.$confirm(`确定要删除选中的${voiceCount}个音色吗?`, "警告", {
|
||||
confirmButtonText: "确定",
|
||||
cancelButtonText: "取消",
|
||||
type: "warning",
|
||||
distinguishCancelAndClose: true
|
||||
}).then(() => {
|
||||
const ids = voices.map(voice => voice.id);
|
||||
if (ids.some(id => !id)) {
|
||||
this.$message.error("存在无效的音色ID");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const voiceCount = voices.length;
|
||||
this.$confirm(`确定要删除选中的${voiceCount}个音色吗?`, "警告", {
|
||||
confirmButtonText: "确定",
|
||||
cancelButtonText: "取消",
|
||||
type: "warning",
|
||||
distinguishCancelAndClose: true
|
||||
}).then(() => {
|
||||
const ids = voices.map(voice => voice.id);
|
||||
if (ids.some(id => !id)) {
|
||||
this.$message.error("存在无效的音色ID");
|
||||
return;
|
||||
}
|
||||
|
||||
Api.timbre.deleteVoice(ids, ({data}) => {
|
||||
if (data.code === 0) {
|
||||
this.$message.success({
|
||||
message: `成功删除${voiceCount}个参数`,
|
||||
showClose: true
|
||||
});
|
||||
this.loadData(); // 刷新参数列表
|
||||
} else {
|
||||
this.$message.error({
|
||||
message: data.msg || '删除失败,请重试',
|
||||
showClose: true
|
||||
});
|
||||
}
|
||||
Api.timbre.deleteVoice(ids, ({ data }) => {
|
||||
if (data.code === 0) {
|
||||
this.$message.success({
|
||||
message: `成功删除${voiceCount}个参数`,
|
||||
showClose: true
|
||||
});
|
||||
this.loadData(); // 刷新参数列表
|
||||
} else {
|
||||
this.$message.error({
|
||||
message: data.msg || '删除失败,请重试',
|
||||
showClose: true
|
||||
});
|
||||
}
|
||||
});
|
||||
}).catch(action => {
|
||||
if (action === 'cancel') {
|
||||
this.$message({
|
||||
type: 'info',
|
||||
message: '已取消删除操作',
|
||||
duration: 1000
|
||||
});
|
||||
} else {
|
||||
this.$message({
|
||||
type: 'info',
|
||||
message: '操作已关闭',
|
||||
duration: 1000
|
||||
});
|
||||
}
|
||||
if (action === 'cancel') {
|
||||
this.$message({
|
||||
type: 'info',
|
||||
message: '已取消删除操作',
|
||||
duration: 1000
|
||||
});
|
||||
} else {
|
||||
this.$message({
|
||||
type: 'info',
|
||||
message: '操作已关闭',
|
||||
duration: 1000
|
||||
});
|
||||
}
|
||||
});
|
||||
},
|
||||
|
||||
@@ -502,7 +515,6 @@ export default {
|
||||
</script>
|
||||
|
||||
<style lang="scss" scoped>
|
||||
|
||||
::v-deep .el-dialog {
|
||||
border-radius: 8px !important;
|
||||
overflow: hidden;
|
||||
@@ -695,6 +707,7 @@ export default {
|
||||
white-space: pre-wrap !important;
|
||||
word-break: break-all !important;
|
||||
}
|
||||
|
||||
/* 按钮组定位调整 */
|
||||
.action-buttons {
|
||||
position: static;
|
||||
@@ -719,5 +732,4 @@ export default {
|
||||
flex-shrink: 0;
|
||||
margin: 2px !important;
|
||||
}
|
||||
|
||||
</style>
|
||||
@@ -77,7 +77,10 @@
|
||||
{{ isAllSelected ? '取消全选' : '全选' }}
|
||||
</el-button>
|
||||
<el-button type="success" size="mini" class="add-device-btn" @click="handleAddDevice">
|
||||
新增
|
||||
验证码绑定
|
||||
</el-button>
|
||||
<el-button type="success" size="mini" class="add-device-btn" @click="handleManualAddDevice">
|
||||
手动添加
|
||||
</el-button>
|
||||
<el-button size="mini" type="danger" icon="el-icon-delete" @click="deleteSelected">解绑</el-button>
|
||||
</div>
|
||||
@@ -103,6 +106,8 @@
|
||||
|
||||
<AddDeviceDialog :visible.sync="addDeviceDialogVisible" :agent-id="currentAgentId"
|
||||
@refresh="fetchBindDevices(currentAgentId)"/>
|
||||
<ManualAddDeviceDialog :visible.sync="manualAddDeviceDialogVisible" :agent-id="currentAgentId"
|
||||
@refresh="fetchBindDevices(currentAgentId)"/>
|
||||
|
||||
</div>
|
||||
</template>
|
||||
@@ -110,13 +115,19 @@
|
||||
<script>
|
||||
import Api from '@/apis/api';
|
||||
import AddDeviceDialog from "@/components/AddDeviceDialog.vue";
|
||||
import ManualAddDeviceDialog from "@/components/ManualAddDeviceDialog.vue";
|
||||
import HeaderBar from "@/components/HeaderBar.vue";
|
||||
|
||||
export default {
|
||||
components: {HeaderBar, AddDeviceDialog},
|
||||
components: {
|
||||
HeaderBar,
|
||||
AddDeviceDialog,
|
||||
ManualAddDeviceDialog
|
||||
},
|
||||
data() {
|
||||
return {
|
||||
addDeviceDialogVisible: false,
|
||||
manualAddDeviceDialogVisible: false,
|
||||
selectedDevices: [],
|
||||
isAllSelected: false,
|
||||
searchKeyword: "",
|
||||
@@ -252,6 +263,9 @@ export default {
|
||||
handleAddDevice() {
|
||||
this.addDeviceDialogVisible = true;
|
||||
},
|
||||
handleManualAddDevice() {
|
||||
this.manualAddDeviceDialogVisible = true;
|
||||
},
|
||||
submitRemark(row) {
|
||||
if (row._submitting) return;
|
||||
|
||||
|
||||
@@ -128,7 +128,7 @@
|
||||
|
||||
<ModelEditDialog :modelType="activeTab" :visible.sync="editDialogVisible" :modelData="editModelData"
|
||||
@save="handleModelSave" />
|
||||
<TtsModel :visible.sync="ttsDialogVisible" :ttsModelId="selectedTtsModelId" />
|
||||
<TtsModel :visible.sync="ttsDialogVisible" :ttsModelId="selectedTtsModelId" :modelConfig="selectedModelConfig" />
|
||||
<AddModelDialog :modelType="activeTab" :visible.sync="addDialogVisible" @confirm="handleAddConfirm" />
|
||||
</div>
|
||||
<el-footer>
|
||||
@@ -162,7 +162,8 @@ export default {
|
||||
total: 0,
|
||||
selectedModels: [],
|
||||
isAllSelected: false,
|
||||
loading: false
|
||||
loading: false,
|
||||
selectedModelConfig: {}
|
||||
};
|
||||
},
|
||||
|
||||
@@ -211,6 +212,7 @@ export default {
|
||||
},
|
||||
openTtsDialog(row) {
|
||||
this.selectedTtsModelId = row.id;
|
||||
this.selectedModelConfig = row;
|
||||
this.ttsDialogVisible = true;
|
||||
},
|
||||
headerCellClassName({ column, columnIndex }) {
|
||||
|
||||
@@ -43,7 +43,7 @@
|
||||
|
||||
<div class="input-box">
|
||||
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
|
||||
<el-input v-model="form.password" placeholder="请输入密码" type="password" />
|
||||
<el-input v-model="form.password" placeholder="请输入密码" type="password" show-password />
|
||||
</div>
|
||||
<div style="display: flex; align-items: center; margin-top: 20px; width: 100%; gap: 10px;">
|
||||
<div class="input-box" style="width: calc(100% - 130px); margin-top: 0;">
|
||||
|
||||
@@ -70,13 +70,13 @@
|
||||
<!-- 密码输入框 -->
|
||||
<div class="input-box">
|
||||
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
|
||||
<el-input v-model="form.password" placeholder="请输入密码" type="password" />
|
||||
<el-input v-model="form.password" placeholder="请输入密码" type="password" show-password />
|
||||
</div>
|
||||
|
||||
<!-- 新增确认密码 -->
|
||||
<div class="input-box">
|
||||
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
|
||||
<el-input v-model="form.confirmPassword" placeholder="请确认密码" type="password" />
|
||||
<el-input v-model="form.confirmPassword" placeholder="请确认密码" type="password" show-password />
|
||||
</div>
|
||||
|
||||
<!-- 验证码部分保持相同 -->
|
||||
|
||||
@@ -61,13 +61,13 @@
|
||||
<!-- 新密码 -->
|
||||
<div class="input-box">
|
||||
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
|
||||
<el-input v-model="form.newPassword" placeholder="请输入新密码" type="password" />
|
||||
<el-input v-model="form.newPassword" placeholder="请输入新密码" type="password" show-password />
|
||||
</div>
|
||||
|
||||
<!-- 确认新密码 -->
|
||||
<div class="input-box">
|
||||
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
|
||||
<el-input v-model="form.confirmPassword" placeholder="请确认新密码" type="password" />
|
||||
<el-input v-model="form.confirmPassword" placeholder="请确认新密码" type="password" show-password />
|
||||
</div>
|
||||
|
||||
<!-- 修改底部链接 -->
|
||||
|
||||
@@ -5,7 +5,7 @@ import asyncio
|
||||
from aioconsole import ainput
|
||||
from config.settings import load_config
|
||||
from config.logger import setup_logging
|
||||
from core.utils.util import get_local_ip
|
||||
from core.utils.util import get_local_ip, validate_mcp_endpoint
|
||||
from core.http_server import SimpleHttpServer
|
||||
from core.websocket_server import WebSocketServer
|
||||
from core.utils.util import check_ffmpeg_installed
|
||||
@@ -77,6 +77,17 @@ async def main():
|
||||
get_local_ip(),
|
||||
port,
|
||||
)
|
||||
mcp_endpoint = config.get("mcp_endpoint", None)
|
||||
if mcp_endpoint is not None and "你" not in mcp_endpoint:
|
||||
# 校验MCP接入点格式
|
||||
if validate_mcp_endpoint(mcp_endpoint):
|
||||
logger.bind(tag=TAG).info("mcp接入点是\t{}", mcp_endpoint)
|
||||
# 将mcp计入点地址转成调用点
|
||||
mcp_endpoint = mcp_endpoint.replace("/mcp/", "/call/")
|
||||
config["mcp_endpoint"] = mcp_endpoint
|
||||
else:
|
||||
logger.bind(tag=TAG).error("mcp接入点不符合规范")
|
||||
config["mcp_endpoint"] = "你的接入点 websocket地址"
|
||||
|
||||
# 获取WebSocket配置,使用安全的默认值
|
||||
websocket_port = 8000
|
||||
|
||||
@@ -104,7 +104,8 @@ wakeup_words:
|
||||
- "喵喵同学"
|
||||
- "小滨小滨"
|
||||
- "小冰小冰"
|
||||
# MCP接入点地址
|
||||
# MCP接入点地址,地址格式为:ws://你的mcp接入点ip或者域名:端口号/mcp/?token=你的token
|
||||
# 详细教程 https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md
|
||||
mcp_endpoint: 你的接入点 websocket地址
|
||||
# 插件的基础配置
|
||||
plugins:
|
||||
@@ -121,7 +122,9 @@ plugins:
|
||||
society_rss_url: "https://www.chinanews.com.cn/rss/society.xml"
|
||||
world_rss_url: "https://www.chinanews.com.cn/rss/world.xml"
|
||||
finance_rss_url: "https://www.chinanews.com.cn/rss/finance.xml"
|
||||
get_news_from_newsnow: {"url": "https://newsnow.busiyi.world/api/s?id="}
|
||||
get_news_from_newsnow:
|
||||
url: "https://newsnow.busiyi.world/api/s?id="
|
||||
news_sources: "澎湃新闻;百度热搜;财联社"
|
||||
home_assistant:
|
||||
devices:
|
||||
- 客厅,玩具灯,switch.cuco_cn_460494544_cp1_on_p_2_1
|
||||
@@ -159,7 +162,7 @@ end_prompt:
|
||||
enable: true # 是否开启结束语
|
||||
# 结束语
|
||||
prompt: |
|
||||
请你以“时间过得真快”未来头,用富有感情、依依不舍的话来结束这场对话吧!
|
||||
请你以"时间过得真快"未来头,用富有感情、依依不舍的话来结束这场对话吧!
|
||||
|
||||
# 具体处理时选择的模块(The module selected for specific processing)
|
||||
selected_module:
|
||||
@@ -196,7 +199,7 @@ Intent:
|
||||
# 如果你的不想使用selected_module.LLM意图识别,这里最好使用独立的LLM作为意图识别,例如使用免费的ChatGLMLLM
|
||||
llm: ChatGLMLLM
|
||||
# plugins_func/functions下的模块,可以通过配置,选择加载哪个模块,加载后对话支持相应的function调用
|
||||
# 系统默认已经记载“handle_exit_intent(退出识别)”、“play_music(音乐播放)”插件,请勿重复加载
|
||||
# 系统默认已经记载"handle_exit_intent(退出识别)"、"play_music(音乐播放)"插件,请勿重复加载
|
||||
# 下面是加载查天气、角色切换、加载查新闻的插件示例
|
||||
functions:
|
||||
- get_weather
|
||||
@@ -206,7 +209,7 @@ Intent:
|
||||
# 不需要动type
|
||||
type: function_call
|
||||
# plugins_func/functions下的模块,可以通过配置,选择加载哪个模块,加载后对话支持相应的function调用
|
||||
# 系统默认已经记载“handle_exit_intent(退出识别)”、“play_music(音乐播放)”插件,请勿重复加载
|
||||
# 系统默认已经记载"handle_exit_intent(退出识别)"、"play_music(音乐播放)"插件,请勿重复加载
|
||||
# 下面是加载查天气、角色切换、加载查新闻的插件示例
|
||||
functions:
|
||||
- change_role
|
||||
@@ -347,6 +350,7 @@ VAD:
|
||||
SileroVAD:
|
||||
type: silero
|
||||
threshold: 0.5
|
||||
threshold_low: 0.3
|
||||
model_dir: models/snakers4_silero-vad
|
||||
min_silence_duration_ms: 200 # 如果说话停顿比较长,可以把这个值设置大一些
|
||||
|
||||
@@ -812,4 +816,4 @@ TTS:
|
||||
# 支持声音克隆,可自行上传音频,填入voice参数,voice参数为空时,使用默认声音
|
||||
access_token: "U4YdYXVfpwWnk2t5Gp822zWPCuORyeJL"
|
||||
voice: "OUeAo1mhq6IBExi"
|
||||
output_dir: tmp/
|
||||
output_dir: tmp/
|
||||
|
||||
@@ -5,7 +5,7 @@ from config.config_loader import load_config
|
||||
from config.settings import check_config_file
|
||||
from datetime import datetime
|
||||
|
||||
SERVER_VERSION = "0.6.1"
|
||||
SERVER_VERSION = "0.6.3"
|
||||
_logger_initialized = False
|
||||
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ server:
|
||||
vision_explain: http://你的ip或者域名:端口号/mcp/vision/explain
|
||||
manager-api:
|
||||
# 你的manager-api的地址,最好使用局域网ip
|
||||
# 如果使用docker部署,请使用填写成 http://xiaozhi-esp32-server-web:8002/xiaozhi
|
||||
url: http://127.0.0.1:8002/xiaozhi
|
||||
# 你的manager-api的token,就是刚才复制出来的server.secret
|
||||
secret: 你的server.secret值
|
||||
@@ -17,6 +17,7 @@ from core.utils.util import (
|
||||
filter_sensitive_info,
|
||||
)
|
||||
from typing import Dict, Any
|
||||
from collections import deque
|
||||
from core.utils.modules_initialize import (
|
||||
initialize_modules,
|
||||
initialize_tts,
|
||||
@@ -110,10 +111,10 @@ class ConnectionHandler:
|
||||
# vad相关变量
|
||||
self.client_audio_buffer = bytearray()
|
||||
self.client_have_voice = False
|
||||
self.client_have_voice_last_time = 0.0
|
||||
self.client_no_voice_last_time = 0.0
|
||||
self.last_activity_time = 0.0 # 统一的活动时间戳(毫秒)
|
||||
self.client_voice_stop = False
|
||||
self.client_voice_frame_count = 0
|
||||
self.client_voice_window = deque(maxlen=5)
|
||||
self.last_is_voice = False
|
||||
|
||||
# asr相关变量
|
||||
# 因为实际部署时可能会用到公共的本地ASR,不能把变量暴露给公共ASR
|
||||
@@ -143,10 +144,10 @@ class ConnectionHandler:
|
||||
self.load_function_plugin = False
|
||||
self.intent_type = "nointent"
|
||||
|
||||
self.timeout_task = None
|
||||
self.timeout_seconds = (
|
||||
int(self.config.get("close_connection_no_voice_time", 120)) + 60
|
||||
) # 在原来第一道关闭的基础上加60秒,进行二道关闭
|
||||
self.timeout_task = None
|
||||
|
||||
# {"mcp":true} 表示启用MCP功能
|
||||
self.features = None
|
||||
@@ -187,12 +188,14 @@ class ConnectionHandler:
|
||||
self.websocket = ws
|
||||
self.device_id = self.headers.get("device-id", None)
|
||||
|
||||
# 初始化活动时间戳
|
||||
self.last_activity_time = time.time() * 1000
|
||||
|
||||
# 启动超时检查任务
|
||||
self.timeout_task = asyncio.create_task(self._check_timeout())
|
||||
|
||||
self.welcome_msg = self.config["xiaozhi"]
|
||||
self.welcome_msg["session_id"] = self.session_id
|
||||
await self.websocket.send(json.dumps(self.welcome_msg))
|
||||
|
||||
# 获取差异化配置
|
||||
self._initialize_private_config()
|
||||
@@ -213,7 +216,17 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).error(f"Connection error: {str(e)}-{stack_trace}")
|
||||
return
|
||||
finally:
|
||||
await self._save_and_close(ws)
|
||||
try:
|
||||
await self._save_and_close(ws)
|
||||
except Exception as final_error:
|
||||
self.logger.bind(tag=TAG).error(f"最终清理时出错: {final_error}")
|
||||
# 确保即使保存记忆失败,也要关闭连接
|
||||
try:
|
||||
await self.close(ws)
|
||||
except Exception as close_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"强制关闭连接时出错: {close_error}"
|
||||
)
|
||||
|
||||
async def _save_and_close(self, ws):
|
||||
"""保存记忆并关闭连接"""
|
||||
@@ -231,7 +244,10 @@ class ConnectionHandler:
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"保存记忆失败: {e}")
|
||||
finally:
|
||||
loop.close()
|
||||
try:
|
||||
loop.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 启动线程保存记忆,不等待完成
|
||||
threading.Thread(target=save_memory_task, daemon=True).start()
|
||||
@@ -239,16 +255,17 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).error(f"保存记忆失败: {e}")
|
||||
finally:
|
||||
# 立即关闭连接,不等待记忆保存完成
|
||||
await self.close(ws)
|
||||
try:
|
||||
await self.close(ws)
|
||||
except Exception as close_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"保存记忆后关闭连接失败: {close_error}"
|
||||
)
|
||||
|
||||
async def _route_message(self, message):
|
||||
"""消息路由"""
|
||||
# 重置超时计时器
|
||||
if self.timeout_task:
|
||||
self.timeout_task.cancel()
|
||||
self.timeout_task = asyncio.create_task(self._check_timeout())
|
||||
|
||||
if isinstance(message, str):
|
||||
self.last_activity_time = time.time() * 1000
|
||||
await handleTextMessage(self, message)
|
||||
elif isinstance(message, bytes):
|
||||
if self.vad is None:
|
||||
@@ -594,13 +611,24 @@ class ConnectionHandler:
|
||||
# 更新系统prompt至上下文
|
||||
self.dialogue.update_system_message(self.prompt)
|
||||
|
||||
def chat(self, query, tool_call=False):
|
||||
def chat(self, query, tool_call=False, depth=0):
|
||||
self.logger.bind(tag=TAG).info(f"大模型收到用户消息: {query}")
|
||||
self.llm_finish_task = False
|
||||
|
||||
if not tool_call:
|
||||
self.dialogue.put(Message(role="user", content=query))
|
||||
|
||||
# 为最顶层时新建会话ID和发送FIRST请求
|
||||
if depth == 0:
|
||||
self.sentence_id = str(uuid.uuid4().hex)
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
|
||||
# Define intent functions
|
||||
functions = None
|
||||
if self.intent_type == "function_call" and hasattr(self, "func_handler"):
|
||||
@@ -616,8 +644,6 @@ class ConnectionHandler:
|
||||
)
|
||||
memory_str = future.result()
|
||||
|
||||
self.sentence_id = str(uuid.uuid4().hex)
|
||||
|
||||
if self.intent_type == "function_call" and functions is not None:
|
||||
# 使用支持functions的streaming接口
|
||||
llm_responses = self.llm.response_with_functions(
|
||||
@@ -640,7 +666,6 @@ class ConnectionHandler:
|
||||
function_id = None
|
||||
function_arguments = ""
|
||||
content_arguments = ""
|
||||
text_index = 0
|
||||
self.client_abort = False
|
||||
for response in llm_responses:
|
||||
if self.client_abort:
|
||||
@@ -670,14 +695,6 @@ class ConnectionHandler:
|
||||
if content is not None and len(content) > 0:
|
||||
if not tool_call_flag:
|
||||
response_message.append(content)
|
||||
if text_index == 0:
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
@@ -686,7 +703,6 @@ class ConnectionHandler:
|
||||
content_detail=content,
|
||||
)
|
||||
)
|
||||
text_index += 1
|
||||
# 处理function call
|
||||
if tool_call_flag:
|
||||
bHasError = False
|
||||
@@ -711,6 +727,11 @@ class ConnectionHandler:
|
||||
f"function call error: {content_arguments}"
|
||||
)
|
||||
if not bHasError:
|
||||
# 如需要大模型先处理一轮,添加相关处理后的日志情况
|
||||
if len(response_message) > 0:
|
||||
self.dialogue.put(
|
||||
Message(role="assistant", content="".join(response_message))
|
||||
)
|
||||
response_message.clear()
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}"
|
||||
@@ -728,14 +749,14 @@ class ConnectionHandler:
|
||||
),
|
||||
self.loop,
|
||||
).result()
|
||||
self._handle_function_result(result, function_call_data)
|
||||
self._handle_function_result(result, function_call_data, depth=depth)
|
||||
|
||||
# 存储对话内容
|
||||
if len(response_message) > 0:
|
||||
self.dialogue.put(
|
||||
Message(role="assistant", content="".join(response_message))
|
||||
)
|
||||
if text_index > 0:
|
||||
if depth == 0:
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
@@ -750,7 +771,7 @@ class ConnectionHandler:
|
||||
|
||||
return True
|
||||
|
||||
def _handle_function_result(self, result, function_call_data):
|
||||
def _handle_function_result(self, result, function_call_data, depth):
|
||||
if result.action == Action.RESPONSE: # 直接回复前端
|
||||
text = result.response
|
||||
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
|
||||
@@ -787,7 +808,7 @@ class ConnectionHandler:
|
||||
content=text,
|
||||
)
|
||||
)
|
||||
self.chat(text, tool_call=True)
|
||||
self.chat(text, tool_call=True, depth=depth + 1)
|
||||
elif result.action == Action.NOTFOUND or result.action == Action.ERROR:
|
||||
text = result.response if result.response else result.result
|
||||
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
|
||||
@@ -840,13 +861,22 @@ class ConnectionHandler:
|
||||
"""资源清理方法"""
|
||||
try:
|
||||
# 取消超时任务
|
||||
if self.timeout_task:
|
||||
if self.timeout_task and not self.timeout_task.done():
|
||||
self.timeout_task.cancel()
|
||||
try:
|
||||
await self.timeout_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self.timeout_task = None
|
||||
|
||||
# 清理工具处理器资源
|
||||
if hasattr(self, "func_handler") and self.func_handler:
|
||||
await self.func_handler.cleanup()
|
||||
try:
|
||||
await self.func_handler.cleanup()
|
||||
except Exception as cleanup_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"清理工具处理器时出错: {cleanup_error}"
|
||||
)
|
||||
|
||||
# 触发停止事件
|
||||
if self.stop_event:
|
||||
@@ -856,19 +886,61 @@ class ConnectionHandler:
|
||||
self.clear_queues()
|
||||
|
||||
# 关闭WebSocket连接
|
||||
if ws:
|
||||
await ws.close()
|
||||
elif self.websocket:
|
||||
await self.websocket.close()
|
||||
try:
|
||||
if ws:
|
||||
# 安全地检查WebSocket状态并关闭
|
||||
try:
|
||||
if hasattr(ws, "closed") and not ws.closed:
|
||||
await ws.close()
|
||||
elif hasattr(ws, "state") and ws.state.name != "CLOSED":
|
||||
await ws.close()
|
||||
else:
|
||||
# 如果没有closed属性,直接尝试关闭
|
||||
await ws.close()
|
||||
except Exception:
|
||||
# 如果关闭失败,忽略错误
|
||||
pass
|
||||
elif self.websocket:
|
||||
try:
|
||||
if (
|
||||
hasattr(self.websocket, "closed")
|
||||
and not self.websocket.closed
|
||||
):
|
||||
await self.websocket.close()
|
||||
elif (
|
||||
hasattr(self.websocket, "state")
|
||||
and self.websocket.state.name != "CLOSED"
|
||||
):
|
||||
await self.websocket.close()
|
||||
else:
|
||||
# 如果没有closed属性,直接尝试关闭
|
||||
await self.websocket.close()
|
||||
except Exception:
|
||||
# 如果关闭失败,忽略错误
|
||||
pass
|
||||
except Exception as ws_error:
|
||||
self.logger.bind(tag=TAG).error(f"关闭WebSocket连接时出错: {ws_error}")
|
||||
|
||||
if self.tts:
|
||||
await self.tts.close()
|
||||
|
||||
# 最后关闭线程池(避免阻塞)
|
||||
if self.executor:
|
||||
self.executor.shutdown(wait=False)
|
||||
try:
|
||||
self.executor.shutdown(wait=False)
|
||||
except Exception as executor_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"关闭线程池时出错: {executor_error}"
|
||||
)
|
||||
self.executor = None
|
||||
|
||||
self.logger.bind(tag=TAG).info("连接资源已释放")
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"关闭连接时出错: {e}")
|
||||
finally:
|
||||
# 确保停止事件被设置
|
||||
if self.stop_event:
|
||||
self.stop_event.set()
|
||||
|
||||
def clear_queues(self):
|
||||
"""清空所有任务队列"""
|
||||
@@ -898,7 +970,6 @@ class ConnectionHandler:
|
||||
def reset_vad_states(self):
|
||||
self.client_audio_buffer = bytearray()
|
||||
self.client_have_voice = False
|
||||
self.client_have_voice_last_time = 0
|
||||
self.client_voice_stop = False
|
||||
self.logger.bind(tag=TAG).debug("VAD states reset.")
|
||||
|
||||
@@ -917,10 +988,28 @@ class ConnectionHandler:
|
||||
"""检查连接超时"""
|
||||
try:
|
||||
while not self.stop_event.is_set():
|
||||
await asyncio.sleep(self.timeout_seconds)
|
||||
if not self.stop_event.is_set():
|
||||
self.logger.bind(tag=TAG).info("连接超时,准备关闭")
|
||||
await self.close(self.websocket)
|
||||
break
|
||||
# 检查是否超时(只有在时间戳已初始化的情况下)
|
||||
if self.last_activity_time > 0.0:
|
||||
current_time = time.time() * 1000
|
||||
if (
|
||||
current_time - self.last_activity_time
|
||||
> self.timeout_seconds * 1000
|
||||
):
|
||||
if not self.stop_event.is_set():
|
||||
self.logger.bind(tag=TAG).info("连接超时,准备关闭")
|
||||
# 设置停止事件,防止重复处理
|
||||
self.stop_event.set()
|
||||
# 使用 try-except 包装关闭操作,确保不会因为异常而阻塞
|
||||
try:
|
||||
await self.close(self.websocket)
|
||||
except Exception as close_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"超时关闭连接时出错: {close_error}"
|
||||
)
|
||||
break
|
||||
# 每10秒检查一次,避免过于频繁
|
||||
await asyncio.sleep(10)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"超时检查任务出错: {e}")
|
||||
finally:
|
||||
self.logger.bind(tag=TAG).info("超时检查任务已退出")
|
||||
|
||||
@@ -6,9 +6,8 @@ from core.handle.helloHandle import checkWakeupWords
|
||||
from core.utils.util import remove_punctuation_and_length
|
||||
from core.providers.tts.dto.dto import ContentType
|
||||
from core.utils.dialogue import Message
|
||||
from core.providers.tools.device_mcp import call_mcp_tool
|
||||
from plugins_func.register import Action, ActionResponse
|
||||
from loguru import logger
|
||||
from core.providers.tts.dto.dto import TTSMessageDTO, SentenceType
|
||||
|
||||
TAG = __name__
|
||||
|
||||
@@ -29,6 +28,8 @@ async def handle_user_intent(conn, text):
|
||||
intent_result = await analyze_intent_with_llm(conn, text)
|
||||
if not intent_result:
|
||||
return False
|
||||
# 会话开始时生成sentence_id
|
||||
conn.sentence_id = str(uuid.uuid4().hex)
|
||||
# 处理各种意图
|
||||
return await process_intent_result(conn, intent_result, text)
|
||||
|
||||
@@ -79,11 +80,6 @@ async def process_intent_result(conn, intent_result, original_text):
|
||||
if function_name == "continue_chat":
|
||||
return False
|
||||
|
||||
if function_name == "play_music":
|
||||
funcItem = conn.func_handler.get_function(function_name)
|
||||
if not funcItem:
|
||||
conn.func_handler.function_registry.register_function("play_music")
|
||||
|
||||
function_args = {}
|
||||
if "arguments" in intent_data["function_call"]:
|
||||
function_args = intent_data["function_call"]["arguments"]
|
||||
@@ -158,5 +154,19 @@ async def process_intent_result(conn, intent_result, original_text):
|
||||
|
||||
|
||||
def speak_txt(conn, text):
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
conn.tts.tts_one_sentence(conn, ContentType.TEXT, content_detail=text)
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
conn.dialogue.put(Message(role="assistant", content=text))
|
||||
|
||||
@@ -66,12 +66,11 @@ async def startToChat(conn, text):
|
||||
|
||||
async def no_voice_close_connect(conn, have_voice):
|
||||
if have_voice:
|
||||
conn.client_no_voice_last_time = 0.0
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
return
|
||||
if conn.client_no_voice_last_time == 0.0:
|
||||
conn.client_no_voice_last_time = time.time() * 1000
|
||||
else:
|
||||
no_voice_time = time.time() * 1000 - conn.client_no_voice_last_time
|
||||
# 只有在已经初始化过时间戳的情况下才进行超时检查
|
||||
if conn.last_activity_time > 0.0:
|
||||
no_voice_time = time.time() * 1000 - conn.last_activity_time
|
||||
close_connection_no_voice_time = int(
|
||||
conn.config.get("close_connection_no_voice_time", 120)
|
||||
)
|
||||
|
||||
@@ -93,7 +93,7 @@ async def sendAudio(conn, audios, pre_buffer=True):
|
||||
break
|
||||
|
||||
# 重置没有声音的状态
|
||||
conn.client_no_voice_last_time = 0.0
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
|
||||
# 计算预期发送时间
|
||||
expected_time = start_time + (play_position / 1000)
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import httpx
|
||||
import openai
|
||||
from openai.types import CompletionUsage
|
||||
from config.logger import setup_logging
|
||||
@@ -16,6 +17,9 @@ class LLMProvider(LLMProviderBase):
|
||||
self.base_url = config.get("base_url")
|
||||
else:
|
||||
self.base_url = config.get("url")
|
||||
# 增加timeout的配置项,单位为秒
|
||||
timeout = config.get("timeout", 300)
|
||||
self.timeout = int(timeout) if timeout else 300
|
||||
|
||||
param_defaults = {
|
||||
"max_tokens": (500, int),
|
||||
@@ -42,7 +46,7 @@ class LLMProvider(LLMProviderBase):
|
||||
model_key_msg = check_model_key("LLM", self.api_key)
|
||||
if model_key_msg:
|
||||
logger.bind(tag=TAG).error(model_key_msg)
|
||||
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url)
|
||||
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url, timeout=httpx.Timeout(self.timeout))
|
||||
|
||||
def response(self, session_id, dialogue, **kwargs):
|
||||
try:
|
||||
|
||||
@@ -161,13 +161,6 @@ class TTSProviderBase(ABC):
|
||||
else:
|
||||
sentence_id = str(uuid.uuid4()).replace("-", "")
|
||||
conn.sentence_id = sentence_id
|
||||
self.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
# 对于单句的文本,进行分段处理
|
||||
segments = re.split(r"([。!?!?;;\n])", content_detail)
|
||||
for seg in segments:
|
||||
@@ -180,13 +173,6 @@ class TTSProviderBase(ABC):
|
||||
content_file=content_file,
|
||||
)
|
||||
)
|
||||
self.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
|
||||
async def open_audio_channels(self, conn):
|
||||
self.conn = conn
|
||||
@@ -209,6 +195,8 @@ class TTSProviderBase(ABC):
|
||||
while not self.conn.stop_event.is_set():
|
||||
try:
|
||||
message = self.tts_text_queue.get(timeout=1)
|
||||
if message.sentence_type == SentenceType.FIRST:
|
||||
self.conn.client_abort = False
|
||||
if self.conn.client_abort:
|
||||
logger.bind(tag=TAG).info("收到打断信息,终止TTS文本处理线程")
|
||||
continue
|
||||
@@ -362,10 +350,8 @@ class TTSProviderBase(ABC):
|
||||
return audio_datas
|
||||
|
||||
def _process_before_stop_play_files(self):
|
||||
for tts_file, text in self.before_stop_play_files:
|
||||
if tts_file and os.path.exists(tts_file):
|
||||
audio_datas = self._process_audio_file(tts_file)
|
||||
self.tts_audio_queue.put((SentenceType.MIDDLE, audio_datas, text))
|
||||
for audio_datas, text in self.before_stop_play_files:
|
||||
self.tts_audio_queue.put((SentenceType.MIDDLE, audio_datas, text))
|
||||
self.before_stop_play_files.clear()
|
||||
self.tts_audio_queue.put((SentenceType.LAST, [], None))
|
||||
|
||||
|
||||
@@ -85,8 +85,12 @@ class TTSProvider(TTSProviderBase):
|
||||
self.reference_id = (
|
||||
None if not config.get("reference_id") else config.get("reference_id")
|
||||
)
|
||||
self.reference_audio = parse_string_to_list(config.get("reference_audio"))
|
||||
self.reference_text = parse_string_to_list(config.get("reference_text"))
|
||||
self.reference_audio = parse_string_to_list(
|
||||
config.get('ref_audio')if config.get('ref_audio') else config.get("reference_audio")
|
||||
)
|
||||
self.reference_text = parse_string_to_list(
|
||||
config.get('ref_text')if config.get('ref_text') else config.get("reference_text")
|
||||
)
|
||||
self.format = config.get("response_format", "wav")
|
||||
self.audio_file_type = config.get("response_format", "wav")
|
||||
self.api_key = config.get("api_key", "YOUR_API_KEY")
|
||||
|
||||
@@ -12,8 +12,8 @@ class TTSProvider(TTSProviderBase):
|
||||
super().__init__(config, delete_audio_file)
|
||||
self.url = config.get("url")
|
||||
self.text_lang = config.get("text_lang", "zh")
|
||||
self.ref_audio_path = config.get("ref_audio_path")
|
||||
self.prompt_text = config.get("prompt_text")
|
||||
self.ref_audio_path = config.get('ref_audio') if config.get('ref_audio') else config.get("ref_audio_path")
|
||||
self.prompt_text = config.get('ref_text') if config.get('ref_text') else config.get("prompt_text")
|
||||
self.prompt_lang = config.get("prompt_lang", "zh")
|
||||
|
||||
# 处理空字符串的情况
|
||||
|
||||
@@ -11,8 +11,8 @@ class TTSProvider(TTSProviderBase):
|
||||
def __init__(self, config, delete_audio_file):
|
||||
super().__init__(config, delete_audio_file)
|
||||
self.url = config.get("url")
|
||||
self.refer_wav_path = config.get("refer_wav_path")
|
||||
self.prompt_text = config.get("prompt_text")
|
||||
self.refer_wav_path = config.get('ref_audio')if config.get('ref_audio') else config.get("refer_wav_path")
|
||||
self.prompt_text = config.get('ref_text')if config.get('ref_text') else config.get("prompt_text")
|
||||
self.prompt_language = config.get("prompt_language")
|
||||
self.text_language = config.get("text_language", "audo")
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ from config.logger import setup_logging
|
||||
from core.utils import opus_encoder_utils
|
||||
from core.utils.util import check_model_key
|
||||
from core.providers.tts.base import TTSProviderBase
|
||||
from core.handle.abortHandle import handleAbortMessage
|
||||
from core.providers.tts.dto.dto import SentenceType, ContentType, InterfaceType
|
||||
from asyncio import Task
|
||||
|
||||
@@ -175,6 +174,9 @@ class TTSProvider(TTSProviderBase):
|
||||
async def _ensure_connection(self):
|
||||
"""建立新的WebSocket连接"""
|
||||
try:
|
||||
if self.ws:
|
||||
logger.bind(tag=TAG).info(f"使用已有链接...")
|
||||
return self.ws
|
||||
logger.bind(tag=TAG).info("开始建立新连接...")
|
||||
ws_header = {
|
||||
"X-Api-App-Key": self.appId,
|
||||
@@ -200,6 +202,10 @@ class TTSProvider(TTSProviderBase):
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"收到TTS任务|{message.sentence_type.name} | {message.content_type.name} | 会话ID: {self.conn.sentence_id}"
|
||||
)
|
||||
|
||||
if message.sentence_type == SentenceType.FIRST:
|
||||
self.conn.client_abort = False
|
||||
|
||||
if self.conn.client_abort:
|
||||
logger.bind(tag=TAG).info("收到打断信息,终止TTS文本处理线程")
|
||||
continue
|
||||
@@ -244,9 +250,12 @@ class TTSProvider(TTSProviderBase):
|
||||
logger.bind(tag=TAG).info(
|
||||
f"添加音频文件到待播放列表: {message.content_file}"
|
||||
)
|
||||
self.before_stop_play_files.append(
|
||||
(message.content_file, message.content_detail)
|
||||
)
|
||||
if message.content_file and os.path.exists(message.content_file):
|
||||
# 先处理文件音频数据
|
||||
file_audio = self._process_audio_file(message.content_file)
|
||||
self.before_stop_play_files.append(
|
||||
(file_audio, message.content_detail)
|
||||
)
|
||||
|
||||
if message.sentence_type == SentenceType.LAST:
|
||||
try:
|
||||
@@ -295,25 +304,15 @@ class TTSProvider(TTSProviderBase):
|
||||
async def start_session(self, session_id):
|
||||
logger.bind(tag=TAG).info(f"开始会话~~{session_id}")
|
||||
try:
|
||||
task = self._monitor_task
|
||||
# 会话开始时检测上个会话的监听状态
|
||||
if (
|
||||
task is not None
|
||||
and isinstance(task, Task)
|
||||
and not task.done()
|
||||
self._monitor_task is not None
|
||||
and isinstance(self._monitor_task, Task)
|
||||
and not self._monitor_task.done()
|
||||
):
|
||||
logger.bind(tag=TAG).info("等待上一个监听任务结束...")
|
||||
if self.ws is not None:
|
||||
logger.bind(tag=TAG).info("强制关闭上一个WebSocket连接以唤醒监听任务...")
|
||||
try:
|
||||
await self.ws.close()
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).warning(f"关闭上一个ws异常: {e}")
|
||||
self.ws = None
|
||||
try:
|
||||
await asyncio.wait_for(task, timeout=8)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).warning(f"等待监听任务异常: {e}")
|
||||
self._monitor_task = None
|
||||
logger.bind(tag=TAG).info("检测到未完成的上个会话,关闭监听任务和连接...")
|
||||
await self.close()
|
||||
|
||||
# 建立新连接
|
||||
await self._ensure_connection()
|
||||
|
||||
@@ -336,19 +335,7 @@ class TTSProvider(TTSProviderBase):
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"启动会话失败: {str(e)}")
|
||||
# 确保清理资源
|
||||
if hasattr(self, "_monitor_task"):
|
||||
try:
|
||||
self._monitor_task.cancel()
|
||||
await self._monitor_task
|
||||
except:
|
||||
pass
|
||||
self._monitor_task = None
|
||||
if self.ws:
|
||||
try:
|
||||
await self.ws.close()
|
||||
except:
|
||||
pass
|
||||
self.ws = None
|
||||
await self.close()
|
||||
raise
|
||||
|
||||
async def finish_session(self, session_id):
|
||||
@@ -368,7 +355,7 @@ class TTSProvider(TTSProviderBase):
|
||||
logger.bind(tag=TAG).info("会话结束请求已发送")
|
||||
|
||||
# 等待监听任务完成
|
||||
if hasattr(self, "_monitor_task"):
|
||||
if self._monitor_task:
|
||||
try:
|
||||
await self._monitor_task
|
||||
except Exception as e:
|
||||
@@ -378,28 +365,25 @@ class TTSProvider(TTSProviderBase):
|
||||
finally:
|
||||
self._monitor_task = None
|
||||
|
||||
# 关闭连接
|
||||
await self.close()
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"关闭会话失败: {str(e)}")
|
||||
# 确保清理资源
|
||||
if hasattr(self, "_monitor_task"):
|
||||
try:
|
||||
self._monitor_task.cancel()
|
||||
await self._monitor_task
|
||||
except:
|
||||
pass
|
||||
self._monitor_task = None
|
||||
if self.ws:
|
||||
try:
|
||||
await self.ws.close()
|
||||
except:
|
||||
pass
|
||||
self.ws = None
|
||||
await self.close()
|
||||
raise
|
||||
|
||||
async def close(self):
|
||||
"""资源清理方法"""
|
||||
# 取消监听任务
|
||||
if self._monitor_task:
|
||||
try:
|
||||
self._monitor_task.cancel()
|
||||
await self._monitor_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).warning(f"关闭时取消监听任务错误: {e}")
|
||||
self._monitor_task = None
|
||||
|
||||
if self.ws:
|
||||
try:
|
||||
await self.ws.close()
|
||||
@@ -413,6 +397,7 @@ class TTSProvider(TTSProviderBase):
|
||||
is_first_sentence = True
|
||||
first_sentence_segment_count = 0 # 添加计数器
|
||||
try:
|
||||
session_finished = False # 标记会话是否正常结束
|
||||
while not self.conn.stop_event.is_set():
|
||||
try:
|
||||
# 确保 `recv()` 运行在同一个 event loop
|
||||
@@ -466,6 +451,7 @@ class TTSProvider(TTSProviderBase):
|
||||
elif res.optional.event == EVENT_SessionFinished:
|
||||
logger.bind(tag=TAG).debug(f"会话结束~~")
|
||||
self._process_before_stop_play_files()
|
||||
session_finished = True
|
||||
break
|
||||
except websockets.ConnectionClosed:
|
||||
logger.bind(tag=TAG).warning("WebSocket连接已关闭")
|
||||
@@ -476,15 +462,15 @@ class TTSProvider(TTSProviderBase):
|
||||
)
|
||||
traceback.print_exc()
|
||||
break
|
||||
finally:
|
||||
# 确保清理资源
|
||||
if self.ws:
|
||||
# 仅在连接异常时才关闭
|
||||
if not session_finished and self.ws:
|
||||
try:
|
||||
await self.ws.close()
|
||||
except:
|
||||
pass
|
||||
self.ws = None
|
||||
# 监听任务退出时清理引用
|
||||
finally:
|
||||
self._monitor_task = None
|
||||
|
||||
async def send_event(
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import os
|
||||
import queue
|
||||
import asyncio
|
||||
import traceback
|
||||
@@ -63,9 +64,12 @@ class TTSProvider(TTSProviderBase):
|
||||
logger.bind(tag=TAG).info(
|
||||
f"添加音频文件到待播放列表: {message.content_file}"
|
||||
)
|
||||
self.before_stop_play_files.append(
|
||||
(message.content_file, message.content_detail)
|
||||
)
|
||||
if message.content_file and os.path.exists(message.content_file):
|
||||
# 先处理文件音频数据
|
||||
file_audio = self._process_audio_file(message.content_file)
|
||||
self.before_stop_play_files.append(
|
||||
(file_audio, message.content_detail)
|
||||
)
|
||||
|
||||
if message.sentence_type == SentenceType.LAST:
|
||||
# 处理剩余的文本
|
||||
|
||||
@@ -23,22 +23,24 @@ class VADProvider(VADProviderBase):
|
||||
|
||||
# 处理空字符串的情况
|
||||
threshold = config.get("threshold", "0.5")
|
||||
threshold_low = config.get("threshold_low", "0.2")
|
||||
min_silence_duration_ms = config.get("min_silence_duration_ms", "1000")
|
||||
|
||||
self.vad_threshold = float(threshold) if threshold else 0.5
|
||||
self.vad_threshold_low = float(threshold_low) if threshold_low else 0.2
|
||||
|
||||
self.silence_threshold_ms = (
|
||||
int(min_silence_duration_ms) if min_silence_duration_ms else 1000
|
||||
)
|
||||
|
||||
# 至少要多少帧才算有语音
|
||||
self.frame_window_threshold = 3
|
||||
|
||||
def is_vad(self, conn, opus_packet):
|
||||
try:
|
||||
pcm_frame = self.decoder.decode(opus_packet, 960)
|
||||
conn.client_audio_buffer.extend(pcm_frame) # 将新数据加入缓冲区
|
||||
|
||||
# 初始化帧计数器
|
||||
if not hasattr(conn, "client_voice_frame_count"):
|
||||
conn.client_voice_frame_count = 0
|
||||
|
||||
# 处理缓冲区中的完整帧(每次处理512采样点)
|
||||
client_have_voice = False
|
||||
while len(conn.client_audio_buffer) >= 512 * 2:
|
||||
@@ -54,26 +56,30 @@ class VADProvider(VADProviderBase):
|
||||
# 检测语音活动
|
||||
with torch.no_grad():
|
||||
speech_prob = self.model(audio_tensor, 16000).item()
|
||||
is_voice = speech_prob >= self.vad_threshold
|
||||
|
||||
if is_voice:
|
||||
conn.client_voice_frame_count += 1
|
||||
# 双阈值判断
|
||||
if speech_prob >= self.vad_threshold:
|
||||
is_voice = True
|
||||
elif speech_prob <= self.vad_threshold_low:
|
||||
is_voice = False
|
||||
else:
|
||||
conn.client_voice_frame_count = 0
|
||||
is_voice = conn.last_is_voice
|
||||
|
||||
# 只有连续4帧检测到语音才认为有语音
|
||||
client_have_voice = conn.client_voice_frame_count >= 4
|
||||
# 声音没低于最低值则延续前一个状态,判断为有声音
|
||||
conn.last_is_voice = is_voice
|
||||
|
||||
# 更新滑动窗口
|
||||
conn.client_voice_window.append(is_voice)
|
||||
client_have_voice = (conn.client_voice_window.count(True) >= self.frame_window_threshold)
|
||||
|
||||
# 如果之前有声音,但本次没有声音,且与上次有声音的时间差已经超过了静默阈值,则认为已经说完一句话
|
||||
if conn.client_have_voice and not client_have_voice:
|
||||
stop_duration = (
|
||||
time.time() * 1000 - conn.client_have_voice_last_time
|
||||
)
|
||||
stop_duration = time.time() * 1000 - conn.last_activity_time
|
||||
if stop_duration >= self.silence_threshold_ms:
|
||||
conn.client_voice_stop = True
|
||||
if client_have_voice:
|
||||
conn.client_have_voice = True
|
||||
conn.client_have_voice_last_time = time.time() * 1000
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
|
||||
return client_have_voice
|
||||
except opuslib_next.OpusError as e:
|
||||
|
||||
@@ -40,6 +40,7 @@ class VLLMProvider(VLLMProviderBase):
|
||||
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url)
|
||||
|
||||
def response(self, question, base64_image):
|
||||
question = question + "(请使用中文回复)"
|
||||
try:
|
||||
messages = [
|
||||
{
|
||||
|
||||
@@ -980,4 +980,30 @@ def is_valid_image_file(file_data: bytes) -> bool:
|
||||
|
||||
def sanitize_tool_name(name: str) -> str:
|
||||
"""Sanitize tool names for OpenAI compatibility."""
|
||||
return re.sub(r"[^a-zA-Z0-9_-]", "_", name)
|
||||
# 支持中文、英文字母、数字、下划线和连字符
|
||||
return re.sub(r"[^a-zA-Z0-9_\-\u4e00-\u9fff]", "_", name)
|
||||
|
||||
|
||||
def validate_mcp_endpoint(mcp_endpoint: str) -> bool:
|
||||
"""
|
||||
校验MCP接入点格式
|
||||
|
||||
Args:
|
||||
mcp_endpoint: MCP接入点字符串
|
||||
|
||||
Returns:
|
||||
bool: 是否有效
|
||||
"""
|
||||
# 1. 检查是否以ws开头
|
||||
if not mcp_endpoint.startswith("ws"):
|
||||
return False
|
||||
|
||||
# 2. 检查是否包含key、call字样
|
||||
if "key" in mcp_endpoint.lower() or "call" in mcp_endpoint.lower():
|
||||
return False
|
||||
|
||||
# 3. 检查是否包含/mcp/字样
|
||||
if "/mcp/" not in mcp_endpoint:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
@@ -57,8 +57,25 @@ class WebSocketServer:
|
||||
self.active_connections.add(handler)
|
||||
try:
|
||||
await handler.handle_connection(websocket)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"处理连接时出错: {e}")
|
||||
finally:
|
||||
# 确保从活动连接集合中移除
|
||||
self.active_connections.discard(handler)
|
||||
# 强制关闭连接(如果还没有关闭的话)
|
||||
try:
|
||||
# 安全地检查WebSocket状态并关闭
|
||||
if hasattr(websocket, "closed") and not websocket.closed:
|
||||
await websocket.close()
|
||||
elif hasattr(websocket, "state") and websocket.state.name != "CLOSED":
|
||||
await websocket.close()
|
||||
else:
|
||||
# 如果没有closed属性,直接尝试关闭
|
||||
await websocket.close()
|
||||
except Exception as close_error:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"服务器端强制关闭连接时出错: {close_error}"
|
||||
)
|
||||
|
||||
async def _http_response(self, websocket, request_headers):
|
||||
# 检查是否为 WebSocket 升级请求
|
||||
|
||||
@@ -48,6 +48,7 @@ services:
|
||||
- SPRING_DATASOURCE_DRUID_USERNAME=root
|
||||
- SPRING_DATASOURCE_DRUID_PASSWORD=123456
|
||||
- SPRING_DATA_REDIS_HOST=xiaozhi-esp32-server-redis
|
||||
- SPRING_DATA_REDIS_PASSWORD=
|
||||
- SPRING_DATA_REDIS_PORT=6379
|
||||
volumes:
|
||||
# 配置文件目录
|
||||
|
||||
@@ -8,30 +8,90 @@ from markitdown import MarkItDown
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
|
||||
# 新闻来源字典,包含名称和对应的API ID
|
||||
NEWS_SOURCES = {
|
||||
"thepaper": "澎湃新闻",
|
||||
"baidu": "百度热搜",
|
||||
"cls-depth": "财联社",
|
||||
CHANNEL_MAP = {
|
||||
"V2EX": "v2ex-share",
|
||||
"知乎": "zhihu",
|
||||
"微博": "weibo",
|
||||
"联合早报": "zaobao",
|
||||
"酷安": "coolapk",
|
||||
"MKTNews": "mktnews-flash",
|
||||
"华尔街见闻": "wallstreetcn-quick",
|
||||
"36氪": "36kr-quick",
|
||||
"抖音": "douyin",
|
||||
"虎扑": "hupu",
|
||||
"百度贴吧": "tieba",
|
||||
"今日头条": "toutiao",
|
||||
"IT之家": "ithome",
|
||||
"澎湃新闻": "thepaper",
|
||||
"卫星通讯社": "sputniknewscn",
|
||||
"参考消息": "cankaoxiaoxi",
|
||||
"远景论坛": "pcbeta-windows11",
|
||||
"财联社": "cls-depth",
|
||||
"雪球": "xueqiu-hotstock",
|
||||
"格隆汇": "gelonghui",
|
||||
"法布财经": "fastbull-express",
|
||||
"Solidot": "solidot",
|
||||
"Hacker News": "hackernews",
|
||||
"Product Hunt": "producthunt",
|
||||
"Github": "github-trending-today",
|
||||
"哔哩哔哩": "bilibili-hot-search",
|
||||
"快手": "kuaishou",
|
||||
"靠谱新闻": "kaopu",
|
||||
"金十数据": "jin10",
|
||||
"百度热搜": "baidu",
|
||||
"牛客": "nowcoder",
|
||||
"少数派": "sspai",
|
||||
"稀土掘金": "juejin",
|
||||
"凤凰网": "ifeng",
|
||||
"虫部落": "chongbuluo-latest",
|
||||
}
|
||||
|
||||
|
||||
# 动态生成新闻源描述
|
||||
def generate_news_sources_description():
|
||||
sources_desc = []
|
||||
for source_id, source_name in NEWS_SOURCES.items():
|
||||
sources_desc.append(f"{source_name}({source_id})")
|
||||
return "、".join(sources_desc)
|
||||
# 默认新闻来源字典,当配置中没有指定时使用
|
||||
DEFAULT_NEWS_SOURCES = "澎湃新闻;百度热搜;财联社"
|
||||
|
||||
|
||||
def get_news_sources_from_config(conn):
|
||||
"""从配置中获取新闻源字符串"""
|
||||
try:
|
||||
# 尝试从插件配置中获取新闻源
|
||||
if (
|
||||
conn.config.get("plugins")
|
||||
and conn.config["plugins"].get("get_news_from_newsnow")
|
||||
and conn.config["plugins"]["get_news_from_newsnow"].get("news_sources")
|
||||
):
|
||||
# 获取配置的新闻源字符串
|
||||
news_sources_config = conn.config["plugins"]["get_news_from_newsnow"][
|
||||
"news_sources"
|
||||
]
|
||||
|
||||
if isinstance(news_sources_config, str) and news_sources_config.strip():
|
||||
logger.bind(tag=TAG).debug(f"使用配置的新闻源: {news_sources_config}")
|
||||
return news_sources_config
|
||||
else:
|
||||
logger.bind(tag=TAG).warning("新闻源配置为空或格式错误,使用默认配置")
|
||||
else:
|
||||
logger.bind(tag=TAG).debug("未找到新闻源配置,使用默认配置")
|
||||
|
||||
return DEFAULT_NEWS_SOURCES
|
||||
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"获取新闻源配置失败: {e},使用默认配置")
|
||||
return DEFAULT_NEWS_SOURCES
|
||||
|
||||
|
||||
# 从CHANNEL_MAP获取所有可用的新闻源名称
|
||||
available_sources = list(CHANNEL_MAP.keys())
|
||||
example_sources_str = "、".join(available_sources)
|
||||
|
||||
GET_NEWS_FROM_NEWSNOW_FUNCTION_DESC = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_news_from_newsnow",
|
||||
"description": (
|
||||
"获取最新新闻,随机选择一条新闻进行播报。"
|
||||
f"用户可以选择不同的新闻源,如{generate_news_sources_description()}等。"
|
||||
"如果没有指定,默认从澎湃新闻获取。"
|
||||
f"用户可以选择不同的新闻源,标准的名称是:{example_sources_str}"
|
||||
"例如用户要求百度新闻,其实就是百度热搜。如果没有指定,默认从澎湃新闻获取。"
|
||||
"用户可以要求获取详细内容,此时会获取新闻的详细内容。"
|
||||
),
|
||||
"parameters": {
|
||||
@@ -39,7 +99,7 @@ GET_NEWS_FROM_NEWSNOW_FUNCTION_DESC = {
|
||||
"properties": {
|
||||
"source": {
|
||||
"type": "string",
|
||||
"description": f"新闻源,例如{generate_news_sources_description()}等。可选参数,如果不提供则使用默认新闻源",
|
||||
"description": f"新闻源的标准中文名称,例如{example_sources_str}等。可选参数,如果不提供则使用默认新闻源",
|
||||
},
|
||||
"detail": {
|
||||
"type": "boolean",
|
||||
@@ -111,10 +171,13 @@ def fetch_news_detail(url):
|
||||
ToolType.SYSTEM_CTL,
|
||||
)
|
||||
def get_news_from_newsnow(
|
||||
conn, source: str = "thepaper", detail: bool = False, lang: str = "zh_CN"
|
||||
conn, source: str = "澎湃新闻", detail: bool = False, lang: str = "zh_CN"
|
||||
):
|
||||
"""获取新闻并随机选择一条进行播报,或获取上一条新闻的详细内容"""
|
||||
try:
|
||||
# 获取当前配置的新闻源
|
||||
news_sources = get_news_sources_from_config(conn)
|
||||
|
||||
# 如果detail为True,获取上一条新闻的详细内容
|
||||
detail = str(detail).lower() == "true"
|
||||
if detail:
|
||||
@@ -132,7 +195,7 @@ def get_news_from_newsnow(
|
||||
url = conn.last_newsnow_link.get("url")
|
||||
title = conn.last_newsnow_link.get("title", "未知标题")
|
||||
source_id = conn.last_newsnow_link.get("source_id", "thepaper")
|
||||
source_name = NEWS_SOURCES.get(source_id, "未知来源")
|
||||
source_name = CHANNEL_MAP.get(source_id, "未知来源")
|
||||
|
||||
if not url or url == "#":
|
||||
return ActionResponse(
|
||||
@@ -166,21 +229,32 @@ def get_news_from_newsnow(
|
||||
return ActionResponse(Action.REQLLM, detail_report, None)
|
||||
|
||||
# 否则,获取新闻列表并随机选择一条
|
||||
# 验证新闻源是否有效,如果无效则使用默认源
|
||||
if source not in NEWS_SOURCES:
|
||||
logger.bind(tag=TAG).warning(f"无效的新闻源: {source},使用默认源thepaper")
|
||||
source = "thepaper"
|
||||
# 将中文名称转换为英文ID
|
||||
english_source_id = None
|
||||
|
||||
source_name = NEWS_SOURCES.get(source, "澎湃新闻")
|
||||
logger.bind(tag=TAG).info(f"获取新闻: 新闻源={source}({source_name})")
|
||||
# 检查输入的中文名称是否在配置的新闻源中
|
||||
news_sources_list = [
|
||||
name.strip() for name in news_sources.split(";") if name.strip()
|
||||
]
|
||||
if source in news_sources_list:
|
||||
# 如果输入的中文名称在配置的新闻源中,在 CHANNEL_MAP 中查找对应的英文ID
|
||||
english_source_id = CHANNEL_MAP.get(source)
|
||||
|
||||
# 如果找不到对应的英文ID,使用默认源
|
||||
if not english_source_id:
|
||||
logger.bind(tag=TAG).warning(f"无效的新闻源: {source},使用默认源澎湃新闻")
|
||||
english_source_id = "thepaper"
|
||||
source = "澎湃新闻"
|
||||
|
||||
logger.bind(tag=TAG).info(f"获取新闻: 新闻源={source}({english_source_id})")
|
||||
|
||||
# 获取新闻列表
|
||||
news_items = fetch_news_from_api(conn, source)
|
||||
news_items = fetch_news_from_api(conn, english_source_id)
|
||||
|
||||
if not news_items:
|
||||
return ActionResponse(
|
||||
Action.REQLLM,
|
||||
f"抱歉,未能从{source_name}获取到新闻信息,请稍后再试或尝试其他新闻源。",
|
||||
f"抱歉,未能从{source}获取到新闻信息,请稍后再试或尝试其他新闻源。",
|
||||
None,
|
||||
)
|
||||
|
||||
@@ -193,14 +267,14 @@ def get_news_from_newsnow(
|
||||
conn.last_newsnow_link = {
|
||||
"url": selected_news.get("url", "#"),
|
||||
"title": selected_news.get("title", "未知标题"),
|
||||
"source_id": source,
|
||||
"source_id": english_source_id,
|
||||
}
|
||||
|
||||
# 构建新闻报告
|
||||
news_report = (
|
||||
f"根据下列数据,用{lang}回应用户的新闻查询请求:\n\n"
|
||||
f"新闻标题: {selected_news['title']}\n"
|
||||
# f"新闻来源: {source_name}\n"
|
||||
# f"新闻来源: {source}\n"
|
||||
f"(请以自然、流畅的方式向用户播报这条新闻标题,"
|
||||
f"提示用户可以要求获取详细内容,此时会获取新闻的详细内容。)"
|
||||
)
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
from config.logger import setup_logging
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import random
|
||||
import asyncio
|
||||
import difflib
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
from core.utils import p3
|
||||
from core.handle.sendAudioHandle import send_stt_message
|
||||
from plugins_func.register import register_function, ToolType, ActionResponse, Action
|
||||
from core.utils.dialogue import Message
|
||||
@@ -51,8 +48,8 @@ def play_music(conn, song_name: str):
|
||||
)
|
||||
|
||||
# 提交异步任务
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
handle_music_command(conn, music_intent), conn.loop
|
||||
task = conn.loop.create_task(
|
||||
handle_music_command(conn, music_intent) # 封装异步逻辑
|
||||
)
|
||||
|
||||
# 非阻塞回调处理
|
||||
@@ -63,7 +60,7 @@ def play_music(conn, song_name: str):
|
||||
except Exception as e:
|
||||
conn.logger.bind(tag=TAG).error(f"播放失败: {e}")
|
||||
|
||||
future.add_done_callback(handle_done)
|
||||
task.add_done_callback(handle_done)
|
||||
|
||||
return ActionResponse(
|
||||
action=Action.NONE, result="指令已接收", response="正在为您播放音乐"
|
||||
@@ -178,13 +175,13 @@ def _get_random_play_prompt(song_name):
|
||||
# 移除文件扩展名
|
||||
clean_name = os.path.splitext(song_name)[0]
|
||||
prompts = [
|
||||
f"正在为您播放,{clean_name}",
|
||||
f"请欣赏歌曲,{clean_name}",
|
||||
f"即将为您播放,{clean_name}",
|
||||
f"为您带来,{clean_name}",
|
||||
f"让我们聆听,{clean_name}",
|
||||
f"接下来请欣赏,{clean_name}",
|
||||
f"为您献上,{clean_name}",
|
||||
f"正在为您播放,《{clean_name}》",
|
||||
f"请欣赏歌曲,《{clean_name}》",
|
||||
f"即将为您播放,《{clean_name}》",
|
||||
f"现在为您带来,《{clean_name}》",
|
||||
f"让我们一起聆听,《{clean_name}》",
|
||||
f"接下来请欣赏,《{clean_name}》",
|
||||
f"此刻为您献上,《{clean_name}》",
|
||||
]
|
||||
# 直接使用random.choice,不设置seed
|
||||
return random.choice(prompts)
|
||||
@@ -218,13 +215,14 @@ async def play_local_music(conn, specific_file=None):
|
||||
await send_stt_message(conn, text)
|
||||
conn.dialogue.put(Message(role="assistant", content=text))
|
||||
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
if conn.intent_type == "intent_llm":
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
)
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
@@ -241,13 +239,14 @@ async def play_local_music(conn, specific_file=None):
|
||||
content_file=music_path,
|
||||
)
|
||||
)
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
if conn.intent_type == "intent_llm":
|
||||
conn.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=conn.sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
conn.logger.bind(tag=TAG).error(f"播放音乐失败: {str(e)}")
|
||||
|
||||
@@ -21,7 +21,7 @@ cozepy==0.12.0
|
||||
mem0ai==0.1.62
|
||||
bs4==0.0.2
|
||||
modelscope==1.23.2
|
||||
sherpa_onnx==1.12.0
|
||||
sherpa_onnx==1.12.4
|
||||
mcp==1.8.1
|
||||
cnlunar==0.2.0
|
||||
PySocks==1.7.1
|
||||
|
||||
Reference in New Issue
Block a user