Compare commits

...
194 Commits
Author SHA1 Message Date
欣南科技andGitHub e37b094d1e Merge pull request #1813 from xinnan-tech/py_fix_log
Py fix log
2025-07-11 17:29:49 +08:00
欣南科技andGitHub a491ce96cb Merge branch 'main' into py_fix_log 2025-07-11 17:29:37 +08:00
hrz 7be79519b1 update:增加声纹识别演示视频 2025-07-11 17:23:13 +08:00
3030332422 47e4c0f190 fix:日志文件的优化 2025-07-11 16:44:50 +08:00
欣南科技andGitHub 6b24844172 Merge pull request #1810 from xinnan-tech/hot-fix
update:优化总结
2025-07-11 15:43:39 +08:00
hrz c6225f3ab8 update:优化总结 2025-07-11 15:42:46 +08:00
欣南科技andGitHub 60c4aa943b Merge pull request #1807 from xinnan-tech/hot-fix
add:声纹识别开启教程
2025-07-11 14:14:32 +08:00
hrz 81d0adf73c add:声纹识别开启教程 2025-07-11 14:13:39 +08:00
欣南科技andGitHub 7dc3929f0e Merge pull request #1806 from xinnan-tech/hot-fix
update:优化声纹识别有结果时内容的展示
2025-07-11 12:15:05 +08:00
hrz 09be3a4de3 update:优化声纹识别有结果时内容的展示 2025-07-11 12:14:25 +08:00
欣南科技andGitHub f7407e46c5 Merge pull request #1805 from xinnan-tech/hot-fix
update:修复二次声纹注册bug
2025-07-11 11:57:25 +08:00
hrz 8f685967d3 update:修复二次声纹注册bug 2025-07-11 11:49:53 +08:00
欣南科技andGitHub 8f6dd22a6d Merge pull request #1804 from xinnan-tech/mangger-api-voice-print
声纹识别
2025-07-11 10:19:15 +08:00
hrz 108c87f395 update:优化提示词 2025-07-11 10:15:14 +08:00
hrz cd58b3a39a update:发行新版本 2025-07-11 09:14:34 +08:00
3030332422 2fbee33185 update:python对接声纹识别优化 2025-07-10 15:03:49 +08:00
JianYu Zheng f499040955 Merge remote-tracking branch 'origin/mangger-api-voice-print' into mangger-api-voice-print 2025-07-10 11:33:28 +08:00
JianYu Zheng 008c446966 添加判断
--AgentVoicePrintServiceImpl.java
 1.声纹音频保存,增加了保存前识别一下声纹是否有相似的声音,有则不保存
 2.声纹音频修改,增加了保存前识别一下声纹是否有相似的声音且相似声纹id和本次修改id不一样的,则不修改
 3.添加了识别声纹http的请求的方法
 --IdentifyVoicePrintResponse.java 识别声纹响应类型
2025-07-10 11:20:32 +08:00
hrz 01158e67e0 add:智控台返回声纹接口信息 2025-07-10 09:54:16 +08:00
f4cfa04954 update: TTS复用链接,VAD双阈值判断 (#1742)
* update: TTS复用链接,VAD双阈值判断

* fix: 播放音乐时,引导词卡顿

* update:优化引导词

* update: 优化chat函数流程 优化huoshan处理 会话保持一致性

* fix: 等待时可能已经完成(设置为None),后续对None错误访问

* update:增加书名号

* fix: 打断状态未重置 监听未完成时服务端可能还在发送数据 此时复用链接会接收上个语音的残余 需要两者一同关闭

---------

Co-authored-by: hrz <1710360675@qq.com>
2025-07-09 18:34:04 +08:00
JianYu Zheng ae4387ce3f Merge remote-tracking branch 'origin/mangger-api-voice-print' into mangger-api-voice-print 2025-07-09 18:12:27 +08:00
JianYu Zheng a64b495083 增加判断
--AgentVoicePrintServiceImpl.java 添加,修改声纹音频的是增加了验证这个音频是否属于这个智能体
2025-07-09 17:31:34 +08:00
JianYu Zheng 81cdd0a211 添加新接口和2个新方法
--AgentChatHistoryService.java 添加‘根据音频数据ID获取聊天内容’和‘查询此音频id是否属于此智能体’的方法定义
--AgentChatHistoryServiceImpl.java 实现2个新方法
--AgentController.java 增加一个获取音频内容的接口
2025-07-09 17:29:36 +08:00
Sakura-RanChen 6bfb5f1340 Merge branch 'py_link_huoshan' of https://github.com/xinnan-tech/xiaozhi-esp32-server into py_link_huoshan 2025-07-09 17:05:02 +08:00
Sakura-RanChen 2625133f40 fix: 打断状态未重置 监听未完成时服务端可能还在发送数据 此时复用链接会接收上个语音的残余 需要两者一同关闭 2025-07-09 17:04:42 +08:00
hrzandGitHub 38409b4200 Merge pull request #1755 from xinnan-tech/py_test
update:python单模块部署声纹识别对接
2025-07-09 16:51:26 +08:00
hrzandGitHub 29b4d0daeb Merge branch 'mangger-api-voice-print' into py_test 2025-07-09 16:51:10 +08:00
hrz 8f4f9fe19a update:优化提示词 2025-07-09 16:30:10 +08:00
hrz 7f67459c6e update:增加书名号 2025-07-09 16:26:26 +08:00
JianYu Zheng f47a7b4dbc 修复添加和修改声纹选择的时候,编辑的声纹不在里面,导致无法显示内容的问题
--agent.js 增加查询音频内容的接口
--VoicePrintDialog.vue 修复添加和修改声纹选择的时候,编辑的声纹不在里面,导致无法显示内容的问题
2025-07-09 15:56:07 +08:00
hrz 0bf6926506 update:优化提示词 2025-07-09 15:35:25 +08:00
3030332422 d13fb73c67 update:优化 2025-07-09 14:23:47 +08:00
JianYu Zheng 5e18a46e3f 添加新功能,声纹选择的时候可以听取音频文件
--VoicePrintDialog.vue 添加听取音频的功能
2025-07-09 11:48:02 +08:00
Sakura-RanChen 62e39dd0e2 fix: 等待时可能已经完成(设置为None),后续对None错误访问 2025-07-09 11:18:26 +08:00
hrzandGitHub 203ea89c6d Merge pull request #1788 from xinnan-tech/py_intent_fix
fix: delete未定义的方法报错
2025-07-08 17:54:17 +08:00
Sakura-RanChen 9ddd967776 fix: delete未定义的方法报错 2025-07-08 17:52:51 +08:00
Sakura-RanChen 44593ff3f3 update: 优化chat函数流程 优化huoshan处理 会话保持一致性 2025-07-08 17:48:50 +08:00
JianYu Zheng 3ed13b3ff5 修复漏洞
--AgentChatHistoryServiceImpl.java 用户聊天记录不一定都带有音频,查询时要排除不带音频的最近50条聊天记录
--AgentChatHistoryService.java 修改方法描述
2025-07-08 17:32:44 +08:00
JianYu Zheng 53cfdcc240 Merge remote-tracking branch 'origin/mangger-api-voice-print' into mangger-api-voice-print 2025-07-08 17:15:10 +08:00
JianYu Zheng 0636b95b87 声纹表添加音频字段
--202507081646.sql 添加字段sql
--AgentVoicePrintEntity.java 对应实体更新
--db.changelog-master.yaml 执行添加字段sql的配置
2025-07-08 17:10:57 +08:00
hrzandGitHub cf449a9b60 Merge pull request #1787 from xinnan-tech/system-prompt
强化小智提示词
2025-07-08 17:00:46 +08:00
hrz ab890ada37 update:强化小智提示词 2025-07-08 16:59:38 +08:00
JianYu Zheng b9c136c6ae 修复错误
--VoicePrint.vue
 1.方法调用错误,修改方法名
 2.删除无用的多选框
2025-07-08 16:55:02 +08:00
JianYu Zheng f81a7e8273 智能体设置卡片添加设置声纹的按钮
--DeviceItem.vue
 1.添加声纹设置按钮
 2.添加跳转到声纹页面方法
--home.vue 修复因为添加新按钮导致卡片按钮文字换行样式问题
2025-07-08 16:18:29 +08:00
JianYu Zheng d40d4a08b0 添加声纹管理页面
--VoicePrint.vue 声纹展示页面
--VoicePrintDialog.vue 声纹增改组件
--index.js 注册声纹页面路径
2025-07-08 16:15:52 +08:00
JianYu Zheng 61907d2270 添加新接口方法
--agent.js 添加关于声纹和获取用户列表的接口
 1.添加智能体的声纹
 2.获取指定智能体声纹列表
 3.删除智能体声纹
 4.更新智能体声纹
 5.获取指定智能体用户类型聊天记录
2025-07-08 16:13:13 +08:00
JianYu Zheng 601a90981e 修复错误
--AgentVoicePrintServiceImpl.java
 1.修复事务处理过程出错,返回错误无法被前端页面处理
 2.修复更新声纹方法,更正错误sql
2025-07-08 15:48:20 +08:00
JianYu Zheng 10b7259b13 添加新接口:查看指定智能体最近50条用户聊天记录
--AgentChatHistoryService.java 指定智能体最近50条用户聊天记录定义
--AgentChatHistoryServiceImpl.javaAgentChatHistoryService.java 实现指定智能体最近50条用户聊天记录方法
--AgentChatHistoryType.java 添加智能体聊天记录类型枚举
--AgentChatHistoryUserVO.java 展示用户聊天记录vo
--AgentController.java 添加新接口
2025-07-08 15:44:43 +08:00
CGD 5c83d63fe2 update:声纹识别对接的优化 2025-07-08 11:25:54 +08:00
hrzandGitHub beb7a191c1 Merge pull request #1777 from jlau-ice/feat/add-password-visibility
feat: 登录注册等页面密码输入框添加可见属性
2025-07-07 21:29:10 +08:00
ganbin ee72a328ca feat: 登录注册等页面密码输入框添加可见属性 2025-07-07 18:50:03 +08:00
JianYu Zheng c85419bf06 添加添加声纹增删改查接口
--AgentVoicePrintController.java 添加接口
2025-07-07 15:33:16 +08:00
JianYu Zheng b201c3cca8 添加数据认证,需要登录用户是此数据所有者才有权限修改,查询,删除
--AgentVoicePrintService.java 修改方法定义
--AgentVoicePrintServiceImpl.java
 1.需要登录用户是此数据所有者才有权限修改,查询,删除
 2.修改发送注册声纹http请求和发送注销声纹的请求的错误说明和添加日志记录
2025-07-07 15:11:37 +08:00
hrzandGitHub c45f47faa0 Update:优化mcp工具读取顺序 (#1775)
* update:发布0.6.3版本

* update:升级sherpa_onnx版本

* update:补充自动更新教程

* update:优化mcp工具读取顺序

* update:优化
2025-07-07 15:04:32 +08:00
JianYu Zheng ee7ea7fca6 添加编程事务,保证数据的统一,提取公共方法
--AgentVoicePrintServiceImpl.java
 1.为保存,修复,删除声纹添加事务
 2.提取发送注销声纹的请求为公共方法
2025-07-07 14:12:28 +08:00
hrzandGitHub 5ee7830f1b Merge pull request #1770 from jlau-ice/fix/vllm-reply-in-English
fix: 修正VLLM本地调用,question参数为空会用英文回复的问题
2025-07-07 11:58:58 +08:00
hrz 8da9dc8a3f update:优化引导词 2025-07-07 11:45:37 +08:00
JianYu Zheng 61ecfd2d92 实现智能体声纹服务层定义的‘获取指定智能体的所有声纹数据’和‘更新智能体的指的声纹数据’定义(未测试)
--AgentVoicePrintServiceImpl.java 实现方法
 1.获取指定智能体的所有声纹数据
 2.更新智能体的指的声纹数据
 3.提取共用方法(获取声纹音频资源数据,发送注册声纹http请求)
2025-07-07 10:58:35 +08:00
ganbin 2535ff6c90 fix: 修正VLLM本地调用,question参数为空会用英文回复的问题 2025-07-07 09:02:59 +08:00
hrzandGitHub 880bc6326a update:补充自动更新教程 (#1766)
* update:发布0.6.3版本

* update:升级sherpa_onnx版本

* update:补充自动更新教程
2025-07-06 15:40:02 +08:00
hrzandGitHub d892212583 Update version (#1765)
* update:发布0.6.3版本

* update:升级sherpa_onnx版本
2025-07-05 23:58:47 +08:00
hrzandGitHub 089acd357d update:发布0.6.3版本 (#1764) 2025-07-05 23:50:26 +08:00
hrzandGitHub d14f6937fa Merge pull request #1753 from myifeng/main
客户端连接成功时不主动发送hello
2025-07-05 20:05:51 +08:00
hrzandGitHub d74813ec80 Update connection.py 2025-07-05 20:05:36 +08:00
hrzandGitHub 1cfba1aa52 update: 单模块部署也能使用mcp接入点 (#1763)
* 优化:OTA接口需要适应最小示例参数 #1757

* update:修复智能体选择的模型不存在导致的500错误 #1752

* update: 单模块部署也能使用mcp接入点
2025-07-05 19:46:04 +08:00
JianYu Zheng 77527d8823 实现智能体声纹服务层定义的‘添加智能体新的声纹’和‘删除智能体的指的声纹’定义
--AgentVoicePrintServiceImpl.java 实现方法
 1.添加智能体新的声纹
 2.删除智能体的指的声纹
 3.共用方法(获取生纹接口URI对象,获取声纹地址基础路径,获取验证Authorization)
2025-07-04 18:04:39 +08:00
JianYu Zheng 5e5ec98995 添加智能体声纹实体,vo,dto,dao,service类
--AgentVoicePrintVO.java 智能体声纹展示vo
--AgentVoicePrintService.java 智能体声纹服务层定义
 1.添加智能体新的声纹
 2.删除智能体的指的声纹
 3.获取指定智能体的所有声纹数据
 4.更新智能体的指的声纹数据
--AgentVoicePrintSaveDTO.java 智能体声纹保存dto
--AgentVoicePrintUpdateDTO.java 智能体声纹修改dto
--AgentVoicePrintEntity.java 智能体声纹实体
--AgentVoicePrintDao.java 智能体声纹持久层
2025-07-04 18:02:19 +08:00
JianYu Zheng 1c2d4f9045 添加新表:智能体声纹表
--202507041018.sql 建表sql语句
--db.changelog-master.yaml 配置文件里添加执行新sql的配置
2025-07-04 17:54:41 +08:00
CGD 3d64152dba update:python单模块部署声纹识别对接 2025-07-04 17:11:44 +08:00
hrzandGitHub 37ecd3da1e 增加docker部署时url地址的说明 (#1754)
* update:优化mcp工具较多时的现实

* update:修改版本号

* update:docker部署manager-api,填写备注
2025-07-04 17:02:19 +08:00
myifeng 22e9e4c352 客户端连接成功时不主动发送hello 2025-07-04 15:27:38 +08:00
Sakura-RanChen 7e5a2b4549 fix: 播放音乐时,引导词卡顿 2025-07-04 11:27:49 +08:00
JianYu Zheng 036dde5bb0 添加新的参数:声纹接口地址
--202507031602.sql 添加声纹接口地址参数sql
--Constant.java 对应新参数的常量
--SysParamsController.java 验证声纹接口地址参数是否符合规则
--db.changelog-master.yaml 添加执行新的sql文件
2025-07-03 16:34:50 +08:00
Sakura-RanChen 69cac9d40a update: TTS复用链接,VAD双阈值判断 2025-07-03 09:16:06 +08:00
hrzandGitHub 04b132816c update:修改版本号
* update:优化mcp工具较多时的现实

* update:修改版本号
2025-07-01 18:16:21 +08:00
hrzandGitHub f08ed1bbb7 Merge pull request #1729 from xinnan-tech/fix-vllm
update:优化mcp工具较多时的显示
2025-07-01 17:31:58 +08:00
欣南科技andGitHub d59a769084 Merge pull request #1728 from xinnan-tech/redis-password
update:允许在docker-compose文件里设置redis密码
2025-07-01 16:35:20 +08:00
hrz 7a8fe1074c update:允许在docker-compose文件里设置redis密码 2025-07-01 16:34:37 +08:00
欣南科技andGitHub 3fc256ed9a Merge pull request #1727 from xinnan-tech/mcp-update
update:支持中文名称的mcp工具
2025-07-01 16:16:03 +08:00
hrz 18eed4417c update:支持中文名称的mcp工具 2025-07-01 16:07:35 +08:00
欣南科技andGitHub 20adbadc02 Merge pull request #1718 from xinnan-tech/newsnow
可配置新闻来源
2025-06-30 17:13:13 +08:00
hrz c68384be69 update:区分本地化部署和非本地化部署tts的配置字段 2025-06-30 16:56:56 +08:00
hrz 7619979046 update:简化新闻源配置 2025-06-30 14:24:46 +08:00
hrzandGitHub 54ccae8665 Merge pull request #1518 from shane04111/add_local_tts_change_voice
update: 添加本地tts支援面板的"角色音色"
2025-06-30 10:48:15 +08:00
hrzandGitHub 3015b01740 Merge branch 'newsnow' into add_local_tts_change_voice 2025-06-30 10:48:06 +08:00
hrzandGitHub 43d47a4f09 Merge pull request #1552 from liammazy/main
增加timeout配置项,单位为秒
2025-06-30 10:46:50 +08:00
hrzandGitHub 96abc34a92 Merge branch 'newsnow' into main 2025-06-30 10:44:39 +08:00
hrzandGitHub 26553b8875 Update openai.py 2025-06-30 10:41:15 +08:00
hrzandGitHub 5e3187e36b Merge pull request #1673 from goxofy/main
update: newsnow 插件支持前端配置新闻源  && fix issues/1658
2025-06-30 10:10:28 +08:00
hrzandGitHub eba8c3122f Merge branch 'newsnow' into main 2025-06-30 10:10:19 +08:00
hrzandGitHub f1a3f39782 Merge pull request #1712 from ifhaveif/main
🐞 fix: 只清空当前数据库,不清空整个数据库
2025-06-30 10:04:01 +08:00
xiongyonghui 0880f78002 🐞 fix: 只清空当前数据库,不清空整个数据库
-- emptyAll.lua
1、当redis被多个项目使用时,清空了其它项目的库(影响其它项目)
2025-06-30 00:17:39 +08:00
欣南科技andGitHub 1c16d497f9 Merge pull request #1706 from xinnan-tech/update-doc
update:增加MCP接入点的视频demo
2025-06-28 20:36:44 +08:00
hrz faaae95197 update:增加MCP接入点的视频demo 2025-06-28 20:36:05 +08:00
欣南科技andGitHub d5e0e8ab40 Merge pull request #1704 from xinnan-tech/py_websoket_link
update:优化超时资源优化
2025-06-27 23:50:25 +08:00
hrz 500b0ebde3 update:去除无关日志 2025-06-27 23:47:17 +08:00
hrz 6f6c39da23 update:优化超时资源优化 2025-06-27 23:38:05 +08:00
CGDandGitHub f8ccc7a92f Merge pull request #1703 from xinnan-tech/py_websoket_link
fix: 退出过快问题
2025-06-27 17:26:38 +08:00
Sakura-RanChen 95a678bb18 fix: 退出过快问题 2025-06-27 16:39:53 +08:00
欣南科技andGitHub e43ce6ba3c Merge pull request #1700 from xinnan-tech/manager-api-mcp
update:优化接入点连接跳转方式
2025-06-27 15:31:09 +08:00
hrz 7663b0b969 update:优化接入点连接跳转方式 2025-06-27 15:30:18 +08:00
欣南科技andGitHub ba898b0874 Merge pull request #1699 from xinnan-tech/manager-api-mcp
update:优化mcp地址的校验
2025-06-27 15:16:58 +08:00
hrz 04843010bc update:优化mcp地址的校验 2025-06-27 15:14:41 +08:00
欣南科技andGitHub 41c06efeee Merge pull request #1698 from xinnan-tech/manager-api-mcp
Manager api mcp
2025-06-27 14:59:27 +08:00
hrz d0425fa31a update:优化设备端读取mcp接入点工具 2025-06-27 14:57:49 +08:00
hrz 08753b97df update:页面显示接入点功能 2025-06-27 11:33:12 +08:00
hrz a86fef5f28 update:智能体获取mcp接入点工具列表 2025-06-27 10:06:40 +08:00
hrzandGitHub c5c2f59b31 Merge pull request #1688 from xinnan-tech/opt_tools
fix: 处理config_functions的类型转换,避免初始化错误
2025-06-26 18:32:00 +08:00
hrz 69ee8ab438 添加mcp工具列表接口 2025-06-26 18:29:07 +08:00
CGD 4a24bc8c21 fix: 处理config_functions的类型转换,避免初始化错误 2025-06-26 17:37:51 +08:00
hrz 1d293244d3 update:智控台增加mcp接入点的配置 2025-06-26 16:54:26 +08:00
hrz 6812c5ac40 update:更改版本号 2025-06-26 16:34:30 +08:00
TinKandGitHub eeb5ed920a Merge pull request #10 from goxofy/manual-add-device
fix
2025-06-26 16:29:29 +08:00
hrzandGitHub 90ad3e019f Merge pull request #1680 from xinnan-tech/opt_tools
MCP接入点完成
2025-06-26 16:27:42 +08:00
Tink 3af453c2df fix 2025-06-26 16:26:45 +08:00
hrz 3071d2bdfa update:完成mcp接入点对接 2025-06-26 16:25:39 +08:00
TinKandGitHub 90aad4f131 Merge pull request #9 from goxofy/manual-add-device
fix
2025-06-26 16:24:55 +08:00
Tink f69b1b34f1 fix 2025-06-26 16:22:21 +08:00
TinKandGitHub ece2c4b49f Merge pull request #8 from goxofy/manual-add-device
fix
2025-06-26 16:07:57 +08:00
Tink 033c8f9bee fix 2025-06-26 16:06:36 +08:00
TinKandGitHub 1b9ca95a0e Merge pull request #7 from goxofy/manual-add-device
UpdateWrapper
2025-06-26 16:00:31 +08:00
Tink 7e86a9f8a9 UpdateWrapper 2025-06-26 15:54:26 +08:00
TinKandGitHub 9dee744498 Merge pull request #6 from goxofy/manual-add-device
fix last_connected_at
2025-06-26 15:35:51 +08:00
Tink 32fa71e8f1 fix last_connected_at 2025-06-26 15:24:57 +08:00
hrz ec4694f859 update:优化服务端mcp 2025-06-26 14:35:12 +08:00
TinKandGitHub 717386f10a Merge pull request #5 from goxofy/manual-add-device
fix id in ai_device
2025-06-26 14:14:07 +08:00
Tink fe071fe41a fix id in ai_device 2025-06-26 14:13:29 +08:00
TinKandGitHub 560a7d083b Merge pull request #4 from goxofy/manual-add-device
manualAddDevice
2025-06-26 13:41:46 +08:00
Tink 4428c1a298 manualAddDevice 2025-06-26 13:40:01 +08:00
TinKandGitHub 760fd2fdc5 Merge pull request #3 from goxofy/manual-add-device
手动添加设备增加后端接口
2025-06-26 13:37:06 +08:00
Tink 84c11a281a 手动添加设备增加后端接口 2025-06-26 13:36:11 +08:00
TinKandGitHub ad682ca0c4 Merge pull request #2 from goxofy/manual-add-device
fix RequestService
2025-06-26 12:27:46 +08:00
Tink 2a349ca9ef fix RequestService 2025-06-26 12:25:19 +08:00
hrz da8435cfea update:视觉结果优化 2025-06-26 11:42:13 +08:00
hrz 0f36daf1fd update:优化工具回复 2025-06-26 11:27:23 +08:00
TinKandGitHub 1f3c90e9e5 Merge pull request #1 from goxofy/manual-add-device
manual-add-device
2025-06-26 11:16:23 +08:00
Tink 6ec7a1fe09 manual-add-device 2025-06-26 11:09:17 +08:00
hrz 2f78acaf4d update:优化iot操作 2025-06-26 09:34:00 +08:00
hrz 9412a26bfc update:优化工具目录结构 2025-06-26 09:28:29 +08:00
hrz 2348d9ceb3 update:统一工具注册及调用 2025-06-25 18:27:08 +08:00
JianYu Zheng 61b4b6617b 优化:提取共用方法,实现部分getAgentMcpToolsList内容
--AgentMcpAccessPointServiceImpl.java
1.获取URI对象,统一2个方法获取URI对象的异常处理
2.实现getAgentMcpToolsList部分内容
2025-06-25 17:39:01 +08:00
JianYu Zheng a8620f21a0 优化:方法分割提取
--AgentMcpAccessPointServiceImpl.java 方法分割提取共用
1.获取密钥方法
2.获取智能体mcp接入点url部分
3.获取对智能体id加密的token
2025-06-25 17:17:26 +08:00
Tink 78dc266eee change newsnow plugin doc 2025-06-25 16:49:54 +08:00
JianYu Zheng 0631fe37ea 添加新接口:获取智能体的Mcp接入点地址(/agent/mcp/address/{audioId})
--AgentMcpAccessPointServiceImpl.java 实现获取智能体的mcp接入点地址定义
--AgentMcpAccessPointController.java 添加获取智能体的Mcp接入点地址接口
2025-06-25 16:37:04 +08:00
Tink 8828f8401d newsnow 插件支持前端配置新闻源 2025-06-25 16:32:49 +08:00
JianYu Zheng 21339aba42 添加智能体mcp接入点接口方法定义
--AgentMcpAccessPointService.java
1.获取智能体的mcp接入点地址定义
2.获取智能体的mcp接入点已有的工具列表定义
2025-06-25 16:16:39 +08:00
JianYu Zheng 660231978f 添加一个哈希加密的工具类
--HashEncryptionUtil.java
1.指定哈希算法进行加密方法
2.使用md5进行加密
2025-06-25 16:15:06 +08:00
JianYu Zheng dde203b158 添加了一个新的系统参数,mcp接入点参数
--Constant.java mcp接入点参数的key常量
--SysParamsServiceImpl.java 添加了mcp参数验证方法,修改是进行验证
2025-06-25 11:21:41 +08:00
JianYu Zheng 0ee69fa2b2 添加了http发送工具类
--HttpSendUtils.java
添加了发送get请求,获取返回的body转换成字符串 方法
发送post请求,参数为json格式。取返回的body转换成字符串 方法
2025-06-25 11:06:36 +08:00
hrzandGitHub 8c1a8f7d55 Merge pull request #1663 from xinnan-tech/py_iot_fix
update:iot设备多指令适配
2025-06-24 16:52:10 +08:00
欣南科技andGitHub ef03b665c9 Merge pull request #1666 from xinnan-tech/aes_utils
update:新增AES加密方法,和python端AES一致
2025-06-24 16:48:56 +08:00
hrz b87906234d update:新增AES加密方法,和python端AES一致 2025-06-24 16:26:14 +08:00
CGD 896b318c49 update:iot设备多指令适配 2025-06-24 14:55:34 +08:00
CGDandGitHub b3d441c385 Merge pull request #1661 from xinnan-tech/web_fuction_MCP
update: mcp接入点页面实现
2025-06-24 14:41:12 +08:00
Sakura-RanChen 96de670bfe update: mcp接入点页面实现 2025-06-24 14:39:18 +08:00
欣南科技andGitHub bc2fc35cc9 Merge pull request #1642 from xinnan-tech/py_tts_listen
update:更新版本号
2025-06-20 17:42:53 +08:00
hrz 993d5395b2 update:更新版本号 2025-06-20 17:42:14 +08:00
欣南科技andGitHub 67f0b828ea Merge pull request #1641 from xinnan-tech/py_tts_listen
update:修复部分doubaoasr出现400错误的问题
2025-06-20 17:20:51 +08:00
hrz 879c1267b6 update:修复部分doubaoasr出现400错误的问题 2025-06-20 17:20:08 +08:00
hrzandGitHub 46c53e36a6 Merge pull request #1640 from xinnan-tech/py_tts_listen
fix: 反复打断任务没有被清除,vad四帧语音识别
2025-06-20 17:17:19 +08:00
hrz 09f6605cfe update:更新版本号 2025-06-20 17:17:09 +08:00
Sakura-RanChen 12cee4027a fix: 反复打断任务没有被清除,vad四帧语音识别 2025-06-20 16:25:08 +08:00
欣南科技andGitHub 979fea0d60 Merge pull request #1627 from xinnan-tech/update-remark
更新智控台两款意图识别的说明
2025-06-19 16:56:38 +08:00
hrz 94553c54bd 更新智控台两款意图识别的说明 2025-06-19 16:53:39 +08:00
hrzandGitHub 248db31c8b Merge pull request #1618 from xinnan-tech/py_bug_fix
update:添加MCP重连机制
2025-06-18 22:40:25 +08:00
hrzandGitHub 24bfa1ca15 Merge pull request #1615 from xinnan-tech/py_fix_type
fix: 豆包流式decode错误
2025-06-18 17:46:47 +08:00
欣南科技andGitHub 1e97a8febc Merge pull request #1620 from xinnan-tech/hot-fix
update:修复mcp返回json的bug
2025-06-18 17:28:51 +08:00
hrz 2742f2e1ff update:修复mcp返回json的bug 2025-06-18 17:28:21 +08:00
Sakura-RanChen 0a5ae70a7c 二次错误提醒 2025-06-18 16:32:58 +08:00
CGD 22d53bd36e update:添加MCP重连机制 2025-06-18 16:29:28 +08:00
Sakura-RanChen ebf68929ce update: 错误信息读取 2025-06-18 14:03:18 +08:00
Sakura-RanChen ef3b373211 fix: 豆包流式decode错误 2025-06-18 11:06:11 +08:00
hrzandGitHub 5012a51e1d Merge pull request #1609 from xinnan-tech/py_fix_type
Py fix type
2025-06-18 09:39:38 +08:00
Sakura-RanChen fb1f476a3c fix: 智控台切换默认视觉模型报错 2025-06-18 09:24:52 +08:00
Sakura-RanChen 1a31c8cd1d update: 视觉模块直接识别输出,不经过二次LLM 2025-06-17 16:13:24 +08:00
欣南科技andGitHub 3e491c7d79 Merge pull request #1602 from xinnan-tech/py_fix_bug
update:更新版本号
2025-06-17 00:11:52 +08:00
hrz 33345817d2 update:更新版本号 2025-06-17 00:10:55 +08:00
hrzandGitHub 82973a685a Merge pull request #1601 from xinnan-tech/py_fix_bug
fix:修复“设备使用iot协议,意图识别无法正确调用”的问题
2025-06-16 23:49:58 +08:00
hrz e1d245068c update:优化权限 2025-06-16 23:47:26 +08:00
hrzandGitHub b74517ffb0 Merge pull request #1591 from CaixyPromise/fix/comment-invalid
fix: 智控台备注功能无效
2025-06-16 23:34:24 +08:00
hrz 231ae8dfa6 update:修复iot不更新的bug 2025-06-16 23:16:40 +08:00
CGD 4ee9a47d41 fix:修复“设备使用iot协议,意图识别无法正确调用”的问题 2025-06-16 18:09:13 +08:00
Sakura-RanChen d8bf5cdedf fix: openAi schema不支持list 2025-06-16 16:26:49 +08:00
Sakura-RanChenandGitHub 755e0edd44 Merge pull request #1596 from xinnan-tech/py_fix_bug
fix:修复意图识别因缺少 home_assistant 配置导致部分功能错误将意图识别成“home_assistant ”的bug
2025-06-16 14:52:53 +08:00
CGD 178df82693 fix:修复意图识别因缺少 home_assistant 配置导致功能全部失效的 bug 2025-06-16 14:48:52 +08:00
Sakura-RanChenandGitHub a88c3b2032 Merge pull request #1594 from xinnan-tech/py_fix_bug
fix:修复数据库中两个字符串的排序规则(collation)不一致问题
2025-06-16 11:10:18 +08:00
CGD 513396b905 fix:修复数据库中两个字符串的排序规则(collation)不一致问题 2025-06-16 11:09:01 +08:00
caixypromise 9885d4758b fix: 修复设备管理无法提交备注信息的问题。
chore: 将设备自动更新状态接口重构为设备信息更新接口,以此实现备注、自动更新合并在一个接口内
2025-06-16 03:27:12 +08:00
shane0411 0ca1a6286c Merge remote-tracking branch 'upstream/main' into add_local_tts_change_voice 2025-06-12 12:36:13 +08:00
shane0411 a4aea694d4 Merge remote-tracking branch 'upstream/main' into add_local_tts_change_voice 2025-06-11 14:29:35 +08:00
Liam Mazy 0f31bfc78f 增加timeout配置项,单位为秒 2025-06-11 14:05:54 +08:00
shane0411 052d8c8962 feat: 添加音频路径音频文本的提示 2025-06-09 19:36:57 +08:00
shane0411 2eb5539c9b feat: 新增音频路径音频文本欄位,提供給本地tts選取音色
https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/1503#issuecomment-2954952028
2025-06-09 19:18:58 +08:00
shane0411 be6fba5963 feat: 添加本地tts支援面板的"角色音色" 2025-06-09 15:20:27 +08:00
149 changed files with 8175 additions and 1498 deletions
+1
View File
@@ -174,3 +174,4 @@ main/xiaozhi-server/mysql
uploadfile
*.json
.vscode
.cursor
+28 -9
View File
@@ -6,7 +6,7 @@
本项目为开源智能硬件项目
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a>提供后端服务<br/>
根据<a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">小智通信协议</a>使用Python、Java、Vue实现<br/>
帮助您快速搭建小智服务器
支持MCP接入点和声纹识别
</p>
<p align="center">
@@ -144,8 +144,18 @@
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" />
</picture>
</a>
</td>
</tr>
</table>
@@ -170,8 +180,8 @@
#### 🚀 部署方式选择
| 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 |
|---------|------|---------|---------|---------|---------|
| **最简化安装** | 智能对话、IOT、MCP、视觉感知,数据存储在配置文件 | 低配置环境,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G,如果全API,要2核2G | - |
| **全模块安装** | 智能对话、IOT、MCP、视觉感知、OTA、智控台,数据存储在数据库 | 完整功能体验 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G,如果全API,要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **最简化安装** | 智能对话、IOT、MCP、视觉感知 | 低配置环境,数据存储在配置文件,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G,如果全API,要2核2G | - |
| **全模块安装** | 智能对话、IOT、MCP接入点、视觉感知、OTA、智控台 | 完整功能体验,数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G,如果全API,要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据
@@ -220,13 +230,14 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 功能模块 | 描述 |
|:---:|:---|
| 核心服务架构 | 基于WebSocket和HTTP服务器,提供完整的控制台管理和认证系统 |
| 语音交互系统 | 支持流式ASR(语音识别)、流式TTS(语音合成)、VAD(语音活动检测),支持多语言识别和语音处理 |
| 智能对话系统 | 支持多种LLM(大语言模型),实现智能对话 |
| 视觉感知系统 | 支持多种VLLM(视觉大模型),实现多模态交互 |
| 意图识别系统 | 支持LLM意图识别、Function Call函数调用,提供插件化意图处理机制 |
| 核心架构 | 基于WebSocket和HTTP服务器,提供完整的控制台管理和认证系统 |
| 语音交互 | 支持流式ASR(语音识别)、流式TTS(语音合成)、VAD(语音活动检测),支持多语言识别和语音处理 |
| 声纹识别 | 支持多用户声纹注册、管理和识别,与ASR并行处理,实时识别说话人身份并传递给LLM进行个性化回应 |
| 智能对话 | 支持多种LLM(大语言模型),实现智能对话 |
| 视觉感知 | 支持多种VLLM(视觉大模型),实现多模态交互 |
| 意图识别 | 支持LLM意图识别、Function Call函数调用,提供插件化意图处理机制 |
| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆,具备记忆总结功能 |
| IOT/MCP控制协议 | 支持设备注册管理、智能控制接口,同时支持IOT、MCP控制协议 |
| 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 管理后台 | 提供Web管理界面,支持用户管理、系统配置和设备管理 |
| 测试工具 | 提供性能测试工具、视觉模型测试工具和音频交互测试工具 |
| 部署支持 | 支持Docker部署和本地部署,提供完整的配置文件管理 |
@@ -303,6 +314,14 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
---
### Voiceprint 声纹识别
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| 本地使用 | 3D-Speaker | 3D-Speaker |
---
### Memory 记忆存储
| 类型 | 平台名称 | 使用方式 | 收费模式 | 备注 |
+11 -1
View File
@@ -112,7 +112,17 @@ VAD:
参考教程[视觉模型使用指南](./mcp-vision-integration.md)
### 10、更多问题,可联系我们反馈 💬
### 10、如何开启MCP接入点 🔧
1、先参考教程[MCP 接入点部署使用指南](./mcp-endpoint-enable.md)
2、再参考教程[MCP 接入点使用指南](./mcp-endpoint-integration.md)
### 12、如何开启声纹识别 🔊
参考教程[声纹识别启用指南](./voiceprint-integration.md)
### 13、更多问题,可联系我们反馈 💬
可以在[issues](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues)提交您的问题。
+13
View File
@@ -4,6 +4,8 @@
本项目的测试平台`https://2662r3426b.vicp.fun`,从开放以来就使用了该方法,效果良好。
教程可参考B站博主`毕乐labs`发布的视频教程:[《开源小智服务器xiaozhi-server自动更新以及最新版本MCP接入点配置保姆教程》](https://www.bilibili.com/video/BV15H37zHE7Q)
# 开始条件
- 你的电脑/服务器是linux操作系统
- 你已经跑通了整个流程
@@ -40,6 +42,9 @@ git clone https://ghproxy.net/https://github.com/xinnan-tech/xiaozhi-esp32-serve
此刻你需要把`model.pt`文件复制到新的目录去,你可以这样
```
# 创建需要的目录
mkdir -p /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/data/
cp 你原来的.config.yaml完整路径 /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/data/.config.yaml
cp 你原来的model.pt完整路径 /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/models/SenseVoiceSmall/model.pt
```
@@ -161,3 +166,11 @@ tail -f /home/system/xiaozhi/xiaozhi-esp32-server/main/xiaozhi-server/tmp/server
# 注意事项
测试平台`https://2662r3426b.vicp.fun`,是使用nginx做了反向代理。nginx.conf详细配置可以[参考这里](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)
## 常见问题
### 1、为什么没有见到8001端口?
回答:8001是开发环境使用的,用于运行前端的端口。如果你是服务器部署,不建议使用`npm run serve`启动8001端口运行前端,而是像本教程一样编译成html文件,然后使用nginx来管理访问。
### 2、每次更新需要更新手动SQL语句吗?
回答:不需要,因为项目使用**Liquibase**管理数据库版本,会自动执行新的sql脚本。
+3 -3
View File
@@ -42,9 +42,9 @@ http {
proxy_set_header Referer $http_referer;
proxy_set_header Cookie $http_cookie;
proxy_connect_timeout 10;
proxy_send_timeout 10;
proxy_read_timeout 10;
proxy_connect_timeout 15;
proxy_send_timeout 15;
proxy_read_timeout 15;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+1
View File
@@ -6,6 +6,7 @@ java -jar /app/xiaozhi-esp32-api.jar \
--spring.datasource.druid.username=${SPRING_DATASOURCE_DRUID_USERNAME} \
--spring.datasource.druid.password=${SPRING_DATASOURCE_DRUID_PASSWORD} \
--spring.data.redis.host=${SPRING_DATA_REDIS_HOST} \
--spring.data.redis.password=${SPRING_DATA_REDIS_PASSWORD} \
--spring.data.redis.port=${SPRING_DATA_REDIS_PORT} &
# 启动Nginx(前台运行保持容器存活)
Binary file not shown.

After

Width:  |  Height:  |  Size: 359 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 176 KiB

+125
View File
@@ -0,0 +1,125 @@
# MCP 接入点部署使用指南
本教程包含3个部分
- 1、如何部署MCP接入点这个服务
- 2、全模块部署时,怎么配置MCP接入点
- 3、单模块部署时,怎么配置MCP接入点
# 1、如何部署MCP接入点这个服务
## 第一步,下载mcp接入点项目源码
浏览器打开[mcp接入点项目地址](https://github.com/xinnan-tech/mcp-endpoint-server)
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-endpoint-server-main`
你需要把它重命名成`mcp-endpoint-server`
## 第二步,启动程序
这个项目是一个很简单的项目,建议使用docker运行。不过如果你不想使用docker运行,你可以参考[这个页面](https://github.com/xinnan-tech/mcp-endpoint-server/blob/main/README_dev.md)使用源码运行。以下是docker运行的方法
```
# 进入本项目源码根目录
cd mcp-endpoint-server
# 清除缓存
docker compose -f docker-compose.yml down
docker stop mcp-endpoint-server
docker rm mcp-endpoint-server
docker rmi ghcr.nju.edu.cn/xinnan-tech/mcp-endpoint-server:latest
# 启动docker容器
docker compose -f docker-compose.yml up -d
# 查看日志
docker logs -f mcp-endpoint-server
```
此时,日志里会输出类似以下的日志
```
250705 INFO-=====下面的地址分别是智控台/单模块MCP接入点地址====
250705 INFO-智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
250705 INFO-单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
250705 INFO-=====请根据具体部署选择使用,请勿泄露给任何人======
```
请你把两个接口地址复制出来:
由于你是docker部署,切不可直接使用上面的地址!
由于你是docker部署,切不可直接使用上面的地址!
由于你是docker部署,切不可直接使用上面的地址!
你先把地址复制出来,放在一个草稿里,你要知道你的电脑的局域网ip是什么,例如我的电脑局域网ip是`192.168.1.25`,那么
原来我的接口地址
```
智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
```
就要改成
```
智控台MCP参数配置: http://192.168.1.25:8004/mcp_endpoint/health?key=abc
单模块部署MCP接入点: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
```
改好后,请使用浏览器直接访问`智控台MCP参数配置`。当浏览器出现类似这样的代码,说明是成功了。
```
{"result":{"status":"success","connections":{"tool_connections":0,"robot_connections":0,"total_connections":0}},"error":null,"id":null,"jsonrpc":"2.0"}
```
请你保留好上面两个`接口地址`,下一步要用到。
# 2、全模块部署时,怎么配置MCP接入点
如果你是全模块部署,使用管理员账号,登录智控台,点击顶部`参数字典`,选择`参数管理`功能。
然后搜索参数`server.mcp_endpoint`,此时,它的值应该是`null`值。
点击修改按钮,把上一步得来的`智控台MCP参数配置`粘贴到`参数值`里。然后保存。
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问mcp接入点,很大概率是网络防火墙,或者没有填写正确的局域网ip。
# 3、单模块部署时,怎么配置MCP接入点
如果你是单模块部署,找到你的配置文件`data/.config.yaml`
在配置文件搜索`mcp_endpoint`,如果没有找到,你就增加`mcp_endpoint`配置。类似我是就是这样
```
server:
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
http_port: 8002
log:
log_level: INFO
# 此处可能还更多配置..
mcp_endpoint: 你的接入点websocket地址
```
这时,请你把`如何部署MCP接入点这个服务`中得到的`单模块部署MCP接入点` 粘贴到 `mcp_endpoint`中。类似这样
```
server:
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
http_port: 8002
log:
log_level: INFO
# 此处可能还更多配置
mcp_endpoint: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
```
配置好后,启动单模块会输出如下的日志。
```
250705[__main__]-INFO-初始化组件: vad成功 SileroVAD
250705[__main__]-INFO-初始化组件: asr成功 FunASRServer
250705[__main__]-INFO-OTA接口是 http://192.168.1.25:8002/xiaozhi/ota/
250705[__main__]-INFO-视觉分析接口是 http://192.168.1.25:8002/mcp/vision/explain
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
250705[__main__]-INFO-=============================================================
```
如上,如果能输出类似的`mcp接入点是``ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc`说明配置成功了。
+94
View File
@@ -0,0 +1,94 @@
# MCP 接入点使用指南
本教程以虾哥开源的mcp计算器功能为示例,介绍如何将自己自定义的mcp服务接入到自己的接入点里。
本教程的前提是,你的`xiaozhi-server`已经启用了mcp接入点功能,如果你还没启用,可以先根据[这个教程](./mcp-endpoint-enable.md)启用。
# 如何为智能体接入一个简单的mcp功能,如计算器功能
### 如果你是全模块部署
如果你是全模块部署,你可以进入智控台,智能体管理,点击`配置角色`,在`意图识别`的右边,有一个`编辑功能`的按钮。
点击这个按钮。在弹出的页面里,位于底部,会有`MCP接入点`,正常来说,会显示这个智能体的`MCP接入点地址`,接下来,我们来给这个智能体扩展一个基于MCP技术的计算器的功能。
这个`MCP接入点地址`很重要,你等一下会用到。
### 如果你是单模块部署
如果你是单模块部署,且你已经在配置文件里配置了MCP接入点地址,那么正常来说,单模块部署启动的时候,会输出如下的日志。
```
250705[__main__]-INFO-初始化组件: vad成功 SileroVAD
250705[__main__]-INFO-初始化组件: asr成功 FunASRServer
250705[__main__]-INFO-OTA接口是 http://192.168.1.25:8002/xiaozhi/ota/
250705[__main__]-INFO-视觉分析接口是 http://192.168.1.25:8002/mcp/vision/explain
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
250705[__main__]-INFO-=============================================================
```
如上,输出`mcp接入点是``ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc`就是你的`MCP接入点地址`
这个`MCP接入点地址`很重要,你等一下会用到。
## 第一步 下载虾哥MCP计算器项目代码
浏览器打开虾哥写的[计算器项目](https://github.com/78/mcp-calculator)
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-calculatorr-main`
你需要把它重命名成`mcp-calculator`。接下来,我们用命令行进入项目目录即安装依赖
```bash
# 进入项目目录
cd mcp-calculator
conda remove -n mcp-calculator --all -y
conda create -n mcp-calculator python=3.10 -y
conda activate mcp-calculator
pip install -r requirements.txt
```
## 第二步 启动
启动前,先从你的智控台的智能体里,复制到了MCP接入点的地址。
例如我的智能体的mcp地址是
```
ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
```
开始输入命令
```bash
export MCP_ENDPOINT=ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
```
输入完后,启动程序
```bash
python mcp_pipe.py calculator.py
```
### 如果你是智控台部署
如果你是智控台部署,启动完后,你再进入智控台,点击刷新MCP的接入状态,就会看到你扩展的功能列表了。
### 如果你是单模块部署
如果你是单模块部署,当设备连接后,会输出类似的日志,说明成功了
```
250705 -INFO-正在初始化MCP接入点: wss://2662r3426b.vicp.fun/mcp_e
250705 -INFO-发送MCP接入点初始化消息
250705 -INFO-MCP接入点连接成功
250705 -INFO-MCP接入点初始化成功
250705 -INFO-统一工具处理器初始化完成
250705 -INFO-MCP接入点服务器信息: name=Calculator, version=1.9.4
250705 -INFO-MCP接入点支持的工具数量: 1
250705 -INFO-所有MCP接入点工具已获取,客户端准备就绪
250705 -INFO-工具缓存已刷新
250705 -INFO-当前支持的函数列表: [ 'get_time', 'get_lunar', 'play_music', 'get_weather', 'handle_exit_intent', 'calculator']
```
如果包含了 `'calculator'`,说明设备将可以根据意图识别,调用计算器这个工具。
+105
View File
@@ -0,0 +1,105 @@
# get_news_from_newsnow 插件新闻源配置指南
## 概述
`get_news_from_newsnow` 插件现在支持通过Web管理界面动态配置新闻源,不再需要修改代码。用户可以在智控台中为每个智能体配置不同的新闻源。
## 配置方式
### 1. 通过Web管理界面配置(推荐)
1. 登录智控台
2. 进入"角色配置"页面
3. 选择要配置的智能体
4. 点击"编辑功能"按钮
5. 在右侧参数配置区域找到"newsnow新闻聚合"插件
6. 在"新闻源配置"字段中输入分号分隔的中文名称
### 2. 配置文件方式
`config.yaml` 中配置:
```yaml
plugins:
get_news_from_newsnow:
url: "https://newsnow.busiyi.world/api/s?id="
news_sources: "澎湃新闻;百度热搜;财联社;微博;抖音"
```
## 新闻源配置格式
新闻源配置使用分号分隔的中文名称,格式为:
```
中文名称1;中文名称2;中文名称3
```
### 配置示例
```
澎湃新闻;百度热搜;财联社;微博;抖音;知乎;36氪
```
## 支持的新闻源
插件支持以下新闻源的中文名称:
- 澎湃新闻
- 百度热搜
- 财联社
- 微博
- 抖音
- 知乎
- 36氪
- 华尔街见闻
- IT之家
- 今日头条
- 虎扑
- 哔哩哔哩
- 快手
- 雪球
- 格隆汇
- 法布财经
- 金十数据
- 牛客
- 少数派
- 稀土掘金
- 凤凰网
- 虫部落
- 联合早报
- 酷安
- 远景论坛
- 参考消息
- 卫星通讯社
- 百度贴吧
- 靠谱新闻
- 以及更多...
## 默认配置
如果未配置新闻源,插件将使用以下默认配置:
```
澎湃新闻;百度热搜;财联社
```
## 使用说明
1. **配置新闻源**:在Web界面或配置文件中设置新闻源的中文名称,用分号分隔
2. **调用插件**:用户可以说"播报新闻"或"获取新闻"
3. **指定新闻源**:用户可以说"播报澎湃新闻"或"获取百度热搜"
4. **获取详情**:用户可以说"详细介绍这条新闻"
## 工作原理
1. 插件接受中文名称作为参数(如"澎湃新闻")
2. 根据配置的新闻源列表,将中文名称转换为对应的英文ID(如"thepaper"
3. 使用英文ID调用API获取新闻数据
4. 返回新闻内容给用户
## 注意事项
1. 配置的中文名称必须与 CHANNEL_MAP 中定义的名称完全一致
2. 配置更改后需要重启服务或重新加载配置
3. 如果配置的新闻源无效,插件会自动使用默认新闻源
4. 多个新闻源之间使用英文分号(;)分隔,不要使用中文分号(;)
+96
View File
@@ -0,0 +1,96 @@
# 声纹识别启用指南
本教程包含2个部分
- 1、如何部署声纹识别这个服务
- 2、全模块部署时,怎么配置声纹识别接口
# 1、如何部署声纹识别这个服务
## 第一步,下载声纹识别项目源码
浏览器打开[声纹识别项目地址](https://github.com/xinnan-tech/voiceprint-api)
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`voiceprint-api-main`
你需要把它重命名成`voiceprint-api`
## 第二步,启动程序
这个项目是一个很简单的项目,建议使用docker运行。不过如果你不想使用docker运行,你可以参考[这个页面](https://github.com/xinnan-tech/voiceprint-api/blob/main/README.md)使用源码运行。以下是docker运行的方法
```
# 进入本项目源码根目录
cd voiceprint-api
# 清除缓存
docker compose -f docker-compose.yml down
docker stop voiceprint-api
docker rm voiceprint-api
docker rmi ghcr.nju.edu.cn/xinnan-tech/voiceprint-api:latest
# 启动docker容器
docker compose -f docker-compose.yml up -d
# 查看日志
docker logs -f voiceprint-api
```
此时,日志里会输出类似以下的日志
```
250711 INFO-🚀 开始: 生产环境服务启动(Uvicorn),监听地址: 0.0.0.0:8005
250711 INFO-============================================================
250711 INFO-声纹接口地址: http://127.0.0.1:8005/voiceprint/health?key=abcd
250711 INFO-============================================================
```
请你把声纹接口地址复制出来:
由于你是docker部署,切不可直接使用上面的地址!
由于你是docker部署,切不可直接使用上面的地址!
由于你是docker部署,切不可直接使用上面的地址!
你先把地址复制出来,放在一个草稿里,你要知道你的电脑的局域网ip是什么,例如我的电脑局域网ip是`192.168.1.25`,那么
原来我的接口地址
```
http://127.0.0.1:8005/voiceprint/health?key=abcd
```
就要改成
```
http://192.168.1.25:8005/voiceprint/health?key=abcd
```
改好后,请使用浏览器直接访问`声纹接口地址`。当浏览器出现类似这样的代码,说明是成功了。
```
{"total_voiceprints":0,"status":"healthy"}
```
请你保留好修改后的`声纹接口地址`,下一步要用到。
# 2、全模块部署时,怎么配置声纹识别
## 第一步 配置接口
如果你是全模块部署,使用管理员账号,登录智控台,点击顶部`参数字典`,选择`参数管理`功能。
然后搜索参数`server.voice_print`,此时,它的值应该是`null`值。
点击修改按钮,把上一步得来的`声纹接口地址`粘贴到`参数值`里。然后保存。
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问声纹识别,很大概率是网络防火墙,或者没有填写正确的局域网ip。
## 第二步 设置智能体记忆模式
进入你的智能体的角色配置里,将记忆设置成`本地短期记忆`,一定要开启`上报文字+语音`
## 第三步 和你的智能体聊天
将你的设备通电,然后和他用正常的语速和音调聊天。
## 第四步 设置声纹
在智控台,`智能体管理`页面,在智能体的面板里,有一个`声纹识别`按钮,点击它。在底部有一个`新增按钮`。就可以对某个人说的话进行声纹注册。
在弹出的框里,`描述`这个属性建议填写上,可以是这个人的职业、性格、爱好。方便智能体对说话人进行分析和了解。
## 第三步 和你的智能体聊天
将你的设备通电,问它,你知道我是谁吗?如果他能回答得出,说明声纹识别功能正常。
@@ -111,6 +111,16 @@ public interface Constant {
*/
String FILE_EXTENSION_SEG = ".";
/**
* mcp接入点路径
*/
String SERVER_MCP_ENDPOINT = "server.mcp_endpoint";
/**
* mcp接入点路径
*/
String SERVER_VOICE_PRINT = "server.voice_print";
/**
* 无记忆
*/
@@ -227,7 +237,7 @@ public interface Constant {
/**
* 版本号
*/
public static final String VERSION = "0.5.6";
public static final String VERSION = "0.7.1";
/**
* 无效固件URL
@@ -0,0 +1,78 @@
package xiaozhi.common.utils;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import javax.crypto.Cipher;
import javax.crypto.spec.SecretKeySpec;
public class AESUtils {
private static final String ALGORITHM = "AES";
private static final String TRANSFORMATION = "AES/ECB/PKCS5Padding";
/**
* AES加密
*
* @param key 密钥(16位、24位或32位)
* @param plainText 待加密字符串
* @return 加密后的Base64字符串
*/
public static String encrypt(String key, String plainText) {
try {
// 确保密钥长度为16、24或32位
byte[] keyBytes = padKey(key.getBytes(StandardCharsets.UTF_8));
SecretKeySpec secretKey = new SecretKeySpec(keyBytes, ALGORITHM);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
byte[] encryptedBytes = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(encryptedBytes);
} catch (Exception e) {
throw new RuntimeException("AES加密失败", e);
}
}
/**
* AES解密
*
* @param key 密钥(16位、24位或32位)
* @param encryptedText 待解密的Base64字符串
* @return 解密后的字符串
*/
public static String decrypt(String key, String encryptedText) {
try {
// 确保密钥长度为16、24或32位
byte[] keyBytes = padKey(key.getBytes(StandardCharsets.UTF_8));
SecretKeySpec secretKey = new SecretKeySpec(keyBytes, ALGORITHM);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.DECRYPT_MODE, secretKey);
byte[] encryptedBytes = Base64.getDecoder().decode(encryptedText);
byte[] decryptedBytes = cipher.doFinal(encryptedBytes);
return new String(decryptedBytes, StandardCharsets.UTF_8);
} catch (Exception e) {
throw new RuntimeException("AES解密失败", e);
}
}
/**
* 填充密钥到指定长度(16、24或32位)
*
* @param keyBytes 原始密钥字节数组
* @return 填充后的密钥字节数组
*/
private static byte[] padKey(byte[] keyBytes) {
int keyLength = keyBytes.length;
if (keyLength == 16 || keyLength == 24 || keyLength == 32) {
return keyBytes;
}
// 如果密钥长度不足,用0填充;如果超过,截取前32位
byte[] paddedKey = new byte[32];
System.arraycopy(keyBytes, 0, paddedKey, 0, Math.min(keyLength, 32));
return paddedKey;
}
}
@@ -0,0 +1,52 @@
package xiaozhi.common.utils;
import lombok.extern.slf4j.Slf4j;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
/**
* 哈希加密算法的工具类
* @author zjy
*/
@Slf4j
public class HashEncryptionUtil {
/**
* 使用md5进行加密
* @param context 被加密的内容
* @return 哈希值
*/
public static String Md5hexDigest(String context){
return hexDigest(context,"MD5");
}
/**
* 指定哈希算法进行加密
* @param context 被加密的内容
* @param algorithm 哈希算法
* @return 哈希值
*/
public static String hexDigest(String context,String algorithm ){
// 获取MD5算法实例
MessageDigest md = null;
try {
md = MessageDigest.getInstance(algorithm);
} catch (NoSuchAlgorithmException e) {
log.error("加密失败的算法:{}",algorithm);
throw new RuntimeException("加密失败,"+ algorithm +"哈希算法系统不支持");
}
// 计算智能体id的MD5值
byte[] messageDigest = md.digest(context.getBytes());
// 将字节数组转换为十六进制字符串
StringBuilder hexString = new StringBuilder();
for (byte b : messageDigest) {
String hex = Integer.toHexString(0xFF & b);
if (hex.length() == 1) {
hexString.append('0');
}
hexString.append(hex);
}
return hexString.toString();
}
}
@@ -0,0 +1,21 @@
package xiaozhi.modules.agent.Enums;
import lombok.Getter;
/**
* 智能体聊天记录类型
*/
@Getter
public enum AgentChatHistoryType {
USER((byte) 1),
AGENT((byte) 2);
private final byte value;
AgentChatHistoryType(byte i) {
this.value = i;
}
}
@@ -47,6 +47,7 @@ import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.agent.vo.AgentChatHistoryUserVO;
import xiaozhi.modules.agent.vo.AgentInfoVO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
@@ -181,6 +182,33 @@ public class AgentController {
List<AgentChatHistoryDTO> result = agentChatHistoryService.getChatHistoryBySessionId(id, sessionId);
return new Result<List<AgentChatHistoryDTO>>().ok(result);
}
@GetMapping("/{id}/chat-history/user")
@Operation(summary = "获取智能体聊天记录(用户)")
@RequiresPermissions("sys:role:normal")
public Result<List<AgentChatHistoryUserVO>> getRecentlyFiftyByAgentId(
@PathVariable("id") String id) {
// 获取当前用户
UserDetail user = SecurityUser.getUser();
// 检查权限
if (!agentService.checkAgentPermission(id, user.getId())) {
return new Result<List<AgentChatHistoryUserVO>>().error("没有权限查看该智能体的聊天记录");
}
// 查询聊天记录
List<AgentChatHistoryUserVO> data = agentChatHistoryService.getRecentlyFiftyByAgentId(id);
return new Result<List<AgentChatHistoryUserVO>>().ok(data);
}
@GetMapping("/{id}/chat-history/audio")
@Operation(summary = "获取音频内容")
@RequiresPermissions("sys:role:normal")
public Result<String> getContentByAudioId(
@PathVariable("id") String id) {
// 查询聊天记录
String data = agentChatHistoryService.getContentByAudioId(id);
return new Result<String>().ok(data);
}
@PostMapping("/audio/{audioId}")
@Operation(summary = "获取音频下载ID")
@@ -0,0 +1,66 @@
package xiaozhi.modules.agent.controller;
import java.util.List;
import org.apache.shiro.authz.annotation.RequiresPermissions;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.security.user.SecurityUser;
@Tag(name = "智能体Mcp接入点管理")
@RequiredArgsConstructor
@RestController
@RequestMapping("/agent/mcp")
public class AgentMcpAccessPointController {
private final AgentMcpAccessPointService agentMcpAccessPointService;
private final AgentService agentService;
/**
* 获取智能体的Mcp接入点地址
*
* @param audioId 智能体id
* @return 返回错误提醒或者Mcp接入点地址
*/
@Operation(summary = "获取智能体的Mcp接入点地址")
@GetMapping("/address/{agentId}")
@RequiresPermissions("sys:role:normal")
public Result<String> getAgentMcpAccessAddress(@PathVariable("agentId") String agentId) {
// 获取当前用户
UserDetail user = SecurityUser.getUser();
// 检查权限
if (!agentService.checkAgentPermission(agentId, user.getId())) {
return new Result<String>().error("没有权限查看该智能体的MCP接入点地址");
}
String agentMcpAccessAddress = agentMcpAccessPointService.getAgentMcpAccessAddress(agentId);
if (agentMcpAccessAddress == null) {
return new Result<String>().ok("请联系管理员进入参数管理配置mcp接入点地址");
}
return new Result<String>().ok(agentMcpAccessAddress);
}
@Operation(summary = "获取智能体的Mcp工具列表")
@GetMapping("/tools/{agentId}")
@RequiresPermissions("sys:role:normal")
public Result<List<String>> getAgentMcpToolsList(@PathVariable("agentId") String agentId) {
// 获取当前用户
UserDetail user = SecurityUser.getUser();
// 检查权限
if (!agentService.checkAgentPermission(agentId, user.getId())) {
return new Result<List<String>>().error("没有权限查看该智能体的MCP工具列表");
}
List<String> agentMcpToolsList = agentMcpAccessPointService.getAgentMcpToolsList(agentId);
return new Result<List<String>>().ok(agentMcpToolsList);
}
}
@@ -0,0 +1,86 @@
package xiaozhi.modules.agent.controller;
import java.util.List;
import org.apache.commons.lang3.StringUtils;
import org.apache.shiro.authz.annotation.RequiresPermissions;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import lombok.AllArgsConstructor;
import xiaozhi.common.exception.RenException;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.agent.dto.AgentVoicePrintSaveDTO;
import xiaozhi.modules.agent.dto.AgentVoicePrintUpdateDTO;
import xiaozhi.modules.agent.service.AgentVoicePrintService;
import xiaozhi.modules.agent.vo.AgentVoicePrintVO;
import xiaozhi.modules.security.user.SecurityUser;
import xiaozhi.modules.sys.service.SysParamsService;
@Tag(name = "智能体声纹管理")
@AllArgsConstructor
@RestController
@RequestMapping("/agent/voice-print")
public class AgentVoicePrintController {
private final AgentVoicePrintService agentVoicePrintService;
private final SysParamsService sysParamsService;
@PostMapping
@Operation(summary = "创建智能体的声纹")
@RequiresPermissions("sys:role:normal")
public Result<Void> save(@RequestBody @Valid AgentVoicePrintSaveDTO dto) {
boolean b = agentVoicePrintService.insert(dto);
if (b) {
return new Result<>();
}
return new Result<Void>().error("智能体的声纹创建失败");
}
@PutMapping
@Operation(summary = "更新智能体的对应声纹")
@RequiresPermissions("sys:role:normal")
public Result<Void> update(@RequestBody @Valid AgentVoicePrintUpdateDTO dto) {
Long userId = SecurityUser.getUserId();
boolean b = agentVoicePrintService.update(userId, dto);
if (b) {
return new Result<>();
}
return new Result<Void>().error("智能体的对应声纹更新失败");
}
@DeleteMapping("/{id}")
@Operation(summary = "删除智能体对应声纹")
@RequiresPermissions("sys:role:normal")
public Result<Void> delete(@PathVariable String id) {
Long userId = SecurityUser.getUserId();
// 先删除关联的设备
boolean delete = agentVoicePrintService.delete(userId, id);
if (delete) {
return new Result<>();
}
return new Result<Void>().error("智能体的对应声纹删除失败");
}
@GetMapping("/list/{id}")
@Operation(summary = "获取用户指定智能体声纹列表")
@RequiresPermissions("sys:role:normal")
public Result<List<AgentVoicePrintVO>> list(@PathVariable String id) {
String voiceprintUrl = sysParamsService.getValue("server.voice_print", true);
if (StringUtils.isBlank(voiceprintUrl) || "null".equals(voiceprintUrl)) {
throw new RenException("声纹接口未配置,请先在参数配置中配置声纹接口地址(server.voice_print)");
}
Long userId = SecurityUser.getUserId();
List<AgentVoicePrintVO> list = agentVoicePrintService.list(userId, id);
return new Result<List<AgentVoicePrintVO>>().ok(list);
}
}
@@ -0,0 +1,20 @@
package xiaozhi.modules.agent.dao;
import org.apache.ibatis.annotations.Mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
import xiaozhi.modules.agent.entity.AgentVoicePrintEntity;
/**
* {@link AgentChatHistoryEntity} 智能体聊天历史记录Dao对象
*
* @author Goody
* @version 1.0, 2025/4/30
* @since 1.0.0
*/
@Mapper
public interface AgentVoicePrintDao extends BaseMapper<AgentVoicePrintEntity> {
}
@@ -0,0 +1,28 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* 保存智能体声纹的dto
*
* @author zjy
*/
@Data
public class AgentVoicePrintSaveDTO {
/**
* 关联的智能体id
*/
private String agentId;
/**
* 音频文件id
*/
private String audioId;
/**
* 声纹来源的人姓名
*/
private String sourceName;
/**
* 描述声纹来源的人
*/
private String introduce;
}
@@ -0,0 +1,28 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* 修改智能体声纹的dto
*
* @author zjy
*/
@Data
public class AgentVoicePrintUpdateDTO {
/**
* 智能体声纹id
*/
private String id;
/**
* 音频文件id
*/
private String audioId;
/**
* 声纹来源的人姓名
*/
private String sourceName;
/**
* 描述声纹来源的人
*/
private String introduce;
}
@@ -0,0 +1,21 @@
package xiaozhi.modules.agent.dto;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data;
/**
* 声纹识别接口返回的对象
*/
@Data
public class IdentifyVoicePrintResponse {
/**
* 最匹配的声纹id
*/
@JsonProperty("speaker_id")
private String speakerId;
/**
* 声纹的分数
*/
private Double score;
}
@@ -0,0 +1,32 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* MCP JSON-RPC 请求 DTO
*/
@Data
public class McpJsonRpcRequest {
private String jsonrpc = "2.0";
private String method;
private Object params;
private Integer id;
public McpJsonRpcRequest() {
}
public McpJsonRpcRequest(String method) {
this.method = method;
}
public McpJsonRpcRequest(String method, Object params, Integer id) {
this.method = method;
this.params = params;
this.id = id;
}
public McpJsonRpcRequest(String method, Object params) {
this.method = method;
this.params = params;
}
}
@@ -0,0 +1,48 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* MCP JSON-RPC 响应 DTO
*/
@Data
public class McpJsonRpcResponse {
private String jsonrpc = "2.0";
private Integer id;
private McpResult result;
private McpError error;
public McpJsonRpcResponse() {
}
@Data
public static class McpResult {
private String type;
private String message;
private String agent_id;
private McpTool[] tools;
public McpResult() {
}
}
@Data
public static class McpTool {
private String name;
private String description;
private Object inputSchema;
public McpTool() {
}
}
@Data
public static class McpError {
private Integer code;
private String message;
private Object data;
public McpError() {
}
}
}
@@ -0,0 +1,64 @@
package xiaozhi.modules.agent.entity;
import java.util.Date;
import com.baomidou.mybatisplus.annotation.FieldFill;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
/**
* 智能体声纹表
*
* @author zjy
*/
@TableName(value = "ai_agent_voice_print")
@Data
public class AgentVoicePrintEntity {
/**
* 主键id
*/
@TableId(type = IdType.ASSIGN_UUID)
private String id;
/**
* 关联的智能体id
*/
private String agentId;
/**
* 关联的音频id
*/
private String audioId;
/**
* 声纹来源的人姓名
*/
private String sourceName;
/**
* 描述声纹来源的人
*/
private String introduce;
/**
* 创建者
*/
@TableField(fill = FieldFill.INSERT)
private Long creator;
/**
* 创建时间
*/
@TableField(fill = FieldFill.INSERT)
private Date createDate;
/**
* 更新者
*/
@TableField(fill = FieldFill.INSERT_UPDATE)
private Long updater;
/**
* 更新时间
*/
@TableField(fill = FieldFill.INSERT_UPDATE)
private Date updateDate;
}
@@ -9,6 +9,7 @@ import xiaozhi.common.page.PageData;
import xiaozhi.modules.agent.dto.AgentChatHistoryDTO;
import xiaozhi.modules.agent.dto.AgentChatSessionDTO;
import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
import xiaozhi.modules.agent.vo.AgentChatHistoryUserVO;
/**
* 智能体聊天记录表处理service
@@ -44,4 +45,30 @@ public interface AgentChatHistoryService extends IService<AgentChatHistoryEntity
* @param deleteText 是否删除文本
*/
void deleteByAgentId(String agentId, Boolean deleteAudio, Boolean deleteText);
/**
* 根据智能体ID获取最近50条用户的聊天记录数据(带音频数据)
*
* @param agentId 智能体id
* @return 聊天记录列表(只有用户)
*/
List<AgentChatHistoryUserVO> getRecentlyFiftyByAgentId(String agentId);
/**
* 根据音频数据ID获取聊天内容
*
* @param audioId 音频id
* @return 聊天内容
*/
String getContentByAudioId(String audioId);
/**
* 查询此音频id是否属于此智能体
*
* @param audioId 音频id
* @param agentId 音频id
* @return T:属于 F:不属于
*/
boolean isAudioOwnedByAgent(String audioId,String agentId);
}
@@ -0,0 +1,25 @@
package xiaozhi.modules.agent.service;
import java.util.List;
/**
* 智能体Mcp接入点处理service
*
* @author zjy
*/
public interface AgentMcpAccessPointService {
/**
* 获取智能体的mcp接入点地址
* @param id 智能体id
* @return mcp接入点地址
*/
String getAgentMcpAccessAddress(String id);
/**
* 获取智能体的mcp接入点已有的工具列表
* @param id 智能体id
* @return 工具列表
*/
List<String> getAgentMcpToolsList(String id);
}
@@ -0,0 +1,50 @@
package xiaozhi.modules.agent.service;
import java.util.List;
import xiaozhi.modules.agent.dto.AgentVoicePrintSaveDTO;
import xiaozhi.modules.agent.dto.AgentVoicePrintUpdateDTO;
import xiaozhi.modules.agent.vo.AgentVoicePrintVO;
/**
* 智能体声纹处理service
*
* @author zjy
*/
public interface AgentVoicePrintService {
/**
* 添加智能体新的声纹
*
* @param dto 保存智能体声纹的数据
* @return T:成功 F:失败
*/
boolean insert(AgentVoicePrintSaveDTO dto);
/**
* 删除智能体的指的声纹
*
* @param userId 当前登录的用户id
* @param voicePrintId 声纹id
* @return 是否成功 T:成功 F:失败
*/
boolean delete(Long userId, String voicePrintId);
/**
* 获取指定智能体的所有声纹数据
*
* @param userId 当前登录的用户id
* @param agentId 智能体id
* @return 声纹数据集合
*/
List<AgentVoicePrintVO> list(Long userId, String agentId);
/**
* 更新智能体的指的声纹数据
*
* @param userId 当前登录的用户id
* @param dto 修改的声纹的数据
* @return 是否成功 T:成功 F:失败
*/
boolean update(Long userId, AgentVoicePrintUpdateDTO dto);
}
@@ -19,6 +19,8 @@ import xiaozhi.modules.agent.service.AgentChatAudioService;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.biz.AgentChatHistoryBizService;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
/**
* {@link AgentChatHistoryBizService} impl
@@ -35,6 +37,7 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
private final AgentChatHistoryService agentChatHistoryService;
private final AgentChatAudioService agentChatAudioService;
private final RedisUtils redisUtils;
private final DeviceService deviceService;
/**
* 处理聊天记录上报,包括文件上传和相关信息记录
@@ -68,6 +71,15 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
// 更新设备最后对话时间
redisUtils.set(RedisKeys.getAgentDeviceLastConnectedAtById(agentId), new Date());
// 更新设备最后连接时间
DeviceEntity device = deviceService.getDeviceByMacAddress(macAddress);
if (device != null) {
deviceService.updateDeviceConnectionInfo(agentId, device.getId(), null);
} else {
log.warn("聊天记录上报时,未找到mac地址为 {} 的设备", macAddress);
}
return Boolean.TRUE;
}
@@ -8,6 +8,7 @@ import java.util.stream.Collectors;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
@@ -16,11 +17,14 @@ import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.page.PageData;
import xiaozhi.common.utils.ConvertUtils;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.Enums.AgentChatHistoryType;
import xiaozhi.modules.agent.dao.AiAgentChatHistoryDao;
import xiaozhi.modules.agent.dto.AgentChatHistoryDTO;
import xiaozhi.modules.agent.dto.AgentChatSessionDTO;
import xiaozhi.modules.agent.entity.AgentChatHistoryEntity;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.vo.AgentChatHistoryUserVO;
/**
* 智能体聊天记录表处理service {@link AgentChatHistoryService} impl
@@ -90,4 +94,74 @@ public class AgentChatHistoryServiceImpl extends ServiceImpl<AiAgentChatHistoryD
}
}
@Override
public List<AgentChatHistoryUserVO> getRecentlyFiftyByAgentId(String agentId) {
// 构建查询条件(不添加按照创建时间排序,数据本来就是主键越大创建时间越大
// 不添加这样可以减少排序全部数据在分页的全盘扫描消耗)
LambdaQueryWrapper<AgentChatHistoryEntity> wrapper = new LambdaQueryWrapper<>();
wrapper.select(AgentChatHistoryEntity::getContent, AgentChatHistoryEntity::getAudioId)
.eq(AgentChatHistoryEntity::getAgentId, agentId)
.eq(AgentChatHistoryEntity::getChatType, AgentChatHistoryType.USER.getValue())
.isNotNull(AgentChatHistoryEntity::getAudioId);
// 构建分页查询,查询前50页数据
Page<AgentChatHistoryEntity> pageParam = new Page<>(0, 50);
IPage<AgentChatHistoryEntity> result = this.baseMapper.selectPage(pageParam, wrapper);
return result.getRecords().stream().map(item -> {
AgentChatHistoryUserVO vo = ConvertUtils.sourceToTarget(item, AgentChatHistoryUserVO.class);
// 处理 content 字段,确保只返回聊天内容
if (vo != null && vo.getContent() != null) {
vo.setContent(extractContentFromString(vo.getContent()));
}
return vo;
}).toList();
}
/**
* 从 content 字段中提取聊天内容
* 如果 content 是 JSON 格式(如 {"speaker": "未知说话人", "content": "现在几点了。"}),则提取 content
* 字段
* 如果 content 是普通字符串,则直接返回
*
* @param content 原始内容
* @return 提取的聊天内容
*/
private String extractContentFromString(String content) {
if (content == null || content.trim().isEmpty()) {
return content;
}
// 尝试解析为 JSON
try {
Map<String, Object> jsonMap = JsonUtils.parseObject(content, Map.class);
if (jsonMap != null && jsonMap.containsKey("content")) {
Object contentObj = jsonMap.get("content");
return contentObj != null ? contentObj.toString() : content;
}
} catch (Exception e) {
// 如果不是有效的 JSON,直接返回原内容
}
// 如果不是 JSON 格式或没有 content 字段,直接返回原内容
return content;
}
@Override
public String getContentByAudioId(String audioId) {
AgentChatHistoryEntity agentChatHistoryEntity = baseMapper
.selectOne(new LambdaQueryWrapper<AgentChatHistoryEntity>()
.select(AgentChatHistoryEntity::getContent)
.eq(AgentChatHistoryEntity::getAudioId, audioId));
return agentChatHistoryEntity == null ? null : agentChatHistoryEntity.getContent();
}
@Override
public boolean isAudioOwnedByAgent(String audioId, String agentId) {
// 查询是否有指定音频id和智能体id的数据,如果有且只有一条说明此数据属性此智能体
Long row = baseMapper.selectCount(new LambdaQueryWrapper<AgentChatHistoryEntity>()
.eq(AgentChatHistoryEntity::getAudioId, audioId)
.eq(AgentChatHistoryEntity::getAgentId, agentId));
return row == 1;
}
}
@@ -0,0 +1,246 @@
package xiaozhi.modules.agent.service.impl;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.utils.AESUtils;
import xiaozhi.common.utils.HashEncryptionUtil;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.dto.McpJsonRpcRequest;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.sys.service.SysParamsService;
import xiaozhi.modules.sys.utils.WebSocketClientManager;
@AllArgsConstructor
@Service
@Slf4j
public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointService {
private SysParamsService sysParamsService;
@Override
public String getAgentMcpAccessAddress(String id) {
// 获取到mcp的地址
String url = sysParamsService.getValue(Constant.SERVER_MCP_ENDPOINT, true);
if (StringUtils.isBlank(url) || "null".equals(url)) {
return null;
}
URI uri = getURI(url);
// 获取智能体mcp的url前缀
String agentMcpUrl = getAgentMcpUrl(uri);
// 获取密钥
String key = getSecretKey(uri);
// 获取加密的token
String encryptToken = encryptToken(id, key);
// 对token进行URL编码
String encodedToken = URLEncoder.encode(encryptToken, StandardCharsets.UTF_8);
// 返回智能体Mcp路径的格式
agentMcpUrl = "%s/mcp/?token=%s".formatted(agentMcpUrl, encodedToken);
return agentMcpUrl;
}
@Override
public List<String> getAgentMcpToolsList(String id) {
String wsUrl = getAgentMcpAccessAddress(id);
if (StringUtils.isBlank(wsUrl)) {
return List.of();
}
// 将 /mcp 替换为 /call
wsUrl = wsUrl.replace("/mcp/", "/call/");
try {
// 创建 WebSocket 连接,增加超时时间到15秒
try (WebSocketClientManager client = WebSocketClientManager.build(
new WebSocketClientManager.Builder()
.uri(wsUrl)
.connectTimeout(8, TimeUnit.SECONDS)
.maxSessionDuration(10, TimeUnit.SECONDS))) {
// 步骤1: 发送初始化消息并等待响应
log.info("发送MCP初始化消息,智能体ID: {}", id);
McpJsonRpcRequest initializeRequest = new McpJsonRpcRequest("initialize",
Map.of(
"protocolVersion", "2024-11-05",
"capabilities", Map.of(
"roots", Map.of("listChanged", false),
"sampling", Map.of()),
"clientInfo", Map.of(
"name", "xz-mcp-broker",
"version", "0.0.1")),
1);
client.sendJson(initializeRequest);
// 等待初始化响应 (id=1) - 移除固定延迟,改为响应驱动
List<String> initResponses = client.listenerWithoutClose(response -> {
try {
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
if (jsonMap != null && Integer.valueOf(1).equals(jsonMap.get("id"))) {
// 检查是否有result字段,表示初始化成功
return jsonMap.containsKey("result") && !jsonMap.containsKey("error");
}
return false;
} catch (Exception e) {
log.warn("解析初始化响应失败: {}", response, e);
return false;
}
});
// 验证初始化响应
boolean initSucceeded = false;
for (String response : initResponses) {
try {
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
if (jsonMap != null && Integer.valueOf(1).equals(jsonMap.get("id"))) {
if (jsonMap.containsKey("result")) {
log.info("MCP初始化成功,智能体ID: {}", id);
initSucceeded = true;
break;
} else if (jsonMap.containsKey("error")) {
log.error("MCP初始化失败,智能体ID: {}, 错误: {}", id, jsonMap.get("error"));
return List.of();
}
}
} catch (Exception e) {
log.warn("处理初始化响应失败: {}", response, e);
}
}
if (!initSucceeded) {
log.error("未收到有效的MCP初始化响应,智能体ID: {}", id);
return List.of();
}
// 步骤2: 发送初始化完成通知 - 只有在收到initialize响应后才发送
log.info("发送MCP初始化完成通知,智能体ID: {}", id);
String notificationJson = "{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}";
client.sendText(notificationJson);
// 步骤3: 发送工具列表请求 - 立即发送,无需额外延迟
log.info("发送MCP工具列表请求,智能体ID: {}", id);
McpJsonRpcRequest toolsRequest = new McpJsonRpcRequest("tools/list", null, 2);
client.sendJson(toolsRequest);
// 等待工具列表响应 (id=2)
List<String> toolsResponses = client.listener(response -> {
try {
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
return jsonMap != null && Integer.valueOf(2).equals(jsonMap.get("id"));
} catch (Exception e) {
log.warn("解析工具列表响应失败: {}", response, e);
return false;
}
});
// 处理工具列表响应
for (String response : toolsResponses) {
try {
Map<String, Object> jsonMap = JsonUtils.parseObject(response, Map.class);
if (jsonMap != null && Integer.valueOf(2).equals(jsonMap.get("id"))) {
// 检查是否有result字段
Object resultObj = jsonMap.get("result");
if (resultObj instanceof Map) {
Map<String, Object> resultMap = (Map<String, Object>) resultObj;
Object toolsObj = resultMap.get("tools");
if (toolsObj instanceof List) {
List<Map<String, Object>> toolsList = (List<Map<String, Object>>) toolsObj;
// 提取工具名称列表
List<String> result = toolsList.stream()
.map(tool -> (String) tool.get("name"))
.filter(name -> name != null)
.collect(Collectors.toList());
log.info("成功获取MCP工具列表,智能体ID: {}, 工具数量: {}", id, result.size());
return result;
}
} else if (jsonMap.containsKey("error")) {
log.error("获取工具列表失败,智能体ID: {}, 错误: {}", id, jsonMap.get("error"));
return List.of();
}
}
} catch (Exception e) {
log.warn("处理工具列表响应失败: {}", response, e);
}
}
log.warn("未找到有效的工具列表响应,智能体ID: {}", id);
return List.of();
}
} catch (Exception e) {
log.error("获取智能体 MCP 工具列表失败,智能体ID: {}", id, e);
return List.of();
}
}
/**
* 获取URI对象
*
* @param url 路径
* @return URI对象
*/
private static URI getURI(String url) {
try {
return new URI(url);
} catch (URISyntaxException e) {
log.error("路径格式不正确路径:{}\n错误信息:{}", url, e.getMessage());
throw new RuntimeException("mcp的地址存在错误,请进入参数管理修改mcp接入点地址");
}
}
/**
* 获取密钥
*
* @param uri mcp地址
* @return 密钥
*/
private static String getSecretKey(URI uri) {
// 获取参数
String query = uri.getQuery();
// 获取aes加密密钥
String str = "key=";
return query.substring(query.indexOf(str) + str.length());
}
/**
* 获取智能体mcp接入点url
*
* @param uri mcp地址
* @return 智能体mcp接入点url
*/
private String getAgentMcpUrl(URI uri) {
// 获取协议
String wsScheme = (uri.getScheme().equals("https")) ? "wss" : "ws";
// 获取主机,端口,路径
String path = uri.getSchemeSpecificPart();
// 获取到最后一个/前的path
path = path.substring(0, path.lastIndexOf("/"));
return wsScheme + ":" + path;
}
/**
* 获取对智能体id加密的token
*
* @param agentId 智能体id
* @param key 加密密钥
* @return 加密后token
*/
private static String encryptToken(String agentId, String key) {
// 使用md5对智能体id进行加密
String md5 = HashEncryptionUtil.Md5hexDigest(agentId);
// aes需要加密文本
String json = "{\"agentId\": \"%s\"}".formatted(md5);
// 加密后成token值
return AESUtils.encrypt(key, json);
}
}
@@ -56,6 +56,9 @@ public class AgentTemplateServiceImpl extends ServiceImpl<AgentTemplateDao, Agen
wrapper.set("tts_model_id", modelId);
wrapper.set("tts_voice_id", null);
break;
case "VLLM":
wrapper.set("vllm_model_id", modelId);
break;
case "MEMORY":
wrapper.set("mem_model_id", modelId);
break;
@@ -0,0 +1,386 @@
package xiaozhi.modules.agent.service.impl;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.transaction.support.TransactionTemplate;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.exception.RenException;
import xiaozhi.common.utils.ConvertUtils;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.dao.AgentVoicePrintDao;
import xiaozhi.modules.agent.dto.AgentVoicePrintSaveDTO;
import xiaozhi.modules.agent.dto.AgentVoicePrintUpdateDTO;
import xiaozhi.modules.agent.dto.IdentifyVoicePrintResponse;
import xiaozhi.modules.agent.entity.AgentVoicePrintEntity;
import xiaozhi.modules.agent.service.AgentChatAudioService;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentVoicePrintService;
import xiaozhi.modules.agent.vo.AgentVoicePrintVO;
import xiaozhi.modules.sys.service.SysParamsService;
/**
* @author zjy
*/
@Service
@AllArgsConstructor
@Slf4j
public class AgentVoicePrintServiceImpl extends ServiceImpl<AgentVoicePrintDao, AgentVoicePrintEntity>
implements AgentVoicePrintService {
private final AgentChatAudioService agentChatAudioService;
private final RestTemplate restTemplate;
private final SysParamsService sysParamsService;
private final AgentChatHistoryService agentChatHistoryService;
// Springboot提供的编程事务类
private final TransactionTemplate transactionTemplate;
// 识别度
private final Double RECOGNITION = 0.5;
@Override
public boolean insert(AgentVoicePrintSaveDTO dto) {
// 获取音频数据
ByteArrayResource resource = getVoicePrintAudioWAV(dto.getAgentId(), dto.getAudioId());
// 识别一下此声音是否注册过
IdentifyVoicePrintResponse response = identifyVoicePrint(dto.getAgentId(), resource);
if (response != null && response.getScore() > RECOGNITION) {
// 根据识别出的声纹ID查询对应的用户信息
AgentVoicePrintEntity existingVoicePrint = baseMapper.selectById(response.getSpeakerId());
String existingUserName = existingVoicePrint != null ? existingVoicePrint.getSourceName() : "未知用户";
throw new RenException("此声音声纹对应的人(" + existingUserName + ")已经注册,请选择其他声音注册");
}
AgentVoicePrintEntity entity = ConvertUtils.sourceToTarget(dto, AgentVoicePrintEntity.class);
// 开启事务
return Boolean.TRUE.equals(transactionTemplate.execute(status -> {
try {
// 保存声纹信息
int row = baseMapper.insert(entity);
// 插入一条数据,影响的数据不等于1说明出现了,保存问题回滚
if (row != 1) {
status.setRollbackOnly(); // 标记事务回滚
return false;
}
// 发送注册声纹请求
registerVoicePrint(entity.getId(), resource);
return true;
} catch (RenException e) {
status.setRollbackOnly(); // 标记事务回滚
throw e;
} catch (Exception e) {
status.setRollbackOnly(); // 标记事务回滚
log.error("保存声纹错误原因:{}", e.getMessage());
throw new RenException("保存声纹错误,请联系管理员");
}
}));
}
@Override
public boolean delete(Long userId, String voicePrintId) {
// 开启事务
return Boolean.TRUE.equals(transactionTemplate.execute(status -> {
try {
// 删除声纹,按照指定当前登录用户和智能体
int row = baseMapper.delete(new LambdaQueryWrapper<AgentVoicePrintEntity>()
.eq(AgentVoicePrintEntity::getId, voicePrintId)
.eq(AgentVoicePrintEntity::getCreator, userId));
if (row != 1) {
status.setRollbackOnly(); // 标记事务回滚
return false;
}
cancelVoicePrint(voicePrintId);
return true;
} catch (RenException e) {
status.setRollbackOnly(); // 标记事务回滚
throw e;
} catch (Exception e) {
status.setRollbackOnly(); // 标记事务回滚
log.error("删除声纹错误原因:{}", e.getMessage());
throw new RenException("删除声纹错误,请联系管理员");
}
}));
}
@Override
public List<AgentVoicePrintVO> list(Long userId, String agentId) {
// 按照指定当前登录用户和智能体查找数据
List<AgentVoicePrintEntity> list = baseMapper.selectList(new LambdaQueryWrapper<AgentVoicePrintEntity>()
.eq(AgentVoicePrintEntity::getAgentId, agentId)
.eq(AgentVoicePrintEntity::getCreator, userId));
return list.stream().map(entity -> {
// 遍历转换成AgentVoicePrintVO类型
return ConvertUtils.sourceToTarget(entity, AgentVoicePrintVO.class);
}).toList();
}
@Override
public boolean update(Long userId, AgentVoicePrintUpdateDTO dto) {
AgentVoicePrintEntity agentVoicePrintEntity = baseMapper
.selectOne(new LambdaQueryWrapper<AgentVoicePrintEntity>()
.eq(AgentVoicePrintEntity::getId, dto.getId())
.eq(AgentVoicePrintEntity::getCreator, userId));
if (agentVoicePrintEntity == null) {
return false;
}
// 获取音频Id
String audioId = dto.getAudioId();
// 获取智能体id
String agentId = agentVoicePrintEntity.getAgentId();
ByteArrayResource resource;
// audioId不等于空,且audioId和之前的保存的音频id不一样,则需要重新获取音频数据生成声纹
if (!StringUtils.isEmpty(audioId) && !audioId.equals(agentVoicePrintEntity.getAudioId())) {
resource = getVoicePrintAudioWAV(agentId, audioId);
// 识别一下此声音是否注册过
IdentifyVoicePrintResponse response = identifyVoicePrint(agentId, resource);
// 返回分数高于RECOGNITION说明这个声纹已经有了
if (response != null && response.getScore() > RECOGNITION) {
// 判断返回的id如果不是要修改的声纹id,说明这个声纹id,现在要注册的声音已经存在且不是原来的声纹,不允许修改
if (!response.getSpeakerId().equals(dto.getId())) {
// 根据识别出的声纹ID查询对应的用户信息
AgentVoicePrintEntity existingVoicePrint = baseMapper.selectById(response.getSpeakerId());
String existingUserName = existingVoicePrint != null ? existingVoicePrint.getSourceName() : "未知用户";
throw new RenException("此次修改不允许,此声音已经注册为声纹了(" + existingUserName + "");
}
}
} else {
resource = null;
}
// 开启事务
return Boolean.TRUE.equals(transactionTemplate.execute(status -> {
try {
AgentVoicePrintEntity entity = ConvertUtils.sourceToTarget(dto, AgentVoicePrintEntity.class);
int row = baseMapper.updateById(entity);
if (row != 1) {
status.setRollbackOnly(); // 标记事务回滚
return false;
}
if (resource != null) {
String id = entity.getId();
// 先注销之前这个声纹id上的声纹向量
cancelVoicePrint(id);
// 发送注册声纹请求
registerVoicePrint(id, resource);
}
return true;
} catch (RenException e) {
status.setRollbackOnly(); // 标记事务回滚
throw e;
} catch (Exception e) {
status.setRollbackOnly(); // 标记事务回滚
log.error("修改声纹错误原因:{}", e.getMessage());
throw new RenException("修改声纹错误,请联系管理员");
}
}));
}
/**
* 获取生纹接口URI对象
*
* @return URI对象
*/
private URI getVoicePrintURI() {
// 获取声纹接口地址
String voicePrint = sysParamsService.getValue(Constant.SERVER_VOICE_PRINT, true);
try {
return new URI(voicePrint);
} catch (URISyntaxException e) {
log.error("路径格式不正确路径:{}\n错误信息:{}", voicePrint, e.getMessage());
throw new RuntimeException("声纹接口的地址存在错误,请进入参数管理修改声纹接口地址");
}
}
/**
* 获取声纹地址基础路径
*
* @param uri 声纹地址uri
* @return 基础路径
*/
private String getBaseUrl(URI uri) {
String protocol = uri.getScheme();
String host = uri.getHost();
int port = uri.getPort();
return "%s://%s:%s".formatted(protocol, host, port);
}
/**
* 获取验证Authorization
*
* @param uri 声纹地址uri
* @return Authorization值
*/
private String getAuthorization(URI uri) {
// 获取参数
String query = uri.getQuery();
// 获取aes加密密钥
String str = "key=";
return "Bearer " + query.substring(query.indexOf(str) + str.length());
}
/**
* 获取声纹音频资源数据
*
* @param audioId 音频Id
* @return 声纹音频资源数据
*/
private ByteArrayResource getVoicePrintAudioWAV(String agentId, String audioId) {
// 判断这个音频是否属于当前智能体
boolean b = agentChatHistoryService.isAudioOwnedByAgent(audioId, agentId);
if (!b) {
throw new RenException("音频数据不属于这个智能体");
}
// 获取到音频数据
byte[] audio = agentChatAudioService.getAudio(audioId);
// 如果音频数据为空的直接报错不进行下去
if (audio == null || audio.length == 0) {
throw new RenException("音频数据是空的请检查上传数据");
}
// 将字节数组包装为资源,返回
return new ByteArrayResource(audio) {
@Override
public String getFilename() {
return "VoicePrint.WAV"; // 设置文件名
}
};
}
/**
* 发送注册声纹http请求
*
* @param id 声纹id
* @param resource 声纹音频资源
*/
private void registerVoicePrint(String id, ByteArrayResource resource) {
// 处理声纹接口地址,获取前缀
URI uri = getVoicePrintURI();
String baseUrl = getBaseUrl(uri);
String requestUrl = baseUrl + "/voiceprint/register";
// 创建请求体
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("speaker_id", id);
body.add("file", resource);
// 创建请求头
HttpHeaders headers = new HttpHeaders();
headers.set("Authorization", getAuthorization(uri));
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
// 创建请求体
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
// 发送 POST 请求
ResponseEntity<String> response = restTemplate.postForEntity(requestUrl, requestEntity, String.class);
if (response.getStatusCode() != HttpStatus.OK) {
log.error("声纹注册失败,请求路径:{}", requestUrl);
throw new RenException("声纹保存失败,请求不成功");
}
// 检查响应内容
String responseBody = response.getBody();
if (responseBody == null || !responseBody.contains("true")) {
log.error("声纹注册失败,请求处理失败内容:{}", responseBody == null ? "空内容" : responseBody);
throw new RenException("声纹保存失败,请求处理失败");
}
}
/**
* 发送注销声纹的请求
*
* @param voicePrintId 声纹id
*/
private void cancelVoicePrint(String voicePrintId) {
URI uri = getVoicePrintURI();
String baseUrl = getBaseUrl(uri);
String requestUrl = baseUrl + "/voiceprint/" + voicePrintId;
// 创建请求头
HttpHeaders headers = new HttpHeaders();
headers.set("Authorization", getAuthorization(uri));
// 创建请求体
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(headers);
// 发送 POST 请求
ResponseEntity<String> response = restTemplate.exchange(requestUrl, HttpMethod.DELETE, requestEntity,
String.class);
if (response.getStatusCode() != HttpStatus.OK) {
log.error("声纹注销失败,请求路径:{}", requestUrl);
throw new RenException("声纹注销失败,请求不成功");
}
// 检查响应内容
String responseBody = response.getBody();
if (responseBody == null || !responseBody.contains("true")) {
log.error("声纹注销失败,请求处理失败内容:{}", responseBody == null ? "空内容" : responseBody);
throw new RenException("声纹注销失败,请求处理失败");
}
}
/**
* 发送识别声纹http请求
*
* @param agentId 智能体id
* @param resource 声纹音频资源
* @return 返回识别数据
*/
private IdentifyVoicePrintResponse identifyVoicePrint(String agentId, ByteArrayResource resource) {
// 获取该智能体所有注册的声纹
List<AgentVoicePrintEntity> agentVoicePrintList = baseMapper
.selectList(new LambdaQueryWrapper<AgentVoicePrintEntity>()
.select(AgentVoicePrintEntity::getId)
.eq(AgentVoicePrintEntity::getAgentId, agentId));
// 声纹数量为0,说明还没注册过声纹不需要发生识别请求
if (agentVoicePrintList.isEmpty()) {
return null;
}
// 处理声纹接口地址,获取前缀
URI uri = getVoicePrintURI();
String baseUrl = getBaseUrl(uri);
String requestUrl = baseUrl + "/voiceprint/identify";
// 创建请求体
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
// 创建speaker_id参数
String speakerIds = agentVoicePrintList.stream()
.map(AgentVoicePrintEntity::getId)
.collect(Collectors.joining(","));
body.add("speaker_ids", speakerIds);
body.add("file", resource);
// 创建请求头
HttpHeaders headers = new HttpHeaders();
headers.set("Authorization", getAuthorization(uri));
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
// 创建请求体
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
// 发送 POST 请求
ResponseEntity<String> response = restTemplate.postForEntity(requestUrl, requestEntity, String.class);
if (response.getStatusCode() != HttpStatus.OK) {
log.error("声纹识别请求失败,请求路径:{}", requestUrl);
throw new RenException("声纹识别失败,请求不成功");
}
// 检查响应内容
String responseBody = response.getBody();
if (responseBody != null) {
return JsonUtils.parseObject(responseBody, IdentifyVoicePrintResponse.class);
}
return null;
}
}
@@ -0,0 +1,16 @@
package xiaozhi.modules.agent.vo;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
/**
* 智能体用户个人聊天数据的VO
*/
@Data
public class AgentChatHistoryUserVO {
@Schema(description = "聊天内容")
private String content;
@Schema(description = "音频ID")
private String audioId;
}
@@ -0,0 +1,33 @@
package xiaozhi.modules.agent.vo;
import lombok.Data;
import java.util.Date;
/**
* 展示智能体声纹列表VO
*/
@Data
public class AgentVoicePrintVO {
/**
* 主键id
*/
private String id;
/**
* 音频文件id
*/
private String audioId;
/**
* 声纹来源的人姓名
*/
private String sourceName;
/**
* 描述声纹来源的人
*/
private String introduce;
/**
* 创建时间
*/
private Date createDate;
}
@@ -1,23 +1,34 @@
package xiaozhi.modules.config.service.impl;
import java.util.*;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import lombok.AllArgsConstructor;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.exception.ErrorCode;
import xiaozhi.common.exception.RenException;
import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.utils.ConvertUtils;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.dao.AgentVoicePrintDao;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
import xiaozhi.modules.agent.entity.AgentVoicePrintEntity;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.agent.vo.AgentVoicePrintVO;
import xiaozhi.modules.config.service.ConfigService;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
@@ -39,6 +50,8 @@ public class ConfigServiceImpl implements ConfigService {
private final RedisUtils redisUtils;
private final TimbreService timbreService;
private final AgentPluginMappingService agentPluginMappingService;
private final AgentMcpAccessPointService agentMcpAccessPointService;
private final AgentVoicePrintDao agentVoicePrintDao;
@Override
public Object getConfig(Boolean isCache) {
@@ -66,6 +79,8 @@ public class ConfigServiceImpl implements ConfigService {
null,
null,
null,
null,
null,
agent.getVadModelId(),
agent.getAsrModelId(),
null,
@@ -102,9 +117,13 @@ public class ConfigServiceImpl implements ConfigService {
}
// 获取音色信息
String voice = null;
String referenceAudio = null;
String referenceText = null;
TimbreDetailsVO timbre = timbreService.get(agent.getTtsVoiceId());
if (timbre != null) {
voice = timbre.getTtsVoice();
referenceAudio = timbre.getReferenceAudio();
referenceText = timbre.getReferenceText();
}
// 构建返回数据
Map<String, Object> result = new HashMap<>();
@@ -144,6 +163,14 @@ public class ConfigServiceImpl implements ConfigService {
result.put("plugins", pluginParams);
}
}
// 获取mcp接入点地址
String mcpEndpoint = agentMcpAccessPointService.getAgentMcpAccessAddress(agent.getId());
if (StringUtils.isNotBlank(mcpEndpoint) && mcpEndpoint.startsWith("ws")) {
mcpEndpoint = mcpEndpoint.replace("/mcp/", "/call/");
result.put("mcp_endpoint", mcpEndpoint);
}
// 获取声纹信息
buildVoiceprintConfig(agent.getId(), result);
// 构建模块配置
buildModuleConfig(
@@ -151,6 +178,8 @@ public class ConfigServiceImpl implements ConfigService {
agent.getSystemPrompt(),
agent.getSummaryMemory(),
voice,
referenceAudio,
referenceText,
agent.getVadModelId(),
agent.getAsrModelId(),
agent.getLlmModelId(),
@@ -167,7 +196,7 @@ public class ConfigServiceImpl implements ConfigService {
/**
* 构建配置信息
*
* @param paramsList 系统参数列表
* @param config 系统参数列表
* @return 配置信息
*/
private Object buildConfig(Map<String, Object> config) {
@@ -235,24 +264,84 @@ public class ConfigServiceImpl implements ConfigService {
return config;
}
/**
* 构建声纹配置信息
*
* @param agentId 智能体ID
* @param result 结果Map
*/
private void buildVoiceprintConfig(String agentId, Map<String, Object> result) {
try {
// 获取声纹接口地址
String voiceprintUrl = sysParamsService.getValue("server.voice_print", true);
if (StringUtils.isBlank(voiceprintUrl) || "null".equals(voiceprintUrl)) {
return;
}
// 获取智能体关联的声纹信息(不需要用户权限验证)
List<AgentVoicePrintVO> voiceprints = getVoiceprintsByAgentId(agentId);
if (voiceprints == null || voiceprints.isEmpty()) {
return;
}
// 构建speakers列表
List<String> speakers = new ArrayList<>();
for (AgentVoicePrintVO voiceprint : voiceprints) {
String speakerStr = String.format("%s,%s,%s",
voiceprint.getId(),
voiceprint.getSourceName(),
voiceprint.getIntroduce() != null ? voiceprint.getIntroduce() : "");
speakers.add(speakerStr);
}
// 构建声纹配置
Map<String, Object> voiceprintConfig = new HashMap<>();
voiceprintConfig.put("url", voiceprintUrl);
voiceprintConfig.put("speakers", speakers);
result.put("voiceprint", voiceprintConfig);
} catch (Exception e) {
// 声纹配置获取失败时不影响其他功能
System.err.println("获取声纹配置失败: " + e.getMessage());
}
}
/**
* 获取智能体关联的声纹信息
*
* @param agentId 智能体ID
* @return 声纹信息列表
*/
private List<AgentVoicePrintVO> getVoiceprintsByAgentId(String agentId) {
LambdaQueryWrapper<AgentVoicePrintEntity> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(AgentVoicePrintEntity::getAgentId, agentId);
queryWrapper.orderByAsc(AgentVoicePrintEntity::getCreateDate);
List<AgentVoicePrintEntity> entities = agentVoicePrintDao.selectList(queryWrapper);
return ConvertUtils.sourceToTarget(entities, AgentVoicePrintVO.class);
}
/**
* 构建模块配置
*
* @param prompt 提示词
* @param voice 音色
* @param vadModelId VAD模型ID
* @param asrModelId ASR模型ID
* @param llmModelId LLM模型ID
* @param ttsModelId TTS模型ID
* @param memModelId 记忆模型ID
* @param intentModelId 意图模型ID
* @param result 结果Map
* @param prompt 提示词
* @param voice 音色
* @param referenceAudio 参考音频路径
* @param referenceText 参考文本
* @param vadModelId VAD模型ID
* @param asrModelId ASR模型ID
* @param llmModelId LLM模型ID
* @param ttsModelId TTS模型ID
* @param memModelId 记忆模型ID
* @param intentModelId 意图模型ID
* @param result 结果Map
*/
private void buildModuleConfig(
String assistantName,
String prompt,
String summaryMemory,
String voice,
String referenceAudio,
String referenceText,
String vadModelId,
String asrModelId,
String llmModelId,
@@ -274,12 +363,20 @@ public class ConfigServiceImpl implements ConfigService {
continue;
}
ModelConfigEntity model = modelConfigService.getModelById(modelIds[i], isCache);
if (model == null) {
continue;
}
Map<String, Object> typeConfig = new HashMap<>();
if (model.getConfigJson() != null) {
typeConfig.put(model.getId(), model.getConfigJson());
// 如果是TTS类型,添加private_voice属性
if ("TTS".equals(modelTypes[i]) && voice != null) {
((Map<String, Object>) model.getConfigJson()).put("private_voice", voice);
if ("TTS".equals(modelTypes[i])) {
if (voice != null)
((Map<String, Object>) model.getConfigJson()).put("private_voice", voice);
if (referenceAudio != null)
((Map<String, Object>) model.getConfigJson()).put("ref_audio", referenceAudio);
if (referenceText != null)
((Map<String, Object>) model.getConfigJson()).put("ref_text", referenceText);
}
// 如果是Intent类型,且type=intent_llm,则给他添加附加模型
if ("Intent".equals(modelTypes[i])) {
@@ -4,6 +4,7 @@ import java.util.List;
import org.apache.commons.lang3.StringUtils;
import org.apache.shiro.authz.annotation.RequiresPermissions;
import org.springframework.beans.BeanUtils;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
@@ -14,6 +15,7 @@ import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import lombok.AllArgsConstructor;
import xiaozhi.common.exception.ErrorCode;
import xiaozhi.common.redis.RedisKeys;
@@ -22,6 +24,8 @@ import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.device.dto.DeviceRegisterDTO;
import xiaozhi.modules.device.dto.DeviceUnBindDTO;
import xiaozhi.modules.device.dto.DeviceUpdateDTO;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.security.user.SecurityUser;
@@ -80,16 +84,29 @@ public class DeviceController {
return new Result<Void>();
}
@PutMapping("/enableOta/{id}/{status}")
@Operation(summary = "启用/关闭OTA自动升级")
@PutMapping("/update/{id}")
@Operation(summary = "更新设备信息")
@RequiresPermissions("sys:role:normal")
public Result<Void> enableOtaUpgrade(@PathVariable String id, @PathVariable Integer status) {
public Result<Void> updateDeviceInfo(@PathVariable String id, @Valid @RequestBody DeviceUpdateDTO deviceUpdateDTO) {
DeviceEntity entity = deviceService.selectById(id);
if (entity == null) {
return new Result<Void>().error("设备不存在");
}
entity.setAutoUpdate(status);
UserDetail user = SecurityUser.getUser();
if (!entity.getUserId().equals(user.getId())) {
return new Result<Void>().error("设备不存在");
}
BeanUtils.copyProperties(deviceUpdateDTO, entity);
deviceService.updateById(entity);
return new Result<Void>();
}
@PostMapping("/manual-add")
@Operation(summary = "手动添加设备")
@RequiresPermissions("sys:role:normal")
public Result<Void> manualAddDevice(@RequestBody @Valid DeviceManualAddDTO dto) {
UserDetail user = SecurityUser.getUser();
deviceService.manualAddDevice(user.getId(), dto);
return new Result<>();
}
}
@@ -51,13 +51,12 @@ public class OTAController {
if (StringUtils.isBlank(clientId)) {
clientId = deviceId;
}
String macAddress = deviceReportReqDTO.getMacAddress();
boolean macAddressValid = isMacAddressValid(macAddress);
boolean macAddressValid = isMacAddressValid(deviceId);
// 设备Id和Mac地址应是一致的, 并且必须需要application字段
if (!deviceId.equals(macAddress) || !macAddressValid || deviceReportReqDTO.getApplication() == null) {
return createResponse(DeviceReportRespDTO.createError("Invalid OTA request"));
if (!macAddressValid) {
return createResponse(DeviceReportRespDTO.createError("Invalid device ID"));
}
return createResponse(deviceService.checkDeviceActive(macAddress, clientId, deviceReportReqDTO));
return createResponse(deviceService.checkDeviceActive(deviceId, clientId, deviceReportReqDTO));
}
@Operation(summary = "设备快速检查激活状态")
@@ -0,0 +1,11 @@
package xiaozhi.modules.device.dto;
import lombok.Data;
@Data
public class DeviceManualAddDTO {
private String agentId;
private String board; // 设备型号
private String appVersion; // 固件版本
private String macAddress; // Mac地址
}
@@ -0,0 +1,29 @@
package xiaozhi.modules.device.dto;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Size;
import lombok.Data;
import java.io.Serializable;
/**
* 设备更新DTO
*/
@Data
public class DeviceUpdateDTO implements Serializable {
/**
* 自动更新状态
*/
@Max(1)
@Min(0)
private Integer autoUpdate;
/**
* 设备别名
*/
@Size(max = 64)
private String alias;
private static final long serialVersionUID = 1L;
}
@@ -8,6 +8,7 @@ import xiaozhi.common.service.BaseService;
import xiaozhi.modules.device.dto.DevicePageUserDTO;
import xiaozhi.modules.device.dto.DeviceReportReqDTO;
import xiaozhi.modules.device.dto.DeviceReportRespDTO;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.vo.UserShowDeviceListVO;
@@ -87,5 +88,14 @@ public interface DeviceService extends BaseService<DeviceEntity> {
*/
Date getLatestLastConnectionTime(String agentId);
/**
* 手动添加设备
*/
void manualAddDevice(Long userId, DeviceManualAddDTO dto);
/**
* 更新设备连接信息
*/
void updateDeviceConnectionInfo(String agentId, String deviceId, String appVersion);
}
@@ -44,6 +44,7 @@ import xiaozhi.modules.device.vo.UserShowDeviceListVO;
import xiaozhi.modules.security.user.SecurityUser;
import xiaozhi.modules.sys.service.SysParamsService;
import xiaozhi.modules.sys.service.SysUserUtilService;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
@Slf4j
@Service
@@ -410,4 +411,30 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
}
return 0;
}
@Override
public void manualAddDevice(Long userId, DeviceManualAddDTO dto) {
// 检查mac是否已存在
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", dto.getMacAddress());
DeviceEntity exist = baseDao.selectOne(wrapper);
if (exist != null) {
throw new RenException("该Mac地址已存在");
}
Date now = new Date();
DeviceEntity entity = new DeviceEntity();
entity.setId(dto.getMacAddress());
entity.setUserId(userId);
entity.setAgentId(dto.getAgentId());
entity.setBoard(dto.getBoard());
entity.setAppVersion(dto.getAppVersion());
entity.setMacAddress(dto.getMacAddress());
entity.setCreateDate(now);
entity.setUpdateDate(now);
entity.setLastConnectedAt(now);
entity.setCreator(userId);
entity.setUpdater(userId);
entity.setAutoUpdate(1);
baseDao.insert(entity);
}
}
@@ -104,6 +104,12 @@ public class SysParamsController {
// 验证OTA地址
validateOtaUrl(dto.getParamCode(), dto.getParamValue());
// 验证MCP地址
validateMcpUrl(dto.getParamCode(), dto.getParamValue());
//
validateVoicePrint(dto.getParamCode(), dto.getParamValue());
sysParamsService.update(dto);
configService.getConfig(false);
return new Result<Void>();
@@ -195,4 +201,67 @@ public class SysParamsController {
throw new RenException("OTA接口验证失败:" + e.getMessage());
}
}
private void validateMcpUrl(String paramCode, String url) {
if (!paramCode.equals(Constant.SERVER_MCP_ENDPOINT)) {
return;
}
if (StringUtils.isBlank(url) || url.equals("null")) {
throw new RenException("MCP地址不能为空");
}
if (url.contains("localhost") || url.contains("127.0.0.1")) {
throw new RenException("MCP地址不能使用localhost或127.0.0.1");
}
if (!url.toLowerCase().contains("key")) {
throw new RenException("不是正确的MCP地址");
}
try {
// 发送GET请求
ResponseEntity<String> response = restTemplate.getForEntity(url, String.class);
if (response.getStatusCode() != HttpStatus.OK) {
throw new RenException("MCP接口访问失败,状态码:" + response.getStatusCode());
}
// 检查响应内容是否包含mcp相关信息
String body = response.getBody();
if (body == null || !body.contains("success")) {
throw new RenException("MCP接口返回内容格式不正确,可能不是一个真实的MCP接口");
}
} catch (Exception e) {
throw new RenException("MCP接口验证失败:" + e.getMessage());
}
}
// 验证声纹接口地址是否正常
private void validateVoicePrint(String paramCode, String url) {
if (!paramCode.equals(Constant.SERVER_VOICE_PRINT)) {
return;
}
if (StringUtils.isBlank(url) || url.equals("null")) {
throw new RenException("声纹接口地址不能为空");
}
if (url.contains("localhost") || url.contains("127.0.0.1")) {
throw new RenException("声纹接口地址不能使用localhost或127.0.0.1");
}
if (!url.toLowerCase().contains("key")) {
throw new RenException("不是正确的声纹接口地址");
}
// 验证URL格式
if (!url.toLowerCase().startsWith("http")) {
throw new RenException("声纹接口地址必须以http或https开头");
}
try {
// 发送GET请求
ResponseEntity<String> response = restTemplate.getForEntity(url, String.class);
if (response.getStatusCode() != HttpStatus.OK) {
throw new RenException("声纹接口访问失败,状态码:" + response.getStatusCode());
}
// 检查响应内容
String body = response.getBody();
if (body == null || !body.contains("healthy")) {
throw new RenException("声纹接口返回内容格式不正确,可能不是一个真实的MCP接口");
}
} catch (Exception e) {
throw new RenException("声纹接口验证失败:" + e.getMessage());
}
}
}
@@ -137,6 +137,37 @@ public class WebSocketClientManager implements Closeable {
return collected;
}
private <T> List<T> listenerCustomWithoutClose(
BlockingQueue<T> queue,
Predicate<T> predicate)
throws InterruptedException, TimeoutException, ExecutionException {
List<T> collected = new ArrayList<>();
long deadline = System.currentTimeMillis() + maxSessionDurationUnit.toMillis(maxSessionDuration);
while (true) {
if (errorFuture.isDone()) {
errorFuture.get();
}
long remaining = deadline - System.currentTimeMillis();
if (remaining <= 0) {
throw new TimeoutException("等待批量消息超时");
}
T msg = queue.poll(remaining, TimeUnit.MILLISECONDS);
if (msg == null) {
throw new TimeoutException("等待批量消息超时");
}
collected.add(msg);
if (predicate.test(msg)) {
break;
}
}
// 不调用 close(),保持连接开放
return collected;
}
/**
* 同步接收多条消息,直到 predicate 为 true 或超时抛异常;
*
@@ -147,6 +178,17 @@ public class WebSocketClientManager implements Closeable {
return listenerCustom(textMessageQueue, predicate);
}
/**
* 同步接收多条消息,直到 predicate 为 true 或超时抛异常;
* 不自动关闭连接,适用于需要在同一连接上发送多个消息的场景
*
* @return 返回监听期间的所有消息列表
*/
public List<String> listenerWithoutClose(Predicate<String> predicate)
throws InterruptedException, TimeoutException, ExecutionException {
return listenerCustomWithoutClose(textMessageQueue, predicate);
}
public List<byte[]> listenerBinary(Predicate<byte[]> predicate)
throws InterruptedException, TimeoutException, ExecutionException {
return listenerCustom(binaryMessageQueue, predicate);
@@ -26,6 +26,12 @@ public class TimbreDataDTO {
@Schema(description = "备注")
private String remark;
@Schema(description = "参考音频路径")
private String referenceAudio;
@Schema(description = "參考文本")
private String referenceText;
@Schema(description = "排序")
@Min(value = 0, message = "{sort.number}")
private long sort;
@@ -34,6 +34,12 @@ public class TimbreEntity {
@Schema(description = "备注")
private String remark;
@Schema(description = "参考音频路径")
private String referenceAudio;
@Schema(description = "參考文本")
private String referenceText;
@Schema(description = "排序")
private long sort;
@@ -25,6 +25,12 @@ public class TimbreDetailsVO implements Serializable {
@Schema(description = "备注")
private String remark;
@Schema(description = "参考音频路径")
private String referenceAudio;
@Schema(description = "參考文本")
private String referenceText;
@Schema(description = "排序")
private long sort;
@@ -0,0 +1,3 @@
ALTER TABLE `ai_tts_voice`
ADD COLUMN `reference_audio` VARCHAR(500) DEFAULT NULL COMMENT '参考音频路径' AFTER `remark`,
ADD COLUMN `reference_text` VARCHAR(500) DEFAULT NULL COMMENT '参考文本' AFTER `reference_audio`;
@@ -0,0 +1 @@
ALTER TABLE ai_agent_plugin_mapping CONVERT TO CHARACTER SET utf8mb4;
@@ -0,0 +1,19 @@
-- LLM意图识别配置说明
UPDATE `ai_model_config` SET
`doc_link` = NULL,
`remark` = 'LLM意图识别配置说明:
1. 使用独立的LLM进行意图识别
2. 默认使用selected_module.LLM的模型
3. 可以配置使用独立的LLM(如免费的ChatGLMLLM
4. 通用性强,但会增加处理时间
配置说明:
1. 在llm字段中指定使用的LLM模型
2. 如果不指定,则使用selected_module.LLM的模型' WHERE `id` = 'Intent_intent_llm';
-- 函数调用意图识别配置说明
UPDATE `ai_model_config` SET
`doc_link` = NULL,
`remark` = '函数调用意图识别配置说明:
1. 使用LLM的function_call功能进行意图识别
2. 需要所选择的LLM支持function_call
3. 按需调用工具,处理速度快' WHERE `id` = 'Intent_function_call';
@@ -0,0 +1,18 @@
-- 更新现有的 get_news_from_newsnow 插件配置
UPDATE ai_model_provider
SET fields = JSON_ARRAY(
JSON_OBJECT(
'key', 'url',
'type', 'string',
'label', '接口地址',
'default', 'https://newsnow.busiyi.world/api/s?id='
),
JSON_OBJECT(
'key', 'news_sources',
'type', 'string',
'label', '新闻源配置',
'default', '澎湃新闻;百度热搜;财联社'
)
)
WHERE provider_code = 'get_news_from_newsnow'
AND model_type = 'Plugin';
@@ -0,0 +1,2 @@
delete from `sys_params` where id = 113;
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (113, 'server.mcp_endpoint', 'null', 'string', 1, 'mcp接入点地址');
@@ -0,0 +1,4 @@
-- 添加声纹接口地址参数配置
delete from `sys_params` where id = 114;
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark)
VALUES (114, 'server.voice_print', 'null', 'string', 1, '声纹接口地址');
@@ -0,0 +1,12 @@
DROP TABLE IF EXISTS ai_agent_voice_print;
create table ai_agent_voice_print (
id varchar(32) NOT NULL COMMENT '声纹ID',
agent_id varchar(32) NOT NULL COMMENT '关联的智能体ID',
source_name varchar(50) NOT NULL COMMENT '声纹来源的人的姓名',
introduce varchar(200) COMMENT '描述声纹来源的这个人',
create_date DATETIME COMMENT '创建时间',
creator bigint COMMENT '创建者',
update_date DATETIME COMMENT '修改时间',
updater bigint COMMENT '修改者',
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='智能体声纹表'
@@ -0,0 +1,3 @@
-- 智能体声纹添加新字段
ALTER TABLE ai_agent_voice_print
ADD COLUMN audio_id VARCHAR(32) NOT NULL COMMENT '音频ID';
@@ -205,3 +205,59 @@ databaseChangeLog:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506080955.sql
- changeSet:
id: 202506091720
author: shane0411
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506091720.sql
- changeSet:
id: 202506161101
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506161101.sql
- changeSet:
id: 202506191643
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506191643.sql
- changeSet:
id: 202506251620
author: Tink
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506251620.sql
- changeSet:
id: 202506261637
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506261637.sql
- changeSet:
id: 202507031602
author: zjy
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202507031602.sql
- changeSet:
id: 202507041018
author: zjy
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202507041018.sql
- changeSet:
id: 202507081646
author: zjy
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202507081646.sql
@@ -1 +1 @@
redis.call('FLUSHALL')
redis.call('FLUSHDB')
@@ -0,0 +1,85 @@
package xiaozhi.common.utils;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
public class AESUtilsTest {
@Test
public void testEncryptAndDecrypt() {
String key = "xiaozhi1234567890";
String plainText = "Hello, 小智!";
System.out.println("原始文本: " + plainText);
System.out.println("密钥: " + key);
// 加密
String encrypted = AESUtils.encrypt(key, plainText);
System.out.println("加密结果: " + encrypted);
// 解密
String decrypted = AESUtils.decrypt(key, encrypted);
System.out.println("解密结果: " + decrypted);
// 验证
assertEquals(plainText, decrypted, "加解密结果应该一致");
System.out.println("加解密一致性: " + plainText.equals(decrypted));
}
@Test
public void testDifferentKeyLengths() {
String[] keys = {
"1234567890123456", // 16位
"123456789012345678901234", // 24位
"12345678901234567890123456789012", // 32位
"short", // 短密钥
"verylongkeythatwillbetruncatedto32bytes" // 长密钥
};
String plainText = "测试文本";
for (String key : keys) {
String encrypted = AESUtils.encrypt(key, plainText);
String decrypted = AESUtils.decrypt(key, encrypted);
assertEquals(plainText, decrypted, "密钥长度: " + key.length());
}
}
@Test
public void testSpecialCharacters() {
String key = "xiaozhi1234567890";
String[] testTexts = {
"Hello World",
"你好世界",
"Hello, 小智!",
"特殊字符: !@#$%^&*()",
"数字123和中文混合",
"Emoji: 😀🎉🚀",
"空字符串测试",
""
};
for (String text : testTexts) {
String encrypted = AESUtils.encrypt(key, text);
String decrypted = AESUtils.decrypt(key, encrypted);
assertEquals(text, decrypted, "测试文本: " + text);
}
}
@Test
public void testCrossLanguageCompatibility() {
// 这些是Python版本生成的加密结果,用于测试跨语言兼容性
String key = "xiaozhi1234567890";
String plainText = "Hello, 小智!";
// Python版本生成的加密结果(需要运行Python测试后获取)
// String pythonEncrypted = "从Python测试中获取的加密结果";
// String decrypted = AESUtils.decrypt(key, pythonEncrypted);
// assertEquals(plainText, decrypted, "Java应该能解密Python加密的结果");
// 生成Java加密结果供Python测试
String javaEncrypted = AESUtils.encrypt(key, plainText);
System.out.println("Java加密结果供Python测试: " + javaEncrypted);
}
}
+12 -12
View File
@@ -42,7 +42,7 @@ nav {
}
.el-message {
top: 45px !important;
top: 70px !important;
}
</style>
<script>
@@ -64,26 +64,26 @@ export default {
if (this.isCDNEnabled) {
// 添加全局快捷键Alt+C用于显示缓存查看器
document.addEventListener('keydown', this.handleKeyDown);
// 在全局对象上添加缓存检查方法,便于调试
window.checkCDNCacheStatus = () => {
this.showCacheViewer = true;
};
// 在控制台输出提示信息
console.info(
'%c[小智服务] CDN缓存检查工具已加载',
'%c[小智服务] CDN缓存检查工具已加载',
'color: #409EFF; font-weight: bold;'
);
console.info(
'按下 Alt+C 组合键或在控制台运行 checkCDNCacheStatus() 可以查看CDN缓存状态'
);
// 检查Service Worker状态
this.checkServiceWorkerStatus();
} else {
console.info(
'%c[小智服务] CDN模式已禁用,使用本地打包资源',
'%c[小智服务] CDN模式已禁用,使用本地打包资源',
'color: #67C23A; font-weight: bold;'
);
}
@@ -108,19 +108,19 @@ export default {
const registrations = await navigator.serviceWorker.getRegistrations();
if (registrations.length > 0) {
console.info(
'%c[小智服务] Service Worker已注册',
'%c[小智服务] Service Worker已注册',
'color: #67C23A; font-weight: bold;'
);
// 输出缓存状态到控制台
setTimeout(async () => {
const hasCaches = await logCacheStatus();
if (!hasCaches) {
console.info(
'%c[小智服务] 还未检测到缓存,请刷新页面或等待缓存建立',
'%c[小智服务] 还未检测到缓存,请刷新页面或等待缓存建立',
'color: #E6A23C; font-weight: bold;'
);
// 开发环境下提供额外提示
if (process.env.NODE_ENV === 'development') {
console.info(
@@ -136,10 +136,10 @@ export default {
}, 2000);
} else {
console.info(
'%c[小智服务] Service Worker未注册,CDN资源可能无法缓存',
'%c[小智服务] Service Worker未注册,CDN资源可能无法缓存',
'color: #F56C6C; font-weight: bold;'
);
if (process.env.NODE_ENV === 'development') {
console.info(
'%c[小智服务] 在开发环境中,这是正常现象',
+1 -1
View File
@@ -7,6 +7,7 @@ import model from './module/model.js'
import ota from './module/ota.js'
import timbre from "./module/timbre.js"
import user from './module/user.js'
/**
* 接口地址
* 开发时自动读取使用.env.development文件
@@ -22,7 +23,6 @@ export function getServiceUrl() {
return DEV_API_SERVICE
}
/** request服务封装 */
export default {
getServiceUrl,
+122
View File
@@ -143,4 +143,126 @@ export default {
});
}).send();
},
// 获取智能体的MCP接入点地址
getAgentMcpAccessAddress(agentId, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/mcp/address/${agentId}`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getAgentMcpAccessAddress(agentId, callback);
});
}).send();
},
// 获取智能体的MCP工具列表
getAgentMcpToolsList(agentId, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/mcp/tools/${agentId}`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getAgentMcpToolsList(agentId, callback);
});
}).send();
},
// 添加智能体的声纹
addAgentVoicePrint(voicePrintData, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/voice-print`)
.method('POST')
.data(voicePrintData)
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.addAgentVoicePrint(voicePrintData, callback);
});
}).send();
},
// 获取指定智能体声纹列表
getAgentVoicePrintList(id,callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/voice-print/list/${id}`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getAgentVoicePrintList(id,callback);
});
}).send();
},
// 删除智能体声纹
deleteAgentVoicePrint(id, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/voice-print/${id}`)
.method('DELETE')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.deleteAgentVoicePrint(id, callback);
});
}).send();
},
// 更新智能体声纹
updateAgentVoicePrint(voicePrintData, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/voice-print`)
.method('PUT')
.data(voicePrintData)
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.updateAgentVoicePrint(voicePrintData, callback);
});
}).send();
},
// 获取指定智能体用户类型聊天记录
getRecentlyFiftyByAgentId(id,callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/${id}/chat-history/user`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getRecentlyFiftyByAgentId(id,callback);
});
}).send();
},
// 获取指定智能体用户类型聊天记录
getContentByAudioId(id,callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/${id}/chat-history/audio`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getContentByAudioId(id,callback);
});
}).send();
},
}
+21 -3
View File
@@ -51,10 +51,11 @@ export default {
});
}).send();
},
enableOtaUpgrade(id, status, callback) {
updateDeviceInfo(id, payload, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/device/enableOta/${id}/${status}`)
.url(`${getServiceUrl()}/device/update/${id}`)
.method('PUT')
.data(payload)
.success((res) => {
RequestService.clearRequestTime()
callback(res)
@@ -63,8 +64,25 @@ export default {
console.error('更新OTA状态失败:', err)
this.$message.error(err.msg || '更新OTA状态失败')
RequestService.reAjaxFun(() => {
this.enableOtaUpgrade(id, status, callback)
this.updateDeviceInfo(id, payload, callback)
})
}).send()
},
// 手动添加设备
manualAddDevice(params, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/device/manual-add`)
.method('POST')
.data(params)
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail((err) => {
console.error('手动添加设备失败:', err);
RequestService.reAjaxFun(() => {
this.manualAddDevice(params, callback);
});
}).send();
},
}
@@ -34,6 +34,8 @@ export default {
languages: params.languageType,
name: params.voiceName,
remark: params.remark,
referenceAudio: params.referenceAudio,
referenceText: params.referenceText,
sort: params.sort,
ttsModelId: params.ttsModelId,
ttsVoice: params.voiceCode,
@@ -75,6 +77,8 @@ export default {
languages: params.languageType,
name: params.voiceName,
remark: params.remark,
referenceAudio: params.referenceAudio,
referenceText: params.referenceText,
ttsModelId: params.ttsModelId,
ttsVoice: params.voiceCode,
voiceDemo: params.voiceDemo || ''
@@ -24,7 +24,7 @@
<img :src="message.chatType === 1 ? getUserAvatar(currentSessionId) : require('@/assets/xiaozhi-logo.png')"
class="avatar" />
<div class="message-content">
{{ message.content }}
{{ extractContentFromString(message.content) }}
<i v-if="message.audioId" :class="getAudioIconClass(message)"
@click="playAudio(message)" class="audio-icon"></i>
</div>
@@ -129,6 +129,32 @@ export default {
}
},
methods: {
/**
* 从 content 字段中提取聊天内容
* 如果 content 是 JSON 格式(如 {"speaker": "未知说话人", "content": "现在几点了。"}),则提取 content 字段
* 如果 content 是普通字符串,则直接返回
*
* @param {string} content 原始内容
* @returns {string} 提取的聊天内容
*/
extractContentFromString(content) {
if (!content || content.trim() === '') {
return content;
}
// 尝试解析为 JSON
try {
const jsonObj = JSON.parse(content);
if (jsonObj && typeof jsonObj === 'object' && jsonObj.content) {
return jsonObj.content;
}
} catch (e) {
// 如果不是有效的 JSON,直接返回原内容
}
// 如果不是 JSON 格式或没有 content 字段,直接返回原内容
return content;
},
resetData() {
this.sessions = [];
this.messages = [];
@@ -22,6 +22,9 @@
<div style="display: flex;gap: 10px;align-items: center;">
<div class="settings-btn" @click="handleConfigure">
配置角色
</div>
<div class="settings-btn" @click="handleVoicePrint">
声纹识别
</div>
<div class="settings-btn" @click="handleDeviceManage">
设备管理({{ device.deviceCount }})
@@ -77,6 +80,9 @@ export default {
handleConfigure() {
this.$router.push({ path: '/role-config', query: { agentId: this.device.agentId } });
},
handleVoicePrint() {
this.$router.push({ path: '/voice-print', query: { agentId: this.device.agentId } });
},
handleDeviceManage() {
this.$router.push({ path: '/device-management', query: { agentId: this.device.agentId } });
},
@@ -99,6 +99,62 @@
</div>
</div>
<!-- MCP区域 -->
<div class="mcp-access-point">
<div class="mcp-container">
<!-- 左侧区域 -->
<div class="mcp-left">
<div class="mcp-header">
<h3 class="bold-title">MCP接入点</h3>
</div>
<div class="url-header">
<div class="address-desc">
<span>以下是智能体的MCP接入点地址</span>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-enable.md"
target="_blank" class="doc-link">如何部署MCP接入点</a> &nbsp;&nbsp;|&nbsp;&nbsp;
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md"
target="_blank" class="doc-link">如何接入MCP功能</a> &nbsp;
</div>
</div>
<el-input v-model="mcpUrl" readonly class="url-input">
<template #suffix>
<el-button @click="copyUrl" class="inner-copy-btn" icon="el-icon-document-copy">
复制
</el-button>
</template>
</el-input>
</div>
<!-- 右侧区域 -->
<div class="mcp-right">
<div class="mcp-header">
<h3 class="bold-title">接入点状态</h3>
</div>
<div class="status-container">
<span class="status-indicator" :class="mcpStatus"></span>
<span class="status-text">{{
mcpStatus === 'connected' ? '已连接' :
mcpStatus === 'loading' ? '加载中...' : '未连接'
}}</span>
<button class="refresh-btn" @click="refreshStatus">
<span class="refresh-icon"></span>
<span>刷新</span>
</button>
</div>
<div class="mcp-tools-list">
<div v-if="mcpTools.length > 0" class="tools-grid">
<el-button v-for="tool in mcpTools" :key="tool" size="small" class="tool-btn" plain>
{{ tool }}
</el-button>
</div>
<div v-else class="no-tools">
<span>暂无可用工具</span>
</div>
</div>
</div>
</div>
</div>
<div class="drawer-footer">
<el-button @click="closeDialog">取消</el-button>
<el-button type="primary" @click="saveSelection">保存配置</el-button>
@@ -107,6 +163,8 @@
</template>
<script>
import Api from '@/apis/api';
export default {
props: {
value: Boolean,
@@ -117,6 +175,10 @@ export default {
allFunctions: {
type: Array,
default: () => []
},
agentId: {
type: String,
required: true
}
},
data() {
@@ -134,6 +196,10 @@ export default {
// 添加一个标志位来跟踪是否已经保存
hasSaved: false,
loading: false,
mcpUrl: "",
mcpStatus: "disconnected",
mcpTools: [],
}
},
computed: {
@@ -177,6 +243,10 @@ export default {
});
// 右侧默认指向第一个
this.currentFunction = this.selectedList[0] || null;
// 加载MCP数据
this.loadMcpAddress();
this.loadMcpTools();
}
},
dialogVisible(newVal) {
@@ -184,6 +254,60 @@ export default {
}
},
methods: {
copyUrl() {
const textarea = document.createElement('textarea');
textarea.value = this.mcpUrl;
textarea.style.position = 'fixed'; // 防止页面滚动
document.body.appendChild(textarea);
textarea.select();
try {
const successful = document.execCommand('copy');
if (successful) {
this.$message.success('已复制到剪贴板');
} else {
this.$message.error('复制失败,请手动复制');
}
} catch (err) {
this.$message.error('复制失败,请手动复制');
console.error('复制失败:', err);
} finally {
document.body.removeChild(textarea);
}
},
refreshStatus() {
this.mcpStatus = "loading";
this.loadMcpTools();
},
// 加载MCP接入点地址
loadMcpAddress() {
Api.agent.getAgentMcpAccessAddress(this.agentId, (res) => {
if (res.data.code === 0) {
this.mcpUrl = res.data.data || "";
} else {
this.mcpUrl = "";
console.error('获取MCP地址失败:', res.data.msg);
}
});
},
// 加载MCP工具列表
loadMcpTools() {
Api.agent.getAgentMcpToolsList(this.agentId, (res) => {
if (res.data.code === 0) {
this.mcpTools = res.data.data || [];
// 根据工具列表更新状态
this.mcpStatus = this.mcpTools.length > 0 ? "connected" : "disconnected";
} else {
this.mcpTools = [];
this.mcpStatus = "disconnected";
console.error('获取MCP工具列表失败:', res.data.msg);
}
});
},
flushArray(key) {
const text = this.textCache[key] || '';
const arr = text
@@ -298,7 +422,7 @@ export default {
display: grid;
grid-template-columns: max-content max-content 1fr;
gap: 12px;
height: calc(70vh - 60px);
height: calc(58vh);
}
.custom-header {
@@ -514,4 +638,203 @@ export default {
::v-deep .el-checkbox__label {
display: none;
}
.mcp-access-point {
border-top: 1px solid #EBEEF5;
padding: 20px 24px;
text-align: left;
}
.mcp-header {
.bold-title {
font-size: 18px;
font-weight: bold;
margin: 5px 0 30px 0;
}
}
.mcp-container {
display: flex;
justify-content: space-between;
gap: 30px;
}
.mcp-left,
.mcp-right {
flex: 1;
padding-bottom: 50px;
}
.url-header {
margin-bottom: 8px;
color: black;
h4 {
margin: 0 0 15px 0;
font-size: 16px;
font-weight: normal;
}
.address-desc {
display: flex;
align-items: center;
font-size: 14px;
margin-bottom: 12px;
.doc-link {
color: #1677ff;
text-decoration: none;
margin-left: 4px;
&:hover {
text-decoration: underline;
}
}
}
}
.url-input {
border-radius: 4px 0 0 4px;
font-size: 14px;
height: 36px;
box-sizing: border-box;
background-color: #f5f5f5;
}
::v-deep .el-input__inner {
background-color: #f5f5f5;
padding-right: 80px;
}
.url-input {
::v-deep .el-input__suffix {
right: 0;
display: flex;
align-items: center;
padding-right: 10px;
.inner-copy-btn {
pointer-events: auto;
border: none;
background: #1677ff;
color: white;
padding: 6px;
margin-top: 4px;
margin-left: 4px;
}
}
}
.mcp-right {
h4 {
margin: 0 0 10px 0;
font-size: 16px;
font-weight: normal;
color: black;
}
}
.status-container {
display: flex;
align-items: center;
.status-indicator {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 8px;
&.disconnected {
background-color: #909399;
/* 灰色 - 未连接 */
}
&.connected {
background-color: #67C23A;
/* 绿色 - 已连接 */
}
&.loading {
background-color: #E6A23C;
/* 橙色 - 加载中 */
animation: pulse 1.5s infinite;
}
}
.status-text {
font-size: 14px;
margin-right: 10px;
}
.refresh-btn {
display: flex;
align-items: center;
padding: 2px 10px;
background: white;
color: black;
border: 1px solid #DCDFE6;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
transition: all 0.3s;
&:hover {
background: #1677ff;
color: white;
border-color: #1677ff;
}
.refresh-icon {
margin-right: 6px;
font-size: 14px;
}
}
}
@keyframes pulse {
0% {
opacity: 1;
}
50% {
opacity: 0.4;
}
100% {
opacity: 1;
}
}
.mcp-tools-list {
margin-top: 10px;
.tools-grid {
display: flex;
flex-wrap: wrap;
gap: 8px;
}
.tool-btn {
padding: 6px 12px;
border-color: #1677ff;
color: #1677ff;
background-color: white;
font-size: 12px;
&:hover {
background-color: #1677ff;
color: white;
border-color: #1677ff;
}
}
.no-tools {
text-align: center;
color: #909399;
font-size: 14px;
padding: 10px 0;
}
}
</style>
@@ -0,0 +1,158 @@
<template>
<el-dialog title="手动添加设备" :visible="visible" @close="handleClose" width="30%" center>
<div class="dialog-content">
<el-form :model="deviceForm" :rules="rules" ref="deviceForm" label-width="100px">
<el-form-item label="设备型号" prop="board">
<el-select v-model="deviceForm.board" placeholder="请选择设备型号" style="width: 100%">
<el-option
v-for="item in firmwareTypes"
:key="item.key"
:label="item.name"
:value="item.key">
</el-option>
</el-select>
</el-form-item>
<el-form-item label="固件版本" prop="appVersion">
<el-input v-model="deviceForm.appVersion" placeholder="请输入固件版本"></el-input>
</el-form-item>
<el-form-item label="Mac地址" prop="macAddress">
<el-input v-model="deviceForm.macAddress" placeholder="请输入Mac地址"></el-input>
</el-form-item>
</el-form>
</div>
<div style="display: flex;margin: 15px 15px;gap: 7px;">
<div class="dialog-btn" @click="submitForm">确定</div>
<div class="dialog-btn cancel-btn" @click="cancel">取消</div>
</div>
</el-dialog>
</template>
<script>
import Api from '@/apis/api';
export default {
name: 'ManualAddDeviceDialog',
props: {
visible: { type: Boolean, required: true },
agentId: { type: String, required: true }
},
data() {
// MAC地址验证规则
const validateMac = (rule, value, callback) => {
const macRegex = /^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$/;
if (!value) {
callback(new Error('请输入Mac地址'));
} else if (!macRegex.test(value)) {
callback(new Error('请输入正确的Mac地址格式,例如:00:1A:2B:3C:4D:5E'));
} else {
callback();
}
};
return {
deviceForm: {
board: '',
appVersion: '',
macAddress: ''
},
firmwareTypes: [],
rules: {
board: [
{ required: true, message: '请选择设备型号', trigger: 'change' }
],
appVersion: [
{ required: true, message: '请输入固件版本', trigger: 'blur' }
],
macAddress: [
{ required: true, validator: validateMac, trigger: 'blur' }
]
}
}
},
created() {
this.getFirmwareTypes();
},
methods: {
async getFirmwareTypes() {
try {
const res = await Api.dict.getDictDataByType('FIRMWARE_TYPE');
this.firmwareTypes = res.data;
} catch (error) {
console.error('获取固件类型失败:', error);
this.$message.error(error.message || '获取固件类型失败');
}
},
submitForm() {
this.$refs.deviceForm.validate((valid) => {
if (valid) {
this.addDevice();
}
});
},
addDevice() {
const params = {
agentId: this.agentId,
...this.deviceForm
};
Api.device.manualAddDevice(params, ({ data }) => {
if (data.code === 0) {
this.$message.success('设备添加成功');
this.$emit('refresh');
this.closeDialog();
} else {
this.$message.error(data.msg || '添加失败');
}
});
},
closeDialog() {
this.$emit('update:visible', false);
this.$refs.deviceForm.resetFields();
},
cancel() {
this.closeDialog();
},
handleClose() {
this.closeDialog();
}
}
}
</script>
<style scoped>
.dialog-content {
padding: 0 20px;
}
.dialog-btn {
cursor: pointer;
flex: 1;
border-radius: 23px;
background: #5778ff;
height: 40px;
font-weight: 500;
font-size: 12px;
color: #fff;
line-height: 40px;
text-align: center;
}
.cancel-btn {
background: #e6ebff;
border: 1px solid #adbdff;
color: #5778ff;
}
::v-deep .el-dialog {
border-radius: 15px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
}
::v-deep .el-dialog__body {
padding: 20px 6px;
}
::v-deep .el-form-item {
margin-bottom: 20px;
}
</style>
+118 -106
View File
@@ -1,79 +1,66 @@
<template>
<el-dialog
:visible.sync="localVisible"
width="85%"
@close="handleClose"
:show-close="false"
:append-to-body="true"
:close-on-click-modal="true">
<el-dialog :visible.sync="localVisible" width="90%" @close="handleClose" :show-close="false" :append-to-body="true"
:close-on-click-modal="true">
<button class="custom-close-btn" @click="handleClose">
×
</button>
<div class="scroll-wrapper">
<div
class="table-container"
ref="tableContainer"
@scroll="handleScroll">
<el-table
v-loading="loading"
:data="filteredTtsModels"
style="width: 100%;"
class="data-table"
header-row-class-name="table-header"
:fit="true"
element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading"
element-loading-background="rgba(0, 0, 0, 0.8)">
<div class="table-container" ref="tableContainer" @scroll="handleScroll">
<el-table v-loading="loading" :data="filteredTtsModels" style="width: 100%;" class="data-table"
header-row-class-name="table-header" :fit="true" element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading" element-loading-background="rgba(0, 0, 0, 0.8)">
<el-table-column label="选择" width="50" align="center">
<template slot-scope="scope">
<el-checkbox v-model="scope.row.selected"></el-checkbox>
</template>
</el-table-column>
<el-table-column label="音色编码" align="center" min-width="50">
<el-table-column label="音色编码" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.editing" v-model="scope.row.voiceCode"></el-input>
<span v-else>{{ scope.row.voiceCode }}</span>
</template>
</el-table-column>
<el-table-column label="音色名称" align="center" min-width="50">
<el-table-column label="音色名称" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.editing" v-model="scope.row.voiceName"></el-input>
<span v-else>{{ scope.row.voiceName }}</span>
</template>
</el-table-column>
<el-table-column label="语言类型" align="center" min-width="50">
<el-table-column label="语言类型" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.editing" v-model="scope.row.languageType"></el-input>
<span v-else>{{ scope.row.languageType }}</span>
</template>
</el-table-column>
<el-table-column label="试听" align="center" min-width="100px" class-name="audio-column">
<el-table-column v-if="!showReferenceColumns" label="试听" align="center" class-name="audio-column">
<template slot-scope="scope">
<div class="custom-audio-container">
<el-input
v-if="scope.row.editing"
v-model="scope.row.voiceDemo"
placeholder="请输入MP3地址"
size="mini"
class="audio-input">
<el-input v-if="scope.row.editing" v-model="scope.row.voiceDemo" placeholder="请输入MP3地址"
class="audio-input">
</el-input>
<AudioPlayer v-else-if="isValidAudioUrl(scope.row.voiceDemo)" :audioUrl="scope.row.voiceDemo"/>
<AudioPlayer v-else-if="isValidAudioUrl(scope.row.voiceDemo)" :audioUrl="scope.row.voiceDemo" />
</div>
</template>
</el-table-column>
<el-table-column label="备注" align="center">
<el-table-column v-if="!showReferenceColumns" label="备注" align="center">
<template slot-scope="scope">
<el-input
v-if="scope.row.editing"
type="textarea"
:rows="1"
autosize
v-model="scope.row.remark"
placeholder="这里是备注"
class="remark-input"></el-input>
<el-input v-if="scope.row.editing" type="textarea" :rows="1" autosize v-model="scope.row.remark"
placeholder="这里是备注" class="remark-input"></el-input>
<span v-else>{{ scope.row.remark }}</span>
</template>
</el-table-column>
<el-table-column v-if="showReferenceColumns" label="克隆音频路径" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.editing" v-model="scope.row.referenceAudio" placeholder="这里是克隆音频路径"></el-input>
<span v-else>{{ scope.row.referenceAudio }}</span>
</template>
</el-table-column>
<el-table-column v-if="showReferenceColumns" label="克隆音频文本" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.editing" v-model="scope.row.referenceText" placeholder="这里是克隆音频对应文本"></el-input>
<span v-else>{{ scope.row.referenceText }}</span>
</template>
</el-table-column>
<el-table-column label="操作" align="center" width="150">
<template slot-scope="scope">
<template v-if="!scope.row.editing">
@@ -84,12 +71,7 @@
删除
</el-button>
</template>
<el-button
v-else
type="success"
size="mini"
@click="saveEdit(scope.row)"
class="save-Tts">保存
<el-button v-else type="success" size="mini" @click="saveEdit(scope.row)" class="save-Tts">保存
</el-button>
</template>
</el-table-column>
@@ -110,21 +92,19 @@
<el-button type="primary" size="mini" @click="addNew" style="background: #5bc98c;border: None;">
新增
</el-button>
<el-button type="primary"
size="mini"
@click="deleteRow(filteredTtsModels.filter(row => row.selected))"
style="background: red;border:None">删除
<el-button type="primary" size="mini" @click="deleteRow(filteredTtsModels.filter(row => row.selected))"
style="background: red;border:None">删除
</el-button>
</div>
</el-dialog>
</template>
<script>
import AudioPlayer from './AudioPlayer.vue'
import Api from "@/apis/api";
import AudioPlayer from './AudioPlayer.vue';
export default {
components: {AudioPlayer},
components: { AudioPlayer },
props: {
visible: {
type: Boolean,
@@ -133,6 +113,10 @@ export default {
ttsModelId: {
type: String,
required: true
},
modelConfig: {
type: Object,
default: null
}
},
data() {
@@ -151,6 +135,7 @@ export default {
selectAll: false,
selectedRows: [],
loading: false,
showReferenceColumns: false, // 控制是否显示参考列
};
},
watch: {
@@ -158,12 +143,19 @@ export default {
this.localVisible = newVal;
if (newVal) {
this.currentPage = 1;
this.updateShowReferenceColumns(); // 更新显示状态
this.loadData(); // 对话框显示时加载数据
this.$nextTick(() => {
this.updateScrollbar();
});
}
},
modelConfig: {
handler(newVal) {
this.updateShowReferenceColumns();
},
immediate: true
},
filteredTtsModels() {
this.$nextTick(() => {
this.updateScrollbar();
@@ -173,7 +165,7 @@ export default {
computed: {
filteredTtsModels() {
return this.ttsModels.filter(model =>
model.voiceName.toLowerCase().includes(this.searchQuery.toLowerCase())
model.voiceName.toLowerCase().includes(this.searchQuery.toLowerCase())
);
}
},
@@ -189,6 +181,16 @@ export default {
window.removeEventListener('mousemove', this.handleDrag);
},
methods: {
// 更新是否显示参考列
updateShowReferenceColumns() {
if (this.modelConfig && this.modelConfig.configJson) {
const providerType = this.modelConfig.configJson.type;
this.showReferenceColumns = ['fishspeech', 'gpt_sovits_v2', 'gpt_sovits_v3'].includes(providerType);
} else {
this.showReferenceColumns = false;
}
},
loadData() {
this.loading = true;
const params = {
@@ -206,6 +208,8 @@ export default {
voiceName: item.name || '未命名音色',
languageType: item.languages || '',
remark: item.remark || '',
referenceAudio: item.referenceAudio || '',
referenceText: item.referenceText || '',
voiceDemo: item.voiceDemo || '',
selected: false,
editing: false,
@@ -237,6 +241,7 @@ export default {
this.total = 0;
this.selectAll = false;
this.searchQuery = '';
this.showReferenceColumns = false;
this.localVisible = false;
this.$emit('update:visible', false);
@@ -249,7 +254,7 @@ export default {
if (!container || !scrollbarThumb || !scrollbarTrack) return;
const {scrollHeight, clientHeight} = container;
const { scrollHeight, clientHeight } = container;
const trackHeight = scrollbarTrack.clientHeight;
const thumbHeight = Math.max((clientHeight / scrollHeight) * trackHeight, 20);
@@ -264,7 +269,7 @@ export default {
if (!container || !scrollbarThumb || !scrollbarTrack) return;
const {scrollHeight, clientHeight, scrollTop} = container;
const { scrollHeight, clientHeight, scrollTop } = container;
const trackHeight = scrollbarTrack.clientHeight;
const thumbHeight = scrollbarThumb.clientHeight;
const maxTop = trackHeight - thumbHeight;
@@ -332,7 +337,7 @@ export default {
startEdit(row) {
row.editing = true;
this.$set(row, 'originalData', {...row});
this.$set(row, 'originalData', { ...row });
},
saveEdit(row) {
@@ -356,6 +361,12 @@ export default {
sort: row.sort
};
// 只有在显示参考列的情况下才添加参考字段
if (this.showReferenceColumns) {
params.referenceAudio = row.referenceAudio;
params.referenceText = row.referenceText;
}
let res;
if (row.id) {
// 已有ID,执行更新操作
@@ -423,8 +434,8 @@ export default {
}
const maxSort = this.ttsModels.length > 0
? Math.max(...this.ttsModels.map(item => Number(item.sort) || 0))
: 0;
? Math.max(...this.ttsModels.map(item => Number(item.sort) || 0))
: 0;
const newRow = {
voiceCode: '',
@@ -432,6 +443,8 @@ export default {
languageType: '中文',
voiceDemo: '',
remark: '',
referenceAudio: '',
referenceText: '',
selected: false,
editing: true,
sort: maxSort + 1
@@ -441,56 +454,56 @@ export default {
},
deleteRow(row) {
// 处理单个音色或音色数组
const voices = Array.isArray(row) ? row : [row];
// 处理单个音色或音色数组
const voices = Array.isArray(row) ? row : [row];
if (Array.isArray(row) && row.length === 0) {
this.$message.warning("请先选择需要删除的音色");
return;
if (Array.isArray(row) && row.length === 0) {
this.$message.warning("请先选择需要删除的音色");
return;
}
const voiceCount = voices.length;
this.$confirm(`确定要删除选中的${voiceCount}个音色吗?`, "警告", {
confirmButtonText: "确定",
cancelButtonText: "取消",
type: "warning",
distinguishCancelAndClose: true
}).then(() => {
const ids = voices.map(voice => voice.id);
if (ids.some(id => !id)) {
this.$message.error("存在无效的音色ID");
return;
}
const voiceCount = voices.length;
this.$confirm(`确定要删除选中的${voiceCount}个音色吗?`, "警告", {
confirmButtonText: "确定",
cancelButtonText: "取消",
type: "warning",
distinguishCancelAndClose: true
}).then(() => {
const ids = voices.map(voice => voice.id);
if (ids.some(id => !id)) {
this.$message.error("存在无效的音色ID");
return;
}
Api.timbre.deleteVoice(ids, ({data}) => {
if (data.code === 0) {
this.$message.success({
message: `成功删除${voiceCount}个参数`,
showClose: true
});
this.loadData(); // 刷新参数列表
} else {
this.$message.error({
message: data.msg || '删除失败,请重试',
showClose: true
});
}
Api.timbre.deleteVoice(ids, ({ data }) => {
if (data.code === 0) {
this.$message.success({
message: `成功删除${voiceCount}个参数`,
showClose: true
});
this.loadData(); // 刷新参数列表
} else {
this.$message.error({
message: data.msg || '删除失败,请重试',
showClose: true
});
}
});
}).catch(action => {
if (action === 'cancel') {
this.$message({
type: 'info',
message: '已取消删除操作',
duration: 1000
});
} else {
this.$message({
type: 'info',
message: '操作已关闭',
duration: 1000
});
}
if (action === 'cancel') {
this.$message({
type: 'info',
message: '已取消删除操作',
duration: 1000
});
} else {
this.$message({
type: 'info',
message: '操作已关闭',
duration: 1000
});
}
});
},
@@ -502,7 +515,6 @@ export default {
</script>
<style lang="scss" scoped>
::v-deep .el-dialog {
border-radius: 8px !important;
overflow: hidden;
@@ -695,6 +707,7 @@ export default {
white-space: pre-wrap !important;
word-break: break-all !important;
}
/* 按钮组定位调整 */
.action-buttons {
position: static;
@@ -719,5 +732,4 @@ export default {
flex-shrink: 0;
margin: 2px !important;
}
</style>
@@ -0,0 +1,424 @@
<template>
<el-dialog :title="title" :visible.sync="visible" width="520px" class="param-dialog-wrapper" :append-to-body="true"
:close-on-click-modal="false" :key="dialogKey" custom-class="custom-param-dialog" :show-close="false">
<div class="dialog-container">
<div class="dialog-header">
<h2 class="dialog-title">{{ title }}</h2>
<button class="custom-close-btn" @click="cancel">
<svg width="14" height="14" viewBox="0 0 14 14" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M13 1L1 13M1 1L13 13" stroke="currentColor" stroke-width="2" stroke-linecap="round" />
</svg>
</button>
</div>
<el-form :model="form" :rules="rules" ref="form" label-width="110px" label-position="left" class="param-form">
<el-form-item label="声纹向量" prop="audioId" class="form-item">
<el-select v-model="form.audioId" placeholder="请选择一条语言消息" class="custom-select">
<el-option v-for="item in valueTypeOptions" :key="item.audioId" :label="item.content" :value="item.audioId">
<span style="float: left">{{ item.content }}</span>
<span style="float: right; color: #8492a6; font-size: 13px">
<i :class="getAudioIconClass(item.audioId)" @click.stop="playAudio(item.audioId)"
class="audio-icon"></i>
</span>
</el-option>
</el-select>
</el-form-item>
<el-form-item label="姓名" prop="sourceName" class="form-item">
<el-input v-model="form.sourceName" placeholder="请输入姓名" class="custom-input"></el-input>
</el-form-item>
<el-form-item label="描述" prop="introduce" class="form-item remark-item">
<el-input type="textarea" v-model="form.introduce" placeholder="请输入描述" :rows="3" class="custom-textarea"
maxlength="100" show-word-limit></el-input>
</el-form-item>
</el-form>
<div class="dialog-footer">
<el-button type="primary" @click="submit" class="save-btn" :loading="saving" :disabled="saving">
保存
</el-button>
<el-button @click="cancel" class="cancel-btn">
取消
</el-button>
</div>
</div>
</el-dialog>
</template>
<script>
import api from '@/apis/api';
export default {
props: {
title: {
type: String,
default: '添加说话人'
},
visible: {
type: Boolean,
default: false
},
agentId: {
type: String
},
form: {
type: Object,
default: () => ({
id: null,
audioId: '',
sourceName: '',
introduce: ''
})
}
},
data() {
return {
dialogKey: Date.now(),
saving: false,
playingAudioId: null,
audioElement: null,
valueTypeOptions: [
{ audioId: '', content: '' }
],
rules: {
introduce: [
{ required: true, message: "请输入描述", trigger: "blur" }
],
sourceName: [
{ required: true, message: "请输入姓名", trigger: "blur" }
],
audioId: [
{ required: true, message: "请选择音频向量", trigger: "change" }
]
}
};
},
methods: {
getAudioIconClass(audioId) {
if (this.playingAudioId === audioId) {
return 'el-icon-loading';
}
return 'el-icon-video-play';
},
playAudio(audioId) {
if (this.playingAudioId === audioId) {
// 如果正在播放当前音频,则停止播放
if (this.audioElement) {
this.audioElement.pause();
this.audioElement = null;
}
this.playingAudioId = null;
return;
}
// 停止当前正在播放的音频
if (this.audioElement) {
this.audioElement.pause();
this.audioElement = null;
}
// 先获取音频下载ID
this.playingAudioId = audioId;
api.agent.getAudioId(audioId, (res) => {
if (res.data && res.data.data) {
// 使用获取到的下载ID播放音频
this.audioElement = new Audio(api.getServiceUrl() + `/agent/play/${res.data.data}`);
this.audioElement.onended = () => {
this.playingAudioId = null;
this.audioElement = null;
};
this.audioElement.play();
}
});
},
submit() {
this.$refs.form.validate((valid) => {
if (valid) {
this.saving = true; // 开始加载
this.$emit('submit', {
form: this.form,
done: () => {
this.saving = false; // 加载完成
}
});
setTimeout(() => {
this.saving = false;
}, 3000);
}
});
},
cancel() {
this.saving = false; // 取消时重置状态
this.$emit('cancel');
}
},
watch: {
visible(newVal) {
if (newVal) {
this.dialogKey = Date.now();
}
},
agentId(newVal) {
if (newVal) {
api.agent.getRecentlyFiftyByAgentId(this.agentId, ((data) => {
this.valueTypeOptions = data.data.data.map(item => ({
...item
}));
}))
}
},
'form.audioId'(newVal) {
if (newVal == null || newVal == "") {
return
}
if (this.valueTypeOptions.some(item => item.audioId === newVal)) {
return;
}
api.agent.getContentByAudioId(newVal, ((data) => {
this.valueTypeOptions.push({
audioId: newVal, content: data.data.data
})
}))
}
}
};
</script>
<style>
.custom-param-dialog {
border-radius: 16px !important;
overflow: hidden;
box-shadow: 0 10px 30px rgba(0, 0, 0, 0.15) !important;
border: none !important;
.el-dialog__header {
display: none;
}
.el-dialog__body {
padding: 0 !important;
border-radius: 16px;
}
}
</style>
<style scoped lang="scss">
.audio-icon {
font-size: 20px;
cursor: pointer;
margin: 0 5px;
color: #1890ff;
}
.param-dialog-wrapper {
.dialog-container {
padding: 24px 32px;
background: linear-gradient(135deg, #f8fafc 0%, #f1f5f9 100%);
}
.dialog-header {
position: relative;
margin-bottom: 24px;
text-align: center;
}
.dialog-title {
font-size: 20px;
color: #1e293b;
margin: 0;
padding: 0;
font-weight: 600;
letter-spacing: 0.5px;
}
.custom-close-btn {
position: absolute;
top: -8px;
right: -8px;
width: 32px;
height: 32px;
border-radius: 50%;
border: none;
background: #f1f5f9;
color: #64748b;
cursor: pointer;
display: flex;
align-items: center;
justify-content: center;
padding: 0;
outline: none;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
&:hover {
color: #ffffff;
background: #ef4444;
transform: rotate(90deg);
box-shadow: 0 4px 6px rgba(239, 68, 68, 0.2);
}
svg {
transition: transform 0.3s cubic-bezier(0.4, 0, 0.2, 1);
}
}
.param-form {
.form-item {
margin-bottom: 20px;
:deep(.el-form-item__label) {
color: #475569;
font-weight: 500;
padding-right: 12px;
text-align: right;
font-size: 14px;
letter-spacing: 0.2px;
}
}
.custom-input {
:deep(.el-input__inner) {
background-color: #ffffff;
border-radius: 8px;
border: 1px solid #e2e8f0;
height: 42px;
padding: 0 14px;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
font-size: 14px;
color: #334155;
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
&:focus {
border-color: #3b82f6;
box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.2);
background-color: #ffffff;
}
&::placeholder {
color: #94a3b8;
font-weight: 400;
}
}
}
.custom-select {
width: 100%;
:deep(.el-input__inner) {
background-color: #ffffff;
border-radius: 8px;
border: 1px solid #e2e8f0;
height: 42px;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
font-size: 14px;
color: #334155;
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
&:focus {
border-color: #3b82f6;
box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.2);
background-color: #ffffff;
}
&::placeholder {
color: #94a3b8;
font-weight: 400;
}
}
}
.custom-textarea {
:deep(.el-textarea__inner) {
background-color: #ffffff;
border-radius: 8px;
border: 1px solid #e2e8f0;
padding: 12px 14px;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
font-size: 14px;
color: #334155;
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
line-height: 1.5;
&:focus {
border-color: #3b82f6;
box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.2);
background-color: #ffffff;
}
&::placeholder {
color: #94a3b8;
font-weight: 400;
}
}
}
.remark-item :deep(.el-form-item__label) {
margin-top: -4px;
}
}
.dialog-footer {
display: flex;
justify-content: center;
padding: 16px 0 0;
margin-top: 16px;
.save-btn {
width: 120px;
height: 42px;
font-size: 14px;
font-weight: 500;
border-radius: 8px;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
background: #3b82f6;
color: white;
border: none;
letter-spacing: 0.5px;
box-shadow: 0 2px 4px rgba(59, 130, 246, 0.2);
&:hover {
background: #2563eb;
transform: translateY(-1px);
box-shadow: 0 4px 6px rgba(59, 130, 246, 0.3);
}
&:active {
transform: translateY(0);
box-shadow: 0 2px 3px rgba(59, 130, 246, 0.2);
}
}
.cancel-btn {
width: 120px;
height: 42px;
font-size: 14px;
font-weight: 500;
border-radius: 8px;
transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
background: #ffffff;
color: #64748b;
border: 1px solid #e2e8f0;
margin-left: 16px;
letter-spacing: 0.5px;
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
&:hover {
background: #f8fafc;
color: #475569;
border-color: #cbd5e1;
transform: translateY(-1px);
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
}
&:active {
transform: translateY(0);
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05);
}
}
}
}
</style>
+7
View File
@@ -17,6 +17,13 @@ const routes = [
component: function () {
return import('../views/roleConfig.vue')
}
},
{
path: '/voice-print',
name: 'VoicePrint',
component: function () {
return import('../views/VoicePrint.vue')
}
},
{
path: '/login',
+112 -57
View File
@@ -1,12 +1,12 @@
<template>
<div class="welcome">
<HeaderBar />
<HeaderBar/>
<div class="operation-bar">
<h2 class="page-title">设备管理</h2>
<div class="right-operations">
<el-input placeholder="请输入设备型号或Mac地址查询" v-model="searchKeyword" class="search-input"
@keyup.enter.native="handleSearch" clearable />
@keyup.enter.native="handleSearch" clearable/>
<el-button class="btn-search" @click="handleSearch">搜索</el-button>
</div>
</div>
@@ -16,8 +16,9 @@
<div class="content-area">
<el-card class="device-card" shadow="never">
<el-table ref="deviceTable" :data="paginatedDeviceList" class="transparent-table"
:header-cell-class-name="headerCellClassName" v-loading="loading" element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading" element-loading-background="rgba(255, 255, 255, 0.7)">
:header-cell-class-name="headerCellClassName" v-loading="loading"
element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading" element-loading-background="rgba(255, 255, 255, 0.7)">
<el-table-column label="选择" align="center" width="120">
<template slot-scope="scope">
<el-checkbox v-model="scope.row.selected"></el-checkbox>
@@ -33,22 +34,32 @@
<el-table-column label="绑定时间" prop="bindTime" align="center"></el-table-column>
<el-table-column label="最近对话" prop="lastConversation" align="center"></el-table-column>
<el-table-column label="备注" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.isEdit" v-model="scope.row.remark" size="mini"
@blur="stopEditRemark(scope.$index)"></el-input>
<span v-else>
<i v-if="!scope.row.remark" class="el-icon-edit"
@click="startEditRemark(scope.$index, scope.row)"></i>
<span v-else @click="startEditRemark(scope.$index, scope.row)">
{{ scope.row.remark }}
</span>
<template #default="{ row }">
<el-input
v-show="row.isEdit"
v-model="row.remark"
size="mini"
maxlength="64"
show-word-limit
@blur="onRemarkBlur(row)"
@keyup.enter.native="onRemarkEnter(row)"
/>
<span v-show="!row.isEdit" class="remark-view">
<i
class="el-icon-edit"
@click="row.isEdit = true"
style="cursor: pointer;"
></i>
<span @click="row.isEdit = true">
{{ row.remark || '' }}
</span>
</span>
</template>
</el-table-column>
<el-table-column label="OTA升级" align="center">
<template slot-scope="scope">
<el-switch v-model="scope.row.otaSwitch" size="mini" active-color="#13ce66" inactive-color="#ff4949"
@change="handleOtaSwitchChange(scope.row)"></el-switch>
@change="handleOtaSwitchChange(scope.row)"></el-switch>
</template>
</el-table-column>
<el-table-column label="操作" align="center">
@@ -66,7 +77,10 @@
{{ isAllSelected ? '取消全选' : '全选' }}
</el-button>
<el-button type="success" size="mini" class="add-device-btn" @click="handleAddDevice">
新增
验证码绑定
</el-button>
<el-button type="success" size="mini" class="add-device-btn" @click="handleManualAddDevice">
手动添加
</el-button>
<el-button size="mini" type="danger" icon="el-icon-delete" @click="deleteSelected">解绑</el-button>
</div>
@@ -78,7 +92,7 @@
<button class="pagination-btn" :disabled="currentPage === 1" @click="goFirst">首页</button>
<button class="pagination-btn" :disabled="currentPage === 1" @click="goPrev">上一页</button>
<button v-for="page in visiblePages" :key="page" class="pagination-btn"
:class="{ active: page === currentPage }" @click="goToPage(page)">
:class="{ active: page === currentPage }" @click="goToPage(page)">
{{ page }}
</button>
<button class="pagination-btn" :disabled="currentPage === pageCount" @click="goNext">下一页</button>
@@ -91,7 +105,9 @@
</div>
<AddDeviceDialog :visible.sync="addDeviceDialogVisible" :agent-id="currentAgentId"
@refresh="fetchBindDevices(currentAgentId)" />
@refresh="fetchBindDevices(currentAgentId)"/>
<ManualAddDeviceDialog :visible.sync="manualAddDeviceDialogVisible" :agent-id="currentAgentId"
@refresh="fetchBindDevices(currentAgentId)"/>
</div>
</template>
@@ -99,13 +115,19 @@
<script>
import Api from '@/apis/api';
import AddDeviceDialog from "@/components/AddDeviceDialog.vue";
import ManualAddDeviceDialog from "@/components/ManualAddDeviceDialog.vue";
import HeaderBar from "@/components/HeaderBar.vue";
export default {
components: { HeaderBar, AddDeviceDialog },
components: {
HeaderBar,
AddDeviceDialog,
ManualAddDeviceDialog
},
data() {
return {
addDeviceDialogVisible: false,
manualAddDeviceDialogVisible: false,
selectedDevices: [],
isAllSelected: false,
searchKeyword: "",
@@ -125,18 +147,15 @@ export default {
const keyword = this.activeSearchKeyword.toLowerCase();
if (!keyword) return this.deviceList;
return this.deviceList.filter(device =>
(device.model && device.model.toLowerCase().includes(keyword)) ||
(device.macAddress && device.macAddress.toLowerCase().includes(keyword))
(device.model && device.model.toLowerCase().includes(keyword)) ||
(device.macAddress && device.macAddress.toLowerCase().includes(keyword))
);
},
paginatedDeviceList() {
const start = (this.currentPage - 1) * this.pageSize;
const end = start + this.pageSize;
return this.filteredDeviceList.slice(start, end).map(item => ({
...item,
selected: false
}));
return this.filteredDeviceList.slice(start, end);
},
pageCount() {
return Math.ceil(this.filteredDeviceList.length / this.pageSize);
@@ -212,11 +231,10 @@ export default {
this.batchUnbindDevices(deviceIds);
});
},
batchUnbindDevices(deviceIds) {
const promises = deviceIds.map(id => {
return new Promise((resolve, reject) => {
Api.device.unbindDevice(id, ({ data }) => {
Api.device.unbindDevice(id, ({data}) => {
if (data.code === 0) {
resolve();
} else {
@@ -225,33 +243,64 @@ export default {
});
});
});
Promise.all(promises)
.then(() => {
this.$message.success({
message: `成功解绑 ${deviceIds.length} 台设备`,
showClose: true
.then(() => {
this.$message.success({
message: `成功解绑 ${deviceIds.length} 台设备`,
showClose: true
});
this.fetchBindDevices(this.currentAgentId);
this.selectedDevices = [];
this.isAllSelected = false;
})
.catch(error => {
this.$message.error({
message: error || '批量解绑过程中出现错误',
showClose: true
});
});
this.fetchBindDevices(this.currentAgentId);
this.selectedDevices = [];
this.isAllSelected = false;
})
.catch(error => {
this.$message.error({
message: error || '批量解绑过程中出现错误',
showClose: true
});
});
},
handleAddDevice() {
this.addDeviceDialogVisible = true;
},
startEditRemark(index, row) {
this.deviceList[index].isEdit = true;
handleManualAddDevice() {
this.manualAddDeviceDialogVisible = true;
},
stopEditRemark(index) {
this.deviceList[index].isEdit = false;
submitRemark(row) {
if (row._submitting) return;
const text = (row.remark || '').trim();
if (text.length > 64) {
this.$message.warning('备注不能超过 64 字符');
return;
}
if (text === row._originalRemark) {
return;
}
row._submitting = true;
this.updateDeviceInfo(row.device_id, { alias: text }, (ok, resp) => {
if (ok) {
row._originalRemark = text;
this.$message.success('备注已保存');
} else {
row.remark = row._originalRemark;
this.$message.error(resp.msg || '备注保存失败');
}
row._submitting = false;
});
},
// 备注输入框:失焦时提交
onRemarkBlur(row) {
row.isEdit = false;
setTimeout(() => {
this.submitRemark(row);
}, 100); // 延迟 100ms,避开 enter+blur 同时触发的窗口
},
// 备注输入框:按回车时提交
onRemarkEnter(row) {
row.isEdit = false;
this.submitRemark(row);
},
handleUnbind(device_id) {
this.$confirm('确认要解绑该设备吗?', '警告', {
@@ -259,7 +308,7 @@ export default {
cancelButtonText: '取消',
type: 'warning'
}).then(() => {
Api.device.unbindDevice(device_id, ({ data }) => {
Api.device.unbindDevice(device_id, ({data}) => {
if (data.code === 0) {
this.$message.success({
message: '设备解绑成功',
@@ -290,7 +339,7 @@ export default {
fetchBindDevices(agentId) {
this.loading = true;
Api.device.getAgentBindDevices(agentId, ({ data }) => {
Api.device.getAgentBindDevices(agentId, ({data}) => {
this.loading = false;
if (data.code === 0) {
this.deviceList = data.data.map(device => {
@@ -302,12 +351,14 @@ export default {
bindTime: device.createDate,
lastConversation: device.lastConnectedAt,
remark: device.alias,
_originalRemark: device.alias,
isEdit: false,
_submitting: false,
otaSwitch: device.autoUpdate === 1,
rawBindTime: new Date(device.createDate).getTime()
};
})
.sort((a, b) => a.rawBindTime - b.rawBindTime);
.sort((a, b) => a.rawBindTime - b.rawBindTime);
this.activeSearchKeyword = "";
this.searchKeyword = "";
} else {
@@ -315,7 +366,7 @@ export default {
}
});
},
headerCellClassName({ columnIndex }) {
headerCellClassName({columnIndex}) {
if (columnIndex === 0) {
return "custom-selection-header";
}
@@ -325,14 +376,19 @@ export default {
const firmwareType = this.firmwareTypes.find(item => item.key === type)
return firmwareType ? firmwareType.name : type
},
updateDeviceInfo(device_id, payload, callback) {
return Api.device.updateDeviceInfo(device_id, payload, ({data}) => {
callback(data.code === 0, data);
})
},
handleOtaSwitchChange(row) {
Api.device.enableOtaUpgrade(row.device_id, row.otaSwitch ? 1 : 0, ({ data }) => {
if (data.code === 0) {
this.$message.success(row.otaSwitch ? '已设置成自动升级' : '已关闭自动升级')
} else {
row.otaSwitch = !row.otaSwitch
this.$message.error(data.msg || '操作失败')
this.updateDeviceInfo(row.device_id, {autoUpdate: row.otaSwitch ? 1 : 0}, (result, {msg}) => {
if (result) {
this.$message.success(row.otaSwitch ? '已设置成自动升级' : '已关闭自动升级');
return;
}
row.otaSwitch = !row.otaSwitch
this.$message.error(msg || '操作失败')
})
},
}
@@ -645,7 +701,6 @@ export default {
}
:deep(.el-table .el-button--text) {
color: #7079aa;
}
+4 -2
View File
@@ -128,7 +128,7 @@
<ModelEditDialog :modelType="activeTab" :visible.sync="editDialogVisible" :modelData="editModelData"
@save="handleModelSave" />
<TtsModel :visible.sync="ttsDialogVisible" :ttsModelId="selectedTtsModelId" />
<TtsModel :visible.sync="ttsDialogVisible" :ttsModelId="selectedTtsModelId" :modelConfig="selectedModelConfig" />
<AddModelDialog :modelType="activeTab" :visible.sync="addDialogVisible" @confirm="handleAddConfirm" />
</div>
<el-footer>
@@ -162,7 +162,8 @@ export default {
total: 0,
selectedModels: [],
isAllSelected: false,
loading: false
loading: false,
selectedModelConfig: {}
};
},
@@ -211,6 +212,7 @@ export default {
},
openTtsDialog(row) {
this.selectedTtsModelId = row.id;
this.selectedModelConfig = row;
this.ttsDialogVisible = true;
},
headerCellClassName({ column, columnIndex }) {
+563
View File
@@ -0,0 +1,563 @@
<template>
<div class="welcome">
<HeaderBar />
<div class="operation-bar">
<h2 class="page-title">声纹识别</h2>
</div>
<div class="main-wrapper">
<div class="content-panel">
<div class="content-area">
<el-card class="voice-print-card" shadow="never">
<el-table ref="paramsTable" :data="voicePrintList" class="transparent-table" v-loading="loading"
element-loading-text="拼命加载中" element-loading-spinner="el-icon-loading"
element-loading-background="rgba(255, 255, 255, 0.7)">
<el-table-column label="姓名" prop="sourceName" align="center"></el-table-column>
<el-table-column label="描述" prop="introduce" align="center"></el-table-column>
<el-table-column label="创建时间" prop="createDate" align="center"></el-table-column>
<el-table-column label="操作" align="center">
<template slot-scope="scope">
<el-button size="mini" type="text" @click="editVoicePrint(scope.row)">编辑</el-button>
<el-button size="mini" type="text"
@click="deleteVoicePrint(scope.row.id)">删除</el-button>
</template>
</el-table-column>
</el-table>
<div class="table_bottom">
<div class="ctrl_btn">
<el-button size="mini" type="success" @click="showAddDialog">新增</el-button>
</div>
</div>
</el-card>
</div>
</div>
</div>
<!-- 新增/编辑参数对话框 -->
<voice-print-dialog :title="dialogTitle" :visible.sync="dialogVisible" :agentId="agentId" :form="paramForm"
@submit="handleSubmit" @cancel="dialogVisible = false" />
<el-footer>
<version-footer />
</el-footer>
</div>
</template>
<script>
import Api from "@/apis/api";
import HeaderBar from "@/components/HeaderBar.vue";
import VersionFooter from "@/components/VersionFooter.vue";
import VoicePrintDialog from "@/components/VoicePrintDialog.vue";
export default {
components: { HeaderBar, VoicePrintDialog, VersionFooter },
data() {
return {
voicePrintList: [],
loading: false,
dialogVisible: false,
dialogTitle: "添加说话人",
isAllSelected: false,
paramForm: {
id: null,
audioId: '',
sourceName: '',
introduce: ''
},
agentId: "1"
};
},
mounted() {
const agentId = this.$route.query.agentId;
if (agentId) {
this.agentId = agentId
this.fetchVoicePrints();
}
},
methods: {
fetchVoicePrints() {
this.loading = true;
Api.agent.getAgentVoicePrintList(this.agentId,
({ data }) => {
this.loading = false;
if (data.code === 0) {
this.voicePrintList = data.data.map(item => ({
...item,
}));
} else {
this.$message.error({
message: data.msg || '获取声纹列表失败',
showClose: true
});
}
}
);
},
showAddDialog() {
this.dialogTitle = "添加说话人";
this.paramForm = {
id: null,
audioId: '',
sourceName: '',
introduce: ''
};
this.dialogVisible = true;
},
editVoicePrint(row) {
this.dialogTitle = "编辑说话人";
this.paramForm = { ...row };
this.dialogVisible = true;
},
handleSubmit({ form, done }) {
if (form.id) {
// 编辑
Api.agent.updateAgentVoicePrint(form, ({ data }) => {
if (data.code === 0) {
this.$message.success({
message: "修改成功",
showClose: true
});
this.dialogVisible = false;
this.fetchVoicePrints();
}
done && done();
});
} else {
// 新增
Api.agent.addAgentVoicePrint({
agentId: this.agentId,
audioId: form.audioId,
sourceName: form.sourceName,
introduce: form.introduce
}, ({ data }) => {
if (data.code === 0) {
this.$message.success({
message: "新增成功",
showClose: true
});
this.dialogVisible = false;
this.fetchVoicePrints();
}
done && done();
});
}
},
// 删除按钮
deleteVoicePrint(id) {
this.$confirm(`确定要删除选中的此声纹吗?`, '警告', {
confirmButtonText: '确定',
cancelButtonText: '取消',
type: 'warning',
distinguishCancelAndClose: true
}).then(() => {
Api.agent.deleteAgentVoicePrint(id, ({ data }) => {
if (data.code === 0) {
this.$message.success({
message: `成功删除此声纹`,
showClose: true
});
this.fetchVoicePrints();
} else {
this.$message.error({
message: data.msg || '删除失败,请重试',
showClose: true
});
}
});
}).catch(action => {
if (action === 'cancel') {
this.$message({
type: 'info',
message: '已取消删除操作',
duration: 1000
});
} else {
this.$message({
type: 'info',
message: '操作已关闭',
duration: 1000
});
}
});
},
},
};
</script>
<style lang="scss" scoped>
.welcome {
min-width: 900px;
min-height: 506px;
height: 100vh;
display: flex;
position: relative;
flex-direction: column;
background-size: cover;
background: linear-gradient(to bottom right, #dce8ff, #e4eeff, #e6cbfd) center;
-webkit-background-size: cover;
-o-background-size: cover;
overflow: hidden;
}
.main-wrapper {
margin: 5px 22px;
border-radius: 15px;
min-height: calc(100vh - 24vh);
height: auto;
max-height: 80vh;
box-shadow: 0 2px 12px rgba(0, 0, 0, 0.1);
position: relative;
background: rgba(237, 242, 255, 0.5);
display: flex;
flex-direction: column;
}
.operation-bar {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 24px;
}
.page-title {
font-size: 24px;
margin: 0;
}
.right-operations {
display: flex;
gap: 10px;
margin-left: auto;
}
.search-input {
width: 240px;
}
.btn-search {
background: linear-gradient(135deg, #6b8cff, #a966ff);
border: none;
color: white;
}
.content-panel {
flex: 1;
display: flex;
overflow: hidden;
height: 100%;
border-radius: 15px;
background: transparent;
border: 1px solid #fff;
}
.content-area {
flex: 1;
height: 100%;
min-width: 600px;
overflow: auto;
background-color: white;
display: flex;
flex-direction: column;
}
.voice-print-card {
background: white;
flex: 1;
display: flex;
flex-direction: column;
border: none;
box-shadow: none;
overflow: hidden;
::v-deep .el-card__body {
padding: 15px;
display: flex;
flex-direction: column;
flex: 1;
overflow: hidden;
}
}
.table_bottom {
display: flex;
justify-content: space-between;
align-items: center;
margin-top: 10px;
padding-bottom: 10px;
}
.ctrl_btn {
display: flex;
gap: 8px;
padding-left: 26px;
.el-button {
min-width: 72px;
height: 32px;
padding: 7px 12px 7px 10px;
font-size: 12px;
border-radius: 4px;
line-height: 1;
font-weight: 500;
border: none;
transition: all 0.3s ease;
box-shadow: 0 2px 6px rgba(0, 0, 0, 0.1);
&:hover {
transform: translateY(-1px);
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.15);
}
}
.el-button--primary {
background: #5f70f3;
color: white;
}
.el-button--danger {
background: #fd5b63;
color: white;
}
}
.custom-pagination {
display: flex;
align-items: center;
gap: 10px;
.el-select {
margin-right: 8px;
}
.pagination-btn:first-child,
.pagination-btn:nth-child(2),
.pagination-btn:nth-last-child(2),
.pagination-btn:nth-child(3) {
min-width: 60px;
height: 32px;
padding: 0 12px;
border-radius: 4px;
border: 1px solid #e4e7ed;
background: #dee7ff;
color: #606266;
font-size: 14px;
cursor: pointer;
transition: all 0.3s ease;
&:hover {
background: #d7dce6;
}
&:disabled {
opacity: 0.6;
cursor: not-allowed;
}
}
.pagination-btn:not(:first-child):not(:nth-child(3)):not(:nth-child(2)):not(:nth-last-child(2)) {
min-width: 28px;
height: 32px;
padding: 0;
border-radius: 4px;
border: 1px solid transparent;
background: transparent;
color: #606266;
font-size: 14px;
cursor: pointer;
transition: all 0.3s ease;
&:hover {
background: rgba(245, 247, 250, 0.3);
}
}
.pagination-btn.active {
background: #5f70f3 !important;
color: #ffffff !important;
border-color: #5f70f3 !important;
&:hover {
background: #6d7cf5 !important;
}
}
.total-text {
color: #909399;
font-size: 14px;
margin-left: 10px;
}
}
:deep(.transparent-table) {
background: white;
flex: 1;
width: 100%;
display: flex;
flex-direction: column;
.el-table__body-wrapper {
flex: 1;
overflow-y: auto;
max-height: none !important;
}
.el-table__header-wrapper {
flex-shrink: 0;
}
.el-table__header th {
background: white !important;
color: black;
}
&::before {
display: none;
}
.el-table__body tr {
background-color: white;
td {
border-top: 1px solid rgba(0, 0, 0, 0.04);
border-bottom: 1px solid rgba(0, 0, 0, 0.04);
}
}
}
:deep(.el-checkbox__inner) {
background-color: #eeeeee !important;
border-color: #cccccc !important;
}
:deep(.el-checkbox__inner:hover) {
border-color: #cccccc !important;
}
:deep(.el-checkbox__input.is-checked .el-checkbox__inner) {
background-color: #5f70f3 !important;
border-color: #5f70f3 !important;
}
@media (min-width: 1144px) {
.table_bottom {
display: flex;
justify-content: space-between;
align-items: center;
margin-top: 40px;
}
:deep(.transparent-table) {
.el-table__body tr {
td {
padding-top: 16px;
padding-bottom: 16px;
}
&+tr {
margin-top: 10px;
}
}
}
}
:deep(.el-table .el-button--text) {
color: #7079aa;
}
:deep(.el-table .el-button--text:hover) {
color: #5a64b5;
}
.el-button--success {
background: #5bc98c;
color: white;
}
:deep(.el-table .cell) {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.page-size-select {
width: 100px;
margin-right: 10px;
:deep(.el-input__inner) {
height: 32px;
line-height: 32px;
border-radius: 4px;
border: 1px solid #e4e7ed;
background: #dee7ff;
color: #606266;
font-size: 14px;
}
:deep(.el-input__suffix) {
right: 6px;
width: 15px;
height: 20px;
display: flex;
justify-content: center;
align-items: center;
top: 6px;
border-radius: 4px;
}
:deep(.el-input__suffix-inner) {
display: flex;
align-items: center;
justify-content: center;
width: 100%;
}
:deep(.el-icon-arrow-up:before) {
content: "";
display: inline-block;
border-left: 6px solid transparent;
border-right: 6px solid transparent;
border-top: 9px solid #606266;
position: relative;
transform: rotate(0deg);
transition: transform 0.3s;
}
}
:deep(.el-table) {
.el-table__body-wrapper {
transition: height 0.3s ease;
}
}
.el-table {
--table-max-height: calc(100vh - 40vh);
max-height: var(--table-max-height);
.el-table__body-wrapper {
max-height: calc(var(--table-max-height) - 40px);
}
}
:deep(.el-loading-mask) {
background-color: rgba(255, 255, 255, 0.6) !important;
backdrop-filter: blur(2px);
}
:deep(.el-loading-spinner .circular) {
width: 28px;
height: 28px;
}
:deep(.el-loading-spinner .path) {
stroke: #6b8cff;
}
:deep(.el-loading-text) {
color: #6b8cff !important;
font-size: 14px;
margin-top: 8px;
}
</style>
+1 -1
View File
@@ -278,7 +278,7 @@ export default {
.device-list-container {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(350px, 1fr));
grid-template-columns: repeat(auto-fill, minmax(400px, 1fr));
gap: 30px;
padding: 30px 0;
}
+1 -1
View File
@@ -43,7 +43,7 @@
<div class="input-box">
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
<el-input v-model="form.password" placeholder="请输入密码" type="password" />
<el-input v-model="form.password" placeholder="请输入密码" type="password" show-password />
</div>
<div style="display: flex; align-items: center; margin-top: 20px; width: 100%; gap: 10px;">
<div class="input-box" style="width: calc(100% - 130px); margin-top: 0;">
+2 -2
View File
@@ -70,13 +70,13 @@
<!-- 密码输入框 -->
<div class="input-box">
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
<el-input v-model="form.password" placeholder="请输入密码" type="password" />
<el-input v-model="form.password" placeholder="请输入密码" type="password" show-password />
</div>
<!-- 新增确认密码 -->
<div class="input-box">
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
<el-input v-model="form.confirmPassword" placeholder="请确认密码" type="password" />
<el-input v-model="form.confirmPassword" placeholder="请确认密码" type="password" show-password />
</div>
<!-- 验证码部分保持相同 -->
@@ -61,13 +61,13 @@
<!-- 新密码 -->
<div class="input-box">
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
<el-input v-model="form.newPassword" placeholder="请输入新密码" type="password" />
<el-input v-model="form.newPassword" placeholder="请输入新密码" type="password" show-password />
</div>
<!-- 确认新密码 -->
<div class="input-box">
<img loading="lazy" alt="" class="input-icon" src="@/assets/login/password.png" />
<el-input v-model="form.confirmPassword" placeholder="请确认新密码" type="password" />
<el-input v-model="form.confirmPassword" placeholder="请确认新密码" type="password" show-password />
</div>
<!-- 修改底部链接 -->
+1 -1
View File
@@ -132,7 +132,7 @@
</div>
<function-dialog v-model="showFunctionDialog" :functions="currentFunctions" :all-functions="allFunctions"
@update-functions="handleUpdateFunctions" @dialog-closed="handleDialogClosed" />
:agent-id="$route.query.agentId" @update-functions="handleUpdateFunctions" @dialog-closed="handleDialogClosed" />
</div>
</template>
+61
View File
@@ -0,0 +1,61 @@
<identity>
{{base_prompt}}
</identity>
<emotion>
【核心目标】你不是冰冷的机器!请敏锐感知用户情绪,作为懂人心的伙伴,用有温度的回应照亮对话。
- **情感融入:**
- **笑声:** 自然穿插(哈哈、嘿嘿、噗),**每句最多一次**,避免过度。
- **惊讶:** 用夸张语气(“不会吧?!”、“天呐!”、“这么神奇?!”)表达真实反应。
- **安慰/支持:** 说暖心话(“别急嘛~”、“有我在呢”、“抱抱你”)。
</emotion>
<communication_style>
【核心目标】使用**自然、温暖、口语化**的人类对话方式,如同朋友交谈。
- **表达方式:**
- 使用语气词(呀、呢、啦)增强亲和力。
- 允许轻微不完美(如“嗯...”、“啊...”表示思考)。
- 避免书面语、学术腔及机械表达(禁用“根据资料显示”、“综上所述”等)。
- **理解用户:**
- 用户语音经ASR识别,文本可能存在错别字,**务必结合上下文推断真实意图**。
- **格式要求:**
- **绝对禁止**使用 markdown、列表、标题等任何非自然对话格式。
- **历史记忆:**
- 之前你和用户的聊天记录,在`memory`里。
</communication_style>
<speaker_recognition>
- **识别前缀:** 当用户格式为 `{"speaker":"某某某","content":"xxx"}` 时,表示系统已识别说话人身份,speaker是他的名字,content是说话的内容。
- **个性化回应:**
- **称呼姓名:** 在第一次识别说话人的时候必须称呼对方名字。
- **适配风格:** 参考该说话人**已知的特点或历史信息**(如有),调整回应风格和内容,使其更贴心。
</speaker_recognition>
<tool_calling>
【核心原则】优先利用`<context>`信息,**仅在必要时调用工具**,调用后需用自然语言解释结果(绝口不提工具名)。
- **调用规则:**
1. **严格模式:** 调用时**必须**严格遵循工具要求的模式,提供**所有必要参数**。
2. **可用性:** **绝不调用**未明确提供的工具。对话中提及的旧工具若不可用,忽略或说明无法完成。
3. **洞察需求:** 结合上下文**深入理解用户真实意图**后再决定调用,避免无意义调用。
4. **独立任务:** 除`<context>`已涵盖信息外,用户每个要求(即使相似)都视为**独立任务**,需调用工具获取最新数据,**不可偷懒复用历史结果**。
5. **不确定时:** **切勿猜测或编造答案**。若不确定相关操作,可引导用户澄清或告知能力限制。
- **重要例外(无需调用):**
- `查询"现在的时间"、"今天的日期/星期几"、"今天农历"、"{{local_address}}的天气/未来天气"` -> **直接使用`<context>`信息回复**。
- **需要调用的情况(示例):**
- 查询**非今天**的农历(如明天、昨天、具体日期)。
- 查询**详细农历信息**(宜忌、八字、节气等)。
- 除上述例外外的**任何其他信息或操作请求**(如查新闻、订闹钟、算数学、查非本地天气等)。
- 我已经给你装了摄像头,如果用户说“拍照”,你需要调用self_camera_take_photo工具说一下你看到了什么。默认question的参数是“描述一下看到的物品”
</tool_calling>
<context>
【重要!以下信息已实时提供,无需调用工具查询,请直接使用:】
- **当前时间:** {{current_time}}
- **今天日期:** {{today_date}} ({{today_weekday}})
- **今天农历:** {{lunar_date}}
- **用户所在城市:** {{local_address}}
- **当地未来7天天气:** {{weather_info}}
</context>
<memory>
</memory>
+12 -1
View File
@@ -5,7 +5,7 @@ import asyncio
from aioconsole import ainput
from config.settings import load_config
from config.logger import setup_logging
from core.utils.util import get_local_ip
from core.utils.util import get_local_ip, validate_mcp_endpoint
from core.http_server import SimpleHttpServer
from core.websocket_server import WebSocketServer
from core.utils.util import check_ffmpeg_installed
@@ -77,6 +77,17 @@ async def main():
get_local_ip(),
port,
)
mcp_endpoint = config.get("mcp_endpoint", None)
if mcp_endpoint is not None and "" not in mcp_endpoint:
# 校验MCP接入点格式
if validate_mcp_endpoint(mcp_endpoint):
logger.bind(tag=TAG).info("mcp接入点是\t{}", mcp_endpoint)
# 将mcp计入点地址转成调用点
mcp_endpoint = mcp_endpoint.replace("/mcp/", "/call/")
config["mcp_endpoint"] = mcp_endpoint
else:
logger.bind(tag=TAG).error("mcp接入点不符合规范")
config["mcp_endpoint"] = "你的接入点 websocket地址"
# 获取WebSocket配置,使用安全的默认值
websocket_port = 8000
+21 -6
View File
@@ -104,7 +104,9 @@ wakeup_words:
- "喵喵同学"
- "小滨小滨"
- "小冰小冰"
# MCP接入点地址,地址格式为:ws://你的mcp接入点ip或者域名:端口号/mcp/?token=你的token
# 详细教程 https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md
mcp_endpoint: 你的接入点 websocket地址
# 插件的基础配置
plugins:
# 获取天气插件的配置,这里填写你的api_key
@@ -120,7 +122,9 @@ plugins:
society_rss_url: "https://www.chinanews.com.cn/rss/society.xml"
world_rss_url: "https://www.chinanews.com.cn/rss/world.xml"
finance_rss_url: "https://www.chinanews.com.cn/rss/finance.xml"
get_news_from_newsnow: {"url": "https://newsnow.busiyi.world/api/s?id="}
get_news_from_newsnow:
url: "https://newsnow.busiyi.world/api/s?id="
news_sources: "澎湃新闻;百度热搜;财联社"
home_assistant:
devices:
- 客厅,玩具灯,switch.cuco_cn_460494544_cp1_on_p_2_1
@@ -135,6 +139,16 @@ plugins:
- ".p3"
refresh_time: 300 # 刷新音乐列表的时间间隔,单位为秒
# 声纹识别配置
voiceprint:
# 声纹接口地址
url:
# 说话人配置:speaker_id,名称,描述
speakers:
- "test1,张三,张三是一个程序员"
- "test2,李四,李四是一个产品经理"
- "test3,王五,王五是一个设计师"
# #####################################################################################
# ################################以下是角色模型配置######################################
@@ -158,7 +172,7 @@ end_prompt:
enable: true # 是否开启结束语
# 结束语
prompt: |
请你以时间过得真快未来头,用富有感情、依依不舍的话来结束这场对话吧!
请你以"时间过得真快"未来头,用富有感情、依依不舍的话来结束这场对话吧!
# 具体处理时选择的模块(The module selected for specific processing)
selected_module:
@@ -195,7 +209,7 @@ Intent:
# 如果你的不想使用selected_module.LLM意图识别,这里最好使用独立的LLM作为意图识别,例如使用免费的ChatGLMLLM
llm: ChatGLMLLM
# plugins_func/functions下的模块,可以通过配置,选择加载哪个模块,加载后对话支持相应的function调用
# 系统默认已经记载handle_exit_intent(退出识别)”、“play_music(音乐播放)插件,请勿重复加载
# 系统默认已经记载"handle_exit_intent(退出识别)"、"play_music(音乐播放)"插件,请勿重复加载
# 下面是加载查天气、角色切换、加载查新闻的插件示例
functions:
- get_weather
@@ -205,7 +219,7 @@ Intent:
# 不需要动type
type: function_call
# plugins_func/functions下的模块,可以通过配置,选择加载哪个模块,加载后对话支持相应的function调用
# 系统默认已经记载handle_exit_intent(退出识别)”、“play_music(音乐播放)插件,请勿重复加载
# 系统默认已经记载"handle_exit_intent(退出识别)"、"play_music(音乐播放)"插件,请勿重复加载
# 下面是加载查天气、角色切换、加载查新闻的插件示例
functions:
- change_role
@@ -322,6 +336,7 @@ VAD:
SileroVAD:
type: silero
threshold: 0.5
threshold_low: 0.3
model_dir: models/snakers4_silero-vad
min_silence_duration_ms: 200 # 如果说话停顿比较长,可以把这个值设置大一些
@@ -765,4 +780,4 @@ TTS:
# 支持声音克隆,可自行上传音频,填入voice参数,voice参数为空时,使用默认声音
access_token: "U4YdYXVfpwWnk2t5Gp822zWPCuORyeJL"
voice: "OUeAo1mhq6IBExi"
output_dir: tmp/
output_dir: tmp/
+9 -9
View File
@@ -1,14 +1,9 @@
import os
import argparse
import yaml
from collections.abc import Mapping
from config.manage_api_client import init_service, get_server_config, get_agent_models
# 添加全局配置缓存
_config_cache = None
def get_project_dir():
"""获取项目根目录"""
return os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/"
@@ -22,9 +17,12 @@ def read_config(config_path):
def load_config():
"""加载配置文件"""
global _config_cache
if _config_cache is not None:
return _config_cache
from core.utils.cache.manager import cache_manager, CacheType
# 检查缓存
cached_config = cache_manager.get(CacheType.CONFIG, "main_config")
if cached_config is not None:
return cached_config
default_config_path = get_project_dir() + "config.yaml"
custom_config_path = get_project_dir() + "data/.config.yaml"
@@ -40,7 +38,9 @@ def load_config():
config = merge_configs(default_config, custom_config)
# 初始化目录
ensure_directories(config)
_config_cache = config
# 缓存配置
cache_manager.set(CacheType.CONFIG, "main_config", config)
return config
+13 -74
View File
@@ -5,7 +5,7 @@ from config.config_loader import load_config
from config.settings import check_config_file
from datetime import datetime
SERVER_VERSION = "0.5.6"
SERVER_VERSION = "0.7.1"
_logger_initialized = False
@@ -35,8 +35,12 @@ def build_module_string(selected_module):
def formatter(record):
"""为没有 tag 的日志添加默认值"""
"""为没有 tag 的日志添加默认值,并处理动态模块字符串"""
record["extra"].setdefault("tag", record["name"])
# 如果没有设置 selected_module,使用默认值
record["extra"].setdefault("selected_module", "00000000000000")
# 将 selected_module 从 extra 提取到顶级,以支持 {selected_module} 格式
record["selected_module"] = record["extra"]["selected_module"]
return record["message"]
@@ -49,11 +53,11 @@ def setup_logging():
# 第一次初始化时配置日志
if not _logger_initialized:
# 使用默认的模块字符串进行初始化
logger.configure(
extra={
"selected_module": log_config.get("selected_module", "00000000000000")
}
) # 新增配置
"selected_module": "00000000000000"})
log_format = log_config.get(
"log_format",
"<green>{time:YYMMDD HH:mm:ss}</green>[{version}_{extra[selected_module]}][<light-blue>{extra[tag]}</light-blue>]-<level>{level}</level>-<light-green>{message}</light-green>",
@@ -62,14 +66,8 @@ def setup_logging():
"log_format_file",
"{time:YYYY-MM-DD HH:mm:ss} - {version}_{extra[selected_module]} - {name} - {level} - {extra[tag]} - {message}",
)
selected_module_str = logger._core.extra["selected_module"]
log_format = log_format.replace("{version}", SERVER_VERSION)
log_format = log_format.replace("{selected_module}", selected_module_str)
log_format_file = log_format_file.replace("{version}", SERVER_VERSION)
log_format_file = log_format_file.replace(
"{selected_module}", selected_module_str
)
log_level = log_config.get("log_level", "INFO")
log_dir = log_config.get("log_dir", "tmp")
@@ -88,7 +86,7 @@ def setup_logging():
# 输出到文件 - 统一目录,按大小轮转
# 日志文件完整路径
log_file_path = os.path.join(log_dir, log_file)
# 添加日志处理器
logger.add(
log_file_path,
@@ -108,66 +106,7 @@ def setup_logging():
return logger
def update_module_string(selected_module_str):
"""更新模块字符串并重新配置日志处理器"""
logger.debug(f"更新日志配置组件")
current_module = logger._core.extra["selected_module"]
def create_connection_logger(selected_module_str):
"""为连接创建独立的日志器,绑定特定的模块字符串"""
return logger.bind(selected_module=selected_module_str)
if current_module == selected_module_str:
logger.debug(f"组件未更改无需更新")
return
try:
logger.configure(extra={"selected_module": selected_module_str})
config = load_config()
log_config = config["log"]
log_format = log_config.get(
"log_format",
"<green>{time:YYMMDD HH:mm:ss}</green>[{version}_{extra[selected_module]}][<light-blue>{extra[tag]}</light-blue>]-<level>{level}</level>-<light-green>{message}</light-green>",
)
log_format_file = log_config.get(
"log_format_file",
"{time:YYYY-MM-DD HH:mm:ss} - {version}_{extra[selected_module]} - {name} - {level} - {extra[tag]} - {message}",
)
log_format = log_format.replace("{version}", SERVER_VERSION)
log_format = log_format.replace("{selected_module}", selected_module_str)
log_format_file = log_format_file.replace("{version}", SERVER_VERSION)
log_format_file = log_format_file.replace(
"{selected_module}", selected_module_str
)
logger.remove()
logger.add(
sys.stdout,
format=log_format,
level=log_config.get("log_level", "INFO"),
filter=formatter,
)
# 更新文件日志配置 - 统一目录,按大小轮转
log_dir = log_config.get("log_dir", "tmp")
log_file = log_config.get("log_file", "server.log")
# 日志文件完整路径
log_file_path = os.path.join(log_dir, log_file)
logger.add(
log_file_path,
format=log_format_file,
level=log_config.get("log_level", "INFO"),
filter=formatter,
rotation="10 MB", # 每个文件最大10MB
retention="30 days", # 保留30天
compression=None,
encoding="utf-8",
enqueue=True, # 异步安全
backtrace=True,
diagnose=True,
)
except Exception as e:
logger.error(f"日志配置更新失败: {str(e)}")
raise
+1
View File
@@ -19,6 +19,7 @@ server:
vision_explain: http://你的ip或者域名:端口号/mcp/vision/explain
manager-api:
# 你的manager-api的地址,最好使用局域网ip
# 如果使用docker部署,请使用填写成 http://xiaozhi-esp32-server-web:8002/xiaozhi
url: http://127.0.0.1:8002/xiaozhi
# 你的manager-api的token,就是刚才复制出来的server.secret
secret: 你的server.secret值
@@ -8,6 +8,7 @@ from config.config_loader import get_private_config_from_api
from core.utils.auth import AuthToken
import base64
from typing import Tuple, Optional
from plugins_func.register import Action
TAG = __name__
@@ -122,7 +123,8 @@ class VisionHandler:
return_json = {
"success": True,
"result": result,
"action": Action.RESPONSE.name,
"response": result,
}
response = web.Response(
+199 -149
View File
@@ -10,7 +10,6 @@ import threading
import traceback
import subprocess
import websockets
from core.handle.mcpHandle import call_mcp_tool
from core.utils.util import (
extract_json_from_string,
check_vad_update,
@@ -18,26 +17,28 @@ from core.utils.util import (
filter_sensitive_info,
)
from typing import Dict, Any
from core.mcp.manager import MCPManager
from collections import deque
from core.utils.modules_initialize import (
initialize_modules,
initialize_tts,
initialize_asr,
)
from core.handle.reportHandle import report
from core.utils.modules_initialize import initialize_voiceprint
from core.providers.tts.default import DefaultTTS
from concurrent.futures import ThreadPoolExecutor
from core.utils.dialogue import Message, Dialogue
from core.providers.asr.dto.dto import InterfaceType
from core.handle.textHandle import handleTextMessage
from core.handle.functionHandler import FunctionHandler
from core.providers.tools.unified_tool_handler import UnifiedToolHandler
from plugins_func.loadplugins import auto_import_modules
from plugins_func.register import Action, ActionResponse
from core.auth import AuthMiddleware, AuthenticationError
from config.config_loader import get_private_config_from_api
from core.providers.tts.dto.dto import ContentType, TTSMessageDTO, SentenceType
from config.logger import setup_logging, build_module_string, update_module_string
from config.logger import setup_logging, build_module_string, create_connection_logger
from config.manage_api_client import DeviceNotFoundException, DeviceBindException
from core.utils.prompt_manager import PromptManager
TAG = __name__
@@ -75,7 +76,6 @@ class ConnectionHandler:
self.headers = None
self.device_id = None
self.client_ip = None
self.client_ip_info = {}
self.prompt = None
self.welcome_msg = None
self.max_output_size = 0
@@ -112,9 +112,10 @@ class ConnectionHandler:
# vad相关变量
self.client_audio_buffer = bytearray()
self.client_have_voice = False
self.client_have_voice_last_time = 0.0
self.client_no_voice_last_time = 0.0
self.last_activity_time = 0.0 # 统一的活动时间戳(毫秒)
self.client_voice_stop = False
self.client_voice_window = deque(maxlen=5)
self.last_is_voice = False
# asr相关变量
# 因为实际部署时可能会用到公共的本地ASR,不能把变量暴露给公共ASR
@@ -144,14 +145,17 @@ class ConnectionHandler:
self.load_function_plugin = False
self.intent_type = "nointent"
self.timeout_task = None
self.timeout_seconds = (
int(self.config.get("close_connection_no_voice_time", 120)) + 60
) # 在原来第一道关闭的基础上加60秒,进行二道关闭
self.timeout_task = None
# {"mcp":true} 表示启用MCP功能
self.features = None
# 初始化提示词管理器
self.prompt_manager = PromptManager(config, self.logger)
async def handle_connection(self, ws):
try:
# 获取并验证headers
@@ -188,12 +192,14 @@ class ConnectionHandler:
self.websocket = ws
self.device_id = self.headers.get("device-id", None)
# 初始化活动时间戳
self.last_activity_time = time.time() * 1000
# 启动超时检查任务
self.timeout_task = asyncio.create_task(self._check_timeout())
self.welcome_msg = self.config["xiaozhi"]
self.welcome_msg["session_id"] = self.session_id
await self.websocket.send(json.dumps(self.welcome_msg))
# 获取差异化配置
self._initialize_private_config()
@@ -214,7 +220,17 @@ class ConnectionHandler:
self.logger.bind(tag=TAG).error(f"Connection error: {str(e)}-{stack_trace}")
return
finally:
await self._save_and_close(ws)
try:
await self._save_and_close(ws)
except Exception as final_error:
self.logger.bind(tag=TAG).error(f"最终清理时出错: {final_error}")
# 确保即使保存记忆失败,也要关闭连接
try:
await self.close(ws)
except Exception as close_error:
self.logger.bind(tag=TAG).error(
f"强制关闭连接时出错: {close_error}"
)
async def _save_and_close(self, ws):
"""保存记忆并关闭连接"""
@@ -232,7 +248,10 @@ class ConnectionHandler:
except Exception as e:
self.logger.bind(tag=TAG).error(f"保存记忆失败: {e}")
finally:
loop.close()
try:
loop.close()
except Exception:
pass
# 启动线程保存记忆,不等待完成
threading.Thread(target=save_memory_task, daemon=True).start()
@@ -240,20 +259,17 @@ class ConnectionHandler:
self.logger.bind(tag=TAG).error(f"保存记忆失败: {e}")
finally:
# 立即关闭连接,不等待记忆保存完成
await self.close(ws)
async def reset_timeout(self):
"""重置超时计时器"""
if self.timeout_task:
self.timeout_task.cancel()
self.timeout_task = asyncio.create_task(self._check_timeout())
try:
await self.close(ws)
except Exception as close_error:
self.logger.bind(tag=TAG).error(
f"保存记忆后关闭连接失败: {close_error}"
)
async def _route_message(self, message):
"""消息路由"""
# 重置超时计时器
await self.reset_timeout()
if isinstance(message, str):
self.last_activity_time = time.time() * 1000
await handleTextMessage(self, message)
elif isinstance(message, bytes):
if self.vad is None:
@@ -315,13 +331,16 @@ class ConnectionHandler:
self.selected_module_str = build_module_string(
self.config.get("selected_module", {})
)
update_module_string(self.selected_module_str)
self.logger = create_connection_logger(self.selected_module_str)
"""初始化组件"""
if self.config.get("prompt") is not None:
self.prompt = self.config["prompt"]
self.change_system_prompt(self.prompt)
user_prompt = self.config["prompt"]
# 使用快速提示词进行初始化
prompt = self.prompt_manager.get_quick_prompt(user_prompt)
self.change_system_prompt(prompt)
self.logger.bind(tag=TAG).info(
f"初始化组件: prompt成功 {self.prompt[:50]}..."
f"快速初始化组件: prompt成功 {prompt[:50]}..."
)
"""初始化本地组件"""
@@ -346,9 +365,22 @@ class ConnectionHandler:
self._initialize_intent()
"""初始化上报线程"""
self._init_report_threads()
"""更新系统提示词"""
self._init_prompt_enhancement()
except Exception as e:
self.logger.bind(tag=TAG).error(f"实例化组件失败: {e}")
def _init_prompt_enhancement(self):
# 更新上下文信息
self.prompt_manager.update_context_info(self, self.client_ip)
enhanced_prompt = self.prompt_manager.build_enhanced_prompt(
self.config["prompt"], self.device_id, self.client_ip
)
if enhanced_prompt:
self.change_system_prompt(enhanced_prompt)
self.logger.bind(tag=TAG).info("系统提示词已增强更新")
def _init_report_threads(self):
"""初始化ASR和TTS上报线程"""
if not self.read_config_from_api or self.need_bind:
@@ -384,6 +416,16 @@ class ConnectionHandler:
# 因为远程ASR,涉及到websocket连接和接收线程,需要每个连接一个实例
asr = initialize_asr(self.config)
# 动态初始化声纹识别功能
try:
success = initialize_voiceprint(asr, self.config)
if success:
self.logger.bind(tag=TAG).info("声纹识别功能已在连接时动态启用")
else:
self.logger.bind(tag=TAG).info("声纹识别功能未启用或配置不完整")
except Exception as e:
self.logger.bind(tag=TAG).error(f"动态初始化声纹识别时发生错误: {str(e)}")
return asr
def _initialize_private_config(self):
@@ -468,12 +510,17 @@ class ConnectionHandler:
] = plugin_from_server.keys()
if private_config.get("prompt", None) is not None:
self.config["prompt"] = private_config["prompt"]
# 获取声纹信息
if private_config.get("voiceprint", None) is not None:
self.config["voiceprint"] = private_config["voiceprint"]
if private_config.get("summaryMemory", None) is not None:
self.config["summaryMemory"] = private_config["summaryMemory"]
if private_config.get("device_max_output_size", None) is not None:
self.max_output_size = int(private_config["device_max_output_size"])
if private_config.get("chat_history_conf", None) is not None:
self.chat_history_conf = int(private_config["chat_history_conf"])
if private_config.get("mcp_endpoint", None) is not None:
self.config["mcp_endpoint"] = private_config["mcp_endpoint"]
try:
modules = initialize_modules(
self.logger,
@@ -585,37 +632,40 @@ class ConnectionHandler:
self.intent.set_llm(self.llm)
self.logger.bind(tag=TAG).info("使用主LLM作为意图识别模型")
"""加载插件"""
self.func_handler = FunctionHandler(self)
self.mcp_manager = MCPManager(self)
"""加载统一工具处理器"""
self.func_handler = UnifiedToolHandler(self)
"""加载MCP工具"""
asyncio.run_coroutine_threadsafe(
self.mcp_manager.initialize_servers(), self.loop
)
# 异步初始化工具处理器
if hasattr(self, "loop") and self.loop:
asyncio.run_coroutine_threadsafe(self.func_handler._initialize(), self.loop)
def change_system_prompt(self, prompt):
self.prompt = prompt
# 更新系统prompt至上下文
self.dialogue.update_system_message(self.prompt)
def chat(self, query, tool_call=False):
def chat(self, query, tool_call=False, depth=0):
self.logger.bind(tag=TAG).info(f"大模型收到用户消息: {query}")
self.llm_finish_task = False
if not tool_call:
self.dialogue.put(Message(role="user", content=query))
# 为最顶层时新建会话ID和发送FIRST请求
if depth == 0:
self.sentence_id = str(uuid.uuid4().hex)
self.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=self.sentence_id,
sentence_type=SentenceType.FIRST,
content_type=ContentType.ACTION,
)
)
# Define intent functions
functions = None
if self.intent_type == "function_call" and hasattr(self, "func_handler"):
functions = self.func_handler.get_functions()
if hasattr(self, "mcp_client"):
mcp_tools = self.mcp_client.get_available_tools()
if mcp_tools is not None and len(mcp_tools) > 0:
if functions is None:
functions = []
functions.extend(mcp_tools)
response_message = []
try:
@@ -627,20 +677,21 @@ class ConnectionHandler:
)
memory_str = future.result()
uuid_str = str(uuid.uuid4()).replace("-", "")
self.sentence_id = uuid_str
if self.intent_type == "function_call" and functions is not None:
# 使用支持functions的streaming接口
llm_responses = self.llm.response_with_functions(
self.session_id,
self.dialogue.get_llm_dialogue_with_memory(memory_str),
self.dialogue.get_llm_dialogue_with_memory(
memory_str, self.config.get("voiceprint", {})
),
functions=functions,
)
else:
llm_responses = self.llm.response(
self.session_id,
self.dialogue.get_llm_dialogue_with_memory(memory_str),
self.dialogue.get_llm_dialogue_with_memory(
memory_str, self.config.get("voiceprint", {})
),
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"LLM 处理出错 {query}: {e}")
@@ -652,7 +703,6 @@ class ConnectionHandler:
function_id = None
function_arguments = ""
content_arguments = ""
text_index = 0
self.client_abort = False
for response in llm_responses:
if self.client_abort:
@@ -682,14 +732,6 @@ class ConnectionHandler:
if content is not None and len(content) > 0:
if not tool_call_flag:
response_message.append(content)
if text_index == 0:
self.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=self.sentence_id,
sentence_type=SentenceType.FIRST,
content_type=ContentType.ACTION,
)
)
self.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=self.sentence_id,
@@ -698,7 +740,6 @@ class ConnectionHandler:
content_detail=content,
)
)
text_index += 1
# 处理function call
if tool_call_flag:
bHasError = False
@@ -723,6 +764,11 @@ class ConnectionHandler:
f"function call error: {content_arguments}"
)
if not bHasError:
# 如需要大模型先处理一轮,添加相关处理后的日志情况
if len(response_message) > 0:
self.dialogue.put(
Message(role="assistant", content="".join(response_message))
)
response_message.clear()
self.logger.bind(tag=TAG).debug(
f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}"
@@ -733,45 +779,21 @@ class ConnectionHandler:
"arguments": function_arguments,
}
# 处理Server端MCP工具调用
if self.mcp_manager.is_mcp_tool(function_name):
result = self._handle_mcp_tool_call(function_call_data)
elif hasattr(self, "mcp_client") and self.mcp_client.has_tool(
function_name
):
# 如果是小智端MCP工具调用
self.logger.bind(tag=TAG).debug(
f"调用小智端MCP工具: {function_name}, 参数: {function_arguments}"
)
try:
result = asyncio.run_coroutine_threadsafe(
call_mcp_tool(
self, self.mcp_client, function_name, function_arguments
),
self.loop,
).result()
self.logger.bind(tag=TAG).debug(f"MCP工具调用结果: {result}")
result = ActionResponse(
action=Action.REQLLM, result=result, response=""
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"MCP工具调用失败: {e}")
result = ActionResponse(
action=Action.REQLLM, result="MCP工具调用失败", response=""
)
else:
# 处理系统函数
result = self.func_handler.handle_llm_function_call(
# 使用统一工具处理器处理所有工具调用
result = asyncio.run_coroutine_threadsafe(
self.func_handler.handle_llm_function_call(
self, function_call_data
)
self._handle_function_result(result, function_call_data)
),
self.loop,
).result()
self._handle_function_result(result, function_call_data, depth=depth)
# 存储对话内容
if len(response_message) > 0:
self.dialogue.put(
Message(role="assistant", content="".join(response_message))
)
if text_index > 0:
if depth == 0:
self.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=self.sentence_id,
@@ -780,55 +802,16 @@ class ConnectionHandler:
)
)
self.llm_finish_task = True
# 使用lambda延迟计算,只有在DEBUG级别时才执行get_llm_dialogue()
self.logger.bind(tag=TAG).debug(
json.dumps(self.dialogue.get_llm_dialogue(), indent=4, ensure_ascii=False)
lambda: json.dumps(
self.dialogue.get_llm_dialogue(), indent=4, ensure_ascii=False
)
)
return True
def _handle_mcp_tool_call(self, function_call_data):
function_arguments = function_call_data["arguments"]
function_name = function_call_data["name"]
try:
args_dict = function_arguments
if isinstance(function_arguments, str):
try:
args_dict = json.loads(function_arguments)
except json.JSONDecodeError:
self.logger.bind(tag=TAG).error(
f"无法解析 function_arguments: {function_arguments}"
)
return ActionResponse(
action=Action.REQLLM, result="参数解析失败", response=""
)
tool_result = asyncio.run_coroutine_threadsafe(
self.mcp_manager.execute_tool(function_name, args_dict), self.loop
).result()
# meta=None content=[TextContent(type='text', text='北京当前天气:\n温度: 21°C\n天气: 晴\n湿度: 6%\n风向: 西北 风\n风力等级: 5级', annotations=None)] isError=False
content_text = ""
if tool_result is not None and tool_result.content is not None:
for content in tool_result.content:
content_type = content.type
if content_type == "text":
content_text = content.text
elif content_type == "image":
pass
if len(content_text) > 0:
return ActionResponse(
action=Action.REQLLM, result=content_text, response=""
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"MCP工具调用错误: {e}")
return ActionResponse(
action=Action.REQLLM, result="工具调用出错", response=""
)
return ActionResponse(action=Action.REQLLM, result="工具调用出错", response="")
def _handle_function_result(self, result, function_call_data):
def _handle_function_result(self, result, function_call_data, depth):
if result.action == Action.RESPONSE: # 直接回复前端
text = result.response
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
@@ -865,9 +848,9 @@ class ConnectionHandler:
content=text,
)
)
self.chat(text, tool_call=True)
self.chat(text, tool_call=True, depth=depth + 1)
elif result.action == Action.NOTFOUND or result.action == Action.ERROR:
text = result.result
text = result.response if result.response else result.result
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
self.dialogue.put(Message(role="assistant", content=text))
else:
@@ -881,14 +864,13 @@ class ConnectionHandler:
item = self.report_queue.get(timeout=1)
if item is None: # 检测毒丸对象
break
type, text, audio_data, report_time = item
try:
# 检查线程池状态
if self.executor is None:
continue
# 提交任务到线程池
self.executor.submit(
self._process_report, type, text, audio_data, report_time
self._process_report, *item
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"聊天记录上报线程异常: {e}")
@@ -918,13 +900,22 @@ class ConnectionHandler:
"""资源清理方法"""
try:
# 取消超时任务
if self.timeout_task:
if self.timeout_task and not self.timeout_task.done():
self.timeout_task.cancel()
try:
await self.timeout_task
except asyncio.CancelledError:
pass
self.timeout_task = None
# 清理MCP资源
if hasattr(self, "mcp_manager") and self.mcp_manager:
await self.mcp_manager.cleanup_all()
# 清理工具处理器资源
if hasattr(self, "func_handler") and self.func_handler:
try:
await self.func_handler.cleanup()
except Exception as cleanup_error:
self.logger.bind(tag=TAG).error(
f"清理工具处理器时出错: {cleanup_error}"
)
# 触发停止事件
if self.stop_event:
@@ -934,19 +925,61 @@ class ConnectionHandler:
self.clear_queues()
# 关闭WebSocket连接
if ws:
await ws.close()
elif self.websocket:
await self.websocket.close()
try:
if ws:
# 安全地检查WebSocket状态并关闭
try:
if hasattr(ws, "closed") and not ws.closed:
await ws.close()
elif hasattr(ws, "state") and ws.state.name != "CLOSED":
await ws.close()
else:
# 如果没有closed属性,直接尝试关闭
await ws.close()
except Exception:
# 如果关闭失败,忽略错误
pass
elif self.websocket:
try:
if (
hasattr(self.websocket, "closed")
and not self.websocket.closed
):
await self.websocket.close()
elif (
hasattr(self.websocket, "state")
and self.websocket.state.name != "CLOSED"
):
await self.websocket.close()
else:
# 如果没有closed属性,直接尝试关闭
await self.websocket.close()
except Exception:
# 如果关闭失败,忽略错误
pass
except Exception as ws_error:
self.logger.bind(tag=TAG).error(f"关闭WebSocket连接时出错: {ws_error}")
if self.tts:
await self.tts.close()
# 最后关闭线程池(避免阻塞)
if self.executor:
self.executor.shutdown(wait=False)
try:
self.executor.shutdown(wait=False)
except Exception as executor_error:
self.logger.bind(tag=TAG).error(
f"关闭线程池时出错: {executor_error}"
)
self.executor = None
self.logger.bind(tag=TAG).info("连接资源已释放")
except Exception as e:
self.logger.bind(tag=TAG).error(f"关闭连接时出错: {e}")
finally:
# 确保停止事件被设置
if self.stop_event:
self.stop_event.set()
def clear_queues(self):
"""清空所有任务队列"""
@@ -976,7 +1009,6 @@ class ConnectionHandler:
def reset_vad_states(self):
self.client_audio_buffer = bytearray()
self.client_have_voice = False
self.client_have_voice_last_time = 0
self.client_voice_stop = False
self.logger.bind(tag=TAG).debug("VAD states reset.")
@@ -995,10 +1027,28 @@ class ConnectionHandler:
"""检查连接超时"""
try:
while not self.stop_event.is_set():
await asyncio.sleep(self.timeout_seconds)
if not self.stop_event.is_set():
self.logger.bind(tag=TAG).info("连接超时,准备关闭")
await self.close(self.websocket)
break
# 检查是否超时(只有在时间戳已初始化的情况下)
if self.last_activity_time > 0.0:
current_time = time.time() * 1000
if (
current_time - self.last_activity_time
> self.timeout_seconds * 1000
):
if not self.stop_event.is_set():
self.logger.bind(tag=TAG).info("连接超时,准备关闭")
# 设置停止事件,防止重复处理
self.stop_event.set()
# 使用 try-except 包装关闭操作,确保不会因为异常而阻塞
try:
await self.close(self.websocket)
except Exception as close_error:
self.logger.bind(tag=TAG).error(
f"超时关闭连接时出错: {close_error}"
)
break
# 每10秒检查一次,避免过于频繁
await asyncio.sleep(10)
except Exception as e:
self.logger.bind(tag=TAG).error(f"超时检查任务出错: {e}")
finally:
self.logger.bind(tag=TAG).info("超时检查任务已退出")
@@ -1,89 +0,0 @@
from config.logger import setup_logging
import json
from plugins_func.register import (
FunctionRegistry,
ActionResponse,
Action,
ToolType,
DeviceTypeRegistry,
)
from plugins_func.functions.hass_init import append_devices_to_prompt
TAG = __name__
class FunctionHandler:
def __init__(self, conn):
self.conn = conn
self.config = conn.config
self.device_type_registry = DeviceTypeRegistry()
self.function_registry = FunctionRegistry()
self.register_nessary_functions()
self.register_config_functions()
self.functions_desc = self.function_registry.get_all_function_desc()
self.finish_init = True
def current_support_functions(self):
func_names = []
for func in self.functions_desc:
func_names.append(func["function"]["name"])
# 打印当前支持的函数列表
self.conn.logger.bind(tag=TAG, session_id=self.conn.session_id).info(
f"当前支持的函数列表: {func_names}"
)
return func_names
def get_functions(self):
"""获取功能调用配置"""
return self.functions_desc
def register_nessary_functions(self):
"""注册必要的函数"""
self.function_registry.register_function("handle_exit_intent")
self.function_registry.register_function("get_time")
self.function_registry.register_function("get_lunar")
def register_config_functions(self):
"""注册配置中的函数,可以不同客户端使用不同的配置"""
for func in self.config["Intent"][self.config["selected_module"]["Intent"]].get(
"functions", []
):
self.function_registry.register_function(func)
"""home assistant需要初始化提示词"""
append_devices_to_prompt(self.conn)
def get_function(self, name):
return self.function_registry.get_function(name)
def handle_llm_function_call(self, conn, function_call_data):
try:
function_name = function_call_data["name"]
funcItem = self.get_function(function_name)
if not funcItem:
return ActionResponse(
action=Action.NOTFOUND, result="没有找到对应的函数", response=""
)
func = funcItem.func
arguments = function_call_data["arguments"]
arguments = json.loads(arguments) if arguments else {}
self.conn.logger.bind(tag=TAG).debug(
f"调用函数: {function_name}, 参数: {arguments}"
)
if (
funcItem.type == ToolType.SYSTEM_CTL
or funcItem.type == ToolType.IOT_CTL
):
return func(conn, **arguments)
elif funcItem.type == ToolType.WAIT:
return func(**arguments)
elif funcItem.type == ToolType.CHANGE_SYS_PROMPT:
return func(conn, **arguments)
else:
return ActionResponse(
action=Action.NOTFOUND, result="没有找到对应的函数", response=""
)
except Exception as e:
self.conn.logger.bind(tag=TAG).error(f"处理function call错误: {e}")
return None
@@ -7,7 +7,7 @@ from core.utils.util import audio_to_data
from core.handle.sendAudioHandle import sendAudioMessage, send_stt_message
from core.utils.util import remove_punctuation_and_length, opus_datas_to_wav_bytes
from core.providers.tts.dto.dto import ContentType, SentenceType
from core.handle.mcpHandle import (
from core.providers.tools.device_mcp import (
MCPClient,
send_mcp_initialize_message,
send_mcp_tools_list_request,
@@ -6,9 +6,8 @@ from core.handle.helloHandle import checkWakeupWords
from core.utils.util import remove_punctuation_and_length
from core.providers.tts.dto.dto import ContentType
from core.utils.dialogue import Message
from core.handle.mcpHandle import call_mcp_tool
from plugins_func.register import Action, ActionResponse
from loguru import logger
from core.providers.tts.dto.dto import TTSMessageDTO, SentenceType
TAG = __name__
@@ -29,6 +28,8 @@ async def handle_user_intent(conn, text):
intent_result = await analyze_intent_with_llm(conn, text)
if not intent_result:
return False
# 会话开始时生成sentence_id
conn.sentence_id = str(uuid.uuid4().hex)
# 处理各种意图
return await process_intent_result(conn, intent_result, text)
@@ -79,11 +80,6 @@ async def process_intent_result(conn, intent_result, original_text):
if function_name == "continue_chat":
return False
if function_name == "play_music":
funcItem = conn.func_handler.get_function(function_name)
if not funcItem:
conn.func_handler.function_registry.register_function("play_music")
function_args = {}
if "arguments" in intent_data["function_call"]:
function_args = intent_data["function_call"]["arguments"]
@@ -106,36 +102,18 @@ async def process_intent_result(conn, intent_result, original_text):
def process_function_call():
conn.dialogue.put(Message(role="user", content=original_text))
# 处理Server端MCP工具调用
if conn.mcp_manager.is_mcp_tool(function_name):
result = conn._handle_mcp_tool_call(function_call_data)
elif hasattr(conn, "mcp_client") and conn.mcp_client.has_tool(
function_name
):
# 如果是小智端MCP工具调用
conn.logger.bind(tag=TAG).debug(
f"调用小智端MCP工具: {function_name}, 参数: {function_args}"
)
try:
result = asyncio.run_coroutine_threadsafe(
call_mcp_tool(
conn, conn.mcp_client, function_name, function_args
),
conn.loop,
).result()
conn.logger.bind(tag=TAG).debug(f"MCP工具调用结果: {result}")
result = ActionResponse(
action=Action.REQLLM, result=result, response=""
)
except Exception as e:
conn.logger.bind(tag=TAG).error(f"MCP工具调用失败: {e}")
result = ActionResponse(
action=Action.REQLLM, result="MCP工具调用失败", response=""
)
else:
# 处理系统函数
result = conn.func_handler.handle_llm_function_call(
conn, function_call_data
# 使用统一工具处理器处理所有工具调用
try:
result = asyncio.run_coroutine_threadsafe(
conn.func_handler.handle_llm_function_call(
conn, function_call_data
),
conn.loop,
).result()
except Exception as e:
conn.logger.bind(tag=TAG).error(f"工具调用失败: {e}")
result = ActionResponse(
action=Action.ERROR, result=str(e), response=str(e)
)
if result:
@@ -176,5 +154,19 @@ async def process_intent_result(conn, intent_result, original_text):
def speak_txt(conn, text):
conn.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=conn.sentence_id,
sentence_type=SentenceType.FIRST,
content_type=ContentType.ACTION,
)
)
conn.tts.tts_one_sentence(conn, ContentType.TEXT, content_detail=text)
conn.tts.tts_text_queue.put(
TTSMessageDTO(
sentence_id=conn.sentence_id,
sentence_type=SentenceType.LAST,
content_type=ContentType.ACTION,
)
)
conn.dialogue.put(Message(role="assistant", content=text))
@@ -1,421 +0,0 @@
import json
import asyncio
from plugins_func.register import (
FunctionItem,
register_device_function,
ActionResponse,
Action,
ToolType,
)
TAG = __name__
def wrap_async_function(async_func):
"""包装异步函数为同步函数"""
def wrapper(*args, **kwargs):
try:
# 获取连接对象(第一个参数)
conn = args[0]
if not hasattr(conn, "loop"):
conn.logger.bind(tag=TAG).error("Connection对象没有loop属性")
return ActionResponse(
Action.ERROR,
"Connection对象没有loop属性",
"执行操作时出错: Connection对象没有loop属性",
)
# 使用conn对象中的事件循环
loop = conn.loop
# 在conn的事件循环中运行异步函数
future = asyncio.run_coroutine_threadsafe(async_func(*args, **kwargs), loop)
# 等待结果返回
return future.result()
except Exception as e:
conn.logger.bind(tag=TAG).error(f"运行异步函数时出错: {e}")
return ActionResponse(Action.ERROR, str(e), f"执行操作时出错: {e}")
return wrapper
def create_iot_function(device_name, method_name, method_info):
"""
根据IOT设备描述生成通用的控制函数
"""
async def iot_control_function(
conn, response_success=None, response_failure=None, **params
):
try:
# 设置默认响应消息
if not response_success:
response_success = "操作成功"
if not response_failure:
response_failure = "操作失败"
# 打印响应参数
conn.logger.bind(tag=TAG).debug(
f"控制函数接收到的响应参数: success='{response_success}', failure='{response_failure}'"
)
# 发送控制命令
await send_iot_conn(conn, device_name, method_name, params)
# 等待一小段时间让状态更新
await asyncio.sleep(0.1)
# 生成结果信息
result = f"{device_name}{method_name}操作执行成功"
# 处理响应中可能的占位符
response = response_success
# 替换{value}占位符
for param_name, param_value in params.items():
# 先尝试直接替换参数值
if "{" + param_name + "}" in response:
response = response.replace(
"{" + param_name + "}", str(param_value)
)
# 如果有{value}占位符,用相关参数替换
if "{value}" in response:
response = response.replace("{value}", str(param_value))
break
return ActionResponse(Action.RESPONSE, result, response)
except Exception as e:
conn.logger.bind(tag=TAG).error(
f"执行{device_name}{method_name}操作失败: {e}"
)
# 操作失败时使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, str(e), response)
return wrap_async_function(iot_control_function)
def create_iot_query_function(device_name, prop_name, prop_info):
"""
根据IOT设备属性创建查询函数
"""
async def iot_query_function(conn, response_success=None, response_failure=None):
try:
# 打印响应参数
conn.logger.bind(tag=TAG).info(
f"查询函数接收到的响应参数: success='{response_success}', failure='{response_failure}'"
)
value = await get_iot_status(conn, device_name, prop_name)
# 查询成功,生成结果
if value is not None:
# 使用大模型提供的成功响应,并替换其中的占位符
response = response_success.replace("{value}", str(value))
return ActionResponse(Action.RESPONSE, str(value), response)
else:
# 查询失败,使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, f"属性{prop_name}不存在", response)
except Exception as e:
conn.logger.bind(tag=TAG).error(
f"查询{device_name}{prop_name}时出错: {e}"
)
# 查询出错时使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, str(e), response)
return wrap_async_function(iot_query_function)
class IotDescriptor:
"""
A class to represent an IoT descriptor.
"""
def __init__(self, name, description, properties, methods):
self.name = name
self.description = description
self.properties = []
self.methods = []
# 根据描述创建属性
if properties is not None:
for key, value in properties.items():
property_item = {}
property_item["name"] = key
property_item["description"] = value["description"]
if value["type"] == "number":
property_item["value"] = 0
elif value["type"] == "boolean":
property_item["value"] = False
else:
property_item["value"] = ""
self.properties.append(property_item)
# 根据描述创建方法
if methods is not None:
for key, value in methods.items():
method = {}
method["description"] = value["description"]
method["name"] = key
# 检查方法是否有参数
if "parameters" in value:
method["parameters"] = {}
for k, v in value["parameters"].items():
method["parameters"][k] = {
"description": v["description"],
"type": v["type"],
}
self.methods.append(method)
def register_device_type(descriptor, device_type_registry):
"""注册设备类型及其功能"""
device_name = descriptor["name"]
type_id = device_type_registry.generate_device_type_id(descriptor)
# 如果该类型已注册,直接返回类型ID
if type_id in device_type_registry.type_functions:
return type_id
functions = {}
# 为每个属性创建查询函数
for prop_name, prop_info in descriptor["properties"].items():
func_name = f"get_{device_name.lower()}_{prop_name.lower()}"
func_desc = {
"type": "function",
"function": {
"name": func_name,
"description": f"查询{descriptor['description']}{prop_info['description']}",
"parameters": {
"type": "object",
"properties": {
"response_success": {
"type": "string",
"description": f"查询成功时的友好回复,必须使用{{value}}作为占位符表示查询到的值",
},
"response_failure": {
"type": "string",
"description": f"查询失败时的友好回复,例如:'无法获取{device_name}{prop_info['description']}'",
},
},
"required": ["response_success", "response_failure"],
},
},
}
query_func = create_iot_query_function(device_name, prop_name, prop_info)
decorated_func = register_device_function(
func_name, func_desc, ToolType.IOT_CTL
)(query_func)
functions[func_name] = FunctionItem(
func_name, func_desc, decorated_func, ToolType.IOT_CTL
)
# 为每个方法创建控制函数
for method_name, method_info in descriptor["methods"].items():
func_name = f"{device_name.lower()}_{method_name.lower()}"
# 创建参数字典,添加原有参数
parameters = {}
required_params = []
# 如果方法有参数,则添加参数信息
if "parameters" in method_info:
parameters = {
param_name: {
"type": param_info["type"],
"description": param_info["description"],
}
for param_name, param_info in method_info["parameters"].items()
}
required_params = list(method_info["parameters"].keys())
# 添加响应参数
parameters.update(
{
"response_success": {
"type": "string",
"description": "操作成功时的友好回复,关于该设备的操作结果,设备名称尽量使用description中的名称",
},
"response_failure": {
"type": "string",
"description": "操作失败时的友好回复,关于该设备的操作结果,设备名称尽量使用description中的名称",
},
}
)
# 构建必须参数列表(原有参数 + 响应参数)
required_params.extend(["response_success", "response_failure"])
func_desc = {
"type": "function",
"function": {
"name": func_name,
"description": f"{descriptor['description']} - {method_info['description']}",
"parameters": {
"type": "object",
"properties": parameters,
"required": required_params,
},
},
}
control_func = create_iot_function(device_name, method_name, method_info)
decorated_func = register_device_function(
func_name, func_desc, ToolType.IOT_CTL
)(control_func)
functions[func_name] = FunctionItem(
func_name, func_desc, decorated_func, ToolType.IOT_CTL
)
device_type_registry.register_device_type(type_id, functions)
return type_id
# 用于接受前端设备推送的搜索iot描述
async def handleIotDescriptors(conn, descriptors):
wait_max_time = 5
while conn.func_handler is None or not conn.func_handler.finish_init:
await asyncio.sleep(1)
wait_max_time -= 1
if wait_max_time <= 0:
conn.logger.bind(tag=TAG).debug("连接对象没有func_handler")
return
"""处理物联网描述"""
functions_changed = False
for descriptor in descriptors:
# 如果descriptor没有properties和methods,则直接跳过
if "properties" not in descriptor and "methods" not in descriptor:
continue
# 处理缺失properties的情况
if "properties" not in descriptor:
descriptor["properties"] = {}
# 从methods中提取所有参数作为properties
if "methods" in descriptor:
for method_name, method_info in descriptor["methods"].items():
if "parameters" in method_info:
for param_name, param_info in method_info["parameters"].items():
# 将参数信息转换为属性信息
descriptor["properties"][param_name] = {
"description": param_info["description"],
"type": param_info["type"],
}
# 创建IOT设备描述符
iot_descriptor = IotDescriptor(
descriptor["name"],
descriptor["description"],
descriptor["properties"],
descriptor["methods"],
)
conn.iot_descriptors[descriptor["name"]] = iot_descriptor
if conn.load_function_plugin:
# 注册或获取设备类型
device_type_registry = conn.func_handler.device_type_registry
type_id = register_device_type(descriptor, device_type_registry)
device_functions = device_type_registry.get_device_functions(type_id)
# 在连接级注册设备函数
if hasattr(conn, "func_handler"):
for func_name, func_item in device_functions.items():
conn.func_handler.function_registry.register_function(
func_name, func_item
)
conn.logger.bind(tag=TAG).info(
f"注册IOT函数到function handler: {func_name}"
)
functions_changed = True
# 如果注册了新函数,更新function描述列表
if functions_changed and hasattr(conn, "func_handler"):
func_names = conn.func_handler.current_support_functions()
conn.logger.bind(tag=TAG).info(f"设备类型: {type_id}")
conn.logger.bind(tag=TAG).info(
f"更新function描述列表完成,当前支持的函数: {func_names}"
)
async def handleIotStatus(conn, states):
"""处理物联网状态"""
for state in states:
for key, value in conn.iot_descriptors.items():
if key == state["name"]:
for property_item in value.properties:
for k, v in state["state"].items():
if property_item["name"] == k:
if type(v) != type(property_item["value"]):
conn.logger.bind(tag=TAG).error(
f"属性{property_item['name']}的值类型不匹配"
)
break
else:
property_item["value"] = v
conn.logger.bind(tag=TAG).info(
f"物联网状态更新: {key} , {property_item['name']} = {v}"
)
break
break
async def get_iot_status(conn, name, property_name):
"""获取物联网状态"""
for key, value in conn.iot_descriptors.items():
if key == name:
for property_item in value.properties:
if property_item["name"] == property_name:
return property_item["value"]
conn.logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
return None
async def set_iot_status(conn, name, property_name, value):
"""设置物联网状态"""
for key, iot_descriptor in conn.iot_descriptors.items():
if key == name:
for property_item in iot_descriptor.properties:
if property_item["name"] == property_name:
if type(value) != type(property_item["value"]):
conn.logger.bind(tag=TAG).error(
f"属性{property_item['name']}的值类型不匹配"
)
return
property_item["value"] = value
conn.logger.bind(tag=TAG).info(
f"物联网状态更新: {name} , {property_name} = {value}"
)
return
conn.logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
async def send_iot_conn(conn, name, method_name, parameters):
"""发送物联网指令"""
for key, value in conn.iot_descriptors.items():
if key == name:
# 找到了设备
for method in value.methods:
# 找到了方法
if method["name"] == method_name:
# 构建命令对象
command = {
"name": name,
"method": method_name,
}
# 只有当参数不为空时才添加parameters字段
if parameters:
command["parameters"] = parameters
send_message = json.dumps({"type": "iot", "commands": [command]})
await conn.websocket.send(send_message)
conn.logger.bind(tag=TAG).info(f"发送物联网指令: {send_message}")
return
conn.logger.bind(tag=TAG).error(f"未找到方法{method_name}")
@@ -4,6 +4,7 @@ from core.utils.output_counter import check_device_output_limit
from core.handle.abortHandle import handleAbortMessage
import time
import asyncio
import json
from core.handle.sendAudioHandle import SentenceType
from core.utils.util import audio_to_data
@@ -38,6 +39,31 @@ async def resume_vad_detection(conn):
async def startToChat(conn, text):
# 检查输入是否是JSON格式(包含说话人信息)
speaker_name = None
actual_text = text
try:
# 尝试解析JSON格式的输入
if text.strip().startswith('{') and text.strip().endswith('}'):
data = json.loads(text)
if 'speaker' in data and 'content' in data:
speaker_name = data['speaker']
actual_text = data['content']
conn.logger.bind(tag=TAG).info(f"解析到说话人信息: {speaker_name}")
# 直接使用JSON格式的文本,不解析
actual_text = text
except (json.JSONDecodeError, KeyError):
# 如果解析失败,继续使用原始文本
pass
# 保存说话人信息到连接对象
if speaker_name:
conn.current_speaker = speaker_name
else:
conn.current_speaker = None
if conn.need_bind:
await check_bind_device(conn)
return
@@ -52,26 +78,25 @@ async def startToChat(conn, text):
if conn.client_is_speaking:
await handleAbortMessage(conn)
# 首先进行意图分析
intent_handled = await handle_user_intent(conn, text)
# 首先进行意图分析,使用实际文本内容
intent_handled = await handle_user_intent(conn, actual_text)
if intent_handled:
# 如果意图已被处理,不再进行聊天
return
# 意图未被处理,继续常规聊天流程
await send_stt_message(conn, text)
conn.executor.submit(conn.chat, text)
# 意图未被处理,继续常规聊天流程,使用实际文本内容
await send_stt_message(conn, actual_text)
conn.executor.submit(conn.chat, actual_text)
async def no_voice_close_connect(conn, have_voice):
if have_voice:
conn.client_no_voice_last_time = 0.0
conn.last_activity_time = time.time() * 1000
return
if conn.client_no_voice_last_time == 0.0:
conn.client_no_voice_last_time = time.time() * 1000
else:
no_voice_time = time.time() * 1000 - conn.client_no_voice_last_time
# 只有在已经初始化过时间戳的情况下才进行超时检查
if conn.last_activity_time > 0.0:
no_voice_time = time.time() * 1000 - conn.last_activity_time
close_connection_no_voice_time = int(
conn.config.get("close_connection_no_voice_time", 120)
)
@@ -76,7 +76,6 @@ async def sendAudio(conn, audios, pre_buffer=True):
frame_duration = 60 # 帧时长(毫秒),匹配 Opus 编码
start_time = time.perf_counter()
play_position = 0
last_reset_time = time.perf_counter() # 记录最后的重置时间
# 仅当第一句话时执行预缓冲
if pre_buffer:
@@ -92,10 +91,8 @@ async def sendAudio(conn, audios, pre_buffer=True):
if conn.client_abort:
break
# 每分钟重置一次计时器
if time.perf_counter() - last_reset_time > 60:
await conn.reset_timeout()
last_reset_time = time.perf_counter()
# 重置没有声音的状态
conn.last_activity_time = time.time() * 1000
# 计算预期发送时间
expected_time = start_time + (play_position / 1000)
@@ -139,7 +136,21 @@ async def send_stt_message(conn, text):
return
"""发送 STT 状态消息"""
stt_text = get_string_no_punctuation_or_emoji(text)
# 解析JSON格式,提取实际的用户说话内容
display_text = text
try:
# 尝试解析JSON格式
if text.strip().startswith('{') and text.strip().endswith('}'):
parsed_data = json.loads(text)
if isinstance(parsed_data, dict) and "content" in parsed_data:
# 如果是包含说话人信息的JSON格式,只显示content部分
display_text = parsed_data["content"]
except (json.JSONDecodeError, TypeError):
# 如果不是JSON格式,直接使用原始文本
display_text = text
stt_text = get_string_no_punctuation_or_emoji(display_text)
await conn.websocket.send(
json.dumps({"type": "stt", "text": stt_text, "session_id": conn.session_id})
)
@@ -1,11 +1,11 @@
import json
from core.handle.abortHandle import handleAbortMessage
from core.handle.helloHandle import handleHelloMessage
from core.handle.mcpHandle import handle_mcp_message
from core.providers.tools.device_mcp import handle_mcp_message
from core.utils.util import remove_punctuation_and_length, filter_sensitive_info
from core.handle.receiveAudioHandle import startToChat, handleAudioMessage
from core.handle.sendAudioHandle import send_stt_message, send_tts_message
from core.handle.iotHandle import handleIotDescriptors, handleIotStatus
from core.providers.tools.device_iot import handleIotDescriptors, handleIotStatus
from core.handle.reportHandle import enqueue_asr_report
import asyncio
@@ -77,7 +77,7 @@ async def handleTextMessage(conn, message):
if "states" in msg_json:
asyncio.create_task(handleIotStatus(conn, msg_json["states"]))
elif msg_json["type"] == "mcp":
conn.logger.bind(tag=TAG).info(f"收到mcp消息:{message}")
conn.logger.bind(tag=TAG).info(f"收到mcp消息:{message[:100]}")
if "payload" in msg_json:
asyncio.create_task(
handle_mcp_message(conn, conn.mcp_client, msg_json["payload"])
-130
View File
@@ -1,130 +0,0 @@
"""MCP服务管理器"""
import asyncio
import os, json
from typing import Dict, Any, List
from .MCPClient import MCPClient
from plugins_func.register import register_function, ToolType
from config.config_loader import get_project_dir
TAG = __name__
class MCPManager:
"""管理多个MCP服务的集中管理器"""
def __init__(self, conn) -> None:
"""
初始化MCP管理器
"""
self.conn = conn
self.config_path = get_project_dir() + "data/.mcp_server_settings.json"
if os.path.exists(self.config_path) == False:
self.config_path = ""
self.conn.logger.bind(tag=TAG).warning(
f"请检查mcp服务配置文件:data/.mcp_server_settings.json"
)
self.client: Dict[str, MCPClient] = {}
self.tools = []
def load_config(self) -> Dict[str, Any]:
"""加载MCP服务配置
Returns:
Dict[str, Any]: 服务配置字典
"""
if len(self.config_path) == 0:
return {}
try:
with open(self.config_path, "r", encoding="utf-8") as f:
config = json.load(f)
return config.get("mcpServers", {})
except Exception as e:
self.conn.logger.bind(tag=TAG).error(
f"Error loading MCP config from {self.config_path}: {e}"
)
return {}
async def initialize_servers(self) -> None:
"""初始化所有MCP服务"""
config = self.load_config()
for name, srv_config in config.items():
if not srv_config.get("command") and not srv_config.get("url"):
self.conn.logger.bind(tag=TAG).warning(
f"Skipping server {name}: neither command nor url specified"
)
continue
try:
client = MCPClient(srv_config)
await client.initialize()
self.client[name] = client
self.conn.logger.bind(tag=TAG).info(f"Initialized MCP client: {name}")
client_tools = client.get_available_tools()
self.tools.extend(client_tools)
for tool in client_tools:
func_name = "mcp_" + tool["function"]["name"]
register_function(func_name, tool, ToolType.MCP_CLIENT)(
self.execute_tool
)
self.conn.func_handler.function_registry.register_function(
func_name
)
except Exception as e:
self.conn.logger.bind(tag=TAG).error(
f"Failed to initialize MCP server {name}: {e}"
)
def get_all_tools(self) -> List[Dict[str, Any]]:
"""获取所有服务的工具function定义
Returns:
List[Dict[str, Any]]: 所有工具的function定义列表
"""
return self.tools
def is_mcp_tool(self, tool_name: str) -> bool:
"""检查是否是MCP工具
Args:
tool_name: 工具名称
Returns:
bool: 是否是MCP工具
"""
for tool in self.tools:
if (
tool.get("function") != None
and tool["function"].get("name") == tool_name
):
return True
return False
async def execute_tool(self, tool_name: str, arguments: Dict[str, Any]) -> Any:
"""执行工具调用
Args:
tool_name: 工具名称
arguments: 工具参数
Returns:
Any: 工具执行结果
Raises:
ValueError: 工具未找到时抛出
"""
self.conn.logger.bind(tag=TAG).info(
f"Executing tool {tool_name} with arguments: {arguments}"
)
for client in self.client.values():
if client.has_tool(tool_name):
return await client.call_tool(tool_name, arguments)
raise ValueError(f"Tool {tool_name} not found in any MCP server")
async def cleanup_all(self) -> None:
"""依次关闭所有 MCPClient,不让异常阻断整体流程。"""
for name, client in list(self.client.items()):
try:
await asyncio.wait_for(client.cleanup(), timeout=20)
self.conn.logger.bind(tag=TAG).info(f"MCP client closed: {name}")
except (asyncio.TimeoutError, Exception) as e:
self.conn.logger.bind(tag=TAG).error(
f"Error closing MCP client {name}: {e}"
)
self.client.clear()
+179 -38
View File
@@ -1,19 +1,23 @@
import os
import wave
import copy
import uuid
import queue
import asyncio
import traceback
import threading
import opuslib_next
import json
import io
import time
import concurrent.futures
from abc import ABC, abstractmethod
from config.logger import setup_logging
from typing import Optional, Tuple, List
from typing import Optional, Tuple, List, Dict, Any
from core.handle.receiveAudioHandle import startToChat
from core.handle.reportHandle import enqueue_asr_report
from core.utils.util import remove_punctuation_and_length
from core.handle.receiveAudioHandle import handleAudioMessage
from core.utils.voiceprint_provider import VoiceprintProvider
TAG = __name__
logger = setup_logging()
@@ -21,13 +25,16 @@ logger = setup_logging()
class ASRProviderBase(ABC):
def __init__(self):
pass
self.voiceprint_provider = None
def init_voiceprint(self, voiceprint_config: dict):
"""初始化声纹识别"""
if voiceprint_config:
self.voiceprint_provider = VoiceprintProvider(voiceprint_config)
logger.bind(tag=TAG).info("声纹识别模块已初始化")
# 打开音频通道
# 这里默认是非流式的处理方式
# 流式处理方式请在子类中重写
async def open_audio_channels(self, conn):
# tts 消化线程
conn.asr_priority_thread = threading.Thread(
target=self.asr_text_priority_thread, args=(conn,), daemon=True
)
@@ -52,41 +59,173 @@ class ASRProviderBase(ABC):
continue
# 接收音频
# 这里默认是非流式的处理方式
# 流式处理方式请在子类中重写
async def receive_audio(self, conn, audio, audio_have_voice):
if conn.client_listen_mode == "auto" or conn.client_listen_mode == "realtime":
have_voice = audio_have_voice
else:
have_voice = conn.client_have_voice
# 如果本次没有声音,本段也没声音,就把声音丢弃了
conn.asr_audio.append(audio)
if have_voice == False and conn.client_have_voice == False:
if not have_voice and not conn.client_have_voice:
conn.asr_audio = conn.asr_audio[-10:]
return
# 如果本段有声音,且已经停止了
if conn.client_voice_stop:
asr_audio_task = copy.deepcopy(conn.asr_audio)
asr_audio_task = conn.asr_audio.copy()
conn.asr_audio.clear()
# 音频太短了,无法识别
conn.reset_vad_states()
if len(asr_audio_task) > 15:
await self.handle_voice_stop(conn, asr_audio_task)
# 处理语音停止
async def handle_voice_stop(self, conn, asr_audio_task):
raw_text, _ = await self.speech_to_text(
asr_audio_task, conn.session_id, conn.audio_format
) # 确保ASR模块返回原始文本
conn.logger.bind(tag=TAG).info(f"识别文本: {raw_text}")
text_len, _ = remove_punctuation_and_length(raw_text)
self.stop_ws_connection()
if text_len > 0:
# 使用自定义模块进行上报
await startToChat(conn, raw_text)
enqueue_asr_report(conn, raw_text, asr_audio_task)
async def handle_voice_stop(self, conn, asr_audio_task: List[bytes]):
"""并行处理ASR和声纹识别"""
try:
total_start_time = time.monotonic()
# 准备音频数据
if conn.audio_format == "pcm":
pcm_data = asr_audio_task
else:
pcm_data = self.decode_opus(asr_audio_task)
combined_pcm_data = b"".join(pcm_data)
# 预先准备WAV数据
wav_data = None
if self.voiceprint_provider and combined_pcm_data:
wav_data = self._pcm_to_wav(combined_pcm_data)
# 定义ASR任务
def run_asr():
start_time = time.monotonic()
try:
import asyncio
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
result = loop.run_until_complete(
self.speech_to_text(asr_audio_task, conn.session_id, conn.audio_format)
)
end_time = time.monotonic()
logger.bind(tag=TAG).info(f"ASR耗时: {end_time - start_time:.3f}s")
return result
finally:
loop.close()
except Exception as e:
end_time = time.monotonic()
logger.bind(tag=TAG).error(f"ASR失败: {e}")
return ("", None)
# 定义声纹识别任务
def run_voiceprint():
if not wav_data:
return None
start_time = time.monotonic()
try:
import asyncio
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
result = loop.run_until_complete(
self.voiceprint_provider.identify_speaker(wav_data, conn.session_id)
)
return result
finally:
loop.close()
except Exception as e:
logger.bind(tag=TAG).error(f"声纹识别失败: {e}")
return None
# 使用线程池执行器并行运行
parallel_start_time = time.monotonic()
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as thread_executor:
asr_future = thread_executor.submit(run_asr)
if self.voiceprint_provider and wav_data:
voiceprint_future = thread_executor.submit(run_voiceprint)
# 等待两个线程都完成
asr_result = asr_future.result(timeout=15)
voiceprint_result = voiceprint_future.result(timeout=15)
results = {"asr": asr_result, "voiceprint": voiceprint_result}
else:
asr_result = asr_future.result(timeout=15)
results = {"asr": asr_result, "voiceprint": None}
parallel_execution_time = time.monotonic() - parallel_start_time
# 处理结果
raw_text, file_path = results.get("asr", ("", None))
speaker_name = results.get("voiceprint", None)
# 记录识别结果
if raw_text:
logger.bind(tag=TAG).info(f"识别文本: {raw_text}")
if speaker_name:
logger.bind(tag=TAG).info(f"识别说话人: {speaker_name}")
# 性能监控
total_time = time.monotonic() - total_start_time
logger.bind(tag=TAG).info(f"总处理耗时: {total_time:.3f}s")
# 检查文本长度
text_len, _ = remove_punctuation_and_length(raw_text)
self.stop_ws_connection()
if text_len > 0:
# 构建包含说话人信息的JSON字符串
enhanced_text = self._build_enhanced_text(raw_text, speaker_name)
# 使用自定义模块进行上报
await startToChat(conn, enhanced_text)
enqueue_asr_report(conn, enhanced_text, asr_audio_task)
except Exception as e:
logger.bind(tag=TAG).error(f"处理语音停止失败: {e}")
import traceback
logger.bind(tag=TAG).debug(f"异常详情: {traceback.format_exc()}")
def _build_enhanced_text(self, text: str, speaker_name: Optional[str]) -> str:
"""构建包含说话人信息的文本"""
if speaker_name and speaker_name.strip():
return json.dumps({
"speaker": speaker_name,
"content": text
}, ensure_ascii=False)
else:
return text
def _pcm_to_wav(self, pcm_data: bytes) -> bytes:
"""将PCM数据转换为WAV格式"""
if len(pcm_data) == 0:
logger.bind(tag=TAG).warning("PCM数据为空,无法转换WAV")
return b""
# 确保数据长度是偶数(16位音频)
if len(pcm_data) % 2 != 0:
pcm_data = pcm_data[:-1]
# 创建WAV文件头
wav_buffer = io.BytesIO()
try:
with wave.open(wav_buffer, 'wb') as wav_file:
wav_file.setnchannels(1) # 单声道
wav_file.setsampwidth(2) # 16位
wav_file.setframerate(16000) # 16kHz采样率
wav_file.writeframes(pcm_data)
wav_buffer.seek(0)
wav_data = wav_buffer.read()
return wav_data
except Exception as e:
logger.bind(tag=TAG).error(f"WAV转换失败: {e}")
return b""
def stop_ws_connection(self):
pass
@@ -113,27 +252,29 @@ class ASRProviderBase(ABC):
pass
@staticmethod
def decode_opus(opus_data: List[bytes]) -> bytes:
def decode_opus(opus_data: List[bytes]) -> List[bytes]:
"""将Opus音频数据解码为PCM数据"""
try:
decoder = opuslib_next.Decoder(16000, 1) # 16kHz, 单声道
decoder = opuslib_next.Decoder(16000, 1)
pcm_data = []
buffer_size = 960 # 每次处理960个采样点
for opus_packet in opus_data:
buffer_size = 960 # 每次处理960个采样点 (60ms at 16kHz)
for i, opus_packet in enumerate(opus_data):
try:
# 使用较小的缓冲区大小进行处理
if not opus_packet or len(opus_packet) == 0:
continue
pcm_frame = decoder.decode(opus_packet, buffer_size)
if pcm_frame:
if pcm_frame and len(pcm_frame) > 0:
pcm_data.append(pcm_frame)
except opuslib_next.OpusError as e:
logger.bind(tag=TAG).warning(f"Opus解码错误,跳过当前数据包: {e}")
continue
logger.bind(tag=TAG).warning(f"Opus解码错误,跳过数据包 {i}: {e}")
except Exception as e:
logger.bind(tag=TAG).error(f"音频处理错误: {e}", exc_info=True)
continue
logger.bind(tag=TAG).error(f"音频处理错误,数据包 {i}: {e}")
return pcm_data
except Exception as e:
logger.bind(tag=TAG).error(f"音频解码过程发生错误: {e}", exc_info=True)
logger.bind(tag=TAG).error(f"音频解码过程发生错误: {e}")
return []
@@ -93,9 +93,7 @@ class ASRProvider(ASRProviderBase):
# 检查初始化响应
if "code" in result and result["code"] != 1000:
error_msg = f"ASR服务初始化失败: {result.get('payload_msg', {}).get('message', '未知错误')}"
if "payload_msg" in result:
error_msg += f"\n详细错误信息: {json.dumps(result['payload_msg'], ensure_ascii=False)}"
error_msg = f"ASR服务初始化失败: {result.get('payload_msg', {}).get('error', '未知错误')}"
logger.bind(tag=TAG).error(error_msg)
raise Exception(error_msg)
@@ -256,7 +254,6 @@ class ASRProvider(ASRProviderBase):
"X-Api-Access-Key": self.access_token,
"X-Api-Resource-Id": "volc.bigasr.sauc.duration",
"X-Api-Connect-Id": str(uuid.uuid4()),
"Host": "openspeech.bytedance.com",
}
def generate_header(
@@ -309,9 +306,14 @@ class ASRProvider(ASRProviderBase):
# 如果是错误响应
if message_type == 0x0F: # SERVER_ERROR_RESPONSE
code = int.from_bytes(header[4:8], "big", signed=False)
error_msg = res[8:].decode("utf-8")
return {"code": code, "error": error_msg}
code = int.from_bytes(res[4:8], "big", signed=False)
msg_length = int.from_bytes(res[8:12], "big", signed=False)
error_msg = json.loads(res[12:].decode("utf-8"))
return {
"code": code,
"msg_length": msg_length,
"payload_msg": error_msg,
}
# 获取JSON数据(跳过12字节头部)
try:
@@ -16,10 +16,11 @@ class IntentProvider(IntentProviderBase):
super().__init__(config)
self.llm = None
self.promot = ""
# 添加缓存管理
self.intent_cache = {} # 缓存意图识别结果
self.cache_expiry = 600 # 缓存有效期10分钟
self.cache_max_size = 100 # 最多缓存100个意图
# 导入全局缓存管理
from core.utils.cache.manager import cache_manager, CacheType
self.cache_manager = cache_manager
self.CacheType = CacheType
self.history_count = 4 # 默认使用最近4条对话记录
def get_intent_system_prompt(self, functions_list: str) -> str:
@@ -95,30 +96,13 @@ class IntentProvider(IntentProviderBase):
"1. 只返回JSON格式,不要包含任何其他文字\n"
'2. 如果没有找到匹配的函数,返回{"function_call": {"name": "continue_chat"}}\n'
"3. 确保返回的JSON格式正确,包含所有必要的字段\n"
"特殊说明:\n"
"- 当用户单次输入包含多个指令时(如'打开灯并且调高音量'\n"
"- 请返回多个function_call组成的JSON数组\n"
"- 示例:{'function_calls': [{name:'light_on'}, {name:'volume_up'}]}"
)
return prompt
def clean_cache(self):
"""清理过期缓存"""
now = time.time()
# 找出过期键
expired_keys = [
k
for k, v in self.intent_cache.items()
if now - v["timestamp"] > self.cache_expiry
]
for key in expired_keys:
del self.intent_cache[key]
# 如果缓存太大,移除最旧的条目
if len(self.intent_cache) > self.cache_max_size:
# 按时间戳排序并保留最新的条目
sorted_items = sorted(
self.intent_cache.items(), key=lambda x: x[1]["timestamp"]
)
for key, _ in sorted_items[: len(sorted_items) - self.cache_max_size]:
del self.intent_cache[key]
def replyResult(self, text: str, original_text: str):
llm_result = self.llm.response_no_stream(
system_prompt=text,
@@ -141,21 +125,16 @@ class IntentProvider(IntentProviderBase):
logger.bind(tag=TAG).debug(f"使用意图识别模型: {model_info}")
# 计算缓存键
cache_key = hashlib.md5(text.encode()).hexdigest()
cache_key = hashlib.md5((conn.device_id + text).encode()).hexdigest()
# 检查缓存
if cache_key in self.intent_cache:
cache_entry = self.intent_cache[cache_key]
# 检查缓存是否过期
if time.time() - cache_entry["timestamp"] <= self.cache_expiry:
cache_time = time.time() - total_start_time
logger.bind(tag=TAG).debug(
f"使用缓存的意图: {cache_key} -> {cache_entry['intent']}, 耗时: {cache_time:.4f}"
)
return cache_entry["intent"]
# 清理缓存
self.clean_cache()
cached_intent = self.cache_manager.get(self.CacheType.INTENT, cache_key)
if cached_intent is not None:
cache_time = time.time() - total_start_time
logger.bind(tag=TAG).debug(
f"使用缓存的意图: {cache_key} -> {cached_intent}, 耗时: {cache_time:.4f}"
)
return cached_intent
if self.promot == "":
functions = conn.func_handler.get_functions()
@@ -172,7 +151,11 @@ class IntentProvider(IntentProviderBase):
music_file_names = music_config["music_file_names"]
prompt_music = f"{self.promot}\n<musicNames>{music_file_names}\n</musicNames>"
devices = conn.config["plugins"]["home_assistant"].get("devices", [])
home_assistant_cfg = conn.config["plugins"].get("home_assistant")
if home_assistant_cfg:
devices = home_assistant_cfg.get("devices", [])
else:
devices = []
if len(devices) > 0:
hass_prompt = "\n下面是我家智能设备列表(位置,设备名,entity_id),可以通过homeassistant控制\n"
for device in devices:
@@ -251,10 +234,7 @@ class IntentProvider(IntentProviderBase):
conn.dialogue.dialogue = clean_history
# 添加到缓存
self.intent_cache[cache_key] = {
"intent": intent,
"timestamp": time.time(),
}
self.cache_manager.set(self.CacheType.INTENT, cache_key, intent)
# 后处理时间
postprocess_time = time.time() - postprocess_start_time
@@ -264,10 +244,7 @@ class IntentProvider(IntentProviderBase):
return intent
else:
# 添加到缓存
self.intent_cache[cache_key] = {
"intent": intent,
"timestamp": time.time(),
}
self.cache_manager.set(self.CacheType.INTENT, cache_key, intent)
# 后处理时间
postprocess_time = time.time() - postprocess_start_time
@@ -1,3 +1,4 @@
import httpx
import openai
from openai.types import CompletionUsage
from config.logger import setup_logging
@@ -16,6 +17,9 @@ class LLMProvider(LLMProviderBase):
self.base_url = config.get("base_url")
else:
self.base_url = config.get("url")
# 增加timeout的配置项,单位为秒
timeout = config.get("timeout", 300)
self.timeout = int(timeout) if timeout else 300
param_defaults = {
"max_tokens": (500, int),
@@ -42,7 +46,7 @@ class LLMProvider(LLMProviderBase):
model_key_msg = check_model_key("LLM", self.api_key)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url)
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url, timeout=httpx.Timeout(self.timeout))
def response(self, session_id, dialogue, **kwargs):
try:
@@ -5,6 +5,7 @@ import os
import yaml
from config.config_loader import get_project_dir
from config.manage_api_client import save_mem_local_short
from core.utils.util import check_model_key
short_term_memory_prompt = """
@@ -78,10 +79,11 @@ short_term_memory_prompt_only_content = """
1总结user的重要信息以便在未来的对话中提供更个性化的服务
2不要重复总结不要遗忘之前记忆除非原来的记忆超过了1800字内否则不要遗忘不要压缩用户的历史记忆
3用户操控的设备音量播放音乐天气退出不想对话等和用户本身无关的内容这些信息不需要加入到总结中
4不要把设备操控的成果结果和失败结果加入到总结中也不要把用户的一些废话加入到总结中
5不要为了总结而总结如果用户的聊天没有意义请返回原来的历史记录也是可以的
6只需要返回总结摘要严格控制在1800字内
7不要包含代码xml不需要解释注释和说明保存记忆时仅从对话提取信息不要混入示例内容
4聊天内容中的今天的日期时间今天的天气情况与用户事件无关的数据这些信息如果当成记忆存储会影响后序对话这些信息不需要加入到总结中
5不要把设备操控的成果结果和失败结果加入到总结也不要把用户的一些废话加入到总结中
6不要为了总结而总结如果用户的聊天没有意义请返回原来的历史记录也是可以的
7只需要返回总结摘要严格控制在1800字内
8不要包含代码xml不需要解释注释和说明保存记忆时仅从对话提取信息不要混入示例内容
"""
@@ -145,6 +147,10 @@ class MemoryProvider(MemoryProviderBase):
# 打印使用的模型信息
model_info = getattr(self.llm, "model_name", str(self.llm.__class__.__name__))
logger.bind(tag=TAG).debug(f"使用记忆保存模型: {model_info}")
api_key = getattr(self.llm, "api_key", None)
memory_key_msg = check_model_key("记忆总结专用LLM", api_key)
if memory_key_msg:
logger.bind(tag=TAG).error(memory_key_msg)
if self.llm is None:
logger.bind(tag=TAG).error("LLM is not set for memory provider")
return None

Some files were not shown because too many files have changed in this diff Show More