Hass/xiaozhi-esp32-server
1
0
Fork 0
mirror of https://github.com/xinnan-tech/xiaozhi-esp32-server.git synced 2026-07-22 07:03:53 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'newsnow' into add_local_tts_change_voice

Browse Source
This commit is contained in:
hrz
2025-06-30 10:48:06 +08:00
committed by GitHub
parent 0ca1a6286c 43d47a4f09
commit 3015b01740
131 changed files with 5529 additions and 1786 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+18 -9
View File
@@ -144,6 +144,11 @@
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
</td>
@@ -170,8 +175,8 @@
#### 🚀 部署方式选择
| 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 |
|---------|------|---------|---------|---------|---------|
| **最简化安装** | 智能对话、IOT、MCP、视觉感知，数据存储在配置文件 | 低配置环境，无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G，如果全API，要2核2G | - |
| **全模块安装** | 智能对话、IOT、MCP、视觉感知、OTA、智控台，数据存储在数据库 | 完整功能体验 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **最简化安装** | 智能对话、IOT、MCP、视觉感知 | 低配置环境，数据存储在配置文件，无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G，如果全API，要2核2G | - |
| **全模块安装** | 智能对话、IOT、MCP接入点、视觉感知、OTA、智控台 | 完整功能体验，数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G，如果全API，要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
> 💡 提示：以下是按最新代码部署后的测试平台，有需要可烧录测试，并发为6个，每天会清空数据
@@ -186,18 +191,22 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
#### 🚩 配置说明和推荐
> [!Note]
> 本项目默认的配置是`入门全免费`设置，如果想效果更优，推荐使用`流式配置`。
> 本项目提供两种配置方案：
>
> 本项目自`0.5.2`版本，已支持使用流式配置，相比`0.5`版本以前，响应速度提升约`2.5秒`
> 1. `入门全免费`配置：适合个人家庭使用，所有组件均采用免费方案，无需额外付费。
>
> 2. `流式配置`：适合演示、培训、超过2个并发等场景，采用流式处理技术，响应速度更快，体验更佳。
>
> 自`0.5.2`版本起，项目支持流式配置，相比早期版本，响应速度提升约`2.5秒`，显著改善用户体验。
| 模块名称 | 入门全免费设置 | 流式配置 |
|:---:|:---:|:---:|
| ASR(语音识别) | FunASR(本地) | 👍DoubaoStreamASR(火山流式语音识别) |
| ASR(语音识别) | FunASR(本地) | 👍FunASRServer 或 👍DoubaoStreamASR |
| LLM(大模型) | ChatGLMLLM(智谱glm-4-flash) | 👍DoubaoLLM(火山doubao-1-5-pro-32k-250115) |
| VLLM(视觉大模型) | ChatGLMVLLM(智谱glm-4v-flash) | 👍QwenVLVLLM(千问qwen2.5-vl-3b-instructh) |
| TTS(语音合成) | 👍LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) |
| Intent(意图识别) | function_call(函数调用) | ✅function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆） | ✅mem_local_short（本地短期记忆） |
| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山双流式语音合成) |
| Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆） | mem_local_short（本地短期记忆） |
#### 🔧 测试工具
本项目提供以下测试工具，帮助您验证系统和选择合适的模型：
@@ -222,7 +231,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 视觉感知系统 | 支持多种VLLM(视觉大模型)，实现多模态交互 |
| 意图识别系统 | 支持LLM意图识别、Function Call函数调用，提供插件化意图处理机制 |
| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆，具备记忆总结功能 |
| IOT/MCP控制协议 | 支持设备注册管理、智能控制接口，同时支持IOT、MCP控制协议 |
| 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 管理后台 | 提供Web管理界面，支持用户管理、系统配置和设备管理 |
| 测试工具 | 提供性能测试工具、视觉模型测试工具和音频交互测试工具 |
| 部署支持 | 支持Docker部署和本地部署，提供完整的配置文件管理 |
docs/homeassistant-integration.md
+9 -39
View File
@@ -89,55 +89,25 @@ http://homeassistant.local:8123
公司,台灯,switch.iot_cn_831898993_socn1_on_p_2_1;
```
这段字符，我们成为“设备清单字符”需要保存好，等一下有用。
这段字符，我们称为“设备清单字符”需要保存好，等一下有用。
#### 2. 登录`智控台`
使用管理员账号，登录`智控台`。点击顶部菜单`参数管理`，搜索`plugins.home_assistant.`，会有三条结果出来
![image-20250504051716417](images/image-ha-integration-06.png)
编辑`plugins.home_assistant.devices`，把刚才整理的设备清单字符粘贴进去。
使用管理员账号，登录`智控台`。在`智能体管理`，找到你的智能体，再点击`配置角色`。
将意图识别设置成`函数调用`或`LLM意图识别`。这时你会看到右侧有一个`编辑功能`。点击`编辑功能`按钮，会弹出`功能管理`的框。
编辑`plugins.home_assistant.base_url`，把你部署的`HomeAssistant`接口地址粘贴进去，我粘贴进去的地址是这样的
在`功能管理`的框里，你需要勾选`HomeAssistant设备状态查询`和`HomeAssistant设备状态修改`。
```
http://192.168.4.7:8123
```
勾选后，在`已选功能`点击`HomeAssistant设备状态查询`，然后在`参数配置`里配置你的`HomeAssistant`地址、密钥、设备清单字符。
编辑`plugins.home_assistant.api_key`，把你从`HomeAssistant`复制过来的密钥，粘贴进去
编辑好后，点击`保存配置`，这时`功能管理`的框会隐藏，这时你再点击保存智能体配置。
保存成功后，即可唤醒设备操作。
#### 3. 设置`意图识别`函数
在智控台，点击顶部菜单“模型配置”，在左侧栏，找到“意图识别”，找到id为`Intent_function_call`的意图，点击编辑
然后在弹框中，在原来的基础上追加两个函数：“hass_get_state”和“hass_set_state”
修改前
```
change_role;get_weather;get_news;play_music
```
修改后
```
change_role;get_weather;get_news;play_music;hass_get_state;hass_set_state
```
#### 4. 手动重启xiaozhi-server
重启xiaozhi-server程序
#### 5. 确认角色配置是否设置了函数意图识别
在智控台，点击顶部菜单“智能体管理”，找到设备所在的智能体，点击“配置角色”
确认意图识别(Intent)，是否选择“函数调用意图识别”
#### 6. 唤醒设别进行控制
#### 3. 唤醒设别进行控制
尝试和esp32说，“打开XXX灯”
docs/images/demo13.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 359 KiB

docs/images/image-ha-integration-06.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 260 KiB

docs/mcp-endpoint-integration.md
+134
View File
@@ -0,0 +1,134 @@
# MCP 接入点部署使用指南
本教程包含2个部分
- 1、如何开启mcp接入点
- 2、如何为智能体接入一个简单的mcp功能，如计算器功能
部署的前提条件：
- 1、你已经部署了全模块，因为mcp接入点需要全模块中的智控台功能
- 2、你想在不修改xiaozhi-server项目的前提下，扩展小智的功能
# 如何开启mcp接入点
## 第一步，下载mcp接入点项目源码
浏览器打开[mcp接入点项目地址](https://github.com/xinnan-tech/mcp-endpoint-server)
打开完，找到页面中一个绿色的按钮，写着`Code`的按钮，点开它，然后你就看到`Download ZIP`的按钮。
点击它，下载本项目源码压缩包。下载到你电脑后，解压它，此时它的名字可能叫`mcp-endpoint-server-main`
你需要把它重命名成`mcp-endpoint-server`。
## 第二步，启动程序
这个项目是一个很简单的项目，建议使用docker运行。不过如果你不想使用docker运行，你可以参考[这个页面](https://github.com/xinnan-tech/mcp-endpoint-server/blob/main/README_dev.md)使用源码运行。以下是docker运行的方法
```
# 进入本项目源码根目录
cd mcp-endpoint-server
# 清除缓存
docker compose -f docker-compose.yml down
docker stop mcp-endpoint-server
docker rm mcp-endpoint-server
docker rmi ghcr.nju.edu.cn/xinnan-tech/mcp-endpoint-server:latest
# 启动docker容器
docker compose -f docker-compose.yml up -d
# 查看日志
docker logs -f mcp-endpoint-server
```
此时，日志里会输出类似以下的日志
```
======================================================
接口地址: http://172.1.1.1:8004/mcp_endpoint/health?key=xxxx
=======上面的地址是MCP接入点地址，请勿泄露给任何人============
```
请你把接口地址复制出来：
由于你是docker部署，切不可直接使用上面的地址！
由于你是docker部署，切不可直接使用上面的地址！
由于你是docker部署，切不可直接使用上面的地址！
你先把地址复制出来，放在一个草稿里，你要知道你的电脑的局域网ip是什么，例如我的电脑局域网ip是`192.168.1.25`，那么
原来我的接口地址
```
http://172.1.1.1:8004/mcp_endpoint/health?key=xxxx
```
就要改成
```
http://192.168.1.25:8004/mcp_endpoint/health?key=xxxx
```
改好后，请使用浏览器直接访问这个接口。当浏览器出现类似这样的代码，说明是成功了。
```
{"result":{"status":"success","connections":{"tool_connections":0,"robot_connections":0,"total_connections":0}},"error":null,"id":null,"jsonrpc":"2.0"}
```
请你保留好这个`接口地址`，下一步要用到。
## 第三步，配置智控台
使用管理员账号，登录智控台，点击顶部`参数字典`，选择`参数管理`功能。
然后搜索参数`server.mcp_endpoint`，此时，它的值应该是`null`值。
点击修改按钮，把上一步得来的`接口地址`粘贴到`参数值`里。然后保存。
如果能保存成功，说明一切顺利，你可以去智能体查看效果了。如果不成功，说明智控台无法访问mcp接入点，很大概率是网络防火墙，或者没有填写正确的局域网ip。
# 如何为智能体接入一个简单的mcp功能，如计算器功能
如果以上步骤顺利，你可以进入智能体管理，点击`配置角色`，在`意图识别`的右边，有一个`编辑功能`的按钮。
点击这个按钮。在弹出的页面里，位于底部，会有`MCP接入点`，正常来说，会显示这个智能体的`MCP接入点地址`，接下来，我们来给这个智能体扩展一个基于MCP技术的计算器的功能。
这个`MCP接入点地址`很重要，你等一下会用到。
## 第一步 下载虾哥MCP计算器项目代码
浏览器打开虾哥写的[计算器项目](https://github.com/78/mcp-calculator)，
打开完，找到页面中一个绿色的按钮，写着`Code`的按钮，点开它，然后你就看到`Download ZIP`的按钮。
点击它，下载本项目源码压缩包。下载到你电脑后，解压它，此时它的名字可能叫`mcp-calculatorr-main`
你需要把它重命名成`mcp-calculator`。接下来，我们用命令行进入项目目录即安装依赖
```bash
# 进入项目目录
cd mcp-calculator
conda remove -n mcp-calculator --all -y
conda create -n mcp-calculator python=3.10 -y
conda activate mcp-calculator
pip install -r requirements.txt
```
## 第二步 启动
启动前，先从你的智控台的智能体里，复制到了MCP接入点的地址。
例如我的智能体的mcp地址是
```
ws://192.168.4.7:8004/mcp_endpoint/mcp/?token=abc
```
开始输入命令
```bash
export MCP_ENDPOINT=ws://192.168.4.7:8004/mcp_endpoint/mcp/?token=abc
```
输入完后，启动程序
```bash
python mcp_pipe.py calculator.py
```
启动完后，你再进入智控台，点击刷新MCP的接入状态，就会看到你扩展的功能列表了。
docs/newsnow_plugin_config.md
+84
View File
@@ -0,0 +1,84 @@
# get_news_from_newsnow 插件新闻源配置指南
## 概述
`get_news_from_newsnow` 插件现在支持通过Web管理界面动态配置新闻源，不再需要修改代码。用户可以在智控台中为每个智能体配置不同的新闻源。
## 配置方式
### 1. 通过Web管理界面配置（推荐）
1. 登录智控台
2. 进入"角色配置"页面
3. 选择要配置的智能体
4. 点击"编辑功能"按钮
5. 在右侧参数配置区域找到"newsnow新闻聚合"插件
6. 在"新闻源配置"字段中输入JSON格式的新闻源配置
### 2. 配置文件方式
在 `config.yaml` 中配置：
```yaml
plugins:
get_news_from_newsnow:
url: "https://newsnow.busiyi.world/api/s?id="
news_sources:
thepaper: "澎湃新闻"
baidu: "百度热搜"
cls-depth: "财联社"
# 可以添加更多新闻源
weibo: "微博头条"
douyin: "抖音热搜"
solidot: "奇客Solidot"
```
## 新闻源配置格式
新闻源配置使用JSON格式，格式为：
```json
{
"source_id": "显示名称",
"source_id2": "显示名称2"
}
```
### 配置示例
```json
{
"thepaper": "澎湃新闻",
"baidu": "百度热搜",
"cls-depth": "财联社",
"weibo": "微博头条",
"douyin": "抖音热搜",
"solidot": "奇客Solidot"
}
```
## 默认配置
如果未配置新闻源，插件将使用以下默认配置：
```json
{
"thepaper": "澎湃新闻",
"baidu": "百度热搜",
"cls-depth": "财联社"
}
```
## 使用说明
1. **配置新闻源**：在Web界面或配置文件中设置新闻源
2. **调用插件**：用户可以说"播报新闻"或"获取新闻"
3. **指定新闻源**：用户可以说"播报澎湃新闻"或"获取百度热搜"
4. **获取详情**：用户可以说"详细介绍这条新闻"
## 注意事项
1. 新闻源的 `source_id` 必须与API接口支持的ID一致
2. 配置更改后需要重启服务或重新加载配置
3. 如果配置的新闻源无效，插件会自动使用默认新闻源
4. JSON格式必须正确，否则会使用默认配置
main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java
+6 -1
View File
@@ -111,6 +111,11 @@ public interface Constant {
*/
String FILE_EXTENSION_SEG = ".";
/**
* mcp接入点路径
*/
String SERVER_MCP_ENDPOINT = "server.mcp_endpoint";
/**
* 无记忆
*/
@@ -227,7 +232,7 @@ public interface Constant {
/**
* 版本号
*/
public static final String VERSION = "0.5.5";
public static final String VERSION = "0.6.1";
/**
* 无效固件URL
main/manager-api/src/main/java/xiaozhi/common/utils/AESUtils.java
+78
View File
@@ -0,0 +1,78 @@
package xiaozhi.common.utils;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import javax.crypto.Cipher;
import javax.crypto.spec.SecretKeySpec;
public class AESUtils {
private static final String ALGORITHM = "AES";
private static final String TRANSFORMATION = "AES/ECB/PKCS5Padding";
/**
* AES加密
*
* @param key 密钥（16位、24位或32位）
* @param plainText 待加密字符串
* @return 加密后的Base64字符串
*/
public static String encrypt(String key, String plainText) {
try {
// 确保密钥长度为16、24或32位
byte[] keyBytes = padKey(key.getBytes(StandardCharsets.UTF_8));
SecretKeySpec secretKey = new SecretKeySpec(keyBytes, ALGORITHM);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
byte[] encryptedBytes = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(encryptedBytes);
} catch (Exception e) {
throw new RuntimeException("AES加密失败", e);
}
}
/**
* AES解密
*
* @param key 密钥（16位、24位或32位）
* @param encryptedText 待解密的Base64字符串
* @return 解密后的字符串
*/
public static String decrypt(String key, String encryptedText) {
try {
// 确保密钥长度为16、24或32位
byte[] keyBytes = padKey(key.getBytes(StandardCharsets.UTF_8));
SecretKeySpec secretKey = new SecretKeySpec(keyBytes, ALGORITHM);
Cipher cipher = Cipher.getInstance(TRANSFORMATION);
cipher.init(Cipher.DECRYPT_MODE, secretKey);
byte[] encryptedBytes = Base64.getDecoder().decode(encryptedText);
byte[] decryptedBytes = cipher.doFinal(encryptedBytes);
return new String(decryptedBytes, StandardCharsets.UTF_8);
} catch (Exception e) {
throw new RuntimeException("AES解密失败", e);
}
}
/**
* 填充密钥到指定长度（16、24或32位）
*
* @param keyBytes 原始密钥字节数组
* @return 填充后的密钥字节数组
*/
private static byte[] padKey(byte[] keyBytes) {
int keyLength = keyBytes.length;
if (keyLength == 16 || keyLength == 24 || keyLength == 32) {
return keyBytes;
}
// 如果密钥长度不足，用0填充；如果超过，截取前32位
byte[] paddedKey = new byte[32];
System.arraycopy(keyBytes, 0, paddedKey, 0, Math.min(keyLength, 32));
return paddedKey;
}
}
main/manager-api/src/main/java/xiaozhi/common/utils/HashEncryptionUtil.java
+52
View File
@@ -0,0 +1,52 @@
package xiaozhi.common.utils;
import lombok.extern.slf4j.Slf4j;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
/**
* 哈希加密算法的工具类
* @author zjy
*/
@Slf4j
public class HashEncryptionUtil {
/**
* 使用md5进行加密
* @param context 被加密的内容
* @return 哈希值
*/
public static String Md5hexDigest(String context){
return hexDigest(context,"MD5");
}
/**
* 指定哈希算法进行加密
* @param context 被加密的内容
* @param algorithm 哈希算法
* @return 哈希值
*/
public static String hexDigest(String context,String algorithm ){
// 获取MD5算法实例
MessageDigest md = null;
try {
md = MessageDigest.getInstance(algorithm);
} catch (NoSuchAlgorithmException e) {
log.error("加密失败的算法：{}",algorithm);
throw new RuntimeException("加密失败，"+ algorithm +"哈希算法系统不支持");
}
// 计算智能体id的MD5值
byte[] messageDigest = md.digest(context.getBytes());
// 将字节数组转换为十六进制字符串
StringBuilder hexString = new StringBuilder();
for (byte b : messageDigest) {
String hex = Integer.toHexString(0xFF & b);
if (hex.length() == 1) {
hexString.append('0');
}
hexString.append(hex);
}
return hexString.toString();
}
}
main/manager-api/src/main/java/xiaozhi/common/utils/ResultUtils.java
+31
View File
@@ -0,0 +1,31 @@
package xiaozhi.common.utils;
/**
* 返回响应体工具类
*/
public class ResultUtils
{
public static <T> Result<T> success(T data) {
return new Result<T>().ok(data);
}
public static <T> Result<T> error() {
return new Result<T>().error();
}
public static <T> Result<T> error(String msg) {
return new Result<T>().error(msg);
}
public static <T> Result<T> error(int errorCode, String msg) {
return new Result<T>().error(errorCode, msg);
}
public static <T> Result<T> error(int errorCode) {
return new Result<T>().error(errorCode);
}
public static <T> Result<T> empty() {
return new Result<T>();
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java
+14 -112
View File
@@ -1,6 +1,5 @@
package xiaozhi.modules.agent.controller;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.UUID;
@@ -33,8 +32,8 @@ import xiaozhi.common.page.PageData;
import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.ConvertUtils;
import xiaozhi.common.utils.Result;
import xiaozhi.common.utils.ResultUtils;
import xiaozhi.modules.agent.dto.AgentChatHistoryDTO;
import xiaozhi.modules.agent.dto.AgentChatSessionDTO;
import xiaozhi.modules.agent.dto.AgentCreateDTO;
@@ -45,8 +44,10 @@ import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
import xiaozhi.modules.agent.service.AgentChatAudioService;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.agent.vo.AgentInfoVO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.security.user.SecurityUser;
@@ -61,6 +62,7 @@ public class AgentController {
private final DeviceService deviceService;
private final AgentChatHistoryService agentChatHistoryService;
private final AgentChatAudioService agentChatAudioService;
private final AgentPluginMappingService agentPluginMappingService;
private final RedisUtils redisUtils;
@GetMapping("/list")
@@ -88,46 +90,17 @@ public class AgentController {
@GetMapping("/{id}")
@Operation(summary = "获取智能体详情")
@RequiresPermissions("sys:role:normal")
public Result<AgentEntity> getAgentById(@PathVariable("id") String id) {
AgentEntity agent = agentService.getAgentById(id);
return new Result<AgentEntity>().ok(agent);
public Result<AgentInfoVO> getAgentById(@PathVariable("id") String id) {
AgentInfoVO agent = agentService.getAgentById(id);
return ResultUtils.success(agent);
}
@PostMapping
@Operation(summary = "创建智能体")
@RequiresPermissions("sys:role:normal")
public Result<String> save(@RequestBody @Valid AgentCreateDTO dto) {
AgentEntity entity = ConvertUtils.sourceToTarget(dto, AgentEntity.class);
// 获取默认模板
AgentTemplateEntity template = agentTemplateService.getDefaultTemplate();
if (template != null) {
// 设置模板中的默认值
entity.setAsrModelId(template.getAsrModelId());
entity.setVadModelId(template.getVadModelId());
entity.setLlmModelId(template.getLlmModelId());
entity.setVllmModelId(template.getVllmModelId());
entity.setTtsModelId(template.getTtsModelId());
entity.setTtsVoiceId(template.getTtsVoiceId());
entity.setMemModelId(template.getMemModelId());
entity.setIntentModelId(template.getIntentModelId());
entity.setSystemPrompt(template.getSystemPrompt());
entity.setSummaryMemory(template.getSummaryMemory());
entity.setChatHistoryConf(template.getChatHistoryConf());
entity.setLangCode(template.getLangCode());
entity.setLanguage(template.getLanguage());
}
// 设置用户ID和创建者信息
UserDetail user = SecurityUser.getUser();
entity.setUserId(user.getId());
entity.setCreator(user.getId());
entity.setCreatedAt(new Date());
// ID、智能体编码和排序会在Service层自动生成
agentService.insert(entity);
return new Result<String>().ok(entity.getId());
String agentId = agentService.createAgent(dto);
return new Result<String>().ok(agentId);
}
@PutMapping("/saveMemory/{macAddress}")
@@ -139,88 +112,15 @@ public class AgentController {
}
AgentUpdateDTO agentUpdateDTO = new AgentUpdateDTO();
agentUpdateDTO.setSummaryMemory(dto.getSummaryMemory());
return updateAgentById(device.getAgentId(), agentUpdateDTO);
agentService.updateAgentById(device.getAgentId(), agentUpdateDTO);
return new Result<>();
}
@PutMapping("/{id}")
@Operation(summary = "更新智能体")
@RequiresPermissions("sys:role:normal")
public Result<Void> update(@PathVariable String id, @RequestBody @Valid AgentUpdateDTO dto) {
return updateAgentById(id, dto);
}
private Result<Void> updateAgentById(String id, AgentUpdateDTO dto) {
// 先查询现有实体
AgentEntity existingEntity = agentService.getAgentById(id);
if (existingEntity == null) {
return new Result<Void>().error("智能体不存在");
}
// 只更新提供的非空字段
if (dto.getAgentName() != null) {
existingEntity.setAgentName(dto.getAgentName());
}
if (dto.getAgentCode() != null) {
existingEntity.setAgentCode(dto.getAgentCode());
}
if (dto.getAsrModelId() != null) {
existingEntity.setAsrModelId(dto.getAsrModelId());
}
if (dto.getVadModelId() != null) {
existingEntity.setVadModelId(dto.getVadModelId());
}
if (dto.getLlmModelId() != null) {
existingEntity.setLlmModelId(dto.getLlmModelId());
}
if (dto.getVllmModelId() != null) {
existingEntity.setVllmModelId(dto.getVllmModelId());
}
if (dto.getTtsModelId() != null) {
existingEntity.setTtsModelId(dto.getTtsModelId());
}
if (dto.getTtsVoiceId() != null) {
existingEntity.setTtsVoiceId(dto.getTtsVoiceId());
}
if (dto.getMemModelId() != null) {
existingEntity.setMemModelId(dto.getMemModelId());
}
if (dto.getIntentModelId() != null) {
existingEntity.setIntentModelId(dto.getIntentModelId());
}
if (dto.getSystemPrompt() != null) {
existingEntity.setSystemPrompt(dto.getSystemPrompt());
}
if (dto.getSummaryMemory() != null) {
existingEntity.setSummaryMemory(dto.getSummaryMemory());
}
if (dto.getChatHistoryConf() != null) {
existingEntity.setChatHistoryConf(dto.getChatHistoryConf());
}
if (dto.getLangCode() != null) {
existingEntity.setLangCode(dto.getLangCode());
}
if (dto.getLanguage() != null) {
existingEntity.setLanguage(dto.getLanguage());
}
if (dto.getSort() != null) {
existingEntity.setSort(dto.getSort());
}
// 设置更新者信息
UserDetail user = SecurityUser.getUser();
existingEntity.setUpdater(user.getId());
existingEntity.setUpdatedAt(new Date());
// 更新记忆策略
if (existingEntity.getMemModelId() == null || existingEntity.getMemModelId().equals(Constant.MEMORY_NO_MEM)) {
// 删除所有记录
agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, true);
existingEntity.setSummaryMemory("");
} else if (existingEntity.getChatHistoryConf() != null && existingEntity.getChatHistoryConf() == 1) {
// 删除音频数据
agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, false);
}
agentService.updateById(existingEntity);
agentService.updateAgentById(id, dto);
return new Result<>();
}
@@ -232,6 +132,8 @@ public class AgentController {
deviceService.deleteByAgentId(id);
// 删除关联的聊天记录
agentChatHistoryService.deleteByAgentId(id, true, true);
// 删除关联的插件
agentPluginMappingService.deleteByAgentId(id);
// 再删除智能体
agentService.deleteById(id);
return new Result<>();
main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentMcpAccessPointController.java
+66
View File
@@ -0,0 +1,66 @@
package xiaozhi.modules.agent.controller;
import java.util.List;
import org.apache.shiro.authz.annotation.RequiresPermissions;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.security.user.SecurityUser;
@Tag(name = "智能体Mcp接入点管理")
@RequiredArgsConstructor
@RestController
@RequestMapping("/agent/mcp")
public class AgentMcpAccessPointController {
private final AgentMcpAccessPointService agentMcpAccessPointService;
private final AgentService agentService;
/**
* 获取智能体的Mcp接入点地址
*
* @param audioId 智能体id
* @return 返回错误提醒或者Mcp接入点地址
*/
@Operation(summary = "获取智能体的Mcp接入点地址")
@GetMapping("/address/{agentId}")
@RequiresPermissions("sys:role:normal")
public Result<String> getAgentMcpAccessAddress(@PathVariable("agentId") String agentId) {
// 获取当前用户
UserDetail user = SecurityUser.getUser();
// 检查权限
if (!agentService.checkAgentPermission(agentId, user.getId())) {
return new Result<String>().error("没有权限查看该智能体的MCP接入点地址");
}
String agentMcpAccessAddress = agentMcpAccessPointService.getAgentMcpAccessAddress(agentId);
if (agentMcpAccessAddress == null) {
return new Result<String>().ok("请联系管理员进入参数管理配置mcp接入点地址");
}
return new Result<String>().ok(agentMcpAccessAddress);
}
@Operation(summary = "获取智能体的Mcp工具列表")
@GetMapping("/tools/{agentId}")
@RequiresPermissions("sys:role:normal")
public Result<List<String>> getAgentMcpToolsList(@PathVariable("agentId") String agentId) {
// 获取当前用户
UserDetail user = SecurityUser.getUser();
// 检查权限
if (!agentService.checkAgentPermission(agentId, user.getId())) {
return new Result<List<String>>().error("没有权限查看该智能体的MCP工具列表");
}
List<String> agentMcpToolsList = agentMcpAccessPointService.getAgentMcpToolsList(agentId);
return new Result<List<String>>().ok(agentMcpToolsList);
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/dao/AgentDao.java
+8
View File
@@ -6,6 +6,7 @@ import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Select;
import xiaozhi.common.dao.BaseDao;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.vo.AgentInfoVO;
@Mapper
public interface AgentDao extends BaseDao<AgentEntity> {
@@ -28,4 +29,11 @@ public interface AgentDao extends BaseDao<AgentEntity> {
" WHERE d.mac_address = #{macAddress} " +
" ORDER BY d.id DESC LIMIT 1")
AgentEntity getDefaultAgentByMacAddress(@Param("macAddress") String macAddress);
/**
* 根据id查询agent信息，包括插件信息
*
* @param agentId 智能体ID
*/
AgentInfoVO selectAgentInfoById(@Param("agentId") String agentId);
}
main/manager-api/src/main/java/xiaozhi/modules/agent/dao/AgentPluginMappingMapper.java
+22
View File
@@ -0,0 +1,22 @@
package xiaozhi.modules.agent.dao;
import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import java.util.List;
/**
* @description 针对表【ai_agent_plugin_mapping(Agent与插件的唯一映射表)】的数据库操作Mapper
* @createDate 2025-05-25 22:33:17
* @Entity xiaozhi.modules.agent.entity.AgentPluginMapping
*/
@Mapper
public interface AgentPluginMappingMapper extends BaseMapper<AgentPluginMapping> {
List<AgentPluginMapping> selectPluginsByAgentId(@Param("agentId") String agentId);
}
main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentUpdateDTO.java
+32 -15
View File
@@ -1,6 +1,8 @@
package xiaozhi.modules.agent.dto;
import java.io.Serializable;
import java.util.HashMap;
import java.util.List;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
@@ -15,19 +17,19 @@ import lombok.Data;
public class AgentUpdateDTO implements Serializable {
private static final long serialVersionUID = 1L;
@Schema(description = "智能体编码", example = "AGT_1234567890", required = false)
@Schema(description = "智能体编码", example = "AGT_1234567890", nullable = true)
private String agentCode;
@Schema(description = "智能体名称", example = "客服助手", required = false)
@Schema(description = "智能体名称", example = "客服助手", nullable = true)
private String agentName;
@Schema(description = "语音识别模型标识", example = "asr_model_02", required = false)
@Schema(description = "语音识别模型标识", example = "asr_model_02", nullable = true)
private String asrModelId;
@Schema(description = "语音活动检测标识", example = "vad_model_02", required = false)
@Schema(description = "语音活动检测标识", example = "vad_model_02", nullable = true)
private String vadModelId;
@Schema(description = "大语言模型标识", example = "llm_model_02", required = false)
@Schema(description = "大语言模型标识", example = "llm_model_02", nullable = true)
private String llmModelId;
@Schema(description = "VLLM模型标识", example = "vllm_model_02", required = false)
@@ -36,31 +38,46 @@ public class AgentUpdateDTO implements Serializable {
@Schema(description = "语音合成模型标识", example = "tts_model_02", required = false)
private String ttsModelId;
@Schema(description = "音色标识", example = "voice_02", required = false)
@Schema(description = "音色标识", example = "voice_02", nullable = true)
private String ttsVoiceId;
@Schema(description = "记忆模型标识", example = "mem_model_02", required = false)
@Schema(description = "记忆模型标识", example = "mem_model_02", nullable = true)
private String memModelId;
@Schema(description = "意图模型标识", example = "intent_model_02", required = false)
@Schema(description = "意图模型标识", example = "intent_model_02", nullable = true)
private String intentModelId;
@Schema(description = "角色设定参数", example = "你是一个专业的客服助手，负责回答用户问题并提供帮助", required = false)
@Schema(description = "插件函数信息", nullable = true)
private List<FunctionInfo> functions;
@Schema(description = "角色设定参数", example = "你是一个专业的客服助手，负责回答用户问题并提供帮助", nullable = true)
private String systemPrompt;
@Schema(description = "总结记忆", example = "构建可生长的动态记忆网络，在有限空间内保留关键信息的同时，智能维护信息演变轨迹\n" +
"根据对话记录，总结user的重要信息，以便在未来的对话中提供更个性化的服务", required = false)
@Schema(description = "总结记忆", example = "构建可生长的动态记忆网络，在有限空间内保留关键信息的同时，智能维护信息演变轨迹\n"
+ "根据对话记录，总结user的重要信息，以便在未来的对话中提供更个性化的服务", nullable = true)
private String summaryMemory;
@Schema(description = "聊天记录配置（0不记录 1仅记录文本 2记录文本和语音）", example = "3", required = false)
@Schema(description = "聊天记录配置（0不记录 1仅记录文本 2记录文本和语音）", example = "3", nullable = true)
private Integer chatHistoryConf;
@Schema(description = "语言编码", example = "zh_CN", required = false)
@Schema(description = "语言编码", example = "zh_CN", nullable = true)
private String langCode;
@Schema(description = "交互语种", example = "中文", required = false)
@Schema(description = "交互语种", example = "中文", nullable = true)
private String language;
@Schema(description = "排序", example = "1", required = false)
@Schema(description = "排序", example = "1", nullable = true)
private Integer sort;
@Data
@Schema(description = "插件函数信息")
public static class FunctionInfo implements Serializable {
@Schema(description = "插件ID", example = "plugin_01")
private String pluginId;
@Schema(description = "函数参数信息", nullable = true)
private HashMap<String, Object> paramInfo;
private static final long serialVersionUID = 1L;
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/dto/McpJsonRpcRequest.java
+32
View File
@@ -0,0 +1,32 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* MCP JSON-RPC 请求 DTO
*/
@Data
public class McpJsonRpcRequest {
private String jsonrpc = "2.0";
private String method;
private Object params;
private Integer id;
public McpJsonRpcRequest() {
}
public McpJsonRpcRequest(String method) {
this.method = method;
}
public McpJsonRpcRequest(String method, Object params, Integer id) {
this.method = method;
this.params = params;
this.id = id;
}
public McpJsonRpcRequest(String method, Object params) {
this.method = method;
this.params = params;
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/dto/McpJsonRpcResponse.java
+48
View File
@@ -0,0 +1,48 @@
package xiaozhi.modules.agent.dto;
import lombok.Data;
/**
* MCP JSON-RPC 响应 DTO
*/
@Data
public class McpJsonRpcResponse {
private String jsonrpc = "2.0";
private Integer id;
private McpResult result;
private McpError error;
public McpJsonRpcResponse() {
}
@Data
public static class McpResult {
private String type;
private String message;
private String agent_id;
private McpTool[] tools;
public McpResult() {
}
}
@Data
public static class McpTool {
private String name;
private String description;
private Object inputSchema;
public McpTool() {
}
}
@Data
public static class McpError {
private Integer code;
private String message;
private Object data;
public McpError() {
}
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentPluginMapping.java
+54
View File
@@ -0,0 +1,54 @@
package xiaozhi.modules.agent.entity;
import java.io.Serializable;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
/**
* Agent与插件的唯一映射表
*
* @TableName ai_agent_plugin_mapping
*/
@Data
@TableName(value = "ai_agent_plugin_mapping")
@Schema(description = "Agent与插件的唯一映射表")
public class AgentPluginMapping implements Serializable {
/**
* 主键
*/
@TableId(type = IdType.ASSIGN_ID)
@Schema(description = "映射信息主键ID")
private Long id;
/**
* 智能体ID
*/
@Schema(description = "智能体ID")
private String agentId;
/**
* 插件ID
*/
@Schema(description = "插件ID")
private String pluginId;
/**
* 插件参数(Json)格式
*/
@Schema(description = "插件参数(Json)格式")
private String paramInfo;
// 冗余字段，用于方便在根据id查询插件时，对照查出插件的Provider_code,详见dao层xml文件
@TableField(exist = false)
@Schema(description = "插件provider_code, 对应表ai_model_provider")
private String providerCode;
@TableField(exist = false)
private static final long serialVersionUID = 1L;
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/AgentMcpAccessPointService.java
+25
View File
@@ -0,0 +1,25 @@
package xiaozhi.modules.agent.service;
import java.util.List;
/**
* 智能体Mcp接入点处理service
*
* @author zjy
*/
public interface AgentMcpAccessPointService {
/**
* 获取智能体的mcp接入点地址
* @param id 智能体id
* @return mcp接入点地址
*/
String getAgentMcpAccessAddress(String id);
/**
* 获取智能体的mcp接入点已有的工具列表
* @param id 智能体id
* @return 工具列表
*/
List<String> getAgentMcpToolsList(String id);
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/AgentPluginMappingService.java
+29
View File
@@ -0,0 +1,29 @@
package xiaozhi.modules.agent.service;
import java.util.List;
import com.baomidou.mybatisplus.extension.service.IService;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
/**
* @description 针对表【ai_agent_plugin_mapping(Agent与插件的唯一映射表)】的数据库操作Service
* @createDate 2025-05-25 22:33:17
*/
public interface AgentPluginMappingService extends IService<AgentPluginMapping> {
/**
* 根据智能体id获取插件参数
*
* @param agentId
* @return
*/
List<AgentPluginMapping> agentPluginParamsByAgentId(String agentId);
/**
* 根据智能体id删除插件参数
*
* @param agentId
*/
void deleteByAgentId(String agentId);
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/AgentService.java
+20 -1
View File
@@ -5,8 +5,11 @@ import java.util.Map;
import xiaozhi.common.page.PageData;
import xiaozhi.common.service.BaseService;
import xiaozhi.modules.agent.dto.AgentCreateDTO;
import xiaozhi.modules.agent.dto.AgentDTO;
import xiaozhi.modules.agent.dto.AgentUpdateDTO;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.vo.AgentInfoVO;
/**
* 智能体表处理service
@@ -30,7 +33,7 @@ public interface AgentService extends BaseService<AgentEntity> {
* @param id 智能体ID
* @return 智能体实体
*/
AgentEntity getAgentById(String id);
AgentInfoVO getAgentById(String id);
/**
* 插入智能体
@@ -79,4 +82,20 @@ public interface AgentService extends BaseService<AgentEntity> {
* @return 是否有权限
*/
boolean checkAgentPermission(String agentId, Long userId);
/**
* 更新智能体
*
* @param agentId 智能体ID
* @param dto 更新智能体所需的信息
*/
void updateAgentById(String agentId, AgentUpdateDTO dto);
/**
* 创建智能体
*
* @param dto 创建智能体所需的信息
* @return 创建的智能体ID
*/
String createAgent(AgentCreateDTO dto);
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java
+12
View File
@@ -19,6 +19,8 @@ import xiaozhi.modules.agent.service.AgentChatAudioService;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.biz.AgentChatHistoryBizService;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
/**
* {@link AgentChatHistoryBizService} impl
@@ -35,6 +37,7 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
private final AgentChatHistoryService agentChatHistoryService;
private final AgentChatAudioService agentChatAudioService;
private final RedisUtils redisUtils;
private final DeviceService deviceService;
/**
* 处理聊天记录上报，包括文件上传和相关信息记录
@@ -68,6 +71,15 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic
// 更新设备最后对话时间
redisUtils.set(RedisKeys.getAgentDeviceLastConnectedAtById(agentId), new Date());
// 更新设备最后连接时间
DeviceEntity device = deviceService.getDeviceByMacAddress(macAddress);
if (device != null) {
deviceService.updateDeviceConnectionInfo(agentId, device.getId(), null);
} else {
log.warn("聊天记录上报时，未找到mac地址为 {} 的设备", macAddress);
}
return Boolean.TRUE;
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentMcpAccessPointServiceImpl.java
+180
View File
@@ -0,0 +1,180 @@
package xiaozhi.modules.agent.service.impl;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.utils.AESUtils;
import xiaozhi.common.utils.HashEncryptionUtil;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.dto.McpJsonRpcRequest;
import xiaozhi.modules.agent.dto.McpJsonRpcResponse;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.sys.service.SysParamsService;
import xiaozhi.modules.sys.utils.WebSocketClientManager;
@AllArgsConstructor
@Service
@Slf4j
public class AgentMcpAccessPointServiceImpl implements AgentMcpAccessPointService {
private SysParamsService sysParamsService;
@Override
public String getAgentMcpAccessAddress(String id) {
// 获取到mcp的地址
String url = sysParamsService.getValue(Constant.SERVER_MCP_ENDPOINT, true);
if (StringUtils.isBlank(url) || "null".equals(url)) {
return null;
}
URI uri = getURI(url);
// 获取智能体mcp的url前缀
String agentMcpUrl = getAgentMcpUrl(uri);
// 获取密钥
String key = getSecretKey(uri);
// 获取加密的token
String encryptToken = encryptToken(id, key);
// 对token进行URL编码
String encodedToken = URLEncoder.encode(encryptToken, StandardCharsets.UTF_8);
// 返回智能体Mcp路径的格式
agentMcpUrl = "%s/mcp/?token=%s".formatted(agentMcpUrl, encodedToken);
return agentMcpUrl;
}
@Override
public List<String> getAgentMcpToolsList(String id) {
String wsUrl = getAgentMcpAccessAddress(id);
if (StringUtils.isBlank(wsUrl)) {
return List.of();
}
// 将 /mcp 替换为 /call
wsUrl = wsUrl.replace("/mcp/", "/call/");
try {
// 创建 WebSocket 连接
try (WebSocketClientManager client = WebSocketClientManager.build(
new WebSocketClientManager.Builder()
.uri(wsUrl)
.connectTimeout(5, TimeUnit.SECONDS)
.maxSessionDuration(20, TimeUnit.SECONDS))) {
// 发送初始化通知
McpJsonRpcRequest initRequest = new McpJsonRpcRequest("notifications/initialized");
client.sendJson(initRequest);
// 等待 0.2 秒
Thread.sleep(200);
// 发送工具列表请求
McpJsonRpcRequest toolsRequest = new McpJsonRpcRequest("tools/list", null, 1);
client.sendJson(toolsRequest);
// 监听响应，直到收到包含 id=1 的响应
List<String> responses = client.listener(response -> {
try {
McpJsonRpcResponse jsonResponse = JsonUtils.parseObject(response, McpJsonRpcResponse.class);
return jsonResponse != null && Integer.valueOf(1).equals(jsonResponse.getId());
} catch (Exception e) {
log.warn("解析响应失败: {}", response, e);
return false;
}
});
// 处理响应
for (String response : responses) {
try {
McpJsonRpcResponse jsonResponse = JsonUtils.parseObject(response, McpJsonRpcResponse.class);
if (jsonResponse != null && Integer.valueOf(1).equals(jsonResponse.getId())
&& jsonResponse.getResult() != null && jsonResponse.getResult().getTools() != null) {
// 提取工具名称列表
return java.util.Arrays.stream(jsonResponse.getResult().getTools())
.map(McpJsonRpcResponse.McpTool::getName)
.collect(Collectors.toList());
}
} catch (Exception e) {
log.warn("处理工具列表响应失败: {}", response, e);
}
}
log.warn("未找到有效的工具列表响应");
return List.of();
}
} catch (Exception e) {
log.error("获取智能体 MCP 工具列表失败，智能体ID: {}", id, e);
return List.of();
}
}
/**
* 获取URI对象
*
* @param url 路径
* @return URI对象
*/
private static URI getURI(String url) {
try {
return new URI(url);
} catch (URISyntaxException e) {
log.error("路径格式不正确路径：{}，\n错误信息:{}", url, e.getMessage());
throw new RuntimeException("mcp的地址存在错误，请进入参数管理修改mcp接入点地址");
}
}
/**
* 获取密钥
*
* @param uri mcp地址
* @return 密钥
*/
private static String getSecretKey(URI uri) {
// 获取参数
String query = uri.getQuery();
// 获取aes加密密钥
String str = "key=";
return query.substring(query.indexOf(str) + str.length());
}
/**
* 获取智能体mcp接入点url
*
* @param uri mcp地址
* @return 智能体mcp接入点url
*/
private String getAgentMcpUrl(URI uri) {
// 获取协议
String wsScheme = (uri.getScheme().equals("https")) ? "wss" : "ws";
// 获取主机，端口，路径
String path = uri.getSchemeSpecificPart();
// 获取到最后一个/前的path
path = path.substring(0, path.lastIndexOf("/"));
return wsScheme + ":" + path;
}
/**
* 获取对智能体id加密的token
*
* @param agentId 智能体id
* @param key 加密密钥
* @return 加密后token
*/
private static String encryptToken(String agentId, String key) {
// 使用md5对智能体id进行加密
String md5 = HashEncryptionUtil.Md5hexDigest(agentId);
// aes需要加密文本
String json = "{\"agentId\": \"%s\"}".formatted(md5);
// 加密后成token值
return AESUtils.encrypt(key, json);
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentPluginMappingServiceImpl.java
+37
View File
@@ -0,0 +1,37 @@
package xiaozhi.modules.agent.service.impl;
import java.util.List;
import org.springframework.stereotype.Service;
import com.baomidou.mybatisplus.core.conditions.update.UpdateWrapper;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import lombok.RequiredArgsConstructor;
import xiaozhi.modules.agent.dao.AgentPluginMappingMapper;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
/**
* @description 针对表【ai_agent_plugin_mapping(Agent与插件的唯一映射表)】的数据库操作Service实现
* @createDate 2025-05-25 22:33:17
*/
@Service
@RequiredArgsConstructor
public class AgentPluginMappingServiceImpl extends ServiceImpl<AgentPluginMappingMapper, AgentPluginMapping>
implements AgentPluginMappingService {
private final AgentPluginMappingMapper agentPluginMappingMapper;
@Override
public List<AgentPluginMapping> agentPluginParamsByAgentId(String agentId) {
return agentPluginMappingMapper.selectPluginsByAgentId(agentId);
}
@Override
public void deleteByAgentId(String agentId) {
UpdateWrapper<AgentPluginMapping> updateWrapper = new UpdateWrapper<>();
updateWrapper.eq("agent_id", agentId);
agentPluginMappingMapper.delete(updateWrapper);
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentServiceImpl.java
+230 -8
View File
@@ -1,12 +1,17 @@
package xiaozhi.modules.agent.service.impl;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.function.Function;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.conditions.update.UpdateWrapper;
@@ -14,16 +19,30 @@ import com.baomidou.mybatisplus.core.metadata.IPage;
import lombok.AllArgsConstructor;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.exception.RenException;
import xiaozhi.common.page.PageData;
import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.service.impl.BaseServiceImpl;
import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.ConvertUtils;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.dao.AgentDao;
import xiaozhi.modules.agent.dto.AgentCreateDTO;
import xiaozhi.modules.agent.dto.AgentDTO;
import xiaozhi.modules.agent.dto.AgentUpdateDTO;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.agent.vo.AgentInfoVO;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.model.dto.ModelProviderDTO;
import xiaozhi.modules.model.service.ModelConfigService;
import xiaozhi.modules.model.service.ModelProviderService;
import xiaozhi.modules.security.user.SecurityUser;
import xiaozhi.modules.sys.enums.SuperAdminEnum;
import xiaozhi.modules.timbre.service.TimbreService;
@@ -36,6 +55,10 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
private final ModelConfigService modelConfigService;
private final RedisUtils redisUtils;
private final DeviceService deviceService;
private final AgentPluginMappingService agentPluginMappingService;
private final AgentChatHistoryService agentChatHistoryService;
private final AgentTemplateService agentTemplateService;
private final ModelProviderService modelProviderService;
@Override
public PageData<AgentEntity> adminAgentList(Map<String, Object> params) {
@@ -46,15 +69,20 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
}
@Override
public AgentEntity getAgentById(String id) {
AgentEntity agent = agentDao.selectById(id);
if (agent != null && agent.getMemModelId() != null && agent.getMemModelId().equals(Constant.MEMORY_NO_MEM)) {
agent.setChatHistoryConf(Constant.ChatHistoryConfEnum.IGNORE.getCode());
} else if (agent != null && agent.getMemModelId() != null
&& !agent.getMemModelId().equals(Constant.MEMORY_NO_MEM)
&& agent.getChatHistoryConf() == null) {
agent.setChatHistoryConf(Constant.ChatHistoryConfEnum.RECORD_TEXT_AUDIO.getCode());
public AgentInfoVO getAgentById(String id) {
AgentInfoVO agent = agentDao.selectAgentInfoById(id);
if (agent == null) {
throw new RenException("智能体不存在");
}
if (agent.getMemModelId() != null && agent.getMemModelId().equals(Constant.MEMORY_NO_MEM)) {
agent.setChatHistoryConf(Constant.ChatHistoryConfEnum.IGNORE.getCode());
if (agent.getChatHistoryConf() == null) {
agent.setChatHistoryConf(Constant.ChatHistoryConfEnum.RECORD_TEXT_AUDIO.getCode());
}
}
// 无需额外查询插件列表，已通过SQL查询出来
return agent;
}
@@ -167,4 +195,198 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
// 检查是否是智能体的所有者
return userId.equals(agent.getUserId());
}
// 根据id更新智能体信息
@Override
@Transactional(rollbackFor = Exception.class)
public void updateAgentById(String agentId, AgentUpdateDTO dto) {
// 先查询现有实体
AgentEntity existingEntity = this.getAgentById(agentId);
if (existingEntity == null) {
throw new RuntimeException("智能体不存在");
}
// 只更新提供的非空字段
if (dto.getAgentName() != null) {
existingEntity.setAgentName(dto.getAgentName());
}
if (dto.getAgentCode() != null) {
existingEntity.setAgentCode(dto.getAgentCode());
}
if (dto.getAsrModelId() != null) {
existingEntity.setAsrModelId(dto.getAsrModelId());
}
if (dto.getVadModelId() != null) {
existingEntity.setVadModelId(dto.getVadModelId());
}
if (dto.getLlmModelId() != null) {
existingEntity.setLlmModelId(dto.getLlmModelId());
}
if (dto.getVllmModelId() != null) {
existingEntity.setVllmModelId(dto.getVllmModelId());
}
if (dto.getTtsModelId() != null) {
existingEntity.setTtsModelId(dto.getTtsModelId());
}
if (dto.getTtsVoiceId() != null) {
existingEntity.setTtsVoiceId(dto.getTtsVoiceId());
}
if (dto.getMemModelId() != null) {
existingEntity.setMemModelId(dto.getMemModelId());
}
if (dto.getIntentModelId() != null) {
existingEntity.setIntentModelId(dto.getIntentModelId());
}
if (dto.getSystemPrompt() != null) {
existingEntity.setSystemPrompt(dto.getSystemPrompt());
}
if (dto.getSummaryMemory() != null) {
existingEntity.setSummaryMemory(dto.getSummaryMemory());
}
if (dto.getChatHistoryConf() != null) {
existingEntity.setChatHistoryConf(dto.getChatHistoryConf());
}
if (dto.getLangCode() != null) {
existingEntity.setLangCode(dto.getLangCode());
}
if (dto.getLanguage() != null) {
existingEntity.setLanguage(dto.getLanguage());
}
if (dto.getSort() != null) {
existingEntity.setSort(dto.getSort());
}
// 更新函数插件信息
List<AgentUpdateDTO.FunctionInfo> functions = dto.getFunctions();
if (functions != null) {
// 1. 收集本次提交的 pluginId
List<String> newPluginIds = functions.stream()
.map(AgentUpdateDTO.FunctionInfo::getPluginId)
.toList();
// 2. 查询当前agent现有的所有映射
List<AgentPluginMapping> existing = agentPluginMappingService.list(
new QueryWrapper<AgentPluginMapping>()
.eq("agent_id", agentId));
Map<String, AgentPluginMapping> existMap = existing.stream()
.collect(Collectors.toMap(AgentPluginMapping::getPluginId, Function.identity()));
// 3. 构造所有要 保存或更新 的实体
List<AgentPluginMapping> allToPersist = functions.stream().map(info -> {
AgentPluginMapping m = new AgentPluginMapping();
m.setAgentId(agentId);
m.setPluginId(info.getPluginId());
m.setParamInfo(JsonUtils.toJsonString(info.getParamInfo()));
AgentPluginMapping old = existMap.get(info.getPluginId());
if (old != null) {
// 已存在，设置id表示更新
m.setId(old.getId());
}
return m;
}).toList();
// 4. 拆分：已有ID的走更新，无ID的走插入
List<AgentPluginMapping> toUpdate = allToPersist.stream()
.filter(m -> m.getId() != null)
.toList();
List<AgentPluginMapping> toInsert = allToPersist.stream()
.filter(m -> m.getId() == null)
.toList();
if (!toUpdate.isEmpty()) {
agentPluginMappingService.updateBatchById(toUpdate);
}
if (!toInsert.isEmpty()) {
agentPluginMappingService.saveBatch(toInsert);
}
// 5. 删除本次不在提交列表里的插件映射
List<Long> toDelete = existing.stream()
.filter(old -> !newPluginIds.contains(old.getPluginId()))
.map(AgentPluginMapping::getId)
.toList();
if (!toDelete.isEmpty()) {
agentPluginMappingService.removeBatchByIds(toDelete);
}
}
// 设置更新者信息
UserDetail user = SecurityUser.getUser();
existingEntity.setUpdater(user.getId());
existingEntity.setUpdatedAt(new Date());
// 更新记忆策略
if (existingEntity.getMemModelId() == null || existingEntity.getMemModelId().equals(Constant.MEMORY_NO_MEM)) {
// 删除所有记录
agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, true);
existingEntity.setSummaryMemory("");
} else if (existingEntity.getChatHistoryConf() != null && existingEntity.getChatHistoryConf() == 1) {
// 删除音频数据
agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, false);
}
this.updateById(existingEntity);
}
@Override
@Transactional(rollbackFor = Exception.class)
public String createAgent(AgentCreateDTO dto) {
// 转换为实体
AgentEntity entity = ConvertUtils.sourceToTarget(dto, AgentEntity.class);
// 获取默认模板
AgentTemplateEntity template = agentTemplateService.getDefaultTemplate();
if (template != null) {
// 设置模板中的默认值
entity.setAsrModelId(template.getAsrModelId());
entity.setVadModelId(template.getVadModelId());
entity.setLlmModelId(template.getLlmModelId());
entity.setVllmModelId(template.getVllmModelId());
entity.setTtsModelId(template.getTtsModelId());
entity.setTtsVoiceId(template.getTtsVoiceId());
entity.setMemModelId(template.getMemModelId());
entity.setIntentModelId(template.getIntentModelId());
entity.setSystemPrompt(template.getSystemPrompt());
entity.setSummaryMemory(template.getSummaryMemory());
entity.setChatHistoryConf(template.getChatHistoryConf());
entity.setLangCode(template.getLangCode());
entity.setLanguage(template.getLanguage());
}
// 设置用户ID和创建者信息
UserDetail user = SecurityUser.getUser();
entity.setUserId(user.getId());
entity.setCreator(user.getId());
entity.setCreatedAt(new Date());
// 保存智能体
insert(entity);
// 设置默认插件
List<AgentPluginMapping> toInsert = new ArrayList<>();
// 播放音乐、查天气、查新闻
String[] pluginIds = new String[] { "SYSTEM_PLUGIN_MUSIC", "SYSTEM_PLUGIN_WEATHER",
"SYSTEM_PLUGIN_NEWS_NEWSNOW" };
for (String pluginId : pluginIds) {
ModelProviderDTO provider = modelProviderService.getById(pluginId);
if (provider == null) {
continue;
}
AgentPluginMapping mapping = new AgentPluginMapping();
mapping.setPluginId(pluginId);
Map<String, Object> paramInfo = new HashMap<>();
List<Map<String, Object>> fields = JsonUtils.parseObject(provider.getFields(), List.class);
if (fields != null) {
for (Map<String, Object> field : fields) {
paramInfo.put((String) field.get("key"), field.get("default"));
}
}
mapping.setParamInfo(JsonUtils.toJsonString(paramInfo));
mapping.setAgentId(entity.getId());
toInsert.add(mapping);
}
// 保存默认插件
agentPluginMappingService.saveBatch(toInsert);
return entity.getId();
}
}
main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentTemplateServiceImpl.java
+3
View File
@@ -56,6 +56,9 @@ public class AgentTemplateServiceImpl extends ServiceImpl<AgentTemplateDao, Agen
wrapper.set("tts_model_id", modelId);
wrapper.set("tts_voice_id", null);
break;
case "VLLM":
wrapper.set("vllm_model_id", modelId);
break;
case "MEMORY":
wrapper.set("mem_model_id", modelId);
break;
main/manager-api/src/main/java/xiaozhi/modules/agent/vo/AgentInfoVO.java
+24
View File
@@ -0,0 +1,24 @@
package xiaozhi.modules.agent.vo;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
import lombok.EqualsAndHashCode;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import java.util.List;
/**
* Agent信息返回体VO
* 这里直接extend了Agent实体类AgentEntity，后续需要规范返回字段可以copy字段出来
*/
@EqualsAndHashCode(callSuper = true)
@Data
public class AgentInfoVO extends AgentEntity
{
@Schema(description = "插件列表Id")
@TableField(typeHandler = JacksonTypeHandler.class)
private List<AgentPluginMapping> functions;
}
main/manager-api/src/main/java/xiaozhi/modules/config/controller/ConfigController.java
+1 -1
View File
@@ -27,7 +27,7 @@ public class ConfigController {
private final ConfigService configService;
@PostMapping("server-base")
@Operation(summary = "获取配置")
@Operation(summary = "服务端获取配置接口")
public Result<Object> getConfig() {
Object config = configService.getConfig(true);
return new Result<Object>().ok(config);
main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java
+26
View File
@@ -4,6 +4,7 @@ import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
@@ -16,7 +17,10 @@ import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.utils.JsonUtils;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentPluginMapping;
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
import xiaozhi.modules.agent.service.AgentPluginMappingService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.config.service.ConfigService;
@@ -39,6 +43,8 @@ public class ConfigServiceImpl implements ConfigService {
private final AgentTemplateService agentTemplateService;
private final RedisUtils redisUtils;
private final TimbreService timbreService;
private final AgentPluginMappingService agentPluginMappingService;
private final AgentMcpAccessPointService agentMcpAccessPointService;
@Override
public Object getConfig(Boolean isCache) {
@@ -138,6 +144,25 @@ public class ConfigServiceImpl implements ConfigService {
agent.setAsrModelId(null);
}
// 添加函数调用参数信息
if (!Objects.equals(agent.getIntentModelId(), "Intent_nointent")) {
String agentId = agent.getId();
List<AgentPluginMapping> pluginMappings = agentPluginMappingService.agentPluginParamsByAgentId(agentId);
if (pluginMappings != null && !pluginMappings.isEmpty()) {
Map<String, Object> pluginParams = new HashMap<>();
for (AgentPluginMapping pluginMapping : pluginMappings) {
pluginParams.put(pluginMapping.getProviderCode(), pluginMapping.getParamInfo());
}
result.put("plugins", pluginParams);
}
}
// 获取mcp接入点地址
String mcpEndpoint = agentMcpAccessPointService.getAgentMcpAccessAddress(agent.getId());
if (StringUtils.isNotBlank(mcpEndpoint) && mcpEndpoint.startsWith("ws")) {
mcpEndpoint = mcpEndpoint.replace("/mcp/", "/call/");
result.put("mcp_endpoint", mcpEndpoint);
}
// 构建模块配置
buildModuleConfig(
agent.getAgentName(),
@@ -298,6 +323,7 @@ public class ConfigServiceImpl implements ConfigService {
map.put("functions", functions);
}
}
System.out.println("map: " + map);
}
if ("Memory".equals(modelTypes[i])) {
Map<String, Object> map = (Map<String, Object>) model.getConfigJson();
main/manager-api/src/main/java/xiaozhi/modules/device/controller/DeviceController.java
+21 -4
View File
@@ -4,6 +4,7 @@ import java.util.List;
import org.apache.commons.lang3.StringUtils;
import org.apache.shiro.authz.annotation.RequiresPermissions;
import org.springframework.beans.BeanUtils;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
@@ -14,6 +15,7 @@ import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import lombok.AllArgsConstructor;
import xiaozhi.common.exception.ErrorCode;
import xiaozhi.common.redis.RedisKeys;
@@ -22,6 +24,8 @@ import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.device.dto.DeviceRegisterDTO;
import xiaozhi.modules.device.dto.DeviceUnBindDTO;
import xiaozhi.modules.device.dto.DeviceUpdateDTO;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.security.user.SecurityUser;
@@ -80,16 +84,29 @@ public class DeviceController {
return new Result<Void>();
}
@PutMapping("/enableOta/{id}/{status}")
@Operation(summary = "启用/关闭OTA自动升级")
@PutMapping("/update/{id}")
@Operation(summary = "更新设备信息")
@RequiresPermissions("sys:role:normal")
public Result<Void> enableOtaUpgrade(@PathVariable String id, @PathVariable Integer status) {
public Result<Void> updateDeviceInfo(@PathVariable String id, @Valid @RequestBody DeviceUpdateDTO deviceUpdateDTO) {
DeviceEntity entity = deviceService.selectById(id);
if (entity == null) {
return new Result<Void>().error("设备不存在");
}
entity.setAutoUpdate(status);
UserDetail user = SecurityUser.getUser();
if (!entity.getUserId().equals(user.getId())) {
return new Result<Void>().error("设备不存在");
}
BeanUtils.copyProperties(deviceUpdateDTO, entity);
deviceService.updateById(entity);
return new Result<Void>();
}
@PostMapping("/manual-add")
@Operation(summary = "手动添加设备")
@RequiresPermissions("sys:role:normal")
public Result<Void> manualAddDevice(@RequestBody @Valid DeviceManualAddDTO dto) {
UserDetail user = SecurityUser.getUser();
deviceService.manualAddDevice(user.getId(), dto);
return new Result<>();
}
}
main/manager-api/src/main/java/xiaozhi/modules/device/dto/DeviceManualAddDTO.java
+11
View File
@@ -0,0 +1,11 @@
package xiaozhi.modules.device.dto;
import lombok.Data;
@Data
public class DeviceManualAddDTO {
private String agentId;
private String board; // 设备型号
private String appVersion; // 固件版本
private String macAddress; // Mac地址
}
main/manager-api/src/main/java/xiaozhi/modules/device/dto/DeviceUpdateDTO.java
+29
View File
@@ -0,0 +1,29 @@
package xiaozhi.modules.device.dto;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Size;
import lombok.Data;
import java.io.Serializable;
/**
* 设备更新DTO
*/
@Data
public class DeviceUpdateDTO implements Serializable {
/**
* 自动更新状态
*/
@Max(1)
@Min(0)
private Integer autoUpdate;
/**
* 设备别名
*/
@Size(max = 64)
private String alias;
private static final long serialVersionUID = 1L;
}
main/manager-api/src/main/java/xiaozhi/modules/device/service/DeviceService.java
+10
View File
@@ -8,6 +8,7 @@ import xiaozhi.common.service.BaseService;
import xiaozhi.modules.device.dto.DevicePageUserDTO;
import xiaozhi.modules.device.dto.DeviceReportReqDTO;
import xiaozhi.modules.device.dto.DeviceReportRespDTO;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.vo.UserShowDeviceListVO;
@@ -87,5 +88,14 @@ public interface DeviceService extends BaseService<DeviceEntity> {
*/
Date getLatestLastConnectionTime(String agentId);
/**
* 手动添加设备
*/
void manualAddDevice(Long userId, DeviceManualAddDTO dto);
/**
* 更新设备连接信息
*/
void updateDeviceConnectionInfo(String agentId, String deviceId, String appVersion);
}
main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java
+27
View File
@@ -44,6 +44,7 @@ import xiaozhi.modules.device.vo.UserShowDeviceListVO;
import xiaozhi.modules.security.user.SecurityUser;
import xiaozhi.modules.sys.service.SysParamsService;
import xiaozhi.modules.sys.service.SysUserUtilService;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
@Slf4j
@Service
@@ -410,4 +411,30 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
}
return 0;
}
@Override
public void manualAddDevice(Long userId, DeviceManualAddDTO dto) {
// 检查mac是否已存在
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", dto.getMacAddress());
DeviceEntity exist = baseDao.selectOne(wrapper);
if (exist != null) {
throw new RenException("该Mac地址已存在");
}
Date now = new Date();
DeviceEntity entity = new DeviceEntity();
entity.setId(dto.getMacAddress());
entity.setUserId(userId);
entity.setAgentId(dto.getAgentId());
entity.setBoard(dto.getBoard());
entity.setAppVersion(dto.getAppVersion());
entity.setMacAddress(dto.getMacAddress());
entity.setCreateDate(now);
entity.setUpdateDate(now);
entity.setLastConnectedAt(now);
entity.setCreator(userId);
entity.setUpdater(userId);
entity.setAutoUpdate(1);
baseDao.insert(entity);
}
}
main/manager-api/src/main/java/xiaozhi/modules/model/controller/ModelProviderController.java
+7
View File
@@ -18,6 +18,7 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.AllArgsConstructor;
import xiaozhi.common.page.PageData;
import xiaozhi.common.utils.Result;
import xiaozhi.common.utils.ResultUtils;
import xiaozhi.common.validator.group.UpdateGroup;
import xiaozhi.modules.model.dto.ModelProviderDTO;
import xiaozhi.modules.model.service.ModelProviderService;
@@ -65,4 +66,10 @@ public class ModelProviderController {
return new Result<>();
}
@GetMapping("/plugin/names")
@Tag(name = "获取插件名称列表")
public Result<List<ModelProviderDTO>> getPluginNameList() {
return ResultUtils.success(modelProviderService.getPluginList());
}
}
main/manager-api/src/main/java/xiaozhi/modules/model/service/ModelProviderService.java
+6 -1
View File
@@ -1,5 +1,6 @@
package xiaozhi.modules.model.service;
import java.util.Collection;
import java.util.List;
import xiaozhi.common.page.PageData;
@@ -7,7 +8,11 @@ import xiaozhi.modules.model.dto.ModelProviderDTO;
public interface ModelProviderService {
// List<String> getModelNames(String modelType, String modelName);
List<ModelProviderDTO> getPluginList();
ModelProviderDTO getById(String id);
List<ModelProviderDTO> getPluginListByIds(Collection<String> ids);
List<ModelProviderDTO> getListByModelType(String modelType);
main/manager-api/src/main/java/xiaozhi/modules/model/service/impl/ModelProviderServiceImpl.java
+25
View File
@@ -1,5 +1,6 @@
package xiaozhi.modules.model.service.impl;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
@@ -8,6 +9,7 @@ import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
@@ -32,6 +34,29 @@ public class ModelProviderServiceImpl extends BaseServiceImpl<ModelProviderDao,
private final ModelProviderDao modelProviderDao;
@Override
public List<ModelProviderDTO> getPluginList() {
LambdaQueryWrapper<ModelProviderEntity> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(ModelProviderEntity::getModelType, "Plugin");
List<ModelProviderEntity> providerEntities = modelProviderDao.selectList(queryWrapper);
return ConvertUtils.sourceToTarget(providerEntities, ModelProviderDTO.class);
}
@Override
public ModelProviderDTO getById(String id) {
ModelProviderEntity entity = modelProviderDao.selectById(id);
return ConvertUtils.sourceToTarget(entity, ModelProviderDTO.class);
}
@Override
public List<ModelProviderDTO> getPluginListByIds(Collection<String> ids) {
LambdaQueryWrapper<ModelProviderEntity> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.in(ModelProviderEntity::getId, ids);
queryWrapper.eq(ModelProviderEntity::getModelType, "Plugin");
List<ModelProviderEntity> providerEntities = modelProviderDao.selectList(queryWrapper);
return ConvertUtils.sourceToTarget(providerEntities, ModelProviderDTO.class);
}
@Override
public List<ModelProviderDTO> getListByModelType(String modelType) {
main/manager-api/src/main/java/xiaozhi/modules/sys/controller/SysParamsController.java
+32
View File
@@ -104,6 +104,9 @@ public class SysParamsController {
// 验证OTA地址
validateOtaUrl(dto.getParamCode(), dto.getParamValue());
// 验证MCP地址
validateMcpUrl(dto.getParamCode(), dto.getParamValue());
sysParamsService.update(dto);
configService.getConfig(false);
return new Result<Void>();
@@ -195,4 +198,33 @@ public class SysParamsController {
throw new RenException("OTA接口验证失败：" + e.getMessage());
}
}
private void validateMcpUrl(String paramCode, String url) {
if (!paramCode.equals(Constant.SERVER_MCP_ENDPOINT)) {
return;
}
if (StringUtils.isBlank(url) || url.equals("null")) {
throw new RenException("MCP地址不能为空");
}
if (url.contains("localhost") || url.contains("127.0.0.1")) {
throw new RenException("MCP地址不能使用localhost或127.0.0.1");
}
if (!url.toLowerCase().contains("key")) {
throw new RenException("不是正确的MCP地址");
}
try {
// 发送GET请求
ResponseEntity<String> response = restTemplate.getForEntity(url, String.class);
if (response.getStatusCode() != HttpStatus.OK) {
throw new RenException("MCP接口访问失败，状态码：" + response.getStatusCode());
}
// 检查响应内容是否包含OTA相关信息
String body = response.getBody();
if (body == null || !body.contains("success")) {
throw new RenException("MCP接口返回内容格式不正确，可能不是一个真实的MCP接口");
}
} catch (Exception e) {
throw new RenException("MCP接口验证失败：" + e.getMessage());
}
}
}
main/manager-api/src/main/java/xiaozhi/modules/sys/enums/ServerActionEnum.java
+6 -13
View File
@@ -2,36 +2,29 @@ package xiaozhi.modules.sys.enums;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonValue;
import xiaozhi.common.exception.RenException;
/**
* 服务端动作枚举
*/
public enum ServerActionEnum
{
public enum ServerActionEnum {
RESTART("restart"),
UPDATE_CONFIG("update_config");
private final String value;
ServerActionEnum(String value)
{
ServerActionEnum(String value) {
this.value = value;
}
@JsonValue
public String getValue()
{
public String getValue() {
return value;
}
@JsonCreator
public static ServerActionEnum fromValue(String value)
{
for (ServerActionEnum action : ServerActionEnum.values())
{
if (action.value.equalsIgnoreCase(value))
{
public static ServerActionEnum fromValue(String value) {
for (ServerActionEnum action : ServerActionEnum.values()) {
if (action.value.equalsIgnoreCase(value)) {
return action;
}
}
main/manager-api/src/main/java/xiaozhi/modules/sys/enums/ServerActionResponseEnum.java
+5 -6
View File
@@ -1,16 +1,16 @@
package xiaozhi.modules.sys.enums;
import org.apache.commons.lang3.StringUtils;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonValue;
import lombok.Getter;
import org.apache.commons.lang3.StringUtils;
/**
* 服务端调用响应枚举
*/
public enum ServerActionResponseEnum
{
public enum ServerActionResponseEnum {
SUCCESS("success"), FAIL("fail");
private final String value;
ServerActionResponseEnum(String value) {
@@ -18,8 +18,7 @@ public enum ServerActionResponseEnum
}
@JsonValue
public String getValue()
{
public String getValue() {
return value;
}
main/manager-api/src/main/java/xiaozhi/modules/sys/utils/WebSocketClientManager.java
+55 -50
View File
@@ -1,13 +1,5 @@
package xiaozhi.modules.sys.utils;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.springframework.util.StopWatch;
import org.springframework.web.socket.*;
import org.springframework.web.socket.client.standard.StandardWebSocketClient;
import org.springframework.web.socket.handler.AbstractWebSocketHandler;
import xiaozhi.common.utils.DateUtils;
import java.io.Closeable;
import java.io.IOException;
import java.net.URI;
@@ -15,30 +7,51 @@ import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.*;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Consumer;
import java.util.function.Predicate;
import org.springframework.util.StopWatch;
import org.springframework.web.socket.BinaryMessage;
import org.springframework.web.socket.CloseStatus;
import org.springframework.web.socket.TextMessage;
import org.springframework.web.socket.WebSocketHttpHeaders;
import org.springframework.web.socket.WebSocketSession;
import org.springframework.web.socket.client.standard.StandardWebSocketClient;
import org.springframework.web.socket.handler.AbstractWebSocketHandler;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import xiaozhi.common.utils.DateUtils;
/**
* WebSocketClientResource：支持 try-with-resources 模式
*/
@Slf4j
public class WebSocketClientManager implements Closeable
{
public class WebSocketClientManager implements Closeable {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
// 全局回调线程池
private static final ExecutorService CALLBACK_EXECUTOR = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
private final AtomicInteger cnt = new AtomicInteger();
private static final ExecutorService CALLBACK_EXECUTOR = Executors
.newFixedThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
private final AtomicInteger cnt = new AtomicInteger();
public Thread newThread(Runnable r)
{
Thread t = new Thread(r, "ws-callback-" + cnt.getAndIncrement());
t.setDaemon(true);
return t;
}
});
public Thread newThread(Runnable r) {
Thread t = new Thread(r, "ws-callback-" + cnt.getAndIncrement());
t.setDaemon(true);
return t;
}
});
private volatile WebSocketSession session;
private final BlockingQueue<String> textMessageQueue;
@@ -51,18 +64,10 @@ public class WebSocketClientManager implements Closeable
private volatile Consumer<byte[]> onBinary;
private volatile Consumer<Throwable> onError;
private final String uri;
private final WebSocketHttpHeaders headers;
private final long connectTimeout;
private final TimeUnit connectUnit;
private final int queueCapacity;
// 私有构造，仅由 Builder 调用
private WebSocketClientManager(Builder b) {
this.uri = b.uri;
this.headers = b.headers != null ? b.headers : new WebSocketHttpHeaders();
this.connectTimeout = b.connectTimeout;
this.connectUnit = b.connectUnit;
this.maxSessionDuration = b.maxSessionDuration;
this.maxSessionDurationUnit = b.maxSessionDurationUnit;
this.queueCapacity = b.queueCapacity;
@@ -71,13 +76,14 @@ public class WebSocketClientManager implements Closeable
this.errorFuture = new CompletableFuture<>();
}
public static WebSocketClientManager build(Builder b) throws InterruptedException, ExecutionException, TimeoutException, IOException {
public static WebSocketClientManager build(Builder b)
throws InterruptedException, ExecutionException, TimeoutException, IOException {
WebSocketClientManager ws = new WebSocketClientManager(b);
StandardWebSocketClient client = new StandardWebSocketClient();
CompletableFuture<WebSocketSession> future = client.execute(ws.new InternalHandler(b.uri), b.headers, URI.create(b.uri));
CompletableFuture<WebSocketSession> future = client.execute(ws.new InternalHandler(b.uri), b.headers,
URI.create(b.uri));
WebSocketSession sess = future.get(b.connectTimeout, b.connectUnit);
if (sess == null || !sess.isOpen())
{
if (sess == null || !sess.isOpen()) {
throw new IOException("握手失败或会话未打开");
}
ws.session = sess;
@@ -100,13 +106,10 @@ public class WebSocketClientManager implements Closeable
session.sendMessage(new TextMessage(json));
}
private <T> List<T> listenerCustom(
BlockingQueue<T> queue,
Predicate<T> predicate)
throws InterruptedException, TimeoutException, ExecutionException
{
throws InterruptedException, TimeoutException, ExecutionException {
List<T> collected = new ArrayList<>();
long deadline = System.currentTimeMillis() + maxSessionDurationUnit.toMillis(maxSessionDuration);
@@ -136,17 +139,16 @@ public class WebSocketClientManager implements Closeable
/**
* 同步接收多条消息，直到 predicate 为 true 或超时抛异常；
*
* @return 返回监听期间的所有消息列表
*/
public List<String> listener(Predicate<String> predicate)
throws InterruptedException, TimeoutException, ExecutionException
{
throws InterruptedException, TimeoutException, ExecutionException {
return listenerCustom(textMessageQueue, predicate);
}
public List<byte[]> listenerBinary(Predicate<byte[]> predicate)
throws InterruptedException, TimeoutException, ExecutionException
{
throws InterruptedException, TimeoutException, ExecutionException {
return listenerCustom(binaryMessageQueue, predicate);
}
@@ -183,8 +185,8 @@ public class WebSocketClientManager implements Closeable
if (session != null && session.isOpen()) {
session.close(CloseStatus.NORMAL);
}
} catch (IOException ignored) {
}
catch (IOException ignored) {}
textMessageQueue.clear();
binaryMessageQueue.clear();
errorFuture.completeExceptionally(new IOException("WebSocket 已关闭"));
@@ -207,7 +209,8 @@ public class WebSocketClientManager implements Closeable
// 保存会话
WebSocketClientManager.this.session = session;
this.stopWatch.start();
log.info("ws连接成功, 目标URI: {}, 连接时间: {}", targetUri, DateUtils.getDateTimeNow(DateUtils.DATE_TIME_MILLIS_PATTERN));
log.info("ws连接成功, 目标URI: {}, 连接时间: {}", targetUri,
DateUtils.getDateTimeNow(DateUtils.DATE_TIME_MILLIS_PATTERN));
}
/**
@@ -264,18 +267,19 @@ public class WebSocketClientManager implements Closeable
stopWatch.stop();
}
log.info("ws连接关闭, 目标URI: {}, 关闭时间: {}, 连接总时长: {}s",
targetUri, DateUtils.getDateTimeNow(DateUtils.DATE_TIME_MILLIS_PATTERN), DateUtils.millsToSecond(stopWatch.getTotalTimeMillis()));
targetUri, DateUtils.getDateTimeNow(DateUtils.DATE_TIME_MILLIS_PATTERN),
DateUtils.millsToSecond(stopWatch.getTotalTimeMillis()));
}
}
public static class Builder {
private String uri; // 目标 WS URI
private long connectTimeout = 3; // 请求连接等待时间
private TimeUnit connectUnit = TimeUnit.SECONDS; // 请求连接等待时间单位
private long maxSessionDuration = 5; // 最大连线时间，默认5秒
private String uri; // 目标 WS URI
private long connectTimeout = 3; // 请求连接等待时间
private TimeUnit connectUnit = TimeUnit.SECONDS; // 请求连接等待时间单位
private long maxSessionDuration = 5; // 最大连线时间，默认5秒
private TimeUnit maxSessionDurationUnit = TimeUnit.SECONDS; // 最大连线时间单位
private int queueCapacity = 100; // 消息队列容量
private WebSocketHttpHeaders headers; // 请求头
private int queueCapacity = 100; // 消息队列容量
private WebSocketHttpHeaders headers; // 请求头
/**
* 目标 WS URI
@@ -307,7 +311,8 @@ public class WebSocketClientManager implements Closeable
return this;
}
public WebSocketClientManager build() throws InterruptedException, ExecutionException, TimeoutException, IOException {
public WebSocketClientManager build()
throws InterruptedException, ExecutionException, TimeoutException, IOException {
return WebSocketClientManager.build(this);
}
main/manager-api/src/main/resources/db/changelog/202505091409.sql
-6
View File
@@ -1,6 +0,0 @@
-- 更新intent_llmM供应器
update `ai_model_provider` set fields = '[{"key":"llm","label":"LLM模型","type":"string"},{"key":"functions","label":"函数列表","type":"dict","dict_name":"functions"}]' where id = 'SYSTEM_Intent_intent_llm';
-- 更新ChatGLMLLM的意图识别配置
update `ai_model_config` set config_json = '{\"type\": \"intent_llm\", \"llm\": \"LLM_ChatGLMLLM\", \"functions\": \"get_weather;get_news_from_newsnow;play_music\"}' where id = 'Intent_intent_llm';
-- 更新函数调用意图识别配置
UPDATE `ai_model_config` SET config_json = REPLACE(config_json, ';get_news;', ';get_news_from_newsnow;') WHERE id = 'Intent_function_call';
main/manager-api/src/main/resources/db/changelog/202505292203.sql
+189
View File
@@ -0,0 +1,189 @@
-- ===============================
-- 一、在ai_model_provider中插入plugin 记录
-- ===============================
START TRANSACTION;
-- intent_llm和function_call不设置函数列表
update `ai_model_provider` set fields = '[{"key":"llm","label":"LLM模型","type":"string"}]' where id = 'SYSTEM_Intent_intent_llm';
update `ai_model_provider` set fields = '[]' where id = 'SYSTEM_Intent_function_call';
update `ai_model_config` set config_json = '{\"type\": \"intent_llm\", \"llm\": \"LLM_ChatGLMLLM\"}' where id = 'Intent_intent_llm';
UPDATE `ai_model_config` SET config_json = '{\"type\": \"function_call\"}' WHERE id = 'Intent_function_call';
delete from ai_model_provider where model_type = 'Plugin';
-- 1. 天气查询
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_WEATHER',
'Plugin',
'get_weather',
'天气查询',
JSON_ARRAY(
JSON_OBJECT(
'key', 'api_key',
'type', 'string',
'label', '天气插件 API 密钥',
'default', (SELECT param_value FROM sys_params WHERE param_code = 'plugins.get_weather.api_key')
),
JSON_OBJECT(
'key', 'default_location',
'type', 'string',
'label', '默认查询城市',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.get_weather.default_location')
),
JSON_OBJECT(
'key', 'api_host',
'type', 'string',
'label', '开发者 API Host',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.get_weather.api_host')
)
),
10, 0, NOW(), 0, NOW());
-- 6. 本地播放音乐
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_MUSIC',
'Plugin',
'play_music',
'服务器音乐播放',
JSON_ARRAY(),
20, 0, NOW(), 0, NOW());
-- 2. 新闻订阅
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_NEWS_CHINANEWS',
'Plugin',
'get_news_from_chinanews',
'中新网新闻',
JSON_ARRAY(
JSON_OBJECT(
'key', 'default_rss_url',
'type', 'string',
'label', '默认 RSS 源',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.get_news.default_rss_url')
),
JSON_OBJECT(
'key', 'society_rss_url',
'type', 'string',
'label', '社会新闻 RSS 地址',
'default',
'https://www.chinanews.com.cn/rss/society.xml'
),
JSON_OBJECT(
'key', 'world_rss_url',
'type', 'string',
'label', '国际新闻 RSS 地址',
'default',
'https://www.chinanews.com.cn/rss/world.xml'
),
JSON_OBJECT(
'key', 'finance_rss_url',
'type', 'string',
'label', '财经新闻 RSS 地址',
'default',
'https://www.chinanews.com.cn/rss/finance.xml'
)
),
30, 0, NOW(), 0, NOW());
-- 3. 新闻订阅
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_NEWS_NEWSNOW',
'Plugin',
'get_news_from_newsnow',
'newsnow新闻聚合',
JSON_ARRAY(
JSON_OBJECT(
'key', 'url',
'type', 'string',
'label', '接口地址',
'default',
'https://newsnow.busiyi.world/api/s?id='
)
),
40, 0, NOW(), 0, NOW());
-- 4. HomeAssistant 状态查询
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_HA_GET_STATE',
'Plugin',
'hass_get_state',
'HomeAssistant设备状态查询',
JSON_ARRAY(
JSON_OBJECT(
'key', 'base_url',
'type', 'string',
'label', 'HA 服务器地址',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.home_assistant.base_url')
),
JSON_OBJECT(
'key', 'api_key',
'type', 'string',
'label', 'HA API 访问令牌',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.home_assistant.api_key')
),
JSON_OBJECT(
'key', 'devices',
'type', 'array',
'label', '设备列表（名称,实体ID;…）',
'default',
(SELECT param_value FROM sys_params WHERE param_code = 'plugins.home_assistant.devices')
)
),
50, 0, NOW(), 0, NOW());
-- 5. HomeAssistant 状态写入
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_HA_SET_STATE',
'Plugin',
'hass_set_state',
'HomeAssistant设备状态修改',
JSON_ARRAY(),
60, 0, NOW(), 0, NOW());
-- 5. HomeAssistant 音乐播放
INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields,
sort, creator, create_date, updater, update_date)
VALUES ('SYSTEM_PLUGIN_HA_PLAY_MUSIC',
'Plugin',
'hass_play_music',
'HomeAssistant音乐播放',
JSON_ARRAY(),
70, 0, NOW(), 0, NOW());
-- ===============================
-- 二、删除sys_params中旧的plugins.*参数
-- ===============================
DELETE
FROM sys_params
WHERE param_code LIKE 'plugins.%';
-- ===============================
-- 三、添加智能体插件id字段
-- ===============================
CREATE TABLE IF NOT EXISTS ai_agent_plugin_mapping
(
id BIGINT AUTO_INCREMENT PRIMARY KEY COMMENT '主键',
agent_id VARCHAR(32) NOT NULL COMMENT '智能体ID',
plugin_id VARCHAR(32) NOT NULL COMMENT '插件ID',
param_info JSON NOT NULL COMMENT '参数信息',
UNIQUE KEY uk_agent_provider (agent_id, plugin_id)
) COMMENT 'Agent与插件的唯一映射表';
COMMIT;
main/manager-api/src/main/resources/db/changelog/202506161101.sql
+1
View File
@@ -0,0 +1 @@
ALTER TABLE ai_agent_plugin_mapping CONVERT TO CHARACTER SET utf8mb4;
main/manager-api/src/main/resources/db/changelog/202506191643.sql
+19
View File
@@ -0,0 +1,19 @@
-- LLM意图识别配置说明
UPDATE `ai_model_config` SET
`doc_link` = NULL,
`remark` = 'LLM意图识别配置说明：
1. 使用独立的LLM进行意图识别
2. 默认使用selected_module.LLM的模型
3. 可以配置使用独立的LLM（如免费的ChatGLMLLM）
4. 通用性强，但会增加处理时间
配置说明：
1. 在llm字段中指定使用的LLM模型
2. 如果不指定，则使用selected_module.LLM的模型' WHERE `id` = 'Intent_intent_llm';
-- 函数调用意图识别配置说明
UPDATE `ai_model_config` SET
`doc_link` = NULL,
`remark` = '函数调用意图识别配置说明：
1. 使用LLM的function_call功能进行意图识别
2. 需要所选择的LLM支持function_call
3. 按需调用工具，处理速度快' WHERE `id` = 'Intent_function_call';
main/manager-api/src/main/resources/db/changelog/202506251619.sql
+21
View File
@@ -0,0 +1,21 @@
-- 更新 get_news_from_newsnow 插件配置，添加新闻源配置字段
-- 执行时间：2025-06-25
-- 更新现有的 get_news_from_newsnow 插件配置
UPDATE ai_model_provider
SET fields = JSON_ARRAY(
JSON_OBJECT(
'key', 'url',
'type', 'string',
'label', '接口地址',
'default', 'https://newsnow.busiyi.world/api/s?id='
),
JSON_OBJECT(
'key', 'news_sources',
'type', 'json',
'label', '新闻源配置',
'default', '{"thepaper":"澎湃新闻","baidu":"百度热搜","cls-depth":"财联社"}'
)
)
WHERE provider_code = 'get_news_from_newsnow'
AND model_type = 'Plugin';
main/manager-api/src/main/resources/db/changelog/202506261637.sql
+2
View File
@@ -0,0 +1,2 @@
delete from `sys_params` where id = 113;
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (113, 'server.mcp_endpoint', 'null', 'string', 1, 'mcp接入点地址');
main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml
+35 -7
View File
@@ -107,13 +107,6 @@ databaseChangeLog:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202505081146.sql
- changeSet:
id: 202505091409
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202505091409.sql
- changeSet:
id: 202505091555
author: whosmyqueen
@@ -170,6 +163,13 @@ databaseChangeLog:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202505271414.sql
- changeSet:
id: 202505292203
author: CAIXYPROMISE
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202505292203.sql
- changeSet:
id: 202506010920
author: hrz
@@ -212,3 +212,31 @@ databaseChangeLog:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506091720.sql
- changeSet:
id: 202506161101
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506161101.sql
- changeSet:
id: 202506191643
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506191643.sql
- changeSet:
id: 202506251619
author: Tink
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506251619.sql
- changeSet:
id: 202506261637
author: hrz
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202506261637.sql
main/manager-api/src/main/resources/lua/emptyAll.lua
+1 -1
View File
@@ -1 +1 @@
redis.call('FLUSHALL')
redis.call('FLUSHDB')
main/manager-api/src/main/resources/mapper/agent/AgentDao.xml
+68
View File
@@ -5,4 +5,72 @@
<select id="getDeviceCountByAgentId" resultType="java.lang.Integer">
SELECT COUNT(*) FROM ai_device WHERE agent_id = #{agentId}
</select>
<resultMap id="AgentInfoMap" type="xiaozhi.modules.agent.vo.AgentInfoVO">
<id column="id" property="id"/>
<result column="userId" property="userId"/>
<result column="agentCode" property="agentCode"/>
<result column="agentName" property="agentName"/>
<result column="asrModelId" property="asrModelId"/>
<result column="vadModelId" property="vadModelId"/>
<result column="llmModelId" property="llmModelId"/>
<result column="ttsModelId" property="ttsModelId"/>
<result column="ttsVoiceId" property="ttsVoiceId"/>
<result column="memModelId" property="memModelId"/>
<result column="intentModelId" property="intentModelId"/>
<result column="functions" property="functions"
typeHandler="com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler"/>
<result column="chatHistoryConf" property="chatHistoryConf"/>
<result column="systemPrompt" property="systemPrompt"/>
<result column="summaryMemory" property="summaryMemory"/>
<result column="langCode" property="langCode"/>
<result column="language" property="language"/>
<result column="sort" property="sort"/>
<result column="creator" property="creator"/>
<result column="createdAt" property="createdAt"/>
<result column="updater" property="updater"/>
<result column="updatedAt" property="updatedAt"/>
</resultMap>
<select id="selectAgentInfoById" resultMap="AgentInfoMap">
SELECT a.id,
a.user_id AS userId,
a.agent_code AS agentCode,
a.agent_name AS agentName,
a.asr_model_id AS asrModelId,
a.vad_model_id AS vadModelId,
a.llm_model_id AS llmModelId,
a.vllm_model_id AS vllmModelId,
a.tts_model_id AS ttsModelId,
a.tts_voice_id AS ttsVoiceId,
a.mem_model_id AS memModelId,
a.intent_model_id AS intentModelId,
COALESCE(
(SELECT JSON_ARRAYAGG(
JSON_OBJECT(
'id', m.id,
'agentId', m.agent_id,
'pluginId', m.plugin_id,
'paramInfo', m.param_info
)
)
FROM ai_agent_plugin_mapping m
WHERE m.agent_id = a.id),
JSON_ARRAY()
) AS functions,
a.chat_history_conf AS chatHistoryConf,
a.system_prompt AS systemPrompt,
a.summary_memory AS summaryMemory,
a.lang_code AS langCode,
a.language AS language,
a.sort,
a.creator,
a.created_at AS createdAt,
a.updater,
a.updated_at AS updatedAt
FROM ai_agent a
WHERE a.id = #{agentId}
</select>
</mapper>
main/manager-api/src/main/resources/mapper/agent/AgentPluginMappingMapper.xml
+48
View File
@@ -0,0 +1,48 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper
PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="xiaozhi.modules.agent.dao.AgentPluginMappingMapper">
<resultMap id="BaseResultMap" type="xiaozhi.modules.agent.entity.AgentPluginMapping">
<id property="id" column="id" jdbcType="BIGINT"/>
<result property="agentId" column="agent_id" jdbcType="VARCHAR"/>
<result property="pluginId" column="plugin_id" jdbcType="VARCHAR"/>
<result property="paramInfo" column="param_info" jdbcType="VARCHAR"/>
</resultMap>
<!-- 用于映射根据agentId查询完整插件信息 -->
<resultMap id="AgentPluginWithCodeMap" type="xiaozhi.modules.agent.entity.AgentPluginMapping">
<id column="id" property="id" jdbcType="BIGINT"/>
<result column="agentId" property="agentId" jdbcType="VARCHAR"/>
<result column="pluginId" property="pluginId" jdbcType="VARCHAR"/>
<result column="paramInfo" property="paramInfo" jdbcType="VARCHAR"/>
<result column="providerCode" property="providerCode" jdbcType="VARCHAR"/>
</resultMap>
<sql id="Base_Column_List">
id,agent_id,plugin_id
</sql>
<select id="selectPluginsByAgentId" resultMap="AgentPluginWithCodeMap">
SELECT m.id AS id,
m.agent_id AS agentId,
m.plugin_id AS pluginId,
m.param_info AS paramInfo,
(
SELECT
p.provider_code
FROM
ai_model_provider p
WHERE
p.id = m.plugin_id
LIMIT
1
) AS providerCode
FROM ai_agent_plugin_mapping m
WHERE m.agent_id = #{agentId}
</select>
</mapper>
main/manager-api/src/test/java/xiaozhi/common/utils/AESUtilsTest.java
+85
View File
@@ -0,0 +1,85 @@
package xiaozhi.common.utils;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
public class AESUtilsTest {
@Test
public void testEncryptAndDecrypt() {
String key = "xiaozhi1234567890";
String plainText = "Hello, 小智!";
System.out.println("原始文本: " + plainText);
System.out.println("密钥: " + key);
// 加密
String encrypted = AESUtils.encrypt(key, plainText);
System.out.println("加密结果: " + encrypted);
// 解密
String decrypted = AESUtils.decrypt(key, encrypted);
System.out.println("解密结果: " + decrypted);
// 验证
assertEquals(plainText, decrypted, "加解密结果应该一致");
System.out.println("加解密一致性: " + plainText.equals(decrypted));
}
@Test
public void testDifferentKeyLengths() {
String[] keys = {
"1234567890123456", // 16位
"123456789012345678901234", // 24位
"12345678901234567890123456789012", // 32位
"short", // 短密钥
"verylongkeythatwillbetruncatedto32bytes" // 长密钥
};
String plainText = "测试文本";
for (String key : keys) {
String encrypted = AESUtils.encrypt(key, plainText);
String decrypted = AESUtils.decrypt(key, encrypted);
assertEquals(plainText, decrypted, "密钥长度: " + key.length());
}
}
@Test
public void testSpecialCharacters() {
String key = "xiaozhi1234567890";
String[] testTexts = {
"Hello World",
"你好世界",
"Hello, 小智!",
"特殊字符: !@#$%^&*()",
"数字123和中文混合",
"Emoji: 😀🎉🚀",
"空字符串测试",
""
};
for (String text : testTexts) {
String encrypted = AESUtils.encrypt(key, text);
String decrypted = AESUtils.decrypt(key, encrypted);
assertEquals(text, decrypted, "测试文本: " + text);
}
}
@Test
public void testCrossLanguageCompatibility() {
// 这些是Python版本生成的加密结果，用于测试跨语言兼容性
String key = "xiaozhi1234567890";
String plainText = "Hello, 小智!";
// Python版本生成的加密结果（需要运行Python测试后获取）
// String pythonEncrypted = "从Python测试中获取的加密结果";
// String decrypted = AESUtils.decrypt(key, pythonEncrypted);
// assertEquals(plainText, decrypted, "Java应该能解密Python加密的结果");
// 生成Java加密结果供Python测试
String javaEncrypted = AESUtils.encrypt(key, plainText);
System.out.println("Java加密结果供Python测试: " + javaEncrypted);
}
}
main/manager-web/src/apis/api.js
+1 -1
View File
@@ -7,6 +7,7 @@ import model from './module/model.js'
import ota from './module/ota.js'
import timbre from "./module/timbre.js"
import user from './module/user.js'
/**
* 接口地址
* 开发时自动读取使用.env.development文件
@@ -22,7 +23,6 @@ export function getServiceUrl() {
return DEV_API_SERVICE
}
/** request服务封装 */
export default {
getServiceUrl,
main/manager-web/src/apis/module/agent.js
+30
View File
@@ -143,4 +143,34 @@ export default {
});
}).send();
},
// 获取智能体的MCP接入点地址
getAgentMcpAccessAddress(agentId, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/mcp/address/${agentId}`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getAgentMcpAccessAddress(agentId, callback);
});
}).send();
},
// 获取智能体的MCP工具列表
getAgentMcpToolsList(agentId, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/agent/mcp/tools/${agentId}`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail(() => {
RequestService.reAjaxFun(() => {
this.getAgentMcpToolsList(agentId, callback);
});
}).send();
},
}
main/manager-web/src/apis/module/device.js
+21 -3
View File
@@ -51,10 +51,11 @@ export default {
});
}).send();
},
enableOtaUpgrade(id, status, callback) {
updateDeviceInfo(id, payload, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/device/enableOta/${id}/${status}`)
.url(`${getServiceUrl()}/device/update/${id}`)
.method('PUT')
.data(payload)
.success((res) => {
RequestService.clearRequestTime()
callback(res)
@@ -63,8 +64,25 @@ export default {
console.error('更新OTA状态失败:', err)
this.$message.error(err.msg || '更新OTA状态失败')
RequestService.reAjaxFun(() => {
this.enableOtaUpgrade(id, status, callback)
this.updateDeviceInfo(id, payload, callback)
})
}).send()
},
// 手动添加设备
manualAddDevice(params, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/device/manual-add`)
.method('POST')
.data(params)
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail((err) => {
console.error('手动添加设备失败:', err);
RequestService.reAjaxFun(() => {
this.manualAddDevice(params, callback);
});
}).send();
},
}
main/manager-web/src/apis/module/model.js
+16
View File
@@ -305,4 +305,20 @@ export default {
})
}).send()
},
// 获取插件列表
getPluginFunctionList(params, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/models/provider/plugin/names`)
.method('GET')
.success((res) => {
RequestService.clearRequestTime()
callback(res)
})
.networkFail((err) => {
this.$message.error(err.msg || '获取插件列表失败')
RequestService.reAjaxFun(() => {
this.getPluginFunctionList(params, callback)
})
}).send()
}
}
main/manager-web/src/components/FirmwareDialog.vue
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<el-dialog :title="title" :visible.sync="dialogVisible" @close="handleClose" @open="handleOpen">
<el-dialog :title="title" :visible.sync="dialogVisible" :close-on-click-modal="false" @close="handleClose" @open="handleOpen">
<el-form ref="form" :model="form" :rules="rules" label-width="100px">
<el-form-item label="固件名称" prop="firmwareName">
<el-input v-model="form.firmwareName" placeholder="请输入固件名称(板子+版本号)"></el-input>
main/manager-web/src/components/FunctionDialog.vue
+483 -56
View File
@@ -1,5 +1,5 @@
<template>
<el-drawer :visible.sync="dialogVisible" direction="rtl" size="50%" :wrapperClosable="false" :withHeader="false">
<el-drawer :visible.sync="dialogVisible" direction="rtl" size="80%" :wrapperClosable="false" :withHeader="false">
<!-- 自定义标题区域 -->
<div class="custom-header">
<div class="header-left">
@@ -16,15 +16,18 @@
<el-button type="text" @click="selectAll" class="select-all-btn">全选</el-button>
</div>
<div class="function-list">
<div v-for="func in unselected" :key="func.name" class="function-item">
<el-checkbox :label="func.name" v-model="selectedNames" @change="(val) => handleCheckboxChange(func, val)" @click.native.stop></el-checkbox>
<div class="func-tag" @click="handleFunctionClick(func)">
<div class="color-dot" :style="{backgroundColor: getFunctionColor(func.name)}"></div>
<span>{{ func.name }}</span>
<div v-if="unselected.length">
<div v-for="func in unselected" :key="func.name" class="function-item">
<el-checkbox :label="func.name" v-model="selectedNames" @change="(val) => handleCheckboxChange(func, val)"
@click.native.stop></el-checkbox>
<div class="func-tag" @click="handleFunctionClick(func)">
<div class="color-dot" :style="{ backgroundColor: getFunctionColor(func.name) }"></div>
<span>{{ func.name }}</span>
</div>
</div>
<el-tooltip class="item" effect="dark" :content="func.description || '暂无功能描述'" placement="top">
<img src="@/assets/home/info.png" alt="" class="info-icon">
</el-tooltip>
</div>
<div v-else style="display: flex; justify-content: center; align-items: center;">
<el-empty description="没有更多的插件了" />
</div>
</div>
</div>
@@ -36,30 +39,120 @@
<el-button type="text" @click="deselectAll" class="select-all-btn">全选</el-button>
</div>
<div class="function-list">
<div v-for="func in selectedList" :key="func.name" class="function-item">
<el-checkbox :label="func.name" v-model="selectedNames" @change="(val) => handleCheckboxChange(func, val)" @click.native.stop></el-checkbox>
<div class="func-tag" @click="handleFunctionClick(func)">
<div class="color-dot" :style="{backgroundColor: getFunctionColor(func.name)}"></div>
<span>{{ func.name }}</span>
<div v-if="selectedList.length > 0">
<div v-for="func in selectedList" :key="func.name" class="function-item">
<el-checkbox :label="func.name" v-model="selectedNames" @change="(val) => handleCheckboxChange(func, val)"
@click.native.stop></el-checkbox>
<div class="func-tag" @click="handleFunctionClick(func)">
<div class="color-dot" :style="{ backgroundColor: getFunctionColor(func.name) }"></div>
<span>{{ func.name }}</span>
</div>
</div>
</div>
<div v-else style="display: flex; justify-content: center; align-items: center;">
<el-empty description="请选择插件功能" />
</div>
</div>
</div>
<!-- 右侧：参数配置 -->
<div class="params-column">
<h4 v-if="currentFunction" class="column-title">参数配置 - {{ currentFunction.name }}</h4>
<div v-if="currentFunction" class="params-container">
<el-form :model="currentFunction" size="mini" class="param-form" v-loading="loading" element-loading-text="拼命加载中" element-loading-spinner="el-icon-loading" element-loading-background="rgba(255, 255, 255, 0.7)">
<el-form-item v-for="(value, key) in currentFunction.params" :key="key" :label="key" class="param-item">
<el-input v-model="currentFunction.params[key]" size="mini" class="param-input" @change="(val) => handleParamChange(currentFunction, key, val)"/>
</el-form-item>
</el-form>
</div>
<div v-if="currentFunction" class="params-container">
<el-form :model="currentFunction" class="param-form">
<!-- 遍历 fieldsMeta，而不是 params 的 keys -->
<div v-if="currentFunction.fieldsMeta.length == 0">
<el-empty :description="currentFunction.name + ' 无需配置参数'" />
</div>
<el-form-item v-for="field in currentFunction.fieldsMeta" :key="field.key" :label="field.label"
class="param-item" :class="{ 'textarea-field': field.type === 'array' || field.type === 'json' }">
<template #label>
<span style="font-size: 16px; margin-right: 6px;">{{ field.label }}</span>
<el-tooltip effect="dark" :content="fieldRemark(field)" placement="top">
<img src="@/assets/home/info.png" alt="" class="info-icon">
</el-tooltip>
</template>
<!-- ARRAY -->
<el-input v-if="field.type === 'array'" type="textarea" v-model="currentFunction.params[field.key]"
@change="val => handleParamChange(currentFunction, field.key, val)" />
<!-- JSON -->
<el-input v-else-if="field.type === 'json'" type="textarea" :rows="6" placeholder="请输入合法的 JSON"
v-model="textCache[field.key]" @blur="flushJson(field)" />
<!-- number -->
<el-input-number v-else-if="field.type === 'number'" :value="currentFunction.params[field.key]"
@change="val => handleParamChange(currentFunction, field.key, val)" />
<!-- boolean -->
<el-switch v-else-if="field.type === 'boolean' || field.type === 'bool'"
:value="currentFunction.params[field.key]"
@change="val => handleParamChange(currentFunction, field.key, val)" />
<!-- string or fallback -->
<el-input v-else v-model="currentFunction.params[field.key]"
@change="val => handleParamChange(currentFunction, field.key, val)" />
</el-form-item>
</el-form>
</div>
<div v-else class="empty-tip">请选择已配置的功能进行参数设置</div>
</div>
</div>
<!-- MCP区域 -->
<div class="mcp-access-point">
<div class="mcp-container">
<!-- 左侧区域 -->
<div class="mcp-left">
<div class="mcp-header">
<h3 class="bold-title">MCP接入点</h3>
</div>
<div class="url-header">
<div class="address-desc">
<span>以下是智能体的MCP接入点地址。</span>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md"
target="_blank" class="doc-link">查看接入点使用文档</a>
</div>
</div>
<el-input v-model="mcpUrl" readonly class="url-input">
<template #suffix>
<el-button @click="copyUrl" class="inner-copy-btn" icon="el-icon-document-copy">
复制
</el-button>
</template>
</el-input>
</div>
<!-- 右侧区域 -->
<div class="mcp-right">
<div class="mcp-header">
<h3 class="bold-title">接入点状态</h3>
</div>
<div class="status-container">
<span class="status-indicator" :class="mcpStatus"></span>
<span class="status-text">{{
mcpStatus === 'connected' ? '已连接' :
mcpStatus === 'loading' ? '加载中...' : '未连接'
}}</span>
<button class="refresh-btn" @click="refreshStatus">
<span class="refresh-icon">↻</span>
<span>刷新</span>
</button>
</div>
<div class="mcp-tools-list">
<div v-if="mcpTools.length > 0" class="tools-grid">
<el-button v-for="tool in mcpTools" :key="tool" size="small" class="tool-btn" plain>
{{ tool }}
</el-button>
</div>
<div v-else class="no-tools">
<span>暂无可用工具</span>
</div>
</div>
</div>
</div>
</div>
<div class="drawer-footer">
<el-button @click="closeDialog">取消</el-button>
<el-button type="primary" @click="saveSelection">保存配置</el-button>
@@ -68,30 +161,31 @@
</template>
<script>
import Api from '@/apis/api';
export default {
props: {
value: Boolean,
functions: {
type: Array,
default: () => []
},
allFunctions: {
type: Array,
default: () => []
},
agentId: {
type: String,
required: true
}
},
data() {
return {
textCache: {},
dialogVisible: this.value,
selectedNames: [],
currentFunction: null,
modifiedFunctions: {},
allFunctions: [
{name: '天气', params: {city: '北京'}, description: '查看指定城市的天气情况'},
{name: '新闻', params: {type: '科技'}, description: '获取最新科技类新闻资讯'},
{name: '工具', params: {category: '常用'}, description: '提供常用工具集合'},
{name: '退出', params: {}, description: '退出当前系统'},
{name: '音乐', params: {genre: '流行'}, description: '播放流行音乐'},
{name: '翻译', params: {from: '中文', to: '英文'}, description: '提供中英文互译功能'},
{name: '计算', params: {precision: '2'}, description: '提供精确计算功能'},
{name: '日历', params: {view: '月'}, description: '查看月历视图'}
],
functionColorMap: [
'#FF6B6B', '#4ECDC4', '#45B7D1',
'#96CEB4', '#FFEEAD', '#D4A5A5', '#A2836E'
@@ -100,6 +194,10 @@ export default {
// 添加一个标志位来跟踪是否已经保存
hasSaved: false,
loading: false,
mcpUrl: "",
mcpStatus: "disconnected",
mcpTools: [],
}
},
computed: {
@@ -111,11 +209,42 @@ export default {
}
},
watch: {
value(newVal) {
this.dialogVisible = newVal;
if (newVal) {
currentFunction(newFn) {
if (!newFn) return;
// 对每个字段，如果是 array 或 json，就在 textCache 里生成初始字符串
newFn.fieldsMeta.forEach(f => {
const v = newFn.params[f.key];
if (f.type === 'array') {
this.$set(this.textCache, f.key, Array.isArray(v) ? v.join('\n') : '');
}
else if (f.type === 'json') {
try {
this.$set(this.textCache, f.key, JSON.stringify(v ?? {}, null, 2));
} catch {
this.$set(this.textCache, f.key, '');
}
}
});
},
value(v) {
this.dialogVisible = v;
if (v) {
// 对话框打开时，初始化选中态
this.selectedNames = this.functions.map(f => f.name);
// 把后端传来的 this.functions（带 params）merge 到 allFunctions 上
this.functions.forEach(saved => {
const idx = this.allFunctions.findIndex(f => f.name === saved.name);
if (idx >= 0) {
// 保留用户之前在 saved.params 上的改动
this.allFunctions[idx].params = { ...saved.params };
}
});
// 右侧默认指向第一个
this.currentFunction = this.selectedList[0] || null;
// 加载MCP数据
this.loadMcpAddress();
this.loadMcpTools();
}
},
dialogVisible(newVal) {
@@ -123,14 +252,86 @@ export default {
}
},
methods: {
copyUrl() {
const textarea = document.createElement('textarea');
textarea.value = this.mcpUrl;
textarea.style.position = 'fixed'; // 防止页面滚动
document.body.appendChild(textarea);
textarea.select();
try {
const successful = document.execCommand('copy');
if (successful) {
this.$message.success('已复制到剪贴板');
} else {
this.$message.error('复制失败，请手动复制');
}
} catch (err) {
this.$message.error('复制失败，请手动复制');
console.error('复制失败:', err);
} finally {
document.body.removeChild(textarea);
}
},
refreshStatus() {
this.mcpStatus = "loading";
this.loadMcpTools();
},
// 加载MCP接入点地址
loadMcpAddress() {
Api.agent.getAgentMcpAccessAddress(this.agentId, (res) => {
if (res.data.code === 0) {
this.mcpUrl = res.data.data || "";
} else {
this.mcpUrl = "";
console.error('获取MCP地址失败:', res.data.msg);
}
});
},
// 加载MCP工具列表
loadMcpTools() {
Api.agent.getAgentMcpToolsList(this.agentId, (res) => {
if (res.data.code === 0) {
this.mcpTools = res.data.data || [];
// 根据工具列表更新状态
this.mcpStatus = this.mcpTools.length > 0 ? "connected" : "disconnected";
} else {
this.mcpTools = [];
this.mcpStatus = "disconnected";
console.error('获取MCP工具列表失败:', res.data.msg);
}
});
},
flushArray(key) {
const text = this.textCache[key] || '';
const arr = text
.split('\n')
.map(s => s.trim())
.filter(Boolean);
this.handleParamChange(this.currentFunction, key, arr);
},
flushJson(field) {
const key = field.key;
if (!key) {
return;
}
const text = this.textCache[key] || '';
try {
const obj = JSON.parse(text);
this.handleParamChange(this.currentFunction, key, obj);
} catch {
this.$message.error(`${this.currentFunction.name}的${key}字段格式错误：JSON格式有误`);
}
},
handleFunctionClick(func) {
if (this.selectedNames.includes(func.name)) {
this.loading = true;
setTimeout(() => {
const tempFunc = this.tempFunctions[func.name];
this.currentFunction = tempFunc ? tempFunc : JSON.parse(JSON.stringify(func));
this.loading = false;
}, 300);
const tempFunc = this.tempFunctions[func.name];
this.currentFunction = tempFunc ? tempFunc : func;
}
},
handleParamChange(func, key, value) {
@@ -185,23 +386,31 @@ export default {
const selected = this.selectedList.map(f => {
const modified = this.modifiedFunctions[f.name];
return modified || f;
}).map(f => ({
...f,
params: JSON.parse(JSON.stringify(f.params))
}));
return {
id: f.id,
name: f.name,
params: modified
? { ...modified.params }
: { ...f.params }
}
});
this.$emit('update-functions', selected);
this.dialogVisible = false;
this.$message.success('配置保存成功');
// 通知父组件对话框已关闭且已保存
this.$emit('dialog-closed', true);
},
getFunctionColor(name) {
const hash = [...name].reduce((acc, char) => acc + char.charCodeAt(0), 0);
return this.functionColorMap[hash % 7];
}
return this.functionColorMap[hash % this.functionColorMap.length];
},
fieldRemark(field) {
let description = (field && field.label) ? field.label : '';
if (field.default) {
description += `（默认值：${field.default}）`;
}
return description;
},
}
}
</script>
@@ -209,9 +418,9 @@ export default {
<style lang="scss" scoped>
.function-manager {
display: grid;
grid-template-columns: minmax(120px, 0.5fr) minmax(120px, 0.5fr) minmax(200px, 2fr);
grid-template-columns: max-content max-content 1fr;
gap: 12px;
height: calc(70vh - 60px);
height: calc(58vh);
}
.custom-header {
@@ -248,6 +457,7 @@ export default {
overflow-y: auto;
border-right: 1px solid #EBEEF5;
scrollbar-width: none;
overflow-x: hidden;
}
.function-column::-webkit-scrollbar {
@@ -257,7 +467,7 @@ export default {
.function-list {
display: flex;
flex-direction: column;
gap: 4px;
gap: 8px;
}
.function-item {
@@ -317,9 +527,31 @@ export default {
}
.param-form {
.param-item {
font-size: 16px;
&.textarea-field {
::v-deep .el-form-item__content {
margin-left: 0 !important;
display: block;
width: 100%;
}
::v-deep .el-form-item__label {
display: block;
width: 100% !important;
margin-bottom: 8px;
}
}
}
.param-input {
width: 100%;
}
::v-deep .el-form-item {
display: flex;
align-items: center;
flex-direction: column;
margin-bottom: 12px;
.el-form-item__label {
@@ -356,9 +588,6 @@ export default {
text-align: center;
}
.param-input {
width: 100%;
}
.drawer-footer {
position: absolute;
@@ -407,4 +636,202 @@ export default {
::v-deep .el-checkbox__label {
display: none;
}
.mcp-access-point {
border-top: 1px solid #EBEEF5;
padding: 20px 24px;
text-align: left;
}
.mcp-header {
.bold-title {
font-size: 18px;
font-weight: bold;
margin: 5px 0 30px 0;
}
}
.mcp-container {
display: flex;
justify-content: space-between;
gap: 30px;
}
.mcp-left,
.mcp-right {
flex: 1;
}
.url-header {
margin-bottom: 8px;
color: black;
h4 {
margin: 0 0 15px 0;
font-size: 16px;
font-weight: normal;
}
.address-desc {
display: flex;
align-items: center;
font-size: 14px;
margin-bottom: 12px;
.doc-link {
color: #1677ff;
text-decoration: none;
margin-left: 4px;
&:hover {
text-decoration: underline;
}
}
}
}
.url-input {
border-radius: 4px 0 0 4px;
font-size: 14px;
height: 36px;
box-sizing: border-box;
background-color: #f5f5f5;
}
::v-deep .el-input__inner {
background-color: #f5f5f5;
padding-right: 80px;
}
.url-input {
::v-deep .el-input__suffix {
right: 0;
display: flex;
align-items: center;
padding-right: 10px;
.inner-copy-btn {
pointer-events: auto;
border: none;
background: #1677ff;
color: white;
padding: 6px;
margin-top: 4px;
margin-left: 4px;
}
}
}
.mcp-right {
h4 {
margin: 0 0 10px 0;
font-size: 16px;
font-weight: normal;
color: black;
}
}
.status-container {
display: flex;
align-items: center;
.status-indicator {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 8px;
&.disconnected {
background-color: #909399;
/* 灰色 - 未连接 */
}
&.connected {
background-color: #67C23A;
/* 绿色 - 已连接 */
}
&.loading {
background-color: #E6A23C;
/* 橙色 - 加载中 */
animation: pulse 1.5s infinite;
}
}
.status-text {
font-size: 14px;
margin-right: 10px;
}
.refresh-btn {
display: flex;
align-items: center;
padding: 2px 10px;
background: white;
color: black;
border: 1px solid #DCDFE6;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
transition: all 0.3s;
&:hover {
background: #1677ff;
color: white;
border-color: #1677ff;
}
.refresh-icon {
margin-right: 6px;
font-size: 14px;
}
}
}
@keyframes pulse {
0% {
opacity: 1;
}
50% {
opacity: 0.4;
}
100% {
opacity: 1;
}
}
.mcp-tools-list {
margin-top: 10px;
.tools-grid {
display: flex;
flex-wrap: wrap;
gap: 8px;
}
.tool-btn {
padding: 6px 12px;
border-color: #1677ff;
color: #1677ff;
background-color: white;
font-size: 12px;
&:hover {
background-color: #1677ff;
color: white;
border-color: #1677ff;
}
}
.no-tools {
text-align: center;
color: #909399;
font-size: 14px;
padding: 10px 0;
}
}
</style>
main/manager-web/src/components/ManualAddDeviceDialog.vue
+158
View File
@@ -0,0 +1,158 @@
<template>
<el-dialog title="手动添加设备" :visible="visible" @close="handleClose" width="30%" center>
<div class="dialog-content">
<el-form :model="deviceForm" :rules="rules" ref="deviceForm" label-width="100px">
<el-form-item label="设备型号" prop="board">
<el-select v-model="deviceForm.board" placeholder="请选择设备型号" style="width: 100%">
<el-option
v-for="item in firmwareTypes"
:key="item.key"
:label="item.name"
:value="item.key">
</el-option>
</el-select>
</el-form-item>
<el-form-item label="固件版本" prop="appVersion">
<el-input v-model="deviceForm.appVersion" placeholder="请输入固件版本"></el-input>
</el-form-item>
<el-form-item label="Mac地址" prop="macAddress">
<el-input v-model="deviceForm.macAddress" placeholder="请输入Mac地址"></el-input>
</el-form-item>
</el-form>
</div>
<div style="display: flex;margin: 15px 15px;gap: 7px;">
<div class="dialog-btn" @click="submitForm">确定</div>
<div class="dialog-btn cancel-btn" @click="cancel">取消</div>
</div>
</el-dialog>
</template>
<script>
import Api from '@/apis/api';
export default {
name: 'ManualAddDeviceDialog',
props: {
visible: { type: Boolean, required: true },
agentId: { type: String, required: true }
},
data() {
// MAC地址验证规则
const validateMac = (rule, value, callback) => {
const macRegex = /^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$/;
if (!value) {
callback(new Error('请输入Mac地址'));
} else if (!macRegex.test(value)) {
callback(new Error('请输入正确的Mac地址格式，例如：00:1A:2B:3C:4D:5E'));
} else {
callback();
}
};
return {
deviceForm: {
board: '',
appVersion: '',
macAddress: ''
},
firmwareTypes: [],
rules: {
board: [
{ required: true, message: '请选择设备型号', trigger: 'change' }
],
appVersion: [
{ required: true, message: '请输入固件版本', trigger: 'blur' }
],
macAddress: [
{ required: true, validator: validateMac, trigger: 'blur' }
]
}
}
},
created() {
this.getFirmwareTypes();
},
methods: {
async getFirmwareTypes() {
try {
const res = await Api.dict.getDictDataByType('FIRMWARE_TYPE');
this.firmwareTypes = res.data;
} catch (error) {
console.error('获取固件类型失败:', error);
this.$message.error(error.message || '获取固件类型失败');
}
},
submitForm() {
this.$refs.deviceForm.validate((valid) => {
if (valid) {
this.addDevice();
}
});
},
addDevice() {
const params = {
agentId: this.agentId,
...this.deviceForm
};
Api.device.manualAddDevice(params, ({ data }) => {
if (data.code === 0) {
this.$message.success('设备添加成功');
this.$emit('refresh');
this.closeDialog();
} else {
this.$message.error(data.msg || '添加失败');
}
});
},
closeDialog() {
this.$emit('update:visible', false);
this.$refs.deviceForm.resetFields();
},
cancel() {
this.closeDialog();
},
handleClose() {
this.closeDialog();
}
}
}
</script>
<style scoped>
.dialog-content {
padding: 0 20px;
}
.dialog-btn {
cursor: pointer;
flex: 1;
border-radius: 23px;
background: #5778ff;
height: 40px;
font-weight: 500;
font-size: 12px;
color: #fff;
line-height: 40px;
text-align: center;
}
.cancel-btn {
background: #e6ebff;
border: 1px solid #adbdff;
color: #5778ff;
}
::v-deep .el-dialog {
border-radius: 15px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
}
::v-deep .el-dialog__body {
padding: 20px 6px;
}
::v-deep .el-form-item {
margin-bottom: 20px;
}
</style>
main/manager-web/src/components/ModelEditDialog.vue
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<el-dialog :visible.sync="dialogVisible" width="57%" center custom-class="custom-dialog" :show-close="false"
<el-dialog :visible.sync="dialogVisible" :close-on-click-modal="false" width="57%" center custom-class="custom-dialog" :show-close="false"
class="center-dialog" >
<div style="margin: 0 18px; text-align: left; padding: 10px; border-radius: 10px;">
<div style="font-size: 30px; color: #3d4566; margin-top: -10px; margin-bottom: 10px; text-align: center;">
main/manager-web/src/components/ProviderDialog.vue
+8 -6
View File
@@ -1,5 +1,5 @@
<template>
<el-dialog :visible="visible" @update:visible="handleVisibleChange" width="57%" center custom-class="custom-dialog"
<el-dialog :visible="visible" :close-on-click-modal="false" @update:visible="handleVisibleChange" width="57%" center custom-class="custom-dialog"
:show-close="false" class="center-dialog">
<div style="margin: 0 18px; text-align: left; padding: 10px; border-radius: 10px;">
@@ -18,7 +18,7 @@
</el-select>
</el-form-item>
<el-form-item label="供应器编码" prop="providerCode" style="flex: 1;">
<el-form-item label="编码" prop="providerCode" style="flex: 1;">
<el-input v-model="form.providerCode" placeholder="请输入供应器编码" class="custom-input-bg"></el-input>
</el-form-item>
</div>
@@ -87,6 +87,7 @@
<el-option label="数字" value="number"></el-option>
<el-option label="布尔值" value="boolean"></el-option>
<el-option label="字典" value="dict"></el-option>
<el-option label="分号分割的列表" value="array"></el-option>
</el-select>
</template>
<template v-else>
@@ -97,10 +98,10 @@
<el-table-column label="默认值">
<template slot-scope="scope">
<template v-if="scope.row.editing">
<el-input v-model="scope.row.default_value" placeholder="请输入默认值"></el-input>
<el-input v-model="scope.row.default" placeholder="请输入默认值"></el-input>
</template>
<template v-else>
{{ scope.row.default_value }}
{{ scope.row.default }}
</template>
</template>
</el-table-column>
@@ -161,7 +162,8 @@ export default {
'string': '字符串',
'number': '数字',
'boolean': '布尔值',
'dict': '字典'
'dict': '字典',
'array': '分号分割的列表'
};
return typeMap[type];
},
@@ -220,7 +222,7 @@ export default {
key: '',
label: '',
type: 'string',
default_value: '',
default: '',
selected: false,
editing: true
});
main/manager-web/src/views/DeviceManagement.vue
+112 -57
View File
@@ -1,12 +1,12 @@
<template>
<div class="welcome">
<HeaderBar />
<HeaderBar/>
<div class="operation-bar">
<h2 class="page-title">设备管理</h2>
<div class="right-operations">
<el-input placeholder="请输入设备型号或Mac地址查询" v-model="searchKeyword" class="search-input"
@keyup.enter.native="handleSearch" clearable />
@keyup.enter.native="handleSearch" clearable/>
<el-button class="btn-search" @click="handleSearch">搜索</el-button>
</div>
</div>
@@ -16,8 +16,9 @@
<div class="content-area">
<el-card class="device-card" shadow="never">
<el-table ref="deviceTable" :data="paginatedDeviceList" class="transparent-table"
:header-cell-class-name="headerCellClassName" v-loading="loading" element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading" element-loading-background="rgba(255, 255, 255, 0.7)">
:header-cell-class-name="headerCellClassName" v-loading="loading"
element-loading-text="拼命加载中"
element-loading-spinner="el-icon-loading" element-loading-background="rgba(255, 255, 255, 0.7)">
<el-table-column label="选择" align="center" width="120">
<template slot-scope="scope">
<el-checkbox v-model="scope.row.selected"></el-checkbox>
@@ -33,22 +34,32 @@
<el-table-column label="绑定时间" prop="bindTime" align="center"></el-table-column>
<el-table-column label="最近对话" prop="lastConversation" align="center"></el-table-column>
<el-table-column label="备注" align="center">
<template slot-scope="scope">
<el-input v-if="scope.row.isEdit" v-model="scope.row.remark" size="mini"
@blur="stopEditRemark(scope.$index)"></el-input>
<span v-else>
<i v-if="!scope.row.remark" class="el-icon-edit"
@click="startEditRemark(scope.$index, scope.row)"></i>
<span v-else @click="startEditRemark(scope.$index, scope.row)">
{{ scope.row.remark }}
</span>
<template #default="{ row }">
<el-input
v-show="row.isEdit"
v-model="row.remark"
size="mini"
maxlength="64"
show-word-limit
@blur="onRemarkBlur(row)"
@keyup.enter.native="onRemarkEnter(row)"
/>
<span v-show="!row.isEdit" class="remark-view">
<i
class="el-icon-edit"
@click="row.isEdit = true"
style="cursor: pointer;"
></i>
<span @click="row.isEdit = true">
{{ row.remark || '—' }}
</span>
</span>
</template>
</el-table-column>
<el-table-column label="OTA升级" align="center">
<template slot-scope="scope">
<el-switch v-model="scope.row.otaSwitch" size="mini" active-color="#13ce66" inactive-color="#ff4949"
@change="handleOtaSwitchChange(scope.row)"></el-switch>
@change="handleOtaSwitchChange(scope.row)"></el-switch>
</template>
</el-table-column>
<el-table-column label="操作" align="center">
@@ -66,7 +77,10 @@
{{ isAllSelected ? '取消全选' : '全选' }}
</el-button>
<el-button type="success" size="mini" class="add-device-btn" @click="handleAddDevice">
新增
验证码绑定
</el-button>
<el-button type="success" size="mini" class="add-device-btn" @click="handleManualAddDevice">
手动添加
</el-button>
<el-button size="mini" type="danger" icon="el-icon-delete" @click="deleteSelected">解绑</el-button>
</div>
@@ -78,7 +92,7 @@
<button class="pagination-btn" :disabled="currentPage === 1" @click="goFirst">首页</button>
<button class="pagination-btn" :disabled="currentPage === 1" @click="goPrev">上一页</button>
<button v-for="page in visiblePages" :key="page" class="pagination-btn"
:class="{ active: page === currentPage }" @click="goToPage(page)">
:class="{ active: page === currentPage }" @click="goToPage(page)">
{{ page }}
</button>
<button class="pagination-btn" :disabled="currentPage === pageCount" @click="goNext">下一页</button>
@@ -91,7 +105,9 @@
</div>
<AddDeviceDialog :visible.sync="addDeviceDialogVisible" :agent-id="currentAgentId"
@refresh="fetchBindDevices(currentAgentId)" />
@refresh="fetchBindDevices(currentAgentId)"/>
<ManualAddDeviceDialog :visible.sync="manualAddDeviceDialogVisible" :agent-id="currentAgentId"
@refresh="fetchBindDevices(currentAgentId)"/>
</div>
</template>
@@ -99,13 +115,19 @@
<script>
import Api from '@/apis/api';
import AddDeviceDialog from "@/components/AddDeviceDialog.vue";
import ManualAddDeviceDialog from "@/components/ManualAddDeviceDialog.vue";
import HeaderBar from "@/components/HeaderBar.vue";
export default {
components: { HeaderBar, AddDeviceDialog },
components: {
HeaderBar,
AddDeviceDialog,
ManualAddDeviceDialog
},
data() {
return {
addDeviceDialogVisible: false,
manualAddDeviceDialogVisible: false,
selectedDevices: [],
isAllSelected: false,
searchKeyword: "",
@@ -125,18 +147,15 @@ export default {
const keyword = this.activeSearchKeyword.toLowerCase();
if (!keyword) return this.deviceList;
return this.deviceList.filter(device =>
(device.model && device.model.toLowerCase().includes(keyword)) ||
(device.macAddress && device.macAddress.toLowerCase().includes(keyword))
(device.model && device.model.toLowerCase().includes(keyword)) ||
(device.macAddress && device.macAddress.toLowerCase().includes(keyword))
);
},
paginatedDeviceList() {
const start = (this.currentPage - 1) * this.pageSize;
const end = start + this.pageSize;
return this.filteredDeviceList.slice(start, end).map(item => ({
...item,
selected: false
}));
return this.filteredDeviceList.slice(start, end);
},
pageCount() {
return Math.ceil(this.filteredDeviceList.length / this.pageSize);
@@ -212,11 +231,10 @@ export default {
this.batchUnbindDevices(deviceIds);
});
},
batchUnbindDevices(deviceIds) {
const promises = deviceIds.map(id => {
return new Promise((resolve, reject) => {
Api.device.unbindDevice(id, ({ data }) => {
Api.device.unbindDevice(id, ({data}) => {
if (data.code === 0) {
resolve();
} else {
@@ -225,33 +243,64 @@ export default {
});
});
});
Promise.all(promises)
.then(() => {
this.$message.success({
message: `成功解绑 ${deviceIds.length} 台设备`,
showClose: true
.then(() => {
this.$message.success({
message: `成功解绑 ${deviceIds.length} 台设备`,
showClose: true
});
this.fetchBindDevices(this.currentAgentId);
this.selectedDevices = [];
this.isAllSelected = false;
})
.catch(error => {
this.$message.error({
message: error || '批量解绑过程中出现错误',
showClose: true
});
});
this.fetchBindDevices(this.currentAgentId);
this.selectedDevices = [];
this.isAllSelected = false;
})
.catch(error => {
this.$message.error({
message: error || '批量解绑过程中出现错误',
showClose: true
});
});
},
handleAddDevice() {
this.addDeviceDialogVisible = true;
},
startEditRemark(index, row) {
this.deviceList[index].isEdit = true;
handleManualAddDevice() {
this.manualAddDeviceDialogVisible = true;
},
stopEditRemark(index) {
this.deviceList[index].isEdit = false;
submitRemark(row) {
if (row._submitting) return;
const text = (row.remark || '').trim();
if (text.length > 64) {
this.$message.warning('备注不能超过 64 字符');
return;
}
if (text === row._originalRemark) {
return;
}
row._submitting = true;
this.updateDeviceInfo(row.device_id, { alias: text }, (ok, resp) => {
if (ok) {
row._originalRemark = text;
this.$message.success('备注已保存');
} else {
row.remark = row._originalRemark;
this.$message.error(resp.msg || '备注保存失败');
}
row._submitting = false;
});
},
// 备注输入框：失焦时提交
onRemarkBlur(row) {
row.isEdit = false;
setTimeout(() => {
this.submitRemark(row);
}, 100); // 延迟 100ms，避开 enter+blur 同时触发的窗口
},
// 备注输入框：按回车时提交
onRemarkEnter(row) {
row.isEdit = false;
this.submitRemark(row);
},
handleUnbind(device_id) {
this.$confirm('确认要解绑该设备吗？', '警告', {
@@ -259,7 +308,7 @@ export default {
cancelButtonText: '取消',
type: 'warning'
}).then(() => {
Api.device.unbindDevice(device_id, ({ data }) => {
Api.device.unbindDevice(device_id, ({data}) => {
if (data.code === 0) {
this.$message.success({
message: '设备解绑成功',
@@ -290,7 +339,7 @@ export default {
fetchBindDevices(agentId) {
this.loading = true;
Api.device.getAgentBindDevices(agentId, ({ data }) => {
Api.device.getAgentBindDevices(agentId, ({data}) => {
this.loading = false;
if (data.code === 0) {
this.deviceList = data.data.map(device => {
@@ -302,12 +351,14 @@ export default {
bindTime: device.createDate,
lastConversation: device.lastConnectedAt,
remark: device.alias,
_originalRemark: device.alias,
isEdit: false,
_submitting: false,
otaSwitch: device.autoUpdate === 1,
rawBindTime: new Date(device.createDate).getTime()
};
})
.sort((a, b) => a.rawBindTime - b.rawBindTime);
.sort((a, b) => a.rawBindTime - b.rawBindTime);
this.activeSearchKeyword = "";
this.searchKeyword = "";
} else {
@@ -315,7 +366,7 @@ export default {
}
});
},
headerCellClassName({ columnIndex }) {
headerCellClassName({columnIndex}) {
if (columnIndex === 0) {
return "custom-selection-header";
}
@@ -325,14 +376,19 @@ export default {
const firmwareType = this.firmwareTypes.find(item => item.key === type)
return firmwareType ? firmwareType.name : type
},
updateDeviceInfo(device_id, payload, callback) {
return Api.device.updateDeviceInfo(device_id, payload, ({data}) => {
callback(data.code === 0, data);
})
},
handleOtaSwitchChange(row) {
Api.device.enableOtaUpgrade(row.device_id, row.otaSwitch ? 1 : 0, ({ data }) => {
if (data.code === 0) {
this.$message.success(row.otaSwitch ? '已设置成自动升级' : '已关闭自动升级')
} else {
row.otaSwitch = !row.otaSwitch
this.$message.error(data.msg || '操作失败')
this.updateDeviceInfo(row.device_id, {autoUpdate: row.otaSwitch ? 1 : 0}, (result, {msg}) => {
if (result) {
this.$message.success(row.otaSwitch ? '已设置成自动升级' : '已关闭自动升级');
return;
}
row.otaSwitch = !row.otaSwitch
this.$message.error(msg || '操作失败')
})
},
}
@@ -645,7 +701,6 @@ export default {
}
:deep(.el-table .el-button--text) {
color: #7079aa;
}
main/manager-web/src/views/ProviderManagement.vue
+3 -1
View File
@@ -143,9 +143,11 @@ export default {
{ value: "ASR", label: "语音识别" },
{ value: "TTS", label: "语音合成" },
{ value: "LLM", label: "大语言模型" },
{ value: "VLLM", label: "视觉大语言模型" },
{ value: "Intent", label: "意图识别" },
{ value: "Memory", label: "记忆模块" },
{ value: "VAD", label: "语音活动检测" }
{ value: "VAD", label: "语音活动检测" },
{ value: "Plugin", label: "插件工具" }
],
currentPage: 1,
loading: false,
main/manager-web/src/views/roleConfig.vue
+74 -41
View File
@@ -97,19 +97,12 @@
popper-class="custom-tooltip">
<div slot="content">
<div><strong>功能名称:</strong> {{ func.name }}</div>
<div v-if="Object.keys(func.params).length > 0">
<strong>参数配置:</strong>
<div v-for="(value, key) in func.params" :key="key">
{{ key }}: {{ value }}
</div>
</div>
<div v-else>无参数配置</div>
</div>
<div class="icon-dot" :style="{ backgroundColor: getFunctionColor(func.name) }">
{{ func.name.charAt(0) }}
</div>
</el-tooltip>
<el-button class="edit-function-btn" @click="showFunctionDialog = true"
<el-button class="edit-function-btn" @click="openFunctionDialog"
:class="{ 'active-btn': showFunctionDialog }">
编辑功能
</el-button>
@@ -138,8 +131,8 @@
</div>
</div>
<function-dialog v-model="showFunctionDialog" :functions="currentFunctions"
@update-functions="handleUpdateFunctions" @dialog-closed="handleDialogClosed" />
<function-dialog v-model="showFunctionDialog" :functions="currentFunctions" :all-functions="allFunctions"
:agent-id="$route.query.agentId" @update-functions="handleUpdateFunctions" @dialog-closed="handleDialogClosed" />
</div>
</template>
@@ -192,12 +185,8 @@ export default {
'#FF6B6B', '#4ECDC4', '#45B7D1',
'#96CEB4', '#FFEEAD', '#D4A5A5', '#A2836E'
],
allFunctions: [
{ name: '天气', params: {} },
{ name: '新闻', params: {} },
{ name: '工具', params: {} },
{ name: '退出', params: {} }
],
allFunctions: [],
originalFunctions: [],
}
},
methods: {
@@ -222,7 +211,12 @@ export default {
langCode: this.form.langCode,
language: this.form.language,
sort: this.form.sort,
functions: this.currentFunctions
functions: this.currentFunctions.map(item => {
return ({
pluginId: item.id,
paramInfo: item.params
})
})
};
Api.agent.updateAgentConfig(this.$route.query.agentId, configData, ({ data }) => {
if (data.code === 0) {
@@ -269,7 +263,8 @@ export default {
message: '配置已重置',
showClose: true
})
}).catch(() => { });
}).catch(() => {
});
},
fetchTemplates() {
Api.agent.getAgentTemplate(({ data }) => {
@@ -335,7 +330,33 @@ export default {
intentModelId: data.data.intentModelId
}
};
this.currentFunctions = data.data.functions || [];
// 后端只给了最小映射：[{ id, agentId, pluginId }, ...]
const savedMappings = data.data.functions || [];
// 先保证 allFunctions 已经加载（如果没有，则先 fetchAllFunctions）
const ensureFuncs = this.allFunctions.length
? Promise.resolve()
: this.fetchAllFunctions();
ensureFuncs.then(() => {
// 合并：按照 pluginId（id 字段）把全量元数据信息补齐
this.currentFunctions = savedMappings.map(mapping => {
const meta = this.allFunctions.find(f => f.id === mapping.pluginId);
if (!meta) {
// 插件定义没找到，退化处理
return { id: mapping.pluginId, name: mapping.pluginId, params: {} };
}
return {
id: mapping.pluginId,
name: meta.name,
// 后端如果还有 paramInfo 字段就用 mapping.paramInfo，否则用 meta.params 默认值
params: mapping.paramInfo || { ...meta.params },
fieldsMeta: meta.fieldsMeta // 保留以便对话框渲染 tooltip
};
});
// 备份原始，以备取消时恢复
this.originalFunctions = JSON.parse(JSON.stringify(this.currentFunctions));
});
} else {
this.$message.error(data.msg || '获取配置失败');
}
@@ -373,17 +394,15 @@ export default {
},
getFunctionColor(name) {
const hash = [...name].reduce((acc, char) => acc + char.charCodeAt(0), 0);
return this.functionColorMap[hash % 7];
return this.functionColorMap[hash % this.functionColorMap.length];
},
showFunctionIcons(type) {
// TODO 暂时不放出来
return false;
// return type === 'Intent' &&
// this.form.model.intentModelId !== 'Intent_nointent';
return type === 'Intent' &&
this.form.model.intentModelId !== 'Intent_nointent';
},
handleModelChange(type, value) {
if (type === 'Intent' && value !== 'Intent_nointent') {
this.fetchFunctionList();
this.fetchAllFunctions();
}
if (type === 'Memory' && value === 'Memory_nomem') {
this.form.chatHistoryConf = 0;
@@ -392,28 +411,44 @@ export default {
this.form.chatHistoryConf = 2;
}
},
fetchFunctionList() {
// 使用假数据代替API调用
return new Promise(resolve => {
setTimeout(() => {
this.currentFunctions = [
{ name: '天气', params: { city: '北京' } },
{ name: '新闻', params: { type: '科技' } }
];
resolve();
}, 500);
fetchAllFunctions() {
return new Promise((resolve, reject) => {
Api.model.getPluginFunctionList(null, ({ data }) => {
if (data.code === 0) {
this.allFunctions = data.data.map(item => {
const meta = JSON.parse(item.fields || '[]');
const params = meta.reduce((m, f) => {
m[f.key] = f.default;
return m;
}, {});
return { ...item, fieldsMeta: meta, params };
});
resolve();
} else {
this.$message.error(data.msg || '获取插件列表失败');
reject();
}
});
});
},
openFunctionDialog() {
// 显示编辑对话框时，确保 allFunctions 已经加载
if (this.allFunctions.length === 0) {
this.fetchAllFunctions().then(() => this.showFunctionDialog = true);
} else {
this.showFunctionDialog = true;
}
},
handleUpdateFunctions(selected) {
this.currentFunctions = selected;
console.log('保存的功能列表:', selected);
this.$message.success('功能配置已保存');
},
handleDialogClosed(saved) {
if (!saved) {
// 如果未保存，恢复原始功能列表
this.currentFunctions = JSON.parse(JSON.stringify(this.originalFunctions));
} else {
this.originalFunctions = JSON.parse(JSON.stringify(this.currentFunctions));
}
this.showFunctionDialog = false;
},
updateChatHistoryConf() {
if (this.form.model.memModelId === 'Memory_nomem') {
@@ -446,9 +481,7 @@ export default {
const agentId = this.$route.query.agentId;
if (agentId) {
this.fetchAgentConfig(agentId);
this.fetchFunctionList().then(() => {
this.originalFunctions = JSON.parse(JSON.stringify(this.currentFunctions));
});
this.fetchAllFunctions();
}
this.fetchModelOptions();
this.fetchTemplates();
main/xiaozhi-server/config.yaml
+12 -7
View File
@@ -104,7 +104,8 @@ wakeup_words:
- "喵喵同学"
- "小滨小滨"
- "小冰小冰"
# MCP接入点地址
mcp_endpoint: 你的接入点 websocket地址
# 插件的基础配置
plugins:
# 获取天气插件的配置，这里填写你的api_key
@@ -117,11 +118,15 @@ plugins:
# 更多类型的新闻列表查看 https://www.chinanews.com.cn/rss/
get_news_from_chinanews:
default_rss_url: "https://www.chinanews.com.cn/rss/society.xml"
category_urls:
society: "https://www.chinanews.com.cn/rss/society.xml"
world: "https://www.chinanews.com.cn/rss/world.xml"
finance: "https://www.chinanews.com.cn/rss/finance.xml"
get_news_from_newsnow: {"url": "https://newsnow.busiyi.world/api/s?id="}
society_rss_url: "https://www.chinanews.com.cn/rss/society.xml"
world_rss_url: "https://www.chinanews.com.cn/rss/world.xml"
finance_rss_url: "https://www.chinanews.com.cn/rss/finance.xml"
get_news_from_newsnow:
url: "https://newsnow.busiyi.world/api/s?id="
news_sources:
thepaper: "澎湃新闻"
baidu: "百度热搜"
cls-depth: "财联社"
home_assistant:
devices:
- 客厅,玩具灯,switch.cuco_cn_460494544_cp1_on_p_2_1
@@ -766,4 +771,4 @@ TTS:
# 支持声音克隆，可自行上传音频，填入voice参数，voice参数为空时，使用默认声音
access_token: "U4YdYXVfpwWnk2t5Gp822zWPCuORyeJL"
voice: "OUeAo1mhq6IBExi"
output_dir: tmp/
output_dir: tmp/
main/xiaozhi-server/config/logger.py
+5 -5
View File
@@ -5,7 +5,7 @@ from config.config_loader import load_config
from config.settings import check_config_file
from datetime import datetime
SERVER_VERSION = "0.5.5"
SERVER_VERSION = "0.6.1"
_logger_initialized = False
@@ -88,7 +88,7 @@ def setup_logging():
# 输出到文件 - 统一目录，按大小轮转
# 日志文件完整路径
log_file_path = os.path.join(log_dir, log_file)
# 添加日志处理器
logger.add(
log_file_path,
@@ -146,14 +146,14 @@ def update_module_string(selected_module_str):
level=log_config.get("log_level", "INFO"),
filter=formatter,
)
# 更新文件日志配置 - 统一目录，按大小轮转
log_dir = log_config.get("log_dir", "tmp")
log_file = log_config.get("log_file", "server.log")
# 日志文件完整路径
log_file_path = os.path.join(log_dir, log_file)
logger.add(
log_file_path,
format=log_format_file,
main/xiaozhi-server/core/api/vision_handler.py
+4 -6
View File
@@ -8,6 +8,7 @@ from config.config_loader import get_private_config_from_api
from core.utils.auth import AuthToken
import base64
from typing import Tuple, Optional
from plugins_func.register import Action
TAG = __name__
@@ -55,11 +56,7 @@ class VisionHandler:
device_id = request.headers.get("Device-Id", "")
client_id = request.headers.get("Client-Id", "")
if device_id != token_device_id:
return web.Response(
text=json.dumps(self._create_error_response("设备ID与token不匹配")),
content_type="application/json",
status=401,
)
raise ValueError("设备ID与token不匹配")
# 解析multipart/form-data请求
reader = await request.multipart()
@@ -126,7 +123,8 @@ class VisionHandler:
return_json = {
"success": True,
"result": result,
"action": Action.RESPONSE.name,
"response": result,
}
response = web.Response(
main/xiaozhi-server/core/connection.py
+51 -115
View File
@@ -10,7 +10,6 @@ import threading
import traceback
import subprocess
import websockets
from core.handle.mcpHandle import call_mcp_tool
from core.utils.util import (
extract_json_from_string,
check_vad_update,
@@ -18,7 +17,6 @@ from core.utils.util import (
filter_sensitive_info,
)
from typing import Dict, Any
from core.mcp.manager import MCPManager
from core.utils.modules_initialize import (
initialize_modules,
initialize_tts,
@@ -30,7 +28,7 @@ from concurrent.futures import ThreadPoolExecutor
from core.utils.dialogue import Message, Dialogue
from core.providers.asr.dto.dto import InterfaceType
from core.handle.textHandle import handleTextMessage
from core.handle.functionHandler import FunctionHandler
from core.providers.tools.unified_tool_handler import UnifiedToolHandler
from plugins_func.loadplugins import auto_import_modules
from plugins_func.register import Action, ActionResponse
from core.auth import AuthMiddleware, AuthenticationError
@@ -112,8 +110,7 @@ class ConnectionHandler:
# vad相关变量
self.client_audio_buffer = bytearray()
self.client_have_voice = False
self.client_have_voice_last_time = 0.0
self.client_no_voice_last_time = 0.0
self.last_activity_time = 0.0 # 统一的活动时间戳（毫秒）
self.client_voice_stop = False
# asr相关变量
@@ -144,10 +141,10 @@ class ConnectionHandler:
self.load_function_plugin = False
self.intent_type = "nointent"
self.timeout_task = None
self.timeout_seconds = (
int(self.config.get("close_connection_no_voice_time", 120)) + 60
) # 在原来第一道关闭的基础上加60秒，进行二道关闭
self.timeout_task = None
# {"mcp":true} 表示启用MCP功能
self.features = None
@@ -188,6 +185,9 @@ class ConnectionHandler:
self.websocket = ws
self.device_id = self.headers.get("device-id", None)
# 初始化活动时间戳
self.last_activity_time = time.time() * 1000
# 启动超时检查任务
self.timeout_task = asyncio.create_task(self._check_timeout())
@@ -242,18 +242,10 @@ class ConnectionHandler:
# 立即关闭连接，不等待记忆保存完成
await self.close(ws)
async def reset_timeout(self):
"""重置超时计时器"""
if self.timeout_task:
self.timeout_task.cancel()
self.timeout_task = asyncio.create_task(self._check_timeout())
async def _route_message(self, message):
"""消息路由"""
# 重置超时计时器
await self.reset_timeout()
if isinstance(message, str):
self.last_activity_time = time.time() * 1000
await handleTextMessage(self, message)
elif isinstance(message, bytes):
if self.vad is None:
@@ -425,10 +417,12 @@ class ConnectionHandler:
init_asr = check_asr_update(self.common_config, private_config)
if init_vad:
self.config["VAD"] = private_config["VAD"]
self.config["selected_module"]["VAD"] = private_config["selected_module"][
"VAD"
]
if init_asr:
self.config["ASR"] = private_config["ASR"]
self.config["selected_module"]["ASR"] = private_config["selected_module"][
"ASR"
]
@@ -453,9 +447,17 @@ class ConnectionHandler:
if private_config.get("Intent", None) is not None:
init_intent = True
self.config["Intent"] = private_config["Intent"]
self.config["selected_module"]["Intent"] = private_config[
"selected_module"
]["Intent"]
model_intent = private_config.get("selected_module", {}).get("Intent", {})
self.config["selected_module"]["Intent"] = model_intent
# 加载插件配置
if model_intent != "Intent_nointent":
plugin_from_server = private_config.get("plugins", {})
for plugin, config_str in plugin_from_server.items():
plugin_from_server[plugin] = json.loads(config_str)
self.config["plugins"] = plugin_from_server
self.config["Intent"][self.config["selected_module"]["Intent"]][
"functions"
] = plugin_from_server.keys()
if private_config.get("prompt", None) is not None:
self.config["prompt"] = private_config["prompt"]
if private_config.get("summaryMemory", None) is not None:
@@ -464,6 +466,8 @@ class ConnectionHandler:
self.max_output_size = int(private_config["device_max_output_size"])
if private_config.get("chat_history_conf", None) is not None:
self.chat_history_conf = int(private_config["chat_history_conf"])
if private_config.get("mcp_endpoint", None) is not None:
self.config["mcp_endpoint"] = private_config["mcp_endpoint"]
try:
modules = initialize_modules(
self.logger,
@@ -575,14 +579,12 @@ class ConnectionHandler:
self.intent.set_llm(self.llm)
self.logger.bind(tag=TAG).info("使用主LLM作为意图识别模型")
"""加载插件"""
self.func_handler = FunctionHandler(self)
self.mcp_manager = MCPManager(self)
"""加载统一工具处理器"""
self.func_handler = UnifiedToolHandler(self)
"""加载MCP工具"""
asyncio.run_coroutine_threadsafe(
self.mcp_manager.initialize_servers(), self.loop
)
# 异步初始化工具处理器
if hasattr(self, "loop") and self.loop:
asyncio.run_coroutine_threadsafe(self.func_handler._initialize(), self.loop)
def change_system_prompt(self, prompt):
self.prompt = prompt
@@ -600,12 +602,6 @@ class ConnectionHandler:
functions = None
if self.intent_type == "function_call" and hasattr(self, "func_handler"):
functions = self.func_handler.get_functions()
if hasattr(self, "mcp_client"):
mcp_tools = self.mcp_client.get_available_tools()
if mcp_tools is not None and len(mcp_tools) > 0:
if functions is None:
functions = []
functions.extend(mcp_tools)
response_message = []
try:
@@ -617,8 +613,7 @@ class ConnectionHandler:
)
memory_str = future.result()
uuid_str = str(uuid.uuid4()).replace("-", "")
self.sentence_id = uuid_str
self.sentence_id = str(uuid.uuid4().hex)
if self.intent_type == "function_call" and functions is not None:
# 使用支持functions的streaming接口
@@ -723,37 +718,13 @@ class ConnectionHandler:
"arguments": function_arguments,
}
# 处理Server端MCP工具调用
if self.mcp_manager.is_mcp_tool(function_name):
result = self._handle_mcp_tool_call(function_call_data)
elif hasattr(self, "mcp_client") and self.mcp_client.has_tool(
function_name
):
# 如果是小智端MCP工具调用
self.logger.bind(tag=TAG).debug(
f"调用小智端MCP工具: {function_name}, 参数: {function_arguments}"
)
try:
result = asyncio.run_coroutine_threadsafe(
call_mcp_tool(
self, self.mcp_client, function_name, function_arguments
),
self.loop,
).result()
self.logger.bind(tag=TAG).debug(f"MCP工具调用结果: {result}")
result = ActionResponse(
action=Action.REQLLM, result=result, response=""
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"MCP工具调用失败: {e}")
result = ActionResponse(
action=Action.REQLLM, result="MCP工具调用失败", response=""
)
else:
# 处理系统函数
result = self.func_handler.handle_llm_function_call(
# 使用统一工具处理器处理所有工具调用
result = asyncio.run_coroutine_threadsafe(
self.func_handler.handle_llm_function_call(
self, function_call_data
)
),
self.loop,
).result()
self._handle_function_result(result, function_call_data)
# 存储对话内容
@@ -776,48 +747,6 @@ class ConnectionHandler:
return True
def _handle_mcp_tool_call(self, function_call_data):
function_arguments = function_call_data["arguments"]
function_name = function_call_data["name"]
try:
args_dict = function_arguments
if isinstance(function_arguments, str):
try:
args_dict = json.loads(function_arguments)
except json.JSONDecodeError:
self.logger.bind(tag=TAG).error(
f"无法解析 function_arguments: {function_arguments}"
)
return ActionResponse(
action=Action.REQLLM, result="参数解析失败", response=""
)
tool_result = asyncio.run_coroutine_threadsafe(
self.mcp_manager.execute_tool(function_name, args_dict), self.loop
).result()
# meta=None content=[TextContent(type='text', text='北京当前天气:\n温度: 21°C\n天气: 晴\n湿度: 6%\n风向: 西北 风\n风力等级: 5级', annotations=None)] isError=False
content_text = ""
if tool_result is not None and tool_result.content is not None:
for content in tool_result.content:
content_type = content.type
if content_type == "text":
content_text = content.text
elif content_type == "image":
pass
if len(content_text) > 0:
return ActionResponse(
action=Action.REQLLM, result=content_text, response=""
)
except Exception as e:
self.logger.bind(tag=TAG).error(f"MCP工具调用错误: {e}")
return ActionResponse(
action=Action.REQLLM, result="工具调用出错", response=""
)
return ActionResponse(action=Action.REQLLM, result="工具调用出错", response="")
def _handle_function_result(self, result, function_call_data):
if result.action == Action.RESPONSE: # 直接回复前端
text = result.response
@@ -857,7 +786,7 @@ class ConnectionHandler:
)
self.chat(text, tool_call=True)
elif result.action == Action.NOTFOUND or result.action == Action.ERROR:
text = result.result
text = result.response if result.response else result.result
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
self.dialogue.put(Message(role="assistant", content=text))
else:
@@ -912,9 +841,9 @@ class ConnectionHandler:
self.timeout_task.cancel()
self.timeout_task = None
# 清理MCP资源
if hasattr(self, "mcp_manager") and self.mcp_manager:
await self.mcp_manager.cleanup_all()
# 清理工具处理器资源
if hasattr(self, "func_handler") and self.func_handler:
await self.func_handler.cleanup()
# 触发停止事件
if self.stop_event:
@@ -966,7 +895,6 @@ class ConnectionHandler:
def reset_vad_states(self):
self.client_audio_buffer = bytearray()
self.client_have_voice = False
self.client_have_voice_last_time = 0
self.client_voice_stop = False
self.logger.bind(tag=TAG).debug("VAD states reset.")
@@ -985,10 +913,18 @@ class ConnectionHandler:
"""检查连接超时"""
try:
while not self.stop_event.is_set():
await asyncio.sleep(self.timeout_seconds)
if not self.stop_event.is_set():
self.logger.bind(tag=TAG).info("连接超时，准备关闭")
await self.close(self.websocket)
break
# 检查是否超时（只有在时间戳已初始化的情况下）
if self.last_activity_time > 0.0:
current_time = time.time() * 1000
if (
current_time - self.last_activity_time
> self.timeout_seconds * 1000
):
if not self.stop_event.is_set():
self.logger.bind(tag=TAG).info("连接超时，准备关闭")
await self.close(self.websocket)
break
# 每10秒检查一次，避免过于频繁
await asyncio.sleep(10)
except Exception as e:
self.logger.bind(tag=TAG).error(f"超时检查任务出错: {e}")
main/xiaozhi-server/core/handle/functionHandler.py
-109
View File
@@ -1,109 +0,0 @@
from config.logger import setup_logging
import json
from plugins_func.register import (
FunctionRegistry,
ActionResponse,
Action,
ToolType,
DeviceTypeRegistry,
)
from plugins_func.functions.hass_init import append_devices_to_prompt
TAG = __name__
class FunctionHandler:
def __init__(self, conn):
self.conn = conn
self.config = conn.config
self.device_type_registry = DeviceTypeRegistry()
self.function_registry = FunctionRegistry()
self.register_nessary_functions()
self.register_config_functions()
self.functions_desc = self.function_registry.get_all_function_desc()
func_names = self.current_support_functions()
self.modify_plugin_loader_des(func_names)
self.finish_init = True
def modify_plugin_loader_des(self, func_names):
if "plugin_loader" not in func_names:
return
# 可编辑的列表中去掉plugin_loader
surport_plugins = [func for func in func_names if func != "plugin_loader"]
func_names = ",".join(surport_plugins)
for function_desc in self.functions_desc:
if function_desc["function"]["name"] == "plugin_loader":
function_desc["function"]["description"] = function_desc["function"][
"description"
].replace("[plugins]", func_names)
break
def upload_functions_desc(self):
self.functions_desc = self.function_registry.get_all_function_desc()
def current_support_functions(self):
func_names = []
for func in self.functions_desc:
func_names.append(func["function"]["name"])
# 打印当前支持的函数列表
self.conn.logger.bind(tag=TAG, session_id=self.conn.session_id).info(
f"当前支持的函数列表: {func_names}"
)
return func_names
def get_functions(self):
"""获取功能调用配置"""
return self.functions_desc
def register_nessary_functions(self):
"""注册必要的函数"""
self.function_registry.register_function("handle_exit_intent")
self.function_registry.register_function("plugin_loader")
self.function_registry.register_function("get_time")
self.function_registry.register_function("get_lunar")
# self.function_registry.register_function("handle_speaker_volume_or_screen_brightness")
def register_config_functions(self):
"""注册配置中的函数,可以不同客户端使用不同的配置"""
for func in self.config["Intent"][self.config["selected_module"]["Intent"]].get(
"functions", []
):
self.function_registry.register_function(func)
"""home assistant需要初始化提示词"""
append_devices_to_prompt(self.conn)
def get_function(self, name):
return self.function_registry.get_function(name)
def handle_llm_function_call(self, conn, function_call_data):
try:
function_name = function_call_data["name"]
funcItem = self.get_function(function_name)
if not funcItem:
return ActionResponse(
action=Action.NOTFOUND, result="没有找到对应的函数", response=""
)
func = funcItem.func
arguments = function_call_data["arguments"]
arguments = json.loads(arguments) if arguments else {}
self.conn.logger.bind(tag=TAG).debug(
f"调用函数: {function_name}, 参数: {arguments}"
)
if (
funcItem.type == ToolType.SYSTEM_CTL
or funcItem.type == ToolType.IOT_CTL
):
return func(conn, **arguments)
elif funcItem.type == ToolType.WAIT:
return func(**arguments)
elif funcItem.type == ToolType.CHANGE_SYS_PROMPT:
return func(conn, **arguments)
else:
return ActionResponse(
action=Action.NOTFOUND, result="没有找到对应的函数", response=""
)
except Exception as e:
self.conn.logger.bind(tag=TAG).error(f"处理function call错误: {e}")
return None
main/xiaozhi-server/core/handle/helloHandle.py
+1 -1
View File
@@ -7,7 +7,7 @@ from core.utils.util import audio_to_data
from core.handle.sendAudioHandle import sendAudioMessage, send_stt_message
from core.utils.util import remove_punctuation_and_length, opus_datas_to_wav_bytes
from core.providers.tts.dto.dto import ContentType, SentenceType
from core.handle.mcpHandle import (
from core.providers.tools.device_mcp import (
MCPClient,
send_mcp_initialize_message,
send_mcp_tools_list_request,
main/xiaozhi-server/core/handle/intentHandler.py
+16 -32
View File
@@ -6,7 +6,7 @@ from core.handle.helloHandle import checkWakeupWords
from core.utils.util import remove_punctuation_and_length
from core.providers.tts.dto.dto import ContentType
from core.utils.dialogue import Message
from core.handle.mcpHandle import call_mcp_tool
from core.providers.tools.device_mcp import call_mcp_tool
from plugins_func.register import Action, ActionResponse
from loguru import logger
@@ -84,9 +84,11 @@ async def process_intent_result(conn, intent_result, original_text):
if not funcItem:
conn.func_handler.function_registry.register_function("play_music")
function_args = None
function_args = {}
if "arguments" in intent_data["function_call"]:
function_args = intent_data["function_call"]["arguments"]
if function_args is None:
function_args = {}
# 确保参数是字符串格式的JSON
if isinstance(function_args, dict):
function_args = json.dumps(function_args)
@@ -104,36 +106,18 @@ async def process_intent_result(conn, intent_result, original_text):
def process_function_call():
conn.dialogue.put(Message(role="user", content=original_text))
# 处理Server端MCP工具调用
if conn.mcp_manager.is_mcp_tool(function_name):
result = conn._handle_mcp_tool_call(function_call_data)
elif hasattr(conn, "mcp_client") and conn.mcp_client.has_tool(
function_name
):
# 如果是小智端MCP工具调用
conn.logger.bind(tag=TAG).debug(
f"调用小智端MCP工具: {function_name}, 参数: {function_args}"
)
try:
result = asyncio.run_coroutine_threadsafe(
call_mcp_tool(
conn, conn.mcp_client, function_name, function_args
),
conn.loop,
).result()
conn.logger.bind(tag=TAG).debug(f"MCP工具调用结果: {result}")
result = ActionResponse(
action=Action.REQLLM, result=result, response=""
)
except Exception as e:
conn.logger.bind(tag=TAG).error(f"MCP工具调用失败: {e}")
result = ActionResponse(
action=Action.REQLLM, result="MCP工具调用失败", response=""
)
else:
# 处理系统函数
result = conn.func_handler.handle_llm_function_call(
conn, function_call_data
# 使用统一工具处理器处理所有工具调用
try:
result = asyncio.run_coroutine_threadsafe(
conn.func_handler.handle_llm_function_call(
conn, function_call_data
),
conn.loop,
).result()
except Exception as e:
conn.logger.bind(tag=TAG).error(f"工具调用失败: {e}")
result = ActionResponse(
action=Action.ERROR, result=str(e), response=str(e)
)
if result:
main/xiaozhi-server/core/handle/iotHandle.py
-422
View File
@@ -1,422 +0,0 @@
import json
import asyncio
from plugins_func.register import (
FunctionItem,
register_device_function,
ActionResponse,
Action,
ToolType,
)
TAG = __name__
def wrap_async_function(async_func):
"""包装异步函数为同步函数"""
def wrapper(*args, **kwargs):
try:
# 获取连接对象（第一个参数）
conn = args[0]
if not hasattr(conn, "loop"):
conn.logger.bind(tag=TAG).error("Connection对象没有loop属性")
return ActionResponse(
Action.ERROR,
"Connection对象没有loop属性",
"执行操作时出错: Connection对象没有loop属性",
)
# 使用conn对象中的事件循环
loop = conn.loop
# 在conn的事件循环中运行异步函数
future = asyncio.run_coroutine_threadsafe(async_func(*args, **kwargs), loop)
# 等待结果返回
return future.result()
except Exception as e:
conn.logger.bind(tag=TAG).error(f"运行异步函数时出错: {e}")
return ActionResponse(Action.ERROR, str(e), f"执行操作时出错: {e}")
return wrapper
def create_iot_function(device_name, method_name, method_info):
"""
根据IOT设备描述生成通用的控制函数
"""
async def iot_control_function(
conn, response_success=None, response_failure=None, **params
):
try:
# 设置默认响应消息
if not response_success:
response_success = "操作成功"
if not response_failure:
response_failure = "操作失败"
# 打印响应参数
conn.logger.bind(tag=TAG).debug(
f"控制函数接收到的响应参数: success='{response_success}', failure='{response_failure}'"
)
# 发送控制命令
await send_iot_conn(conn, device_name, method_name, params)
# 等待一小段时间让状态更新
await asyncio.sleep(0.1)
# 生成结果信息
result = f"{device_name}的{method_name}操作执行成功"
# 处理响应中可能的占位符
response = response_success
# 替换{value}占位符
for param_name, param_value in params.items():
# 先尝试直接替换参数值
if "{" + param_name + "}" in response:
response = response.replace(
"{" + param_name + "}", str(param_value)
)
# 如果有{value}占位符，用相关参数替换
if "{value}" in response:
response = response.replace("{value}", str(param_value))
break
return ActionResponse(Action.RESPONSE, result, response)
except Exception as e:
conn.logger.bind(tag=TAG).error(
f"执行{device_name}的{method_name}操作失败: {e}"
)
# 操作失败时使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, str(e), response)
return wrap_async_function(iot_control_function)
def create_iot_query_function(device_name, prop_name, prop_info):
"""
根据IOT设备属性创建查询函数
"""
async def iot_query_function(conn, response_success=None, response_failure=None):
try:
# 打印响应参数
conn.logger.bind(tag=TAG).info(
f"查询函数接收到的响应参数: success='{response_success}', failure='{response_failure}'"
)
value = await get_iot_status(conn, device_name, prop_name)
# 查询成功，生成结果
if value is not None:
# 使用大模型提供的成功响应，并替换其中的占位符
response = response_success.replace("{value}", str(value))
return ActionResponse(Action.RESPONSE, str(value), response)
else:
# 查询失败，使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, f"属性{prop_name}不存在", response)
except Exception as e:
conn.logger.bind(tag=TAG).error(
f"查询{device_name}的{prop_name}时出错: {e}"
)
# 查询出错时使用大模型提供的失败响应
response = response_failure
return ActionResponse(Action.ERROR, str(e), response)
return wrap_async_function(iot_query_function)
class IotDescriptor:
"""
A class to represent an IoT descriptor.
"""
def __init__(self, name, description, properties, methods):
self.name = name
self.description = description
self.properties = []
self.methods = []
# 根据描述创建属性
if properties is not None:
for key, value in properties.items():
property_item = {}
property_item["name"] = key
property_item["description"] = value["description"]
if value["type"] == "number":
property_item["value"] = 0
elif value["type"] == "boolean":
property_item["value"] = False
else:
property_item["value"] = ""
self.properties.append(property_item)
# 根据描述创建方法
if methods is not None:
for key, value in methods.items():
method = {}
method["description"] = value["description"]
method["name"] = key
# 检查方法是否有参数
if "parameters" in value:
method["parameters"] = {}
for k, v in value["parameters"].items():
method["parameters"][k] = {
"description": v["description"],
"type": v["type"],
}
self.methods.append(method)
def register_device_type(descriptor, device_type_registry):
"""注册设备类型及其功能"""
device_name = descriptor["name"]
type_id = device_type_registry.generate_device_type_id(descriptor)
# 如果该类型已注册，直接返回类型ID
if type_id in device_type_registry.type_functions:
return type_id
functions = {}
# 为每个属性创建查询函数
for prop_name, prop_info in descriptor["properties"].items():
func_name = f"get_{device_name.lower()}_{prop_name.lower()}"
func_desc = {
"type": "function",
"function": {
"name": func_name,
"description": f"查询{descriptor['description']}的{prop_info['description']}",
"parameters": {
"type": "object",
"properties": {
"response_success": {
"type": "string",
"description": f"查询成功时的友好回复，必须使用{{value}}作为占位符表示查询到的值",
},
"response_failure": {
"type": "string",
"description": f"查询失败时的友好回复，例如：'无法获取{device_name}的{prop_info['description']}'",
},
},
"required": ["response_success", "response_failure"],
},
},
}
query_func = create_iot_query_function(device_name, prop_name, prop_info)
decorated_func = register_device_function(
func_name, func_desc, ToolType.IOT_CTL
)(query_func)
functions[func_name] = FunctionItem(
func_name, func_desc, decorated_func, ToolType.IOT_CTL
)
# 为每个方法创建控制函数
for method_name, method_info in descriptor["methods"].items():
func_name = f"{device_name.lower()}_{method_name.lower()}"
# 创建参数字典，添加原有参数
parameters = {}
required_params = []
# 如果方法有参数，则添加参数信息
if "parameters" in method_info:
parameters = {
param_name: {
"type": param_info["type"],
"description": param_info["description"],
}
for param_name, param_info in method_info["parameters"].items()
}
required_params = list(method_info["parameters"].keys())
# 添加响应参数
parameters.update(
{
"response_success": {
"type": "string",
"description": "操作成功时的友好回复,关于该设备的操作结果，设备名称尽量使用description中的名称",
},
"response_failure": {
"type": "string",
"description": "操作失败时的友好回复,关于该设备的操作结果，设备名称尽量使用description中的名称",
},
}
)
# 构建必须参数列表（原有参数 + 响应参数）
required_params.extend(["response_success", "response_failure"])
func_desc = {
"type": "function",
"function": {
"name": func_name,
"description": f"{descriptor['description']} - {method_info['description']}",
"parameters": {
"type": "object",
"properties": parameters,
"required": required_params,
},
},
}
control_func = create_iot_function(device_name, method_name, method_info)
decorated_func = register_device_function(
func_name, func_desc, ToolType.IOT_CTL
)(control_func)
functions[func_name] = FunctionItem(
func_name, func_desc, decorated_func, ToolType.IOT_CTL
)
device_type_registry.register_device_type(type_id, functions)
return type_id
# 用于接受前端设备推送的搜索iot描述
async def handleIotDescriptors(conn, descriptors):
wait_max_time = 5
while conn.func_handler is None or not conn.func_handler.finish_init:
await asyncio.sleep(1)
wait_max_time -= 1
if wait_max_time <= 0:
conn.logger.bind(tag=TAG).debug("连接对象没有func_handler")
return
"""处理物联网描述"""
functions_changed = False
for descriptor in descriptors:
# 如果descriptor没有properties和methods，则直接跳过
if "properties" not in descriptor and "methods" not in descriptor:
continue
# 处理缺失properties的情况
if "properties" not in descriptor:
descriptor["properties"] = {}
# 从methods中提取所有参数作为properties
if "methods" in descriptor:
for method_name, method_info in descriptor["methods"].items():
if "parameters" in method_info:
for param_name, param_info in method_info["parameters"].items():
# 将参数信息转换为属性信息
descriptor["properties"][param_name] = {
"description": param_info["description"],
"type": param_info["type"],
}
# 创建IOT设备描述符
iot_descriptor = IotDescriptor(
descriptor["name"],
descriptor["description"],
descriptor["properties"],
descriptor["methods"],
)
conn.iot_descriptors[descriptor["name"]] = iot_descriptor
if conn.load_function_plugin:
# 注册或获取设备类型
device_type_registry = conn.func_handler.device_type_registry
type_id = register_device_type(descriptor, device_type_registry)
device_functions = device_type_registry.get_device_functions(type_id)
# 在连接级注册设备函数
if hasattr(conn, "func_handler"):
for func_name, func_item in device_functions.items():
conn.func_handler.function_registry.register_function(
func_name, func_item
)
conn.logger.bind(tag=TAG).info(
f"注册IOT函数到function handler: {func_name}"
)
functions_changed = True
# 如果注册了新函数，更新function描述列表
if functions_changed and hasattr(conn, "func_handler"):
conn.func_handler.upload_functions_desc()
func_names = conn.func_handler.current_support_functions()
conn.logger.bind(tag=TAG).info(f"设备类型: {type_id}")
conn.logger.bind(tag=TAG).info(
f"更新function描述列表完成，当前支持的函数: {func_names}"
)
async def handleIotStatus(conn, states):
"""处理物联网状态"""
for state in states:
for key, value in conn.iot_descriptors.items():
if key == state["name"]:
for property_item in value.properties:
for k, v in state["state"].items():
if property_item["name"] == k:
if type(v) != type(property_item["value"]):
conn.logger.bind(tag=TAG).error(
f"属性{property_item['name']}的值类型不匹配"
)
break
else:
property_item["value"] = v
conn.logger.bind(tag=TAG).info(
f"物联网状态更新: {key} , {property_item['name']} = {v}"
)
break
break
async def get_iot_status(conn, name, property_name):
"""获取物联网状态"""
for key, value in conn.iot_descriptors.items():
if key == name:
for property_item in value.properties:
if property_item["name"] == property_name:
return property_item["value"]
conn.logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
return None
async def set_iot_status(conn, name, property_name, value):
"""设置物联网状态"""
for key, iot_descriptor in conn.iot_descriptors.items():
if key == name:
for property_item in iot_descriptor.properties:
if property_item["name"] == property_name:
if type(value) != type(property_item["value"]):
conn.logger.bind(tag=TAG).error(
f"属性{property_item['name']}的值类型不匹配"
)
return
property_item["value"] = value
conn.logger.bind(tag=TAG).info(
f"物联网状态更新: {name} , {property_name} = {value}"
)
return
conn.logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
async def send_iot_conn(conn, name, method_name, parameters):
"""发送物联网指令"""
for key, value in conn.iot_descriptors.items():
if key == name:
# 找到了设备
for method in value.methods:
# 找到了方法
if method["name"] == method_name:
# 构建命令对象
command = {
"name": name,
"method": method_name,
}
# 只有当参数不为空时才添加parameters字段
if parameters:
command["parameters"] = parameters
send_message = json.dumps({"type": "iot", "commands": [command]})
await conn.websocket.send(send_message)
conn.logger.bind(tag=TAG).info(f"发送物联网指令: {send_message}")
return
conn.logger.bind(tag=TAG).error(f"未找到方法{method_name}")
main/xiaozhi-server/core/handle/receiveAudioHandle.py
+4 -5
View File
@@ -66,12 +66,11 @@ async def startToChat(conn, text):
async def no_voice_close_connect(conn, have_voice):
if have_voice:
conn.client_no_voice_last_time = 0.0
conn.last_activity_time = time.time() * 1000
return
if conn.client_no_voice_last_time == 0.0:
conn.client_no_voice_last_time = time.time() * 1000
else:
no_voice_time = time.time() * 1000 - conn.client_no_voice_last_time
# 只有在已经初始化过时间戳的情况下才进行超时检查
if conn.last_activity_time > 0.0:
no_voice_time = time.time() * 1000 - conn.last_activity_time
close_connection_no_voice_time = int(
conn.config.get("close_connection_no_voice_time", 120)
)
main/xiaozhi-server/core/handle/sendAudioHandle.py
+2 -4
View File
@@ -92,10 +92,8 @@ async def sendAudio(conn, audios, pre_buffer=True):
if conn.client_abort:
break
# 每分钟重置一次计时器
if time.perf_counter() - last_reset_time > 60:
await conn.reset_timeout()
last_reset_time = time.perf_counter()
# 重置没有声音的状态
conn.last_activity_time = time.time() * 1000
# 计算预期发送时间
expected_time = start_time + (play_position / 1000)
main/xiaozhi-server/core/handle/textHandle.py
+3 -3
View File
@@ -1,11 +1,11 @@
import json
from core.handle.abortHandle import handleAbortMessage
from core.handle.helloHandle import handleHelloMessage
from core.handle.mcpHandle import handle_mcp_message
from core.providers.tools.device_mcp import handle_mcp_message
from core.utils.util import remove_punctuation_and_length, filter_sensitive_info
from core.handle.receiveAudioHandle import startToChat, handleAudioMessage
from core.handle.sendAudioHandle import send_stt_message, send_tts_message
from core.handle.iotHandle import handleIotDescriptors, handleIotStatus
from core.providers.tools.device_iot import handleIotDescriptors, handleIotStatus
from core.handle.reportHandle import enqueue_asr_report
import asyncio
@@ -77,7 +77,7 @@ async def handleTextMessage(conn, message):
if "states" in msg_json:
asyncio.create_task(handleIotStatus(conn, msg_json["states"]))
elif msg_json["type"] == "mcp":
conn.logger.bind(tag=TAG).info(f"收到mcp消息：{message}")
conn.logger.bind(tag=TAG).info(f"收到mcp消息：{message[:100]}")
if "payload" in msg_json:
asyncio.create_task(
handle_mcp_message(conn, conn.mcp_client, msg_json["payload"])
main/xiaozhi-server/core/mcp/manager.py
-131
View File
@@ -1,131 +0,0 @@
"""MCP服务管理器"""
import asyncio
import os, json
from typing import Dict, Any, List
from .MCPClient import MCPClient
from plugins_func.register import register_function, ToolType
from config.config_loader import get_project_dir
TAG = __name__
class MCPManager:
"""管理多个MCP服务的集中管理器"""
def __init__(self, conn) -> None:
"""
初始化MCP管理器
"""
self.conn = conn
self.config_path = get_project_dir() + "data/.mcp_server_settings.json"
if os.path.exists(self.config_path) == False:
self.config_path = ""
self.conn.logger.bind(tag=TAG).warning(
f"请检查mcp服务配置文件：data/.mcp_server_settings.json"
)
self.client: Dict[str, MCPClient] = {}
self.tools = []
def load_config(self) -> Dict[str, Any]:
"""加载MCP服务配置
Returns:
Dict[str, Any]: 服务配置字典
"""
if len(self.config_path) == 0:
return {}
try:
with open(self.config_path, "r", encoding="utf-8") as f:
config = json.load(f)
return config.get("mcpServers", {})
except Exception as e:
self.conn.logger.bind(tag=TAG).error(
f"Error loading MCP config from {self.config_path}: {e}"
)
return {}
async def initialize_servers(self) -> None:
"""初始化所有MCP服务"""
config = self.load_config()
for name, srv_config in config.items():
if not srv_config.get("command") and not srv_config.get("url"):
self.conn.logger.bind(tag=TAG).warning(
f"Skipping server {name}: neither command nor url specified"
)
continue
try:
client = MCPClient(srv_config)
await client.initialize()
self.client[name] = client
self.conn.logger.bind(tag=TAG).info(f"Initialized MCP client: {name}")
client_tools = client.get_available_tools()
self.tools.extend(client_tools)
for tool in client_tools:
func_name = "mcp_" + tool["function"]["name"]
register_function(func_name, tool, ToolType.MCP_CLIENT)(
self.execute_tool
)
self.conn.func_handler.function_registry.register_function(
func_name
)
except Exception as e:
self.conn.logger.bind(tag=TAG).error(
f"Failed to initialize MCP server {name}: {e}"
)
self.conn.func_handler.upload_functions_desc()
def get_all_tools(self) -> List[Dict[str, Any]]:
"""获取所有服务的工具function定义
Returns:
List[Dict[str, Any]]: 所有工具的function定义列表
"""
return self.tools
def is_mcp_tool(self, tool_name: str) -> bool:
"""检查是否是MCP工具
Args:
tool_name: 工具名称
Returns:
bool: 是否是MCP工具
"""
for tool in self.tools:
if (
tool.get("function") != None
and tool["function"].get("name") == tool_name
):
return True
return False
async def execute_tool(self, tool_name: str, arguments: Dict[str, Any]) -> Any:
"""执行工具调用
Args:
tool_name: 工具名称
arguments: 工具参数
Returns:
Any: 工具执行结果
Raises:
ValueError: 工具未找到时抛出
"""
self.conn.logger.bind(tag=TAG).info(
f"Executing tool {tool_name} with arguments: {arguments}"
)
for client in self.client.values():
if client.has_tool(tool_name):
return await client.call_tool(tool_name, arguments)
raise ValueError(f"Tool {tool_name} not found in any MCP server")
async def cleanup_all(self) -> None:
"""依次关闭所有 MCPClient，不让异常阻断整体流程。"""
for name, client in list(self.client.items()):
try:
await asyncio.wait_for(client.cleanup(), timeout=20)
self.conn.logger.bind(tag=TAG).info(f"MCP client closed: {name}")
except (asyncio.TimeoutError, Exception) as e:
self.conn.logger.bind(tag=TAG).error(
f"Error closing MCP client {name}: {e}"
)
self.client.clear()
main/xiaozhi-server/core/providers/asr/doubao_stream.py
+9 -7
View File
@@ -93,9 +93,7 @@ class ASRProvider(ASRProviderBase):
# 检查初始化响应
if "code" in result and result["code"] != 1000:
error_msg = f"ASR服务初始化失败: {result.get('payload_msg', {}).get('message', '未知错误')}"
if "payload_msg" in result:
error_msg += f"\n详细错误信息: {json.dumps(result['payload_msg'], ensure_ascii=False)}"
error_msg = f"ASR服务初始化失败: {result.get('payload_msg', {}).get('error', '未知错误')}"
logger.bind(tag=TAG).error(error_msg)
raise Exception(error_msg)
@@ -256,7 +254,6 @@ class ASRProvider(ASRProviderBase):
"X-Api-Access-Key": self.access_token,
"X-Api-Resource-Id": "volc.bigasr.sauc.duration",
"X-Api-Connect-Id": str(uuid.uuid4()),
"Host": "openspeech.bytedance.com",
}
def generate_header(
@@ -309,9 +306,14 @@ class ASRProvider(ASRProviderBase):
# 如果是错误响应
if message_type == 0x0F: # SERVER_ERROR_RESPONSE
code = int.from_bytes(header[4:8], "big", signed=False)
error_msg = res[8:].decode("utf-8")
return {"code": code, "error": error_msg}
code = int.from_bytes(res[4:8], "big", signed=False)
msg_length = int.from_bytes(res[8:12], "big", signed=False)
error_msg = json.loads(res[12:].decode("utf-8"))
return {
"code": code,
"msg_length": msg_length,
"payload_msg": error_msg,
}
# 获取JSON数据（跳过12字节头部）
try:
main/xiaozhi-server/core/providers/intent/intent_llm/intent_llm.py
+17 -1
View File
@@ -76,6 +76,14 @@ class IntentProvider(IntentProviderBase):
'返回: {"function_call": {"name": "get_battery_level", "arguments": {"response_success": "当前电池电量为{value}%", "response_failure": "无法获取Battery的当前电量百分比"}}}\n'
"```\n"
"```\n"
"用户: 当前屏幕亮度是多少？\n"
'返回: {"function_call": {"name": "self_screen_get_brightness"}}\n'
"```\n"
"```\n"
"用户: 设置屏幕亮度为50%\n"
'返回: {"function_call": {"name": "self_screen_set_brightness", "arguments": {"brightness": 50}}}\n'
"```\n"
"```\n"
"用户: 我想结束对话\n"
'返回: {"function_call": {"name": "handle_exit_intent", "arguments": {"say_goodbye": "goodbye"}}}\n'
"```\n"
@@ -87,6 +95,10 @@ class IntentProvider(IntentProviderBase):
"1. 只返回JSON格式，不要包含任何其他文字\n"
'2. 如果没有找到匹配的函数，返回{"function_call": {"name": "continue_chat"}}\n'
"3. 确保返回的JSON格式正确，包含所有必要的字段\n"
"特殊说明：\n"
"- 当用户单次输入包含多个指令时（如'打开灯并且调高音量'）\n"
"- 请返回多个function_call组成的JSON数组\n"
"- 示例：{'function_calls': [{name:'light_on'}, {name:'volume_up'}]}"
)
return prompt
@@ -164,7 +176,11 @@ class IntentProvider(IntentProviderBase):
music_file_names = music_config["music_file_names"]
prompt_music = f"{self.promot}\n<musicNames>{music_file_names}\n</musicNames>"
devices = conn.config["plugins"]["home_assistant"].get("devices", [])
home_assistant_cfg = conn.config["plugins"].get("home_assistant")
if home_assistant_cfg:
devices = home_assistant_cfg.get("devices", [])
else:
devices = []
if len(devices) > 0:
hass_prompt = "\n下面是我家智能设备列表（位置，设备名，entity_id），可以通过homeassistant控制\n"
for device in devices:
main/xiaozhi-server/core/providers/llm/coze/coze.py
+3 -1
View File
@@ -23,7 +23,9 @@ class LLMProvider(LLMProviderBase):
self.bot_id = str(config.get("bot_id"))
self.user_id = str(config.get("user_id"))
self.session_conversation_map = {} # 存储session_id和conversation_id的映射
check_model_key("CozeLLM", self.personal_access_token)
model_key_msg = check_model_key("CozeLLM", self.personal_access_token)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
def response(self, session_id, dialogue, **kwargs):
coze_api_token = self.personal_access_token
main/xiaozhi-server/core/providers/llm/dify/dify.py
+3 -1
View File
@@ -15,7 +15,9 @@ class LLMProvider(LLMProviderBase):
self.mode = config.get("mode", "chat-messages")
self.base_url = config.get("base_url", "https://api.dify.ai/v1").rstrip("/")
self.session_conversation_map = {} # 存储session_id和conversation_id的映射
check_model_key("DifyLLM", self.api_key)
model_key_msg = check_model_key("DifyLLM", self.api_key)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
def response(self, session_id, dialogue, **kwargs):
try:
main/xiaozhi-server/core/providers/llm/fastgpt/fastgpt.py
+3 -1
View File
@@ -14,7 +14,9 @@ class LLMProvider(LLMProviderBase):
self.base_url = config.get("base_url")
self.detail = config.get("detail", False)
self.variables = config.get("variables", {})
check_model_key("FastGPTLLM", self.api_key)
model_key_msg = check_model_key("FastGPTLLM", self.api_key)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
def response(self, session_id, dialogue, **kwargs):
try:
main/xiaozhi-server/core/providers/llm/gemini/gemini.py
+3 -2
View File
@@ -73,8 +73,9 @@ class LLMProvider(LLMProviderBase):
http_proxy = cfg.get("http_proxy")
https_proxy = cfg.get("https_proxy")
if not check_model_key("LLM", self.api_key):
raise ValueError("无效的Gemini API Key，请检查是否配置正确")
model_key_msg = check_model_key("LLM", self.api_key)
if model_key_msg:
log.bind(tag=TAG).error(model_key_msg)
if http_proxy or https_proxy:
log.bind(tag=TAG).info(
main/xiaozhi-server/core/providers/llm/openai/openai.py
+25 -10
View File
@@ -1,3 +1,4 @@
import httpx
import openai
from openai.types import CompletionUsage
from config.logger import setup_logging
@@ -16,26 +17,36 @@ class LLMProvider(LLMProviderBase):
self.base_url = config.get("base_url")
else:
self.base_url = config.get("url")
# 增加timeout的配置项，单位为秒
timeout = config.get("timeout", 300)
self.timeout = int(timeout) if timeout else 300
param_defaults = {
"max_tokens": (500, int),
"temperature": (0.7, lambda x: round(float(x), 1)),
"top_p": (1.0, lambda x: round(float(x), 1)),
"frequency_penalty": (0, lambda x: round(float(x), 1))
"frequency_penalty": (0, lambda x: round(float(x), 1)),
}
for param, (default, converter) in param_defaults.items():
value = config.get(param)
try:
setattr(self, param, converter(value) if value not in (None, "") else default)
setattr(
self,
param,
converter(value) if value not in (None, "") else default,
)
except (ValueError, TypeError):
setattr(self, param, default)
logger.debug(
f"意图识别参数初始化: {self.temperature}, {self.max_tokens}, {self.top_p}, {self.frequency_penalty}")
f"意图识别参数初始化: {self.temperature}, {self.max_tokens}, {self.top_p}, {self.frequency_penalty}"
)
check_model_key("LLM", self.api_key)
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url)
model_key_msg = check_model_key("LLM", self.api_key)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url, timeout=httpx.Timeout(self.timeout))
def response(self, session_id, dialogue, **kwargs):
try:
@@ -46,7 +57,9 @@ class LLMProvider(LLMProviderBase):
max_tokens=kwargs.get("max_tokens", self.max_tokens),
temperature=kwargs.get("temperature", self.temperature),
top_p=kwargs.get("top_p", self.top_p),
frequency_penalty=kwargs.get("frequency_penalty", self.frequency_penalty),
frequency_penalty=kwargs.get(
"frequency_penalty", self.frequency_penalty
),
)
is_active = True
@@ -84,12 +97,14 @@ class LLMProvider(LLMProviderBase):
for chunk in stream:
# 检查是否存在有效的choice且content不为空
if getattr(chunk, "choices", None):
yield chunk.choices[0].delta.content, chunk.choices[0].delta.tool_calls
yield chunk.choices[0].delta.content, chunk.choices[
0
].delta.tool_calls
# 存在 CompletionUsage 消息时，生成 Token 消耗 log
elif isinstance(getattr(chunk, 'usage', None), CompletionUsage):
usage_info = getattr(chunk, 'usage', None)
elif isinstance(getattr(chunk, "usage", None), CompletionUsage):
usage_info = getattr(chunk, "usage", None)
logger.bind(tag=TAG).info(
f"Token 消耗：输入 {getattr(usage_info, 'prompt_tokens', '未知')}，"
f"Token 消耗：输入 {getattr(usage_info, 'prompt_tokens', '未知')}，"
f"输出 {getattr(usage_info, 'completion_tokens', '未知')}，"
f"共计 {getattr(usage_info, 'total_tokens', '未知')}"
)
main/xiaozhi-server/core/providers/memory/base.py
-4
View File
@@ -12,10 +12,6 @@ class MemoryProviderBase(ABC):
def set_llm(self, llm):
self.llm = llm
# 获取模型名称和类型信息
model_name = getattr(llm, "model_name", str(llm.__class__.__name__))
# 记录更详细的日志
logger.bind(tag=TAG).info(f"记忆总结设置LLM: {model_name}")
@abstractmethod
async def save_memory(self, msgs):
main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py
+4 -2
View File
@@ -12,12 +12,14 @@ class MemoryProvider(MemoryProviderBase):
super().__init__(config)
self.api_key = config.get("api_key", "")
self.api_version = config.get("api_version", "v1.1")
have_key = check_model_key("Mem0ai", self.api_key)
if not have_key:
model_key_msg = check_model_key("Mem0ai", self.api_key)
if model_key_msg:
logger.bind(tag=TAG).error(model_key_msg)
self.use_mem0 = False
return
else:
self.use_mem0 = True
try:
self.client = MemoryClient(api_key=self.api_key)
logger.bind(tag=TAG).info("成功连接到 Mem0ai 服务")
main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py
+5
View File
@@ -5,6 +5,7 @@ import os
import yaml
from config.config_loader import get_project_dir
from config.manage_api_client import save_mem_local_short
from core.utils.util import check_model_key
short_term_memory_prompt = """
@@ -145,6 +146,10 @@ class MemoryProvider(MemoryProviderBase):
# 打印使用的模型信息
model_info = getattr(self.llm, "model_name", str(self.llm.__class__.__name__))
logger.bind(tag=TAG).debug(f"使用记忆保存模型: {model_info}")
api_key = getattr(self.llm, "api_key", None)
memory_key_msg = check_model_key("记忆总结专用LLM", api_key)
if memory_key_msg:
logger.bind(tag=TAG).error(memory_key_msg)
if self.llm is None:
logger.bind(tag=TAG).error("LLM is not set for memory provider")
return None
main/xiaozhi-server/core/providers/tools/__init__.py
+1
View File
@@ -0,0 +1 @@
main/xiaozhi-server/core/providers/tools/base/__init__.py
+6
View File
@@ -0,0 +1,6 @@
"""基础工具定义模块"""
from .tool_types import ToolType, ToolDefinition
from .tool_executor import ToolExecutor
__all__ = ["ToolType", "ToolDefinition", "ToolExecutor"]
main/xiaozhi-server/core/providers/tools/base/tool_executor.py
+27
View File
@@ -0,0 +1,27 @@
"""工具执行器基类定义"""
from abc import ABC, abstractmethod
from typing import Dict, Any
from .tool_types import ToolDefinition
from plugins_func.register import ActionResponse
class ToolExecutor(ABC):
"""工具执行器抽象基类"""
@abstractmethod
async def execute(
self, conn, tool_name: str, arguments: Dict[str, Any]
) -> ActionResponse:
"""执行工具调用"""
pass
@abstractmethod
def get_tools(self) -> Dict[str, ToolDefinition]:
"""获取该执行器管理的所有工具"""
pass
@abstractmethod
def has_tool(self, tool_name: str) -> bool:
"""检查是否有指定工具"""
pass
main/xiaozhi-server/core/providers/tools/base/tool_types.py
+27
View File
@@ -0,0 +1,27 @@
"""工具系统的类型定义"""
from enum import Enum
from dataclasses import dataclass
from typing import Any, Dict, Optional
from plugins_func.register import Action
class ToolType(Enum):
"""工具类型枚举"""
SERVER_PLUGIN = "server_plugin" # 服务端插件
SERVER_MCP = "server_mcp" # 服务端MCP
DEVICE_IOT = "device_iot" # 设备端IoT
DEVICE_MCP = "device_mcp" # 设备端MCP
MCP_ENDPOINT = "mcp_endpoint" # MCP接入点
@dataclass
class ToolDefinition:
"""工具定义"""
name: str # 工具名称
description: Dict[str, Any] # 工具描述（OpenAI函数调用格式）
tool_type: ToolType # 工具类型
parameters: Optional[Dict[str, Any]] = None # 额外参数
main/xiaozhi-server/core/providers/tools/device_iot/__init__.py
+12
View File
@@ -0,0 +1,12 @@
"""设备端IoT工具模块"""
from .iot_descriptor import IotDescriptor
from .iot_handler import handleIotDescriptors, handleIotStatus
from .iot_executor import DeviceIoTExecutor
__all__ = [
"IotDescriptor",
"handleIotDescriptors",
"handleIotStatus",
"DeviceIoTExecutor",
]
main/xiaozhi-server/core/providers/tools/device_iot/iot_descriptor.py
+46
View File
@@ -0,0 +1,46 @@
"""IoT设备描述符定义"""
from config.logger import setup_logging
TAG = __name__
logger = setup_logging()
class IotDescriptor:
"""IoT设备描述符"""
def __init__(self, name, description, properties, methods):
self.name = name
self.description = description
self.properties = []
self.methods = []
# 根据描述创建属性
if properties is not None:
for key, value in properties.items():
property_item = {}
property_item["name"] = key
property_item["description"] = value["description"]
if value["type"] == "number":
property_item["value"] = 0
elif value["type"] == "boolean":
property_item["value"] = False
else:
property_item["value"] = ""
self.properties.append(property_item)
# 根据描述创建方法
if methods is not None:
for key, value in methods.items():
method = {}
method["description"] = value["description"]
method["name"] = key
# 检查方法是否有参数
if "parameters" in value:
method["parameters"] = {}
for k, v in value["parameters"].items():
method["parameters"][k] = {
"description": v["description"],
"type": v["type"],
}
self.methods.append(method)
main/xiaozhi-server/core/providers/tools/device_iot/iot_executor.py
+238
View File
@@ -0,0 +1,238 @@
"""设备端IoT工具执行器"""
import json
import asyncio
from typing import Dict, Any
from ..base import ToolType, ToolDefinition, ToolExecutor
from plugins_func.register import Action, ActionResponse
class DeviceIoTExecutor(ToolExecutor):
"""设备端IoT工具执行器"""
def __init__(self, conn):
self.conn = conn
self.iot_tools: Dict[str, ToolDefinition] = {}
async def execute(
self, conn, tool_name: str, arguments: Dict[str, Any]
) -> ActionResponse:
"""执行设备端IoT工具"""
if not self.has_tool(tool_name):
return ActionResponse(
action=Action.NOTFOUND, response=f"IoT工具 {tool_name} 不存在"
)
try:
# 解析工具名称，获取设备名和操作类型
if tool_name.startswith("get_"):
# 查询操作：get_devicename_property
parts = tool_name.split("_", 2)
if len(parts) >= 3:
device_name = parts[1]
property_name = parts[2]
value = await self._get_iot_status(device_name, property_name)
if value is not None:
# 处理响应模板
response_success = arguments.get(
"response_success", "查询成功：{value}"
)
response = response_success.replace("{value}", str(value))
return ActionResponse(
action=Action.RESPONSE,
response=response,
)
else:
response_failure = arguments.get(
"response_failure", f"无法获取{device_name}的状态"
)
return ActionResponse(
action=Action.ERROR, response=response_failure
)
else:
# 控制操作：devicename_method
parts = tool_name.split("_", 1)
if len(parts) >= 2:
device_name = parts[0]
method_name = parts[1]
# 提取控制参数（排除响应参数）
control_params = {
k: v
for k, v in arguments.items()
if k not in ["response_success", "response_failure"]
}
# 发送IoT控制命令
await self._send_iot_command(
device_name, method_name, control_params
)
# 等待状态更新
await asyncio.sleep(0.1)
response_success = arguments.get("response_success", "操作成功")
# 处理响应中的占位符
for param_name, param_value in control_params.items():
placeholder = "{" + param_name + "}"
if placeholder in response_success:
response_success = response_success.replace(
placeholder, str(param_value)
)
if "{value}" in response_success:
response_success = response_success.replace(
"{value}", str(param_value)
)
break
return ActionResponse(
action=Action.REQLLM,
result=response_success,
)
return ActionResponse(action=Action.ERROR, response="无法解析IoT工具名称")
except Exception as e:
response_failure = arguments.get("response_failure", "操作失败")
return ActionResponse(action=Action.ERROR, response=response_failure)
async def _get_iot_status(self, device_name: str, property_name: str):
"""获取IoT设备状态"""
for key, value in self.conn.iot_descriptors.items():
if key.lower() == device_name.lower():
for property_item in value.properties:
if property_item["name"].lower() == property_name.lower():
return property_item["value"]
return None
async def _send_iot_command(
self, device_name: str, method_name: str, parameters: Dict[str, Any]
):
"""发送IoT控制命令"""
for key, value in self.conn.iot_descriptors.items():
if key.lower() == device_name.lower():
for method in value.methods:
if method["name"].lower() == method_name.lower():
command = {
"name": key,
"method": method["name"],
}
if parameters:
command["parameters"] = parameters
send_message = json.dumps(
{"type": "iot", "commands": [command]}
)
await self.conn.websocket.send(send_message)
return
raise Exception(f"未找到设备{device_name}的方法{method_name}")
def register_iot_tools(self, descriptors: list):
"""注册IoT工具"""
for descriptor in descriptors:
device_name = descriptor["name"]
device_desc = descriptor["description"]
# 注册查询工具
if "properties" in descriptor:
for prop_name, prop_info in descriptor["properties"].items():
tool_name = f"get_{device_name.lower()}_{prop_name.lower()}"
tool_desc = {
"type": "function",
"function": {
"name": tool_name,
"description": f"查询{device_desc}的{prop_info['description']}",
"parameters": {
"type": "object",
"properties": {
"response_success": {
"type": "string",
"description": f"查询成功时的友好回复，必须使用{{value}}作为占位符表示查询到的值",
},
"response_failure": {
"type": "string",
"description": f"查询失败时的友好回复",
},
},
"required": ["response_success", "response_failure"],
},
},
}
self.iot_tools[tool_name] = ToolDefinition(
name=tool_name,
description=tool_desc,
tool_type=ToolType.DEVICE_IOT,
)
# 注册控制工具
if "methods" in descriptor:
for method_name, method_info in descriptor["methods"].items():
tool_name = f"{device_name.lower()}_{method_name.lower()}"
# 构建参数
parameters = {}
required_params = []
# 添加方法的原始参数
if "parameters" in method_info:
parameters.update(
{
param_name: {
"type": param_info["type"],
"description": param_info["description"],
}
for param_name, param_info in method_info[
"parameters"
].items()
}
)
required_params.extend(method_info["parameters"].keys())
# 添加响应参数
parameters.update(
{
"response_success": {
"type": "string",
"description": "操作成功时的友好回复",
},
"response_failure": {
"type": "string",
"description": "操作失败时的友好回复",
},
}
)
required_params.extend(["response_success", "response_failure"])
tool_desc = {
"type": "function",
"function": {
"name": tool_name,
"description": f"{device_desc} - {method_info['description']}",
"parameters": {
"type": "object",
"properties": parameters,
"required": required_params,
},
},
}
self.iot_tools[tool_name] = ToolDefinition(
name=tool_name,
description=tool_desc,
tool_type=ToolType.DEVICE_IOT,
)
def get_tools(self) -> Dict[str, ToolDefinition]:
"""获取所有设备端IoT工具"""
return self.iot_tools.copy()
def has_tool(self, tool_name: str) -> bool:
"""检查是否有指定的设备端IoT工具"""
return tool_name in self.iot_tools
main/xiaozhi-server/core/providers/tools/device_iot/iot_handler.py
+83
View File
@@ -0,0 +1,83 @@
"""IoT设备支持模块，提供IoT设备描述符和状态处理"""
import asyncio
from config.logger import setup_logging
from .iot_descriptor import IotDescriptor
TAG = __name__
logger = setup_logging()
async def handleIotDescriptors(conn, descriptors):
"""处理物联网描述"""
wait_max_time = 5
while (
not hasattr(conn, "func_handler")
or conn.func_handler is None
or not conn.func_handler.finish_init
):
await asyncio.sleep(1)
wait_max_time -= 1
if wait_max_time <= 0:
logger.bind(tag=TAG).debug("连接对象没有func_handler")
return
functions_changed = False
for descriptor in descriptors:
# 如果descriptor没有properties和methods，则直接跳过
if "properties" not in descriptor and "methods" not in descriptor:
continue
# 处理缺失properties的情况
if "properties" not in descriptor:
descriptor["properties"] = {}
# 从methods中提取所有参数作为properties
if "methods" in descriptor:
for method_name, method_info in descriptor["methods"].items():
if "parameters" in method_info:
for param_name, param_info in method_info["parameters"].items():
# 将参数信息转换为属性信息
descriptor["properties"][param_name] = {
"description": param_info["description"],
"type": param_info["type"],
}
# 创建IOT设备描述符
iot_descriptor = IotDescriptor(
descriptor["name"],
descriptor["description"],
descriptor["properties"],
descriptor["methods"],
)
conn.iot_descriptors[descriptor["name"]] = iot_descriptor
functions_changed = True
# 如果注册了新函数，更新function描述列表
if functions_changed and hasattr(conn, "func_handler"):
# 注册IoT工具到统一工具处理器
await conn.func_handler.register_iot_tools(descriptors)
conn.func_handler.current_support_functions()
async def handleIotStatus(conn, states):
"""处理物联网状态"""
for state in states:
for key, value in conn.iot_descriptors.items():
if key == state["name"]:
for property_item in value.properties:
for k, v in state["state"].items():
if property_item["name"] == k:
if type(v) != type(property_item["value"]):
logger.bind(tag=TAG).error(
f"属性{property_item['name']}的值类型不匹配"
)
break
else:
property_item["value"] = v
logger.bind(tag=TAG).info(
f"物联网状态更新: {key} , {property_item['name']} = {v}"
)
break
break
main/xiaozhi-server/core/providers/tools/device_mcp/__init__.py
+21
View File
@@ -0,0 +1,21 @@
"""设备端MCP工具模块"""
from .mcp_client import MCPClient
from .mcp_handler import (
send_mcp_message,
handle_mcp_message,
send_mcp_initialize_message,
send_mcp_tools_list_request,
call_mcp_tool,
)
from .mcp_executor import DeviceMCPExecutor
__all__ = [
"MCPClient",
"send_mcp_message",
"handle_mcp_message",
"send_mcp_initialize_message",
"send_mcp_tools_list_request",
"call_mcp_tool",
"DeviceMCPExecutor",
]
main/xiaozhi-server/core/providers/tools/device_mcp/mcp_client.py
+93
View File
@@ -0,0 +1,93 @@
"""设备端MCP客户端定义"""
import asyncio
from concurrent.futures import Future
from core.utils.util import sanitize_tool_name
from config.logger import setup_logging
TAG = __name__
logger = setup_logging()
class MCPClient:
"""设备端MCP客户端，用于管理MCP状态和工具"""
def __init__(self):
self.tools = {} # sanitized_name -> tool_data
self.name_mapping = {}
self.ready = False
self.call_results = {} # To store Futures for tool call responses
self.next_id = 1
self.lock = asyncio.Lock()
self._cached_available_tools = None # Cache for get_available_tools
def has_tool(self, name: str) -> bool:
return name in self.tools
def get_available_tools(self) -> list:
# Check if the cache is valid
if self._cached_available_tools is not None:
return self._cached_available_tools
# If cache is not valid, regenerate the list
result = []
for tool_name, tool_data in self.tools.items():
function_def = {
"name": tool_name,
"description": tool_data["description"],
"parameters": {
"type": tool_data["inputSchema"].get("type", "object"),
"properties": tool_data["inputSchema"].get("properties", {}),
"required": tool_data["inputSchema"].get("required", []),
},
}
result.append({"type": "function", "function": function_def})
self._cached_available_tools = result # Store the generated list in cache
return result
async def is_ready(self) -> bool:
async with self.lock:
return self.ready
async def set_ready(self, status: bool):
async with self.lock:
self.ready = status
async def add_tool(self, tool_data: dict):
async with self.lock:
sanitized_name = sanitize_tool_name(tool_data["name"])
self.tools[sanitized_name] = tool_data
self.name_mapping[sanitized_name] = tool_data["name"]
self._cached_available_tools = (
None # Invalidate the cache when a tool is added
)
async def get_next_id(self) -> int:
async with self.lock:
current_id = self.next_id
self.next_id += 1
return current_id
async def register_call_result_future(self, id: int, future: Future):
async with self.lock:
self.call_results[id] = future
async def resolve_call_result(self, id: int, result: any):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_result(result)
async def reject_call_result(self, id: int, exception: Exception):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_exception(exception)
async def cleanup_call_result(self, id: int):
async with self.lock:
if id in self.call_results:
self.call_results.pop(id)
main/xiaozhi-server/core/providers/tools/device_mcp/mcp_executor.py
+89
View File
@@ -0,0 +1,89 @@
"""设备端MCP工具执行器"""
from typing import Dict, Any
from ..base import ToolType, ToolDefinition, ToolExecutor
from plugins_func.register import Action, ActionResponse
from .mcp_handler import call_mcp_tool
class DeviceMCPExecutor(ToolExecutor):
"""设备端MCP工具执行器"""
def __init__(self, conn):
self.conn = conn
async def execute(
self, conn, tool_name: str, arguments: Dict[str, Any]
) -> ActionResponse:
"""执行设备端MCP工具"""
if not hasattr(conn, "mcp_client") or not conn.mcp_client:
return ActionResponse(
action=Action.ERROR,
response="设备端MCP客户端未初始化",
)
if not await conn.mcp_client.is_ready():
return ActionResponse(
action=Action.ERROR,
response="设备端MCP客户端未准备就绪",
)
try:
# 转换参数为JSON字符串
import json
args_str = json.dumps(arguments) if arguments else "{}"
# 调用设备端MCP工具
result = await call_mcp_tool(conn, conn.mcp_client, tool_name, args_str)
resultJson = None
if isinstance(result, str):
try:
resultJson = json.loads(result)
except Exception as e:
pass
# 视觉大模型不经过二次LLM处理
if (
resultJson is not None
and isinstance(resultJson, dict)
and "action" in resultJson
):
return ActionResponse(
action=Action[resultJson["action"]],
response=resultJson.get("response", ""),
)
return ActionResponse(action=Action.REQLLM, result=str(result))
except ValueError as e:
return ActionResponse(action=Action.NOTFOUND, response=str(e))
except Exception as e:
return ActionResponse(action=Action.ERROR, response=str(e))
def get_tools(self) -> Dict[str, ToolDefinition]:
"""获取所有设备端MCP工具"""
if not hasattr(self.conn, "mcp_client") or not self.conn.mcp_client:
return {}
tools = {}
mcp_tools = self.conn.mcp_client.get_available_tools()
for tool in mcp_tools:
func_def = tool.get("function", {})
tool_name = func_def.get("name", "")
if tool_name:
tools[tool_name] = ToolDefinition(
name=tool_name, description=tool, tool_type=ToolType.DEVICE_MCP
)
return tools
def has_tool(self, tool_name: str) -> bool:
"""检查是否有指定的设备端MCP工具"""
if not hasattr(self.conn, "mcp_client") or not self.conn.mcp_client:
return False
return self.conn.mcp_client.has_tool(tool_name)
main/xiaozhi-server/core/handle/mcpHandle.py → main/xiaozhi-server/core/providers/tools/device_mcp/mcp_handler.py
+33 -32
View File
@@ -1,14 +1,19 @@
"""设备端MCP客户端支持模块"""
import json
import asyncio
import re
from concurrent.futures import Future
from core.utils.util import get_vision_url, sanitize_tool_name
from core.utils.auth import AuthToken
from config.logger import setup_logging
TAG = __name__
logger = setup_logging()
class MCPClient:
"""MCPClient，用于管理MCP状态和工具"""
"""设备端MCP客户端，用于管理MCP状态和工具"""
def __init__(self):
self.tools = {} # sanitized_name -> tool_data
@@ -94,24 +99,24 @@ class MCPClient:
async def send_mcp_message(conn, payload: dict):
"""Helper to send MCP messages, encapsulating common logic."""
if not conn.features.get("mcp"):
conn.logger.bind(tag=TAG).warning("客户端不支持MCP，无法发送MCP消息")
logger.bind(tag=TAG).warning("客户端不支持MCP，无法发送MCP消息")
return
message = json.dumps({"type": "mcp", "payload": payload})
try:
await conn.websocket.send(message)
conn.logger.bind(tag=TAG).info(f"成功发送MCP消息: {message}")
logger.bind(tag=TAG).info(f"成功发送MCP消息: {message}")
except Exception as e:
conn.logger.bind(tag=TAG).error(f"发送MCP消息失败: {e}")
logger.bind(tag=TAG).error(f"发送MCP消息失败: {e}")
async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
"""处理MCP消息,包括初始化、工具列表和工具调用响应等"""
conn.logger.bind(tag=TAG).info(f"处理MCP消息: {payload}")
logger.bind(tag=TAG).info(f"处理MCP消息: {str(payload)[:100]}")
if not isinstance(payload, dict):
conn.logger.bind(tag=TAG).error("MCP消息缺少payload字段或格式错误")
logger.bind(tag=TAG).error("MCP消息缺少payload字段或格式错误")
return
# Handle result
@@ -121,32 +126,32 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
# Check for tool call response first
if msg_id in mcp_client.call_results:
conn.logger.bind(tag=TAG).debug(
logger.bind(tag=TAG).debug(
f"收到工具调用响应，ID: {msg_id}, 结果: {result}"
)
await mcp_client.resolve_call_result(msg_id, result)
return
if msg_id == 1: # mcpInitializeID
conn.logger.bind(tag=TAG).debug("收到MCP初始化响应")
logger.bind(tag=TAG).debug("收到MCP初始化响应")
server_info = result.get("serverInfo")
if isinstance(server_info, dict):
name = server_info.get("name")
version = server_info.get("version")
conn.logger.bind(tag=TAG).info(
logger.bind(tag=TAG).info(
f"客户端MCP服务器信息: name={name}, version={version}"
)
return
elif msg_id == 2: # mcpToolsListID
conn.logger.bind(tag=TAG).debug("收到MCP工具列表响应")
logger.bind(tag=TAG).debug("收到MCP工具列表响应")
if isinstance(result, dict) and "tools" in result:
tools_data = result["tools"]
if not isinstance(tools_data, list):
conn.logger.bind(tag=TAG).error("工具列表格式错误")
logger.bind(tag=TAG).error("工具列表格式错误")
return
conn.logger.bind(tag=TAG).info(
logger.bind(tag=TAG).info(
f"客户端设备支持的工具数量: {len(tools_data)}"
)
@@ -172,7 +177,7 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
"inputSchema": input_schema,
}
await mcp_client.add_tool(new_tool)
conn.logger.bind(tag=TAG).debug(f"客户端工具 #{i+1}: {name}")
logger.bind(tag=TAG).debug(f"客户端工具 #{i+1}: {name}")
# 替换所有工具描述中的工具名称
for tool_data in mcp_client.tools.values():
@@ -190,24 +195,27 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
next_cursor = result.get("nextCursor", "")
if next_cursor:
conn.logger.bind(tag=TAG).info(
f"有更多工具，nextCursor: {next_cursor}"
)
logger.bind(tag=TAG).info(f"有更多工具，nextCursor: {next_cursor}")
await send_mcp_tools_list_continue_request(conn, next_cursor)
else:
await mcp_client.set_ready(True)
conn.logger.bind(tag=TAG).info("所有工具已获取，MCP客户端准备就绪")
logger.bind(tag=TAG).info("所有工具已获取，MCP客户端准备就绪")
# 刷新工具缓存，确保MCP工具被包含在函数列表中
if hasattr(conn, "func_handler") and conn.func_handler:
conn.func_handler.tool_manager.refresh_tools()
conn.func_handler.current_support_functions()
return
# Handle method calls (requests from the client)
elif "method" in payload:
method = payload["method"]
conn.logger.bind(tag=TAG).info(f"收到MCP客户端请求: {method}")
logger.bind(tag=TAG).info(f"收到MCP客户端请求: {method}")
elif "error" in payload:
error_data = payload["error"]
error_msg = error_data.get("message", "未知错误")
conn.logger.bind(tag=TAG).error(f"收到MCP错误响应: {error_msg}")
logger.bind(tag=TAG).error(f"收到MCP错误响应: {error_msg}")
msg_id = int(payload.get("id", 0))
if msg_id in mcp_client.call_results:
@@ -216,9 +224,6 @@ async def handle_mcp_message(conn, mcp_client: MCPClient, payload: dict):
)
# --- Outgoing MCP Messages ---
async def send_mcp_initialize_message(conn):
"""发送MCP初始化消息"""
@@ -250,7 +255,7 @@ async def send_mcp_initialize_message(conn):
},
},
}
conn.logger.bind(tag=TAG).info("发送MCP初始化消息")
logger.bind(tag=TAG).info("发送MCP初始化消息")
await send_mcp_message(conn, payload)
@@ -261,7 +266,7 @@ async def send_mcp_tools_list_request(conn):
"id": 2, # mcpToolsListID
"method": "tools/list",
}
conn.logger.bind(tag=TAG).debug("发送MCP工具列表请求")
logger.bind(tag=TAG).debug("发送MCP工具列表请求")
await send_mcp_message(conn, payload)
@@ -273,7 +278,7 @@ async def send_mcp_tools_list_continue_request(conn, cursor: str):
"method": "tools/list",
"params": {"cursor": cursor},
}
conn.logger.bind(tag=TAG).info(f"发送带cursor的MCP工具列表请求: {cursor}")
logger.bind(tag=TAG).info(f"发送带cursor的MCP工具列表请求: {cursor}")
await send_mcp_message(conn, payload)
@@ -307,8 +312,6 @@ async def call_mcp_tool(
# 如果解析失败，尝试合并多个JSON对象
try:
# 使用正则表达式匹配所有JSON对象
import re
json_objects = re.findall(r"\{[^{}]*\}", args)
if len(json_objects) > 1:
# 合并所有JSON对象
@@ -327,7 +330,7 @@ async def call_mcp_tool(
else:
raise ValueError(f"参数JSON解析失败: {args}")
except Exception as e:
conn.logger.bind(tag=TAG).error(
logger.bind(tag=TAG).error(
f"参数JSON解析失败: {str(e)}, 原始参数: {args}"
)
raise ValueError(f"参数JSON解析失败: {str(e)}")
@@ -353,15 +356,13 @@ async def call_mcp_tool(
"params": {"name": actual_name, "arguments": arguments},
}
conn.logger.bind(tag=TAG).info(
f"发送客户端mcp工具调用请求: {actual_name}，参数: {args}"
)
logger.bind(tag=TAG).info(f"发送客户端mcp工具调用请求: {actual_name}，参数: {args}")
await send_mcp_message(conn, payload)
try:
# Wait for response or timeout
raw_result = await asyncio.wait_for(result_future, timeout=timeout)
conn.logger.bind(tag=TAG).info(
logger.bind(tag=TAG).info(
f"客户端mcp工具调用 {actual_name} 成功，原始结果: {raw_result}"
)
main/xiaozhi-server/core/providers/tools/mcp_endpoint/__init__.py
+21
View File
@@ -0,0 +1,21 @@
"""MCP接入点工具模块"""
from .mcp_endpoint_executor import MCPEndpointExecutor
from .mcp_endpoint_client import MCPEndpointClient
from .mcp_endpoint_handler import (
connect_mcp_endpoint,
send_mcp_endpoint_initialize,
send_mcp_endpoint_notification,
send_mcp_endpoint_tools_list,
call_mcp_endpoint_tool,
)
__all__ = [
"MCPEndpointExecutor",
"MCPEndpointClient",
"connect_mcp_endpoint",
"send_mcp_endpoint_initialize",
"send_mcp_endpoint_notification",
"send_mcp_endpoint_tools_list",
"call_mcp_endpoint_tool",
]
main/xiaozhi-server/core/providers/tools/mcp_endpoint/mcp_endpoint_client.py
+112
View File
@@ -0,0 +1,112 @@
"""MCP接入点客户端定义"""
import asyncio
from concurrent.futures import Future
from core.utils.util import sanitize_tool_name
from config.logger import setup_logging
TAG = __name__
logger = setup_logging()
class MCPEndpointClient:
"""MCP接入点客户端，用于管理MCP接入点状态和工具"""
def __init__(self, conn=None):
self.conn = conn
self.tools = {} # sanitized_name -> tool_data
self.name_mapping = {}
self.ready = False
self.call_results = {} # To store Futures for tool call responses
self.next_id = 1
self.lock = asyncio.Lock()
self._cached_available_tools = None # Cache for get_available_tools
self.websocket = None # WebSocket连接
def has_tool(self, name: str) -> bool:
return name in self.tools
def get_available_tools(self) -> list:
# Check if the cache is valid
if self._cached_available_tools is not None:
return self._cached_available_tools
# If cache is not valid, regenerate the list
result = []
for tool_name, tool_data in self.tools.items():
function_def = {
"name": tool_name,
"description": tool_data["description"],
"parameters": {
"type": tool_data["inputSchema"].get("type", "object"),
"properties": tool_data["inputSchema"].get("properties", {}),
"required": tool_data["inputSchema"].get("required", []),
},
}
result.append({"type": "function", "function": function_def})
self._cached_available_tools = result # Store the generated list in cache
return result
async def is_ready(self) -> bool:
async with self.lock:
return self.ready
async def set_ready(self, status: bool):
async with self.lock:
self.ready = status
async def add_tool(self, tool_data: dict):
async with self.lock:
sanitized_name = sanitize_tool_name(tool_data["name"])
self.tools[sanitized_name] = tool_data
self.name_mapping[sanitized_name] = tool_data["name"]
self._cached_available_tools = (
None # Invalidate the cache when a tool is added
)
async def get_next_id(self) -> int:
async with self.lock:
current_id = self.next_id
self.next_id += 1
return current_id
async def register_call_result_future(self, id: int, future: Future):
async with self.lock:
self.call_results[id] = future
async def resolve_call_result(self, id: int, result: any):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_result(result)
async def reject_call_result(self, id: int, exception: Exception):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_exception(exception)
async def cleanup_call_result(self, id: int):
async with self.lock:
if id in self.call_results:
self.call_results.pop(id)
def set_websocket(self, websocket):
"""设置WebSocket连接"""
self.websocket = websocket
async def send_message(self, message: str):
"""发送消息到MCP接入点"""
if self.websocket:
await self.websocket.send(message)
else:
raise RuntimeError("WebSocket连接未建立")
async def close(self):
"""关闭WebSocket连接"""
if self.websocket:
await self.websocket.close()
self.websocket = None

Some files were not shown because too many files have changed in this diff Show More