mirror of
https://github.com/xinnan-tech/xiaozhi-esp32-server.git
synced 2026-07-23 15:43:54 +08:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Chingfeng Li
|
c07ecbf6d1 |
@@ -80,7 +80,6 @@ VAD:
|
||||
7、[如何开启声纹识别](./voiceprint-integration.md)<br/>
|
||||
8、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/>
|
||||
9、[知识库ragflow集成指南](./ragflow-integration.md)<br/>
|
||||
10、[如何部署上下文源](./context-provider-integration.md)<br/>
|
||||
|
||||
### 11、语音克隆、本地语音部署相关教程
|
||||
1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/>
|
||||
|
||||
@@ -1,224 +0,0 @@
|
||||
# 上下文源使用教程
|
||||
|
||||
## 概述
|
||||
|
||||
`上下文源`，就是为小智系统提示词的上下文添加【数据源】。
|
||||
|
||||
`上下文源` 在小智在唤醒那一刻，获取外部系统的数据，并将其动态注入到大模型的系统提示词（System Prompt）中。
|
||||
让其做到唤醒时感知世界某个事物的状态。
|
||||
|
||||
它和MCP、记忆有本质的区别：`上下文源`是强制让小智感知世界的数据；`记忆(Mem)`是让他知道之前聊了什么内容；`MCP(functionc all)`是当需要调用某项能力/知识的时候使用调用。
|
||||
|
||||
通过这个功能，在小智唤醒的一刹那，“感知”到：
|
||||
- 人体健康传感器状态（体温、血压、血氧状态等）
|
||||
- 业务系统的实时数据（服务器负载、待办数据、股票信息等）
|
||||
- 任何可以通过 HTTP API 获取的文本信息
|
||||
|
||||
**注意**：该功能只是方便小智在唤醒的时候感知事物的状态，而如果想要小智唤醒后实时获取事物的状态，建议在此功能上再结合MCP工具的调用。
|
||||
|
||||
## 工作原理
|
||||
|
||||
1. **配置源**：用户配置一个或多个 HTTP API 地址。
|
||||
2. **触发请求**：当系统构建 Prompt 时，如果发现模板中包含 `{{ dynamic_context }}` 占位符，会请求所有配置的 API。
|
||||
3. **自动注入**：系统会自动将 API 返回的数据格式化为 Markdown 列表，替换 `{{ dynamic_context }}` 占位符。
|
||||
|
||||
## 接口规范
|
||||
|
||||
为了让小智正确解析数据，您的 API 需要满足以下规范：
|
||||
|
||||
- **请求方式**：`GET`
|
||||
- **请求头**：系统会自动添加 `device-id` 字段到 Request Header。
|
||||
- **响应格式**：必须返回 JSON 格式，且包含 `code` 和 `data` 字段。
|
||||
|
||||
### 响应示例
|
||||
|
||||
**情况 1：返回键值对**
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "success",
|
||||
"data": {
|
||||
"客厅温度": "26℃",
|
||||
"客厅湿度": "45%",
|
||||
"大门状态": "已关闭"
|
||||
}
|
||||
}
|
||||
```
|
||||
*注入效果：*
|
||||
```markdown
|
||||
<context>
|
||||
- **客厅温度：** 26℃
|
||||
- **客厅湿度：** 45%
|
||||
- **大门状态：** 已关闭
|
||||
</context>
|
||||
```
|
||||
|
||||
**情况 2：返回列表**
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"data": [
|
||||
"您有10个待办事项",
|
||||
"当前汽车的行驶速度是100km每小时"
|
||||
]
|
||||
}
|
||||
```
|
||||
*注入效果：*
|
||||
```markdown
|
||||
<context>
|
||||
- 您有10个待办事项
|
||||
- 当前汽车的行驶速度是100km每小时
|
||||
</context>
|
||||
```
|
||||
|
||||
## 配置指南
|
||||
|
||||
### 方式 1：智控台配置（全模块部署）
|
||||
|
||||
1. 登录智控台，进入**角色配置**页面。
|
||||
2. 找到**上下文源**配置项（点击“编辑源”按钮）。
|
||||
3. 点击**添加**，输入您的 API 地址。
|
||||
4. 如果 API 需要鉴权，可以在**请求头**部分添加 `Authorization` 或其他 Header。
|
||||
5. 保存配置。
|
||||
|
||||
### 方式 2：配置文件配置（单模块部署）
|
||||
|
||||
编辑 `xiaozhi-server/data/.config.yaml` 文件，添加 `context_providers` 配置段：
|
||||
|
||||
```yaml
|
||||
# 上下文源配置
|
||||
context_providers:
|
||||
- url: "http://api.example.com/data"
|
||||
headers:
|
||||
Authorization: "Bearer your-token"
|
||||
- url: "http://another-api.com/data"
|
||||
```
|
||||
|
||||
## 启用功能
|
||||
|
||||
默认情况下，系统的提示词模板文件（`data/.agent-base-prompt.txt`）中已经预置了 `{{ dynamic_context }}` 占位符，您无需手动添加。
|
||||
|
||||
**示例：**
|
||||
|
||||
```markdown
|
||||
<context>
|
||||
【重要！以下信息已实时提供，无需调用工具查询，请直接使用：】
|
||||
- **设备ID：** {{device_id}}
|
||||
- **当前时间：** {{current_time}}
|
||||
...
|
||||
{{ dynamic_context }}
|
||||
</context>
|
||||
```
|
||||
|
||||
**注意**：如果您不需要使用此功能，可以选择**不配置任何上下文源**，也可以从提示词模板文件中**删除** `{{ dynamic_context }}` 占位符。
|
||||
|
||||
## 附录：Mock 测试服务示例
|
||||
|
||||
为了方便您测试和开发，我们提供了一个简单的 Python Mock Server 脚本。您可以运行此脚本在本地模拟 API 接口。
|
||||
|
||||
**mock_api_server.py**
|
||||
|
||||
```python
|
||||
import http.server
|
||||
import socketserver
|
||||
import json
|
||||
from urllib.parse import urlparse, parse_qs
|
||||
|
||||
# 设置端口号
|
||||
PORT = 8081
|
||||
|
||||
class MockRequestHandler(http.server.SimpleHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
# 解析路径和参数
|
||||
parsed_path = urlparse(self.path)
|
||||
path = parsed_path.path
|
||||
query = parse_qs(parsed_path.query)
|
||||
|
||||
response_data = {}
|
||||
status_code = 200
|
||||
|
||||
print(f"收到请求: {path}, 参数: {query}")
|
||||
|
||||
# Case 1: 模拟健康数据 (返回字典 Dict)
|
||||
# 路径参数风格: /health
|
||||
# device_id 从 Header 获取
|
||||
if path == "/health":
|
||||
device_id = self.headers.get("device-id", "unknown_device")
|
||||
print(f"device_id: {device_id}")
|
||||
response_data = {
|
||||
"code": 0,
|
||||
"msg": "success",
|
||||
"data": {
|
||||
"测试设备ID": device_id,
|
||||
"心率": "80 bpm",
|
||||
"血压": "120/80 mmHg",
|
||||
"状态": "良好"
|
||||
}
|
||||
}
|
||||
|
||||
# Case 2: 模拟新闻列表 (返回列表 List)
|
||||
# 无参数: /news/list
|
||||
elif path == "/news/list":
|
||||
response_data = {
|
||||
"code": 0,
|
||||
"msg": "success",
|
||||
"data": [
|
||||
"今日头条：Python 3.14 发布",
|
||||
"科技新闻：AI 助手改变生活",
|
||||
"本地新闻：明日有大雨，记得带伞"
|
||||
]
|
||||
}
|
||||
|
||||
# Case 3: 模拟天气简报 (返回字符串 String)
|
||||
# 无参数: /weather/simple
|
||||
elif path == "/weather/simple":
|
||||
response_data = {
|
||||
"code": 0,
|
||||
"msg": "success",
|
||||
"data": "今日晴转多云，气温 20-25 度，空气质量优，适合出行。"
|
||||
}
|
||||
|
||||
# Case 4: 模拟设备详情 (Query参数风格)
|
||||
# 参数风格: /device/info
|
||||
# device_id 从 Header 获取
|
||||
elif path == "/device/info":
|
||||
device_id = self.headers.get("device-id", "unknown_device")
|
||||
response_data = {
|
||||
"code": 0,
|
||||
"msg": "success",
|
||||
"data": {
|
||||
"查询方式": "Header参数",
|
||||
"设备ID": device_id,
|
||||
"电量": "85%",
|
||||
"固件": "v2.0.1"
|
||||
}
|
||||
}
|
||||
|
||||
# Case 5: 404 Not Found
|
||||
else:
|
||||
status_code = 404
|
||||
response_data = {"error": "接口不存在"}
|
||||
|
||||
# 发送响应
|
||||
self.send_response(status_code)
|
||||
self.send_header('Content-type', 'application/json; charset=utf-8')
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(response_data, ensure_ascii=False).encode('utf-8'))
|
||||
|
||||
# 启动服务
|
||||
# 允许地址重用，防止快速重启报错
|
||||
socketserver.TCPServer.allow_reuse_address = True
|
||||
with socketserver.TCPServer(("", PORT), MockRequestHandler) as httpd:
|
||||
print(f"==================================================")
|
||||
print(f"Mock API Server 已启动: http://localhost:{PORT}")
|
||||
print(f"可用接口列表:")
|
||||
print(f"1. [字典] http://localhost:{PORT}/health")
|
||||
print(f"2. [列表] http://localhost:{PORT}/news/list")
|
||||
print(f"3. [文本] http://localhost:{PORT}/weather/simple")
|
||||
print(f"4. [参数] http://localhost:{PORT}/device/info")
|
||||
print(f"==================================================")
|
||||
try:
|
||||
httpd.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
print("\n服务已停止")
|
||||
```
|
||||
@@ -71,7 +71,6 @@ docker logs -f mcp-endpoint-server
|
||||
请你保留好上面两个`接口地址`，下一步要用到。
|
||||
|
||||
# 2、全模块部署时，怎么配置MCP接入点
|
||||
首先，你要开启MCP接入点功能。在智控台，点击顶部`参数字典`，在下拉菜单中，点击`系统功能配置`页面。在页面上勾选`MCP接入点`，点击`保存配置`。在`角色配置`页面，点击`编辑功能`按钮，即可看到`mcp接入点`功能。
|
||||
|
||||
如果你是全模块部署，使用管理员账号，登录智控台，点击顶部`参数字典`，选择`参数管理`功能。
|
||||
|
||||
|
||||
@@ -156,14 +156,6 @@ services:
|
||||
|
||||
编辑`ragflow/docker`文件夹下的`.env`文件,找到以下配置，逐个搜索，逐个修改！逐个搜索，逐个修改！
|
||||
|
||||
下面对于`.env`文件的修改，60%的人会忽略`MYSQL_USER`配置导致ragflow启动不成功，因此，需要强调三次：
|
||||
|
||||
强调第一次：如果你的`.env`文件如果没有`MYSQL_USER`配置，请在配置文件增加这项！
|
||||
|
||||
强调第二次：如果你的`.env`文件如果没有`MYSQL_USER`配置，请在配置文件增加这项！
|
||||
|
||||
强调第三次：如果你的`.env`文件如果没有`MYSQL_USER`配置，请在配置文件增加这项！
|
||||
|
||||
``` env
|
||||
# 端口设置
|
||||
SVR_WEB_HTTP_PORT=8008 # HTTP端口
|
||||
@@ -238,11 +230,9 @@ docker-compose -f docker-compose.yml up -d
|
||||
在弹框中，点击"Create new Key"按钮，生成一个API Key。复制这个`API Key`，你稍后会用到。
|
||||
|
||||
# 第二步 配置到智控台
|
||||
确保你的智控台版本是`0.8.7`或以上。使用超级管理员账号登录到智控台。
|
||||
确保你的智控台版本是`0.8.7`或以上。使用超级管理员账号登录到智控台。在顶部导航栏中，点击`模型配置`，在左侧导航栏中，点击`知识库`。
|
||||
|
||||
首先，你要先开启知识库功能。在顶部导航栏中，点击`参数字典`，在下拉菜单中，点击`系统功能配置`页面。在页面上勾选`知识库`，点击`保存配置`。即可在导航栏看到`知识库`功能。
|
||||
|
||||
在顶部导航栏中，点击`模型配置`，在左侧导航栏中，点击`知识库`。在列表中找到`RAG_RAGFlow`，点击`编辑`按钮。
|
||||
在列表中找到`RAG_RAGFlow`，点击`编辑`按钮。
|
||||
|
||||
在`服务地址`中，填写`http://你的ragflow服务的局域网IP:8008`，例如我的ragflow服务的局域网IP是`192.168.1.100`，那么我就填写`http://192.168.1.100:8008`。
|
||||
|
||||
|
||||
@@ -164,8 +164,6 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd
|
||||
# 2、全模块部署时，怎么配置声纹识别
|
||||
|
||||
## 第一步 配置接口
|
||||
首先，你要开启声纹识别功能。在智控台，点击顶部`参数字典`，在下拉菜单中，点击`系统功能配置`页面。在页面上勾选`声纹识别`，点击`保存配置`。即可在新建智能体的卡片上看到`声纹识别`按钮。
|
||||
|
||||
如果你是全模块部署，使用管理员账号，登录智控台，点击顶部`参数字典`，选择`参数管理`功能。
|
||||
|
||||
然后搜索参数`server.voice_print`，此时，它的值应该是`null`值。
|
||||
|
||||
@@ -141,11 +141,6 @@ public interface Constant {
|
||||
*/
|
||||
String SERVER_MQTT_SECRET = "server.mqtt_signature_key";
|
||||
|
||||
/**
|
||||
* WebSocket认证开关
|
||||
*/
|
||||
String SERVER_AUTH_ENABLED = "server.auth.enabled";
|
||||
|
||||
/**
|
||||
* 无记忆
|
||||
*/
|
||||
@@ -304,7 +299,7 @@ public interface Constant {
|
||||
/**
|
||||
* 版本号
|
||||
*/
|
||||
public static final String VERSION = "0.8.10";
|
||||
public static final String VERSION = "0.8.8";
|
||||
|
||||
/**
|
||||
* 无效固件URL
|
||||
|
||||
@@ -228,16 +228,4 @@ public interface ErrorCode {
|
||||
|
||||
// 智能体模板相关错误码(补充)
|
||||
int AGENT_TEMPLATE_NOT_FOUND = 10183; // 默认智能体未找到
|
||||
|
||||
// 知识库适配器相关错误码
|
||||
int RAG_ADAPTER_TYPE_NOT_SUPPORTED = 10184; // 不支持的适配器类型
|
||||
int RAG_CONFIG_VALIDATION_FAILED = 10185; // RAG配置验证失败
|
||||
int RAG_ADAPTER_CREATION_FAILED = 10186; // 适配器创建失败
|
||||
int RAG_ADAPTER_INIT_FAILED = 10187; // 适配器初始化失败
|
||||
int RAG_ADAPTER_CONNECTION_FAILED = 10188; // 适配器连接测试失败
|
||||
int RAG_ADAPTER_OPERATION_FAILED = 10189; // 适配器操作失败
|
||||
int RAG_ADAPTER_NOT_FOUND = 10190; // 适配器未找到
|
||||
int RAG_ADAPTER_CACHE_ERROR = 10191; // 适配器缓存错误
|
||||
int RAG_ADAPTER_TYPE_NOT_FOUND = 10192; // 适配器类型未找到
|
||||
|
||||
}
|
||||
|
||||
@@ -44,7 +44,6 @@ import xiaozhi.modules.agent.entity.AgentEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
|
||||
import xiaozhi.modules.agent.service.AgentChatAudioService;
|
||||
import xiaozhi.modules.agent.service.AgentChatHistoryService;
|
||||
import xiaozhi.modules.agent.service.AgentContextProviderService;
|
||||
import xiaozhi.modules.agent.service.AgentPluginMappingService;
|
||||
import xiaozhi.modules.agent.service.AgentService;
|
||||
import xiaozhi.modules.agent.service.AgentTemplateService;
|
||||
@@ -65,7 +64,6 @@ public class AgentController {
|
||||
private final AgentChatHistoryService agentChatHistoryService;
|
||||
private final AgentChatAudioService agentChatAudioService;
|
||||
private final AgentPluginMappingService agentPluginMappingService;
|
||||
private final AgentContextProviderService agentContextProviderService;
|
||||
private final RedisUtils redisUtils;
|
||||
|
||||
@GetMapping("/list")
|
||||
@@ -137,8 +135,6 @@ public class AgentController {
|
||||
agentChatHistoryService.deleteByAgentId(id, true, true);
|
||||
// 删除关联的插件
|
||||
agentPluginMappingService.deleteByAgentId(id);
|
||||
// 删除关联的上下文源配置
|
||||
agentContextProviderService.deleteByAgentId(id);
|
||||
// 再删除智能体
|
||||
agentService.deleteById(id);
|
||||
return new Result<>();
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
package xiaozhi.modules.agent.dao;
|
||||
|
||||
import org.apache.ibatis.annotations.Mapper;
|
||||
import xiaozhi.common.dao.BaseDao;
|
||||
import xiaozhi.modules.agent.entity.AgentContextProviderEntity;
|
||||
|
||||
@Mapper
|
||||
public interface AgentContextProviderDao extends BaseDao<AgentContextProviderEntity> {
|
||||
}
|
||||
@@ -69,9 +69,6 @@ public class AgentUpdateDTO implements Serializable {
|
||||
@Schema(description = "排序", example = "1", nullable = true)
|
||||
private Integer sort;
|
||||
|
||||
@Schema(description = "上下文源配置", nullable = true)
|
||||
private List<ContextProviderDTO> contextProviders;
|
||||
|
||||
@Data
|
||||
@Schema(description = "插件函数信息")
|
||||
public static class FunctionInfo implements Serializable {
|
||||
|
||||
@@ -1,19 +0,0 @@
|
||||
package xiaozhi.modules.agent.dto;
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.Map;
|
||||
|
||||
import io.swagger.v3.oas.annotations.media.Schema;
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
@Schema(description = "上下文源配置DTO")
|
||||
public class ContextProviderDTO implements Serializable {
|
||||
private static final long serialVersionUID = 1L;
|
||||
|
||||
@Schema(description = "URL地址")
|
||||
private String url;
|
||||
|
||||
@Schema(description = "请求头")
|
||||
private Map<String, Object> headers;
|
||||
}
|
||||
-43
@@ -1,43 +0,0 @@
|
||||
package xiaozhi.modules.agent.entity;
|
||||
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
|
||||
import com.baomidou.mybatisplus.annotation.IdType;
|
||||
import com.baomidou.mybatisplus.annotation.TableField;
|
||||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler;
|
||||
|
||||
import io.swagger.v3.oas.annotations.media.Schema;
|
||||
import lombok.Data;
|
||||
import xiaozhi.modules.agent.dto.ContextProviderDTO;
|
||||
|
||||
@Data
|
||||
@TableName(value = "ai_agent_context_provider", autoResultMap = true)
|
||||
@Schema(description = "智能体上下文源配置")
|
||||
public class AgentContextProviderEntity {
|
||||
|
||||
@TableId(type = IdType.ASSIGN_UUID)
|
||||
@Schema(description = "主键")
|
||||
private String id;
|
||||
|
||||
@Schema(description = "智能体ID")
|
||||
private String agentId;
|
||||
|
||||
@Schema(description = "上下文源配置")
|
||||
@TableField(typeHandler = JacksonTypeHandler.class)
|
||||
private List<ContextProviderDTO> contextProviders;
|
||||
|
||||
@Schema(description = "创建者")
|
||||
private Long creator;
|
||||
|
||||
@Schema(description = "创建时间")
|
||||
private Date createdAt;
|
||||
|
||||
@Schema(description = "更新者")
|
||||
private Long updater;
|
||||
|
||||
@Schema(description = "更新时间")
|
||||
private Date updatedAt;
|
||||
}
|
||||
-25
@@ -1,25 +0,0 @@
|
||||
package xiaozhi.modules.agent.service;
|
||||
|
||||
import xiaozhi.common.service.BaseService;
|
||||
import xiaozhi.modules.agent.entity.AgentContextProviderEntity;
|
||||
|
||||
public interface AgentContextProviderService extends BaseService<AgentContextProviderEntity> {
|
||||
/**
|
||||
* 根据智能体ID获取上下文源配置
|
||||
* @param agentId 智能体ID
|
||||
* @return 上下文源配置实体
|
||||
*/
|
||||
AgentContextProviderEntity getByAgentId(String agentId);
|
||||
|
||||
/**
|
||||
* 保存或更新上下文源配置
|
||||
* @param entity 实体
|
||||
*/
|
||||
void saveOrUpdateByAgentId(AgentContextProviderEntity entity);
|
||||
|
||||
/**
|
||||
* 根据智能体ID删除上下文源配置
|
||||
* @param agentId 智能体ID
|
||||
*/
|
||||
void deleteByAgentId(String agentId);
|
||||
}
|
||||
-35
@@ -1,35 +0,0 @@
|
||||
package xiaozhi.modules.agent.service.impl;
|
||||
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
|
||||
|
||||
import xiaozhi.common.service.impl.BaseServiceImpl;
|
||||
import xiaozhi.modules.agent.dao.AgentContextProviderDao;
|
||||
import xiaozhi.modules.agent.entity.AgentContextProviderEntity;
|
||||
import xiaozhi.modules.agent.service.AgentContextProviderService;
|
||||
|
||||
@Service
|
||||
public class AgentContextProviderServiceImpl extends BaseServiceImpl<AgentContextProviderDao, AgentContextProviderEntity> implements AgentContextProviderService {
|
||||
|
||||
@Override
|
||||
public AgentContextProviderEntity getByAgentId(String agentId) {
|
||||
return baseDao.selectOne(new QueryWrapper<AgentContextProviderEntity>().eq("agent_id", agentId));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void saveOrUpdateByAgentId(AgentContextProviderEntity entity) {
|
||||
AgentContextProviderEntity exist = getByAgentId(entity.getAgentId());
|
||||
if (exist != null) {
|
||||
entity.setId(exist.getId());
|
||||
updateById(entity);
|
||||
} else {
|
||||
insert(entity);
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void deleteByAgentId(String agentId) {
|
||||
baseDao.delete(new QueryWrapper<AgentContextProviderEntity>().eq("agent_id", agentId));
|
||||
}
|
||||
}
|
||||
+1
-32
@@ -32,12 +32,10 @@ import xiaozhi.modules.agent.dao.AgentDao;
|
||||
import xiaozhi.modules.agent.dto.AgentCreateDTO;
|
||||
import xiaozhi.modules.agent.dto.AgentDTO;
|
||||
import xiaozhi.modules.agent.dto.AgentUpdateDTO;
|
||||
import xiaozhi.modules.agent.entity.AgentContextProviderEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentPluginMapping;
|
||||
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
|
||||
import xiaozhi.modules.agent.service.AgentChatHistoryService;
|
||||
import xiaozhi.modules.agent.service.AgentContextProviderService;
|
||||
import xiaozhi.modules.agent.service.AgentPluginMappingService;
|
||||
import xiaozhi.modules.agent.service.AgentService;
|
||||
import xiaozhi.modules.agent.service.AgentTemplateService;
|
||||
@@ -64,7 +62,6 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
|
||||
private final AgentChatHistoryService agentChatHistoryService;
|
||||
private final AgentTemplateService agentTemplateService;
|
||||
private final ModelProviderService modelProviderService;
|
||||
private final AgentContextProviderService agentContextProviderService;
|
||||
|
||||
@Override
|
||||
public PageData<AgentEntity> adminAgentList(Map<String, Object> params) {
|
||||
@@ -88,13 +85,6 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
|
||||
agent.setChatHistoryConf(Constant.ChatHistoryConfEnum.RECORD_TEXT_AUDIO.getCode());
|
||||
}
|
||||
}
|
||||
|
||||
// 查询上下文源配置
|
||||
AgentContextProviderEntity contextProviderEntity = agentContextProviderService.getByAgentId(id);
|
||||
if (contextProviderEntity != null) {
|
||||
agent.setContextProviders(contextProviderEntity.getContextProviders());
|
||||
}
|
||||
|
||||
// 无需额外查询插件列表，已通过SQL查询出来
|
||||
return agent;
|
||||
}
|
||||
@@ -341,14 +331,6 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
|
||||
agentChatHistoryService.deleteByAgentId(existingEntity.getId(), true, false);
|
||||
}
|
||||
|
||||
// 更新上下文源配置
|
||||
if (dto.getContextProviders() != null) {
|
||||
AgentContextProviderEntity contextEntity = new AgentContextProviderEntity();
|
||||
contextEntity.setAgentId(agentId);
|
||||
contextEntity.setContextProviders(dto.getContextProviders());
|
||||
agentContextProviderService.saveOrUpdateByAgentId(contextEntity);
|
||||
}
|
||||
|
||||
boolean b = validateLLMIntentParams(dto.getLlmModelId(), dto.getIntentModelId());
|
||||
if (!b) {
|
||||
throw new RenException(ErrorCode.LLM_INTENT_PARAMS_MISMATCH);
|
||||
@@ -413,20 +395,7 @@ public class AgentServiceImpl extends BaseServiceImpl<AgentDao, AgentEntity> imp
|
||||
entity.setIntentModelId(template.getIntentModelId());
|
||||
entity.setSystemPrompt(template.getSystemPrompt());
|
||||
entity.setSummaryMemory(template.getSummaryMemory());
|
||||
|
||||
// 根据记忆模型类型设置默认的chatHistoryConf值
|
||||
if (template.getMemModelId() != null) {
|
||||
if (template.getMemModelId().equals("Memory_nomem")) {
|
||||
// 无记忆功能的模型，默认不记录聊天记录
|
||||
entity.setChatHistoryConf(0);
|
||||
} else {
|
||||
// 有记忆功能的模型，默认记录文本和语音
|
||||
entity.setChatHistoryConf(2);
|
||||
}
|
||||
} else {
|
||||
entity.setChatHistoryConf(template.getChatHistoryConf());
|
||||
}
|
||||
|
||||
entity.setChatHistoryConf(template.getChatHistoryConf());
|
||||
entity.setLangCode(template.getLangCode());
|
||||
entity.setLanguage(template.getLanguage());
|
||||
}
|
||||
|
||||
@@ -5,7 +5,6 @@ import com.baomidou.mybatisplus.extension.handlers.JacksonTypeHandler;
|
||||
import io.swagger.v3.oas.annotations.media.Schema;
|
||||
import lombok.Data;
|
||||
import lombok.EqualsAndHashCode;
|
||||
import xiaozhi.modules.agent.dto.ContextProviderDTO;
|
||||
import xiaozhi.modules.agent.entity.AgentEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentPluginMapping;
|
||||
|
||||
@@ -22,7 +21,4 @@ public class AgentInfoVO extends AgentEntity
|
||||
@Schema(description = "插件列表Id")
|
||||
@TableField(typeHandler = JacksonTypeHandler.class)
|
||||
private List<AgentPluginMapping> functions;
|
||||
|
||||
@Schema(description = "上下文源配置")
|
||||
private List<ContextProviderDTO> contextProviders;
|
||||
}
|
||||
|
||||
-10
@@ -20,12 +20,10 @@ import xiaozhi.common.redis.RedisUtils;
|
||||
import xiaozhi.common.utils.ConvertUtils;
|
||||
import xiaozhi.common.utils.JsonUtils;
|
||||
import xiaozhi.modules.agent.dao.AgentVoicePrintDao;
|
||||
import xiaozhi.modules.agent.entity.AgentContextProviderEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentPluginMapping;
|
||||
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
|
||||
import xiaozhi.modules.agent.entity.AgentVoicePrintEntity;
|
||||
import xiaozhi.modules.agent.service.AgentContextProviderService;
|
||||
import xiaozhi.modules.agent.service.AgentMcpAccessPointService;
|
||||
import xiaozhi.modules.agent.service.AgentPluginMappingService;
|
||||
import xiaozhi.modules.agent.service.AgentService;
|
||||
@@ -55,7 +53,6 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
private final TimbreService timbreService;
|
||||
private final AgentPluginMappingService agentPluginMappingService;
|
||||
private final AgentMcpAccessPointService agentMcpAccessPointService;
|
||||
private final AgentContextProviderService agentContextProviderService;
|
||||
private final VoiceCloneService cloneVoiceService;
|
||||
private final AgentVoicePrintDao agentVoicePrintDao;
|
||||
|
||||
@@ -181,13 +178,6 @@ public class ConfigServiceImpl implements ConfigService {
|
||||
mcpEndpoint = mcpEndpoint.replace("/mcp/", "/call/");
|
||||
result.put("mcp_endpoint", mcpEndpoint);
|
||||
}
|
||||
|
||||
// 获取上下文源配置
|
||||
AgentContextProviderEntity contextProviderEntity = agentContextProviderService.getByAgentId(agent.getId());
|
||||
if (contextProviderEntity != null && contextProviderEntity.getContextProviders() != null && !contextProviderEntity.getContextProviders().isEmpty()) {
|
||||
result.put("context_providers", contextProviderEntity.getContextProviders());
|
||||
}
|
||||
|
||||
// 获取声纹信息
|
||||
buildVoiceprintConfig(agent.getId(), result);
|
||||
|
||||
|
||||
+2
-4
@@ -72,12 +72,10 @@ public class DeviceController {
|
||||
return new Result<String>().error(ErrorCode.MCA_NOT_NULL);
|
||||
}
|
||||
// 生成六位验证码
|
||||
String code;
|
||||
String key;
|
||||
String code = String.valueOf(Math.random()).substring(2, 8);
|
||||
String key = RedisKeys.getDeviceCaptchaKey(code);
|
||||
String existsMac = null;
|
||||
do {
|
||||
code = String.valueOf(Math.random()).substring(2, 8);
|
||||
key = RedisKeys.getDeviceCaptchaKey(code);
|
||||
existsMac = (String) redisUtils.get(key);
|
||||
} while (StringUtils.isNotBlank(existsMac));
|
||||
|
||||
|
||||
+3
-54
@@ -1,8 +1,6 @@
|
||||
package xiaozhi.modules.device.service.impl;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.InvalidKeyException;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.time.Instant;
|
||||
import java.util.Base64;
|
||||
import java.util.Date;
|
||||
@@ -171,22 +169,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
|
||||
DeviceReportRespDTO.Websocket websocket = new DeviceReportRespDTO.Websocket();
|
||||
// 从系统参数获取WebSocket URL，如果未配置则使用默认值
|
||||
String wsUrl = sysParamsService.getValue(Constant.SERVER_WEBSOCKET, true);
|
||||
|
||||
// 检查是否启用认证并生成token
|
||||
String authEnabled = sysParamsService.getValue(Constant.SERVER_AUTH_ENABLED, true);
|
||||
if ("true".equalsIgnoreCase(authEnabled)) {
|
||||
try {
|
||||
// 生成token
|
||||
String token = generateWebSocketToken(clientId, macAddress);
|
||||
websocket.setToken(token);
|
||||
} catch (Exception e) {
|
||||
log.error("生成WebSocket token失败: {}", e.getMessage());
|
||||
websocket.setToken("");
|
||||
}
|
||||
} else {
|
||||
websocket.setToken("");
|
||||
}
|
||||
|
||||
websocket.setToken("");
|
||||
if (StringUtils.isBlank(wsUrl) || wsUrl.equals("null")) {
|
||||
log.error("WebSocket地址未配置，请登录智控台，在参数管理找到【server.websocket】配置");
|
||||
wsUrl = "ws://xiaozhi.server.com:8000/xiaozhi/v1/";
|
||||
@@ -206,7 +189,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
|
||||
|
||||
// 添加MQTT UDP配置
|
||||
// 从系统参数获取MQTT Gateway地址，仅在配置有效时使用
|
||||
String mqttUdpConfig = sysParamsService.getValue(Constant.SERVER_MQTT_GATEWAY, true);
|
||||
String mqttUdpConfig = sysParamsService.getValue(Constant.SERVER_MQTT_GATEWAY, false);
|
||||
if (mqttUdpConfig != null && !mqttUdpConfig.equals("null") && !mqttUdpConfig.isEmpty()) {
|
||||
try {
|
||||
String groupId = deviceById != null && deviceById.getBoard() != null ? deviceById.getBoard()
|
||||
@@ -511,40 +494,6 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
|
||||
return Base64.getEncoder().encodeToString(signature);
|
||||
}
|
||||
|
||||
/**
|
||||
* 生成WebSocket认证token 遵循Python端AuthManager的实现逻辑：token = signature.timestamp
|
||||
*
|
||||
* @param clientId 客户端ID
|
||||
* @param username 用户名 (通常为deviceId/macAddress)
|
||||
* @return 认证token字符串
|
||||
*/
|
||||
private String generateWebSocketToken(String clientId, String username)
|
||||
throws NoSuchAlgorithmException, InvalidKeyException {
|
||||
// 从系统参数获取密钥
|
||||
String secretKey = sysParamsService.getValue(Constant.SERVER_SECRET, false);
|
||||
if (StringUtils.isBlank(secretKey)) {
|
||||
throw new IllegalStateException("WebSocket认证密钥未配置(server.secret)");
|
||||
}
|
||||
|
||||
// 获取当前时间戳(秒)
|
||||
long timestamp = System.currentTimeMillis() / 1000;
|
||||
|
||||
// 构建签名内容: clientId|username|timestamp
|
||||
String content = String.format("%s|%s|%d", clientId, username, timestamp);
|
||||
|
||||
// 生成HMAC-SHA256签名
|
||||
Mac hmac = Mac.getInstance("HmacSHA256");
|
||||
SecretKeySpec keySpec = new SecretKeySpec(secretKey.getBytes(StandardCharsets.UTF_8), "HmacSHA256");
|
||||
hmac.init(keySpec);
|
||||
byte[] signature = hmac.doFinal(content.getBytes(StandardCharsets.UTF_8));
|
||||
|
||||
// Base64 URL-safe编码签名(去除填充符=)
|
||||
String signatureBase64 = Base64.getUrlEncoder().withoutPadding().encodeToString(signature);
|
||||
|
||||
// 返回格式: signature.timestamp
|
||||
return String.format("%s.%d", signatureBase64, timestamp);
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建MQTT配置信息
|
||||
*
|
||||
@@ -555,7 +504,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
|
||||
private DeviceReportRespDTO.MQTT buildMqttConfig(String macAddress, String groupId)
|
||||
throws Exception {
|
||||
// 从环境变量或系统参数获取签名密钥
|
||||
String signatureKey = sysParamsService.getValue("server.mqtt_signature_key", true);
|
||||
String signatureKey = sysParamsService.getValue("server.mqtt_signature_key", false);
|
||||
if (StringUtils.isBlank(signatureKey)) {
|
||||
log.warn("缺少MQTT_SIGNATURE_KEY，跳过MQTT配置生成");
|
||||
return null;
|
||||
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
package xiaozhi.modules.knowledge.config;
|
||||
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import xiaozhi.modules.knowledge.rag.KnowledgeBaseAdapterFactory;
|
||||
|
||||
/**
|
||||
* 知识库配置类
|
||||
* 配置知识库相关的Bean
|
||||
*/
|
||||
@Configuration
|
||||
public class KnowledgeBaseConfig {
|
||||
|
||||
/**
|
||||
* 提供KnowledgeBaseAdapterFactory的Bean实例
|
||||
* @return KnowledgeBaseAdapterFactory实例
|
||||
*/
|
||||
@Bean
|
||||
public KnowledgeBaseAdapterFactory knowledgeBaseAdapterFactory() {
|
||||
return new KnowledgeBaseAdapterFactory();
|
||||
}
|
||||
}
|
||||
@@ -77,7 +77,7 @@ public class KnowledgeFilesDTO implements Serializable {
|
||||
return STATUS_UNSTART;
|
||||
}
|
||||
|
||||
// RAGFlow根据run字段的值直接映射到对应的状态码
|
||||
// 根据run字段的值直接映射到对应的状态码
|
||||
switch (run.toUpperCase()) {
|
||||
case "RUNNING":
|
||||
return STATUS_RUNNING;
|
||||
|
||||
-200
@@ -1,200 +0,0 @@
|
||||
package xiaozhi.modules.knowledge.rag;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
||||
import xiaozhi.common.page.PageData;
|
||||
import xiaozhi.modules.knowledge.dto.KnowledgeFilesDTO;
|
||||
|
||||
/**
|
||||
* 知识库API适配器抽象基类
|
||||
* 定义通用的知识库操作接口，支持多种后端API实现
|
||||
*/
|
||||
public abstract class KnowledgeBaseAdapter {
|
||||
|
||||
/**
|
||||
* 获取适配器类型标识
|
||||
*
|
||||
* @return 适配器类型（如：ragflow, milvus, pinecone等）
|
||||
*/
|
||||
public abstract String getAdapterType();
|
||||
|
||||
/**
|
||||
* 初始化适配器配置
|
||||
*
|
||||
* @param config 配置参数
|
||||
*/
|
||||
public abstract void initialize(Map<String, Object> config);
|
||||
|
||||
/**
|
||||
* 验证配置是否有效
|
||||
*
|
||||
* @param config 配置参数
|
||||
* @return 验证结果
|
||||
*/
|
||||
public abstract boolean validateConfig(Map<String, Object> config);
|
||||
|
||||
/**
|
||||
* 分页查询文档列表
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param queryParams 查询参数
|
||||
* @param page 页码
|
||||
* @param limit 每页数量
|
||||
* @return 分页数据
|
||||
*/
|
||||
public abstract PageData<KnowledgeFilesDTO> getDocumentList(String datasetId,
|
||||
Map<String, Object> queryParams,
|
||||
Integer page,
|
||||
Integer limit);
|
||||
|
||||
/**
|
||||
* 根据文档ID获取文档详情
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @return 文档详情
|
||||
*/
|
||||
public abstract KnowledgeFilesDTO getDocumentById(String datasetId, String documentId);
|
||||
|
||||
/**
|
||||
* 上传文档到知识库
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param file 上传的文件
|
||||
* @param name 文档名称
|
||||
* @param metaFields 元数据字段
|
||||
* @param chunkMethod 分块方法
|
||||
* @param parserConfig 解析器配置
|
||||
* @return 上传的文档信息
|
||||
*/
|
||||
public abstract KnowledgeFilesDTO uploadDocument(String datasetId,
|
||||
MultipartFile file,
|
||||
String name,
|
||||
Map<String, Object> metaFields,
|
||||
String chunkMethod,
|
||||
Map<String, Object> parserConfig);
|
||||
|
||||
/**
|
||||
* 根据状态分页查询文档列表
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param status 文档解析状态
|
||||
* @param page 页码
|
||||
* @param limit 每页数量
|
||||
* @return 分页数据
|
||||
*/
|
||||
public abstract PageData<KnowledgeFilesDTO> getDocumentListByStatus(String datasetId,
|
||||
Integer status,
|
||||
Integer page,
|
||||
Integer limit);
|
||||
|
||||
/**
|
||||
* 删除文档
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param documentId 文档ID
|
||||
*/
|
||||
public abstract void deleteDocument(String datasetId, String documentId);
|
||||
|
||||
/**
|
||||
* 解析文档（切块）
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param documentIds 文档ID列表
|
||||
* @return 解析结果
|
||||
*/
|
||||
public abstract boolean parseDocuments(String datasetId, List<String> documentIds);
|
||||
|
||||
/**
|
||||
* 列出指定文档的切片
|
||||
*
|
||||
* @param datasetId 知识库ID
|
||||
* @param documentId 文档ID
|
||||
* @param keywords 关键词过滤
|
||||
* @param page 页码
|
||||
* @param pageSize 每页数量
|
||||
* @param chunkId 切片ID
|
||||
* @return 切片列表信息
|
||||
*/
|
||||
public abstract Map<String, Object> listChunks(String datasetId,
|
||||
String documentId,
|
||||
String keywords,
|
||||
Integer page,
|
||||
Integer pageSize,
|
||||
String chunkId);
|
||||
|
||||
/**
|
||||
* 召回测试 - 从知识库中检索相关切片
|
||||
*
|
||||
* @param question 用户查询
|
||||
* @param datasetIds 数据集ID列表
|
||||
* @param documentIds 文档ID列表
|
||||
* @param retrievalParams 检索参数
|
||||
* @return 召回测试结果
|
||||
*/
|
||||
public abstract Map<String, Object> retrievalTest(String question,
|
||||
List<String> datasetIds,
|
||||
List<String> documentIds,
|
||||
Map<String, Object> retrievalParams);
|
||||
|
||||
/**
|
||||
* 测试连接
|
||||
*
|
||||
* @return 连接测试结果
|
||||
*/
|
||||
public abstract boolean testConnection();
|
||||
|
||||
/**
|
||||
* 获取适配器状态信息
|
||||
*
|
||||
* @return 状态信息
|
||||
*/
|
||||
public abstract Map<String, Object> getStatus();
|
||||
|
||||
/**
|
||||
* 获取支持的配置参数
|
||||
*
|
||||
* @return 配置参数说明
|
||||
*/
|
||||
public abstract Map<String, Object> getSupportedConfig();
|
||||
|
||||
/**
|
||||
* 获取默认配置
|
||||
*
|
||||
* @return 默认配置
|
||||
*/
|
||||
public abstract Map<String, Object> getDefaultConfig();
|
||||
|
||||
/**
|
||||
* 创建数据集
|
||||
*
|
||||
* @param createParams 创建参数
|
||||
* @return 数据集ID
|
||||
*/
|
||||
public abstract String createDataset(Map<String, Object> createParams);
|
||||
|
||||
/**
|
||||
* 更新数据集
|
||||
*
|
||||
* @param datasetId 数据集ID
|
||||
* @param updateParams 更新参数
|
||||
*/
|
||||
public abstract void updateDataset(String datasetId, Map<String, Object> updateParams);
|
||||
|
||||
/**
|
||||
* 删除数据集
|
||||
*
|
||||
* @param datasetId 数据集ID
|
||||
*/
|
||||
public abstract void deleteDataset(String datasetId);
|
||||
|
||||
/**
|
||||
* 获取数据集的文档数量
|
||||
*
|
||||
* @param datasetId 数据集ID
|
||||
* @return 文档数量
|
||||
*/
|
||||
public abstract Integer getDocumentCount(String datasetId);
|
||||
}
|
||||
-197
@@ -1,197 +0,0 @@
|
||||
package xiaozhi.modules.knowledge.rag;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import xiaozhi.common.exception.ErrorCode;
|
||||
import xiaozhi.common.exception.RenException;
|
||||
|
||||
/**
|
||||
* 知识库适配器工厂类
|
||||
* 负责创建和管理不同类型的知识库API适配器
|
||||
*/
|
||||
@Slf4j
|
||||
public class KnowledgeBaseAdapterFactory {
|
||||
|
||||
// 注册的适配器类型映射
|
||||
private static final Map<String, Class<? extends KnowledgeBaseAdapter>> adapterRegistry = new HashMap<>();
|
||||
|
||||
// 适配器实例缓存
|
||||
private static final Map<String, KnowledgeBaseAdapter> adapterCache = new ConcurrentHashMap<>();
|
||||
|
||||
static {
|
||||
// 注册内置适配器类型
|
||||
registerAdapter("ragflow", xiaozhi.modules.knowledge.rag.impl.RAGFlowAdapter.class);
|
||||
// 可以在这里注册更多适配器类型
|
||||
}
|
||||
|
||||
/**
|
||||
* 注册新的适配器类型
|
||||
*
|
||||
* @param adapterType 适配器类型标识
|
||||
* @param adapterClass 适配器类
|
||||
*/
|
||||
public static void registerAdapter(String adapterType, Class<? extends KnowledgeBaseAdapter> adapterClass) {
|
||||
if (adapterRegistry.containsKey(adapterType)) {
|
||||
log.warn("适配器类型 '{}' 已存在，将被覆盖", adapterType);
|
||||
}
|
||||
adapterRegistry.put(adapterType, adapterClass);
|
||||
log.info("注册适配器类型: {} -> {}", adapterType, adapterClass.getSimpleName());
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取适配器实例
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
* @param config 配置参数
|
||||
* @return 适配器实例
|
||||
*/
|
||||
public static KnowledgeBaseAdapter getAdapter(String adapterType, Map<String, Object> config) {
|
||||
String cacheKey = buildCacheKey(adapterType, config);
|
||||
|
||||
// 检查缓存中是否已存在实例
|
||||
if (adapterCache.containsKey(cacheKey)) {
|
||||
log.debug("从缓存获取适配器实例: {}", cacheKey);
|
||||
return adapterCache.get(cacheKey);
|
||||
}
|
||||
|
||||
// 创建新的适配器实例
|
||||
KnowledgeBaseAdapter adapter = createAdapter(adapterType, config);
|
||||
|
||||
// 缓存适配器实例
|
||||
adapterCache.put(cacheKey, adapter);
|
||||
log.info("创建并缓存适配器实例: {}", cacheKey);
|
||||
|
||||
return adapter;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取适配器实例（无配置）
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
* @return 适配器实例
|
||||
*/
|
||||
public static KnowledgeBaseAdapter getAdapter(String adapterType) {
|
||||
return getAdapter(adapterType, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取所有已注册的适配器类型
|
||||
*
|
||||
* @return 适配器类型集合
|
||||
*/
|
||||
public static Set<String> getRegisteredAdapterTypes() {
|
||||
return adapterRegistry.keySet();
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查适配器类型是否已注册
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
* @return 是否已注册
|
||||
*/
|
||||
public static boolean isAdapterTypeRegistered(String adapterType) {
|
||||
return adapterRegistry.containsKey(adapterType);
|
||||
}
|
||||
|
||||
/**
|
||||
* 清除适配器缓存
|
||||
*/
|
||||
public static void clearCache() {
|
||||
int cacheSize = adapterCache.size();
|
||||
adapterCache.clear();
|
||||
log.info("清除适配器缓存，共清除 {} 个实例", cacheSize);
|
||||
}
|
||||
|
||||
/**
|
||||
* 移除特定适配器类型的缓存
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
*/
|
||||
public static void removeCacheByType(String adapterType) {
|
||||
int removedCount = 0;
|
||||
for (String cacheKey : adapterCache.keySet()) {
|
||||
if (cacheKey.startsWith(adapterType + "@")) {
|
||||
adapterCache.remove(cacheKey);
|
||||
removedCount++;
|
||||
}
|
||||
}
|
||||
log.info("移除适配器类型 '{}' 的缓存，共移除 {} 个实例", adapterType, removedCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取适配器工厂状态信息
|
||||
*
|
||||
* @return 状态信息
|
||||
*/
|
||||
public static Map<String, Object> getFactoryStatus() {
|
||||
Map<String, Object> status = new HashMap<>();
|
||||
status.put("registeredAdapterTypes", adapterRegistry.keySet());
|
||||
status.put("cachedAdapterCount", adapterCache.size());
|
||||
status.put("cacheKeys", adapterCache.keySet());
|
||||
return status;
|
||||
}
|
||||
|
||||
/**
|
||||
* 创建适配器实例
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
* @param config 配置参数
|
||||
* @return 适配器实例
|
||||
*/
|
||||
private static KnowledgeBaseAdapter createAdapter(String adapterType, Map<String, Object> config) {
|
||||
if (!adapterRegistry.containsKey(adapterType)) {
|
||||
throw new RenException(ErrorCode.RAG_ADAPTER_TYPE_NOT_SUPPORTED,
|
||||
"不支持的适配器类型: " + adapterType);
|
||||
}
|
||||
|
||||
try {
|
||||
Class<? extends KnowledgeBaseAdapter> adapterClass = adapterRegistry.get(adapterType);
|
||||
KnowledgeBaseAdapter adapter = adapterClass.getDeclaredConstructor().newInstance();
|
||||
|
||||
// 初始化适配器
|
||||
if (config != null) {
|
||||
adapter.initialize(config);
|
||||
|
||||
// 验证配置
|
||||
if (!adapter.validateConfig(config)) {
|
||||
throw new RenException(ErrorCode.RAG_CONFIG_VALIDATION_FAILED,
|
||||
"适配器配置验证失败: " + adapterType);
|
||||
}
|
||||
}
|
||||
|
||||
log.info("成功创建适配器实例: {}", adapterType);
|
||||
return adapter;
|
||||
|
||||
} catch (Exception e) {
|
||||
log.error("创建适配器实例失败: {}", adapterType, e);
|
||||
throw new RenException(ErrorCode.RAG_ADAPTER_CREATION_FAILED,
|
||||
"创建适配器失败: " + adapterType + ", 错误: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建缓存键
|
||||
*
|
||||
* @param adapterType 适配器类型
|
||||
* @param config 配置参数
|
||||
* @return 缓存键
|
||||
*/
|
||||
private static String buildCacheKey(String adapterType, Map<String, Object> config) {
|
||||
if (config == null || config.isEmpty()) {
|
||||
return adapterType + "@default";
|
||||
}
|
||||
|
||||
// 基于配置参数生成缓存键
|
||||
StringBuilder keyBuilder = new StringBuilder(adapterType + "@");
|
||||
|
||||
// 使用配置的哈希值作为缓存键的一部分
|
||||
int configHash = config.hashCode();
|
||||
keyBuilder.append(configHash);
|
||||
|
||||
return keyBuilder.toString();
|
||||
}
|
||||
}
|
||||
-1155
File diff suppressed because it is too large
Load Diff
+346
-142
@@ -35,8 +35,6 @@ import xiaozhi.common.utils.MessageUtils;
|
||||
import xiaozhi.modules.knowledge.dao.KnowledgeBaseDao;
|
||||
import xiaozhi.modules.knowledge.dto.KnowledgeBaseDTO;
|
||||
import xiaozhi.modules.knowledge.entity.KnowledgeBaseEntity;
|
||||
import xiaozhi.modules.knowledge.rag.KnowledgeBaseAdapter;
|
||||
import xiaozhi.modules.knowledge.rag.KnowledgeBaseAdapterFactory;
|
||||
import xiaozhi.modules.knowledge.service.KnowledgeBaseService;
|
||||
import xiaozhi.modules.model.dao.ModelConfigDao;
|
||||
import xiaozhi.modules.model.entity.ModelConfigEntity;
|
||||
@@ -108,7 +106,7 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
if (pageData != null && pageData.getList() != null) {
|
||||
for (KnowledgeBaseDTO knowledgeBase : pageData.getList()) {
|
||||
try {
|
||||
Integer documentCount = getDocumentCountFromRAG(knowledgeBase.getDatasetId(),
|
||||
Integer documentCount = getDocumentCountFromRAGFlow(knowledgeBase.getDatasetId(),
|
||||
knowledgeBase.getRagModelId());
|
||||
knowledgeBase.setDocumentCount(documentCount);
|
||||
} catch (Exception e) {
|
||||
@@ -148,10 +146,10 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
checkDuplicateKnowledgeBaseName(knowledgeBaseDTO, null);
|
||||
|
||||
String datasetId = null;
|
||||
// 调用RAG API创建数据集
|
||||
// 调用RAGFlow API创建数据集
|
||||
try {
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(knowledgeBaseDTO.getRagModelId());
|
||||
datasetId = createDatasetInRAG(
|
||||
datasetId = createDatasetInRAGFlow(
|
||||
knowledgeBaseDTO.getName(),
|
||||
knowledgeBaseDTO.getDescription(),
|
||||
ragConfig);
|
||||
@@ -164,13 +162,13 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
KnowledgeBaseEntity existingEntity = knowledgeBaseDao.selectOne(
|
||||
new QueryWrapper<KnowledgeBaseEntity>().eq("dataset_id", datasetId));
|
||||
if (existingEntity != null) {
|
||||
// 如果datasetId已存在，删除RAG中的数据集并抛出异常
|
||||
// 如果datasetId已存在，删除RAGFlow中的数据集并抛出异常
|
||||
try {
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(knowledgeBaseDTO.getRagModelId());
|
||||
deleteDatasetInRAG(datasetId, ragConfig);
|
||||
deleteDatasetInRAGFlow(datasetId, ragConfig);
|
||||
} catch (Exception deleteException) {
|
||||
// 提供更详细的错误信息，包括异常类型和消息
|
||||
String errorMessage = "删除重复datasetId的RAG数据集失败: " + deleteException.getClass().getSimpleName();
|
||||
String errorMessage = "删除重复datasetId的RAGFlow数据集失败: " + deleteException.getClass().getSimpleName();
|
||||
if (deleteException.getMessage() != null) {
|
||||
errorMessage += " - " + deleteException.getMessage();
|
||||
}
|
||||
@@ -221,17 +219,17 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
// 先校验RAG配置
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(knowledgeBaseDTO.getRagModelId());
|
||||
|
||||
// 调用RAG API更新数据集
|
||||
updateDatasetInRAG(
|
||||
// 调用RAGFlow API更新数据集
|
||||
updateDatasetInRAGFlow(
|
||||
knowledgeBaseDTO.getDatasetId(),
|
||||
knowledgeBaseDTO.getName(),
|
||||
knowledgeBaseDTO.getDescription(),
|
||||
ragConfig);
|
||||
|
||||
log.info("RAG API更新成功，datasetId: {}", knowledgeBaseDTO.getDatasetId());
|
||||
log.info("RAGFlow API更新成功，datasetId: {}", knowledgeBaseDTO.getDatasetId());
|
||||
} catch (Exception e) {
|
||||
// 提供更详细的错误信息，包括异常类型和消息
|
||||
String errorMessage = "更新RAG数据集失败: " + e.getClass().getSimpleName();
|
||||
String errorMessage = "更新RAGFlow数据集失败: " + e.getClass().getSimpleName();
|
||||
if (e.getMessage() != null) {
|
||||
errorMessage += " - " + e.getMessage();
|
||||
}
|
||||
@@ -239,7 +237,7 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
throw e;
|
||||
}
|
||||
} else {
|
||||
log.warn("datasetId或ragModelId为空，跳过RAG更新");
|
||||
log.warn("datasetId或ragModelId为空，跳过RAGFlow更新");
|
||||
}
|
||||
|
||||
KnowledgeBaseEntity entity = ConvertUtils.sourceToTarget(knowledgeBaseDTO, KnowledgeBaseEntity.class);
|
||||
@@ -290,19 +288,19 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
log.info("找到记录: ID={}, datasetId={}, ragModelId={}",
|
||||
entity.getId(), entity.getDatasetId(), entity.getRagModelId());
|
||||
|
||||
// 先调用RAG API删除数据集
|
||||
// 先调用RAGFlow API删除数据集
|
||||
boolean apiDeleteSuccess = false;
|
||||
if (StringUtils.isNotBlank(entity.getDatasetId()) && StringUtils.isNotBlank(entity.getRagModelId())) {
|
||||
try {
|
||||
log.info("开始调用RAG API删除数据集");
|
||||
log.info("开始调用RAGFlow API删除数据集");
|
||||
// 在删除前进行RAG配置校验
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(entity.getRagModelId());
|
||||
deleteDatasetInRAG(entity.getDatasetId(), ragConfig);
|
||||
log.info("RAG API删除调用完成");
|
||||
deleteDatasetInRAGFlow(entity.getDatasetId(), ragConfig);
|
||||
log.info("RAGFlow API删除调用完成");
|
||||
apiDeleteSuccess = true;
|
||||
} catch (Exception e) {
|
||||
// 提供更详细的错误信息，包括异常类型和消息
|
||||
String errorMessage = "删除RAG数据集失败: " + e.getClass().getSimpleName();
|
||||
String errorMessage = "删除RAGFlow数据集失败: " + e.getClass().getSimpleName();
|
||||
if (e.getMessage() != null) {
|
||||
errorMessage += " - " + e.getMessage();
|
||||
}
|
||||
@@ -310,7 +308,7 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
throw e;
|
||||
}
|
||||
} else {
|
||||
log.warn("datasetId或ragModelId为空，跳过RAG删除");
|
||||
log.warn("datasetId或ragModelId为空，跳过RAGFlow删除");
|
||||
apiDeleteSuccess = true; // 没有RAG数据集，视为成功
|
||||
}
|
||||
|
||||
@@ -434,133 +432,289 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
}
|
||||
|
||||
/**
|
||||
* 从RAG配置中提取适配器类型
|
||||
*
|
||||
* @param config RAG配置
|
||||
* @return 适配器类型
|
||||
* 调用RAGFlow API创建数据集
|
||||
*/
|
||||
private String extractAdapterType(Map<String, Object> config) {
|
||||
if (config == null) {
|
||||
throw new RenException(ErrorCode.RAG_CONFIG_NOT_FOUND);
|
||||
private String createDatasetInRAGFlow(String name, String description, Map<String, Object> ragConfig) {
|
||||
String datasetId = null;
|
||||
String baseUrl = (String) ragConfig.get("base_url");
|
||||
String apiKey = (String) ragConfig.get("api_key");
|
||||
|
||||
log.info("开始调用RAGFlow API创建数据集, name: {}", name);
|
||||
log.debug("RAGFlow配置 - baseUrl: {}, apiKey: {}", baseUrl, StringUtils.isBlank(apiKey) ? "未配置" : "已配置");
|
||||
|
||||
// 构建请求URL
|
||||
String url = baseUrl + "/api/v1/datasets";
|
||||
log.debug("请求URL: {}", url);
|
||||
|
||||
// 构建请求体
|
||||
Map<String, Object> requestBody = new HashMap<>();
|
||||
String username = SecurityUser.getUser().getUsername();
|
||||
requestBody.put("name", username + "_" + name);
|
||||
if (StringUtils.isNotBlank(description)) {
|
||||
requestBody.put("description", description);
|
||||
}
|
||||
log.debug("请求体: {}", requestBody);
|
||||
|
||||
// 从配置中提取适配器类型
|
||||
String adapterType = (String) config.get("type");
|
||||
// 设置请求头
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_JSON);
|
||||
headers.set("Authorization", "Bearer " + apiKey);
|
||||
|
||||
// 验证适配器类型是否存在且非空
|
||||
if (StringUtils.isBlank(adapterType)) {
|
||||
throw new RenException(ErrorCode.RAG_ADAPTER_TYPE_NOT_FOUND);
|
||||
}
|
||||
|
||||
// 验证适配器类型是否已注册
|
||||
if (!KnowledgeBaseAdapterFactory.isAdapterTypeRegistered(adapterType)) {
|
||||
throw new RenException(ErrorCode.RAG_ADAPTER_TYPE_NOT_SUPPORTED,
|
||||
"不支持的适配器类型: " + adapterType);
|
||||
}
|
||||
|
||||
return adapterType;
|
||||
}
|
||||
|
||||
/**
|
||||
* 使用适配器创建数据集
|
||||
*/
|
||||
private String createDatasetInRAG(String name, String description, Map<String, Object> ragConfig) {
|
||||
log.info("开始使用适配器创建数据集, name: {}", name);
|
||||
HttpEntity<Map<String, Object>> requestEntity = new HttpEntity<>(requestBody, headers);
|
||||
|
||||
// 发送POST请求
|
||||
log.info("发送POST请求到RAGFlow API...");
|
||||
ResponseEntity<String> response;
|
||||
try {
|
||||
// 从RAG配置中提取适配器类型
|
||||
String adapterType = extractAdapterType(ragConfig);
|
||||
|
||||
// 使用适配器工厂获取适配器实例
|
||||
KnowledgeBaseAdapter adapter = KnowledgeBaseAdapterFactory.getAdapter(adapterType, ragConfig);
|
||||
|
||||
// 构建数据集创建参数
|
||||
Map<String, Object> createParams = new HashMap<>();
|
||||
String username = SecurityUser.getUser().getUsername();
|
||||
createParams.put("name", username + "_" + name);
|
||||
if (StringUtils.isNotBlank(description)) {
|
||||
createParams.put("description", description);
|
||||
}
|
||||
|
||||
// 调用适配器的创建数据集方法
|
||||
String datasetId = adapter.createDataset(createParams);
|
||||
|
||||
log.info("数据集创建成功，datasetId: {}", datasetId);
|
||||
return datasetId;
|
||||
|
||||
response = restTemplate.exchange(url, HttpMethod.POST, requestEntity, String.class);
|
||||
} catch (Exception e) {
|
||||
// 直接传递底层适配器的详细错误信息
|
||||
log.error("创建数据集失败", e);
|
||||
if (e instanceof RenException) {
|
||||
throw (RenException) e;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, e.getMessage());
|
||||
String errorMessage = url + e.getMessage();
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
log.info("RAGFlow API响应状态码: {}", response.getStatusCode());
|
||||
log.debug("RAGFlow API响应内容: {}", response.getBody());
|
||||
|
||||
if (!response.getStatusCode().is2xxSuccessful()) {
|
||||
String errorMessage = response.getStatusCode() + ", 响应内容: " + response.getBody();
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
// 解析响应体，提取datasetId
|
||||
String responseBody = response.getBody();
|
||||
if (StringUtils.isNotBlank(responseBody)) {
|
||||
try {
|
||||
// 解析RAGFlow API响应，支持多种可能的字段名
|
||||
ObjectMapper objectMapper = new ObjectMapper();
|
||||
Map<String, Object> responseMap = objectMapper.readValue(responseBody, Map.class);
|
||||
|
||||
log.debug("RAGFlow API响应解析结果: {}", responseMap);
|
||||
|
||||
// 首先检查响应码
|
||||
Integer code = (Integer) responseMap.get("code");
|
||||
String message = (String) responseMap.get("message");
|
||||
if (code != null && code == 0) {
|
||||
// 响应码为0表示成功，从data字段中获取datasetId
|
||||
Object dataObj = responseMap.get("data");
|
||||
if (dataObj instanceof Map) {
|
||||
Map<String, Object> dataMap = (Map<String, Object>) dataObj;
|
||||
datasetId = (String) dataMap.get("id");
|
||||
|
||||
if (StringUtils.isBlank(datasetId)) {
|
||||
// 如果id字段为空，尝试其他可能的字段名
|
||||
datasetId = (String) dataMap.get("dataset_id");
|
||||
datasetId = (String) dataMap.get("datasetId");
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// 如果响应码不为0，说明API调用失败
|
||||
String errorMessage = code + (message != null ? message : "null");
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
log.info("从RAGFlow API响应中解析出datasetId: {}", datasetId);
|
||||
log.debug("完整响应内容: {}", responseBody);
|
||||
} catch (IOException e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
} catch (Exception e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
// 如果解析失败，但响应体不为空，尝试直接使用响应体作为错误信息
|
||||
String finalErrorMessage = responseBody;
|
||||
if (e.getMessage() != null) {
|
||||
finalErrorMessage += errorMessage;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, finalErrorMessage);
|
||||
}
|
||||
}
|
||||
|
||||
if (StringUtils.isBlank(datasetId)) {
|
||||
log.error("无法从RAGFlow API响应中获取datasetId，响应内容: {}", responseBody);
|
||||
throw new RenException(ErrorCode.RAG_DATASET_ID_NOT_NULL);
|
||||
}
|
||||
log.info("RAGFlow数据集创建成功，datasetId: {}", datasetId);
|
||||
|
||||
return datasetId;
|
||||
}
|
||||
|
||||
/**
|
||||
* 使用适配器更新数据集
|
||||
* 调用RAGFlow API更新数据集
|
||||
*/
|
||||
private void updateDatasetInRAG(String datasetId, String name, String description,
|
||||
private void updateDatasetInRAGFlow(String datasetId, String name, String description,
|
||||
Map<String, Object> ragConfig) {
|
||||
log.info("开始使用适配器更新数据集，datasetId: {}, name: {}", datasetId, name);
|
||||
String baseUrl = (String) ragConfig.get("base_url");
|
||||
String apiKey = (String) ragConfig.get("api_key");
|
||||
|
||||
try {
|
||||
// 从RAG配置中提取适配器类型
|
||||
String adapterType = extractAdapterType(ragConfig);
|
||||
log.info("开始调用RAGFlow API更新数据集，datasetId: {}, name: {}", datasetId, name);
|
||||
log.debug("RAGFlow配置 - baseUrl: {}, apiKey: {}", baseUrl, StringUtils.isBlank(apiKey) ? "未配置" : "已配置");
|
||||
|
||||
// 使用适配器工厂获取适配器实例
|
||||
KnowledgeBaseAdapter adapter = KnowledgeBaseAdapterFactory.getAdapter(adapterType, ragConfig);
|
||||
|
||||
// 构建数据集更新参数
|
||||
Map<String, Object> updateParams = new HashMap<>();
|
||||
String username = SecurityUser.getUser().getUsername();
|
||||
updateParams.put("name", username + "_" + name);
|
||||
if (StringUtils.isNotBlank(description)) {
|
||||
updateParams.put("description", description);
|
||||
}
|
||||
|
||||
// 调用适配器的更新数据集方法
|
||||
adapter.updateDataset(datasetId, updateParams);
|
||||
|
||||
log.info("数据集更新成功，datasetId: {}", datasetId);
|
||||
|
||||
} catch (Exception e) {
|
||||
// 直接传递底层适配器的详细错误信息
|
||||
log.error("更新数据集失败", e);
|
||||
if (e instanceof RenException) {
|
||||
throw (RenException) e;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, e.getMessage());
|
||||
// 构建请求URL
|
||||
String url = baseUrl + "/api/v1/datasets/" + datasetId;
|
||||
log.debug("请求URL: {}", url);
|
||||
// 构建请求体
|
||||
Map<String, Object> requestBody = new HashMap<>();
|
||||
requestBody.put("dataset_id", datasetId);
|
||||
String username = SecurityUser.getUser().getUsername();
|
||||
requestBody.put("name", username + "_" + name);
|
||||
if (StringUtils.isNotBlank(description)) {
|
||||
requestBody.put("description", description);
|
||||
}
|
||||
log.debug("请求体: {}", requestBody);
|
||||
|
||||
// 设置请求头
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_JSON);
|
||||
headers.set("Authorization", "Bearer " + apiKey);
|
||||
|
||||
HttpEntity<Map<String, Object>> requestEntity = new HttpEntity<>(requestBody, headers);
|
||||
|
||||
// 发送PUT请求
|
||||
log.info("发送PUT请求到RAGFlow API...");
|
||||
ResponseEntity<String> response;
|
||||
try {
|
||||
response = restTemplate.exchange(url, HttpMethod.PUT, requestEntity, String.class);
|
||||
} catch (Exception e) {
|
||||
String errorMessage = url + e.getMessage();
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
log.info("RAGFlow API响应状态码: {}", response.getStatusCode());
|
||||
log.debug("RAGFlow API响应内容: {}", response.getBody());
|
||||
|
||||
if (!response.getStatusCode().is2xxSuccessful()) {
|
||||
String errorMessage = response.getStatusCode() + ", 响应内容: " + response.getBody();
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
// 解析响应体，验证操作是否真正成功
|
||||
String responseBody = response.getBody();
|
||||
if (responseBody != null) {
|
||||
try {
|
||||
Map<String, Object> responseMap = objectMapper.readValue(responseBody, Map.class);
|
||||
Integer code = (Integer) responseMap.get("code");
|
||||
String message = (String) responseMap.get("message");
|
||||
|
||||
if (code != null && code != 0) {
|
||||
String errorMessage = code + (message != null ? message : "null");
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
} catch (Exception e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
// 如果解析失败，但响应体不为空，尝试直接使用响应体作为错误信息
|
||||
String finalErrorMessage = responseBody;
|
||||
if (e.getMessage() != null) {
|
||||
finalErrorMessage += errorMessage;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, finalErrorMessage);
|
||||
}
|
||||
}
|
||||
|
||||
log.info("RAGFlow数据集更新成功，datasetId: {}", datasetId);
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* 使用适配器删除数据集
|
||||
* 调用RAGFlow API删除数据集
|
||||
*/
|
||||
private void deleteDatasetInRAG(String datasetId, Map<String, Object> ragConfig) {
|
||||
log.info("开始使用适配器删除数据集，datasetId: {}", datasetId);
|
||||
private void deleteDatasetInRAGFlow(String datasetId, Map<String, Object> ragConfig) {
|
||||
String baseUrl = (String) ragConfig.get("base_url");
|
||||
String apiKey = (String) ragConfig.get("api_key");
|
||||
|
||||
log.info("开始调用RAGFlow API删除数据集，datasetId: {}", datasetId);
|
||||
log.debug("RAGFlow配置 - baseUrl: {}, apiKey: {}", baseUrl, StringUtils.isBlank(apiKey) ? "未配置" : "已配置");
|
||||
|
||||
// 构建请求URL
|
||||
String url = baseUrl + "/api/v1/datasets";
|
||||
log.debug("请求URL: {}", url);
|
||||
|
||||
// 构建请求体
|
||||
Map<String, Object> requestBody = new HashMap<>();
|
||||
requestBody.put("ids", List.of(datasetId));
|
||||
log.debug("请求体: {}", requestBody);
|
||||
|
||||
// 设置请求头
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_JSON);
|
||||
headers.set("Authorization", "Bearer " + apiKey);
|
||||
|
||||
HttpEntity<Map<String, Object>> requestEntity = new HttpEntity<>(requestBody, headers);
|
||||
|
||||
// 发送DELETE请求
|
||||
log.info("发送DELETE请求到RAGFlow API...");
|
||||
ResponseEntity<String> response;
|
||||
try {
|
||||
// 从RAG配置中提取适配器类型
|
||||
String adapterType = extractAdapterType(ragConfig);
|
||||
|
||||
// 使用适配器工厂获取适配器实例
|
||||
KnowledgeBaseAdapter adapter = KnowledgeBaseAdapterFactory.getAdapter(adapterType, ragConfig);
|
||||
|
||||
// 调用适配器的删除数据集方法
|
||||
adapter.deleteDataset(datasetId);
|
||||
|
||||
log.info("数据集删除成功，datasetId: {}", datasetId);
|
||||
|
||||
response = restTemplate.exchange(url, HttpMethod.DELETE, requestEntity, String.class);
|
||||
} catch (Exception e) {
|
||||
// 直接传递底层适配器的详细错误信息
|
||||
log.error("删除数据集失败", e);
|
||||
if (e instanceof RenException) {
|
||||
throw (RenException) e;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, e.getMessage());
|
||||
String errorMessage = url + e.getMessage();
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
log.info("RAGFlow API响应状态码: {}", response.getStatusCode());
|
||||
log.debug("RAGFlow API响应内容: {}", response.getBody());
|
||||
|
||||
if (!response.getStatusCode().is2xxSuccessful()) {
|
||||
String errorMessage = response.getStatusCode() + response.getBody();
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
|
||||
// 解析响应体，验证操作是否真正成功
|
||||
String responseBody = response.getBody();
|
||||
if (responseBody != null) {
|
||||
try {
|
||||
Map<String, Object> responseMap = objectMapper.readValue(responseBody, Map.class);
|
||||
Integer code = (Integer) responseMap.get("code");
|
||||
String message = (String) responseMap.get("message");
|
||||
|
||||
if (code != null && code != 0) {
|
||||
String errorMessage = code + (message != null ? message : "null");
|
||||
log.error(errorMessage);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, errorMessage);
|
||||
} catch (Exception e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName();
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
// 如果解析失败，但响应体不为空，尝试直接使用响应体作为错误信息
|
||||
String finalErrorMessage = responseBody;
|
||||
if (e.getMessage() != null) {
|
||||
finalErrorMessage += errorMessage;
|
||||
}
|
||||
throw new RenException(ErrorCode.RAG_API_ERROR, finalErrorMessage);
|
||||
}
|
||||
}
|
||||
|
||||
log.info("RAGFlow数据集删除成功，datasetId: {}", datasetId);
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -606,9 +760,9 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
}
|
||||
|
||||
/**
|
||||
* 从适配器获取知识库的文档数量
|
||||
* 从RAGFlow API获取知识库的文档数量
|
||||
*/
|
||||
private Integer getDocumentCountFromRAG(String datasetId, String ragModelId) {
|
||||
private Integer getDocumentCountFromRAGFlow(String datasetId, String ragModelId) {
|
||||
if (StringUtils.isBlank(datasetId) || StringUtils.isBlank(ragModelId)) {
|
||||
log.warn("datasetId或ragModelId为空，无法获取文档数量");
|
||||
return 0;
|
||||
@@ -616,29 +770,79 @@ public class KnowledgeBaseServiceImpl extends BaseServiceImpl<KnowledgeBaseDao,
|
||||
|
||||
log.info("开始获取知识库 {} 的文档数量", datasetId);
|
||||
|
||||
// 获取RAG配置
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(ragModelId);
|
||||
String baseUrl = (String) ragConfig.get("base_url");
|
||||
String apiKey = (String) ragConfig.get("api_key");
|
||||
|
||||
// 构建请求URL - 调用RAGFlow API获取文档列表，但不返回文档详情，只获取总数
|
||||
String url = baseUrl + "/api/v1/datasets/" + datasetId + "/documents?page=1&size=1";
|
||||
log.debug("请求URL: {}", url);
|
||||
|
||||
// 设置请求头
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_JSON);
|
||||
headers.set("Authorization", "Bearer " + apiKey);
|
||||
|
||||
HttpEntity<String> requestEntity = new HttpEntity<>(headers);
|
||||
|
||||
// 发送GET请求
|
||||
log.info("发送GET请求到RAGFlow API获取文档数量...");
|
||||
ResponseEntity<String> response;
|
||||
try {
|
||||
// 获取RAG配置
|
||||
Map<String, Object> ragConfig = getValidatedRAGConfig(ragModelId);
|
||||
|
||||
// 从RAG配置中提取适配器类型
|
||||
String adapterType = extractAdapterType(ragConfig);
|
||||
|
||||
// 使用适配器工厂获取适配器实例
|
||||
KnowledgeBaseAdapter adapter = KnowledgeBaseAdapterFactory.getAdapter(adapterType, ragConfig);
|
||||
|
||||
// 调用适配器的获取文档数量方法
|
||||
Integer documentCount = adapter.getDocumentCount(datasetId);
|
||||
|
||||
log.info("获取知识库 {} 的文档数量成功: {}", datasetId, documentCount);
|
||||
return documentCount;
|
||||
|
||||
response = restTemplate.exchange(url, HttpMethod.GET, requestEntity, String.class);
|
||||
} catch (Exception e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName() + " - 获取知识库文档数量失败";
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
String errorMessage = url + e.getMessage();
|
||||
log.error(errorMessage, e);
|
||||
return 0;
|
||||
}
|
||||
|
||||
log.info("RAGFlow API响应状态码: {}", response.getStatusCode());
|
||||
|
||||
if (!response.getStatusCode().is2xxSuccessful()) {
|
||||
log.error("RAGFlow API调用失败，状态码: {}, 响应内容: {}", response.getStatusCode(), response.getBody());
|
||||
return 0;
|
||||
}
|
||||
|
||||
String responseBody = response.getBody();
|
||||
log.debug("RAGFlow API响应内容: {}", responseBody);
|
||||
|
||||
// 解析响应
|
||||
try {
|
||||
ObjectMapper objectMapper = new ObjectMapper();
|
||||
Map<String, Object> responseMap = objectMapper.readValue(responseBody, Map.class);
|
||||
Integer code = (Integer) responseMap.get("code");
|
||||
|
||||
if (code != null && code == 0) {
|
||||
Object dataObj = responseMap.get("data");
|
||||
if (dataObj instanceof Map) {
|
||||
Map<String, Object> dataMap = (Map<String, Object>) dataObj;
|
||||
Object totalObj = dataMap.get("total");
|
||||
if (totalObj instanceof Integer) {
|
||||
Integer documentCount = (Integer) totalObj;
|
||||
log.info("获取知识库 {} 的文档数量成功: {}", datasetId, documentCount);
|
||||
return documentCount;
|
||||
} else if (totalObj instanceof Long) {
|
||||
Long documentCount = (Long) totalObj;
|
||||
log.info("获取知识库 {} 的文档数量成功: {}", datasetId, documentCount);
|
||||
return documentCount.intValue();
|
||||
}
|
||||
}
|
||||
} else {
|
||||
log.error("RAGFlow API调用失败，响应码: {}, 响应内容: {}", code, responseBody);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName() + " - 解析RAGFlow API响应时发生IO异常";
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
} catch (Exception e) {
|
||||
// 构建详细的错误信息，包含异常类型和消息
|
||||
String baseErrorMessage = e.getClass().getSimpleName() + " - 解析RAGFlow API响应失败";
|
||||
String errorMessage = baseErrorMessage + (e.getMessage() != null ? ": " + e.getMessage() : "");
|
||||
log.error(errorMessage, e);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
}
|
||||
+569
-242
File diff suppressed because it is too large
Load Diff
+14
-19
@@ -6,7 +6,6 @@ import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.PutMapping;
|
||||
@@ -24,9 +23,7 @@ import xiaozhi.common.exception.ErrorCode;
|
||||
import xiaozhi.common.exception.RenException;
|
||||
import xiaozhi.common.page.TokenDTO;
|
||||
import xiaozhi.common.user.UserDetail;
|
||||
import xiaozhi.common.utils.JsonUtils;
|
||||
import xiaozhi.common.utils.Result;
|
||||
import xiaozhi.common.utils.Sm2DecryptUtil;
|
||||
import xiaozhi.common.validator.AssertUtils;
|
||||
import xiaozhi.common.validator.ValidatorUtils;
|
||||
import xiaozhi.modules.security.dto.LoginDTO;
|
||||
@@ -35,6 +32,8 @@ import xiaozhi.modules.security.password.PasswordUtils;
|
||||
import xiaozhi.modules.security.service.CaptchaService;
|
||||
import xiaozhi.modules.security.service.SysUserTokenService;
|
||||
import xiaozhi.modules.security.user.SecurityUser;
|
||||
import xiaozhi.common.utils.Sm2DecryptUtil;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import xiaozhi.modules.sys.dto.PasswordDTO;
|
||||
import xiaozhi.modules.sys.dto.RetrievePasswordDTO;
|
||||
import xiaozhi.modules.sys.dto.SysUserDTO;
|
||||
@@ -90,13 +89,13 @@ public class LoginController {
|
||||
@Operation(summary = "登录")
|
||||
public Result<TokenDTO> login(@RequestBody LoginDTO login) {
|
||||
String password = login.getPassword();
|
||||
|
||||
|
||||
// 使用工具类解密并验证验证码
|
||||
String actualPassword = Sm2DecryptUtil.decryptAndValidateCaptcha(
|
||||
password, login.getCaptchaId(), captchaService, sysParamsService);
|
||||
|
||||
|
||||
login.setPassword(actualPassword);
|
||||
|
||||
|
||||
// 按照用户名获取用户
|
||||
SysUserDTO userDTO = sysUserService.getByUsername(login.getUsername());
|
||||
// 判断用户是否存在
|
||||
@@ -109,6 +108,8 @@ public class LoginController {
|
||||
}
|
||||
return sysUserTokenService.createToken(userDTO.getId());
|
||||
}
|
||||
|
||||
|
||||
|
||||
@PostMapping("/register")
|
||||
@Operation(summary = "注册")
|
||||
@@ -116,15 +117,15 @@ public class LoginController {
|
||||
if (!sysUserService.getAllowUserRegister()) {
|
||||
throw new RenException(ErrorCode.USER_REGISTER_DISABLED);
|
||||
}
|
||||
|
||||
|
||||
String password = login.getPassword();
|
||||
|
||||
|
||||
// 使用工具类解密并验证验证码
|
||||
String actualPassword = Sm2DecryptUtil.decryptAndValidateCaptcha(
|
||||
password, login.getCaptchaId(), captchaService, sysParamsService);
|
||||
|
||||
|
||||
login.setPassword(actualPassword);
|
||||
|
||||
|
||||
// 是否开启手机注册
|
||||
Boolean isMobileRegister = sysParamsService
|
||||
.getValueObject(Constant.SysMSMParam.SERVER_ENABLE_MOBILE_REGISTER.getValue(), Boolean.class);
|
||||
@@ -203,11 +204,11 @@ public class LoginController {
|
||||
}
|
||||
|
||||
String password = dto.getPassword();
|
||||
|
||||
|
||||
// 使用工具类解密并验证验证码
|
||||
String actualPassword = Sm2DecryptUtil.decryptAndValidateCaptcha(
|
||||
password, dto.getCaptchaId(), captchaService, sysParamsService);
|
||||
|
||||
|
||||
dto.setPassword(actualPassword);
|
||||
|
||||
sysUserService.changePasswordDirectly(userDTO.getId(), dto.getPassword());
|
||||
@@ -228,7 +229,7 @@ public class LoginController {
|
||||
config.put("beianIcpNum", sysParamsService.getValue(Constant.SysBaseParam.BEIAN_ICP_NUM.getValue(), true));
|
||||
config.put("beianGaNum", sysParamsService.getValue(Constant.SysBaseParam.BEIAN_GA_NUM.getValue(), true));
|
||||
config.put("name", sysParamsService.getValue(Constant.SysBaseParam.SERVER_NAME.getValue(), true));
|
||||
|
||||
|
||||
// SM2公钥
|
||||
String publicKey = sysParamsService.getValue(Constant.SM2_PUBLIC_KEY, true);
|
||||
if (StringUtils.isBlank(publicKey)) {
|
||||
@@ -236,12 +237,6 @@ public class LoginController {
|
||||
}
|
||||
config.put("sm2PublicKey", publicKey);
|
||||
|
||||
// 获取system-web.menu参数配置
|
||||
String menuConfig = sysParamsService.getValue("system-web.menu", true);
|
||||
if (StringUtils.isNotBlank(menuConfig)) {
|
||||
config.put("systemWebMenu", JsonUtils.parseObject(menuConfig, Object.class));
|
||||
}
|
||||
|
||||
return new Result<Map<String, Object>>().ok(config);
|
||||
}
|
||||
}
|
||||
+3
-3
@@ -174,7 +174,7 @@ public class SysParamsController {
|
||||
return;
|
||||
}
|
||||
if (StringUtils.isBlank(url) || url.equals("null")) {
|
||||
return;
|
||||
throw new RenException(ErrorCode.OTA_URL_EMPTY);
|
||||
}
|
||||
|
||||
// 检查是否包含localhost或127.0.0.1
|
||||
@@ -211,7 +211,7 @@ public class SysParamsController {
|
||||
return;
|
||||
}
|
||||
if (StringUtils.isBlank(url) || url.equals("null")) {
|
||||
return;
|
||||
throw new RenException(ErrorCode.MCP_URL_EMPTY);
|
||||
}
|
||||
if (url.contains("localhost") || url.contains("127.0.0.1")) {
|
||||
throw new RenException(ErrorCode.MCP_URL_LOCALHOST);
|
||||
@@ -242,7 +242,7 @@ public class SysParamsController {
|
||||
return;
|
||||
}
|
||||
if (StringUtils.isBlank(url) || url.equals("null")) {
|
||||
return;
|
||||
throw new RenException(ErrorCode.VOICEPRINT_URL_EMPTY);
|
||||
}
|
||||
if (url.contains("localhost") || url.contains("127.0.0.1")) {
|
||||
throw new RenException(ErrorCode.VOICEPRINT_URL_LOCALHOST);
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
-- 添加系统功能菜单配置参数
|
||||
delete from `sys_params` where param_code = 'system-web.menu';
|
||||
|
||||
-- 添加系统功能菜单配置参数
|
||||
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES
|
||||
(600, 'system-web.menu', '{"features":{"voiceprintRecognition":{"name":"feature.voiceprintRecognition.name","enabled":false,"description":"feature.voiceprintRecognition.description"},"voiceClone":{"name":"feature.voiceClone.name","enabled":false,"description":"feature.voiceClone.description"},"knowledgeBase":{"name":"feature.knowledgeBase.name","enabled":false,"description":"feature.knowledgeBase.description"},"mcpAccessPoint":{"name":"feature.mcpAccessPoint.name","enabled":false,"description":"feature.mcpAccessPoint.description"},"vad":{"name":"feature.vad.name","enabled":true,"description":"feature.vad.description"},"asr":{"name":"feature.asr.name","enabled":true,"description":"feature.asr.description"}},"groups":{"featureManagement":["voiceprintRecognition","voiceClone","knowledgeBase","mcpAccessPoint"],"voiceManagement":["vad","asr"]}}', 'json', 1, '系统功能菜单配置');
|
||||
@@ -1,14 +0,0 @@
|
||||
-- liquibase formatted sql
|
||||
|
||||
-- changeset xiaozhi:202512041515
|
||||
CREATE TABLE ai_agent_context_provider (
|
||||
id VARCHAR(32) NOT NULL COMMENT '主键',
|
||||
agent_id VARCHAR(32) NOT NULL COMMENT '智能体ID',
|
||||
context_providers JSON COMMENT '上下文源配置',
|
||||
creator BIGINT COMMENT '创建者',
|
||||
created_at DATETIME COMMENT '创建时间',
|
||||
updater BIGINT COMMENT '更新者',
|
||||
updated_at DATETIME COMMENT '更新时间',
|
||||
PRIMARY KEY (id),
|
||||
INDEX idx_agent_id (agent_id)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='智能体上下文源配置表';
|
||||
@@ -1,6 +0,0 @@
|
||||
-- 删除server模块是否开启token认证参数
|
||||
delete from `sys_params` where param_code = 'server.auth.enabled';
|
||||
|
||||
-- 添加server模块是否开启token认证参数
|
||||
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES
|
||||
(122, 'server.auth.enabled', 'true', 'boolean', 1, 'server模块是否开启token认证');
|
||||
@@ -423,25 +423,3 @@ databaseChangeLog:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202511131023.sql
|
||||
- changeSet:
|
||||
id: 202512031517
|
||||
author: rainv123
|
||||
changes:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202512031517.sql
|
||||
|
||||
- changeSet:
|
||||
id: 202512041515
|
||||
author: cgd
|
||||
changes:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202512041515.sql
|
||||
- changeSet:
|
||||
id: 202512131453
|
||||
author: hrz
|
||||
changes:
|
||||
- sqlFile:
|
||||
encoding: utf8
|
||||
path: classpath:db/changelog/202512131453.sql
|
||||
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=\u6587\u4EF6\u5185\u5BB9\u4E0D\u80FD\u4E3A\u7A7A
|
||||
10181=\u97F3\u8272\u514B\u9686\u540D\u79F0\u4E0D\u80FD\u4E3A\u7A7A
|
||||
10182=\u97F3\u8272\u514B\u9686\u97F3\u9891\u4E0D\u5B58\u5728
|
||||
10183=\u9ED8\u8BA4\u667A\u80FD\u4F53\u672A\u627E\u5230
|
||||
10184=\u4E0D\u652F\u6301\u7684\u9002\u914D\u5668\u7C7B\u578B
|
||||
10185=RAG\u914D\u7F6E\u9A8C\u8BC1\u5931\u8D25
|
||||
10186=\u9002\u914D\u5668\u521B\u5EFA\u5931\u8D25
|
||||
10187=\u9002\u914D\u5668\u521D\u59CB\u5316\u5931\u8D25
|
||||
10188=\u9002\u914D\u5668\u8FDE\u63A5\u6D4B\u8BD5\u5931\u8D25
|
||||
10189=\u9002\u914D\u5668\u64CD\u4F5C\u5931\u8D25
|
||||
10190=\u9002\u914D\u5668\u672A\u627E\u5230
|
||||
10191=\u9002\u914D\u5668\u7F13\u5B58\u9519\u8BEF
|
||||
10192=\u9002\u914D\u5668\u7C7B\u578B\u672A\u627E\u5230
|
||||
10183=\u9ED8\u8BA4\u667A\u80FD\u4F53\u672A\u627E\u5230
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=Dateiinhalt darf nicht leer sein
|
||||
10181=Stimmenklon-Name darf nicht leer sein
|
||||
10182=Stimmenklon-Audio nicht gefunden
|
||||
10183=Standard-Agent-Vorlage nicht gefunden
|
||||
10184=Nicht unterstützter Adaptertyp
|
||||
10185=RAG-Konfigurationsvalidierung fehlgeschlagen
|
||||
10186=Adapter-Erstellung fehlgeschlagen
|
||||
10187=Adapter-Initialisierung fehlgeschlagen
|
||||
10188=Adapter-Verbindungstest fehlgeschlagen
|
||||
10189=Adapter-Operation fehlgeschlagen
|
||||
10190=Adapter nicht gefunden
|
||||
10191=Adapter-Cache-Fehler
|
||||
10192=Adaptertyp nicht gefunden
|
||||
10183=Standard-Agent-Vorlage nicht gefunden
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=File content cannot be empty
|
||||
10181=Voice clone name cannot be empty
|
||||
10182=Voice clone audio not found
|
||||
10183=Default agent template not found
|
||||
10184=Unsupported adapter type
|
||||
10185=RAG configuration validation failed
|
||||
10186=Adapter creation failed
|
||||
10187=Adapter initialization failed
|
||||
10188=Adapter connection test failed
|
||||
10189=Adapter operation failed
|
||||
10190=Adapter not found
|
||||
10191=Adapter cache error
|
||||
10192=Adapter type not found
|
||||
10183=Default agent template not found
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=Nội dung tệp không thể để trống
|
||||
10181=Tên nhân bản giọng nói không thể để trống
|
||||
10182=Không tìm thấy âm thanh nhân bản giọng nói
|
||||
10183=Không tìm thấy mẫu agent mặc định
|
||||
10184=Loại bộ chuyển đổi không được hỗ trợ
|
||||
10185=Kiểm tra cấu hình RAG thất bại
|
||||
10186=Tạo bộ chuyển đổi thất bại
|
||||
10187=Khởi tạo bộ chuyển đổi thất bại
|
||||
10188=Kiểm tra kết nối bộ chuyển đổi thất bại
|
||||
10189=Thao tác bộ chuyển đổi thất bại
|
||||
10190=Không tìm thấy bộ chuyển đổi
|
||||
10191=Lỗi bộ nhớ đệm bộ chuyển đổi
|
||||
10192=Không tìm thấy loại bộ chuyển đổi
|
||||
10183=Không tìm thấy mẫu agent mặc định
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=\u6587\u4EF6\u5185\u5BB9\u4E0D\u80FD\u4E3A\u7A7A
|
||||
10181=\u97F3\u8272\u514B\u9686\u540D\u79F0\u4E0D\u80FD\u4E3A\u7A7A
|
||||
10182=\u97F3\u8272\u514B\u9686\u97F3\u9891\u4E0D\u5B58\u5728
|
||||
10183=\u9ED8\u8BA4\u667A\u80FD\u4F53\u672A\u627E\u5230
|
||||
10184=\u4E0D\u652F\u6301\u7684\u9002\u914D\u5668\u7C7B\u578B
|
||||
10185=RAG\u914D\u7F6E\u9A8C\u8BC1\u5931\u8D25
|
||||
10186=\u9002\u914D\u5668\u521B\u5EFA\u5931\u8D25
|
||||
10187=\u9002\u914D\u5668\u521D\u59CB\u5316\u5931\u8D25
|
||||
10188=\u9002\u914D\u5668\u8FDE\u63A5\u6D4B\u8BD5\u5931\u8D25
|
||||
10189=\u9002\u914D\u5668\u64CD\u4F5C\u5931\u8D25
|
||||
10190=\u9002\u914D\u5668\u672A\u627E\u5230
|
||||
10191=\u9002\u914D\u5668\u7F13\u5B58\u9519\u8BEF
|
||||
10192=\u9002\u914D\u5668\u7C7B\u578B\u672A\u627E\u5230
|
||||
10183=\u9ED8\u8BA4\u667A\u80FD\u4F53\u672A\u627E\u5230
|
||||
@@ -189,13 +189,4 @@
|
||||
10180=\u6587\u4ef6\u5185\u5bb9\u4e0d\u80fd\u70ba\u7a7a
|
||||
10181=\u97f3\u8272\u514b\u9686\u540d\u7a31\u4e0d\u80fd\u70ba\u7a7a
|
||||
10182=\u97f3\u8272\u514b\u9686\u97f3\u983b\u4e0d\u5b58\u5728
|
||||
10183=\u9ed8\u8ba4\u667a\u80fd\u4f53\u672a\u627e\u5230
|
||||
10184=\u4E0D\u652F\u6301\u7684\u9002\u914D\u5668\u985E\u578B
|
||||
10185=RAG\u914D\u7F6E\u9A57\u8B49\u5931\u6557
|
||||
10186=\u9002\u914D\u5668\u5275\u5EFA\u5931\u6557
|
||||
10187=\u9002\u914D\u5668\u521D\u59CB\u5316\u5931\u6557
|
||||
10188=\u9002\u914D\u5668\u9023\u63A5\u6E2C\u8A66\u5931\u6557
|
||||
10189=\u9002\u914D\u5668\u64CD\u4F5C\u5931\u6557
|
||||
10190=\u9002\u914D\u5668\u672A\u627E\u5230
|
||||
10191=\u9002\u914D\u5668\u7F13\u5B58\u932F\u8AA4
|
||||
10192=\u9002\u914D\u5668\u985E\u578B\u672A\u627E\u5230
|
||||
10183=\u9ed8\u8ba4\u667a\u80fd\u4f53\u672a\u627e\u5230
|
||||
@@ -235,7 +235,7 @@ function showAbout() {
|
||||
title: t('settings.aboutApp', { appName: import.meta.env.VITE_APP_TITLE }),
|
||||
content: t('settings.aboutContent', {
|
||||
appName: import.meta.env.VITE_APP_TITLE,
|
||||
version: '0.8.10'
|
||||
version: '0.8.8'
|
||||
}),
|
||||
showCancel: false,
|
||||
confirmText: t('common.confirm'),
|
||||
|
||||
@@ -200,13 +200,6 @@ export default {
|
||||
confirm() {
|
||||
this.saving = true;
|
||||
|
||||
// 校验模型ID不能为纯文字或空格
|
||||
if (this.formData.id && !this.validateModelId(this.formData.id)) {
|
||||
this.$message.error(this.$t('modelConfigDialog.invalidModelId'));
|
||||
this.saving = false;
|
||||
return;
|
||||
}
|
||||
|
||||
if (!this.formData.supplier) {
|
||||
this.$message.error(this.$t('addModelDialog.requiredSupplier'));
|
||||
this.saving = false;
|
||||
@@ -261,38 +254,6 @@ export default {
|
||||
this.providerFields = [];
|
||||
this.currentProvider = null;
|
||||
},
|
||||
|
||||
// 校验模型ID：不能为纯文字或空格
|
||||
validateModelId(modelId) {
|
||||
if (!modelId || typeof modelId !== 'string') {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 去除首尾空格
|
||||
const trimmedId = modelId.trim();
|
||||
|
||||
// 检查是否为空或纯空格
|
||||
if (trimmedId === '') {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 检查是否只包含字母（纯文字）
|
||||
if (/^[a-zA-Z]+$/.test(trimmedId)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 检查是否包含空格
|
||||
if (/\s/.test(trimmedId)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 允许字母、数字、下划线、连字符
|
||||
if (!/^[a-zA-Z0-9_-]+$/.test(trimmedId)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
@@ -404,7 +365,7 @@ export default {
|
||||
|
||||
.custom-input-bg .el-input__inner,
|
||||
.custom-input-bg .el-textarea__inner {
|
||||
background-color: #ffffff;
|
||||
background-color: #f6f8fc;
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -1,329 +0,0 @@
|
||||
<template>
|
||||
<el-dialog
|
||||
:visible.sync="dialogVisible"
|
||||
width="900px"
|
||||
:title="$t('contextProviderDialog.title')"
|
||||
:close-on-click-modal="false"
|
||||
custom-class="context-provider-dialog"
|
||||
append-to-body
|
||||
>
|
||||
<div class="dialog-content">
|
||||
<el-empty v-if="localProviders.length === 0" :description="$t('contextProviderDialog.noContextApi')">
|
||||
<el-button type="primary" icon="el-icon-plus" @click="addProvider(0)">{{ $t('contextProviderDialog.add') }}</el-button>
|
||||
</el-empty>
|
||||
|
||||
<div
|
||||
v-for="(provider, pIndex) in localProviders"
|
||||
:key="pIndex"
|
||||
class="provider-item"
|
||||
>
|
||||
<el-card class="provider-card" shadow="hover" :body-style="{ padding: '15px 20px' }">
|
||||
<!-- URL Row -->
|
||||
<div class="input-row">
|
||||
<span class="label-text">{{ $t('contextProviderDialog.apiUrl') }}</span>
|
||||
<el-input
|
||||
v-model="provider.url"
|
||||
:placeholder="$t('contextProviderDialog.apiUrlPlaceholder')"
|
||||
size="small"
|
||||
class="flex-1"
|
||||
></el-input>
|
||||
</div>
|
||||
|
||||
<!-- Headers Section -->
|
||||
<div class="headers-section">
|
||||
<div class="label-text" style="margin-top: 6px;">{{ $t('contextProviderDialog.requestHeaders') }}</div>
|
||||
<div class="headers-list">
|
||||
<div
|
||||
v-for="(header, hIndex) in provider.headers"
|
||||
:key="hIndex"
|
||||
class="header-row"
|
||||
>
|
||||
<el-input
|
||||
v-model="header.key"
|
||||
:placeholder="$t('contextProviderDialog.headerKeyPlaceholder')"
|
||||
size="small"
|
||||
style="width: 180px;"
|
||||
></el-input>
|
||||
<span class="separator">:</span>
|
||||
<el-input
|
||||
v-model="header.value"
|
||||
:placeholder="$t('contextProviderDialog.headerValuePlaceholder')"
|
||||
size="small"
|
||||
class="flex-1"
|
||||
></el-input>
|
||||
|
||||
<div class="row-controls">
|
||||
<el-button
|
||||
type="primary"
|
||||
icon="el-icon-plus"
|
||||
circle
|
||||
size="mini"
|
||||
plain
|
||||
@click="addHeader(pIndex, hIndex + 1)"
|
||||
></el-button>
|
||||
<el-button
|
||||
type="danger"
|
||||
icon="el-icon-minus"
|
||||
circle
|
||||
size="mini"
|
||||
plain
|
||||
@click="removeHeader(pIndex, hIndex)"
|
||||
></el-button>
|
||||
</div>
|
||||
</div>
|
||||
<!-- Empty Headers State -->
|
||||
<div v-if="provider.headers.length === 0" class="header-row empty-header">
|
||||
<span class="no-header-text">{{ $t('contextProviderDialog.noHeaders') }}</span>
|
||||
<el-button
|
||||
type="text"
|
||||
icon="el-icon-plus"
|
||||
size="mini"
|
||||
@click="addHeader(pIndex, 0)"
|
||||
>{{ $t('contextProviderDialog.addHeader') }}</el-button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</el-card>
|
||||
|
||||
<!-- Provider Block Controls (Right Side) -->
|
||||
<div class="block-controls">
|
||||
<el-button
|
||||
type="primary"
|
||||
icon="el-icon-plus"
|
||||
circle
|
||||
size="medium"
|
||||
@click="addProvider(pIndex + 1)"
|
||||
></el-button>
|
||||
<el-button
|
||||
type="danger"
|
||||
icon="el-icon-minus"
|
||||
circle
|
||||
size="medium"
|
||||
@click="removeProvider(pIndex)"
|
||||
></el-button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<span slot="footer" class="dialog-footer">
|
||||
<el-button @click="dialogVisible = false">{{ $t('contextProviderDialog.cancel') }}</el-button>
|
||||
<el-button type="primary" @click="handleConfirm">{{ $t('contextProviderDialog.confirm') }}</el-button>
|
||||
</span>
|
||||
</el-dialog>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
export default {
|
||||
name: 'ContextProviderDialog',
|
||||
props: {
|
||||
visible: {
|
||||
type: Boolean,
|
||||
default: false
|
||||
},
|
||||
providers: {
|
||||
type: Array,
|
||||
default: () => []
|
||||
}
|
||||
},
|
||||
data() {
|
||||
return {
|
||||
localProviders: []
|
||||
};
|
||||
},
|
||||
computed: {
|
||||
dialogVisible: {
|
||||
get() {
|
||||
return this.visible;
|
||||
},
|
||||
set(val) {
|
||||
this.$emit('update:visible', val);
|
||||
}
|
||||
}
|
||||
},
|
||||
watch: {
|
||||
visible(val) {
|
||||
if (val) {
|
||||
this.initLocalData();
|
||||
}
|
||||
}
|
||||
},
|
||||
methods: {
|
||||
initLocalData() {
|
||||
// 深拷贝并将 headers 对象转换为数组
|
||||
this.localProviders = this.providers.map(p => {
|
||||
const headers = p.headers || {};
|
||||
return {
|
||||
url: p.url || '',
|
||||
headers: Object.entries(headers).map(([key, value]) => ({ key, value }))
|
||||
};
|
||||
});
|
||||
|
||||
// 如果为空，添加一个默认块
|
||||
if (this.localProviders.length === 0) {
|
||||
this.localProviders.push({ url: '', headers: [{ key: '', value: '' }] });
|
||||
}
|
||||
},
|
||||
addProvider(index) {
|
||||
this.localProviders.splice(index, 0, {
|
||||
url: '',
|
||||
headers: [{ key: '', value: '' }]
|
||||
});
|
||||
},
|
||||
removeProvider(index) {
|
||||
this.localProviders.splice(index, 1);
|
||||
},
|
||||
addHeader(pIndex, hIndex) {
|
||||
this.localProviders[pIndex].headers.splice(hIndex, 0, { key: '', value: '' });
|
||||
},
|
||||
removeHeader(pIndex, hIndex) {
|
||||
this.localProviders[pIndex].headers.splice(hIndex, 1);
|
||||
},
|
||||
handleConfirm() {
|
||||
const result = this.localProviders
|
||||
.filter(p => p.url.trim() !== '')
|
||||
.map(p => {
|
||||
const headersObj = {};
|
||||
p.headers.forEach(h => {
|
||||
if (h.key.trim()) {
|
||||
headersObj[h.key.trim()] = h.value;
|
||||
}
|
||||
});
|
||||
return {
|
||||
url: p.url.trim(),
|
||||
headers: headersObj
|
||||
};
|
||||
});
|
||||
|
||||
this.$emit('confirm', result);
|
||||
this.dialogVisible = false;
|
||||
}
|
||||
}
|
||||
};
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.dialog-content {
|
||||
max-height: 60vh;
|
||||
overflow-y: auto;
|
||||
padding: 20px 25px;
|
||||
}
|
||||
|
||||
.dialog-content::-webkit-scrollbar {
|
||||
width: 6px;
|
||||
}
|
||||
.dialog-content::-webkit-scrollbar-thumb {
|
||||
background: #dcdfe6;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.dialog-content::-webkit-scrollbar-track {
|
||||
background: #f5f7fa;
|
||||
}
|
||||
|
||||
.provider-item {
|
||||
display: flex;
|
||||
gap: 15px;
|
||||
margin-bottom: 20px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.provider-card {
|
||||
flex: 1;
|
||||
border-radius: 12px;
|
||||
border: 1px solid #e4e7ed;
|
||||
border-left: 4px solid #409EFF; /* 左侧强调色 */
|
||||
background-color: #fff;
|
||||
transition: all 0.3s ease;
|
||||
box-shadow: 0 2px 12px 0 rgba(0, 0, 0, 0.05);
|
||||
}
|
||||
|
||||
.provider-card:hover {
|
||||
box-shadow: 0 8px 20px rgba(0, 0, 0, 0.1);
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.block-controls {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.input-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
margin-bottom: 18px;
|
||||
}
|
||||
|
||||
.label-text {
|
||||
width: 60px;
|
||||
font-weight: 600;
|
||||
color: #606266;
|
||||
text-align: right;
|
||||
font-size: 13px;
|
||||
white-space: nowrap;
|
||||
line-height: 32px; /* 垂直居中对齐 */
|
||||
}
|
||||
|
||||
.flex-1 {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.headers-section {
|
||||
display: flex;
|
||||
gap: 12px;
|
||||
align-items: flex-start;
|
||||
}
|
||||
|
||||
.headers-list {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 10px;
|
||||
background: #fcfcfc;
|
||||
padding: 15px;
|
||||
border-radius: 8px;
|
||||
border: 1px dashed #dcdfe6;
|
||||
transition: all 0.3s;
|
||||
}
|
||||
|
||||
.headers-list:hover {
|
||||
border-color: #c0c4cc;
|
||||
background: #fff;
|
||||
}
|
||||
|
||||
.header-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.separator {
|
||||
color: #909399;
|
||||
font-weight: bold;
|
||||
margin: 0 2px;
|
||||
}
|
||||
|
||||
.row-controls {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
margin-left: 8px;
|
||||
flex-shrink: 0;
|
||||
opacity: 0.6;
|
||||
transition: opacity 0.2s;
|
||||
}
|
||||
|
||||
.header-row:hover .row-controls {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.empty-header {
|
||||
justify-content: center;
|
||||
padding: 10px;
|
||||
color: #909399;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.no-header-text {
|
||||
margin-right: 8px;
|
||||
}
|
||||
</style>
|
||||
@@ -23,7 +23,7 @@
|
||||
<div class="settings-btn" @click="handleConfigure">
|
||||
{{ $t('home.configureRole') }}
|
||||
</div>
|
||||
<div v-if="featureStatus.voiceprintRecognition" class="settings-btn" @click="handleVoicePrint">
|
||||
<div class="settings-btn" @click="handleVoicePrint">
|
||||
{{ $t('home.voiceprintRecognition') }}
|
||||
</div>
|
||||
<div class="settings-btn" @click="handleDeviceManage">
|
||||
@@ -49,15 +49,7 @@ import i18n from '@/i18n';
|
||||
export default {
|
||||
name: 'DeviceItem',
|
||||
props: {
|
||||
device: { type: Object, required: true },
|
||||
featureStatus: {
|
||||
type: Object,
|
||||
default: () => ({
|
||||
voiceprintRecognition: false,
|
||||
voiceClone: false,
|
||||
knowledgeBase: false
|
||||
})
|
||||
}
|
||||
device: { type: Object, required: true }
|
||||
},
|
||||
data() {
|
||||
return { switchValue: false }
|
||||
|
||||
@@ -106,7 +106,7 @@
|
||||
</div>
|
||||
|
||||
<!-- MCP区域 -->
|
||||
<div class="mcp-access-point" v-if="featureStatus.mcpAccessPoint">
|
||||
<div class="mcp-access-point">
|
||||
<div class="mcp-container">
|
||||
<!-- 左侧区域 -->
|
||||
<div class="mcp-left">
|
||||
@@ -171,7 +171,6 @@
|
||||
<script>
|
||||
import Api from '@/apis/api';
|
||||
import i18n from '@/i18n';
|
||||
import featureManager from '@/utils/featureManager';
|
||||
|
||||
export default {
|
||||
i18n,
|
||||
@@ -206,11 +205,6 @@ export default {
|
||||
mcpUrl: "",
|
||||
mcpStatus: "disconnected",
|
||||
mcpTools: [],
|
||||
|
||||
// 功能状态
|
||||
featureStatus: {
|
||||
mcpAccessPoint: false
|
||||
}
|
||||
}
|
||||
},
|
||||
computed: {
|
||||
@@ -255,9 +249,6 @@ export default {
|
||||
// 右侧默认指向第一个
|
||||
this.currentFunction = this.selectedList[0] || null;
|
||||
|
||||
// 加载功能状态
|
||||
this.loadFeatureStatus();
|
||||
|
||||
// 加载MCP数据
|
||||
this.loadMcpAddress();
|
||||
this.loadMcpTools();
|
||||
@@ -268,19 +259,6 @@ export default {
|
||||
}
|
||||
},
|
||||
methods: {
|
||||
/**
|
||||
* 加载功能状态
|
||||
*/
|
||||
async loadFeatureStatus() {
|
||||
// 确保featureManager已初始化完成
|
||||
await featureManager.waitForInitialization();
|
||||
|
||||
const config = featureManager.getConfig();
|
||||
this.featureStatus = {
|
||||
mcpAccessPoint: config.mcpAccessPoint || false
|
||||
};
|
||||
},
|
||||
|
||||
copyUrl() {
|
||||
const textarea = document.createElement('textarea');
|
||||
textarea.value = this.mcpUrl;
|
||||
@@ -478,7 +456,6 @@ export default {
|
||||
.function-column {
|
||||
position: relative;
|
||||
width: auto;
|
||||
height:700px;
|
||||
padding: 10px;
|
||||
overflow-y: auto;
|
||||
border-right: 1px solid #EBEEF5;
|
||||
@@ -486,12 +463,6 @@ export default {
|
||||
overflow-x: hidden;
|
||||
}
|
||||
|
||||
.mcp-access-point {
|
||||
position: relative;
|
||||
z-index: 1;
|
||||
background: white;
|
||||
}
|
||||
|
||||
.function-column::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
<span class="nav-text">{{ $t("header.smartManagement") }}</span>
|
||||
</div>
|
||||
<!-- 普通用户显示音色克隆 -->
|
||||
<div v-if="!isSuperAdmin && featureStatus.voiceClone" class="equipment-management"
|
||||
<div v-if="!isSuperAdmin" class="equipment-management"
|
||||
:class="{ 'active-tab': $route.path === '/voice-clone-management' }" @click="goVoiceCloneManagement">
|
||||
<img loading="lazy" alt="" src="@/assets/header/voice.png" :style="{
|
||||
filter:
|
||||
@@ -38,7 +38,7 @@
|
||||
</div>
|
||||
|
||||
<!-- 超级管理员显示音色克隆下拉菜单 -->
|
||||
<el-dropdown v-if="isSuperAdmin && featureStatus.voiceClone" trigger="click" class="equipment-management more-dropdown" :class="{
|
||||
<el-dropdown v-if="isSuperAdmin" trigger="click" class="equipment-management more-dropdown" :class="{
|
||||
'active-tab':
|
||||
$route.path === '/voice-clone-management' ||
|
||||
$route.path === '/voice-resource-management',
|
||||
@@ -72,7 +72,7 @@
|
||||
}" />
|
||||
<span class="nav-text">{{ $t("header.modelConfig") }}</span>
|
||||
</div>
|
||||
<div v-if="featureStatus.knowledgeBase" class="equipment-management"
|
||||
<div class="equipment-management"
|
||||
:class="{ 'active-tab': $route.path === '/knowledge-base-management' || $route.path === '/knowledge-file-upload' }"
|
||||
@click="goKnowledgeBaseManagement">
|
||||
<img loading="lazy" alt="" src="@/assets/header/knowledge_base.png" :style="{
|
||||
@@ -89,8 +89,7 @@
|
||||
$route.path === '/server-side-management' ||
|
||||
$route.path === '/agent-template-management' ||
|
||||
$route.path === '/ota-management' ||
|
||||
$route.path === '/user-management' ||
|
||||
$route.path === '/feature-management',
|
||||
$route.path === '/user-management',
|
||||
}" @visible-change="handleParamDropdownVisibleChange">
|
||||
<span class="el-dropdown-link">
|
||||
<img loading="lazy" alt="" src="@/assets/header/param_management.png" :style="{
|
||||
@@ -101,8 +100,7 @@
|
||||
$route.path === '/server-side-management' ||
|
||||
$route.path === '/agent-template-management' ||
|
||||
$route.path === '/ota-management' ||
|
||||
$route.path === '/user-management' ||
|
||||
$route.path === '/feature-management'
|
||||
$route.path === '/user-management'
|
||||
? 'brightness(0) invert(1)'
|
||||
: 'None',
|
||||
}" />
|
||||
@@ -131,9 +129,6 @@
|
||||
<el-dropdown-item @click.native="goServerSideManagement">
|
||||
{{ $t("header.serverSideManagement") }}
|
||||
</el-dropdown-item>
|
||||
<el-dropdown-item @click.native="goFeatureManagement">
|
||||
{{ $t("header.featureManagement") }}
|
||||
</el-dropdown-item>
|
||||
</el-dropdown-menu>
|
||||
</el-dropdown>
|
||||
</div>
|
||||
@@ -191,7 +186,6 @@ import userApi from "@/apis/module/user";
|
||||
import i18n, { changeLanguage } from "@/i18n";
|
||||
import { mapActions, mapGetters } from "vuex";
|
||||
import ChangePasswordDialog from "./ChangePasswordDialog.vue"; // 引入修改密码弹窗组件
|
||||
import featureManager from "@/utils/featureManager"; // 引入功能管理工具类
|
||||
|
||||
export default {
|
||||
name: "HeaderBar",
|
||||
@@ -223,11 +217,6 @@ export default {
|
||||
label: "label",
|
||||
children: "children",
|
||||
},
|
||||
// 功能状态
|
||||
featureStatus: {
|
||||
voiceClone: false, // 音色克隆功能状态
|
||||
knowledgeBase: false, // 知识库功能状态
|
||||
},
|
||||
};
|
||||
},
|
||||
computed: {
|
||||
@@ -297,14 +286,12 @@ export default {
|
||||
];
|
||||
},
|
||||
},
|
||||
async mounted() {
|
||||
mounted() {
|
||||
this.fetchUserInfo();
|
||||
this.checkScreenSize();
|
||||
window.addEventListener("resize", this.checkScreenSize);
|
||||
// 从localStorage加载搜索历史
|
||||
this.loadSearchHistory();
|
||||
// 等待featureManager初始化完成后再加载功能状态
|
||||
await this.loadFeatureStatus();
|
||||
},
|
||||
//移除事件监听器
|
||||
beforeDestroy() {
|
||||
@@ -351,20 +338,6 @@ export default {
|
||||
goAgentTemplateManagement() {
|
||||
this.$router.push("/agent-template-management");
|
||||
},
|
||||
// 跳转到功能管理
|
||||
goFeatureManagement() {
|
||||
this.$router.push("/feature-management");
|
||||
},
|
||||
// 加载功能状态
|
||||
async loadFeatureStatus() {
|
||||
// 等待featureManager初始化完成
|
||||
await featureManager.waitForInitialization();
|
||||
|
||||
const config = featureManager.getConfig();
|
||||
|
||||
this.featureStatus.voiceClone = config.voiceClone;
|
||||
this.featureStatus.knowledgeBase = config.knowledgeBase;
|
||||
},
|
||||
// 获取用户信息
|
||||
fetchUserInfo() {
|
||||
userApi.getUserInfo(({ data }) => {
|
||||
|
||||
+1208
-1261
File diff suppressed because it is too large
Load Diff
@@ -22,7 +22,6 @@ export default {
|
||||
'header.clearHistory': 'Clear History',
|
||||
'header.providerManagement': 'Provider Management',
|
||||
'header.serverSideManagement': 'Server Management',
|
||||
'header.featureManagement': 'System Feature Management',
|
||||
'header.changePassword': 'Change Password',
|
||||
'header.logout': 'Logout',
|
||||
'header.searchPlaceholder': 'Search by name..',
|
||||
@@ -230,26 +229,6 @@ export default {
|
||||
'voicePrintDialog.requiredName': 'Please enter name',
|
||||
'voicePrintDialog.requiredAudioVector': 'Please select audio vector',
|
||||
|
||||
// Context provider dialog related
|
||||
'contextProviderDialog.title': 'Edit Source',
|
||||
'contextProviderDialog.noContextApi': 'No Context API',
|
||||
'contextProviderDialog.add': 'Add',
|
||||
'contextProviderDialog.apiUrl': 'API URL',
|
||||
'contextProviderDialog.apiUrlPlaceholder': 'http://api.example.com/data',
|
||||
'contextProviderDialog.requestHeaders': 'Request Headers',
|
||||
'contextProviderDialog.headerKeyPlaceholder': 'Key',
|
||||
'contextProviderDialog.headerValuePlaceholder': 'Value',
|
||||
'contextProviderDialog.noHeaders': 'No Headers',
|
||||
'contextProviderDialog.addHeader': 'Add Header',
|
||||
'contextProviderDialog.cancel': 'Cancel',
|
||||
'contextProviderDialog.confirm': 'Confirm',
|
||||
|
||||
// Role config page - context provider related
|
||||
'roleConfig.contextProvider': 'Context',
|
||||
'roleConfig.contextProviderSuccess': 'Successfully added {count} sources.',
|
||||
'roleConfig.contextProviderDocLink': 'How to deploy context provider',
|
||||
'roleConfig.editContextProvider': 'Edit Source',
|
||||
|
||||
// Voice print page related
|
||||
'voicePrint.pageTitle': 'Voice Print Recognition',
|
||||
'voicePrint.name': 'Name',
|
||||
@@ -713,7 +692,7 @@ export default {
|
||||
'paramManagement.deleteFailed': 'Deletion failed, please try again',
|
||||
'paramManagement.operationCancelled': 'Deletion cancelled',
|
||||
'paramManagement.operationClosed': 'Operation closed',
|
||||
'paramManagement.updateSuccess': 'Update successful. Some configurations will take effect only after restarting the xiaozhi-server module.',
|
||||
'paramManagement.updateSuccess': 'Update successful',
|
||||
'paramManagement.addSuccess': 'Add successful',
|
||||
'paramManagement.updateFailed': 'Update failed',
|
||||
'paramManagement.addFailed': 'Add failed',
|
||||
@@ -852,7 +831,7 @@ export default {
|
||||
'modelConfig.enableSuccess': 'Enable successful',
|
||||
'modelConfig.disableSuccess': 'Disable successful',
|
||||
'modelConfig.operationFailed': 'Operation failed',
|
||||
'modelConfig.setDefaultSuccess': 'Set default model successful, please restart the xiaozhi-server module manually in time',
|
||||
'modelConfig.setDefaultSuccess': 'Set default model successful',
|
||||
'modelConfig.itemsPerPage': '{items} items/page',
|
||||
'modelConfig.firstPage': 'First Page',
|
||||
'modelConfig.prevPage': 'Previous Page',
|
||||
@@ -870,7 +849,6 @@ export default {
|
||||
'modelConfigDialog.setDefault': 'Set as Default',
|
||||
'modelConfigDialog.modelId': 'Model ID',
|
||||
'modelConfigDialog.enterModelId': 'If not filled in, it will be generated automatically',
|
||||
'modelConfigDialog.invalidModelId': 'Model ID cannot be pure text or spaces, please use letters, numbers, underscores, or hyphens',
|
||||
'modelConfigDialog.modelName': 'Model Name',
|
||||
'modelConfigDialog.enterModelName': 'Please enter model name',
|
||||
'modelConfigDialog.modelCode': 'Model Code',
|
||||
@@ -1283,35 +1261,4 @@ export default {
|
||||
'knowledgeFileUpload.content': 'Content:',
|
||||
'knowledgeFileUpload.testQuestionRequired': 'Please enter test question',
|
||||
'knowledgeBaseDialog.descriptionRequired': 'Please enter knowledge base description',
|
||||
|
||||
// Feature Management page text
|
||||
'featureManagement.selectAll': 'Select All',
|
||||
'featureManagement.deselectAll': 'Deselect All',
|
||||
'featureManagement.save': 'Save Configuration',
|
||||
'featureManagement.reset': 'Reset',
|
||||
'featureManagement.group.featureManagement': 'Enable/Disable the feature/section',
|
||||
'featureManagement.group.voiceManagement': 'Visible to users during agent configuration',
|
||||
'featureManagement.noFeatures': 'No features available',
|
||||
'featureManagement.contactAdmin': 'Please contact administrator to configure features',
|
||||
'featureManagement.saveSuccess': 'Feature configuration saved successfully',
|
||||
'featureManagement.resetConfirm': 'Are you sure you want to reset all feature configurations?',
|
||||
'featureManagement.confirm': 'Confirm',
|
||||
'featureManagement.cancel': 'Cancel',
|
||||
'featureManagement.resetSuccess': 'Feature configuration reset successfully',
|
||||
'featureManagement.noChanges': 'No changes to save',
|
||||
|
||||
// Feature names and descriptions
|
||||
'feature.voiceprintRecognition.name': 'Voiceprint Recognition',
|
||||
'feature.voiceprintRecognition.description': 'Verify user identity through voiceprint recognition technology, providing secure voice interaction experience',
|
||||
'feature.voiceClone.name': 'Voice Clone',
|
||||
'feature.voiceClone.description': 'Clone specific voice timbre using AI technology to achieve personalized voice synthesis',
|
||||
'feature.knowledgeBase.name': 'Knowledge Base',
|
||||
'feature.knowledgeBase.description': 'Build and manage knowledge base system to provide professional knowledge support for AI assistants',
|
||||
'feature.mcpAccessPoint.name': 'MCP Access Point',
|
||||
'feature.mcpAccessPoint.description': 'Provide MCP protocol access points to support integration of external tools and services',
|
||||
'feature.vad.name': 'Voice Activity Detection',
|
||||
'feature.vad.description': 'Automatically detect voice activity to optimize voice interaction response efficiency',
|
||||
'feature.asr.name': 'Speech Recognition',
|
||||
'feature.asr.description': 'Convert speech to text to enable natural language interaction functionality',
|
||||
|
||||
}
|
||||
+1208
-1261
File diff suppressed because it is too large
Load Diff
@@ -22,7 +22,6 @@ export default {
|
||||
'header.clearHistory': '清空历史',
|
||||
'header.providerManagement': '字段管理',
|
||||
'header.serverSideManagement': '服务端管理',
|
||||
'header.featureManagement': '系统功能配置',
|
||||
'header.changePassword': '修改密码',
|
||||
'header.logout': '退出登录',
|
||||
'header.searchPlaceholder': '输入名称搜索..',
|
||||
@@ -230,26 +229,6 @@ export default {
|
||||
'voicePrintDialog.requiredName': '请输入姓名',
|
||||
'voicePrintDialog.requiredAudioVector': '请选择音频向量',
|
||||
|
||||
// 上下文源对话框相关
|
||||
'contextProviderDialog.title': '编辑源',
|
||||
'contextProviderDialog.noContextApi': '暂无上下文API',
|
||||
'contextProviderDialog.add': '添加',
|
||||
'contextProviderDialog.apiUrl': '接口地址',
|
||||
'contextProviderDialog.apiUrlPlaceholder': 'http://api.example.com/data',
|
||||
'contextProviderDialog.requestHeaders': '请求头',
|
||||
'contextProviderDialog.headerKeyPlaceholder': 'Key',
|
||||
'contextProviderDialog.headerValuePlaceholder': 'Value',
|
||||
'contextProviderDialog.noHeaders': '暂无 Headers',
|
||||
'contextProviderDialog.addHeader': '添加 Header',
|
||||
'contextProviderDialog.cancel': '取消',
|
||||
'contextProviderDialog.confirm': '确定',
|
||||
|
||||
// 角色配置页面-上下文源相关
|
||||
'roleConfig.contextProvider': '上下文源',
|
||||
'roleConfig.contextProviderSuccess': '已成功添加 {count} 个源。',
|
||||
'roleConfig.contextProviderDocLink': '如何部署上下文源',
|
||||
'roleConfig.editContextProvider': '编辑源',
|
||||
|
||||
// 声纹页面相关
|
||||
'voicePrint.pageTitle': '声纹识别',
|
||||
'voicePrint.name': '姓名',
|
||||
@@ -713,7 +692,7 @@ export default {
|
||||
'paramManagement.deleteFailed': '删除失败，请重试',
|
||||
'paramManagement.operationCancelled': '已取消删除操作',
|
||||
'paramManagement.operationClosed': '操作已关闭',
|
||||
'paramManagement.updateSuccess': '修改成功，部分配置需重启xiaozhi-server模块才生效',
|
||||
'paramManagement.updateSuccess': '修改成功',
|
||||
'paramManagement.addSuccess': '新增成功',
|
||||
'paramManagement.updateFailed': '更新失败',
|
||||
'paramManagement.addFailed': '新增失败',
|
||||
@@ -852,7 +831,7 @@ export default {
|
||||
'modelConfig.enableSuccess': '启用成功',
|
||||
'modelConfig.disableSuccess': '禁用成功',
|
||||
'modelConfig.operationFailed': '操作失败',
|
||||
'modelConfig.setDefaultSuccess': '设置默认模型成功，请及时手动重启xiaozhi-server模块',
|
||||
'modelConfig.setDefaultSuccess': '设置默认模型成功',
|
||||
'modelConfig.itemsPerPage': '{items}条/页',
|
||||
'modelConfig.firstPage': '首页',
|
||||
'modelConfig.prevPage': '上一页',
|
||||
@@ -870,7 +849,6 @@ export default {
|
||||
'modelConfigDialog.setDefault': '设为默认',
|
||||
'modelConfigDialog.modelId': '模型ID',
|
||||
'modelConfigDialog.enterModelId': '未填写将自动生成模型ID',
|
||||
'modelConfigDialog.invalidModelId': '模型ID不能为纯文字或空格，请使用字母、数字、下划线或连字符组合',
|
||||
'modelConfigDialog.modelName': '模型名称',
|
||||
'modelConfigDialog.enterModelName': '请输入模型名称',
|
||||
'modelConfigDialog.modelCode': '模型编码',
|
||||
@@ -1283,35 +1261,4 @@ export default {
|
||||
'knowledgeFileUpload.content': '内容:',
|
||||
'knowledgeFileUpload.testQuestionRequired': '请输入测试问题',
|
||||
'knowledgeBaseDialog.descriptionRequired': '请输入知识库描述',
|
||||
|
||||
// 系统功能配置页面文本
|
||||
'featureManagement.selectAll': '全选',
|
||||
'featureManagement.deselectAll': '取消全选',
|
||||
'featureManagement.save': '保存配置',
|
||||
'featureManagement.reset': '重置',
|
||||
'featureManagement.group.featureManagement': '是否开启功能/板块',
|
||||
'featureManagement.group.voiceManagement': '配置智能体时是否对用户可见',
|
||||
'featureManagement.noFeatures': '暂无功能',
|
||||
'featureManagement.contactAdmin': '请联系管理员配置功能',
|
||||
'featureManagement.saveSuccess': '功能配置保存成功',
|
||||
'featureManagement.resetConfirm': '确定要重置所有功能配置吗？',
|
||||
'featureManagement.confirm': '确定',
|
||||
'featureManagement.cancel': '取消',
|
||||
'featureManagement.resetSuccess': '功能配置重置成功',
|
||||
'featureManagement.noChanges': '没有需要保存的更改',
|
||||
|
||||
// 功能名称和描述
|
||||
'feature.voiceprintRecognition.name': '声纹识别',
|
||||
'feature.voiceprintRecognition.description': '通过声纹识别技术验证用户身份，提供安全的语音交互体验',
|
||||
'feature.voiceClone.name': '音色克隆',
|
||||
'feature.voiceClone.description': '使用AI技术克隆特定音色，实现个性化语音合成',
|
||||
'feature.knowledgeBase.name': '知识库',
|
||||
'feature.knowledgeBase.description': '构建和管理知识库系统，为AI助手提供专业知识支持',
|
||||
'feature.mcpAccessPoint.name': 'MCP接入点',
|
||||
'feature.mcpAccessPoint.description': '提供MCP协议接入点，支持外部工具和服务的集成',
|
||||
'feature.vad.name': '语音活动检测',
|
||||
'feature.vad.description': '自动检测语音活动，优化语音交互的响应效率',
|
||||
'feature.asr.name': '语音识别',
|
||||
'feature.asr.description': '将语音转换为文本，实现自然语言交互功能',
|
||||
|
||||
}
|
||||
@@ -22,7 +22,6 @@ export default {
|
||||
'header.clearHistory': '清空歷史',
|
||||
'header.providerManagement': '字段管理',
|
||||
'header.serverSideManagement': '服務端管理',
|
||||
'header.featureManagement': '系統功能配置',
|
||||
'header.changePassword': '修改密碼',
|
||||
'header.logout': '退出登錄',
|
||||
'header.searchPlaceholder': '輸入名稱搜索..',
|
||||
@@ -230,26 +229,6 @@ export default {
|
||||
'voicePrintDialog.requiredName': '請輸入姓名',
|
||||
'voicePrintDialog.requiredAudioVector': '請選擇音頻向量',
|
||||
|
||||
// 上下文源對話框相關
|
||||
'contextProviderDialog.title': '編輯源',
|
||||
'contextProviderDialog.noContextApi': '暫無上下文API',
|
||||
'contextProviderDialog.add': '添加',
|
||||
'contextProviderDialog.apiUrl': '接口地址',
|
||||
'contextProviderDialog.apiUrlPlaceholder': 'http://api.example.com/data',
|
||||
'contextProviderDialog.requestHeaders': '請求頭',
|
||||
'contextProviderDialog.headerKeyPlaceholder': 'Key',
|
||||
'contextProviderDialog.headerValuePlaceholder': 'Value',
|
||||
'contextProviderDialog.noHeaders': '暫無 Headers',
|
||||
'contextProviderDialog.addHeader': '添加 Header',
|
||||
'contextProviderDialog.cancel': '取消',
|
||||
'contextProviderDialog.confirm': '確定',
|
||||
|
||||
// 角色配置頁面-上下文源相關
|
||||
'roleConfig.contextProvider': '上下文源',
|
||||
'roleConfig.contextProviderSuccess': '已成功添加 {count} 個源。',
|
||||
'roleConfig.contextProviderDocLink': '如何部署上下文源',
|
||||
'roleConfig.editContextProvider': '編輯源',
|
||||
|
||||
// 聲紋頁面相關
|
||||
'voicePrint.pageTitle': '聲紋識別',
|
||||
'voicePrint.name': '姓名',
|
||||
@@ -713,7 +692,7 @@ export default {
|
||||
'paramManagement.deleteFailed': '刪除失敗，請重試',
|
||||
'paramManagement.operationCancelled': '已取消刪除操作',
|
||||
'paramManagement.operationClosed': '操作已關閉',
|
||||
'paramManagement.updateSuccess': '修改成功，部分配置需重啟xiaozhi-server模組才生效',
|
||||
'paramManagement.updateSuccess': '修改成功',
|
||||
'paramManagement.addSuccess': '新增成功',
|
||||
'paramManagement.updateFailed': '更新失敗',
|
||||
'paramManagement.addFailed': '新增失敗',
|
||||
@@ -852,7 +831,7 @@ export default {
|
||||
'modelConfig.enableSuccess': '啟用成功',
|
||||
'modelConfig.disableSuccess': '禁用成功',
|
||||
'modelConfig.operationFailed': '操作失敗',
|
||||
'modelConfig.setDefaultSuccess': '設置默認模型成功，請及時手動重啟xiaozhi-server模組',
|
||||
'modelConfig.setDefaultSuccess': '設置默認模型成功',
|
||||
'modelConfig.itemsPerPage': '{items}條/頁',
|
||||
'modelConfig.firstPage': '首頁',
|
||||
'modelConfig.prevPage': '上一頁',
|
||||
@@ -870,7 +849,6 @@ export default {
|
||||
'modelConfigDialog.setDefault': '設為默認',
|
||||
'modelConfigDialog.modelId': '模型ID',
|
||||
'modelConfigDialog.enterModelId': '未填冩將自動生成模型ID',
|
||||
'modelConfigDialog.invalidModelId': '模型ID不能為純文字或空格，請使用字母、數字、底線或連字符組合',
|
||||
'modelConfigDialog.modelName': '模型名稱',
|
||||
'modelConfigDialog.enterModelName': '請輸入模型名稱',
|
||||
'modelConfigDialog.modelCode': '模型編碼',
|
||||
@@ -1283,35 +1261,4 @@ export default {
|
||||
'knowledgeFileUpload.content': '內容:',
|
||||
'knowledgeFileUpload.testQuestionRequired': '請輸入測試問題',
|
||||
'knowledgeBaseDialog.descriptionRequired': '請輸入知识库描述',
|
||||
|
||||
// 功能管理頁面文本
|
||||
'featureManagement.selectAll': '全選',
|
||||
'featureManagement.deselectAll': '取消全選',
|
||||
'featureManagement.save': '儲存配置',
|
||||
'featureManagement.reset': '重置',
|
||||
'featureManagement.group.featureManagement': '是否開啟功能/板块',
|
||||
'featureManagement.group.voiceManagement': '配置智能体時是否對用戶可見',
|
||||
'featureManagement.noFeatures': '暫無功能',
|
||||
'featureManagement.contactAdmin': '請聯繫管理員配置功能',
|
||||
'featureManagement.saveSuccess': '功能配置儲存成功',
|
||||
'featureManagement.resetConfirm': '確定要重置所有功能配置嗎？',
|
||||
'featureManagement.confirm': '確定',
|
||||
'featureManagement.cancel': '取消',
|
||||
'featureManagement.resetSuccess': '功能配置重置成功',
|
||||
'featureManagement.noChanges': '沒有需要儲存的更改',
|
||||
|
||||
// 功能名稱和描述
|
||||
'feature.voiceprintRecognition.name': '聲紋識別',
|
||||
'feature.voiceprintRecognition.description': '通過聲紋識別技術驗證用戶身份，提供安全的語音交互體驗',
|
||||
'feature.voiceClone.name': '音色複刻',
|
||||
'feature.voiceClone.description': '使用AI技術複刻特定音色，實現個性化語音合成',
|
||||
'feature.knowledgeBase.name': '知識庫',
|
||||
'feature.knowledgeBase.description': '構建和管理知識庫系統，為AI助手提供專業知識支持',
|
||||
'feature.mcpAccessPoint.name': 'MCP接入點',
|
||||
'feature.mcpAccessPoint.description': '提供MCP協議接入點，支持外部工具和服務的整合',
|
||||
'feature.vad.name': '語音活動檢測',
|
||||
'feature.vad.description': '自動檢測語音活動，優化語音交互的響應效率',
|
||||
'feature.asr.name': '語音識別',
|
||||
'feature.asr.description': '將語音轉換為文本，實現自然語言交互功能',
|
||||
|
||||
}
|
||||
@@ -8,7 +8,6 @@ import store from './store';
|
||||
import i18n from './i18n';
|
||||
import './styles/global.scss';
|
||||
import { register as registerServiceWorker } from './registerServiceWorker';
|
||||
import featureManager from './utils/featureManager';
|
||||
|
||||
// 创建事件总线，用于组件间通信
|
||||
Vue.prototype.$eventBus = new Vue();
|
||||
|
||||
@@ -76,8 +76,7 @@ const routes = [
|
||||
return import('../views/ModelConfig.vue')
|
||||
}
|
||||
},
|
||||
{
|
||||
path: '/params-management',
|
||||
{ path: '/params-management',
|
||||
name: 'ParamsManagement',
|
||||
component: function () {
|
||||
return import('../views/ParamsManagement.vue')
|
||||
@@ -87,8 +86,7 @@ const routes = [
|
||||
title: '参数管理'
|
||||
}
|
||||
},
|
||||
{
|
||||
path: '/knowledge-base-management',
|
||||
{ path: '/knowledge-base-management',
|
||||
name: 'KnowledgeBaseManagement',
|
||||
component: function () {
|
||||
return import('../views/KnowledgeBaseManagement.vue')
|
||||
@@ -98,8 +96,7 @@ const routes = [
|
||||
title: '知识库管理'
|
||||
}
|
||||
},
|
||||
{
|
||||
path: '/knowledge-file-upload',
|
||||
{ path: '/knowledge-file-upload',
|
||||
name: 'KnowledgeFileUpload',
|
||||
component: function () {
|
||||
return import('../views/KnowledgeFileUpload.vue')
|
||||
@@ -184,18 +181,6 @@ const routes = [
|
||||
return import('../views/TemplateQuickConfig.vue')
|
||||
}
|
||||
},
|
||||
// 功能配置页面路由
|
||||
{
|
||||
path: '/feature-management',
|
||||
name: 'FeatureManagement',
|
||||
component: function () {
|
||||
return import('../views/FeatureManagement.vue')
|
||||
},
|
||||
meta: {
|
||||
requiresAuth: true,
|
||||
title: '功能配置'
|
||||
}
|
||||
},
|
||||
]
|
||||
const router = new VueRouter({
|
||||
base: process.env.VUE_APP_PUBLIC_PATH || '/',
|
||||
|
||||
@@ -1,340 +0,0 @@
|
||||
//功能配置工具
|
||||
import Api from "@/apis/api";
|
||||
|
||||
class FeatureManager {
|
||||
constructor() {
|
||||
this.defaultFeatures = {
|
||||
voiceprintRecognition: {
|
||||
name: 'feature.voiceprintRecognition.name',
|
||||
enabled: false,
|
||||
description: 'feature.voiceprintRecognition.description'
|
||||
},
|
||||
voiceClone: {
|
||||
name: 'feature.voiceClone.name',
|
||||
enabled: false,
|
||||
description: 'feature.voiceClone.description'
|
||||
},
|
||||
knowledgeBase: {
|
||||
name: 'feature.knowledgeBase.name',
|
||||
enabled: false,
|
||||
description: 'feature.knowledgeBase.description'
|
||||
},
|
||||
mcpAccessPoint: {
|
||||
name: 'feature.mcpAccessPoint.name',
|
||||
enabled: false,
|
||||
description: 'feature.mcpAccessPoint.description'
|
||||
},
|
||||
vad: {
|
||||
name: 'feature.vad.name',
|
||||
enabled: false,
|
||||
description: 'feature.vad.description'
|
||||
},
|
||||
asr: {
|
||||
name: 'feature.asr.name',
|
||||
enabled: false,
|
||||
description: 'feature.asr.description'
|
||||
}
|
||||
};
|
||||
this.currentFeatures = { ...this.defaultFeatures }; // 当前内存中的配置
|
||||
this.initialized = false;
|
||||
this.initPromise = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* 等待初始化完成
|
||||
*/
|
||||
async waitForInitialization() {
|
||||
if (!this.initPromise) {
|
||||
this.initPromise = this.init();
|
||||
}
|
||||
await this.initPromise;
|
||||
return this.initialized;
|
||||
}
|
||||
|
||||
/**
|
||||
* 初始化功能配置
|
||||
*/
|
||||
async init() {
|
||||
try {
|
||||
// 从pub-config接口获取配置
|
||||
const config = await this.getConfigFromPubConfig();
|
||||
if (config) {
|
||||
this.currentFeatures = { ...config }; // 保存到内存
|
||||
this.initialized = true;
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('从pub-config接口获取配置失败:', error);
|
||||
}
|
||||
|
||||
// pub-config接口失败，使用默认配置
|
||||
this.currentFeatures = { ...this.defaultFeatures }; // 保存默认配置到内存
|
||||
this.initialized = true;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 从pub-config接口获取配置
|
||||
*/
|
||||
async getConfigFromPubConfig() {
|
||||
return new Promise((resolve) => {
|
||||
// 直接调用pub-config接口获取配置
|
||||
Api.user.getPubConfig((result) => {
|
||||
// 检查返回结果的结构
|
||||
if (result && result.status === 200) {
|
||||
// 检查是否有data字段
|
||||
if (result.data) {
|
||||
// 检查是否有code字段，如果有则按照code判断
|
||||
if (result.data.code !== undefined) {
|
||||
if (result.data.code === 0 && result.data.data && result.data.data.systemWebMenu) {
|
||||
try {
|
||||
let config;
|
||||
if (typeof result.data.data.systemWebMenu === 'string') {
|
||||
// 如果是字符串，需要解析JSON
|
||||
config = JSON.parse(result.data.data.systemWebMenu);
|
||||
} else {
|
||||
// 如果已经是对象，直接使用
|
||||
config = result.data.data.systemWebMenu;
|
||||
}
|
||||
|
||||
// 检查配置中是否包含features对象
|
||||
if (config && config.features) {
|
||||
// 确保knowledgeBase功能存在且配置正确
|
||||
if (!config.features.knowledgeBase) {
|
||||
console.warn('配置中缺少knowledgeBase功能，合并默认配置');
|
||||
config.features = { ...this.defaultFeatures, ...config.features };
|
||||
}
|
||||
resolve(config.features);
|
||||
} else {
|
||||
console.warn('配置中缺少features对象，使用默认配置');
|
||||
resolve(this.defaultFeatures);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('处理systemWebMenu配置失败:', error);
|
||||
resolve(null);
|
||||
}
|
||||
} else {
|
||||
console.warn('接口返回code不为0或缺少必要数据，使用默认配置');
|
||||
resolve(null);
|
||||
}
|
||||
} else {
|
||||
// 如果没有code字段，直接检查systemWebMenu
|
||||
if (result.data && result.data.systemWebMenu) {
|
||||
try {
|
||||
let config;
|
||||
if (typeof result.data.systemWebMenu === 'string') {
|
||||
// 如果是字符串，需要解析JSON
|
||||
config = JSON.parse(result.data.systemWebMenu);
|
||||
} else {
|
||||
// 如果已经是对象，直接使用
|
||||
config = result.data.systemWebMenu;
|
||||
}
|
||||
|
||||
// 检查配置中是否包含features对象
|
||||
if (config && config.features) {
|
||||
// 确保knowledgeBase功能存在且配置正确
|
||||
if (!config.features.knowledgeBase) {
|
||||
console.warn('配置中缺少knowledgeBase功能，合并默认配置');
|
||||
config.features = { ...this.defaultFeatures, ...config.features };
|
||||
}
|
||||
resolve(config.features);
|
||||
} else {
|
||||
console.warn('配置中缺少features对象，使用默认配置');
|
||||
resolve(this.defaultFeatures);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('处理systemWebMenu配置失败:', error);
|
||||
resolve(null);
|
||||
}
|
||||
} else {
|
||||
console.warn('接口返回缺少systemWebMenu数据，使用默认配置');
|
||||
resolve(null);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
console.warn('接口返回数据中缺少data字段，使用默认配置');
|
||||
resolve(null);
|
||||
}
|
||||
} else {
|
||||
console.warn('pub-config接口调用失败，使用默认配置');
|
||||
resolve(null);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取当前配置
|
||||
*/
|
||||
getCurrentConfig() {
|
||||
// 返回内存中的当前配置
|
||||
return this.currentFeatures;
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存配置到后端API
|
||||
*/
|
||||
async saveConfig(config) {
|
||||
try {
|
||||
// 更新内存中的配置
|
||||
this.currentFeatures = { ...config };
|
||||
|
||||
// 异步保存到后端API
|
||||
this.saveConfigToAPI(config).catch(error => {
|
||||
console.warn('保存配置到API失败:', error);
|
||||
});
|
||||
|
||||
// 触发配置变更事件
|
||||
window.dispatchEvent(new CustomEvent('featureConfigChanged', {
|
||||
detail: config
|
||||
}));
|
||||
} catch (error) {
|
||||
console.error('保存功能配置失败:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存配置到后端API
|
||||
*/
|
||||
async saveConfigToAPI(config) {
|
||||
return new Promise((resolve) => {
|
||||
// 直接使用已知的ID（600）更新参数
|
||||
Api.admin.updateParam(
|
||||
{
|
||||
id: 600,
|
||||
paramCode: 'system-web.menu',
|
||||
paramValue: JSON.stringify({
|
||||
features: config,
|
||||
groups: {
|
||||
featureManagement: ["voiceprintRecognition", "voiceClone", "knowledgeBase", "mcpAccessPoint"],
|
||||
voiceManagement: ["vad", "asr"]
|
||||
}
|
||||
}),
|
||||
valueType: 'json',
|
||||
remark: '系统功能菜单配置'
|
||||
},
|
||||
(updateResult) => {
|
||||
if (updateResult.code === 0) {
|
||||
resolve();
|
||||
} else {
|
||||
// 如果更新失败，可能是参数不存在或其他错误，记录但不阻止保存到localStorage
|
||||
console.warn('更新参数失败:', updateResult.msg);
|
||||
resolve(); // 不阻止保存到localStorage
|
||||
}
|
||||
},
|
||||
(error) => {
|
||||
console.warn('更新参数失败:', error);
|
||||
resolve(); // 不阻止保存到localStorage
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 获取所有功能配置
|
||||
*/
|
||||
getAllFeatures() {
|
||||
return this.getCurrentConfig();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取简化的配置对象（用于首页组件）
|
||||
*/
|
||||
getConfig() {
|
||||
const features = this.getAllFeatures();
|
||||
return {
|
||||
voiceprintRecognition: features.voiceprintRecognition?.enabled || false,
|
||||
voiceClone: features.voiceClone?.enabled || false,
|
||||
knowledgeBase: features.knowledgeBase?.enabled || false,
|
||||
mcpAccessPoint: features.mcpAccessPoint?.enabled || false,
|
||||
vad: features.vad?.enabled || false,
|
||||
asr: features.asr?.enabled || false
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取指定功能的状态
|
||||
*/
|
||||
getFeatureStatus(featureKey) {
|
||||
const features = this.getAllFeatures();
|
||||
return features[featureKey]?.enabled || false;
|
||||
}
|
||||
|
||||
/**
|
||||
* 设置功能状态
|
||||
*/
|
||||
setFeatureStatus(featureKey, enabled) {
|
||||
const features = this.getAllFeatures();
|
||||
if (features[featureKey]) {
|
||||
features[featureKey].enabled = enabled;
|
||||
this.saveConfig(features);
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* 启用功能
|
||||
*/
|
||||
enableFeature(featureKey) {
|
||||
return this.setFeatureStatus(featureKey, true);
|
||||
}
|
||||
|
||||
/**
|
||||
* 禁用功能
|
||||
*/
|
||||
disableFeature(featureKey) {
|
||||
return this.setFeatureStatus(featureKey, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* 切换功能状态
|
||||
*/
|
||||
toggleFeature(featureKey) {
|
||||
const currentStatus = this.getFeatureStatus(featureKey);
|
||||
return this.setFeatureStatus(featureKey, !currentStatus);
|
||||
}
|
||||
|
||||
/**
|
||||
* 重置所有功能为默认状态
|
||||
*/
|
||||
resetToDefault() {
|
||||
this.saveConfig(this.defaultFeatures);
|
||||
}
|
||||
|
||||
/**
|
||||
* 批量更新功能状态
|
||||
*/
|
||||
updateFeatures(featureUpdates) {
|
||||
const features = this.getAllFeatures();
|
||||
Object.keys(featureUpdates).forEach(featureKey => {
|
||||
if (features[featureKey]) {
|
||||
features[featureKey].enabled = featureUpdates[featureKey];
|
||||
}
|
||||
});
|
||||
this.saveConfig(features);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取已启用的功能列表
|
||||
*/
|
||||
getEnabledFeatures() {
|
||||
const features = this.getAllFeatures();
|
||||
return Object.keys(features).filter(key => features[key].enabled);
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查功能是否启用
|
||||
*/
|
||||
isFeatureEnabled(featureKey) {
|
||||
return this.getFeatureStatus(featureKey);
|
||||
}
|
||||
}
|
||||
|
||||
// 创建单例实例
|
||||
const featureManager = new FeatureManager();
|
||||
|
||||
export default featureManager;
|
||||
@@ -1,601 +0,0 @@
|
||||
<template>
|
||||
<div class="welcome">
|
||||
<HeaderBar />
|
||||
|
||||
<div class="operation-bar">
|
||||
<h2 class="page-title">{{ $t('header.featureManagement') }}</h2>
|
||||
</div>
|
||||
|
||||
<div class="main-wrapper">
|
||||
<div class="content-panel">
|
||||
<div class="content-area">
|
||||
<el-card class="feature-card" shadow="never">
|
||||
<div class="config-header">
|
||||
<div class="header-icon">
|
||||
<img loading="lazy" src="@/assets/home/equipment.png" alt="" />
|
||||
</div>
|
||||
<div class="header-actions">
|
||||
<el-button @click="!isSaving && toggleSelectAll()" class="btn-select-all" :disabled="isSaving">
|
||||
{{ isAllSelected ? $t('featureManagement.deselectAll') : $t('featureManagement.selectAll') }}
|
||||
</el-button>
|
||||
<el-button type="primary" class="save-btn" @click="handleSave" :disabled="isSaving">
|
||||
{{ isSaving ? $t('featureManagement.saving') : $t('featureManagement.save') }}
|
||||
</el-button>
|
||||
<el-button class="reset-btn" @click="handleReset" :disabled="isSaving">
|
||||
{{ $t('featureManagement.reset') }}
|
||||
</el-button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="divider"></div>
|
||||
|
||||
<!-- 功能分组容器 - 左右布局 -->
|
||||
<div class="feature-groups-container">
|
||||
<!-- 功能管理分组 -->
|
||||
<div v-if="featureManagementFeatures.length > 0" class="feature-group">
|
||||
<h3 class="group-title">{{ $t('featureManagement.group.featureManagement') }}</h3>
|
||||
<div class="features-grid">
|
||||
<div
|
||||
v-for="feature in featureManagementFeatures"
|
||||
:key="feature.id"
|
||||
class="feature-card-item"
|
||||
:class="{ 'feature-enabled': feature.enabled, 'feature-disabled': isSaving }"
|
||||
@click="!isSaving && toggleFeature(feature)"
|
||||
>
|
||||
<div class="feature-header">
|
||||
<h3 class="feature-name">{{ $t(`feature.${feature.id}.name`) }}</h3>
|
||||
<el-checkbox
|
||||
v-model="feature.enabled"
|
||||
@change="!isSaving && toggleFeature(feature)"
|
||||
class="feature-checkbox"
|
||||
:disabled="isSaving"
|
||||
/>
|
||||
</div>
|
||||
<p class="feature-description">{{ $t(`feature.${feature.id}.description`) }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 语音管理分组 -->
|
||||
<div v-if="voiceManagementFeatures.length > 0" class="feature-group">
|
||||
<h3 class="group-title">{{ $t('featureManagement.group.voiceManagement') }}</h3>
|
||||
<div class="features-grid">
|
||||
<div
|
||||
v-for="feature in voiceManagementFeatures"
|
||||
:key="feature.id"
|
||||
class="feature-card-item"
|
||||
:class="{ 'feature-enabled': feature.enabled, 'feature-disabled': isSaving }"
|
||||
@click="!isSaving && toggleFeature(feature)"
|
||||
>
|
||||
<div class="feature-header">
|
||||
<h3 class="feature-name">{{ $t(`feature.${feature.id}.name`) }}</h3>
|
||||
<el-checkbox
|
||||
v-model="feature.enabled"
|
||||
@change="!isSaving && toggleFeature(feature)"
|
||||
class="feature-checkbox"
|
||||
:disabled="isSaving"
|
||||
/>
|
||||
</div>
|
||||
<p class="feature-description">{{ $t(`feature.${feature.id}.description`) }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-if="filteredFeatures.length === 0" class="empty-state">
|
||||
<el-empty :description="$t('featureManagement.noFeatures')">
|
||||
<p class="empty-tip">{{ $t('featureManagement.contactAdmin') }}</p>
|
||||
</el-empty>
|
||||
</div>
|
||||
</el-card>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<el-footer>
|
||||
<VersionFooter />
|
||||
</el-footer>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
import HeaderBar from "@/components/HeaderBar.vue";
|
||||
import VersionFooter from "@/components/VersionFooter.vue";
|
||||
import featureManager from "@/utils/featureManager.js";
|
||||
|
||||
export default {
|
||||
name: "FeatureManagement",
|
||||
components: {
|
||||
HeaderBar,
|
||||
VersionFooter
|
||||
},
|
||||
data() {
|
||||
return {
|
||||
pendingChanges: false,
|
||||
featureManagementFeatures: [],
|
||||
voiceManagementFeatures: [],
|
||||
isSaving: false // 添加保存状态锁定
|
||||
}
|
||||
},
|
||||
computed: {
|
||||
// 所有功能列表
|
||||
filteredFeatures() {
|
||||
return [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
},
|
||||
|
||||
// 判断是否所有功能都已选中
|
||||
isAllSelected() {
|
||||
const allFeatures = [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
return allFeatures.length > 0 && allFeatures.every(feature => feature.enabled)
|
||||
}
|
||||
},
|
||||
async created() {
|
||||
// 等待功能配置管理器初始化完成
|
||||
try {
|
||||
console.log('等待功能配置管理器初始化...')
|
||||
await featureManager.waitForInitialization()
|
||||
console.log('功能配置管理器初始化完成，开始加载功能配置')
|
||||
await this.loadFeatures()
|
||||
this.setupConfigChangeListener()
|
||||
} catch (error) {
|
||||
console.error('功能配置管理器初始化等待失败:', error)
|
||||
await this.loadFeatures()
|
||||
this.setupConfigChangeListener()
|
||||
}
|
||||
},
|
||||
|
||||
beforeDestroy() {
|
||||
this.removeConfigChangeListener()
|
||||
},
|
||||
|
||||
methods: {
|
||||
// 根据ID列表获取功能
|
||||
async getFeaturesByIds(featureIds) {
|
||||
try {
|
||||
const featureConfig = await featureManager.getAllFeatures()
|
||||
console.log('获取到的功能配置:', JSON.stringify(featureConfig, null, 2))
|
||||
console.log('请求的功能ID列表:', featureIds)
|
||||
|
||||
const result = featureIds.map(id => {
|
||||
const feature = featureConfig[id]
|
||||
console.log(`功能 ${id} 的配置:`, feature)
|
||||
console.log(`功能 ${id} 的启用状态:`, feature?.enabled)
|
||||
|
||||
return {
|
||||
id: id,
|
||||
name: this.$t(`feature.${id}.name`),
|
||||
description: this.$t(`feature.${id}.description`),
|
||||
enabled: feature?.enabled || false
|
||||
}
|
||||
})
|
||||
|
||||
console.log('最终返回的功能列表:', JSON.stringify(result, null, 2))
|
||||
return result
|
||||
} catch (error) {
|
||||
console.error('获取功能配置失败:', error)
|
||||
// 如果获取失败，返回默认配置
|
||||
return featureIds.map(id => ({
|
||||
id: id,
|
||||
name: this.$t(`feature.${id}.name`),
|
||||
description: this.$t(`feature.${id}.description`),
|
||||
enabled: false
|
||||
}))
|
||||
}
|
||||
},
|
||||
|
||||
// 加载功能配置
|
||||
async loadFeatures() {
|
||||
// 保存当前用户的选择状态
|
||||
const currentFeatureStates = {}
|
||||
const allCurrentFeatures = [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
allCurrentFeatures.forEach(feature => {
|
||||
currentFeatureStates[feature.id] = feature.enabled
|
||||
})
|
||||
|
||||
// 重新加载配置
|
||||
this.featureManagementFeatures = await this.getFeaturesByIds(['voiceprintRecognition', 'voiceClone', 'knowledgeBase', 'mcpAccessPoint'])
|
||||
this.voiceManagementFeatures = await this.getFeaturesByIds(['vad', 'asr'])
|
||||
|
||||
// 恢复用户的选择状态（如果存在）
|
||||
const allFeatures = [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
allFeatures.forEach(feature => {
|
||||
if (currentFeatureStates.hasOwnProperty(feature.id)) {
|
||||
feature.enabled = currentFeatureStates[feature.id]
|
||||
}
|
||||
})
|
||||
},
|
||||
// 切换功能状态
|
||||
async toggleFeature(feature) {
|
||||
// 如果正在保存，阻止操作
|
||||
if (this.isSaving) {
|
||||
return
|
||||
}
|
||||
|
||||
feature.enabled = !feature.enabled
|
||||
this.pendingChanges = true
|
||||
|
||||
// 不再立即更新到配置管理器，只在保存时统一更新
|
||||
},
|
||||
// 保存配置
|
||||
async handleSave() {
|
||||
if (!this.pendingChanges) {
|
||||
this.$message.info({
|
||||
message: this.$t('featureManagement.noChanges'),
|
||||
showClose: true
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
// 设置保存状态，锁定界面
|
||||
this.isSaving = true
|
||||
|
||||
try {
|
||||
// 获取当前所有功能的状态并保存
|
||||
const featureUpdates = {}
|
||||
const allFeatures = [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
allFeatures.forEach(feature => {
|
||||
featureUpdates[feature.id] = feature.enabled
|
||||
})
|
||||
await featureManager.updateFeatures(featureUpdates)
|
||||
|
||||
this.pendingChanges = false
|
||||
this.$message.success({
|
||||
message: this.$t('featureManagement.saveSuccess'),
|
||||
showClose: true
|
||||
})
|
||||
|
||||
setTimeout(() => {
|
||||
this.loadFeatures()
|
||||
this.$router.go(0)
|
||||
}, 1000)
|
||||
} catch (error) {
|
||||
console.error('保存配置失败:', error)
|
||||
this.$message.error({
|
||||
message: this.$t('featureManagement.saveError'),
|
||||
showClose: true
|
||||
})
|
||||
} finally {
|
||||
// 无论成功与否，都解除保存状态锁定
|
||||
this.isSaving = false
|
||||
}
|
||||
},
|
||||
// 设置配置变化监听器
|
||||
setupConfigChangeListener() {
|
||||
this.configChangeHandler = () => {
|
||||
console.log('检测到配置变化，重新加载功能列表')
|
||||
this.loadFeatures()
|
||||
}
|
||||
window.addEventListener('featureConfigReloaded', this.configChangeHandler)
|
||||
},
|
||||
|
||||
// 移除配置变化监听器
|
||||
removeConfigChangeListener() {
|
||||
if (this.configChangeHandler) {
|
||||
window.removeEventListener('featureConfigReloaded', this.configChangeHandler)
|
||||
}
|
||||
},
|
||||
|
||||
// 重置配置
|
||||
async handleReset() {
|
||||
try {
|
||||
await this.$confirm(
|
||||
this.$t('featureManagement.resetConfirm'),
|
||||
this.$t('featureManagement.reset'),
|
||||
{
|
||||
confirmButtonText: this.$t('featureManagement.confirm'),
|
||||
cancelButtonText: this.$t('featureManagement.cancel'),
|
||||
type: 'warning'
|
||||
}
|
||||
)
|
||||
|
||||
featureManager.resetToDefault()
|
||||
this.loadFeatures()
|
||||
this.pendingChanges = false
|
||||
|
||||
this.$message.success({
|
||||
message: this.$t('featureManagement.resetSuccess'),
|
||||
showClose: true
|
||||
})
|
||||
|
||||
setTimeout(() => {
|
||||
this.loadFeatures()
|
||||
this.$router.go(0)
|
||||
}, 1000)
|
||||
} catch (error) {
|
||||
// 用户取消操作
|
||||
}
|
||||
},
|
||||
// 搜索功能（预留接口）
|
||||
handleSearch() {
|
||||
// 搜索功能待实现
|
||||
},
|
||||
// 全选/取消全选
|
||||
toggleSelectAll() {
|
||||
// 如果正在保存，阻止操作
|
||||
if (this.isSaving) {
|
||||
return
|
||||
}
|
||||
|
||||
const allFeatures = [...this.featureManagementFeatures, ...this.voiceManagementFeatures]
|
||||
const newStatus = !this.isAllSelected
|
||||
|
||||
allFeatures.forEach(feature => {
|
||||
feature.enabled = newStatus
|
||||
})
|
||||
|
||||
this.pendingChanges = true
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.welcome {
|
||||
min-width: 900px;
|
||||
min-height: 506px;
|
||||
height: 100vh;
|
||||
display: flex;
|
||||
position: relative;
|
||||
flex-direction: column;
|
||||
background-size: cover;
|
||||
background: linear-gradient(to bottom right, #dce8ff, #e4eeff, #e6cbfd) center;
|
||||
-webkit-background-size: cover;
|
||||
-o-background-size: cover;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.operation-bar {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 16px 24px;
|
||||
}
|
||||
|
||||
.page-title {
|
||||
font-size: 24px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.config-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 0 0 16px 0;
|
||||
}
|
||||
|
||||
.header-icon {
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
background: #5778ff;
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
margin-right: 12px;
|
||||
}
|
||||
|
||||
.header-icon img {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
}
|
||||
|
||||
.header-actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.divider {
|
||||
height: 1px;
|
||||
background: #e0e0e0;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.btn-select-all {
|
||||
background: #e6ebff;
|
||||
color: #5778ff;
|
||||
border: 1px solid #adbdff;
|
||||
border-radius: 18px;
|
||||
padding: 8px 16px;
|
||||
height: 32px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.btn-select-all:hover {
|
||||
background: #d0d8ff;
|
||||
}
|
||||
|
||||
.save-btn {
|
||||
background: #5778ff;
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 18px;
|
||||
padding: 8px 16px;
|
||||
height: 32px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.save-btn:hover {
|
||||
background: #4a6ae8;
|
||||
}
|
||||
|
||||
.reset-btn {
|
||||
background: #e6ebff;
|
||||
color: #5778ff;
|
||||
border: 1px solid #adbdff;
|
||||
border-radius: 18px;
|
||||
padding: 8px 16px;
|
||||
height: 32px;
|
||||
}
|
||||
|
||||
.reset-btn:hover {
|
||||
background: #d0d8ff;
|
||||
}
|
||||
|
||||
.main-wrapper {
|
||||
margin: 0 22px 5px 22px;
|
||||
border-radius: 15px;
|
||||
min-height: calc(100vh - 24vh);
|
||||
height: auto;
|
||||
max-height: 80vh;
|
||||
box-shadow: 0 2px 12px rgba(0, 0, 0, 0.1);
|
||||
position: relative;
|
||||
background: rgba(237, 242, 255, 0.5);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.content-panel {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
overflow: hidden;
|
||||
height: 100%;
|
||||
border-radius: 15px;
|
||||
background: transparent;
|
||||
border: 1px solid #fff;
|
||||
}
|
||||
|
||||
.content-area {
|
||||
flex: 1;
|
||||
height: 100%;
|
||||
min-width: 600px;
|
||||
overflow: auto;
|
||||
background-color: white;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.feature-card {
|
||||
background: white;
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
border: none;
|
||||
box-shadow: none;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.feature-card ::v-deep .el-card__body {
|
||||
padding: 24px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.features-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
gap: 16px;
|
||||
}
|
||||
|
||||
.feature-card-item {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 20px;
|
||||
border-radius: 12px;
|
||||
border: 2px solid #e0e0e0;
|
||||
background-color: white;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
user-select: none;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.feature-card-item:hover {
|
||||
border-color: #869bf0;
|
||||
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.feature-card-item.feature-enabled {
|
||||
border-color:#5778ff;
|
||||
box-shadow: 0 4px 16px rgba(95, 112, 243, 0.2);
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.feature-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.feature-checkbox ::v-deep .el-checkbox__input {
|
||||
transform: scale(1.2);
|
||||
}
|
||||
|
||||
.feature-checkbox ::v-deep .el-checkbox__input.is-checked .el-checkbox__inner {
|
||||
background-color: #5778ff;
|
||||
border-color: #5778ff;
|
||||
}
|
||||
|
||||
.feature-checkbox ::v-deep .el-checkbox__input.is-checked + .el-checkbox__label {
|
||||
color: #5778ff;
|
||||
}
|
||||
|
||||
|
||||
.feature-name {
|
||||
font-size: 18px;
|
||||
font-weight: 600;
|
||||
color: #333;
|
||||
margin: 0;
|
||||
transition: color 0.3s ease;
|
||||
}
|
||||
|
||||
|
||||
.feature-description {
|
||||
font-size: 14px;
|
||||
line-height: 1.6;
|
||||
color: #666;
|
||||
margin: 0 0 12px 0;
|
||||
transition: color 0.3s ease;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
|
||||
/* 功能分组容器 - 左右布局 */
|
||||
.feature-groups-container {
|
||||
display: flex;
|
||||
gap: 32px;
|
||||
align-items: flex-start;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
/* 分组之间的分隔线 */
|
||||
.feature-groups-container::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
left: 50%;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 1px;
|
||||
height: 550px;
|
||||
background: #e0e0e0;
|
||||
opacity: 0.5;
|
||||
transform: translateX(-50%);
|
||||
}
|
||||
|
||||
/* 分组样式 */
|
||||
.feature-group {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
margin-bottom: 32px;
|
||||
}
|
||||
|
||||
.group-title {
|
||||
font-size: 18px;
|
||||
font-weight: 600;
|
||||
color: #303133;
|
||||
margin-bottom: 12px;
|
||||
padding-left: 12px;
|
||||
border-left: 4px solid #5f70f3;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.features-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
gap: 16px;
|
||||
}
|
||||
</style>
|
||||
@@ -214,11 +214,10 @@
|
||||
<el-dialog :title="$t('knowledgeFileUpload.retrievalTest')" :visible.sync="retrievalTestDialogVisible"
|
||||
width="1200px" class="retrieval-test-dialog">
|
||||
<div class="retrieval-test-form">
|
||||
<el-form :model="retrievalTestForm" label-width="80px" @submit.native.prevent="runRetrievalTest">
|
||||
<el-form :model="retrievalTestForm" label-width="80px">
|
||||
<el-form-item :label="$t('knowledgeFileUpload.testQuestion')" required>
|
||||
<el-input v-model="retrievalTestForm.question"
|
||||
:placeholder="$t('knowledgeFileUpload.testQuestionPlaceholder')" style="width: 100%; max-height: 80px;"
|
||||
@keyup.enter.native="runRetrievalTest">
|
||||
:placeholder="$t('knowledgeFileUpload.testQuestionPlaceholder')" style="width: 100%; max-height: 80px;">
|
||||
</el-input>
|
||||
</el-form-item>
|
||||
</el-form>
|
||||
|
||||
@@ -38,11 +38,11 @@
|
||||
position: absolute;
|
||||
top: 50%;
|
||||
transform: translateY(-50%);
|
||||
right: 18%;
|
||||
right: 15%;
|
||||
background-color: #fff;
|
||||
border-radius: 20px;
|
||||
padding: 35px 0;
|
||||
width: 450px;
|
||||
width: 550px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
|
||||
@@ -39,9 +39,8 @@
|
||||
</template>
|
||||
|
||||
<template v-else>
|
||||
<DeviceItem v-for="(item, index) in devices" :key="index" :device="item" :feature-status="featureStatus"
|
||||
@configure="goToRoleConfig" @deviceManage="handleDeviceManage" @delete="handleDeleteAgent"
|
||||
@chat-history="handleShowChatHistory" />
|
||||
<DeviceItem v-for="(item, index) in devices" :key="index" :device="item" @configure="goToRoleConfig"
|
||||
@deviceManage="handleDeviceManage" @delete="handleDeleteAgent" @chat-history="handleShowChatHistory" />
|
||||
</template>
|
||||
</div>
|
||||
</div>
|
||||
@@ -62,7 +61,6 @@ import ChatHistoryDialog from '@/components/ChatHistoryDialog.vue';
|
||||
import DeviceItem from '@/components/DeviceItem.vue';
|
||||
import HeaderBar from '@/components/HeaderBar.vue';
|
||||
import VersionFooter from '@/components/VersionFooter.vue';
|
||||
import featureManager from '@/utils/featureManager';
|
||||
|
||||
export default {
|
||||
name: 'HomePage',
|
||||
@@ -78,33 +76,15 @@ export default {
|
||||
skeletonCount: localStorage.getItem('skeletonCount') || 8,
|
||||
showChatHistory: false,
|
||||
currentAgentId: '',
|
||||
currentAgentName: '',
|
||||
// 功能状态
|
||||
featureStatus: {
|
||||
voiceprintRecognition: false,
|
||||
voiceClone: false,
|
||||
knowledgeBase: false
|
||||
}
|
||||
currentAgentName: ''
|
||||
}
|
||||
},
|
||||
|
||||
async mounted() {
|
||||
mounted() {
|
||||
this.fetchAgentList();
|
||||
await this.loadFeatureStatus();
|
||||
},
|
||||
|
||||
methods: {
|
||||
// 加载功能状态
|
||||
async loadFeatureStatus() {
|
||||
await featureManager.waitForInitialization();
|
||||
const config = featureManager.getConfig();
|
||||
this.featureStatus = {
|
||||
voiceprintRecognition: config.voiceprintRecognition,
|
||||
voiceClone: config.voiceClone,
|
||||
knowledgeBase: config.knowledgeBase
|
||||
};
|
||||
},
|
||||
|
||||
showAddDialog() {
|
||||
this.addDeviceDialogVisible = true
|
||||
},
|
||||
|
||||
@@ -156,7 +156,6 @@ import VersionFooter from "@/components/VersionFooter.vue";
|
||||
import i18n, { changeLanguage } from "@/i18n";
|
||||
import { getUUID, goToPage, showDanger, showSuccess, sm2Encrypt, validateMobile } from "@/utils";
|
||||
import { mapState } from "vuex";
|
||||
import featureManager from "@/utils/featureManager";
|
||||
|
||||
export default {
|
||||
name: "login",
|
||||
@@ -215,13 +214,6 @@ export default {
|
||||
this.$store.dispatch("fetchPubConfig").then(() => {
|
||||
// 根据配置决定默认登录方式
|
||||
this.isMobileLogin = this.enableMobileRegister;
|
||||
|
||||
// pub-config接口调用完成后，重新初始化featureManager以确保使用最新的配置
|
||||
featureManager.waitForInitialization().then(() => {
|
||||
console.log('featureManager重新初始化完成，使用pub-config配置');
|
||||
}).catch(error => {
|
||||
console.warn('featureManager重新初始化失败:', error);
|
||||
});
|
||||
});
|
||||
},
|
||||
methods: {
|
||||
|
||||
@@ -55,24 +55,10 @@
|
||||
</div>
|
||||
</div>
|
||||
</el-form-item>
|
||||
<el-form-item :label="$t('roleConfig.contextProvider') + '：'" class="context-provider-item">
|
||||
<div style="display: flex; align-items: center; justify-content: space-between;">
|
||||
<span style="color: #606266; font-size: 13px;">
|
||||
{{ $t('roleConfig.contextProviderSuccess', { count: currentContextProviders.length }) }}<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/context-provider-integration.md" target="_blank" class="doc-link">{{ $t('roleConfig.contextProviderDocLink') }}</a>
|
||||
</span>
|
||||
<el-button
|
||||
class="edit-function-btn"
|
||||
size="small"
|
||||
@click="openContextProviderDialog"
|
||||
>
|
||||
{{ $t('roleConfig.editContextProvider') }}
|
||||
</el-button>
|
||||
</div>
|
||||
</el-form-item>
|
||||
<el-form-item :label="$t('roleConfig.roleIntroduction') + '：'">
|
||||
<el-input
|
||||
type="textarea"
|
||||
rows="8"
|
||||
rows="9"
|
||||
resize="none"
|
||||
:placeholder="$t('roleConfig.pleaseEnterContent')"
|
||||
v-model="form.systemPrompt"
|
||||
@@ -85,7 +71,7 @@
|
||||
<el-form-item :label="$t('roleConfig.memoryHis') + '：'">
|
||||
<el-input
|
||||
type="textarea"
|
||||
rows="4"
|
||||
rows="6"
|
||||
resize="none"
|
||||
v-model="form.summaryMemory"
|
||||
maxlength="2000"
|
||||
@@ -121,11 +107,7 @@
|
||||
</div>
|
||||
<div class="form-column">
|
||||
<div class="model-row">
|
||||
<el-form-item
|
||||
v-if="featureStatus.vad"
|
||||
:label="$t('roleConfig.vad')"
|
||||
class="model-item"
|
||||
>
|
||||
<el-form-item :label="$t('roleConfig.vad')" class="model-item">
|
||||
<div class="model-select-wrapper">
|
||||
<el-select
|
||||
v-model="form.model.vadModelId"
|
||||
@@ -143,11 +125,7 @@
|
||||
</el-select>
|
||||
</div>
|
||||
</el-form-item>
|
||||
<el-form-item
|
||||
v-if="featureStatus.asr"
|
||||
:label="$t('roleConfig.asr')"
|
||||
class="model-item"
|
||||
>
|
||||
<el-form-item :label="$t('roleConfig.asr')" class="model-item">
|
||||
<div class="model-select-wrapper">
|
||||
<el-select
|
||||
v-model="form.model.asrModelId"
|
||||
@@ -289,11 +267,6 @@
|
||||
@update-functions="handleUpdateFunctions"
|
||||
@dialog-closed="handleDialogClosed"
|
||||
/>
|
||||
<context-provider-dialog
|
||||
:visible.sync="showContextProviderDialog"
|
||||
:providers="currentContextProviders"
|
||||
@confirm="handleUpdateContext"
|
||||
/>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
@@ -302,17 +275,14 @@ import Api from "@/apis/api";
|
||||
import { getServiceUrl } from "@/apis/api";
|
||||
import RequestService from "@/apis/httpRequest";
|
||||
import FunctionDialog from "@/components/FunctionDialog.vue";
|
||||
import ContextProviderDialog from "@/components/ContextProviderDialog.vue";
|
||||
import HeaderBar from "@/components/HeaderBar.vue";
|
||||
import i18n from "@/i18n";
|
||||
import featureManager from "@/utils/featureManager";
|
||||
|
||||
export default {
|
||||
name: "RoleConfigPage",
|
||||
components: { HeaderBar, FunctionDialog, ContextProviderDialog },
|
||||
components: { HeaderBar, FunctionDialog },
|
||||
data() {
|
||||
return {
|
||||
showContextProviderDialog: false,
|
||||
form: {
|
||||
agentCode: "",
|
||||
agentName: "",
|
||||
@@ -350,18 +320,12 @@ export default {
|
||||
voiceDetails: {}, // 保存完整的音色信息
|
||||
showFunctionDialog: false,
|
||||
currentFunctions: [],
|
||||
currentContextProviders: [],
|
||||
allFunctions: [],
|
||||
originalFunctions: [],
|
||||
playingVoice: false,
|
||||
isPaused: false,
|
||||
currentAudio: null,
|
||||
currentPlayingVoiceId: null,
|
||||
// 功能状态
|
||||
featureStatus: {
|
||||
vad: false, // 语言检测活动功能状态
|
||||
asr: false, // 语音识别功能状态
|
||||
},
|
||||
};
|
||||
},
|
||||
methods: {
|
||||
@@ -392,7 +356,6 @@ export default {
|
||||
paramInfo: item.params,
|
||||
};
|
||||
}),
|
||||
contextProviders: this.currentContextProviders,
|
||||
};
|
||||
Api.agent.updateAgentConfig(this.$route.query.agentId, configData, ({ data }) => {
|
||||
if (data.code === 0) {
|
||||
@@ -509,9 +472,6 @@ export default {
|
||||
};
|
||||
// 后端只给了最小映射：[{ id, agentId, pluginId }, ...]
|
||||
const savedMappings = data.data.functions || [];
|
||||
|
||||
// 加载上下文配置
|
||||
this.currentContextProviders = data.data.contextProviders || [];
|
||||
|
||||
// 先保证 allFunctions 已经加载（如果没有，则先 fetchAllFunctions）
|
||||
const ensureFuncs = this.allFunctions.length
|
||||
@@ -644,14 +604,15 @@ export default {
|
||||
if (type === "Intent" && value !== "Intent_nointent") {
|
||||
this.fetchAllFunctions();
|
||||
}
|
||||
if (type === "Memory") {
|
||||
if (value === "Memory_nomem") {
|
||||
// 无记忆功能的模型，默认不记录聊天记录
|
||||
this.form.chatHistoryConf = 0;
|
||||
} else {
|
||||
// 有记忆功能的模型，默认记录文本和语音
|
||||
this.form.chatHistoryConf = 2;
|
||||
}
|
||||
if (type === "Memory" && value === "Memory_nomem") {
|
||||
this.form.chatHistoryConf = 0;
|
||||
}
|
||||
if (
|
||||
type === "Memory" &&
|
||||
value !== "Memory_nomem" &&
|
||||
(this.form.chatHistoryConf === 0 || this.form.chatHistoryConf === null)
|
||||
) {
|
||||
this.form.chatHistoryConf = 2;
|
||||
}
|
||||
if (type === "LLM") {
|
||||
// 当LLM类型改变时，更新意图识别选项的可见性
|
||||
@@ -686,12 +647,6 @@ export default {
|
||||
this.showFunctionDialog = true;
|
||||
}
|
||||
},
|
||||
openContextProviderDialog() {
|
||||
this.showContextProviderDialog = true;
|
||||
},
|
||||
handleUpdateContext(providers) {
|
||||
this.currentContextProviders = providers;
|
||||
},
|
||||
handleUpdateFunctions(selected) {
|
||||
this.currentFunctions = selected;
|
||||
},
|
||||
@@ -1026,19 +981,6 @@ export default {
|
||||
this.form.chatHistoryConf = 0;
|
||||
}
|
||||
},
|
||||
// 加载功能状态
|
||||
async loadFeatureStatus() {
|
||||
try {
|
||||
// 确保featureManager已初始化完成
|
||||
await featureManager.waitForInitialization();
|
||||
const config = featureManager.getConfig();
|
||||
this.featureStatus.voiceprintRecognition = config.voiceprintRecognition || false;
|
||||
this.featureStatus.vad = config.vad || false;
|
||||
this.featureStatus.asr = config.asr || false;
|
||||
} catch (error) {
|
||||
console.error("加载功能状态失败:", error);
|
||||
}
|
||||
},
|
||||
},
|
||||
watch: {
|
||||
"form.model.ttsModelId": {
|
||||
@@ -1061,7 +1003,7 @@ export default {
|
||||
immediate: true,
|
||||
},
|
||||
},
|
||||
async mounted() {
|
||||
mounted() {
|
||||
const agentId = this.$route.query.agentId;
|
||||
if (agentId) {
|
||||
this.fetchAgentConfig(agentId);
|
||||
@@ -1069,8 +1011,6 @@ export default {
|
||||
}
|
||||
this.fetchModelOptions();
|
||||
this.fetchTemplates();
|
||||
// 加载功能状态，确保featureManager已初始化
|
||||
await this.loadFeatureStatus();
|
||||
},
|
||||
};
|
||||
</script>
|
||||
@@ -1230,8 +1170,7 @@ export default {
|
||||
|
||||
.template-item {
|
||||
height: 4vh;
|
||||
min-width: 60px;
|
||||
padding: 0 12px;
|
||||
width: 76px;
|
||||
border-radius: 8px;
|
||||
background: #e6ebff;
|
||||
line-height: 4vh;
|
||||
@@ -1241,7 +1180,6 @@ export default {
|
||||
color: #5778ff;
|
||||
cursor: pointer;
|
||||
transition: background-color 0.3s ease;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.template-item:hover {
|
||||
@@ -1359,26 +1297,6 @@ export default {
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
.chat-history-options ::v-deep .el-radio-button {
|
||||
border-color: #5778ff;
|
||||
}
|
||||
|
||||
.chat-history-options ::v-deep .el-radio-button .el-radio-button__inner {
|
||||
color: #5778ff;
|
||||
border-color: #5778ff;
|
||||
background-color: transparent;
|
||||
}
|
||||
|
||||
.chat-history-options ::v-deep .el-radio-button.is-active .el-radio-button__inner {
|
||||
background-color: #5778ff;
|
||||
border-color: #5778ff;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.chat-history-options ::v-deep .el-radio-button .el-radio-button__inner:hover {
|
||||
color: #5778ff;
|
||||
}
|
||||
|
||||
.header-actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
@@ -1426,18 +1344,4 @@ export default {
|
||||
height: 32px;
|
||||
margin-left: 8px;
|
||||
}
|
||||
|
||||
.context-provider-item ::v-deep .el-form-item__label {
|
||||
line-height: 42px !important;
|
||||
}
|
||||
|
||||
.doc-link {
|
||||
color: #5778ff;
|
||||
text-decoration: none;
|
||||
margin-left: 4px;
|
||||
|
||||
&:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
|
||||
@@ -56,7 +56,6 @@
|
||||
3. **洞察需求：** 结合上下文**深入理解用户真实意图**后再决定调用，避免无意义调用。
|
||||
4. **独立任务：** 除`<context>`已涵盖信息外，用户每个要求（即使相似）都视为**独立任务**，需调用工具获取最新数据，**不可偷懒复用历史结果**。
|
||||
5. **不确定时：** **切勿猜测或编造答案**。若不确定相关操作，可引导用户澄清或告知能力限制。
|
||||
6. **多工具调用：** 当用户要求执行多个任务时，你会调用多个工具（数量不定）。**重要：在获取到所有工具结果后，你必须依次总结每个工具的查询结果**，不要遗漏任何一个。例如用户问"设备当前状态，某某地方的天气和社会新闻"，你要先说设备状态，再说天气情况，最后说新闻内容。
|
||||
- **重要例外（无需调用）：**
|
||||
- `查询"现在的时间"、"今天的日期/星期几"、"今天农历"、"{{local_address}}的天气/未来天气"` -> **直接使用`<context>`信息回复**。
|
||||
- **需要调用的情况（示例）：**
|
||||
@@ -74,7 +73,6 @@
|
||||
- **今天农历：** {{lunar_date}}
|
||||
- **用户所在城市：** {{local_address}}
|
||||
- **当地未来7天天气：** {{weather_info}}
|
||||
{{ dynamic_context }}
|
||||
</context>
|
||||
|
||||
<memory>
|
||||
|
||||
@@ -9,7 +9,6 @@ from core.utils.util import get_local_ip, validate_mcp_endpoint
|
||||
from core.http_server import SimpleHttpServer
|
||||
from core.websocket_server import WebSocketServer
|
||||
from core.utils.util import check_ffmpeg_installed
|
||||
from core.utils.gc_manager import get_gc_manager
|
||||
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
@@ -64,10 +63,6 @@ async def main():
|
||||
# 添加 stdin 监控任务
|
||||
stdin_task = asyncio.create_task(monitor_stdin())
|
||||
|
||||
# 启动全局GC管理器（5分钟清理一次）
|
||||
gc_manager = get_gc_manager(interval_seconds=300)
|
||||
await gc_manager.start()
|
||||
|
||||
# 启动 WebSocket 服务器
|
||||
ws_server = WebSocketServer(config)
|
||||
ws_task = asyncio.create_task(ws_server.start())
|
||||
@@ -127,9 +122,6 @@ async def main():
|
||||
except asyncio.CancelledError:
|
||||
print("任务被取消，清理资源中...")
|
||||
finally:
|
||||
# 停止全局GC管理器
|
||||
await gc_manager.stop()
|
||||
|
||||
# 取消所有任务（关键修复点）
|
||||
stdin_task.cancel()
|
||||
ws_task.cancel()
|
||||
|
||||
@@ -113,15 +113,6 @@ wakeup_words:
|
||||
# MCP接入点地址，地址格式为：ws://你的mcp接入点ip或者域名:端口号/mcp/?token=你的token
|
||||
# 详细教程 https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mcp-endpoint-integration.md
|
||||
mcp_endpoint: 你的接入点 websocket地址
|
||||
|
||||
# 上下文源配置
|
||||
# 用于在系统提示词中注入动态数据，如健康数据、股票信息等
|
||||
# 可以添加多个上下文源
|
||||
context_providers:
|
||||
- url: ""
|
||||
headers:
|
||||
Authorization: ""
|
||||
|
||||
# 插件的基础配置
|
||||
plugins:
|
||||
# 获取天气插件的配置，这里填写你的api_key
|
||||
@@ -471,6 +462,7 @@ ASR:
|
||||
domain: slm # 识别领域，iat:日常用语，medical:医疗，finance:金融等
|
||||
language: zh_cn # 语言，zh_cn:中文，en_us:英文
|
||||
accent: mandarin # 方言，mandarin:普通话
|
||||
dwa: wpgs # 动态修正，wpgs:实时返回中间结果
|
||||
# 调整音频处理参数以提高长语音识别质量
|
||||
output_dir: tmp/
|
||||
|
||||
@@ -495,6 +487,7 @@ LLM:
|
||||
temperature: 0.7 # 温度值
|
||||
max_tokens: 500 # 最大生成token数
|
||||
top_p: 1
|
||||
top_k: 50
|
||||
frequency_penalty: 0 # 频率惩罚
|
||||
AliAppLLM:
|
||||
# 定义LLM API类型
|
||||
|
||||
@@ -32,16 +32,7 @@ def load_config():
|
||||
custom_config = read_config(custom_config_path)
|
||||
|
||||
if custom_config.get("manager-api", {}).get("url"):
|
||||
import asyncio
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
# 如果已经在事件循环中，使用异步版本
|
||||
config = asyncio.run_coroutine_threadsafe(
|
||||
get_config_from_api_async(custom_config), loop
|
||||
).result()
|
||||
except RuntimeError:
|
||||
# 如果不在事件循环中（启动时），创建新的事件循环
|
||||
config = asyncio.run(get_config_from_api_async(custom_config))
|
||||
config = get_config_from_api(custom_config)
|
||||
else:
|
||||
# 合并配置
|
||||
config = merge_configs(default_config, custom_config)
|
||||
@@ -53,13 +44,13 @@ def load_config():
|
||||
return config
|
||||
|
||||
|
||||
async def get_config_from_api_async(config):
|
||||
"""从Java API获取配置（异步版本）"""
|
||||
def get_config_from_api(config):
|
||||
"""从Java API获取配置"""
|
||||
# 初始化API客户端
|
||||
init_service(config)
|
||||
|
||||
# 获取服务器配置
|
||||
config_data = await get_server_config()
|
||||
config_data = get_server_config()
|
||||
if config_data is None:
|
||||
raise Exception("Failed to fetch server config from API")
|
||||
|
||||
@@ -68,7 +59,6 @@ async def get_config_from_api_async(config):
|
||||
"url": config["manager-api"].get("url", ""),
|
||||
"secret": config["manager-api"].get("secret", ""),
|
||||
}
|
||||
auth_enabled = config_data.get("server", {}).get("auth", {}).get("enabled", False)
|
||||
# server的配置以本地为准
|
||||
if config.get("server"):
|
||||
config_data["server"] = {
|
||||
@@ -78,16 +68,15 @@ async def get_config_from_api_async(config):
|
||||
"vision_explain": config["server"].get("vision_explain", ""),
|
||||
"auth_key": config["server"].get("auth_key", ""),
|
||||
}
|
||||
config_data["server"]["auth"] = {"enabled": auth_enabled}
|
||||
# 如果服务器没有prompt_template，则从本地配置读取
|
||||
if not config_data.get("prompt_template"):
|
||||
config_data["prompt_template"] = config.get("prompt_template")
|
||||
return config_data
|
||||
|
||||
|
||||
async def get_private_config_from_api(config, device_id, client_id):
|
||||
def get_private_config_from_api(config, device_id, client_id):
|
||||
"""从Java API获取私有配置"""
|
||||
return await get_agent_models(device_id, client_id, config["selected_module"])
|
||||
return get_agent_models(device_id, client_id, config["selected_module"])
|
||||
|
||||
|
||||
def ensure_directories(config):
|
||||
|
||||
@@ -5,7 +5,7 @@ from config.config_loader import load_config
|
||||
from config.settings import check_config_file
|
||||
from datetime import datetime
|
||||
|
||||
SERVER_VERSION = "0.8.10"
|
||||
SERVER_VERSION = "0.8.8"
|
||||
_logger_initialized = False
|
||||
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import os
|
||||
import time
|
||||
import base64
|
||||
from typing import Optional, Dict
|
||||
|
||||
@@ -19,7 +20,7 @@ class DeviceBindException(Exception):
|
||||
|
||||
class ManageApiClient:
|
||||
_instance = None
|
||||
_async_clients = {} # 为每个事件循环存储独立的客户端
|
||||
_client = None
|
||||
_secret = None
|
||||
|
||||
def __new__(cls, config):
|
||||
@@ -31,7 +32,7 @@ class ManageApiClient:
|
||||
|
||||
@classmethod
|
||||
def _init_client(cls, config):
|
||||
"""初始化配置（延迟创建客户端）"""
|
||||
"""初始化持久化连接池"""
|
||||
cls.config = config.get("manager-api")
|
||||
|
||||
if not cls.config:
|
||||
@@ -46,40 +47,23 @@ class ManageApiClient:
|
||||
cls._secret = cls.config.get("secret")
|
||||
cls.max_retries = cls.config.get("max_retries", 6) # 最大重试次数
|
||||
cls.retry_delay = cls.config.get("retry_delay", 10) # 初始重试延迟(秒)
|
||||
# 不在这里创建 AsyncClient，延迟到实际使用时创建
|
||||
cls._async_clients = {}
|
||||
# NOTE(goody): 2025/4/16 http相关资源统一管理，后续可以增加线程池或者超时
|
||||
# 后续也可以统一配置apiToken之类的走通用的Auth
|
||||
cls._client = httpx.Client(
|
||||
base_url=cls.config.get("url"),
|
||||
headers={
|
||||
"User-Agent": f"PythonClient/2.0 (PID:{os.getpid()})",
|
||||
"Accept": "application/json",
|
||||
"Authorization": "Bearer " + cls._secret,
|
||||
},
|
||||
timeout=cls.config.get("timeout", 30), # 默认超时时间30秒
|
||||
)
|
||||
|
||||
@classmethod
|
||||
async def _ensure_async_client(cls):
|
||||
"""确保异步客户端已创建（为每个事件循环创建独立的客户端）"""
|
||||
import asyncio
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop_id = id(loop)
|
||||
|
||||
# 为每个事件循环创建独立的客户端
|
||||
if loop_id not in cls._async_clients:
|
||||
cls._async_clients[loop_id] = httpx.AsyncClient(
|
||||
base_url=cls.config.get("url"),
|
||||
headers={
|
||||
"User-Agent": f"PythonClient/2.0 (PID:{os.getpid()})",
|
||||
"Accept": "application/json",
|
||||
"Authorization": "Bearer " + cls._secret,
|
||||
},
|
||||
timeout=cls.config.get("timeout", 30),
|
||||
)
|
||||
return cls._async_clients[loop_id]
|
||||
except RuntimeError:
|
||||
# 如果没有运行中的事件循环，创建一个临时的
|
||||
raise Exception("必须在异步上下文中调用")
|
||||
|
||||
@classmethod
|
||||
async def _async_request(cls, method: str, endpoint: str, **kwargs) -> Dict:
|
||||
"""发送单次异步HTTP请求并处理响应"""
|
||||
# 确保客户端已创建
|
||||
client = await cls._ensure_async_client()
|
||||
def _request(cls, method: str, endpoint: str, **kwargs) -> Dict:
|
||||
"""发送单次HTTP请求并处理响应"""
|
||||
endpoint = endpoint.lstrip("/")
|
||||
response = await client.request(method, endpoint, **kwargs)
|
||||
response = cls._client.request(method, endpoint, **kwargs)
|
||||
response.raise_for_status()
|
||||
|
||||
result = response.json()
|
||||
@@ -112,23 +96,22 @@ class ManageApiClient:
|
||||
return False
|
||||
|
||||
@classmethod
|
||||
async def _execute_async_request(cls, method: str, endpoint: str, **kwargs) -> Dict:
|
||||
"""带重试机制的异步请求执行器"""
|
||||
import asyncio
|
||||
def _execute_request(cls, method: str, endpoint: str, **kwargs) -> Dict:
|
||||
"""带重试机制的请求执行器"""
|
||||
retry_count = 0
|
||||
|
||||
while retry_count <= cls.max_retries:
|
||||
try:
|
||||
# 执行异步请求
|
||||
return await cls._async_request(method, endpoint, **kwargs)
|
||||
# 执行请求
|
||||
return cls._request(method, endpoint, **kwargs)
|
||||
except Exception as e:
|
||||
# 判断是否应该重试
|
||||
if retry_count < cls.max_retries and cls._should_retry(e):
|
||||
retry_count += 1
|
||||
print(
|
||||
f"{method} {endpoint} 异步请求失败，将在 {cls.retry_delay:.1f} 秒后进行第 {retry_count} 次重试"
|
||||
f"{method} {endpoint} 请求失败，将在 {cls.retry_delay:.1f} 秒后进行第 {retry_count} 次重试"
|
||||
)
|
||||
await asyncio.sleep(cls.retry_delay)
|
||||
time.sleep(cls.retry_delay)
|
||||
continue
|
||||
else:
|
||||
# 不重试，直接抛出异常
|
||||
@@ -136,27 +119,22 @@ class ManageApiClient:
|
||||
|
||||
@classmethod
|
||||
def safe_close(cls):
|
||||
"""安全关闭所有异步连接池"""
|
||||
import asyncio
|
||||
for client in list(cls._async_clients.values()):
|
||||
try:
|
||||
asyncio.run(client.aclose())
|
||||
except Exception:
|
||||
pass
|
||||
cls._async_clients.clear()
|
||||
cls._instance = None
|
||||
"""安全关闭连接池"""
|
||||
if cls._client:
|
||||
cls._client.close()
|
||||
cls._instance = None
|
||||
|
||||
|
||||
async def get_server_config() -> Optional[Dict]:
|
||||
def get_server_config() -> Optional[Dict]:
|
||||
"""获取服务器基础配置"""
|
||||
return await ManageApiClient._instance._execute_async_request("POST", "/config/server-base")
|
||||
return ManageApiClient._instance._execute_request("POST", "/config/server-base")
|
||||
|
||||
|
||||
async def get_agent_models(
|
||||
def get_agent_models(
|
||||
mac_address: str, client_id: str, selected_module: Dict
|
||||
) -> Optional[Dict]:
|
||||
"""获取代理模型配置"""
|
||||
return await ManageApiClient._instance._execute_async_request(
|
||||
return ManageApiClient._instance._execute_request(
|
||||
"POST",
|
||||
"/config/agent-models",
|
||||
json={
|
||||
@@ -167,9 +145,9 @@ async def get_agent_models(
|
||||
)
|
||||
|
||||
|
||||
async def save_mem_local_short(mac_address: str, short_momery: str) -> Optional[Dict]:
|
||||
def save_mem_local_short(mac_address: str, short_momery: str) -> Optional[Dict]:
|
||||
try:
|
||||
return await ManageApiClient._instance._execute_async_request(
|
||||
return ManageApiClient._instance._execute_request(
|
||||
"PUT",
|
||||
f"/agent/saveMemory/" + mac_address,
|
||||
json={
|
||||
@@ -181,14 +159,14 @@ async def save_mem_local_short(mac_address: str, short_momery: str) -> Optional[
|
||||
return None
|
||||
|
||||
|
||||
async def report(
|
||||
def report(
|
||||
mac_address: str, session_id: str, chat_type: int, content: str, audio, report_time
|
||||
) -> Optional[Dict]:
|
||||
"""异步聊天记录上报"""
|
||||
"""带熔断的业务方法示例"""
|
||||
if not content or not ManageApiClient._instance:
|
||||
return None
|
||||
try:
|
||||
return await ManageApiClient._instance._execute_async_request(
|
||||
return ManageApiClient._instance._execute_request(
|
||||
"POST",
|
||||
f"/agent/chat-history/report",
|
||||
json={
|
||||
|
||||
@@ -96,7 +96,7 @@ class VisionHandler:
|
||||
current_config = copy.deepcopy(self.config)
|
||||
read_config_from_api = current_config.get("read_config_from_api", False)
|
||||
if read_config_from_api:
|
||||
current_config = await get_private_config_from_api(
|
||||
current_config = get_private_config_from_api(
|
||||
current_config,
|
||||
device_id,
|
||||
client_id,
|
||||
|
||||
@@ -68,12 +68,8 @@ class ConnectionHandler:
|
||||
self.logger = setup_logging()
|
||||
self.server = server # 保存server实例的引用
|
||||
|
||||
self.need_bind = False # 是否需要绑定设备
|
||||
self.bind_completed_event = asyncio.Event()
|
||||
self.bind_code = None # 绑定设备的验证码
|
||||
self.last_bind_prompt_time = 0 # 上次播放绑定提示的时间戳(秒)
|
||||
self.bind_prompt_interval = 60 # 绑定提示播放间隔(秒)
|
||||
|
||||
self.need_bind = False
|
||||
self.bind_code = None
|
||||
self.read_config_from_api = self.config.get("read_config_from_api", False)
|
||||
|
||||
self.websocket = None
|
||||
@@ -92,7 +88,7 @@ class ConnectionHandler:
|
||||
self.client_listen_mode = "auto"
|
||||
|
||||
# 线程任务相关
|
||||
self.loop = None # 在 handle_connection 中获取运行中的事件循环
|
||||
self.loop = asyncio.get_event_loop()
|
||||
self.stop_event = threading.Event()
|
||||
self.executor = ThreadPoolExecutor(max_workers=5)
|
||||
|
||||
@@ -120,7 +116,6 @@ class ConnectionHandler:
|
||||
self.client_audio_buffer = bytearray()
|
||||
self.client_have_voice = False
|
||||
self.client_voice_window = deque(maxlen=5)
|
||||
self.first_activity_time = 0.0 # 记录首次活动的时间（毫秒）
|
||||
self.last_activity_time = 0.0 # 统一的活动时间戳（毫秒）
|
||||
self.client_voice_stop = False
|
||||
self.last_is_voice = False
|
||||
@@ -163,13 +158,10 @@ class ConnectionHandler:
|
||||
self.conn_from_mqtt_gateway = False
|
||||
|
||||
# 初始化提示词管理器
|
||||
self.prompt_manager = PromptManager(self.config, self.logger)
|
||||
self.prompt_manager = PromptManager(config, self.logger)
|
||||
|
||||
async def handle_connection(self, ws):
|
||||
try:
|
||||
# 获取运行中的事件循环（必须在异步上下文中）
|
||||
self.loop = asyncio.get_running_loop()
|
||||
|
||||
# 获取并验证headers
|
||||
self.headers = dict(ws.request.headers)
|
||||
real_ip = self.headers.get("x-real-ip") or self.headers.get(
|
||||
@@ -195,7 +187,6 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).info("连接来自:MQTT网关")
|
||||
|
||||
# 初始化活动时间戳
|
||||
self.first_activity_time = time.time() * 1000
|
||||
self.last_activity_time = time.time() * 1000
|
||||
|
||||
# 启动超时检查任务
|
||||
@@ -204,8 +195,10 @@ class ConnectionHandler:
|
||||
self.welcome_msg = self.config["xiaozhi"]
|
||||
self.welcome_msg["session_id"] = self.session_id
|
||||
|
||||
# 在后台初始化配置和组件（完全不阻塞主循环）
|
||||
asyncio.create_task(self._background_initialize())
|
||||
# 获取差异化配置
|
||||
self._initialize_private_config()
|
||||
# 异步初始化
|
||||
self.executor.submit(self._initialize_components)
|
||||
|
||||
try:
|
||||
async for message in self.websocket:
|
||||
@@ -267,37 +260,8 @@ class ConnectionHandler:
|
||||
f"保存记忆后关闭连接失败: {close_error}"
|
||||
)
|
||||
|
||||
async def _discard_message_with_bind_prompt(self):
|
||||
"""丢弃消息并检查是否需要播放绑定提示"""
|
||||
current_time = time.time()
|
||||
# 检查是否需要播放绑定提示
|
||||
if current_time - self.last_bind_prompt_time >= self.bind_prompt_interval:
|
||||
self.last_bind_prompt_time = current_time
|
||||
# 复用现有的绑定提示逻辑
|
||||
from core.handle.receiveAudioHandle import check_bind_device
|
||||
|
||||
asyncio.create_task(check_bind_device(self))
|
||||
|
||||
async def _route_message(self, message):
|
||||
"""消息路由"""
|
||||
# 检查是否已经获取到真实的绑定状态
|
||||
if not self.bind_completed_event.is_set():
|
||||
# 还没有获取到真实状态，等待直到获取到真实状态或超时
|
||||
try:
|
||||
await asyncio.wait_for(self.bind_completed_event.wait(), timeout=1)
|
||||
except asyncio.TimeoutError:
|
||||
# 超时仍未获取到真实状态，丢弃消息
|
||||
await self._discard_message_with_bind_prompt()
|
||||
return
|
||||
|
||||
# 已经获取到真实状态，检查是否需要绑定
|
||||
if self.need_bind:
|
||||
# 需要绑定，丢弃消息
|
||||
await self._discard_message_with_bind_prompt()
|
||||
return
|
||||
|
||||
# 不需要绑定，继续处理消息
|
||||
|
||||
if isinstance(message, str):
|
||||
await handleTextMessage(self, message)
|
||||
elif isinstance(message, bytes):
|
||||
@@ -427,14 +391,6 @@ class ConnectionHandler:
|
||||
|
||||
def _initialize_components(self):
|
||||
try:
|
||||
if self.tts is None:
|
||||
self.tts = self._initialize_tts()
|
||||
# 打开语音合成通道
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self.tts.open_audio_channels(self), self.loop
|
||||
)
|
||||
if self.need_bind:
|
||||
return
|
||||
self.selected_module_str = build_module_string(
|
||||
self.config.get("selected_module", {})
|
||||
)
|
||||
@@ -458,10 +414,17 @@ class ConnectionHandler:
|
||||
|
||||
# 初始化声纹识别
|
||||
self._initialize_voiceprint()
|
||||
|
||||
# 打开语音识别通道
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self.asr.open_audio_channels(self), self.loop
|
||||
)
|
||||
if self.tts is None:
|
||||
self.tts = self._initialize_tts()
|
||||
# 打开语音合成通道
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self.tts.open_audio_channels(self), self.loop
|
||||
)
|
||||
|
||||
"""加载记忆"""
|
||||
self._initialize_memory()
|
||||
@@ -476,7 +439,6 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).error(f"实例化组件失败: {e}")
|
||||
|
||||
def _init_prompt_enhancement(self):
|
||||
|
||||
# 更新上下文信息
|
||||
self.prompt_manager.update_context_info(self, self.client_ip)
|
||||
enhanced_prompt = self.prompt_manager.build_enhanced_prompt(
|
||||
@@ -512,11 +474,7 @@ class ConnectionHandler:
|
||||
|
||||
def _initialize_asr(self):
|
||||
"""初始化ASR"""
|
||||
if (
|
||||
self._asr is not None
|
||||
and hasattr(self._asr, "interface_type")
|
||||
and self._asr.interface_type == InterfaceType.LOCAL
|
||||
):
|
||||
if self._asr.interface_type == InterfaceType.LOCAL:
|
||||
# 如果公共ASR是本地服务，则直接返回
|
||||
# 因为本地一个实例ASR，可以被多个连接共享
|
||||
asr = self._asr
|
||||
@@ -543,48 +501,32 @@ class ConnectionHandler:
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).warning(f"声纹识别初始化失败: {str(e)}")
|
||||
|
||||
async def _background_initialize(self):
|
||||
"""在后台初始化配置和组件（完全不阻塞主循环）"""
|
||||
try:
|
||||
# 异步获取差异化配置
|
||||
await self._initialize_private_config_async()
|
||||
# 在线程池中初始化组件
|
||||
self.executor.submit(self._initialize_components)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"后台初始化失败: {e}")
|
||||
|
||||
async def _initialize_private_config_async(self):
|
||||
"""从接口异步获取差异化配置（异步版本，不阻塞主循环）"""
|
||||
def _initialize_private_config(self):
|
||||
"""如果是从配置文件获取，则进行二次实例化"""
|
||||
if not self.read_config_from_api:
|
||||
self.need_bind = False
|
||||
self.bind_completed_event.set()
|
||||
return
|
||||
"""从接口获取差异化的配置进行二次实例化，非全量重新实例化"""
|
||||
try:
|
||||
begin_time = time.time()
|
||||
private_config = await get_private_config_from_api(
|
||||
private_config = get_private_config_from_api(
|
||||
self.config,
|
||||
self.headers.get("device-id"),
|
||||
self.headers.get("client-id", self.headers.get("device-id")),
|
||||
)
|
||||
private_config["delete_audio"] = bool(self.config.get("delete_audio", True))
|
||||
self.logger.bind(tag=TAG).info(
|
||||
f"{time.time() - begin_time} 秒，异步获取差异化配置成功: {json.dumps(filter_sensitive_info(private_config), ensure_ascii=False)}"
|
||||
f"{time.time() - begin_time} 秒，获取差异化配置成功: {json.dumps(filter_sensitive_info(private_config), ensure_ascii=False)}"
|
||||
)
|
||||
self.need_bind = False
|
||||
self.bind_completed_event.set()
|
||||
except DeviceNotFoundException as e:
|
||||
self.need_bind = True
|
||||
self.bind_completed_event.set() # 状态已确定，设置事件
|
||||
private_config = {}
|
||||
except DeviceBindException as e:
|
||||
self.need_bind = True
|
||||
self.bind_code = e.bind_code
|
||||
self.bind_completed_event.set() # 状态已确定，设置事件
|
||||
private_config = {}
|
||||
except Exception as e:
|
||||
self.need_bind = True
|
||||
self.bind_completed_event.set() # 状态已确定，设置事件
|
||||
self.logger.bind(tag=TAG).error(f"异步获取差异化配置失败: {e}")
|
||||
self.logger.bind(tag=TAG).error(f"获取差异化配置失败: {e}")
|
||||
private_config = {}
|
||||
|
||||
init_llm, init_tts, init_memory, init_intent = (
|
||||
@@ -657,14 +599,8 @@ class ConnectionHandler:
|
||||
self.chat_history_conf = int(private_config["chat_history_conf"])
|
||||
if private_config.get("mcp_endpoint", None) is not None:
|
||||
self.config["mcp_endpoint"] = private_config["mcp_endpoint"]
|
||||
if private_config.get("context_providers", None) is not None:
|
||||
self.config["context_providers"] = private_config["context_providers"]
|
||||
|
||||
# 使用 run_in_executor 在线程池中执行 initialize_modules，避免阻塞主循环
|
||||
try:
|
||||
modules = await self.loop.run_in_executor(
|
||||
None, # 使用默认线程池
|
||||
initialize_modules,
|
||||
modules = initialize_modules(
|
||||
self.logger,
|
||||
private_config,
|
||||
init_vad,
|
||||
@@ -787,47 +723,25 @@ class ConnectionHandler:
|
||||
self.dialogue.update_system_message(self.prompt)
|
||||
|
||||
def chat(self, query, depth=0):
|
||||
if query is not None:
|
||||
self.logger.bind(tag=TAG).info(f"大模型收到用户消息: {query}")
|
||||
self.logger.bind(tag=TAG).info(f"大模型收到用户消息: {query}")
|
||||
self.llm_finish_task = False
|
||||
|
||||
# 为最顶层时新建会话ID和发送FIRST请求
|
||||
# depth!=0目前只能表示是非用户发起的请求，与对话逻辑无关，这里所有tts对话都应该发送FIRST和LAST，没有例外
|
||||
if depth == 0:
|
||||
self.llm_finish_task = False
|
||||
self.sentence_id = str(uuid.uuid4().hex)
|
||||
self.dialogue.put(Message(role="user", content=query))
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
|
||||
# 设置最大递归深度，避免无限循环，可根据实际需求调整
|
||||
MAX_DEPTH = 5
|
||||
force_final_answer = False # 标记是否强制最终回答
|
||||
|
||||
if depth >= MAX_DEPTH:
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"已达到最大工具调用深度 {MAX_DEPTH}，将强制基于现有信息回答"
|
||||
)
|
||||
force_final_answer = True
|
||||
# 添加系统指令，要求 LLM 基于现有信息回答
|
||||
self.dialogue.put(
|
||||
Message(
|
||||
role="user",
|
||||
content="[系统提示] 已达到最大工具调用次数限制，请你基于目前已经获取的所有信息，直接给出最终答案。不要再尝试调用任何工具。",
|
||||
)
|
||||
# 发送开始标记
|
||||
self.sentence_id = str(uuid.uuid4().hex)
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.FIRST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
|
||||
# Define intent functions
|
||||
functions = None
|
||||
# 达到最大深度时，禁用工具调用，强制 LLM 直接回答
|
||||
if (
|
||||
self.intent_type == "function_call"
|
||||
and hasattr(self, "func_handler")
|
||||
and not force_final_answer
|
||||
):
|
||||
if self.intent_type == "function_call" and hasattr(self, "func_handler"):
|
||||
functions = self.func_handler.get_functions()
|
||||
response_message = []
|
||||
|
||||
@@ -862,8 +776,9 @@ class ConnectionHandler:
|
||||
|
||||
# 处理流式响应
|
||||
tool_call_flag = False
|
||||
# 支持多个并行工具调用 - 使用列表存储
|
||||
tool_calls_list = [] # 格式: [{"id": "", "name": "", "arguments": ""}]
|
||||
function_name = None
|
||||
function_id = None
|
||||
function_arguments = ""
|
||||
content_arguments = ""
|
||||
self.client_abort = False
|
||||
emotion_flag = True
|
||||
@@ -884,7 +799,12 @@ class ConnectionHandler:
|
||||
|
||||
if tools_call is not None and len(tools_call) > 0:
|
||||
tool_call_flag = True
|
||||
self._merge_tool_calls(tool_calls_list, tools_call)
|
||||
if tools_call[0].id is not None and tools_call[0].id != "":
|
||||
function_id = tools_call[0].id
|
||||
if tools_call[0].function.name is not None and tools_call[0].function.name != "":
|
||||
function_name = tools_call[0].function.name
|
||||
if tools_call[0].function.arguments is not None:
|
||||
function_arguments += tools_call[0].function.arguments
|
||||
else:
|
||||
content = response
|
||||
|
||||
@@ -907,25 +827,27 @@ class ConnectionHandler:
|
||||
content_detail=content,
|
||||
)
|
||||
)
|
||||
# 发送结束标记
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
# 处理function call
|
||||
if tool_call_flag:
|
||||
bHasError = False
|
||||
# 处理基于文本的工具调用格式
|
||||
if len(tool_calls_list) == 0 and content_arguments:
|
||||
if function_id is None:
|
||||
a = extract_json_from_string(content_arguments)
|
||||
if a is not None:
|
||||
try:
|
||||
content_arguments_json = json.loads(a)
|
||||
tool_calls_list.append(
|
||||
{
|
||||
"id": str(uuid.uuid4().hex),
|
||||
"name": content_arguments_json["name"],
|
||||
"arguments": json.dumps(
|
||||
content_arguments_json["arguments"],
|
||||
ensure_ascii=False,
|
||||
),
|
||||
}
|
||||
function_name = content_arguments_json["name"]
|
||||
function_arguments = json.dumps(
|
||||
content_arguments_json["arguments"], ensure_ascii=False
|
||||
)
|
||||
function_id = str(uuid.uuid4().hex)
|
||||
except Exception as e:
|
||||
bHasError = True
|
||||
response_message.append(a)
|
||||
@@ -936,120 +858,95 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).error(
|
||||
f"function call error: {content_arguments}"
|
||||
)
|
||||
|
||||
if not bHasError and len(tool_calls_list) > 0:
|
||||
if not bHasError:
|
||||
# 如需要大模型先处理一轮，添加相关处理后的日志情况
|
||||
if len(response_message) > 0:
|
||||
text_buff = "".join(response_message)
|
||||
self.tts_MessageText = text_buff
|
||||
self.dialogue.put(Message(role="assistant", content=text_buff))
|
||||
response_message.clear()
|
||||
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"检测到 {len(tool_calls_list)} 个工具调用"
|
||||
f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}"
|
||||
)
|
||||
function_call_data = {
|
||||
"name": function_name,
|
||||
"id": function_id,
|
||||
"arguments": function_arguments,
|
||||
}
|
||||
|
||||
# 收集所有工具调用的 Future
|
||||
futures_with_data = []
|
||||
for tool_call_data in tool_calls_list:
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"function_name={tool_call_data['name']}, function_id={tool_call_data['id']}, function_arguments={tool_call_data['arguments']}"
|
||||
)
|
||||
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
self.func_handler.handle_llm_function_call(
|
||||
self, tool_call_data
|
||||
),
|
||||
self.loop,
|
||||
)
|
||||
futures_with_data.append((future, tool_call_data))
|
||||
|
||||
# 等待协程结束（实际等待时长为最慢的那个）
|
||||
tool_results = []
|
||||
for future, tool_call_data in futures_with_data:
|
||||
result = future.result()
|
||||
tool_results.append((result, tool_call_data))
|
||||
|
||||
# 统一处理所有工具调用结果
|
||||
if tool_results:
|
||||
self._handle_function_result(tool_results, depth=depth)
|
||||
# 使用统一工具处理器处理所有工具调用
|
||||
result = asyncio.run_coroutine_threadsafe(
|
||||
self.func_handler.handle_llm_function_call(
|
||||
self, function_call_data
|
||||
),
|
||||
self.loop,
|
||||
).result()
|
||||
self._handle_function_result(result, function_call_data, depth=depth)
|
||||
|
||||
# 存储对话内容
|
||||
if len(response_message) > 0:
|
||||
text_buff = "".join(response_message)
|
||||
self.tts_MessageText = text_buff
|
||||
self.dialogue.put(Message(role="assistant", content=text_buff))
|
||||
if depth == 0:
|
||||
self.tts.tts_text_queue.put(
|
||||
TTSMessageDTO(
|
||||
sentence_id=self.sentence_id,
|
||||
sentence_type=SentenceType.LAST,
|
||||
content_type=ContentType.ACTION,
|
||||
)
|
||||
)
|
||||
self.llm_finish_task = True
|
||||
# 使用lambda延迟计算，只有在DEBUG级别时才执行get_llm_dialogue()
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
lambda: json.dumps(
|
||||
self.dialogue.get_llm_dialogue(), indent=4, ensure_ascii=False
|
||||
)
|
||||
|
||||
self.llm_finish_task = True
|
||||
# 使用lambda延迟计算，只有在DEBUG级别时才执行get_llm_dialogue()
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
lambda: json.dumps(
|
||||
self.dialogue.get_llm_dialogue(), indent=4, ensure_ascii=False
|
||||
)
|
||||
)
|
||||
|
||||
return True
|
||||
|
||||
def _handle_function_result(self, tool_results, depth):
|
||||
need_llm_tools = []
|
||||
|
||||
for result, tool_call_data in tool_results:
|
||||
if result.action in [
|
||||
Action.RESPONSE,
|
||||
Action.NOTFOUND,
|
||||
Action.ERROR,
|
||||
]: # 直接回复前端
|
||||
text = result.response if result.response else result.result
|
||||
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
|
||||
self.dialogue.put(Message(role="assistant", content=text))
|
||||
elif result.action == Action.REQLLM:
|
||||
# 收集需要 LLM 处理的工具
|
||||
need_llm_tools.append((result, tool_call_data))
|
||||
else:
|
||||
pass
|
||||
|
||||
if need_llm_tools:
|
||||
all_tool_calls = [
|
||||
{
|
||||
"id": tool_call_data["id"],
|
||||
"function": {
|
||||
"arguments": (
|
||||
"{}"
|
||||
if tool_call_data["arguments"] == ""
|
||||
else tool_call_data["arguments"]
|
||||
),
|
||||
"name": tool_call_data["name"],
|
||||
},
|
||||
"type": "function",
|
||||
"index": idx,
|
||||
}
|
||||
for idx, (_, tool_call_data) in enumerate(need_llm_tools)
|
||||
]
|
||||
self.dialogue.put(Message(role="assistant", tool_calls=all_tool_calls))
|
||||
|
||||
for result, tool_call_data in need_llm_tools:
|
||||
text = result.result
|
||||
if text is not None and len(text) > 0:
|
||||
self.dialogue.put(
|
||||
Message(
|
||||
role="tool",
|
||||
tool_call_id=(
|
||||
str(uuid.uuid4())
|
||||
if tool_call_data["id"] is None
|
||||
else tool_call_data["id"]
|
||||
),
|
||||
content=text,
|
||||
)
|
||||
def _handle_function_result(self, result, function_call_data, depth):
|
||||
if result.action == Action.RESPONSE: # 直接回复前端
|
||||
text = result.response
|
||||
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
|
||||
self.dialogue.put(Message(role="assistant", content=text))
|
||||
elif result.action == Action.REQLLM: # 调用函数后再请求llm生成回复
|
||||
text = result.result
|
||||
if text is not None and len(text) > 0:
|
||||
function_id = function_call_data["id"]
|
||||
function_name = function_call_data["name"]
|
||||
function_arguments = function_call_data["arguments"]
|
||||
self.dialogue.put(
|
||||
Message(
|
||||
role="assistant",
|
||||
tool_calls=[
|
||||
{
|
||||
"id": function_id,
|
||||
"function": {
|
||||
"arguments": (
|
||||
"{}"
|
||||
if function_arguments == ""
|
||||
else function_arguments
|
||||
),
|
||||
"name": function_name,
|
||||
},
|
||||
"type": "function",
|
||||
"index": 0,
|
||||
}
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
self.chat(None, depth=depth + 1)
|
||||
self.dialogue.put(
|
||||
Message(
|
||||
role="tool",
|
||||
tool_call_id=(
|
||||
str(uuid.uuid4()) if function_id is None else function_id
|
||||
),
|
||||
content=text,
|
||||
)
|
||||
)
|
||||
self.chat(text, depth=depth + 1)
|
||||
elif result.action == Action.NOTFOUND or result.action == Action.ERROR:
|
||||
text = result.response if result.response else result.result
|
||||
self.tts.tts_one_sentence(self, ContentType.TEXT, content_detail=text)
|
||||
self.dialogue.put(Message(role="assistant", content=text))
|
||||
else:
|
||||
pass
|
||||
|
||||
def _report_worker(self):
|
||||
"""聊天记录上报工作线程"""
|
||||
@@ -1077,8 +974,8 @@ class ConnectionHandler:
|
||||
def _process_report(self, type, text, audio_data, report_time):
|
||||
"""处理上报任务"""
|
||||
try:
|
||||
# 执行异步上报（在事件循环中运行）
|
||||
asyncio.run(report(self, type, text, audio_data, report_time))
|
||||
# 执行上报（传入二进制数据）
|
||||
report(self, type, text, audio_data, report_time)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"上报处理异常: {e}")
|
||||
finally:
|
||||
@@ -1169,6 +1066,7 @@ class ConnectionHandler:
|
||||
f"关闭线程池时出错: {executor_error}"
|
||||
)
|
||||
self.executor = None
|
||||
|
||||
self.logger.bind(tag=TAG).info("连接资源已释放")
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"关闭连接时出错: {e}")
|
||||
@@ -1198,11 +1096,6 @@ class ConnectionHandler:
|
||||
except queue.Empty:
|
||||
break
|
||||
|
||||
# 重置音频流控器（取消后台任务并清空队列）
|
||||
if hasattr(self, "audio_rate_controller") and self.audio_rate_controller:
|
||||
self.audio_rate_controller.reset()
|
||||
self.logger.bind(tag=TAG).debug("已重置音频流控器")
|
||||
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"清理结束: TTS队列大小={self.tts.tts_text_queue.qsize()}, 音频队列大小={self.tts.tts_audio_queue.qsize()}"
|
||||
)
|
||||
@@ -1228,14 +1121,13 @@ class ConnectionHandler:
|
||||
"""检查连接超时"""
|
||||
try:
|
||||
while not self.stop_event.is_set():
|
||||
last_activity_time = self.last_activity_time
|
||||
if self.need_bind:
|
||||
last_activity_time = self.first_activity_time
|
||||
|
||||
# 检查是否超时（只有在时间戳已初始化的情况下）
|
||||
if last_activity_time > 0.0:
|
||||
if self.last_activity_time > 0.0:
|
||||
current_time = time.time() * 1000
|
||||
if current_time - last_activity_time > self.timeout_seconds * 1000:
|
||||
if (
|
||||
current_time - self.last_activity_time
|
||||
> self.timeout_seconds * 1000
|
||||
):
|
||||
if not self.stop_event.is_set():
|
||||
self.logger.bind(tag=TAG).info("连接超时，准备关闭")
|
||||
# 设置停止事件，防止重复处理
|
||||
@@ -1254,31 +1146,3 @@ class ConnectionHandler:
|
||||
self.logger.bind(tag=TAG).error(f"超时检查任务出错: {e}")
|
||||
finally:
|
||||
self.logger.bind(tag=TAG).info("超时检查任务已退出")
|
||||
|
||||
def _merge_tool_calls(self, tool_calls_list, tools_call):
|
||||
"""合并工具调用列表
|
||||
|
||||
Args:
|
||||
tool_calls_list: 已收集的工具调用列表
|
||||
tools_call: 新的工具调用
|
||||
"""
|
||||
for tool_call in tools_call:
|
||||
tool_index = getattr(tool_call, "index", None)
|
||||
if tool_index is None:
|
||||
if tool_call.function.name:
|
||||
# 有 function_name，说明是新的工具调用
|
||||
tool_index = len(tool_calls_list)
|
||||
else:
|
||||
tool_index = len(tool_calls_list) - 1 if tool_calls_list else 0
|
||||
|
||||
# 确保列表有足够的位置
|
||||
if tool_index >= len(tool_calls_list):
|
||||
tool_calls_list.append({"id": "", "name": "", "arguments": ""})
|
||||
|
||||
# 更新工具调用信息
|
||||
if tool_call.id:
|
||||
tool_calls_list[tool_index]["id"] = tool_call.id
|
||||
if tool_call.function.name:
|
||||
tool_calls_list[tool_index]["name"] = tool_call.function.name
|
||||
if tool_call.function.arguments:
|
||||
tool_calls_list[tool_index]["arguments"] += tool_call.function.arguments
|
||||
|
||||
@@ -101,7 +101,7 @@ async def checkWakeupWords(conn, text):
|
||||
}
|
||||
|
||||
# 获取音频数据
|
||||
opus_packets = await audio_to_data(response.get("file_path"), use_cache=False)
|
||||
opus_packets = audio_to_data(response.get("file_path"))
|
||||
# 播放唤醒词回复
|
||||
conn.client_abort = False
|
||||
|
||||
|
||||
@@ -123,7 +123,7 @@ async def max_out_size(conn):
|
||||
text = "不好意思，我现在有点事情要忙，明天这个时候我们再聊，约好了哦！明天不见不散，拜拜！"
|
||||
await send_stt_message(conn, text)
|
||||
file_path = "config/assets/max_output_size.wav"
|
||||
opus_packets = await audio_to_data(file_path)
|
||||
opus_packets = audio_to_data(file_path)
|
||||
conn.tts.tts_audio_queue.put((SentenceType.LAST, opus_packets, text))
|
||||
conn.close_after_chat = True
|
||||
|
||||
@@ -142,7 +142,7 @@ async def check_bind_device(conn):
|
||||
|
||||
# 播放提示音
|
||||
music_path = "config/assets/bind_code.wav"
|
||||
opus_packets = await audio_to_data(music_path)
|
||||
opus_packets = audio_to_data(music_path)
|
||||
conn.tts.tts_audio_queue.put((SentenceType.FIRST, opus_packets, text))
|
||||
|
||||
# 逐个播放数字
|
||||
@@ -150,7 +150,7 @@ async def check_bind_device(conn):
|
||||
try:
|
||||
digit = conn.bind_code[i]
|
||||
num_path = f"config/assets/bind_code/{digit}.wav"
|
||||
num_packets = await audio_to_data(num_path)
|
||||
num_packets = audio_to_data(num_path)
|
||||
conn.tts.tts_audio_queue.put((SentenceType.MIDDLE, num_packets, None))
|
||||
except Exception as e:
|
||||
conn.logger.bind(tag=TAG).error(f"播放数字音频失败: {e}")
|
||||
@@ -162,5 +162,5 @@ async def check_bind_device(conn):
|
||||
text = f"没有找到该设备的版本信息，请正确配置 OTA地址，然后重新编译固件。"
|
||||
await send_stt_message(conn, text)
|
||||
music_path = "config/assets/bind_not_found.wav"
|
||||
opus_packets = await audio_to_data(music_path)
|
||||
opus_packets = audio_to_data(music_path)
|
||||
conn.tts.tts_audio_queue.put((SentenceType.LAST, opus_packets, text))
|
||||
|
||||
@@ -10,6 +10,7 @@ TTS上报功能已集成到ConnectionHandler类中。
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
import opuslib_next
|
||||
|
||||
from config.manage_api_client import report as manage_report
|
||||
@@ -17,7 +18,7 @@ from config.manage_api_client import report as manage_report
|
||||
TAG = __name__
|
||||
|
||||
|
||||
async def report(conn, type, text, opus_data, report_time):
|
||||
def report(conn, type, text, opus_data, report_time):
|
||||
"""执行聊天记录上报操作
|
||||
|
||||
Args:
|
||||
@@ -32,8 +33,8 @@ async def report(conn, type, text, opus_data, report_time):
|
||||
audio_data = opus_to_wav(conn, opus_data)
|
||||
else:
|
||||
audio_data = None
|
||||
# 执行异步上报
|
||||
await manage_report(
|
||||
# 执行上报
|
||||
manage_report(
|
||||
mac_address=conn.device_id,
|
||||
session_id=conn.session_id,
|
||||
chat_type=type,
|
||||
@@ -55,49 +56,41 @@ def opus_to_wav(conn, opus_data):
|
||||
Returns:
|
||||
bytes: WAV格式的音频数据
|
||||
"""
|
||||
decoder = None
|
||||
try:
|
||||
decoder = opuslib_next.Decoder(16000, 1) # 16kHz, 单声道
|
||||
pcm_data = []
|
||||
decoder = opuslib_next.Decoder(16000, 1) # 16kHz, 单声道
|
||||
pcm_data = []
|
||||
|
||||
for opus_packet in opus_data:
|
||||
try:
|
||||
pcm_frame = decoder.decode(opus_packet, 960) # 960 samples = 60ms
|
||||
pcm_data.append(pcm_frame)
|
||||
except opuslib_next.OpusError as e:
|
||||
conn.logger.bind(tag=TAG).error(f"Opus解码错误: {e}", exc_info=True)
|
||||
for opus_packet in opus_data:
|
||||
try:
|
||||
pcm_frame = decoder.decode(opus_packet, 960) # 960 samples = 60ms
|
||||
pcm_data.append(pcm_frame)
|
||||
except opuslib_next.OpusError as e:
|
||||
conn.logger.bind(tag=TAG).error(f"Opus解码错误: {e}", exc_info=True)
|
||||
|
||||
if not pcm_data:
|
||||
raise ValueError("没有有效的PCM数据")
|
||||
if not pcm_data:
|
||||
raise ValueError("没有有效的PCM数据")
|
||||
|
||||
# 创建WAV文件头
|
||||
pcm_data_bytes = b"".join(pcm_data)
|
||||
num_samples = len(pcm_data_bytes) // 2 # 16-bit samples
|
||||
# 创建WAV文件头
|
||||
pcm_data_bytes = b"".join(pcm_data)
|
||||
num_samples = len(pcm_data_bytes) // 2 # 16-bit samples
|
||||
|
||||
# WAV文件头
|
||||
wav_header = bytearray()
|
||||
wav_header.extend(b"RIFF") # ChunkID
|
||||
wav_header.extend((36 + len(pcm_data_bytes)).to_bytes(4, "little")) # ChunkSize
|
||||
wav_header.extend(b"WAVE") # Format
|
||||
wav_header.extend(b"fmt ") # Subchunk1ID
|
||||
wav_header.extend((16).to_bytes(4, "little")) # Subchunk1Size
|
||||
wav_header.extend((1).to_bytes(2, "little")) # AudioFormat (PCM)
|
||||
wav_header.extend((1).to_bytes(2, "little")) # NumChannels
|
||||
wav_header.extend((16000).to_bytes(4, "little")) # SampleRate
|
||||
wav_header.extend((32000).to_bytes(4, "little")) # ByteRate
|
||||
wav_header.extend((2).to_bytes(2, "little")) # BlockAlign
|
||||
wav_header.extend((16).to_bytes(2, "little")) # BitsPerSample
|
||||
wav_header.extend(b"data") # Subchunk2ID
|
||||
wav_header.extend(len(pcm_data_bytes).to_bytes(4, "little")) # Subchunk2Size
|
||||
# WAV文件头
|
||||
wav_header = bytearray()
|
||||
wav_header.extend(b"RIFF") # ChunkID
|
||||
wav_header.extend((36 + len(pcm_data_bytes)).to_bytes(4, "little")) # ChunkSize
|
||||
wav_header.extend(b"WAVE") # Format
|
||||
wav_header.extend(b"fmt ") # Subchunk1ID
|
||||
wav_header.extend((16).to_bytes(4, "little")) # Subchunk1Size
|
||||
wav_header.extend((1).to_bytes(2, "little")) # AudioFormat (PCM)
|
||||
wav_header.extend((1).to_bytes(2, "little")) # NumChannels
|
||||
wav_header.extend((16000).to_bytes(4, "little")) # SampleRate
|
||||
wav_header.extend((32000).to_bytes(4, "little")) # ByteRate
|
||||
wav_header.extend((2).to_bytes(2, "little")) # BlockAlign
|
||||
wav_header.extend((16).to_bytes(2, "little")) # BitsPerSample
|
||||
wav_header.extend(b"data") # Subchunk2ID
|
||||
wav_header.extend(len(pcm_data_bytes).to_bytes(4, "little")) # Subchunk2Size
|
||||
|
||||
# 返回完整的WAV数据
|
||||
return bytes(wav_header) + pcm_data_bytes
|
||||
finally:
|
||||
if decoder is not None:
|
||||
try:
|
||||
del decoder
|
||||
except Exception as e:
|
||||
conn.logger.bind(tag=TAG).debug(f"释放decoder资源时出错: {e}")
|
||||
# 返回完整的WAV数据
|
||||
return bytes(wav_header) + pcm_data_bytes
|
||||
|
||||
|
||||
def enqueue_tts_report(conn, text, opus_data):
|
||||
|
||||
@@ -4,7 +4,6 @@ import asyncio
|
||||
from core.utils import textUtils
|
||||
from core.utils.util import audio_to_data
|
||||
from core.providers.tts.dto.dto import SentenceType
|
||||
from core.utils.audioRateController import AudioRateController
|
||||
|
||||
TAG = __name__
|
||||
|
||||
@@ -16,19 +15,7 @@ async def sendAudioMessage(conn, sentenceType, audios, text):
|
||||
await send_tts_message(conn, "start", None)
|
||||
|
||||
if sentenceType == SentenceType.FIRST:
|
||||
# 同一句子的后续消息加入流控队列，其他情况立即发送
|
||||
if (
|
||||
hasattr(conn, "audio_rate_controller")
|
||||
and conn.audio_rate_controller
|
||||
and getattr(conn, "audio_flow_control", {}).get("sentence_id")
|
||||
== conn.sentence_id
|
||||
):
|
||||
conn.audio_rate_controller.add_message(
|
||||
lambda: send_tts_message(conn, "sentence_start", text)
|
||||
)
|
||||
else:
|
||||
# 新句子或流控器未初始化，立即发送
|
||||
await send_tts_message(conn, "sentence_start", text)
|
||||
await send_tts_message(conn, "sentence_start", text)
|
||||
|
||||
await sendAudio(conn, audios)
|
||||
# 发送句子开始消息
|
||||
@@ -36,27 +23,36 @@ async def sendAudioMessage(conn, sentenceType, audios, text):
|
||||
conn.logger.bind(tag=TAG).info(f"发送音频消息: {sentenceType}, {text}")
|
||||
|
||||
# 发送结束消息（如果是最后一个文本）
|
||||
if sentenceType == SentenceType.LAST:
|
||||
if conn.llm_finish_task and sentenceType == SentenceType.LAST:
|
||||
await send_tts_message(conn, "stop", None)
|
||||
conn.client_is_speaking = False
|
||||
if conn.close_after_chat:
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def _wait_for_audio_completion(conn):
|
||||
def calculate_timestamp_and_sequence(conn, start_time, packet_index, frame_duration=60):
|
||||
"""
|
||||
等待音频队列清空
|
||||
|
||||
计算音频数据包的时间戳和序列号
|
||||
Args:
|
||||
conn: 连接对象
|
||||
start_time: 起始时间（性能计数器值）
|
||||
packet_index: 数据包索引
|
||||
frame_duration: 帧时长（毫秒），匹配 Opus 编码
|
||||
Returns:
|
||||
tuple: (timestamp, sequence)
|
||||
"""
|
||||
if hasattr(conn, "audio_rate_controller") and conn.audio_rate_controller:
|
||||
rate_controller = conn.audio_rate_controller
|
||||
conn.logger.bind(tag=TAG).debug(
|
||||
f"等待音频发送完成，队列中还有 {len(rate_controller.queue)} 个包"
|
||||
)
|
||||
await rate_controller.queue_empty_event.wait()
|
||||
conn.logger.bind(tag=TAG).debug("音频发送完成")
|
||||
# 计算时间戳（使用播放位置计算）
|
||||
timestamp = int((start_time + packet_index * frame_duration / 1000) * 1000) % (
|
||||
2**32
|
||||
)
|
||||
|
||||
# 计算序列号
|
||||
if hasattr(conn, "audio_flow_control"):
|
||||
sequence = conn.audio_flow_control["sequence"]
|
||||
else:
|
||||
sequence = packet_index # 如果没有流控状态，直接使用索引
|
||||
|
||||
return timestamp, sequence
|
||||
|
||||
|
||||
async def _send_to_mqtt_gateway(conn, opus_packet, timestamp, sequence):
|
||||
@@ -81,153 +77,131 @@ async def _send_to_mqtt_gateway(conn, opus_packet, timestamp, sequence):
|
||||
await conn.websocket.send(complete_packet)
|
||||
|
||||
|
||||
# 播放音频
|
||||
async def sendAudio(conn, audios, frame_duration=60):
|
||||
"""
|
||||
发送音频包，使用 AudioRateController 进行精确的流量控制
|
||||
|
||||
发送单个opus包，支持流控
|
||||
Args:
|
||||
conn: 连接对象
|
||||
audios: 单个opus包(bytes) 或 opus包列表
|
||||
frame_duration: 帧时长（毫秒），默认60ms
|
||||
opus_packet: 单个opus数据包
|
||||
pre_buffer: 快速发送音频
|
||||
frame_duration: 帧时长（毫秒），匹配 Opus 编码
|
||||
"""
|
||||
if audios is None or len(audios) == 0:
|
||||
return
|
||||
|
||||
# 获取发送延迟配置
|
||||
send_delay = conn.config.get("tts_audio_send_delay", -1) / 1000.0
|
||||
is_single_packet = isinstance(audios, bytes)
|
||||
|
||||
# 初始化或获取 RateController
|
||||
rate_controller, flow_control = _get_or_create_rate_controller(
|
||||
conn, frame_duration, is_single_packet
|
||||
)
|
||||
if isinstance(audios, bytes):
|
||||
# 重置流控状态,第一次读取和会话发生转变时
|
||||
if not hasattr(conn, "audio_flow_control") or conn.audio_flow_control.get("sentence_id") != conn.sentence_id:
|
||||
conn.audio_flow_control = {
|
||||
"last_send_time": 0,
|
||||
"packet_count": 0,
|
||||
"start_time": time.perf_counter(),
|
||||
"sequence": 0, # 添加序列号
|
||||
"sentence_id": conn.sentence_id,
|
||||
}
|
||||
|
||||
# 统一转换为列表处理
|
||||
audio_list = [audios] if is_single_packet else audios
|
||||
|
||||
# 发送音频包
|
||||
await _send_audio_with_rate_control(
|
||||
conn, audio_list, rate_controller, flow_control, send_delay
|
||||
)
|
||||
|
||||
|
||||
def _get_or_create_rate_controller(conn, frame_duration, is_single_packet):
|
||||
"""
|
||||
获取或创建 RateController 和 flow_control
|
||||
|
||||
Args:
|
||||
conn: 连接对象
|
||||
frame_duration: 帧时长
|
||||
is_single_packet: 是否单包模式（True: TTS流式单包, False: 批量包）
|
||||
|
||||
Returns:
|
||||
(rate_controller, flow_control)
|
||||
"""
|
||||
# 判断是否需要重置：单包模式且 sentence_id 变化，或者控制器不存在
|
||||
need_reset = (
|
||||
is_single_packet
|
||||
and getattr(conn, "audio_flow_control", {}).get("sentence_id")
|
||||
!= conn.sentence_id
|
||||
) or not hasattr(conn, "audio_rate_controller")
|
||||
|
||||
if need_reset:
|
||||
# 创建或获取 rate_controller
|
||||
if not hasattr(conn, "audio_rate_controller"):
|
||||
conn.audio_rate_controller = AudioRateController(frame_duration)
|
||||
else:
|
||||
conn.audio_rate_controller.reset()
|
||||
|
||||
# 初始化 flow_control
|
||||
conn.audio_flow_control = {
|
||||
"packet_count": 0,
|
||||
"sequence": 0,
|
||||
"sentence_id": conn.sentence_id,
|
||||
}
|
||||
|
||||
# 启动后台发送循环
|
||||
_start_background_sender(
|
||||
conn, conn.audio_rate_controller, conn.audio_flow_control
|
||||
)
|
||||
|
||||
return conn.audio_rate_controller, conn.audio_flow_control
|
||||
|
||||
|
||||
def _start_background_sender(conn, rate_controller, flow_control):
|
||||
"""
|
||||
启动后台发送循环任务
|
||||
|
||||
Args:
|
||||
conn: 连接对象
|
||||
rate_controller: 速率控制器
|
||||
flow_control: 流控状态
|
||||
"""
|
||||
|
||||
async def send_callback(packet):
|
||||
# 检查是否应该中止
|
||||
if conn.client_abort:
|
||||
raise asyncio.CancelledError("客户端已中止")
|
||||
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
await _do_send_audio(conn, packet, flow_control)
|
||||
conn.client_is_speaking = True
|
||||
|
||||
# 使用 start_sending 启动后台循环
|
||||
rate_controller.start_sending(send_callback)
|
||||
|
||||
|
||||
async def _send_audio_with_rate_control(
|
||||
conn, audio_list, rate_controller, flow_control, send_delay
|
||||
):
|
||||
"""
|
||||
使用 rate_controller 发送音频包
|
||||
|
||||
Args:
|
||||
conn: 连接对象
|
||||
audio_list: 音频包列表
|
||||
rate_controller: 速率控制器
|
||||
flow_control: 流控状态
|
||||
send_delay: 固定延迟（秒），-1表示使用动态流控
|
||||
"""
|
||||
pre_buffer_count = 5
|
||||
|
||||
for packet in audio_list:
|
||||
if conn.client_abort:
|
||||
return
|
||||
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
|
||||
# 预缓冲：前5个包直接发送
|
||||
# 预缓冲：前5个包直接发送，不做延迟
|
||||
pre_buffer_count = 5
|
||||
flow_control = conn.audio_flow_control
|
||||
current_time = time.perf_counter()
|
||||
|
||||
if flow_control["packet_count"] < pre_buffer_count:
|
||||
await _do_send_audio(conn, packet, flow_control)
|
||||
conn.client_is_speaking = True
|
||||
# 预缓冲阶段，直接发送不延迟
|
||||
pass
|
||||
elif send_delay > 0:
|
||||
# 固定延迟模式
|
||||
# 使用固定延迟
|
||||
await asyncio.sleep(send_delay)
|
||||
await _do_send_audio(conn, packet, flow_control)
|
||||
conn.client_is_speaking = True
|
||||
else:
|
||||
# 动态流控模式：仅添加到队列，由后台循环负责发送
|
||||
rate_controller.add_audio(packet)
|
||||
effective_packet = flow_control["packet_count"] - pre_buffer_count
|
||||
expected_time = flow_control["start_time"] + (
|
||||
effective_packet * frame_duration / 1000
|
||||
)
|
||||
delay = expected_time - current_time
|
||||
if delay > 0:
|
||||
await asyncio.sleep(delay)
|
||||
else:
|
||||
# 纠正误差
|
||||
flow_control["start_time"] += abs(delay)
|
||||
|
||||
if conn.conn_from_mqtt_gateway:
|
||||
# 计算时间戳和序列号
|
||||
timestamp, sequence = calculate_timestamp_and_sequence(
|
||||
conn,
|
||||
flow_control["start_time"],
|
||||
flow_control["packet_count"],
|
||||
frame_duration,
|
||||
)
|
||||
# 调用通用函数发送带头部的数据包
|
||||
await _send_to_mqtt_gateway(conn, audios, timestamp, sequence)
|
||||
else:
|
||||
# 直接发送opus数据包，不添加头部
|
||||
await conn.websocket.send(audios)
|
||||
|
||||
async def _do_send_audio(conn, opus_packet, flow_control):
|
||||
"""
|
||||
执行实际的音频发送
|
||||
"""
|
||||
packet_index = flow_control.get("packet_count", 0)
|
||||
sequence = flow_control.get("sequence", 0)
|
||||
|
||||
if conn.conn_from_mqtt_gateway:
|
||||
# 计算时间戳（基于播放位置）
|
||||
start_time = time.time()
|
||||
timestamp = int(start_time * 1000) % (2**32)
|
||||
await _send_to_mqtt_gateway(conn, opus_packet, timestamp, sequence)
|
||||
# 更新流控状态
|
||||
flow_control["packet_count"] += 1
|
||||
flow_control["sequence"] += 1
|
||||
flow_control["last_send_time"] = time.perf_counter()
|
||||
else:
|
||||
# 直接发送opus数据包
|
||||
await conn.websocket.send(opus_packet)
|
||||
# 文件型音频走普通播放
|
||||
start_time = time.perf_counter()
|
||||
play_position = 0
|
||||
|
||||
# 更新流控状态
|
||||
flow_control["packet_count"] = packet_index + 1
|
||||
flow_control["sequence"] = sequence + 1
|
||||
# 执行预缓冲
|
||||
pre_buffer_frames = min(5, len(audios))
|
||||
for i in range(pre_buffer_frames):
|
||||
if conn.conn_from_mqtt_gateway:
|
||||
# 计算时间戳和序列号
|
||||
timestamp, sequence = calculate_timestamp_and_sequence(
|
||||
conn, start_time, i, frame_duration
|
||||
)
|
||||
# 调用通用函数发送带头部的数据包
|
||||
await _send_to_mqtt_gateway(conn, audios[i], timestamp, sequence)
|
||||
else:
|
||||
# 直接发送预缓冲包，不添加头部
|
||||
await conn.websocket.send(audios[i])
|
||||
remaining_audios = audios[pre_buffer_frames:]
|
||||
|
||||
# 播放剩余音频帧
|
||||
for i, opus_packet in enumerate(remaining_audios):
|
||||
if conn.client_abort:
|
||||
break
|
||||
|
||||
# 重置没有声音的状态
|
||||
conn.last_activity_time = time.time() * 1000
|
||||
|
||||
if send_delay > 0:
|
||||
# 固定延迟模式
|
||||
await asyncio.sleep(send_delay)
|
||||
else:
|
||||
# 计算预期发送时间
|
||||
expected_time = start_time + (play_position / 1000)
|
||||
current_time = time.perf_counter()
|
||||
delay = expected_time - current_time
|
||||
if delay > 0:
|
||||
await asyncio.sleep(delay)
|
||||
|
||||
if conn.conn_from_mqtt_gateway:
|
||||
# 计算时间戳和序列号（使用当前的数据包索引确保连续性）
|
||||
packet_index = pre_buffer_frames + i
|
||||
timestamp, sequence = calculate_timestamp_and_sequence(
|
||||
conn, start_time, packet_index, frame_duration
|
||||
)
|
||||
# 调用通用函数发送带头部的数据包
|
||||
await _send_to_mqtt_gateway(conn, opus_packet, timestamp, sequence)
|
||||
else:
|
||||
# 直接发送opus数据包，不添加头部
|
||||
await conn.websocket.send(opus_packet)
|
||||
|
||||
play_position += frame_duration
|
||||
|
||||
|
||||
async def send_tts_message(conn, state, text=None):
|
||||
@@ -246,10 +220,8 @@ async def send_tts_message(conn, state, text=None):
|
||||
stop_tts_notify_voice = conn.config.get(
|
||||
"stop_tts_notify_voice", "config/assets/tts_notify.mp3"
|
||||
)
|
||||
audios = await audio_to_data(stop_tts_notify_voice, is_opus=True)
|
||||
audios = audio_to_data(stop_tts_notify_voice, is_opus=True)
|
||||
await sendAudio(conn, audios)
|
||||
# 等待所有音频包发送完成
|
||||
await _wait_for_audio_completion(conn)
|
||||
# 清除服务端讲话状态
|
||||
conn.clearSpeakStatus()
|
||||
|
||||
@@ -283,4 +255,5 @@ async def send_stt_message(conn, text):
|
||||
await conn.websocket.send(
|
||||
json.dumps({"type": "stt", "text": stt_text, "session_id": conn.session_id})
|
||||
)
|
||||
conn.client_is_speaking = True
|
||||
await send_tts_message(conn, "start")
|
||||
|
||||
@@ -1,14 +1,12 @@
|
||||
import time
|
||||
import asyncio
|
||||
from typing import Dict, Any
|
||||
|
||||
from core.handle.receiveAudioHandle import startToChat
|
||||
from core.handle.receiveAudioHandle import handleAudioMessage, startToChat
|
||||
from core.handle.reportHandle import enqueue_asr_report
|
||||
from core.handle.sendAudioHandle import send_stt_message, send_tts_message
|
||||
from core.handle.textMessageHandler import TextMessageHandler
|
||||
from core.handle.textMessageType import TextMessageType
|
||||
from core.utils.util import remove_punctuation_and_length
|
||||
from core.providers.asr.dto.dto import InterfaceType
|
||||
|
||||
TAG = __name__
|
||||
|
||||
@@ -31,18 +29,8 @@ class ListenTextMessageHandler(TextMessageHandler):
|
||||
elif msg_json["state"] == "stop":
|
||||
conn.client_have_voice = True
|
||||
conn.client_voice_stop = True
|
||||
if conn.asr.interface_type == InterfaceType.STREAM:
|
||||
# 流式模式下，发送结束请求
|
||||
asyncio.create_task(conn.asr._send_stop_request())
|
||||
else:
|
||||
# 非流式模式：直接触发ASR识别
|
||||
if len(conn.asr_audio) > 0:
|
||||
asr_audio_task = conn.asr_audio.copy()
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
|
||||
if len(asr_audio_task) > 0:
|
||||
await conn.asr.handle_voice_stop(conn, asr_audio_task)
|
||||
if len(conn.asr_audio) > 0:
|
||||
await handleAudioMessage(conn, b"")
|
||||
elif msg_json["state"] == "detect":
|
||||
conn.client_have_voice = False
|
||||
conn.asr_audio.clear()
|
||||
|
||||
@@ -8,6 +8,8 @@ import asyncio
|
||||
import requests
|
||||
import websockets
|
||||
import opuslib_next
|
||||
import random
|
||||
from typing import Optional, Tuple, List
|
||||
from urllib import parse
|
||||
from datetime import datetime
|
||||
from config.logger import setup_logging
|
||||
@@ -137,13 +139,13 @@ class ASRProvider(ASRProviderBase):
|
||||
conn.asr_audio.append(audio)
|
||||
conn.asr_audio = conn.asr_audio[-10:]
|
||||
|
||||
# 只在有声音且没有连接时建立连接（排除正在停止的情况）
|
||||
if audio_have_voice and not self.is_processing and not self.asr_ws:
|
||||
# 只在有声音且没有连接时建立连接
|
||||
if audio_have_voice and not self.is_processing:
|
||||
try:
|
||||
await self._start_recognition(conn)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"开始识别失败: {str(e)}")
|
||||
await self._cleanup()
|
||||
await self._cleanup(conn)
|
||||
return
|
||||
|
||||
if self.asr_ws and self.is_processing and self.server_ready:
|
||||
@@ -183,8 +185,10 @@ class ASRProvider(ASRProviderBase):
|
||||
"header": {
|
||||
"namespace": "SpeechTranscriber",
|
||||
"name": "StartTranscription",
|
||||
"status": 20000000,
|
||||
"message_id": uuid.uuid4().hex,
|
||||
"task_id": self.task_id,
|
||||
"status_text": "Gateway:SUCCESS:Success.",
|
||||
"appkey": self.appkey
|
||||
},
|
||||
"payload": {
|
||||
@@ -203,21 +207,18 @@ class ASRProvider(ASRProviderBase):
|
||||
async def _forward_results(self, conn):
|
||||
"""转发识别结果"""
|
||||
try:
|
||||
while not conn.stop_event.is_set():
|
||||
while self.asr_ws and not conn.stop_event.is_set():
|
||||
try:
|
||||
response = await asyncio.wait_for(self.asr_ws.recv(), timeout=1.0)
|
||||
result = json.loads(response)
|
||||
|
||||
|
||||
header = result.get("header", {})
|
||||
payload = result.get("payload", {})
|
||||
message_name = header.get("name", "")
|
||||
status = header.get("status", 0)
|
||||
|
||||
|
||||
if status != 20000000:
|
||||
if status == 40010004:
|
||||
logger.bind(tag=TAG).warning(f"请在服务端响应完成后再关闭链接，状态码: {status}")
|
||||
break
|
||||
if status in [40000004, 40010003]: # 连接超时或客户端断开
|
||||
if status in [40000004, 40010004]: # 连接超时或客户端断开
|
||||
logger.bind(tag=TAG).warning(f"连接问题，状态码: {status}")
|
||||
break
|
||||
elif status in [40270002, 40270003]: # 音频问题
|
||||
@@ -226,12 +227,12 @@ class ASRProvider(ASRProviderBase):
|
||||
else:
|
||||
logger.bind(tag=TAG).error(f"识别错误，状态码: {status}, 消息: {header.get('status_text', '')}")
|
||||
continue
|
||||
|
||||
|
||||
# 收到TranscriptionStarted表示服务器准备好接收音频数据
|
||||
if message_name == "TranscriptionStarted":
|
||||
self.server_ready = True
|
||||
logger.bind(tag=TAG).debug("服务器已准备，开始发送缓存音频...")
|
||||
|
||||
|
||||
# 发送缓存音频
|
||||
if conn.asr_audio:
|
||||
for cached_audio in conn.asr_audio[-10:]:
|
||||
@@ -242,89 +243,89 @@ class ASRProvider(ASRProviderBase):
|
||||
logger.bind(tag=TAG).warning(f"发送缓存音频失败: {e}")
|
||||
break
|
||||
continue
|
||||
elif message_name == "SentenceEnd":
|
||||
# 句子结束（每个句子都会触发）
|
||||
|
||||
if message_name == "TranscriptionResultChanged":
|
||||
# 中间结果
|
||||
text = payload.get("result", "")
|
||||
if text:
|
||||
logger.bind(tag=TAG).info(f"识别到文本: {text}")
|
||||
|
||||
# 手动模式下累积识别结果
|
||||
if conn.client_listen_mode == "manual":
|
||||
if self.text:
|
||||
self.text += text
|
||||
else:
|
||||
self.text = text
|
||||
|
||||
# 手动模式下，只有在收到stop信号后才触发处理（仅处理一次）
|
||||
if conn.client_voice_stop:
|
||||
audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
|
||||
if len(audio_data) > 0:
|
||||
logger.bind(tag=TAG).debug("收到最终识别结果，触发处理")
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
# 清理音频缓存
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
break
|
||||
else:
|
||||
# 自动模式下直接覆盖
|
||||
self.text = text
|
||||
conn.reset_vad_states()
|
||||
audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
break
|
||||
|
||||
self.text = text
|
||||
elif message_name == "SentenceEnd":
|
||||
# 最终结果
|
||||
text = payload.get("result", "")
|
||||
if text:
|
||||
self.text = text
|
||||
conn.reset_vad_states()
|
||||
# 传递缓存的音频数据
|
||||
audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
# 清空缓存
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
break
|
||||
elif message_name == "TranscriptionCompleted":
|
||||
# 识别完成
|
||||
self.is_processing = False
|
||||
break
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
logger.bind(tag=TAG).error("接收结果超时")
|
||||
break
|
||||
except websockets.ConnectionClosed:
|
||||
logger.bind(tag=TAG).info("ASR服务连接已关闭")
|
||||
self.is_processing = False
|
||||
continue
|
||||
except websockets.exceptions.ConnectionClosed:
|
||||
break
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"处理结果失败: {str(e)}")
|
||||
break
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"结果转发失败: {str(e)}")
|
||||
finally:
|
||||
# 清理连接的音频缓存
|
||||
await self._cleanup()
|
||||
if conn:
|
||||
if hasattr(conn, 'asr_audio_for_voiceprint'):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, 'asr_audio'):
|
||||
conn.asr_audio = []
|
||||
await self._cleanup(conn)
|
||||
|
||||
async def _send_stop_request(self):
|
||||
"""发送停止识别请求（不关闭连接）"""
|
||||
if self.asr_ws:
|
||||
async def _cleanup(self, conn):
|
||||
"""清理资源"""
|
||||
logger.bind(tag=TAG).debug(f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}")
|
||||
|
||||
# 清理连接的音频缓存
|
||||
if conn and hasattr(conn, 'asr_audio_for_voiceprint'):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
|
||||
# 判断是否需要发送终止请求
|
||||
should_stop = self.is_processing or self.server_ready
|
||||
|
||||
# 发送停止识别请求
|
||||
if self.asr_ws and should_stop:
|
||||
try:
|
||||
# 先停止音频发送
|
||||
self.is_processing = False
|
||||
|
||||
stop_msg = {
|
||||
"header": {
|
||||
"namespace": "SpeechTranscriber",
|
||||
"name": "StopTranscription",
|
||||
"status": 20000000,
|
||||
"message_id": uuid.uuid4().hex,
|
||||
"task_id": self.task_id,
|
||||
"status_text": "Client:Stop",
|
||||
"appkey": self.appkey
|
||||
}
|
||||
}
|
||||
logger.bind(tag=TAG).debug("停止识别请求已发送")
|
||||
logger.bind(tag=TAG).debug("正在发送ASR终止请求")
|
||||
await self.asr_ws.send(json.dumps(stop_msg, ensure_ascii=False))
|
||||
await asyncio.sleep(0.1)
|
||||
logger.bind(tag=TAG).debug("ASR终止请求已发送")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"发送停止识别请求失败: {e}")
|
||||
|
||||
async def _cleanup(self):
|
||||
"""清理资源（关闭连接）"""
|
||||
logger.bind(tag=TAG).debug(f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}")
|
||||
|
||||
# 状态重置
|
||||
logger.bind(tag=TAG).error(f"ASR终止请求发送失败: {e}")
|
||||
|
||||
# 状态重置（在终止请求发送后）
|
||||
self.is_processing = False
|
||||
self.server_ready = False
|
||||
logger.bind(tag=TAG).debug("ASR状态已重置")
|
||||
|
||||
# 清理任务
|
||||
if self.forward_task and not self.forward_task.done():
|
||||
self.forward_task.cancel()
|
||||
try:
|
||||
await asyncio.wait_for(self.forward_task, timeout=1.0)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"forward_task取消异常: {e}")
|
||||
finally:
|
||||
self.forward_task = None
|
||||
|
||||
# 关闭连接
|
||||
if self.asr_ws:
|
||||
try:
|
||||
@@ -335,10 +336,7 @@ class ASRProvider(ASRProviderBase):
|
||||
logger.bind(tag=TAG).error(f"关闭WebSocket连接失败: {e}")
|
||||
finally:
|
||||
self.asr_ws = None
|
||||
|
||||
# 清理任务引用
|
||||
self.forward_task = None
|
||||
|
||||
|
||||
logger.bind(tag=TAG).debug("ASR会话清理完成")
|
||||
|
||||
async def speech_to_text(self, opus_data, session_id, audio_format):
|
||||
@@ -349,11 +347,4 @@ class ASRProvider(ASRProviderBase):
|
||||
|
||||
async def close(self):
|
||||
"""关闭资源"""
|
||||
await self._cleanup(None)
|
||||
if hasattr(self, 'decoder') and self.decoder is not None:
|
||||
try:
|
||||
del self.decoder
|
||||
self.decoder = None
|
||||
logger.bind(tag=TAG).debug("Aliyun decoder resources released")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"释放Aliyun decoder资源时出错: {e}")
|
||||
await self._cleanup()
|
||||
|
||||
@@ -9,6 +9,7 @@ import asyncio
|
||||
import traceback
|
||||
import threading
|
||||
import opuslib_next
|
||||
import concurrent.futures
|
||||
from abc import ABC, abstractmethod
|
||||
from config.logger import setup_logging
|
||||
from typing import Optional, Tuple, List
|
||||
@@ -52,89 +53,121 @@ class ASRProviderBase(ABC):
|
||||
|
||||
# 接收音频
|
||||
async def receive_audio(self, conn, audio, audio_have_voice):
|
||||
if conn.client_listen_mode == "manual":
|
||||
# 手动模式：缓存音频用于ASR识别
|
||||
conn.asr_audio.append(audio)
|
||||
else:
|
||||
# 自动/实时模式：使用VAD检测
|
||||
if conn.client_listen_mode == "auto" or conn.client_listen_mode == "realtime":
|
||||
have_voice = audio_have_voice
|
||||
else:
|
||||
have_voice = conn.client_have_voice
|
||||
|
||||
conn.asr_audio.append(audio)
|
||||
if not have_voice and not conn.client_have_voice:
|
||||
conn.asr_audio = conn.asr_audio[-10:]
|
||||
return
|
||||
|
||||
conn.asr_audio.append(audio)
|
||||
if not have_voice and not conn.client_have_voice:
|
||||
conn.asr_audio = conn.asr_audio[-10:]
|
||||
return
|
||||
if conn.client_voice_stop:
|
||||
asr_audio_task = conn.asr_audio.copy()
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
|
||||
# 自动模式下通过VAD检测到语音停止时触发识别
|
||||
if conn.client_voice_stop:
|
||||
asr_audio_task = conn.asr_audio.copy()
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
|
||||
if len(asr_audio_task) > 15:
|
||||
await self.handle_voice_stop(conn, asr_audio_task)
|
||||
if len(asr_audio_task) > 15 or conn.client_listen_mode == "manual":
|
||||
await self.handle_voice_stop(conn, asr_audio_task)
|
||||
|
||||
# 处理语音停止
|
||||
async def handle_voice_stop(self, conn, asr_audio_task: List[bytes]):
|
||||
"""并行处理ASR和声纹识别"""
|
||||
try:
|
||||
total_start_time = time.monotonic()
|
||||
|
||||
|
||||
# 准备音频数据
|
||||
if conn.audio_format == "pcm":
|
||||
pcm_data = asr_audio_task
|
||||
else:
|
||||
pcm_data = self.decode_opus(asr_audio_task)
|
||||
|
||||
|
||||
combined_pcm_data = b"".join(pcm_data)
|
||||
|
||||
|
||||
# 预先准备WAV数据
|
||||
wav_data = None
|
||||
if conn.voiceprint_provider and combined_pcm_data:
|
||||
wav_data = self._pcm_to_wav(combined_pcm_data)
|
||||
|
||||
|
||||
# 定义ASR任务
|
||||
asr_task = self.speech_to_text(asr_audio_task, conn.session_id, conn.audio_format)
|
||||
|
||||
if conn.voiceprint_provider and wav_data:
|
||||
voiceprint_task = conn.voiceprint_provider.identify_speaker(wav_data, conn.session_id)
|
||||
# 并发等待两个结果
|
||||
asr_result, voiceprint_result = await asyncio.gather(
|
||||
asr_task, voiceprint_task, return_exceptions=True
|
||||
)
|
||||
else:
|
||||
asr_result = await asr_task
|
||||
voiceprint_result = None
|
||||
|
||||
# 记录识别结果 - 检查是否为异常
|
||||
if isinstance(asr_result, Exception):
|
||||
logger.bind(tag=TAG).error(f"ASR识别失败: {asr_result}")
|
||||
raw_text = ""
|
||||
else:
|
||||
raw_text, _ = asr_result
|
||||
|
||||
if isinstance(voiceprint_result, Exception):
|
||||
logger.bind(tag=TAG).error(f"声纹识别失败: {voiceprint_result}")
|
||||
speaker_name = ""
|
||||
else:
|
||||
speaker_name = voiceprint_result
|
||||
|
||||
def run_asr():
|
||||
start_time = time.monotonic()
|
||||
try:
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
result = loop.run_until_complete(
|
||||
self.speech_to_text(asr_audio_task, conn.session_id, conn.audio_format)
|
||||
)
|
||||
end_time = time.monotonic()
|
||||
logger.bind(tag=TAG).debug(f"ASR耗时: {end_time - start_time:.3f}s")
|
||||
return result
|
||||
finally:
|
||||
loop.close()
|
||||
except Exception as e:
|
||||
end_time = time.monotonic()
|
||||
logger.bind(tag=TAG).error(f"ASR失败: {e}")
|
||||
return ("", None)
|
||||
|
||||
# 定义声纹识别任务
|
||||
def run_voiceprint():
|
||||
if not wav_data:
|
||||
return None
|
||||
try:
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
# 使用连接的声纹识别提供者
|
||||
result = loop.run_until_complete(
|
||||
conn.voiceprint_provider.identify_speaker(wav_data, conn.session_id)
|
||||
)
|
||||
return result
|
||||
finally:
|
||||
loop.close()
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"声纹识别失败: {e}")
|
||||
return None
|
||||
|
||||
# 使用线程池执行器并行运行
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as thread_executor:
|
||||
asr_future = thread_executor.submit(run_asr)
|
||||
|
||||
if conn.voiceprint_provider and wav_data:
|
||||
voiceprint_future = thread_executor.submit(run_voiceprint)
|
||||
|
||||
# 等待两个线程都完成
|
||||
asr_result = asr_future.result(timeout=15)
|
||||
voiceprint_result = voiceprint_future.result(timeout=15)
|
||||
|
||||
results = {"asr": asr_result, "voiceprint": voiceprint_result}
|
||||
else:
|
||||
asr_result = asr_future.result(timeout=15)
|
||||
results = {"asr": asr_result, "voiceprint": None}
|
||||
|
||||
|
||||
# 处理结果
|
||||
raw_text, _ = results.get("asr", ("", None))
|
||||
speaker_name = results.get("voiceprint", None)
|
||||
|
||||
# 记录识别结果
|
||||
if raw_text:
|
||||
logger.bind(tag=TAG).info(f"识别文本: {raw_text}")
|
||||
if speaker_name:
|
||||
logger.bind(tag=TAG).info(f"识别说话人: {speaker_name}")
|
||||
|
||||
|
||||
# 性能监控
|
||||
total_time = time.monotonic() - total_start_time
|
||||
logger.bind(tag=TAG).debug(f"总处理耗时: {total_time:.3f}s")
|
||||
|
||||
|
||||
# 检查文本长度
|
||||
text_len, _ = remove_punctuation_and_length(raw_text)
|
||||
self.stop_ws_connection()
|
||||
|
||||
|
||||
if text_len > 0:
|
||||
# 构建包含说话人信息的JSON字符串
|
||||
enhanced_text = self._build_enhanced_text(raw_text, speaker_name)
|
||||
|
||||
|
||||
# 使用自定义模块进行上报
|
||||
await startToChat(conn, enhanced_text)
|
||||
enqueue_asr_report(conn, enhanced_text, asr_audio_task)
|
||||
@@ -208,7 +241,6 @@ class ASRProviderBase(ABC):
|
||||
@staticmethod
|
||||
def decode_opus(opus_data: List[bytes]) -> List[bytes]:
|
||||
"""将Opus音频数据解码为PCM数据"""
|
||||
decoder = None
|
||||
try:
|
||||
decoder = opuslib_next.Decoder(16000, 1)
|
||||
pcm_data = []
|
||||
@@ -233,9 +265,3 @@ class ASRProviderBase(ABC):
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"音频解码过程发生错误: {e}")
|
||||
return []
|
||||
finally:
|
||||
if decoder is not None:
|
||||
try:
|
||||
del decoder
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"释放decoder资源时出错: {e}")
|
||||
|
||||
@@ -18,6 +18,8 @@ class ASRProvider(ASRProviderBase):
|
||||
self.interface_type = InterfaceType.STREAM
|
||||
self.config = config
|
||||
self.text = ""
|
||||
self.max_retries = 3
|
||||
self.retry_delay = 2
|
||||
self.decoder = opuslib_next.Decoder(16000, 1)
|
||||
self.asr_ws = None
|
||||
self.forward_task = None
|
||||
@@ -54,13 +56,14 @@ class ASRProvider(ASRProviderBase):
|
||||
async def receive_audio(self, conn, audio, audio_have_voice):
|
||||
conn.asr_audio.append(audio)
|
||||
conn.asr_audio = conn.asr_audio[-10:]
|
||||
|
||||
# 存储音频数据
|
||||
if not hasattr(conn, 'asr_audio_for_voiceprint'):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
conn.asr_audio_for_voiceprint.append(audio)
|
||||
|
||||
|
||||
# 当没有音频数据时处理完整语音片段
|
||||
if conn.client_listen_mode != "manual" and not audio and len(conn.asr_audio_for_voiceprint) > 0:
|
||||
if not audio and len(conn.asr_audio_for_voiceprint) > 0:
|
||||
await self.handle_voice_stop(conn, conn.asr_audio_for_voiceprint)
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
|
||||
@@ -176,7 +179,6 @@ class ASRProvider(ASRProviderBase):
|
||||
payload.get("audio_info", {}).get("duration", 0) > 2000
|
||||
and not utterances
|
||||
and not payload["result"].get("text")
|
||||
and conn.client_listen_mode != "manual"
|
||||
):
|
||||
logger.bind(tag=TAG).error(f"识别文本：空")
|
||||
self.text = ""
|
||||
@@ -185,44 +187,15 @@ class ASRProvider(ASRProviderBase):
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
break
|
||||
|
||||
# 专门处理没有文本的识别结果（手动模式下可能已经识别完成但是没松按键）
|
||||
elif not payload["result"].get("text") and not utterances:
|
||||
if conn.client_listen_mode == "manual" and conn.client_voice_stop and len(audio_data) > 0:
|
||||
logger.bind(tag=TAG).debug("消息结束收到停止信号，触发处理")
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
# 清理音频缓存
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
break
|
||||
|
||||
for utterance in utterances:
|
||||
if utterance.get("definite", False):
|
||||
current_text = utterance["text"]
|
||||
self.text = utterance["text"]
|
||||
logger.bind(tag=TAG).info(
|
||||
f"识别到文本: {current_text}"
|
||||
f"识别到文本: {self.text}"
|
||||
)
|
||||
|
||||
# 手动模式下累积识别结果
|
||||
if conn.client_listen_mode == "manual":
|
||||
if self.text:
|
||||
self.text += current_text
|
||||
else:
|
||||
self.text = current_text
|
||||
|
||||
# 在接收消息中途时收到停止信号
|
||||
if conn.client_voice_stop and len(audio_data) > 0:
|
||||
logger.bind(tag=TAG).debug("消息中途收到停止信号，触发处理")
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
# 清理音频缓存
|
||||
conn.asr_audio.clear()
|
||||
conn.reset_vad_states()
|
||||
break
|
||||
else:
|
||||
# 自动模式下直接覆盖
|
||||
self.text = current_text
|
||||
conn.reset_vad_states()
|
||||
if len(audio_data) > 15: # 确保有足够音频数据
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
conn.reset_vad_states()
|
||||
if len(audio_data) > 15: # 确保有足够音频数据
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
break
|
||||
elif "error" in payload:
|
||||
error_msg = payload.get("error", "未知错误")
|
||||
@@ -254,6 +227,8 @@ class ASRProvider(ASRProviderBase):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, 'asr_audio'):
|
||||
conn.asr_audio = []
|
||||
if hasattr(conn, 'has_valid_voice'):
|
||||
conn.has_valid_voice = False
|
||||
|
||||
def stop_ws_connection(self):
|
||||
if self.asr_ws:
|
||||
@@ -261,20 +236,6 @@ class ASRProvider(ASRProviderBase):
|
||||
self.asr_ws = None
|
||||
self.is_processing = False
|
||||
|
||||
async def _send_stop_request(self):
|
||||
"""发送最后一个音频帧以通知服务器结束"""
|
||||
if self.asr_ws:
|
||||
try:
|
||||
# 发送结束标记的音频帧（gzip压缩的空数据）
|
||||
empty_payload = gzip.compress(b"")
|
||||
last_audio_request = bytearray(self.generate_last_audio_default_header())
|
||||
last_audio_request.extend(len(empty_payload).to_bytes(4, "big"))
|
||||
last_audio_request.extend(empty_payload)
|
||||
await self.asr_ws.send(last_audio_request)
|
||||
logger.bind(tag=TAG).debug("已发送结束音频帧")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"发送结束音频帧时出错: {e}")
|
||||
|
||||
def construct_request(self, reqid):
|
||||
req = {
|
||||
"app": {
|
||||
@@ -409,16 +370,6 @@ class ASRProvider(ASRProviderBase):
|
||||
pass
|
||||
self.forward_task = None
|
||||
self.is_processing = False
|
||||
|
||||
# 显式释放decoder资源
|
||||
if hasattr(self, 'decoder') and self.decoder is not None:
|
||||
try:
|
||||
del self.decoder
|
||||
self.decoder = None
|
||||
logger.bind(tag=TAG).debug("Doubao decoder resources released")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"释放Doubao decoder资源时出错: {e}")
|
||||
|
||||
# 清理所有连接的音频缓冲区
|
||||
if hasattr(self, '_connections'):
|
||||
for conn in self._connections.values():
|
||||
@@ -426,3 +377,5 @@ class ASRProvider(ASRProviderBase):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, 'asr_audio'):
|
||||
conn.asr_audio = []
|
||||
if hasattr(conn, 'has_valid_voice'):
|
||||
conn.has_valid_voice = False
|
||||
|
||||
@@ -1,16 +1,14 @@
|
||||
import os
|
||||
import io
|
||||
import sys
|
||||
import time
|
||||
import shutil
|
||||
import os
|
||||
import sys
|
||||
import io
|
||||
import psutil
|
||||
import asyncio
|
||||
|
||||
from config.logger import setup_logging
|
||||
from typing import Optional, Tuple, List
|
||||
from core.providers.asr.base import ASRProviderBase
|
||||
from funasr import AutoModel
|
||||
from funasr.utils.postprocess_utils import rich_transcription_postprocess
|
||||
from core.providers.asr.base import ASRProviderBase
|
||||
import shutil
|
||||
from core.providers.asr.dto.dto import InterfaceType
|
||||
|
||||
TAG = __name__
|
||||
@@ -92,17 +90,16 @@ class ASRProvider(ASRProviderBase):
|
||||
else:
|
||||
file_path = self.save_audio_to_file(pcm_data, session_id)
|
||||
|
||||
# 语音识别 - 使用线程池避免阻塞事件循环
|
||||
# 语音识别
|
||||
start_time = time.time()
|
||||
result = await asyncio.to_thread(
|
||||
self.model.generate,
|
||||
result = self.model.generate(
|
||||
input=combined_pcm_data,
|
||||
cache={},
|
||||
language="auto",
|
||||
use_itn=True,
|
||||
batch_size_s=60,
|
||||
)
|
||||
text = await asyncio.to_thread(rich_transcription_postprocess, result[0]["text"])
|
||||
text = rich_transcription_postprocess(result[0]["text"])
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"语音识别耗时: {time.time() - start_time:.3f}s | 结果: {text}"
|
||||
)
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import os
|
||||
import json
|
||||
import asyncio
|
||||
import tempfile
|
||||
import difflib
|
||||
from typing import Optional, Tuple, List
|
||||
import dashscope
|
||||
from config.logger import setup_logging
|
||||
@@ -13,8 +16,7 @@ logger = setup_logging()
|
||||
class ASRProvider(ASRProviderBase):
|
||||
def __init__(self, config: dict, delete_audio_file: bool):
|
||||
super().__init__()
|
||||
# 音频文件上传类型，流式文本识别输出
|
||||
self.interface_type = InterfaceType.NON_STREAM
|
||||
self.interface_type = InterfaceType.STREAM
|
||||
"""Qwen3-ASR-Flash ASR初始化"""
|
||||
|
||||
# 配置参数
|
||||
@@ -128,11 +130,27 @@ class ASRProvider(ASRProviderBase):
|
||||
|
||||
# 处理流式响应
|
||||
full_text = ""
|
||||
last_text = "" # 用于存储上一个文本片段
|
||||
for chunk in response:
|
||||
try:
|
||||
text = chunk["output"]["choices"][0]["message"].content[0]["text"]
|
||||
# 更新为最新的完整文本
|
||||
full_text = text.strip()
|
||||
# 标准化文本片段（去除首尾空格）
|
||||
normalized_text = text.strip()
|
||||
# 只有当新文本片段与上一个不同时才处理
|
||||
if normalized_text != last_text:
|
||||
# 提取新增的文本部分
|
||||
# 通过比较当前文本和上一个文本，找到新增的部分
|
||||
if normalized_text.startswith(last_text):
|
||||
# 如果当前文本以最后一个文本开头，则新增部分是两者的差集
|
||||
new_part = normalized_text[len(last_text):]
|
||||
else:
|
||||
# 如果不以最后一个文本开头，说明识别结果发生了较大变化，直接使用当前文本
|
||||
new_part = normalized_text
|
||||
|
||||
# 将新增部分添加到完整文本中
|
||||
full_text += new_part
|
||||
last_text = normalized_text
|
||||
# 这里可以实时处理文本片段，例如通过回调函数
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
@@ -5,7 +5,6 @@ import hashlib
|
||||
import asyncio
|
||||
import websockets
|
||||
import opuslib_next
|
||||
import gc
|
||||
from time import mktime
|
||||
from datetime import datetime
|
||||
from urllib.parse import urlencode
|
||||
@@ -35,6 +34,9 @@ class ASRProvider(ASRProviderBase):
|
||||
self.forward_task = None
|
||||
self.is_processing = False
|
||||
self.server_ready = False
|
||||
self.last_frame_sent = False # 标记是否已发送最终帧
|
||||
self.best_text = "" # 保存最佳识别结果
|
||||
self.has_final_result = False # 标记是否收到最终识别结果
|
||||
|
||||
# 讯飞配置
|
||||
self.app_id = config.get("app_id")
|
||||
@@ -49,6 +51,7 @@ class ASRProvider(ASRProviderBase):
|
||||
"domain": config.get("domain", "slm"),
|
||||
"language": config.get("language", "zh_cn"),
|
||||
"accent": config.get("accent", "mandarin"),
|
||||
"dwa": config.get("dwa", "wpgs"),
|
||||
"result": {"encoding": "utf8", "compress": "raw", "format": "plain"},
|
||||
}
|
||||
|
||||
@@ -112,7 +115,7 @@ class ASRProvider(ASRProviderBase):
|
||||
await self._start_recognition(conn)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"建立ASR连接失败: {str(e)}")
|
||||
await self._cleanup()
|
||||
await self._cleanup(conn)
|
||||
return
|
||||
|
||||
# 发送当前音频数据
|
||||
@@ -122,7 +125,7 @@ class ASRProvider(ASRProviderBase):
|
||||
await self._send_audio_frame(pcm_frame, STATUS_CONTINUE_FRAME)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).warning(f"发送音频数据时发生错误: {e}")
|
||||
await self._cleanup()
|
||||
await self._cleanup(conn)
|
||||
|
||||
async def _start_recognition(self, conn):
|
||||
"""开始识别会话"""
|
||||
@@ -132,10 +135,6 @@ class ASRProvider(ASRProviderBase):
|
||||
ws_url = self.create_url()
|
||||
logger.bind(tag=TAG).info(f"正在连接ASR服务: {ws_url[:50]}...")
|
||||
|
||||
# 如果为手动模式,设置超时时长为一分钟
|
||||
if conn.client_listen_mode == "manual":
|
||||
self.iat_params["eos"] = 60000
|
||||
|
||||
self.asr_ws = await websockets.connect(
|
||||
ws_url,
|
||||
max_size=1000000000,
|
||||
@@ -146,6 +145,8 @@ class ASRProvider(ASRProviderBase):
|
||||
|
||||
logger.bind(tag=TAG).info("ASR WebSocket连接已建立")
|
||||
self.server_ready = False
|
||||
self.last_frame_sent = False
|
||||
self.best_text = ""
|
||||
self.forward_task = asyncio.create_task(self._forward_results(conn))
|
||||
|
||||
# 发送首帧音频
|
||||
@@ -194,12 +195,23 @@ class ASRProvider(ASRProviderBase):
|
||||
|
||||
await self.asr_ws.send(json.dumps(frame_data, ensure_ascii=False))
|
||||
|
||||
# 标记是否发送了最终帧
|
||||
if status == STATUS_LAST_FRAME:
|
||||
self.last_frame_sent = True
|
||||
logger.bind(tag=TAG).info("标记最终帧已发送")
|
||||
|
||||
async def _forward_results(self, conn):
|
||||
"""转发识别结果"""
|
||||
try:
|
||||
while not conn.stop_event.is_set():
|
||||
while self.asr_ws and not conn.stop_event.is_set():
|
||||
# 获取当前连接的音频数据
|
||||
audio_data = getattr(conn, "asr_audio_for_voiceprint", [])
|
||||
try:
|
||||
response = await asyncio.wait_for(self.asr_ws.recv(), timeout=60)
|
||||
# 如果已发送最终帧，增加超时时间等待完整结果
|
||||
timeout = 3.0 if self.last_frame_sent else 30.0
|
||||
response = await asyncio.wait_for(
|
||||
self.asr_ws.recv(), timeout=timeout
|
||||
)
|
||||
result = json.loads(response)
|
||||
logger.bind(tag=TAG).debug(f"收到ASR结果: {result}")
|
||||
|
||||
@@ -223,27 +235,144 @@ class ASRProvider(ASRProviderBase):
|
||||
# 解码base64文本
|
||||
decoded_text = base64.b64decode(text_data).decode("utf-8")
|
||||
text_json = json.loads(decoded_text)
|
||||
|
||||
# 提取文本内容
|
||||
text_ws = text_json.get("ws", [])
|
||||
result_text = ""
|
||||
for i in text_ws:
|
||||
for j in i.get("cw", []):
|
||||
w = j.get("w", "")
|
||||
self.text += w
|
||||
result_text += w
|
||||
|
||||
# 更新识别文本 - 实时更新策略
|
||||
# 只检查是否为空字符串，不再过滤任何标点符号
|
||||
# 这样可以确保所有识别到的内容，包括标点符号都能被实时更新
|
||||
if result_text and result_text.strip():
|
||||
# 实时更新：正常情况下都更新，提高响应速度
|
||||
should_update = True
|
||||
|
||||
# 保存最佳文本
|
||||
# 1. 如果是识别完成状态或最终帧后收到的结果，优先保存
|
||||
# 2. 否则保存最长的有意义文本
|
||||
# 取消对标点符号的过滤，只检查是否为空
|
||||
# 这样可以保留所有识别到的内容，包括各种标点符号
|
||||
is_valid_text = len(result_text.strip()) > 0
|
||||
|
||||
if (
|
||||
self.last_frame_sent or status == 2
|
||||
) and is_valid_text:
|
||||
self.best_text = result_text
|
||||
self.has_final_result = True # 标记已收到最终结果
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"保存最终识别结果: {self.best_text}"
|
||||
)
|
||||
elif (
|
||||
len(result_text) > len(self.best_text)
|
||||
and is_valid_text
|
||||
and not self.has_final_result
|
||||
):
|
||||
self.best_text = result_text
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"保存中间最佳文本: {self.best_text}"
|
||||
)
|
||||
|
||||
# 如果已发送最终帧，只过滤空文本
|
||||
if self.last_frame_sent:
|
||||
# 只拒绝完全空的结果
|
||||
if not result_text.strip():
|
||||
should_update = False
|
||||
logger.bind(tag=TAG).warning(
|
||||
f"最终帧后拒绝空文本"
|
||||
)
|
||||
|
||||
if should_update:
|
||||
# 处理流式识别结果，避免简单替换导致内容丢失
|
||||
# 1. 如果是中间状态（非最终帧后），可能需要替换为更完整的识别
|
||||
# 2. 如果是最终帧后收到的结果，可能是对前面文本的补充
|
||||
if self.last_frame_sent:
|
||||
# 最终帧后收到的结果可能是标点符号等补充内容
|
||||
# 检查是否需要合并文本而不是替换
|
||||
# 如果当前文本是纯标点而前面已有内容，应该追加而不是替换
|
||||
if len(
|
||||
self.text
|
||||
) > 0 and result_text.strip() in [
|
||||
"。",
|
||||
".",
|
||||
"?",
|
||||
"？",
|
||||
"!",
|
||||
"！",
|
||||
",",
|
||||
"，",
|
||||
";",
|
||||
"；",
|
||||
]:
|
||||
# 对于标点符号，追加到现有文本后
|
||||
self.text = (
|
||||
self.text.rstrip().rstrip("。.")
|
||||
+ result_text
|
||||
)
|
||||
else:
|
||||
# 其他情况保持替换逻辑
|
||||
self.text = result_text
|
||||
else:
|
||||
# 中间状态替换为新的识别结果
|
||||
self.text = result_text
|
||||
|
||||
logger.bind(tag=TAG).info(
|
||||
f"实时更新识别文本: {self.text} (最终帧已发送: {self.last_frame_sent})"
|
||||
)
|
||||
|
||||
# 识别完成，但如果还没发送最终帧，继续等待
|
||||
if status == 2:
|
||||
if conn.client_listen_mode == "manual":
|
||||
audio_data = getattr(conn, 'asr_audio_for_voiceprint', [])
|
||||
if len(audio_data) > 0:
|
||||
logger.bind(tag=TAG).debug("收到最终识别结果，触发处理")
|
||||
await self.handle_voice_stop(conn, audio_data)
|
||||
# 清理音频缓存
|
||||
conn.asr_audio.clear()
|
||||
logger.bind(tag=TAG).info(
|
||||
f"识别完成状态已到达，当前识别文本: {self.text}"
|
||||
)
|
||||
|
||||
# 如果还没发送最终帧，继续等待
|
||||
if not self.last_frame_sent:
|
||||
logger.bind(tag=TAG).info(
|
||||
"识别完成但最终帧未发送，继续等待..."
|
||||
)
|
||||
continue
|
||||
|
||||
# 已发送最终帧且收到完成状态，使用最佳策略选择最终结果
|
||||
# 优先使用识别完成状态下的最新结果，而不是仅仅基于长度
|
||||
if self.best_text:
|
||||
# 如果当前文本是在最终帧发送后或识别完成状态下收到的，优先使用
|
||||
if (
|
||||
self.last_frame_sent or status == 2
|
||||
) and self.text.strip():
|
||||
logger.bind(tag=TAG).info(
|
||||
f"使用完成状态下的最新识别结果: {self.text}"
|
||||
)
|
||||
elif len(self.best_text) > len(self.text):
|
||||
logger.bind(tag=TAG).info(
|
||||
f"使用更长的最佳文本作为最终结果: {self.text} -> {self.best_text}"
|
||||
)
|
||||
self.text = self.best_text
|
||||
|
||||
logger.bind(tag=TAG).info(f"获取到最终完整文本: {self.text}")
|
||||
conn.reset_vad_states()
|
||||
if len(audio_data) > 15: # 确保有足够音频数据
|
||||
# 准备处理结果
|
||||
pass
|
||||
break
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
logger.bind(tag=TAG).error("接收结果超时")
|
||||
break
|
||||
if self.last_frame_sent:
|
||||
# 超时时也使用最佳文本
|
||||
if self.best_text and len(self.best_text) > len(self.text):
|
||||
logger.bind(tag=TAG).info(
|
||||
f"超时，使用最佳文本: {self.text} -> {self.best_text}"
|
||||
)
|
||||
self.text = self.best_text
|
||||
logger.bind(tag=TAG).info(
|
||||
f"最终帧后超时，使用结果: {self.text}"
|
||||
)
|
||||
break
|
||||
# 如果还没发送最终帧，继续等待
|
||||
continue
|
||||
except websockets.ConnectionClosed:
|
||||
logger.bind(tag=TAG).info("ASR服务连接已关闭")
|
||||
self.is_processing = False
|
||||
@@ -260,15 +389,17 @@ class ASRProvider(ASRProviderBase):
|
||||
if hasattr(e, "__cause__") and e.__cause__:
|
||||
logger.bind(tag=TAG).error(f"错误原因: {str(e.__cause__)}")
|
||||
finally:
|
||||
# 清理连接资源
|
||||
await self._cleanup()
|
||||
|
||||
# 清理连接的音频缓存
|
||||
if self.asr_ws:
|
||||
await self.asr_ws.close()
|
||||
self.asr_ws = None
|
||||
self.is_processing = False
|
||||
if conn:
|
||||
if hasattr(conn, "asr_audio_for_voiceprint"):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, "asr_audio"):
|
||||
conn.asr_audio = []
|
||||
if hasattr(conn, "has_valid_voice"):
|
||||
conn.has_valid_voice = False
|
||||
|
||||
async def handle_voice_stop(self, conn, asr_audio_task: List[bytes]):
|
||||
"""处理语音停止，发送最后一帧并处理识别结果"""
|
||||
@@ -276,13 +407,22 @@ class ASRProvider(ASRProviderBase):
|
||||
# 先发送最后一帧表示音频结束
|
||||
if self.asr_ws and self.is_processing:
|
||||
try:
|
||||
await self._send_audio_frame(b"", STATUS_LAST_FRAME)
|
||||
logger.bind(tag=TAG).debug(f"已发送停止请求")
|
||||
# 取最后一个有效的音频帧作为最后一帧数据
|
||||
last_frame = b""
|
||||
if asr_audio_task:
|
||||
last_audio = asr_audio_task[-1]
|
||||
last_frame = self.decoder.decode(last_audio, 960)
|
||||
await self._send_audio_frame(last_frame, STATUS_LAST_FRAME)
|
||||
logger.bind(tag=TAG).info("已发送最后一帧")
|
||||
|
||||
# 发送最终帧后，给_forward_results适当时间处理最终结果
|
||||
await asyncio.sleep(0.25)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"发送停止请求失败: {e}")
|
||||
|
||||
logger.bind(tag=TAG).info(f"准备处理最终识别结果: {self.text}")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"发送最后一帧失败: {e}")
|
||||
|
||||
# 调用父类的handle_voice_stop方法处理识别结果
|
||||
await super().handle_voice_stop(conn, asr_audio_task)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"处理语音停止失败: {e}")
|
||||
@@ -296,27 +436,40 @@ class ASRProvider(ASRProviderBase):
|
||||
self.asr_ws = None
|
||||
self.is_processing = False
|
||||
|
||||
async def _send_stop_request(self):
|
||||
"""发送停止识别请求（不关闭连接）"""
|
||||
if self.asr_ws:
|
||||
try:
|
||||
# 先停止音频发送
|
||||
self.is_processing = False
|
||||
await self._send_audio_frame(b"", STATUS_LAST_FRAME)
|
||||
logger.bind(tag=TAG).debug("已发送停止请求")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"发送停止请求失败: {e}")
|
||||
|
||||
async def _cleanup(self):
|
||||
"""清理资源（关闭连接）"""
|
||||
logger.bind(tag=TAG).debug(
|
||||
async def _cleanup(self, conn):
|
||||
"""清理资源"""
|
||||
logger.bind(tag=TAG).info(
|
||||
f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}"
|
||||
)
|
||||
|
||||
# 发送最后一帧
|
||||
if self.asr_ws and self.is_processing:
|
||||
try:
|
||||
await self._send_audio_frame(b"", STATUS_LAST_FRAME)
|
||||
await asyncio.sleep(0.1)
|
||||
logger.bind(tag=TAG).info("已发送最后一帧")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"发送最后一帧失败: {e}")
|
||||
|
||||
# 状态重置
|
||||
self.is_processing = False
|
||||
self.server_ready = False
|
||||
logger.bind(tag=TAG).debug("ASR状态已重置")
|
||||
self.last_frame_sent = False
|
||||
self.best_text = ""
|
||||
self.has_final_result = False
|
||||
logger.bind(tag=TAG).info("ASR状态已重置")
|
||||
|
||||
# 清理任务
|
||||
if self.forward_task and not self.forward_task.done():
|
||||
self.forward_task.cancel()
|
||||
try:
|
||||
await asyncio.wait_for(self.forward_task, timeout=1.0)
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"forward_task取消异常: {e}")
|
||||
finally:
|
||||
self.forward_task = None
|
||||
|
||||
# 关闭连接
|
||||
if self.asr_ws:
|
||||
@@ -329,10 +482,16 @@ class ASRProvider(ASRProviderBase):
|
||||
finally:
|
||||
self.asr_ws = None
|
||||
|
||||
# 清理任务引用
|
||||
self.forward_task = None
|
||||
# 清理连接的音频缓存
|
||||
if conn:
|
||||
if hasattr(conn, "asr_audio_for_voiceprint"):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, "asr_audio"):
|
||||
conn.asr_audio = []
|
||||
if hasattr(conn, "has_valid_voice"):
|
||||
conn.has_valid_voice = False
|
||||
|
||||
logger.bind(tag=TAG).debug("ASR会话清理完成")
|
||||
logger.bind(tag=TAG).info("ASR会话清理完成")
|
||||
|
||||
async def speech_to_text(self, opus_data, session_id, audio_format):
|
||||
"""获取识别结果"""
|
||||
@@ -353,16 +512,6 @@ class ASRProvider(ASRProviderBase):
|
||||
pass
|
||||
self.forward_task = None
|
||||
self.is_processing = False
|
||||
|
||||
# 显式释放decoder资源
|
||||
if hasattr(self, 'decoder') and self.decoder is not None:
|
||||
try:
|
||||
del self.decoder
|
||||
self.decoder = None
|
||||
logger.bind(tag=TAG).debug("Xunfei decoder resources released")
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).debug(f"释放Xunfei decoder资源时出错: {e}")
|
||||
|
||||
# 清理所有连接的音频缓冲区
|
||||
if hasattr(self, "_connections"):
|
||||
for conn in self._connections.values():
|
||||
@@ -370,3 +519,5 @@ class ASRProvider(ASRProviderBase):
|
||||
conn.asr_audio_for_voiceprint = []
|
||||
if hasattr(conn, "asr_audio"):
|
||||
conn.asr_audio = []
|
||||
if hasattr(conn, "has_valid_voice"):
|
||||
conn.has_valid_voice = False
|
||||
|
||||
@@ -5,7 +5,6 @@ import os
|
||||
import yaml
|
||||
from config.config_loader import get_project_dir
|
||||
from config.manage_api_client import save_mem_local_short
|
||||
import asyncio
|
||||
from core.utils.util import check_model_key
|
||||
|
||||
|
||||
@@ -194,13 +193,7 @@ class MemoryProvider(MemoryProviderBase):
|
||||
max_tokens=2000,
|
||||
temperature=0.2,
|
||||
)
|
||||
# 使用异步版本，需要在事件循环中运行
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.create_task(save_mem_local_short(self.role_id, result))
|
||||
except RuntimeError:
|
||||
# 如果没有运行中的事件循环，创建一个新的
|
||||
asyncio.run(save_mem_local_short(self.role_id, result))
|
||||
save_mem_local_short(self.role_id, result)
|
||||
logger.bind(tag=TAG).info(f"Save memory successful - Role: {self.role_id}")
|
||||
|
||||
return self.short_memory
|
||||
|
||||
@@ -10,13 +10,10 @@ import concurrent.futures
|
||||
from contextlib import AsyncExitStack
|
||||
from typing import Optional, List, Dict, Any
|
||||
|
||||
from mcp import ClientSession, StdioServerParameters, Implementation
|
||||
from mcp.client.session import SamplingFnT, ElicitationFnT, ListRootsFnT, LoggingFnT, MessageHandlerFnT
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
from mcp.client.sse import sse_client
|
||||
from mcp.client.streamable_http import streamablehttp_client
|
||||
from mcp.shared.session import ProgressFnT
|
||||
|
||||
from config.logger import setup_logging
|
||||
from core.utils.util import sanitize_tool_name
|
||||
|
||||
@@ -44,25 +41,13 @@ class ServerMCPClient:
|
||||
self.tools_dict: Dict[str, Any] = {}
|
||||
self.name_mapping: Dict[str, str] = {}
|
||||
|
||||
async def initialize(self, read_timeout_seconds: timedelta | None = None,
|
||||
sampling_callback: SamplingFnT | None = None,
|
||||
elicitation_callback: ElicitationFnT | None = None,
|
||||
list_roots_callback: ListRootsFnT | None = None,
|
||||
logging_callback: LoggingFnT | None = None,
|
||||
message_handler: MessageHandlerFnT | None = None,
|
||||
client_info: Implementation | None = None):
|
||||
async def initialize(self):
|
||||
"""初始化MCP客户端连接"""
|
||||
if self._worker_task:
|
||||
return
|
||||
|
||||
self._worker_task = asyncio.create_task(
|
||||
self._worker(read_timeout_seconds=read_timeout_seconds,
|
||||
sampling_callback=sampling_callback,
|
||||
elicitation_callback=elicitation_callback,
|
||||
list_roots_callback=list_roots_callback,
|
||||
logging_callback=logging_callback,
|
||||
message_handler=message_handler,
|
||||
client_info=client_info), name="ServerMCPClientWorker"
|
||||
self._worker(), name="ServerMCPClientWorker"
|
||||
)
|
||||
await self._ready_evt.wait()
|
||||
|
||||
@@ -112,15 +97,12 @@ class ServerMCPClient:
|
||||
for name, tool in self.tools_dict.items()
|
||||
]
|
||||
|
||||
async def call_tool(self, name: str, arguments: dict, read_timeout_seconds: timedelta | None = None, progress_callback: ProgressFnT | None = None, *, meta: dict[str, Any] | None = None) -> Any:
|
||||
async def call_tool(self, name: str, args: dict) -> Any:
|
||||
"""调用指定工具
|
||||
|
||||
Args:
|
||||
name: 工具名称
|
||||
arguments: 工具参数
|
||||
read_timeout_seconds:
|
||||
progress_callback: 进度回调函数
|
||||
meta:
|
||||
args: 工具参数
|
||||
|
||||
Returns:
|
||||
Any: 工具执行结果
|
||||
@@ -133,7 +115,7 @@ class ServerMCPClient:
|
||||
|
||||
real_name = self.name_mapping.get(name, name)
|
||||
loop = self._worker_task.get_loop()
|
||||
coro = self.session.call_tool(real_name, arguments=arguments, read_timeout_seconds=read_timeout_seconds, progress_callback=progress_callback, meta=meta)
|
||||
coro = self.session.call_tool(real_name, args)
|
||||
|
||||
if loop is asyncio.get_running_loop():
|
||||
return await coro
|
||||
@@ -162,13 +144,7 @@ class ServerMCPClient:
|
||||
# 所有检查都通过，连接正常
|
||||
return True
|
||||
|
||||
async def _worker(self, read_timeout_seconds: timedelta | None = None,
|
||||
sampling_callback: SamplingFnT | None = None,
|
||||
elicitation_callback: ElicitationFnT | None = None,
|
||||
list_roots_callback: ListRootsFnT | None = None,
|
||||
logging_callback: LoggingFnT | None = None,
|
||||
message_handler: MessageHandlerFnT | None = None,
|
||||
client_info: Implementation | None = None):
|
||||
async def _worker(self):
|
||||
"""MCP客户端工作协程"""
|
||||
async with AsyncExitStack() as stack:
|
||||
try:
|
||||
@@ -232,13 +208,7 @@ class ServerMCPClient:
|
||||
ClientSession(
|
||||
read_stream=read_stream,
|
||||
write_stream=write_stream,
|
||||
read_timeout_seconds=read_timeout_seconds,
|
||||
sampling_callback=sampling_callback,
|
||||
elicitation_callback=elicitation_callback,
|
||||
list_roots_callback=list_roots_callback,
|
||||
logging_callback=logging_callback,
|
||||
message_handler=message_handler,
|
||||
client_info=client_info
|
||||
read_timeout_seconds=timedelta(seconds=15),
|
||||
)
|
||||
)
|
||||
await self.session.initialize()
|
||||
|
||||
@@ -3,14 +3,7 @@
|
||||
import asyncio
|
||||
import os
|
||||
import json
|
||||
from datetime import timedelta
|
||||
from typing import Dict, Any, List
|
||||
|
||||
from mcp import Implementation
|
||||
from mcp.client.session import SamplingFnT, ElicitationFnT, ListRootsFnT, LoggingFnT, MessageHandlerFnT
|
||||
from mcp.shared.session import ProgressFnT
|
||||
from mcp.types import LoggingMessageNotificationParams
|
||||
|
||||
from config.config_loader import get_project_dir
|
||||
from config.logger import setup_logging
|
||||
from .mcp_client import ServerMCPClient
|
||||
@@ -63,7 +56,7 @@ class ServerMCPManager:
|
||||
# 初始化服务端MCP客户端
|
||||
logger.bind(tag=TAG).info(f"初始化服务端MCP客户端: {name}")
|
||||
client = ServerMCPClient(srv_config)
|
||||
await client.initialize(logging_callback=self.logging_callback)
|
||||
await client.initialize()
|
||||
self.clients[name] = client
|
||||
client_tools = client.get_available_tools()
|
||||
self.tools.extend(client_tools)
|
||||
@@ -116,7 +109,7 @@ class ServerMCPManager:
|
||||
# 带重试机制的工具调用
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
return await target_client.call_tool(tool_name, arguments, progress_callback=self.progress_callback)
|
||||
return await target_client.call_tool(tool_name, arguments)
|
||||
except Exception as e:
|
||||
# 最后一次尝试失败时直接抛出异常
|
||||
if attempt == max_retries - 1:
|
||||
@@ -138,7 +131,7 @@ class ServerMCPManager:
|
||||
config = self.load_config()
|
||||
if client_name in config:
|
||||
client = ServerMCPClient(config[client_name])
|
||||
await client.initialize(logging_callback=self.logging_callback)
|
||||
await client.initialize()
|
||||
self.clients[client_name] = client
|
||||
target_client = client
|
||||
logger.bind(tag=TAG).info(
|
||||
@@ -166,11 +159,3 @@ class ServerMCPManager:
|
||||
except (asyncio.TimeoutError, Exception) as e:
|
||||
logger.bind(tag=TAG).error(f"关闭服务端MCP客户端 {name} 时出错: {e}")
|
||||
self.clients.clear()
|
||||
|
||||
# 可选回调方法
|
||||
|
||||
async def logging_callback(self, params: LoggingMessageNotificationParams):
|
||||
logger.bind(tag=TAG).info(f"[Server Log - {params.level.upper()}] {params.data}")
|
||||
|
||||
async def progress_callback(self, progress: float, total: float | None, message: str | None) -> None:
|
||||
logger.bind(tag=TAG).info(f"[Progress {progress}/{total}]: {message}")
|
||||
@@ -36,18 +36,7 @@ class VADProvider(VADProviderBase):
|
||||
# 至少要多少帧才算有语音
|
||||
self.frame_window_threshold = 3
|
||||
|
||||
def __del__(self):
|
||||
if hasattr(self, 'decoder') and self.decoder is not None:
|
||||
try:
|
||||
del self.decoder
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def is_vad(self, conn, opus_packet):
|
||||
# 手动模式：直接返回True，不进行实时VAD检测，所有音频都缓存
|
||||
if conn.client_listen_mode == "manual":
|
||||
return True
|
||||
|
||||
try:
|
||||
pcm_frame = self.decoder.decode(opus_packet, 960)
|
||||
conn.client_audio_buffer.extend(pcm_frame) # 将新数据加入缓冲区
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
import time
|
||||
import asyncio
|
||||
from config.logger import setup_logging
|
||||
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
|
||||
|
||||
class AudioRateController:
|
||||
"""
|
||||
音频速率控制器 - 按照60ms帧时长精确控制音频发送
|
||||
解决高并发下的时间累积误差问题
|
||||
"""
|
||||
|
||||
def __init__(self, frame_duration=60):
|
||||
"""
|
||||
Args:
|
||||
frame_duration: 单个音频帧时长（毫秒），默认60ms
|
||||
"""
|
||||
self.frame_duration = frame_duration
|
||||
self.queue = []
|
||||
self.play_position = 0 # 虚拟播放位置（毫秒）
|
||||
self.start_timestamp = None # 开始时间戳（只读，不修改）
|
||||
self.pending_send_task = None
|
||||
self.logger = logger
|
||||
self.queue_empty_event = asyncio.Event() # 队列清空事件
|
||||
self.queue_empty_event.set() # 初始为空状态
|
||||
|
||||
def reset(self):
|
||||
"""重置控制器状态"""
|
||||
if self.pending_send_task and not self.pending_send_task.done():
|
||||
self.pending_send_task.cancel()
|
||||
# 取消任务后，任务会在下次事件循环时清理，无需阻塞等待
|
||||
|
||||
self.queue.clear()
|
||||
self.play_position = 0
|
||||
self.start_timestamp = time.time()
|
||||
self.queue_empty_event.set() # 队列已清空
|
||||
|
||||
def add_audio(self, opus_packet):
|
||||
"""添加音频包到队列"""
|
||||
self.queue.append(("audio", opus_packet))
|
||||
self.queue_empty_event.clear() # 队列非空，清除事件
|
||||
|
||||
def add_message(self, message_callback):
|
||||
"""
|
||||
添加消息到队列（立即发送，不占用播放时间）
|
||||
|
||||
Args:
|
||||
message_callback: 消息发送回调函数 async def()
|
||||
"""
|
||||
self.queue.append(("message", message_callback))
|
||||
self.queue_empty_event.clear() # 队列非空，清除事件
|
||||
|
||||
def _get_elapsed_ms(self):
|
||||
"""获取已经过的时间（毫秒）"""
|
||||
if self.start_timestamp is None:
|
||||
return 0
|
||||
return (time.time() - self.start_timestamp) * 1000
|
||||
|
||||
async def check_queue(self, send_audio_callback):
|
||||
"""
|
||||
检查队列并按时发送音频/消息
|
||||
|
||||
Args:
|
||||
send_audio_callback: 发送音频的回调函数 async def(opus_packet)
|
||||
"""
|
||||
if self.start_timestamp is None:
|
||||
self.start_timestamp = time.time()
|
||||
|
||||
while self.queue:
|
||||
item = self.queue[0]
|
||||
item_type = item[0]
|
||||
|
||||
if item_type == "message":
|
||||
# 消息类型：立即发送，不占用播放时间
|
||||
_, message_callback = item
|
||||
self.queue.pop(0)
|
||||
try:
|
||||
await message_callback()
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"发送消息失败: {e}")
|
||||
raise
|
||||
|
||||
elif item_type == "audio":
|
||||
_, opus_packet = item
|
||||
|
||||
# 循环等待直到时间到达
|
||||
while True:
|
||||
# 计算时间差
|
||||
elapsed_ms = self._get_elapsed_ms()
|
||||
output_ms = self.play_position
|
||||
|
||||
if elapsed_ms < output_ms:
|
||||
# 还不到发送时间，计算等待时长
|
||||
wait_ms = output_ms - elapsed_ms
|
||||
|
||||
# 等待后继续检查（允许被中断）
|
||||
try:
|
||||
await asyncio.sleep(wait_ms / 1000)
|
||||
except asyncio.CancelledError:
|
||||
self.logger.bind(tag=TAG).debug("音频发送任务被取消")
|
||||
raise
|
||||
# 等待结束后重新检查时间（循环回到 while True）
|
||||
else:
|
||||
# 时间已到，跳出等待循环
|
||||
break
|
||||
|
||||
# 时间已到，从队列移除并发送
|
||||
self.queue.pop(0)
|
||||
self.play_position += self.frame_duration
|
||||
|
||||
try:
|
||||
await send_audio_callback(opus_packet)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"发送音频失败: {e}")
|
||||
raise
|
||||
|
||||
self.queue_empty_event.set()
|
||||
|
||||
def start_sending(self, send_audio_callback):
|
||||
"""
|
||||
启动异步发送任务
|
||||
|
||||
Args:
|
||||
send_audio_callback: 发送音频的回调函数
|
||||
|
||||
Returns:
|
||||
asyncio.Task: 发送任务
|
||||
"""
|
||||
|
||||
async def _send_loop():
|
||||
try:
|
||||
while True:
|
||||
await self.check_queue(send_audio_callback)
|
||||
# 如果队列空了，短暂等待后再检查（避免 busy loop）
|
||||
await asyncio.sleep(0.01)
|
||||
except asyncio.CancelledError:
|
||||
self.logger.bind(tag=TAG).debug("音频发送循环已停止")
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"音频发送循环异常: {e}")
|
||||
|
||||
self.pending_send_task = asyncio.create_task(_send_loop())
|
||||
return self.pending_send_task
|
||||
|
||||
def stop_sending(self):
|
||||
"""停止发送任务"""
|
||||
if self.pending_send_task and not self.pending_send_task.done():
|
||||
self.pending_send_task.cancel()
|
||||
self.logger.bind(tag=TAG).debug("已取消音频发送任务")
|
||||
@@ -19,7 +19,6 @@ class CacheType(Enum):
|
||||
CONFIG = "config"
|
||||
DEVICE_PROMPT = "device_prompt"
|
||||
VOICEPRINT_HEALTH = "voiceprint_health" # 声纹识别健康检查
|
||||
AUDIO_DATA = "audio_data" # 音频数据缓存
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -59,8 +58,5 @@ class CacheConfig:
|
||||
CacheType.VOICEPRINT_HEALTH: cls(
|
||||
strategy=CacheStrategy.TTL, ttl=600, max_size=100 # 10分钟过期
|
||||
),
|
||||
CacheType.AUDIO_DATA: cls(
|
||||
strategy=CacheStrategy.TTL, ttl=600, max_size=100 # 10分钟过期
|
||||
),
|
||||
}
|
||||
return configs.get(cache_type, cls())
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
import httpx
|
||||
from typing import Dict, Any, List
|
||||
from config.logger import setup_logging
|
||||
|
||||
TAG = __name__
|
||||
|
||||
class ContextDataProvider:
|
||||
"""数据上下文填充，负责从配置的API获取数据"""
|
||||
|
||||
def __init__(self, config: Dict[str, Any], logger=None):
|
||||
self.config = config
|
||||
self.logger = logger or setup_logging()
|
||||
self.context_data = ""
|
||||
|
||||
def fetch_all(self, device_id: str) -> str:
|
||||
"""获取所有配置的上下文数据"""
|
||||
context_providers = self.config.get("context_providers", [])
|
||||
if not context_providers:
|
||||
return ""
|
||||
|
||||
formatted_lines = []
|
||||
for provider in context_providers:
|
||||
url = provider.get("url")
|
||||
headers = provider.get("headers", {})
|
||||
|
||||
if not url:
|
||||
continue
|
||||
|
||||
try:
|
||||
headers = headers.copy() if isinstance(headers, dict) else {}
|
||||
# 将 device_id 添加到请求头
|
||||
headers["device-id"] = device_id
|
||||
|
||||
# 发送请求
|
||||
response = httpx.get(url, headers=headers, timeout=3)
|
||||
|
||||
if response.status_code == 200:
|
||||
result = response.json()
|
||||
if isinstance(result, dict):
|
||||
if result.get("code") == 0:
|
||||
data = result.get("data")
|
||||
# 格式化数据
|
||||
if isinstance(data, dict):
|
||||
for k, v in data.items():
|
||||
formatted_lines.append(f"- **{k}：** {v}")
|
||||
elif isinstance(data, list):
|
||||
for item in data:
|
||||
formatted_lines.append(f"- {item}")
|
||||
else:
|
||||
formatted_lines.append(f"- {data}")
|
||||
else:
|
||||
self.logger.bind(tag=TAG).warning(f"API {url} 返回错误码: {result.get('msg')}")
|
||||
else:
|
||||
self.logger.bind(tag=TAG).warning(f"API {url} 返回的不是JSON字典")
|
||||
else:
|
||||
self.logger.bind(tag=TAG).warning(f"API {url} 请求失败: {response.status_code}")
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"获取上下文数据 {url} 失败: {e}")
|
||||
|
||||
# 将所有格式化后的行拼接成一个字符串
|
||||
self.context_data = "\n".join(formatted_lines)
|
||||
if self.context_data:
|
||||
self.logger.bind(tag=TAG).debug(f"已注入动态上下文数据:\n{self.context_data}")
|
||||
return self.context_data
|
||||
@@ -1,122 +0,0 @@
|
||||
"""
|
||||
全局GC管理模块
|
||||
定期执行垃圾回收，避免频繁触发GC导致的GIL锁问题
|
||||
"""
|
||||
|
||||
import gc
|
||||
import asyncio
|
||||
import threading
|
||||
from config.logger import setup_logging
|
||||
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
|
||||
|
||||
class GlobalGCManager:
|
||||
"""全局垃圾回收管理器"""
|
||||
|
||||
def __init__(self, interval_seconds=300):
|
||||
"""
|
||||
初始化GC管理器
|
||||
|
||||
Args:
|
||||
interval_seconds: GC执行间隔（秒），默认300秒（5分钟）
|
||||
"""
|
||||
self.interval_seconds = interval_seconds
|
||||
self._task = None
|
||||
self._stop_event = asyncio.Event()
|
||||
self._lock = threading.Lock()
|
||||
|
||||
async def start(self):
|
||||
"""启动定时GC任务"""
|
||||
if self._task is not None:
|
||||
logger.bind(tag=TAG).warning("GC管理器已经在运行")
|
||||
return
|
||||
|
||||
logger.bind(tag=TAG).info(f"启动全局GC管理器，间隔{self.interval_seconds}秒")
|
||||
self._stop_event.clear()
|
||||
self._task = asyncio.create_task(self._gc_loop())
|
||||
|
||||
async def stop(self):
|
||||
"""停止定时GC任务"""
|
||||
if self._task is None:
|
||||
return
|
||||
|
||||
logger.bind(tag=TAG).info("停止全局GC管理器")
|
||||
self._stop_event.set()
|
||||
|
||||
if self._task and not self._task.done():
|
||||
self._task.cancel()
|
||||
try:
|
||||
await self._task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
self._task = None
|
||||
|
||||
async def _gc_loop(self):
|
||||
"""GC循环任务"""
|
||||
try:
|
||||
while not self._stop_event.is_set():
|
||||
# 等待指定间隔
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
self._stop_event.wait(), timeout=self.interval_seconds
|
||||
)
|
||||
# 如果stop_event被设置，退出循环
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
# 超时表示到了执行GC的时间
|
||||
pass
|
||||
|
||||
# 执行GC
|
||||
await self._run_gc()
|
||||
|
||||
except asyncio.CancelledError:
|
||||
logger.bind(tag=TAG).info("GC循环任务被取消")
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"GC循环任务异常: {e}")
|
||||
finally:
|
||||
logger.bind(tag=TAG).info("GC循环任务已退出")
|
||||
|
||||
async def _run_gc(self):
|
||||
"""执行垃圾回收"""
|
||||
try:
|
||||
# 在线程池中执行GC，避免阻塞事件循环
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
def do_gc():
|
||||
with self._lock:
|
||||
before = len(gc.get_objects())
|
||||
collected = gc.collect()
|
||||
after = len(gc.get_objects())
|
||||
return before, collected, after
|
||||
|
||||
before, collected, after = await loop.run_in_executor(None, do_gc)
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"全局GC执行完成 - 回收对象: {collected}, "
|
||||
f"对象数量: {before} -> {after}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"执行GC时出错: {e}")
|
||||
|
||||
|
||||
# 全局单例
|
||||
_gc_manager_instance = None
|
||||
|
||||
|
||||
def get_gc_manager(interval_seconds=300):
|
||||
"""
|
||||
获取全局GC管理器实例（单例模式）
|
||||
|
||||
Args:
|
||||
interval_seconds: GC执行间隔（秒），默认300秒（5分钟）
|
||||
|
||||
Returns:
|
||||
GlobalGCManager实例
|
||||
"""
|
||||
global _gc_manager_instance
|
||||
if _gc_manager_instance is None:
|
||||
_gc_manager_instance = GlobalGCManager(interval_seconds)
|
||||
return _gc_manager_instance
|
||||
@@ -102,9 +102,6 @@ class OpusEncoderUtils:
|
||||
def _encode(self, frame: np.ndarray) -> Optional[bytes]:
|
||||
"""编码一帧音频数据"""
|
||||
try:
|
||||
# 编码器已释放，跳过编码
|
||||
if not hasattr(self, 'encoder') or self.encoder is None:
|
||||
return None
|
||||
# 将numpy数组转换为bytes
|
||||
frame_bytes = frame.tobytes()
|
||||
# opuslib要求输入字节数必须是channels*2的倍数
|
||||
@@ -131,9 +128,5 @@ class OpusEncoderUtils:
|
||||
|
||||
def close(self):
|
||||
"""关闭编码器并释放资源"""
|
||||
if hasattr(self, 'encoder') and self.encoder:
|
||||
try:
|
||||
del self.encoder
|
||||
self.encoder = None
|
||||
except Exception as e:
|
||||
logging.error(f"Error releasing Opus encoder: {e}")
|
||||
# opuslib没有明确的关闭方法，Python的垃圾回收会处理
|
||||
pass
|
||||
@@ -4,6 +4,7 @@
|
||||
"""
|
||||
|
||||
import os
|
||||
import cnlunar
|
||||
from typing import Dict, Any
|
||||
from config.logger import setup_logging
|
||||
from jinja2 import Template
|
||||
@@ -59,11 +60,6 @@ class PromptManager:
|
||||
|
||||
self.cache_manager = cache_manager
|
||||
self.CacheType = CacheType
|
||||
|
||||
# 初始化上下文源
|
||||
from core.utils.context_provider import ContextDataProvider
|
||||
self.context_provider = ContextDataProvider(config, self.logger)
|
||||
self.context_data = {}
|
||||
|
||||
self._load_base_template()
|
||||
|
||||
@@ -188,14 +184,6 @@ class PromptManager:
|
||||
local_address = self._get_location_info(client_ip)
|
||||
# 获取天气信息（使用全局缓存）
|
||||
self._get_weather_info(conn, local_address)
|
||||
|
||||
# 获取配置的上下文数据
|
||||
if hasattr(conn, "device_id") and conn.device_id:
|
||||
if self.base_prompt_template and "dynamic_context" in self.base_prompt_template:
|
||||
self.context_data = self.context_provider.fetch_all(conn.device_id)
|
||||
else:
|
||||
self.context_data = ""
|
||||
|
||||
self.logger.bind(tag=TAG).debug(f"上下文信息更新完成")
|
||||
|
||||
except Exception as e:
|
||||
@@ -242,7 +230,6 @@ class PromptManager:
|
||||
emojiList=EMOJI_List,
|
||||
device_id=device_id,
|
||||
client_ip=client_ip,
|
||||
dynamic_context=self.context_data,
|
||||
*args,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -4,7 +4,6 @@ import json
|
||||
import copy
|
||||
import wave
|
||||
import socket
|
||||
import asyncio
|
||||
import requests
|
||||
import subprocess
|
||||
import numpy as np
|
||||
@@ -269,82 +268,56 @@ def audio_to_data_stream(
|
||||
pcm_to_data_stream(raw_data, is_opus, callback)
|
||||
|
||||
|
||||
async def audio_to_data(
|
||||
audio_file_path: str, is_opus: bool = True, use_cache: bool = True
|
||||
) -> list[bytes]:
|
||||
def audio_to_data(audio_file_path: str, is_opus: bool = True) -> list[bytes]:
|
||||
"""
|
||||
将音频文件转换为Opus/PCM编码的帧列表
|
||||
Args:
|
||||
audio_file_path: 音频文件路径
|
||||
is_opus: 是否进行Opus编码
|
||||
use_cache: 是否使用缓存
|
||||
"""
|
||||
from core.utils.cache.manager import cache_manager
|
||||
from core.utils.cache.config import CacheType
|
||||
# 获取文件后缀名
|
||||
file_type = os.path.splitext(audio_file_path)[1]
|
||||
if file_type:
|
||||
file_type = file_type.lstrip(".")
|
||||
# 读取音频文件，-nostdin 参数：不要从标准输入读取数据，否则FFmpeg会阻塞
|
||||
audio = AudioSegment.from_file(
|
||||
audio_file_path, format=file_type, parameters=["-nostdin"]
|
||||
)
|
||||
|
||||
# 生成缓存键，包含文件路径和编码类型
|
||||
cache_key = f"{audio_file_path}:{is_opus}"
|
||||
# 转换为单声道/16kHz采样率/16位小端编码（确保与编码器匹配）
|
||||
audio = audio.set_channels(1).set_frame_rate(16000).set_sample_width(2)
|
||||
|
||||
# 尝试从缓存获取结果
|
||||
if use_cache:
|
||||
cached_result = cache_manager.get(CacheType.AUDIO_DATA, cache_key)
|
||||
if cached_result is not None:
|
||||
return cached_result
|
||||
# 获取原始PCM数据（16位小端）
|
||||
raw_data = audio.raw_data
|
||||
|
||||
def _sync_audio_to_data():
|
||||
# 获取文件后缀名
|
||||
file_type = os.path.splitext(audio_file_path)[1]
|
||||
if file_type:
|
||||
file_type = file_type.lstrip(".")
|
||||
# 读取音频文件，-nostdin 参数：不要从标准输入读取数据，否则FFmpeg会阻塞
|
||||
audio = AudioSegment.from_file(
|
||||
audio_file_path, format=file_type, parameters=["-nostdin"]
|
||||
)
|
||||
# 初始化Opus编码器
|
||||
encoder = opuslib_next.Encoder(16000, 1, opuslib_next.APPLICATION_AUDIO)
|
||||
|
||||
# 转换为单声道/16kHz采样率/16位小端编码（确保与编码器匹配）
|
||||
audio = audio.set_channels(1).set_frame_rate(16000).set_sample_width(2)
|
||||
# 编码参数
|
||||
frame_duration = 60 # 60ms per frame
|
||||
frame_size = int(16000 * frame_duration / 1000) # 960 samples/frame
|
||||
|
||||
# 获取原始PCM数据（16位小端）
|
||||
raw_data = audio.raw_data
|
||||
datas = []
|
||||
# 按帧处理所有音频数据（包括最后一帧可能补零）
|
||||
for i in range(0, len(raw_data), frame_size * 2): # 16bit=2bytes/sample
|
||||
# 获取当前帧的二进制数据
|
||||
chunk = raw_data[i : i + frame_size * 2]
|
||||
|
||||
# 初始化Opus编码器
|
||||
encoder = opuslib_next.Encoder(16000, 1, opuslib_next.APPLICATION_AUDIO)
|
||||
# 如果最后一帧不足，补零
|
||||
if len(chunk) < frame_size * 2:
|
||||
chunk += b"\x00" * (frame_size * 2 - len(chunk))
|
||||
|
||||
# 编码参数
|
||||
frame_duration = 60 # 60ms per frame
|
||||
frame_size = int(16000 * frame_duration / 1000) # 960 samples/frame
|
||||
if is_opus:
|
||||
# 转换为numpy数组处理
|
||||
np_frame = np.frombuffer(chunk, dtype=np.int16)
|
||||
# 编码Opus数据
|
||||
frame_data = encoder.encode(np_frame.tobytes(), frame_size)
|
||||
else:
|
||||
frame_data = chunk if isinstance(chunk, bytes) else bytes(chunk)
|
||||
|
||||
datas = []
|
||||
# 按帧处理所有音频数据（包括最后一帧可能补零）
|
||||
for i in range(0, len(raw_data), frame_size * 2): # 16bit=2bytes/sample
|
||||
# 获取当前帧的二进制数据
|
||||
chunk = raw_data[i : i + frame_size * 2]
|
||||
datas.append(frame_data)
|
||||
|
||||
# 如果最后一帧不足，补零
|
||||
if len(chunk) < frame_size * 2:
|
||||
chunk += b"\x00" * (frame_size * 2 - len(chunk))
|
||||
|
||||
if is_opus:
|
||||
# 转换为numpy数组处理
|
||||
np_frame = np.frombuffer(chunk, dtype=np.int16)
|
||||
# 编码Opus数据
|
||||
frame_data = encoder.encode(np_frame.tobytes(), frame_size)
|
||||
else:
|
||||
frame_data = chunk if isinstance(chunk, bytes) else bytes(chunk)
|
||||
|
||||
datas.append(frame_data)
|
||||
|
||||
return datas
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
# 在单独的线程中执行同步的音频处理操作
|
||||
result = await loop.run_in_executor(None, _sync_audio_to_data)
|
||||
|
||||
# 将结果存入缓存，使用配置中定义的TTL（10分钟）
|
||||
if use_cache:
|
||||
cache_manager.set(CacheType.AUDIO_DATA, cache_key, result)
|
||||
|
||||
return result
|
||||
return datas
|
||||
|
||||
|
||||
def audio_bytes_to_data_stream(
|
||||
@@ -399,33 +372,26 @@ def opus_datas_to_wav_bytes(opus_datas, sample_rate=16000, channels=1):
|
||||
将opus帧列表解码为wav字节流
|
||||
"""
|
||||
decoder = opuslib_next.Decoder(sample_rate, channels)
|
||||
try:
|
||||
pcm_datas = []
|
||||
pcm_datas = []
|
||||
|
||||
frame_duration = 60 # ms
|
||||
frame_size = int(sample_rate * frame_duration / 1000) # 960
|
||||
frame_duration = 60 # ms
|
||||
frame_size = int(sample_rate * frame_duration / 1000) # 960
|
||||
|
||||
for opus_frame in opus_datas:
|
||||
# 解码为PCM（返回bytes，2字节/采样点）
|
||||
pcm = decoder.decode(opus_frame, frame_size)
|
||||
pcm_datas.append(pcm)
|
||||
for opus_frame in opus_datas:
|
||||
# 解码为PCM（返回bytes，2字节/采样点）
|
||||
pcm = decoder.decode(opus_frame, frame_size)
|
||||
pcm_datas.append(pcm)
|
||||
|
||||
pcm_bytes = b"".join(pcm_datas)
|
||||
pcm_bytes = b"".join(pcm_datas)
|
||||
|
||||
# 写入wav字节流
|
||||
wav_buffer = BytesIO()
|
||||
with wave.open(wav_buffer, "wb") as wf:
|
||||
wf.setnchannels(channels)
|
||||
wf.setsampwidth(2) # 16bit
|
||||
wf.setframerate(sample_rate)
|
||||
wf.writeframes(pcm_bytes)
|
||||
return wav_buffer.getvalue()
|
||||
finally:
|
||||
if decoder is not None:
|
||||
try:
|
||||
del decoder
|
||||
except Exception:
|
||||
pass
|
||||
# 写入wav字节流
|
||||
wav_buffer = BytesIO()
|
||||
with wave.open(wav_buffer, "wb") as wf:
|
||||
wf.setnchannels(channels)
|
||||
wf.setsampwidth(2) # 16bit
|
||||
wf.setframerate(sample_rate)
|
||||
wf.writeframes(pcm_bytes)
|
||||
return wav_buffer.getvalue()
|
||||
|
||||
|
||||
def check_vad_update(before_config, new_config):
|
||||
|
||||
@@ -1,37 +1,10 @@
|
||||
import asyncio
|
||||
import logging
|
||||
import json
|
||||
|
||||
import websockets
|
||||
from config.logger import setup_logging
|
||||
|
||||
|
||||
class SuppressInvalidHandshakeFilter(logging.Filter):
|
||||
"""过滤掉无效握手错误日志（如HTTPS访问WS端口）"""
|
||||
|
||||
def filter(self, record):
|
||||
msg = record.getMessage()
|
||||
suppress_keywords = [
|
||||
"opening handshake failed",
|
||||
"did not receive a valid HTTP request",
|
||||
"connection closed while reading HTTP request",
|
||||
"line without CRLF",
|
||||
]
|
||||
return not any(keyword in msg for keyword in suppress_keywords)
|
||||
|
||||
|
||||
def _setup_websockets_logger():
|
||||
"""配置 websockets 相关的所有 logger，过滤无效握手错误"""
|
||||
filter_instance = SuppressInvalidHandshakeFilter()
|
||||
for logger_name in ["websockets", "websockets.server", "websockets.client"]:
|
||||
logger = logging.getLogger(logger_name)
|
||||
logger.addFilter(filter_instance)
|
||||
|
||||
|
||||
_setup_websockets_logger()
|
||||
|
||||
|
||||
from core.connection import ConnectionHandler
|
||||
from config.config_loader import get_config_from_api_async
|
||||
from config.config_loader import get_config_from_api
|
||||
from core.auth import AuthManager, AuthenticationError
|
||||
from core.utils.modules_initialize import initialize_modules
|
||||
from core.utils.util import check_vad_update, check_asr_update
|
||||
@@ -60,6 +33,8 @@ class WebSocketServer:
|
||||
self._intent = modules["intent"] if "intent" in modules else None
|
||||
self._memory = modules["memory"] if "memory" in modules else None
|
||||
|
||||
self.active_connections = set()
|
||||
|
||||
auth_config = self.config["server"].get("auth", {})
|
||||
self.auth_enable = auth_config.get("enabled", False)
|
||||
# 设备白名单
|
||||
@@ -123,11 +98,14 @@ class WebSocketServer:
|
||||
self._intent,
|
||||
self, # 传入server实例
|
||||
)
|
||||
self.active_connections.add(handler)
|
||||
try:
|
||||
await handler.handle_connection(websocket)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"处理连接时出错: {e}")
|
||||
finally:
|
||||
# 确保从活动连接集合中移除
|
||||
self.active_connections.discard(handler)
|
||||
# 强制关闭连接（如果还没有关闭的话）
|
||||
try:
|
||||
# 安全地检查WebSocket状态并关闭
|
||||
@@ -160,8 +138,8 @@ class WebSocketServer:
|
||||
"""
|
||||
try:
|
||||
async with self.config_lock:
|
||||
# 重新获取配置（使用异步版本）
|
||||
new_config = await get_config_from_api_async(self.config)
|
||||
# 重新获取配置
|
||||
new_config = get_config_from_api(self.config)
|
||||
if new_config is None:
|
||||
self.logger.bind(tag=TAG).error("获取新配置失败")
|
||||
return False
|
||||
|
||||
@@ -50,9 +50,7 @@ class BaseASRTester:
|
||||
raise NotImplementedError
|
||||
|
||||
def _calculate_result(self, service_name, latencies, test_count):
|
||||
"""计算测试结果（修复：正确处理None值，剔除失败测试）"""
|
||||
# 剔除None值（失败的测试）和无效延迟，只统计有效延迟
|
||||
valid_latencies = [l for l in latencies if l is not None and l > 0]
|
||||
valid_latencies = [l for l in latencies if l > 0]
|
||||
if valid_latencies:
|
||||
avg_latency = sum(valid_latencies) / len(valid_latencies)
|
||||
status = f"成功（{len(valid_latencies)}/{test_count}次有效）"
|
||||
@@ -66,45 +64,16 @@ class DoubaoStreamASRTester(BaseASRTester):
|
||||
def __init__(self):
|
||||
super().__init__("DoubaoStreamASR")
|
||||
|
||||
def _generate_header(
|
||||
self,
|
||||
version=0x01,
|
||||
message_type=0x01,
|
||||
message_type_specific_flags=0x00,
|
||||
serial_method=0x01,
|
||||
compression_type=0x01,
|
||||
reserved_data=0x00,
|
||||
extension_header: bytes = b"",
|
||||
):
|
||||
"""生成协议头（修复：使用正确的Header格式）"""
|
||||
def _generate_header(self):
|
||||
header = bytearray()
|
||||
header_size = int(len(extension_header) / 4) + 1
|
||||
header.append((version << 4) | header_size)
|
||||
header.append((message_type << 4) | message_type_specific_flags)
|
||||
header.append((serial_method << 4) | compression_type)
|
||||
header.append(reserved_data)
|
||||
header.extend(extension_header)
|
||||
header.append((0x01 << 4) | 0x01)
|
||||
header.append((0x01 << 4) | 0x00)
|
||||
header.append((0x01 << 4) | 0x01)
|
||||
header.append(0x00)
|
||||
return header
|
||||
|
||||
def _generate_audio_default_header(self):
|
||||
"""生成音频数据Header"""
|
||||
return self._generate_header(
|
||||
version=0x01,
|
||||
message_type=0x02,
|
||||
message_type_specific_flags=0x00, # 普通音频帧
|
||||
serial_method=0x01,
|
||||
compression_type=0x01,
|
||||
)
|
||||
|
||||
def _generate_last_audio_header(self):
|
||||
"""生成最后一帧音频的Header（标记音频结束）"""
|
||||
return self._generate_header(
|
||||
version=0x01,
|
||||
message_type=0x02,
|
||||
message_type_specific_flags=0x02, # 0x02表示这是最后一帧
|
||||
serial_method=0x01,
|
||||
compression_type=0x01,
|
||||
)
|
||||
return self._generate_header()
|
||||
|
||||
def _parse_response(self, res: bytes) -> dict:
|
||||
try:
|
||||
@@ -141,7 +110,6 @@ class DoubaoStreamASRTester(BaseASRTester):
|
||||
ws_url = "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel"
|
||||
appid = self.asr_config["appid"]
|
||||
access_token = self.asr_config["access_token"]
|
||||
cluster = self.asr_config.get("cluster", "volcengine_input_common")
|
||||
uid = self.asr_config.get("uid", "streaming_asr_service")
|
||||
|
||||
start_time = time.time()
|
||||
@@ -162,7 +130,7 @@ class DoubaoStreamASRTester(BaseASRTester):
|
||||
close_timeout=10
|
||||
) as ws:
|
||||
request_params = {
|
||||
"app": {"appid": appid, "cluster": cluster, "token": access_token},
|
||||
"app": {"appid": appid, "token": access_token},
|
||||
"user": {"uid": uid},
|
||||
"request": {
|
||||
"reqid": str(uuid.uuid4()),
|
||||
@@ -198,9 +166,8 @@ class DoubaoStreamASRTester(BaseASRTester):
|
||||
if audio_data.startswith(b'RIFF'):
|
||||
audio_data = audio_data[44:]
|
||||
|
||||
# 发送音频数据（使用最后一帧标记，告诉服务端音频已结束）
|
||||
payload = gzip.compress(audio_data)
|
||||
audio_request = bytearray(self._generate_last_audio_header()) # 修复：使用最后一帧Header
|
||||
audio_request = bytearray(self._generate_audio_default_header())
|
||||
audio_request.extend(len(payload).to_bytes(4, "big"))
|
||||
audio_request.extend(payload)
|
||||
await ws.send(audio_request)
|
||||
@@ -208,12 +175,11 @@ class DoubaoStreamASRTester(BaseASRTester):
|
||||
first_chunk = await ws.recv()
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[豆包ASR] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
await ws.close()
|
||||
|
||||
except Exception as e:
|
||||
print(f"[豆包ASR] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("豆包流式ASR", latencies, test_count)
|
||||
|
||||
@@ -223,12 +189,11 @@ class QwenASRFlashTester(BaseASRTester):
|
||||
super().__init__("Qwen3ASRFlash")
|
||||
|
||||
async def _test_single(self, audio_file_info):
|
||||
start_time = time.time()
|
||||
temp_file_path = None
|
||||
|
||||
try:
|
||||
audio_data = audio_file_info['data']
|
||||
|
||||
# 优化：将临时文件准备工作移到计时前，减少磁盘IO对性能测试的影响
|
||||
with tempfile.NamedTemporaryFile(suffix='.wav', delete=False) as f:
|
||||
temp_file_path = f.name
|
||||
|
||||
@@ -256,9 +221,6 @@ class QwenASRFlashTester(BaseASRTester):
|
||||
|
||||
dashscope.api_key = api_key
|
||||
|
||||
# 统一计时起点：在API调用前开始计时（但文件准备已完成）
|
||||
start_time = time.time()
|
||||
|
||||
response = dashscope.MultiModalConversation.call(
|
||||
model="qwen3-asr-flash",
|
||||
messages=messages,
|
||||
@@ -295,10 +257,10 @@ class QwenASRFlashTester(BaseASRTester):
|
||||
# print(f"\n[通义ASR] 开始第 {i+1} 次测试...")
|
||||
latency = await self._test_single(self.test_audio_files[0])
|
||||
latencies.append(latency)
|
||||
print(f"[通义ASR] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
# print(f"[通义ASR] 第{i+1}次成功 延迟: {latency:.3f}s")
|
||||
except Exception as e:
|
||||
# print(f"[通义ASR] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("通义千问ASR", latencies, test_count)
|
||||
|
||||
@@ -306,115 +268,134 @@ class QwenASRFlashTester(BaseASRTester):
|
||||
class XunfeiStreamASRTester(BaseASRTester):
|
||||
def __init__(self):
|
||||
super().__init__("XunfeiStreamASR")
|
||||
|
||||
|
||||
def _create_url(self):
|
||||
url = "wss://iat-api.xfyun.cn/v2/iat"
|
||||
"""生成讯飞ASR认证URL"""
|
||||
url = 'ws://iat.cn-huabei-1.xf-yun.com/v1'
|
||||
# 生成RFC1123格式的时间戳
|
||||
now = datetime.now()
|
||||
date = format_date_time(mktime(now.timetuple()))
|
||||
|
||||
signature_origin = f"host: iat-api.xfyun.cn\ndate: {date}\nGET /v2/iat HTTP/1.1"
|
||||
signature_sha = hmac.new(
|
||||
self.asr_config["api_secret"].encode('utf-8'),
|
||||
signature_origin.encode('utf-8'),
|
||||
hashlib.sha256
|
||||
).digest()
|
||||
signature_sha = base64.b64encode(signature_sha).decode()
|
||||
# 拼接字符串
|
||||
signature_origin = "host: " + "iat.cn-huabei-1.xf-yun.com" + "\n"
|
||||
signature_origin += "date: " + date + "\n"
|
||||
signature_origin += "GET " + "/v1 " + "HTTP/1.1"
|
||||
|
||||
authorization_origin = f'api_key="{self.asr_config["api_key"]}", algorithm="hmac-sha256", headers="host date request-line", signature="{signature_sha}"'
|
||||
authorization = base64.b64encode(authorization_origin.encode()).decode()
|
||||
# 进行hmac-sha256进行加密
|
||||
signature_sha = hmac.new(self.asr_config["api_secret"].encode('utf-8'), signature_origin.encode('utf-8'),
|
||||
digestmod=hashlib.sha256).digest()
|
||||
signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8')
|
||||
|
||||
v = {"authorization": authorization, "date": date, "host": "iat-api.xfyun.cn"}
|
||||
return url + "?" + parse.urlencode(v)
|
||||
authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % (
|
||||
self.asr_config["api_key"], "hmac-sha256", "host date request-line", signature_sha)
|
||||
authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8')
|
||||
|
||||
async def test(self, test_count: int = 5):
|
||||
# 将请求的鉴权参数组合为字典
|
||||
v = {
|
||||
"authorization": authorization,
|
||||
"date": date,
|
||||
"host": "iat.cn-huabei-1.xf-yun.com"
|
||||
}
|
||||
|
||||
# 拼接鉴权参数，生成url
|
||||
url = url + '?' + parse.urlencode(v)
|
||||
return url
|
||||
|
||||
async def test(self, test_count=5):
|
||||
if not self.test_audio_files:
|
||||
return {"name": "讯飞流式ASR", "latency": 0, "status": "失败: 未找到测试音频"}
|
||||
if not self.asr_config:
|
||||
return {"name": "讯飞流式ASR", "latency": 0, "status": "失败: 未配置"}
|
||||
|
||||
required = ["app_id", "api_key", "api_secret"]
|
||||
for k in required:
|
||||
if k not in self.asr_config:
|
||||
return {"name": "讯飞流式ASR", "latency": 0, "status": f"失败: 缺少配置 {k}"}
|
||||
|
||||
|
||||
# 检查必要的配置参数
|
||||
required_keys = ["app_id", "api_key", "api_secret"]
|
||||
for key in required_keys:
|
||||
if key not in self.asr_config:
|
||||
return {"name": "讯飞流式ASR", "latency": 0, "status": f"失败: 缺少配置项 {key}"}
|
||||
|
||||
latencies = []
|
||||
frame_size = 1280
|
||||
audio_raw = self.test_audio_files[0]['data']
|
||||
if audio_raw.startswith(b'RIFF'):
|
||||
audio_raw = audio_raw[44:]
|
||||
|
||||
STATUS_FIRST_FRAME = 0
|
||||
|
||||
for i in range(test_count):
|
||||
try:
|
||||
start_time = time.time()
|
||||
# 生成认证URL
|
||||
ws_url = self._create_url()
|
||||
|
||||
|
||||
# 获取音频数据
|
||||
audio_data = self.test_audio_files[0]['data']
|
||||
if audio_data.startswith(b'RIFF'):
|
||||
audio_data = audio_data[44:] # 跳过WAV文件头
|
||||
|
||||
# 识别参数
|
||||
iat_params = {
|
||||
"domain": self.asr_config.get("domain", "slm"),
|
||||
"language": self.asr_config.get("language", "zh_cn"),
|
||||
"accent": self.asr_config.get("accent", "mandarin"),
|
||||
"dwa": self.asr_config.get("dwa", "wpgs"),
|
||||
"result": {
|
||||
"encoding": "utf8",
|
||||
"compress": "raw",
|
||||
"format": "plain"
|
||||
}
|
||||
}
|
||||
|
||||
# 准备首帧数据
|
||||
first_frame_data = {
|
||||
"header": {
|
||||
"status": STATUS_FIRST_FRAME,
|
||||
"app_id": self.asr_config["app_id"]
|
||||
},
|
||||
"parameter": {
|
||||
"iat": iat_params
|
||||
},
|
||||
"payload": {
|
||||
"audio": {
|
||||
"audio": base64.b64encode(audio_data[:960]).decode('utf-8'),
|
||||
"sample_rate": 16000,
|
||||
"encoding": "raw"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# 启动连接并测量时间
|
||||
start_time = time.time()
|
||||
|
||||
async with websockets.connect(
|
||||
ws_url,
|
||||
additional_headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"},
|
||||
max_size=1 << 30,
|
||||
max_size=1000000000,
|
||||
ping_interval=None,
|
||||
ping_timeout=None,
|
||||
close_timeout=30,
|
||||
) as ws:
|
||||
|
||||
# 第一帧：移除 punc 字段，避免未知参数错误
|
||||
await ws.send(json.dumps({
|
||||
"common": {"app_id": self.asr_config["app_id"]},
|
||||
"business": {
|
||||
"domain": "iat",
|
||||
"language": "zh_cn",
|
||||
"accent": "mandarin",
|
||||
"dwa": "wpgs",
|
||||
"vad_eos": 5000
|
||||
# 已移除 "punc": True
|
||||
},
|
||||
"data": {
|
||||
"status": 0,
|
||||
"format": "audio/L16;rate=16000",
|
||||
"encoding": "raw",
|
||||
"audio": base64.b64encode(audio_raw[:frame_size]).decode()
|
||||
}
|
||||
}, ensure_ascii=False))
|
||||
|
||||
# 后续所有帧
|
||||
pos = frame_size
|
||||
while pos < len(audio_raw):
|
||||
chunk = audio_raw[pos:pos + frame_size]
|
||||
status = 2 if (pos + frame_size >= len(audio_raw)) else 1
|
||||
await ws.send(json.dumps({
|
||||
"data": {
|
||||
"status": status,
|
||||
"format": "audio/L16;rate=16000",
|
||||
"encoding": "raw",
|
||||
"audio": base64.b64encode(chunk).decode()
|
||||
}
|
||||
}, ensure_ascii=False))
|
||||
if status == 2:
|
||||
# 发送首帧数据
|
||||
await ws.send(json.dumps(first_frame_data, ensure_ascii=False))
|
||||
print(f"[讯飞ASR] 第{i+1}次测试：已发送首帧，等待响应...")
|
||||
|
||||
# 直接等待第一个响应并计算延迟
|
||||
# 参考豆包和通义千问的实现方式，简化逻辑
|
||||
response_received = False
|
||||
while not response_received:
|
||||
try:
|
||||
# 设置较大的超时时间
|
||||
response = await asyncio.wait_for(ws.recv(), timeout=30.0)
|
||||
|
||||
# 收到响应立即计算延迟，不管内容是什么
|
||||
# 这样可以准确测量首包到达时间
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
response_received = True
|
||||
|
||||
print(f"[讯飞ASR] 第{i+1}次测试：收到首包响应，延迟: {latency:.3f}s")
|
||||
break
|
||||
pos += frame_size
|
||||
|
||||
# 接收首词
|
||||
first_token = True
|
||||
async for message in ws:
|
||||
data = json.loads(message)
|
||||
if data.get("code") != 0:
|
||||
raise Exception(f"讯飞错误: {data.get('message')}")
|
||||
|
||||
ws_result = data.get("data", {}).get("result", {}).get("ws")
|
||||
if ws_result:
|
||||
text = "".join(cw.get("w", "") for seg in ws_result for cw in seg.get("cw", []))
|
||||
if text.strip() and first_token:
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[讯飞ASR] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
first_token = False
|
||||
break
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
print(f"[讯飞ASR] 第{i+1}次测试：响应超时")
|
||||
raise Exception("获取响应超时")
|
||||
except Exception as e:
|
||||
print(f"[讯飞ASR] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("讯飞流式ASR", latencies, test_count)
|
||||
|
||||
class ASRPerformanceSuite:
|
||||
def __init__(self):
|
||||
self.testers = []
|
||||
@@ -457,9 +438,8 @@ class ASRPerformanceSuite:
|
||||
|
||||
print(tabulate(table_data, headers=["ASR服务", "首词延迟", "状态"], tablefmt="grid"))
|
||||
print("\n测试说明：")
|
||||
print("- 计时起点: 建立连接前（包含握手、发送音频、接收首个识别结果全流程）")
|
||||
print("- 通义千问优化: 临时文件准备在计时前完成，减少磁盘IO对测试的影响")
|
||||
print("- 错误处理: 失败的测试不计入平均值，只统计成功测试的延迟")
|
||||
print("- 测量从发送请求到接收第一个有效识别文本的时间")
|
||||
print("- 超时控制: DashScope 默认超时，豆包 WebSocket 超时10秒")
|
||||
print("- 排序规则: 成功的按延迟升序，失败的排在后面")
|
||||
|
||||
async def run(self, test_count=5):
|
||||
|
||||
@@ -35,12 +35,11 @@ class StreamTTSPerformanceTester:
|
||||
host = tts_config["host"]
|
||||
ws_url = f"wss://{host}/ws/v1"
|
||||
|
||||
# 统一计时起点：在建立连接前开始计时
|
||||
start_time = time.time()
|
||||
async with websockets.connect(ws_url, extra_headers={"X-NLS-Token": token}) as ws:
|
||||
task_id = str(uuid.uuid4())
|
||||
message_id = str(uuid.uuid4())
|
||||
|
||||
|
||||
start_request = {
|
||||
"header": {
|
||||
"message_id": message_id,
|
||||
@@ -56,15 +55,14 @@ class StreamTTSPerformanceTester:
|
||||
"volume": 50,
|
||||
"speech_rate": 0,
|
||||
"pitch_rate": 0,
|
||||
"enable_subtitle": True,
|
||||
}
|
||||
}
|
||||
await ws.send(json.dumps(start_request))
|
||||
|
||||
|
||||
start_response = json.loads(await ws.recv())
|
||||
if start_response["header"]["name"] != "SynthesisStarted":
|
||||
raise Exception("启动合成失败")
|
||||
|
||||
|
||||
run_request = {
|
||||
"header": {
|
||||
"message_id": str(uuid.uuid4()),
|
||||
@@ -76,142 +74,23 @@ class StreamTTSPerformanceTester:
|
||||
"payload": {"text": text}
|
||||
}
|
||||
await ws.send(json.dumps(run_request))
|
||||
|
||||
|
||||
while True:
|
||||
response = await ws.recv()
|
||||
if isinstance(response, bytes):
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[阿里云TTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
break
|
||||
elif isinstance(response, str):
|
||||
data = json.loads(response)
|
||||
if data["header"]["name"] == "TaskFailed":
|
||||
raise Exception(f"合成失败: {data['payload']['error_info']}")
|
||||
|
||||
|
||||
except Exception as e:
|
||||
print(f"[阿里云TTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("阿里云TTS", latencies, test_count)
|
||||
|
||||
async def test_alibl_tts(self, text=None, test_count=5):
|
||||
"""测试阿里云百炼CosyVoice流式TTS首词延迟"""
|
||||
text = text or self.test_texts[0]
|
||||
latencies = []
|
||||
|
||||
for i in range(test_count):
|
||||
try:
|
||||
tts_config = self.config["TTS"]["AliBLTTS"]
|
||||
api_key = tts_config["api_key"]
|
||||
model = tts_config.get("model", "cosyvoice-v2")
|
||||
voice = tts_config.get("voice", "longxiaochun_v2")
|
||||
format_type = tts_config.get("format", "pcm")
|
||||
sample_rate = int(tts_config.get("sample_rate", "24000"))
|
||||
|
||||
ws_url = "wss://dashscope.aliyuncs.com/api-ws/v1/inference/"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"X-DashScope-DataInspection": "enable",
|
||||
}
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
async with websockets.connect(
|
||||
ws_url,
|
||||
additional_headers=headers,
|
||||
ping_interval=30,
|
||||
ping_timeout=10,
|
||||
close_timeout=10,
|
||||
max_size=10 * 1024 * 1024,
|
||||
) as ws:
|
||||
session_id = uuid.uuid4().hex
|
||||
|
||||
# 1. 发送 run-task（启动任务）
|
||||
run_task_message = {
|
||||
"header": {
|
||||
"action": "run-task",
|
||||
"task_id": session_id,
|
||||
"streaming": "duplex",
|
||||
},
|
||||
"payload": {
|
||||
"task_group": "audio",
|
||||
"task": "tts",
|
||||
"function": "SpeechSynthesizer",
|
||||
"model": model,
|
||||
"parameters": {
|
||||
"text_type": "PlainText",
|
||||
"voice": voice,
|
||||
"format": format_type,
|
||||
"sample_rate": sample_rate,
|
||||
"volume": 50,
|
||||
"rate": 1.0,
|
||||
"pitch": 1.0,
|
||||
},
|
||||
"input": {}
|
||||
},
|
||||
}
|
||||
await ws.send(json.dumps(run_task_message))
|
||||
|
||||
# 2. 等待 task-started 事件（关键！必须等这个再发文本）
|
||||
task_started = False
|
||||
while not task_started:
|
||||
msg = await ws.recv()
|
||||
if isinstance(msg, str):
|
||||
data = json.loads(msg)
|
||||
header = data.get("header", {})
|
||||
event = header.get("event")
|
||||
if event == "task-started":
|
||||
task_started = True
|
||||
print(f"[阿里云百炼TTS] 第{i+1}次 任务启动成功")
|
||||
elif event == "task-failed":
|
||||
raise Exception(f"启动失败: {header.get('error_message', '未知错误')}")
|
||||
|
||||
# 3. 发送 continue-task（发送文本！这是正确动作）
|
||||
continue_task_message = {
|
||||
"header": {
|
||||
"action": "continue-task", # 改回 continue-task
|
||||
"task_id": session_id,
|
||||
"streaming": "duplex",
|
||||
},
|
||||
"payload": {"input": {"text": text}},
|
||||
}
|
||||
await ws.send(json.dumps(continue_task_message))
|
||||
|
||||
# 4. 发送 finish-task（结束任务）
|
||||
finish_task_message = {
|
||||
"header": {
|
||||
"action": "finish-task",
|
||||
"task_id": session_id,
|
||||
"streaming": "duplex",
|
||||
},
|
||||
"payload": {"input": {}}
|
||||
}
|
||||
await ws.send(json.dumps(finish_task_message))
|
||||
|
||||
# 5. 等待第一个音频数据块
|
||||
while True:
|
||||
msg = await asyncio.wait_for(ws.recv(), timeout=15.0)
|
||||
if isinstance(msg, (bytes, bytearray)) and len(msg) > 0:
|
||||
latency = time.time() - start_time
|
||||
print(f"[阿里云百炼TTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
latencies.append(latency)
|
||||
break
|
||||
elif isinstance(msg, str):
|
||||
data = json.loads(msg)
|
||||
event = data.get("header", {}).get("event")
|
||||
if event == "task-failed":
|
||||
raise Exception(f"合成失败: {data}")
|
||||
elif event == "task-finished":
|
||||
if not latencies or latencies[-1] is None:
|
||||
raise Exception("任务结束但未收到音频")
|
||||
|
||||
except Exception as e:
|
||||
print(f"[阿里云百炼TTS] 第{i+1}次失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
|
||||
return self._calculate_result("阿里云百炼TTS", latencies, test_count)
|
||||
|
||||
async def test_doubao_tts(self, text=None, test_count=5):
|
||||
"""测试火山引擎流式TTS首词延迟（测试多次取平均）"""
|
||||
text = text or self.test_texts[0]
|
||||
@@ -235,12 +114,13 @@ class StreamTTSPerformanceTester:
|
||||
}
|
||||
async with websockets.connect(ws_url, additional_headers=ws_header, max_size=1000000000) as ws:
|
||||
session_id = uuid.uuid4().hex
|
||||
|
||||
|
||||
# 发送会话启动请求
|
||||
header = bytes([
|
||||
(0b0001 << 4) | 0b0001,
|
||||
0b0001 << 4 | 0b1011,
|
||||
0b0001 << 4 | 0b0000,
|
||||
(0b0001 << 4) | 0b0001,
|
||||
0b0001 << 4 | 0b100,
|
||||
0b0001 << 4 | 0b0000,
|
||||
0
|
||||
])
|
||||
optional = bytearray()
|
||||
optional.extend((1).to_bytes(4, "big", signed=True))
|
||||
@@ -249,13 +129,13 @@ class StreamTTSPerformanceTester:
|
||||
optional.extend(session_id_bytes)
|
||||
payload = json.dumps({"speaker": speaker}).encode()
|
||||
await ws.send(header + optional + len(payload).to_bytes(4, "big", signed=True) + payload)
|
||||
|
||||
|
||||
# 发送文本
|
||||
header = bytes([
|
||||
(0b0001 << 4) | 0b0001,
|
||||
0b0001 << 4 | 0b1011,
|
||||
0b0001 << 4 | 0b0000,
|
||||
0
|
||||
(0b0001 << 4) | 0b0001,
|
||||
0b0001 << 4 | 0b100,
|
||||
0b0001 << 4 | 0b0000,
|
||||
0
|
||||
])
|
||||
optional = bytearray()
|
||||
optional.extend((200).to_bytes(4, "big", signed=True))
|
||||
@@ -264,15 +144,13 @@ class StreamTTSPerformanceTester:
|
||||
optional.extend(session_id_bytes)
|
||||
payload = json.dumps({"text": text, "speaker": speaker}).encode()
|
||||
await ws.send(header + optional + len(payload).to_bytes(4, "big", signed=True) + payload)
|
||||
|
||||
|
||||
first_chunk = await ws.recv()
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[火山引擎TTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
|
||||
|
||||
except Exception as e:
|
||||
print(f"[火山引擎TTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("火山引擎TTS", latencies, test_count)
|
||||
|
||||
@@ -313,24 +191,22 @@ class StreamTTSPerformanceTester:
|
||||
first_chunk = await ws.recv()
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[PaddleSpeechTTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
|
||||
|
||||
# 发送结束请求
|
||||
end_request = {
|
||||
"task": "tts",
|
||||
"signal": "end"
|
||||
}
|
||||
await ws.send(json.dumps(end_request))
|
||||
|
||||
|
||||
# 确保连接正常关闭
|
||||
try:
|
||||
await ws.recv()
|
||||
except websockets.exceptions.ConnectionClosedOK:
|
||||
pass
|
||||
|
||||
|
||||
except Exception as e:
|
||||
print(f"[PaddleSpeechTTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("PaddleSpeechTTS", latencies, test_count)
|
||||
|
||||
@@ -344,32 +220,29 @@ class StreamTTSPerformanceTester:
|
||||
tts_config = self.config["TTS"]["IndexStreamTTS"]
|
||||
api_url = tts_config.get("api_url")
|
||||
voice = tts_config.get("voice")
|
||||
|
||||
# 统一计时起点：在建立连接前开始计时
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
payload = {"text": text, "character": voice}
|
||||
async with session.post(api_url, json=payload, timeout=10) as resp:
|
||||
if resp.status != 200:
|
||||
raise Exception(f"请求失败: {resp.status}, {await resp.text()}")
|
||||
|
||||
|
||||
async for chunk in resp.content.iter_any():
|
||||
data = chunk[0] if isinstance(chunk, (list, tuple)) else chunk
|
||||
if not data:
|
||||
continue
|
||||
|
||||
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[IndexStreamTTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
resp.close()
|
||||
break
|
||||
else:
|
||||
latencies.append(None)
|
||||
|
||||
latencies.append(0)
|
||||
|
||||
except Exception as e:
|
||||
print(f"[IndexStreamTTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("IndexStreamTTS", latencies, test_count)
|
||||
|
||||
@@ -384,8 +257,7 @@ class StreamTTSPerformanceTester:
|
||||
api_url = tts_config["api_url"]
|
||||
access_token = tts_config["access_token"]
|
||||
voice = tts_config["voice"]
|
||||
|
||||
# 统一计时起点：在建立连接前开始计时
|
||||
|
||||
start_time = time.time()
|
||||
async with aiohttp.ClientSession() as session:
|
||||
params = {
|
||||
@@ -401,23 +273,21 @@ class StreamTTSPerformanceTester:
|
||||
"Authorization": f"Bearer {access_token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
|
||||
async with session.get(api_url, params=params, headers=headers, timeout=10) as resp:
|
||||
if resp.status != 200:
|
||||
raise Exception(f"请求失败: {resp.status}, {await resp.text()}")
|
||||
|
||||
|
||||
# 接收第一个数据块
|
||||
async for _ in resp.content.iter_any():
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[LinkeraiTTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
break
|
||||
else:
|
||||
latencies.append(None)
|
||||
|
||||
latencies.append(0)
|
||||
|
||||
except Exception as e:
|
||||
print(f"[LinkeraiTTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("LinkeraiTTS", latencies, test_count)
|
||||
|
||||
@@ -435,9 +305,10 @@ class StreamTTSPerformanceTester:
|
||||
api_secret = tts_config["api_secret"]
|
||||
api_url = tts_config.get("api_url", "wss://cbm01.cn-huabei-1.xf-yun.com/v1/private/mcd9m97e6")
|
||||
voice = tts_config.get("voice", "x5_lingxiaoxuan_flow")
|
||||
|
||||
# 生成认证URL
|
||||
auth_url = self._create_xunfei_auth_url(api_key, api_secret, api_url)
|
||||
start_time = time.time()
|
||||
|
||||
async with websockets.connect(
|
||||
auth_url,
|
||||
ping_interval=30,
|
||||
@@ -447,7 +318,10 @@ class StreamTTSPerformanceTester:
|
||||
) as ws:
|
||||
# 构造请求
|
||||
request = self._build_xunfei_request(app_id, text, voice)
|
||||
# 发送请求后立即计时，确保准确测量从发送文本到接收首块的时间
|
||||
await ws.send(json.dumps(request))
|
||||
start_time = time.time()
|
||||
|
||||
# 等待第一个音频数据块
|
||||
first_audio_received = False
|
||||
while not first_audio_received:
|
||||
@@ -455,14 +329,14 @@ class StreamTTSPerformanceTester:
|
||||
data = json.loads(msg)
|
||||
header = data.get("header", {})
|
||||
code = header.get("code")
|
||||
|
||||
|
||||
if code != 0:
|
||||
message = header.get("message", "未知错误")
|
||||
raise Exception(f"合成失败: {code} - {message}")
|
||||
|
||||
|
||||
payload = data.get("payload", {})
|
||||
audio_payload = payload.get("audio", {})
|
||||
|
||||
|
||||
if audio_payload:
|
||||
status = audio_payload.get("status", 0)
|
||||
audio_data = audio_payload.get("audio", "")
|
||||
@@ -470,12 +344,10 @@ class StreamTTSPerformanceTester:
|
||||
# 收到第一个音频数据块
|
||||
latency = time.time() - start_time
|
||||
latencies.append(latency)
|
||||
print(f"[讯飞TTS] 第{i+1}次 首词延迟: {latency:.3f}s")
|
||||
first_audio_received = True
|
||||
break
|
||||
except Exception as e:
|
||||
print(f"[讯飞TTS] 第{i+1}次测试失败: {str(e)}")
|
||||
latencies.append(None)
|
||||
latencies.append(0)
|
||||
|
||||
return self._calculate_result("讯飞TTS", latencies, test_count)
|
||||
|
||||
@@ -559,9 +431,8 @@ class StreamTTSPerformanceTester:
|
||||
|
||||
|
||||
def _calculate_result(self, service_name, latencies, test_count):
|
||||
"""计算测试结果（正确处理None值，剔除失败测试）"""
|
||||
# 剔除失败的测试（None值和<=0延迟），只统计有效延迟
|
||||
valid_latencies = [l for l in latencies if l is not None and l > 0]
|
||||
"""计算测试结果"""
|
||||
valid_latencies = [l for l in latencies if l > 0]
|
||||
if valid_latencies:
|
||||
avg_latency = sum(valid_latencies) / len(valid_latencies)
|
||||
status = f"成功（{len(valid_latencies)}/{test_count}次有效）"
|
||||
@@ -595,10 +466,9 @@ class StreamTTSPerformanceTester:
|
||||
]
|
||||
|
||||
print(tabulate(table_data, headers=["TTS服务", "首词延迟(秒)", "状态"], tablefmt="grid"))
|
||||
print("\n测试说明：测量从建立连接到接收第一个音频数据块的时间（包含握手、鉴权、发送文本），取多次测试平均值")
|
||||
print("- 计时起点: 建立WebSocket/HTTP连接前（统一包含网络建连、握手、发送文本全流程）")
|
||||
print("\n测试说明：测量从发送请求到接收第一个音频数据块的时间，取多次测试平均值")
|
||||
print("- 超时控制: 单个请求最大等待时间为10秒")
|
||||
print("- 错误处理: 失败的测试不计入平均值，只统计成功测试的延迟")
|
||||
print("- 错误处理: 无法连接和超时的列为网络错误")
|
||||
print("- 排序规则: 按平均耗时从快到慢排序")
|
||||
|
||||
|
||||
@@ -624,12 +494,7 @@ class StreamTTSPerformanceTester:
|
||||
# 测试阿里云TTS
|
||||
result = await self.test_aliyun_tts(test_text, test_count)
|
||||
self.results.append(result)
|
||||
|
||||
# 测试阿里云百炼TTS
|
||||
if self.config.get("TTS", {}).get("AliBLTTS"):
|
||||
result = await self.test_alibl_tts(test_text, test_count)
|
||||
self.results.append(result)
|
||||
|
||||
|
||||
# 测试火山引擎TTS
|
||||
result = await self.test_doubao_tts(test_text, test_count)
|
||||
self.results.append(result)
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding:utf-8 -*-
|
||||
#--------- 本项目推荐环境是python3.10，以下暂时不推荐升级的依赖
|
||||
torch==2.2.2
|
||||
torchaudio==2.2.2
|
||||
@@ -11,9 +10,9 @@ silero_vad==6.1.0
|
||||
opuslib_next==1.1.5
|
||||
pydub==0.25.1
|
||||
funasr==1.2.7
|
||||
openai==2.8.1
|
||||
openai==2.7.1
|
||||
google-generativeai==0.8.5
|
||||
edge_tts==7.2.6
|
||||
edge_tts==7.2.3
|
||||
httpx==0.28.1
|
||||
aiohttp==3.13.2
|
||||
aiohttp_cors==0.8.1
|
||||
@@ -25,11 +24,11 @@ cozepy==0.20.0
|
||||
mem0ai==1.0.0
|
||||
bs4==0.0.2
|
||||
modelscope==1.23.2
|
||||
sherpa_onnx==1.12.17
|
||||
sherpa_onnx==1.12.15
|
||||
mcp==1.20.0
|
||||
cnlunar==0.2.0
|
||||
PySocks==1.7.1
|
||||
dashscope==1.25.2
|
||||
dashscope==1.24.6
|
||||
baidu-aip==4.16.13
|
||||
chardet==5.2.0
|
||||
aioconsole==0.8.2
|
||||
@@ -39,4 +38,4 @@ PyJWT==2.10.1
|
||||
psutil==7.0.0
|
||||
portalocker==3.2.0
|
||||
Jinja2==3.1.6
|
||||
vosk==0.3.45
|
||||
vosk==0.3.44
|
||||
@@ -240,24 +240,6 @@ export class AudioRecorder {
|
||||
if (this.isRecording) return false;
|
||||
|
||||
try {
|
||||
// 检查是否有WebSocketHandler实例
|
||||
const { getWebSocketHandler } = await import('../network/websocket.js');
|
||||
const wsHandler = getWebSocketHandler();
|
||||
|
||||
// 如果机器正在说话，发送打断消息
|
||||
if (wsHandler && wsHandler.isRemoteSpeaking && wsHandler.currentSessionId) {
|
||||
const abortMessage = {
|
||||
session_id: wsHandler.currentSessionId,
|
||||
type: 'abort',
|
||||
reason: 'wake_word_detected'
|
||||
};
|
||||
|
||||
if (this.websocket && this.websocket.readyState === WebSocket.OPEN) {
|
||||
this.websocket.send(JSON.stringify(abortMessage));
|
||||
log('发送打断消息', 'info');
|
||||
}
|
||||
}
|
||||
|
||||
if (!this.initEncoder()) {
|
||||
log('无法启动录音: Opus编码器初始化失败', 'error');
|
||||
return false;
|
||||
|
||||
Reference in New Issue
Block a user