mirror of
https://github.com/xinnan-tech/xiaozhi-esp32-server.git
synced 2026-07-23 07:33:53 +08:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
  欣南科技andGitHub
|
2d5ef3011a | ||
 hrz
|
63dab6d224 | ||
 CGDandGitHub
|
38dd76dfb3 | ||
|
fe642de526 | ||
|
hrz
|
f4b7eefc38 | ||
|
hrz
|
5bbbad02b4 | ||
 Sakura-RanChen
|
bd85dd5faf | ||
|
hrzandGitHub
|
d91ba3d219 | ||
|
MakerZorky
|
7992c40815 | ||
|
hrzandGitHub
|
574b01db97 | ||
|
hrz
|
2d6049b7bd | ||
|
hrz
|
96d0f2e6f9 | ||
|
hrz
|
b46bd4d0dd | ||
|
hrzandGitHub
|
411fc46189 | ||
|
Sakura-RanChen
|
48427e330f | ||
 王华侨
|
5ddc2451c1 | ||
|
王华侨
|
e357cad2bb | ||
 wnk
|
9a5aaea8ca |
@@ -156,8 +156,8 @@ server:
|
||||
|
||||
| 部署方式 | 特点 | 适用场景 | Docker部署文档 | 源码部署文档 |
|
||||
|---------|------|---------|---------|---------|
|
||||
| **最简化安装** | 智能对话、IOT功能，数据存储在配置文件 | 低配置环境，无需数据库，仅支持虾哥1.6.0及以下固件版本 | [Docker只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) | [本地源码只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)|
|
||||
| **全模块安装** | 智能对话、IOT、OTA、智控台，数据存储在数据库 | 完整功能体验，支持虾哥最新固件 |[Docker运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) | [本地源码运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) |
|
||||
| **最简化安装** | 智能对话、IOT功能，数据存储在配置文件 | 低配置环境，无需数据库 | [Docker只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) | [本地源码只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)|
|
||||
| **全模块安装** | 智能对话、IOT、OTA、智控台，数据存储在数据库 | 完整功能体验 |[Docker运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) | [本地源码运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) |
|
||||
|
||||
> 💡 提示：以下是按最新代码部署后的测试平台，有需要可烧录测试，并发为6个，每天会清空数据
|
||||
|
||||
|
||||
+12
-7
@@ -240,7 +240,7 @@ selected_module:
|
||||
LLM: ChatGLMLLM
|
||||
TTS: EdgeTTS
|
||||
# 默认不开启记忆，如需开启请看配置文件里的描述
|
||||
Memory: nomem
|
||||
Memory: nomem
|
||||
# 默认不开启意图识别，如需开启请看配置文件里的描述
|
||||
Intent: nointent
|
||||
```
|
||||
@@ -276,21 +276,26 @@ LLM:
|
||||
如果你能看到，类似以下日志,则是本项目服务启动成功的标志。
|
||||
|
||||
```
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - Server is running at ws://xxx.xx.xx.xx:8000/xiaozhi/v1/
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - =======上面的地址是websocket协议地址，请勿用浏览器访问=======
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - =======================================================
|
||||
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-OTA接口是 http://192.168.4.123:8002/xiaozhi/ota/
|
||||
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
|
||||
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======上面的地址是websocket协议地址，请勿用浏览器访问=======
|
||||
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
|
||||
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======================================================
|
||||
```
|
||||
|
||||
正常来说，如果您是通过源码运行本项目，日志会有你的接口地址信息。
|
||||
但是如果你用docker部署，那么你的日志里给出的接口地址信息就不是真实的接口地址。
|
||||
|
||||
最正确的方法，是根据电脑的局域网IP来确定你的接口地址。
|
||||
如果你的电脑的局域网IP比如是`192.168.1.25`，那么你的接口地址就是：`ws://192.168.1.25:8000/xiaozhi/v1/`。
|
||||
如果你的电脑的局域网IP比如是`192.168.1.25`，那么你的接口地址就是：`ws://192.168.1.25:8000/xiaozhi/v1/`，对应的OTA地址就是：`http://192.168.1.25:8002/xiaozhi/ota/`。
|
||||
|
||||
这个信息很有用的，后面`编译esp32固件`需要用到。
|
||||
|
||||
接下来，你就可以开始 [编译1.6.0版本esp32固件](firmware-build.md)了。
|
||||
接下来，你就可以开始操作你的esp32设备了，你可以`自行编译esp32固件`也可以配置使用`虾哥编译好的1.6.1以上版本的固件`。两个任选一个
|
||||
|
||||
1、 [编译自己的esp32固件](firmware-build.md)了。
|
||||
|
||||
2、 [基于虾哥编译好的固件配置自定义服务器](firmware-setting.md)了。
|
||||
|
||||
|
||||
以下是一些常见问题，供参考：
|
||||
|
||||
@@ -169,7 +169,7 @@ docker logs -f xiaozhi-esp32-server
|
||||
如果你能看到，类似以下日志,则是Server启动成功的标志。
|
||||
|
||||
```
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - Server is running at ws://xxx.xx.xx.xx:8000/xiaozhi/v1/
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - Websocket地址是 ws://xxx.xx.xx.xx:8000/xiaozhi/v1/
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - =======上面的地址是websocket协议地址，请勿用浏览器访问=======
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
|
||||
25-02-23 12:01:09[core.websocket_server] - INFO - =======================================================
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
# 编译1.6.0版本esp32固件
|
||||
|
||||
## 第1步 配置环境
|
||||
先按照这个教程配置项目环境[《Windows搭建 ESP IDF 5.3.2开发环境以及编译小智》](https://icnynnzcwou8.feishu.cn/wiki/JEYDwTTALi5s2zkGlFGcDiRknXf)
|
||||
|
||||
## 第2步 打开配置文件
|
||||
配置好编译环境后，下载虾哥iaozhi-esp32项目源码，
|
||||
|
||||
|
||||
从这里下载虾哥[xiaozhi-esp32项目1.6.0版本源码](https://github.com/78/xiaozhi-esp32/archive/refs/tags/v1.6.0.zip)。
|
||||
|
||||
从这里下载虾哥[xiaozhi-esp32项目1.6.0版本源码](https://github.com/78/xiaozhi-esp32/archive/refs/tags/v1.6.0.zip)。
|
||||
|
||||
从这里下载虾哥[xiaozhi-esp32项目1.6.0版本源码](https://github.com/78/xiaozhi-esp32/archive/refs/tags/v1.6.0.zip)。
|
||||
|
||||
下载后，解压缩包，打开`xiaozhi-esp32/main/Kconfig.projbuild`文件。
|
||||
|
||||
|
||||
## 第3步 修改WEBSOCKET地址
|
||||
找到`WEBSOCKET_URL`的`default`的内容，把`wss://api.tenclass.net/xiaozhi/v1/`
|
||||
改成你自己的地址，例如，我的接口地址是`ws://192.168.1.25:8000/xiaozhi/v1/`，就把内容改成这个。
|
||||
|
||||
修改前：
|
||||
|
||||
```
|
||||
config WEBSOCKET_URL
|
||||
depends on CONNECTION_TYPE_WEBSOCKET
|
||||
string "Websocket URL"
|
||||
default "wss://api.tenclass.net/xiaozhi/v1/"
|
||||
help
|
||||
Communication with the server through websocket after wake up.
|
||||
```
|
||||
|
||||
修改后(示例)：
|
||||
|
||||
```
|
||||
config WEBSOCKET_URL
|
||||
depends on CONNECTION_TYPE_WEBSOCKET
|
||||
string "Websocket URL"
|
||||
default "ws://192.168.1.25:8000/xiaozhi/v1/"
|
||||
help
|
||||
Communication with the server through websocket after wake up.
|
||||
```
|
||||
|
||||
注意：你的地址是`ws://`开头，不是`wss://`开头，一定不要写错了。
|
||||
|
||||
注意：你的地址是`ws://`开头，不是`wss://`开头，一定不要写错了。
|
||||
|
||||
注意：你的地址是`ws://`开头，不是`wss://`开头，一定不要写错了。
|
||||
|
||||
## 第4步 设置编译参数
|
||||
|
||||
设置编译参数
|
||||
|
||||
```
|
||||
# 终端命令行进入xiaozhi-esp32的根目录
|
||||
cd xiaozhi-esp32
|
||||
# 例如我使用的板子是esp32s3，所以设置编译目标为esp32s3，如果你的板子是其他型号，请替换成对应的型号
|
||||
idf.py set-target esp32s3
|
||||
# 进入菜单配置
|
||||
idf.py menuconfig
|
||||
```
|
||||
|
||||
|
||||
|
||||
进入菜单配置后，再进入`Xiaozhi Assistant`，将`CONNECTION_TYPE`设置为`Websocket`
|
||||
回退到主菜单，再进入`Xiaozhi Assistant`，将`BOARD_TYPE`设置你板子的具体型号
|
||||
保存退出，回到终端命令行。
|
||||
|
||||
|
||||
|
||||
## 第5步 编译固件
|
||||
|
||||
```
|
||||
idf.py build
|
||||
```
|
||||
|
||||
## 第6步 打包bin固件
|
||||
|
||||
```
|
||||
cd scripts
|
||||
python release.py
|
||||
```
|
||||
|
||||
上面的打包命令执行完成后，会在项目根目录下的`build`目录下生成固件文件`merged-binary.bin`。
|
||||
这个`merged-binary.bin`就是要烧录到硬件上的固件文件。
|
||||
|
||||
注意：如果执行到第二命令后，报了“zip”相关的错误，请忽略这个错误，只要`build`目录下生成固件文件`merged-binary.bin`
|
||||
，对你没有太大影响，请继续。
|
||||
|
||||
## 第7步 烧录固件
|
||||
将esp32设备连接电脑，使用chrome浏览器，打开以下网址
|
||||
|
||||
```
|
||||
https://espressif.github.io/esp-launchpad/
|
||||
```
|
||||
|
||||
打开这个教程，[Flash工具/Web端烧录固件（无IDF开发环境）](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)。
|
||||
翻到：`方式二：ESP-Launchpad 浏览器WEB端烧录`，从`3. 烧录固件/下载到开发板`开始，按照教程操作。
|
||||
|
||||
烧录成功且联网成功后，通过唤醒词唤醒小智，留意server端输出的控制台信息。
|
||||
|
||||
## 常见问题
|
||||
以下是一些常见问题，供参考：
|
||||
|
||||
[1、为什么我说的话，小智识别出来很多韩文、日文、英文](./FAQ.md)
|
||||
|
||||
[2、为什么会出现“TTS 任务出错 文件不存在”？](./FAQ.md)
|
||||
|
||||
[3、TTS 经常失败，经常超时](./FAQ.md)
|
||||
|
||||
[4、使用Wifi能连接自建服务器，但是4G模式却接不上](./FAQ.md)
|
||||
|
||||
[5、如何提高小智对话响应速度？](./FAQ.md)
|
||||
|
||||
[6、我说话很慢，停顿时小智老是抢话](./FAQ.md)
|
||||
|
||||
[7、我想通过小智控制电灯、空调、远程开关机等操作](./FAQ.md)
|
||||
+19
-2
@@ -1,14 +1,31 @@
|
||||
# esp32固件编译
|
||||
|
||||
## 第1步 准备你的ota地址
|
||||
如果你按照教程使用的是全模块部署，就应该会有ota地址。
|
||||
|
||||
如果你，使用的是本项目0.3.12版本，不管是简单Server部署还是全模块部署，都会有ota地址。
|
||||
|
||||
由于简单Server部署和全模块部署的OTA地址设置方式不一样，请你选择下面的具体方式：
|
||||
|
||||
### 如果你用的是简单Server部署
|
||||
此刻，请你用浏览器打开你的ota地址，例如我的ota地址
|
||||
```
|
||||
http://192.168.1.25:8002/xiaozhi/ota/
|
||||
```
|
||||
如果显示“OTA接口运行正常，向设备发送的websocket地址是：ws://xxx:8000/xiaozhi/v1/
|
||||
|
||||
你可以使用项目自带的`test_page.html`测试一下，是否能连上ota页面输出的websocket地址。
|
||||
|
||||
如果访问不到，你需要到配置文件`.config.yaml`里修改`server.websocket`的地址，重启后再重新测试，直到`test_page.html`能正常访问。
|
||||
|
||||
成功后，请往下进行第2步
|
||||
|
||||
### 如果你用的是全模块部署
|
||||
此刻，请你用浏览器打开你的ota地址，例如我的ota地址
|
||||
```
|
||||
http://192.168.1.25:8002/xiaozhi/ota/
|
||||
```
|
||||
|
||||
如果显示“OTA接口运行正常，websocket集群数量：X”。那就往下。
|
||||
如果显示“OTA接口运行正常，websocket集群数量：X”。那就往下进行2步。
|
||||
|
||||
如果显示“OTA接口运行不正常”，大概是你还没在`智控台`配置`Websocket`地址。那就：
|
||||
|
||||
|
||||
@@ -177,5 +177,5 @@ public interface Constant {
|
||||
/**
|
||||
* 版本号
|
||||
*/
|
||||
public static final String VERSION = "0.3.11";
|
||||
public static final String VERSION = "0.3.12";
|
||||
}
|
||||
+64
@@ -3,6 +3,7 @@ package xiaozhi.modules.model.service.impl;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.springframework.stereotype.Service;
|
||||
@@ -19,6 +20,10 @@ import xiaozhi.common.redis.RedisKeys;
|
||||
import xiaozhi.common.redis.RedisUtils;
|
||||
import xiaozhi.common.service.impl.BaseServiceImpl;
|
||||
import xiaozhi.common.utils.ConvertUtils;
|
||||
import xiaozhi.modules.agent.dao.AgentDao;
|
||||
import xiaozhi.modules.agent.dao.AgentTemplateDao;
|
||||
import xiaozhi.modules.agent.entity.AgentEntity;
|
||||
import xiaozhi.modules.agent.service.AgentTemplateService;
|
||||
import xiaozhi.modules.model.dao.ModelConfigDao;
|
||||
import xiaozhi.modules.model.dto.ModelBasicInfoDTO;
|
||||
import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
|
||||
@@ -36,6 +41,9 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
|
||||
private final ModelConfigDao modelConfigDao;
|
||||
private final ModelProviderService modelProviderService;
|
||||
private final RedisUtils redisUtils;
|
||||
private final AgentTemplateDao agentTemplateDao;
|
||||
private final AgentTemplateService agentTemplateService;
|
||||
private final AgentDao agentDao;
|
||||
|
||||
@Override
|
||||
public List<ModelBasicInfoDTO> getModelCodeList(String modelType, String modelName) {
|
||||
@@ -75,6 +83,7 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
|
||||
// 再保存供应器提供的模型
|
||||
ModelConfigEntity modelConfigEntity = ConvertUtils.sourceToTarget(modelConfigBodyDTO, ModelConfigEntity.class);
|
||||
modelConfigEntity.setModelType(modelType);
|
||||
modelConfigEntity.setIsDefault(0);
|
||||
modelConfigDao.insert(modelConfigEntity);
|
||||
return ConvertUtils.sourceToTarget(modelConfigEntity, ModelConfigDTO.class);
|
||||
}
|
||||
@@ -102,9 +111,64 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
|
||||
|
||||
@Override
|
||||
public void delete(String id) {
|
||||
// 查看是否是默认
|
||||
ModelConfigEntity modelConfig = modelConfigDao.selectById(id);
|
||||
if (modelConfig != null && modelConfig.getIsDefault() == 1) {
|
||||
throw new RenException("该模型为默认模型，请先设置其他模型为默认模型");
|
||||
}
|
||||
// 验证是否有引用
|
||||
checkAgentReference(id);
|
||||
checkIntentConfigReference(id);
|
||||
|
||||
modelConfigDao.deleteById(id);
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查智能体配置是否有引用
|
||||
*
|
||||
* @param modelId 模型ID
|
||||
*/
|
||||
private void checkAgentReference(String modelId) {
|
||||
List<AgentEntity> agents = agentDao.selectList(
|
||||
new QueryWrapper<AgentEntity>()
|
||||
.eq("vad_model_id", modelId)
|
||||
.or()
|
||||
.eq("asr_model_id", modelId)
|
||||
.or()
|
||||
.eq("llm_model_id", modelId)
|
||||
.or()
|
||||
.eq("tts_model_id", modelId)
|
||||
.or()
|
||||
.eq("mem_model_id", modelId)
|
||||
.or()
|
||||
.eq("intent_model_id", modelId));
|
||||
if (!agents.isEmpty()) {
|
||||
String agentNames = agents.stream()
|
||||
.map(AgentEntity::getAgentName)
|
||||
.collect(Collectors.joining("、"));
|
||||
throw new RenException(String.format("该模型配置已被智能体[%s]引用，无法删除", agentNames));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查意图识别配置是否有引用
|
||||
*
|
||||
* @param modelId 模型ID
|
||||
*/
|
||||
private void checkIntentConfigReference(String modelId) {
|
||||
ModelConfigEntity modelConfig = modelConfigDao.selectById(modelId);
|
||||
if (modelConfig != null
|
||||
&& "LLM".equals(modelConfig.getModelType() == null ? null : modelConfig.getModelType().toUpperCase())) {
|
||||
List<ModelConfigEntity> intentConfigs = modelConfigDao.selectList(
|
||||
new QueryWrapper<ModelConfigEntity>()
|
||||
.eq("model_type", "Intent")
|
||||
.like("config_json", "%" + modelId + "%"));
|
||||
if (!intentConfigs.isEmpty()) {
|
||||
throw new RenException("该LLM模型已被意图识别配置引用，无法删除");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getModelNameById(String id) {
|
||||
if (StringUtils.isBlank(id)) {
|
||||
|
||||
@@ -3,9 +3,13 @@ import sys
|
||||
import signal
|
||||
from config.settings import load_config, check_config_file
|
||||
from core.websocket_server import WebSocketServer
|
||||
from core.ota_server import SimpleOtaServer
|
||||
from core.utils.util import check_ffmpeg_installed
|
||||
from config.logger import setup_logging
|
||||
from core.utils.util import get_local_ip
|
||||
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
|
||||
|
||||
async def wait_for_exit():
|
||||
@@ -34,6 +38,41 @@ async def main():
|
||||
# 启动 WebSocket 服务器
|
||||
ws_server = WebSocketServer(config)
|
||||
ws_task = asyncio.create_task(ws_server.start())
|
||||
ota_task = None
|
||||
|
||||
read_config_from_api = config.get("read_config_from_api", False)
|
||||
if not read_config_from_api:
|
||||
# 启动 Simple OTA 服务器
|
||||
ota_server = SimpleOtaServer(config)
|
||||
ota_task = asyncio.create_task(ota_server.start())
|
||||
|
||||
logger.bind(tag=TAG).info(
|
||||
"OTA接口是\t\thttp://{}:{}/xiaozhi/ota/",
|
||||
get_local_ip(),
|
||||
config["server"]["ota_port"],
|
||||
)
|
||||
|
||||
# 获取WebSocket配置，使用安全的默认值
|
||||
websocket_port = 8000
|
||||
server_config = config.get("server", {})
|
||||
if isinstance(server_config, dict):
|
||||
websocket_port = int(server_config.get("port", 8000))
|
||||
|
||||
logger.bind(tag=TAG).info(
|
||||
"Websocket地址是\tws://{}:{}/xiaozhi/v1/",
|
||||
get_local_ip(),
|
||||
websocket_port,
|
||||
)
|
||||
|
||||
logger.bind(tag=TAG).info(
|
||||
"=======上面的地址是websocket协议地址，请勿用浏览器访问======="
|
||||
)
|
||||
logger.bind(tag=TAG).info(
|
||||
"如想测试websocket请用谷歌浏览器打开test目录下的test_page.html"
|
||||
)
|
||||
logger.bind(tag=TAG).info(
|
||||
"=============================================================\n"
|
||||
)
|
||||
|
||||
try:
|
||||
await wait_for_exit() # 监听退出信号
|
||||
@@ -41,8 +80,12 @@ async def main():
|
||||
print("任务被取消，清理资源中...")
|
||||
finally:
|
||||
ws_task.cancel()
|
||||
if ota_task:
|
||||
ota_task.cancel()
|
||||
try:
|
||||
await ws_task
|
||||
if ota_task:
|
||||
await ota_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
print("服务器已关闭，程序退出。")
|
||||
|
||||
@@ -9,6 +9,15 @@ server:
|
||||
# 服务器监听地址和端口(Server listening address and port)
|
||||
ip: 0.0.0.0
|
||||
port: 8000
|
||||
# OTA接口的端口号
|
||||
ota_port: 8002
|
||||
# 这个websocket配置是指ota接口向设备发送的websocket地址
|
||||
# 如果按默认的写法，ota接口会自动生成websocket地址。这个地址你可以直接用浏览器访问ota接口确认一下
|
||||
# 当你使用docker部署或使用公网部署(使用ssl、域名)时，不一定准确
|
||||
# 所以如果你使用docker部署或使用公网部署时，请设置正确的websocket地址
|
||||
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
|
||||
# OTA返回信息时区偏移量
|
||||
timezone_offset: +8
|
||||
# 认证配置
|
||||
auth:
|
||||
# 是否启用认证
|
||||
@@ -239,8 +248,8 @@ LLM:
|
||||
api_key: 你的deepseek web key
|
||||
temperature: 0.7 # 温度值
|
||||
max_tokens: 500 # 最大生成token数
|
||||
top_p: 1
|
||||
top_k: 50
|
||||
top_p: 1
|
||||
top_k: 50
|
||||
frequency_penalty: 0 # 频率惩罚
|
||||
AliAppLLM:
|
||||
# 定义LLM API类型
|
||||
@@ -249,7 +258,7 @@ LLM:
|
||||
app_id: 你的app_id
|
||||
# 可在这里找到你的 api_key https://bailian.console.aliyun.com/?apiKey=1#/api-key
|
||||
api_key: 你的api_key
|
||||
# 是否不使用本地prompt：true|false (默不用请在百练应用中设置prompt)
|
||||
# 是否不使用本地prompt：true|false (默不用请在百练应用中设置prompt)
|
||||
is_no_prompt: true
|
||||
# Ali_memory_id：false（不使用）|你的memory_id（请在百练应用中设置中获取）
|
||||
# Tips！：Ali_memory未实现多用户存储记忆(记忆按id调用)
|
||||
|
||||
@@ -3,7 +3,7 @@ import sys
|
||||
from loguru import logger
|
||||
from config.config_loader import load_config
|
||||
|
||||
SERVER_VERSION = "0.3.11"
|
||||
SERVER_VERSION = "0.3.12"
|
||||
|
||||
|
||||
def get_module_abbreviation(module_name, module_dict):
|
||||
|
||||
@@ -441,11 +441,16 @@ class ConnectionHandler:
|
||||
current_text = full_text[processed_chars:] # 从未处理的位置开始
|
||||
|
||||
# 查找最后一个有效标点
|
||||
punctuations = ("。", "？", "！", "；", "：")
|
||||
punctuations = ("。", ".", "？", "?", "！", "!", "；", ";", "：")
|
||||
last_punct_pos = -1
|
||||
number_flag = True
|
||||
for punct in punctuations:
|
||||
pos = current_text.rfind(punct)
|
||||
if pos > last_punct_pos:
|
||||
prev_char = current_text[pos - 1] if pos - 1 >= 0 else ""
|
||||
# 如果.前面是数字统一判断为小数
|
||||
if prev_char.isdigit() and punct == '.':
|
||||
number_flag = False
|
||||
if pos > last_punct_pos and number_flag:
|
||||
last_punct_pos = pos
|
||||
|
||||
# 找到分割点则处理
|
||||
@@ -567,11 +572,16 @@ class ConnectionHandler:
|
||||
current_text = full_text[processed_chars:] # 从未处理的位置开始
|
||||
|
||||
# 查找最后一个有效标点
|
||||
punctuations = ("。", "？", "！", "；", "：")
|
||||
punctuations = ("。", ".", "？", "?", "！", "!", "；", ";", "：")
|
||||
last_punct_pos = -1
|
||||
number_flag = True
|
||||
for punct in punctuations:
|
||||
pos = current_text.rfind(punct)
|
||||
if pos > last_punct_pos:
|
||||
prev_char = current_text[pos - 1] if pos - 1 >= 0 else ""
|
||||
# 如果.前面是数字统一判断为小数
|
||||
if prev_char.isdigit() and punct == '.':
|
||||
number_flag = False
|
||||
if pos > last_punct_pos and number_flag:
|
||||
last_punct_pos = pos
|
||||
|
||||
# 找到分割点则处理
|
||||
@@ -888,7 +898,7 @@ class ConnectionHandler:
|
||||
|
||||
def clear_queues(self):
|
||||
# 清空所有任务队列
|
||||
self.logger.bind(tag=TAG).info(
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"开始清理: TTS队列大小={self.tts_queue.qsize()}, 音频队列大小={self.audio_play_queue.qsize()}"
|
||||
)
|
||||
for q in [self.tts_queue, self.audio_play_queue]:
|
||||
@@ -902,7 +912,7 @@ class ConnectionHandler:
|
||||
q.queue.clear()
|
||||
# 添加毒丸信号到队列，确保线程退出
|
||||
# q.queue.put(None)
|
||||
self.logger.bind(tag=TAG).info(
|
||||
self.logger.bind(tag=TAG).debug(
|
||||
f"清理结束: TTS队列大小={self.tts_queue.qsize()}, 音频队列大小={self.audio_play_queue.qsize()}"
|
||||
)
|
||||
|
||||
|
||||
@@ -4,14 +4,49 @@ import asyncio
|
||||
import time
|
||||
from core.utils.util import (
|
||||
get_string_no_punctuation_or_emoji,
|
||||
analyze_emotion
|
||||
)
|
||||
|
||||
TAG = __name__
|
||||
logger = setup_logging()
|
||||
|
||||
emoji_map = {
|
||||
'neutral': '😶',
|
||||
'happy': '🙂',
|
||||
'laughing': '😆',
|
||||
'funny': '😂',
|
||||
'sad': '😔',
|
||||
'angry': '😠',
|
||||
'crying': '😭',
|
||||
'loving': '😍',
|
||||
'embarrassed': '😳',
|
||||
'surprised': '😲',
|
||||
'shocked': '😱',
|
||||
'thinking': '🤔',
|
||||
'winking': '😉',
|
||||
'cool': '😎',
|
||||
'relaxed': '😌',
|
||||
'delicious': '🤤',
|
||||
'kissy': '😘',
|
||||
'confident': '😏',
|
||||
'sleepy': '😴',
|
||||
'silly': '😜',
|
||||
'confused': '🙄'
|
||||
}
|
||||
|
||||
async def sendAudioMessage(conn, audios, text, text_index=0):
|
||||
# 发送句子开始消息
|
||||
emotion = analyze_emotion(text)
|
||||
emoji = emoji_map.get(emotion, '🙂') # 默认使用笑脸
|
||||
await conn.websocket.send(json.dumps(
|
||||
{
|
||||
"type": "llm",
|
||||
"text": emoji,
|
||||
"emotion": emotion,
|
||||
"session_id": conn.session_id,
|
||||
}
|
||||
)
|
||||
)
|
||||
if text_index == conn.tts_first_text_index:
|
||||
logger.bind(tag=TAG).info(f"发送第一段语音: {text}")
|
||||
await send_tts_message(conn, "sentence_start", text)
|
||||
@@ -86,14 +121,4 @@ async def send_stt_message(conn, text):
|
||||
await conn.websocket.send(
|
||||
json.dumps({"type": "stt", "text": stt_text, "session_id": conn.session_id})
|
||||
)
|
||||
await conn.websocket.send(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "llm",
|
||||
"text": "😊",
|
||||
"emotion": "happy",
|
||||
"session_id": conn.session_id,
|
||||
}
|
||||
)
|
||||
)
|
||||
await send_tts_message(conn, "start")
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
import json
|
||||
import time
|
||||
import asyncio
|
||||
from aiohttp import web
|
||||
from config.logger import setup_logging
|
||||
from core.connection import ConnectionHandler
|
||||
from core.utils.util import get_local_ip, initialize_modules
|
||||
|
||||
TAG = __name__
|
||||
|
||||
|
||||
class SimpleOtaServer:
|
||||
def __init__(self, config: dict):
|
||||
self.config = config
|
||||
self.logger = setup_logging()
|
||||
|
||||
def _get_websocket_url(self, local_ip: str, port: int) -> str:
|
||||
"""获取websocket地址
|
||||
|
||||
Args:
|
||||
local_ip: 本地IP地址
|
||||
port: 端口号
|
||||
|
||||
Returns:
|
||||
str: websocket地址
|
||||
"""
|
||||
server_config = self.config["server"]
|
||||
websocket_config = server_config.get("websocket")
|
||||
|
||||
if websocket_config and "你" not in websocket_config:
|
||||
return websocket_config
|
||||
else:
|
||||
return f"ws://{local_ip}:{port}/xiaozhi/v1/"
|
||||
|
||||
async def start(self):
|
||||
server_config = self.config["server"]
|
||||
host = server_config.get("ip", "0.0.0.0")
|
||||
port = int(server_config.get("ota_port"))
|
||||
|
||||
if port:
|
||||
app = web.Application()
|
||||
# 添加路由
|
||||
app.add_routes(
|
||||
[
|
||||
web.get("/xiaozhi/ota/", self._handle_ota_get_request),
|
||||
web.post("/xiaozhi/ota/", self._handle_ota_request),
|
||||
web.options("/xiaozhi/ota/", self._handle_ota_request),
|
||||
]
|
||||
)
|
||||
|
||||
# 运行服务
|
||||
runner = web.AppRunner(app)
|
||||
await runner.setup()
|
||||
site = web.TCPSite(runner, host, port)
|
||||
await site.start()
|
||||
|
||||
# 保持服务运行
|
||||
while True:
|
||||
await asyncio.sleep(3600) # 每隔 1 小时检查一次
|
||||
|
||||
async def _handle_ota_request(self, request):
|
||||
"""处理 /xiaozhi/ota/ 的 POST 请求"""
|
||||
try:
|
||||
data = await request.text()
|
||||
self.logger.bind(tag=TAG).debug(f"OTA请求方法: {request.method}")
|
||||
self.logger.bind(tag=TAG).debug(f"OTA请求头: {request.headers}")
|
||||
self.logger.bind(tag=TAG).debug(f"OTA请求数据: {data}")
|
||||
|
||||
device_id = request.headers.get("device-id", "")
|
||||
if device_id:
|
||||
self.logger.bind(tag=TAG).info(f"OTA请求设备ID: {device_id}")
|
||||
else:
|
||||
raise Exception("OTA请求设备ID为空")
|
||||
|
||||
data_json = json.loads(data)
|
||||
|
||||
server_config = self.config["server"]
|
||||
host = server_config.get("ip", "0.0.0.0")
|
||||
port = int(server_config.get("port", 8000))
|
||||
local_ip = get_local_ip()
|
||||
|
||||
# OTA基础信息
|
||||
return_json = {
|
||||
"server_time": {
|
||||
"timestamp": int(round(time.time() * 1000)),
|
||||
"timezone_offset": server_config.get("timezone_offset", 8) * 60,
|
||||
},
|
||||
"firmware": {
|
||||
"version": data_json["application"].get("version", "1.0.0"),
|
||||
"url": "",
|
||||
},
|
||||
"websocket": {
|
||||
"url": self._get_websocket_url(local_ip, port),
|
||||
},
|
||||
}
|
||||
response = web.Response(
|
||||
text=json.dumps(return_json, separators=(",", ":")),
|
||||
content_type="application/json",
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"OTA请求异常: {e}")
|
||||
return_json = {"success": False, "message": "request error."}
|
||||
response = web.Response(
|
||||
text=json.dumps(return_json, separators=(",", ":")),
|
||||
content_type="application/json",
|
||||
)
|
||||
finally:
|
||||
# 添加header，允许跨域访问
|
||||
response.headers["Access-Control-Allow-Headers"] = (
|
||||
"client-id, content-type, device-id"
|
||||
)
|
||||
response.headers["Access-Control-Allow-Credentials"] = "true"
|
||||
response.headers["Access-Control-Allow-Origin"] = "*"
|
||||
return response
|
||||
|
||||
async def _handle_ota_get_request(self, request):
|
||||
"""处理 /xiaozhi/ota/ 的 GET 请求"""
|
||||
try:
|
||||
server_config = self.config["server"]
|
||||
local_ip = get_local_ip()
|
||||
port = int(server_config.get("port", 8000))
|
||||
websocket_url = self._get_websocket_url(local_ip, port)
|
||||
message = f"OTA接口运行正常，向设备发送的websocket地址是：{websocket_url}"
|
||||
response = web.Response(text=message, content_type="text/plain")
|
||||
except Exception as e:
|
||||
self.logger.bind(tag=TAG).error(f"OTA GET请求异常: {e}")
|
||||
response = web.Response(text="OTA接口异常", content_type="text/plain")
|
||||
finally:
|
||||
# 添加header，允许跨域访问
|
||||
response.headers["Access-Control-Allow-Headers"] = (
|
||||
"client-id, content-type, device-id"
|
||||
)
|
||||
response.headers["Access-Control-Allow-Credentials"] = "true"
|
||||
response.headers["Access-Control-Allow-Origin"] = "*"
|
||||
return response
|
||||
@@ -45,14 +45,14 @@ def parse_response(res):
|
||||
payload 类似与http 请求体
|
||||
"""
|
||||
protocol_version = res[0] >> 4
|
||||
header_size = res[0] & 0x0f
|
||||
header_size = res[0] & 0x0F
|
||||
message_type = res[1] >> 4
|
||||
message_type_specific_flags = res[1] & 0x0f
|
||||
message_type_specific_flags = res[1] & 0x0F
|
||||
serialization_method = res[2] >> 4
|
||||
message_compression = res[2] & 0x0f
|
||||
message_compression = res[2] & 0x0F
|
||||
reserved = res[3]
|
||||
header_extensions = res[4:header_size * 4]
|
||||
payload = res[header_size * 4:]
|
||||
header_extensions = res[4 : header_size * 4]
|
||||
payload = res[header_size * 4 :]
|
||||
result = {}
|
||||
payload_msg = None
|
||||
payload_size = 0
|
||||
@@ -61,13 +61,13 @@ def parse_response(res):
|
||||
payload_msg = payload[4:]
|
||||
elif message_type == SERVER_ACK:
|
||||
seq = int.from_bytes(payload[:4], "big", signed=True)
|
||||
result['seq'] = seq
|
||||
result["seq"] = seq
|
||||
if len(payload) >= 8:
|
||||
payload_size = int.from_bytes(payload[4:8], "big", signed=False)
|
||||
payload_msg = payload[8:]
|
||||
elif message_type == SERVER_ERROR_RESPONSE:
|
||||
code = int.from_bytes(payload[:4], "big", signed=False)
|
||||
result['code'] = code
|
||||
result["code"] = code
|
||||
payload_size = int.from_bytes(payload[4:8], "big", signed=False)
|
||||
payload_msg = payload[8:]
|
||||
if payload_msg is None:
|
||||
@@ -78,8 +78,8 @@ def parse_response(res):
|
||||
payload_msg = json.loads(str(payload_msg, "utf-8"))
|
||||
elif serialization_method != NO_SERIALIZATION:
|
||||
payload_msg = str(payload_msg, "utf-8")
|
||||
result['payload_msg'] = payload_msg
|
||||
result['payload_size'] = payload_size
|
||||
result["payload_msg"] = payload_msg
|
||||
result["payload_size"] = payload_size
|
||||
return result
|
||||
|
||||
|
||||
@@ -122,7 +122,9 @@ class ASRProvider(ASRProviderBase):
|
||||
return file_path
|
||||
|
||||
@staticmethod
|
||||
def _generate_header(message_type=CLIENT_FULL_REQUEST, message_type_specific_flags=NO_SEQUENCE) -> bytearray:
|
||||
def _generate_header(
|
||||
message_type=CLIENT_FULL_REQUEST, message_type_specific_flags=NO_SEQUENCE
|
||||
) -> bytearray:
|
||||
"""Generate protocol header."""
|
||||
header = bytearray()
|
||||
header_size = 1
|
||||
@@ -143,13 +145,9 @@ class ASRProvider(ASRProviderBase):
|
||||
"user": {
|
||||
"uid": str(uuid.uuid4()),
|
||||
},
|
||||
"request": {
|
||||
"reqid": reqid,
|
||||
"show_utterances": False,
|
||||
"sequence": 1
|
||||
},
|
||||
"request": {"reqid": reqid, "show_utterances": False, "sequence": 1},
|
||||
"audio": {
|
||||
"format": "wav",
|
||||
"format": "raw",
|
||||
"rate": 16000,
|
||||
"language": "zh-CN",
|
||||
"bits": 16,
|
||||
@@ -158,18 +156,23 @@ class ASRProvider(ASRProviderBase):
|
||||
},
|
||||
}
|
||||
|
||||
async def _send_request(self, audio_data: List[bytes], segment_size: int) -> Optional[str]:
|
||||
async def _send_request(
|
||||
self, audio_data: List[bytes], segment_size: int
|
||||
) -> Optional[str]:
|
||||
"""Send request to Volcano ASR service."""
|
||||
try:
|
||||
auth_header = {'Authorization': 'Bearer; {}'.format(self.access_token)}
|
||||
async with websockets.connect(self.ws_url, additional_headers=auth_header) as websocket:
|
||||
auth_header = {"Authorization": "Bearer; {}".format(self.access_token)}
|
||||
async with websockets.connect(
|
||||
self.ws_url, additional_headers=auth_header
|
||||
) as websocket:
|
||||
# Prepare request data
|
||||
request_params = self._construct_request(str(uuid.uuid4()))
|
||||
print(request_params)
|
||||
payload_bytes = str.encode(json.dumps(request_params))
|
||||
payload_bytes = gzip.compress(payload_bytes)
|
||||
full_client_request = self._generate_header()
|
||||
full_client_request.extend((len(payload_bytes)).to_bytes(4, 'big')) # payload size(4 bytes)
|
||||
full_client_request.extend(
|
||||
(len(payload_bytes)).to_bytes(4, "big")
|
||||
) # payload size(4 bytes)
|
||||
full_client_request.extend(payload_bytes) # payload
|
||||
|
||||
# Send header and metadata
|
||||
@@ -177,22 +180,29 @@ class ASRProvider(ASRProviderBase):
|
||||
await websocket.send(full_client_request)
|
||||
res = await websocket.recv()
|
||||
result = parse_response(res)
|
||||
if 'payload_msg' in result and result['payload_msg']['code'] != self.success_code:
|
||||
if (
|
||||
"payload_msg" in result
|
||||
and result["payload_msg"]["code"] != self.success_code
|
||||
):
|
||||
logger.bind(tag=TAG).error(f"ASR error: {result}")
|
||||
return None
|
||||
|
||||
for seq, (chunk, last) in enumerate(self.slice_data(audio_data, segment_size), 1):
|
||||
for seq, (chunk, last) in enumerate(
|
||||
self.slice_data(audio_data, segment_size), 1
|
||||
):
|
||||
if last:
|
||||
audio_only_request = self._generate_header(
|
||||
message_type=CLIENT_AUDIO_ONLY_REQUEST,
|
||||
message_type_specific_flags=NEG_SEQUENCE
|
||||
message_type_specific_flags=NEG_SEQUENCE,
|
||||
)
|
||||
else:
|
||||
audio_only_request = self._generate_header(
|
||||
message_type=CLIENT_AUDIO_ONLY_REQUEST
|
||||
)
|
||||
payload_bytes = gzip.compress(chunk)
|
||||
audio_only_request.extend((len(payload_bytes)).to_bytes(4, 'big')) # payload size(4 bytes)
|
||||
audio_only_request.extend(
|
||||
(len(payload_bytes)).to_bytes(4, "big")
|
||||
) # payload size(4 bytes)
|
||||
audio_only_request.extend(payload_bytes) # payload
|
||||
# Send audio data
|
||||
await websocket.send(audio_only_request)
|
||||
@@ -201,9 +211,12 @@ class ASRProvider(ASRProviderBase):
|
||||
response = await websocket.recv()
|
||||
result = parse_response(response)
|
||||
|
||||
if 'payload_msg' in result and result['payload_msg']['code'] == self.success_code:
|
||||
if len(result['payload_msg']['result']) > 0:
|
||||
return result['payload_msg']['result'][0]["text"]
|
||||
if (
|
||||
"payload_msg" in result
|
||||
and result["payload_msg"]["code"] == self.success_code
|
||||
):
|
||||
if len(result["payload_msg"]["result"]) > 0:
|
||||
return result["payload_msg"]["result"][0]["text"]
|
||||
return None
|
||||
else:
|
||||
logger.bind(tag=TAG).error(f"ASR error: {result}")
|
||||
@@ -231,7 +244,7 @@ class ASRProvider(ASRProviderBase):
|
||||
@staticmethod
|
||||
def read_wav_info(data: io.BytesIO = None) -> (int, int, int, int, int):
|
||||
with io.BytesIO(data) as _f:
|
||||
wave_fp = wave.open(_f, 'rb')
|
||||
wave_fp = wave.open(_f, "rb")
|
||||
nchannels, sampwidth, framerate, nframes = wave_fp.getparams()[:4]
|
||||
wave_bytes = wave_fp.readframes(nframes)
|
||||
return nchannels, sampwidth, framerate, nframes, len(wave_bytes)
|
||||
@@ -247,37 +260,32 @@ class ASRProvider(ASRProviderBase):
|
||||
data_len = len(data)
|
||||
offset = 0
|
||||
while offset + chunk_size < data_len:
|
||||
yield data[offset: offset + chunk_size], False
|
||||
yield data[offset : offset + chunk_size], False
|
||||
offset += chunk_size
|
||||
else:
|
||||
yield data[offset: data_len], True
|
||||
yield data[offset:data_len], True
|
||||
|
||||
async def speech_to_text(self, opus_data: List[bytes], session_id: str) -> Tuple[Optional[str], Optional[str]]:
|
||||
async def speech_to_text(
|
||||
self, opus_data: List[bytes], session_id: str
|
||||
) -> Tuple[Optional[str], Optional[str]]:
|
||||
"""将语音数据转换为文本"""
|
||||
try:
|
||||
# 合并所有opus数据包
|
||||
pcm_data = self.decode_opus(opus_data, session_id)
|
||||
combined_pcm_data = b''.join(pcm_data)
|
||||
combined_pcm_data = b"".join(pcm_data)
|
||||
|
||||
wav_buffer = io.BytesIO()
|
||||
|
||||
with wave.open(wav_buffer, "wb") as wav_file:
|
||||
wav_file.setnchannels(1) # 设置声道数
|
||||
wav_file.setsampwidth(2) # 设置采样宽度
|
||||
wav_file.setframerate(16000) # 设置采样率
|
||||
wav_file.writeframes(combined_pcm_data) # 写入 PCM 数据
|
||||
|
||||
# 获取封装后的 WAV 数据
|
||||
wav_data = wav_buffer.getvalue()
|
||||
nchannels, sampwidth, framerate, nframes, wav_len = self.read_wav_info(wav_data)
|
||||
size_per_sec = nchannels * sampwidth * framerate
|
||||
# 直接使用PCM数据
|
||||
# 计算分段大小 (单声道, 16bit, 16kHz采样率)
|
||||
size_per_sec = 1 * 2 * 16000 # nchannels * sampwidth * framerate
|
||||
segment_size = int(size_per_sec * self.seg_duration / 1000)
|
||||
|
||||
# 语音识别
|
||||
start_time = time.time()
|
||||
text = await self._send_request(wav_data, segment_size)
|
||||
text = await self._send_request(combined_pcm_data, segment_size)
|
||||
if text:
|
||||
logger.bind(tag=TAG).debug(f"语音识别耗时: {time.time() - start_time:.3f}s | 结果: {text}")
|
||||
logger.bind(tag=TAG).debug(
|
||||
f"语音识别耗时: {time.time() - start_time:.3f}s | 结果: {text}"
|
||||
)
|
||||
return text, None
|
||||
return "", None
|
||||
|
||||
|
||||
@@ -29,16 +29,20 @@ class TTSProviderBase(ABC):
|
||||
try:
|
||||
asyncio.run(self.text_to_speak(text, tmp_file))
|
||||
except Exception as e:
|
||||
logger.bind(tag=TAG).error(f"语音生成失败: {text}，错误: {e}")
|
||||
if not os.path.exists(tmp_file):
|
||||
logger.bind(tag=TAG).warning(f"语音生成失败{5 - max_repeat_time + 1}次: {text}，错误: {e}")
|
||||
# 未执行成功，删除文件
|
||||
if os.path.exists(tmp_file):
|
||||
os.remove(tmp_file)
|
||||
max_repeat_time -= 1
|
||||
if max_repeat_time > 0:
|
||||
logger.bind(tag=TAG).error(f"再试{max_repeat_time}次")
|
||||
|
||||
if max_repeat_time > 0:
|
||||
logger.bind(tag=TAG).info(
|
||||
f"语音生成成功: {text}:{tmp_file}，重试{5 - max_repeat_time}次"
|
||||
)
|
||||
else:
|
||||
logger.bind(tag=TAG).error(
|
||||
f"语音生成失败: {text}，请检查网络或服务是否正常"
|
||||
)
|
||||
|
||||
return tmp_file
|
||||
except Exception as e:
|
||||
|
||||
@@ -7,7 +7,29 @@ from typing import Dict, Any
|
||||
from core.utils import tts, llm, intent, memory, vad, asr
|
||||
|
||||
TAG = __name__
|
||||
|
||||
emoji_map = {
|
||||
'neutral': '😶',
|
||||
'happy': '🙂',
|
||||
'laughing': '😆',
|
||||
'funny': '😂',
|
||||
'sad': '😔',
|
||||
'angry': '😠',
|
||||
'crying': '😭',
|
||||
'loving': '😍',
|
||||
'embarrassed': '😳',
|
||||
'surprised': '😲',
|
||||
'shocked': '😱',
|
||||
'thinking': '🤔',
|
||||
'winking': '😉',
|
||||
'cool': '😎',
|
||||
'relaxed': '😌',
|
||||
'delicious': '🤤',
|
||||
'kissy': '😘',
|
||||
'confident': '😏',
|
||||
'sleepy': '😴',
|
||||
'silly': '😜',
|
||||
'confused': '🙄'
|
||||
}
|
||||
|
||||
def get_local_ip():
|
||||
try:
|
||||
@@ -90,13 +112,14 @@ def is_punctuation_or_emoji(char):
|
||||
punctuation_set = {
|
||||
"，",
|
||||
",", # 中文逗号 + 英文逗号
|
||||
"。",
|
||||
".", # 中文句号 + 英文句号
|
||||
"！",
|
||||
"!", # 中文感叹号 + 英文感叹号
|
||||
"-",
|
||||
"－", # 英文连字符 + 中文全角横线
|
||||
"、", # 中文顿号
|
||||
"“",
|
||||
"”",
|
||||
"\"", # 中文双引号 + 英文引号
|
||||
"：",
|
||||
":", # 中文冒号 + 英文冒号
|
||||
}
|
||||
if char.isspace() or char in punctuation_set:
|
||||
return True
|
||||
@@ -329,3 +352,136 @@ def initialize_modules(
|
||||
logger.bind(tag=TAG).info(f"初始化组件: prompt成功 {modules['prompt'][:50]}...")
|
||||
|
||||
return modules
|
||||
|
||||
def analyze_emotion(text):
|
||||
"""
|
||||
分析文本情感并返回对应的emoji名称（支持中英文）
|
||||
"""
|
||||
if not text or not isinstance(text, str):
|
||||
return 'neutral'
|
||||
|
||||
original_text = text
|
||||
text = text.lower().strip()
|
||||
|
||||
# 检查是否包含现有emoji
|
||||
for emotion, emoji in emoji_map.items():
|
||||
if emoji in original_text:
|
||||
return emotion
|
||||
|
||||
# 标点符号分析
|
||||
has_exclamation = '!' in original_text or '！' in original_text
|
||||
has_question = '?' in original_text or '？' in original_text
|
||||
has_ellipsis = '...' in original_text or '…' in original_text
|
||||
|
||||
# 定义情感关键词映射（中英文扩展版）
|
||||
emotion_keywords = {
|
||||
'happy': ['开心', '高兴', '快乐', '愉快', '幸福', '满意', '棒', '好', '不错', '完美', '棒极了', '太好了',
|
||||
'好呀', '好的', 'happy', 'joy', 'great', 'good', 'nice', 'awesome', 'fantastic', 'wonderful'],
|
||||
'laughing': ['哈哈', '哈哈哈', '呵呵', '嘿嘿', '嘻嘻', '笑死', '太好笑了', '笑死我了', 'lol', 'lmao', 'haha',
|
||||
'hahaha', 'hehe', 'rofl', 'funny', 'laugh'],
|
||||
'funny': ['搞笑', '滑稽', '逗', '幽默', '笑点', '段子', '笑话', '太逗了', 'hilarious', 'joke', 'comedy'],
|
||||
'sad': ['伤心', '难过', '悲哀', '悲伤', '忧郁', '郁闷', '沮丧', '失望', '想哭', '难受', '不开心', '唉', '呜呜',
|
||||
'sad', 'upset', 'unhappy', 'depressed', 'sorrow', 'gloomy'],
|
||||
'angry': ['生气', '愤怒', '气死', '讨厌', '烦人', '可恶', '烦死了', '恼火', '暴躁', '火大', '愤怒', '气炸了',
|
||||
'angry', 'mad', 'annoyed', 'furious', 'pissed', 'hate'],
|
||||
'crying': ['哭泣', '泪流', '大哭', '伤心欲绝', '泪目', '流泪', '哭死', '哭晕', '想哭', '泪崩',
|
||||
'cry', 'crying', 'tears', 'sob', 'weep'],
|
||||
'loving': ['爱你', '喜欢', '爱', '亲爱的', '宝贝', '么么哒', '抱抱', '想你', '思念', '最爱', '亲亲', '喜欢你',
|
||||
'love', 'like', 'adore', 'darling', 'sweetie', 'honey', 'miss you', 'heart'],
|
||||
'embarrassed': ['尴尬', '不好意思', '害羞', '脸红', '难为情', '社死', '丢脸', '出丑',
|
||||
'embarrassed', 'awkward', 'shy', 'blush'],
|
||||
'surprised': ['惊讶', '吃惊', '天啊', '哇塞', '哇', '居然', '竟然', '没想到', '出乎意料',
|
||||
'surprise', 'wow', 'omg', 'oh my god', 'amazing', 'unbelievable'],
|
||||
'shocked': ['震惊', '吓到', '惊呆了', '不敢相信', '震撼', '吓死', '恐怖', '害怕', '吓人',
|
||||
'shocked', 'shocking', 'scared', 'frightened', 'terrified', 'horror'],
|
||||
'thinking': ['思考', '考虑', '想一下', '琢磨', '沉思', '冥想', '想', '思考中', '在想',
|
||||
'think', 'thinking', 'consider', 'ponder', 'meditate'],
|
||||
'winking': ['调皮', '眨眼', '你懂的', '坏笑', '邪恶', '奸笑', '使眼色',
|
||||
'wink', 'teasing', 'naughty', 'mischievous'],
|
||||
'cool': ['酷', '帅', '厉害', '棒极了', '真棒', '牛逼', '强', '优秀', '杰出', '出色', '完美',
|
||||
'cool', 'awesome', 'amazing', 'great', 'impressive', 'perfect'],
|
||||
'relaxed': ['放松', '舒服', '惬意', '悠闲', '轻松', '舒适', '安逸', '自在',
|
||||
'relax', 'relaxed', 'comfortable', 'cozy', 'chill', 'peaceful'],
|
||||
'delicious': ['好吃', '美味', '香', '馋', '可口', '香甜', '大餐', '大快朵颐', '流口水', '垂涎',
|
||||
'delicious', 'yummy', 'tasty', 'yum', 'appetizing', 'mouthwatering'],
|
||||
'kissy': ['亲亲', '么么', '吻', 'mua', 'muah', '亲一下', '飞吻',
|
||||
'kiss', 'xoxo', 'hug', 'muah', 'smooch'],
|
||||
'confident': ['自信', '肯定', '确定', '毫无疑问', '当然', '必须的', '毫无疑问', '确信', '坚信',
|
||||
'confident', 'sure', 'certain', 'definitely', 'positive'],
|
||||
'sleepy': ['困', '睡觉', '晚安', '想睡', '好累', '疲惫', '疲倦', '困了', '想休息', '睡意',
|
||||
'sleep', 'sleepy', 'tired', 'exhausted', 'bedtime', 'good night'],
|
||||
'silly': ['傻', '笨', '呆', '憨', '蠢', '二', '憨憨', '傻乎乎', '呆萌',
|
||||
'silly', 'stupid', 'dumb', 'foolish', 'goofy', 'ridiculous'],
|
||||
'confused': ['疑惑', '不明白', '不懂', '困惑', '疑问', '为什么', '怎么回事', '啥意思', '不清楚',
|
||||
'confused', 'puzzled', 'doubt', 'question', 'what', 'why', 'how']
|
||||
}
|
||||
|
||||
# 特殊句型判断（中英文）
|
||||
# 赞美他人
|
||||
if any(phrase in text for phrase in
|
||||
['你真', '你好', '您真', '你真棒', '你好厉害', '你太强了', '你真好', '你真聪明',
|
||||
'you are', 'you\'re', 'you look', 'you seem', 'so smart', 'so kind']):
|
||||
return 'loving'
|
||||
# 自我赞美
|
||||
if any(phrase in text for phrase in ['我真', '我最', '我太棒了', '我厉害', '我聪明', '我优秀',
|
||||
'i am', 'i\'m', 'i feel', 'so good', 'so happy']):
|
||||
return 'cool'
|
||||
# 晚安/睡觉相关
|
||||
if any(phrase in text for phrase in ['睡觉', '晚安', '睡了', '好梦', '休息了', '去睡了',
|
||||
'sleep', 'good night', 'bedtime', 'go to bed']):
|
||||
return 'sleepy'
|
||||
# 疑问句
|
||||
if has_question and not has_exclamation:
|
||||
return 'thinking'
|
||||
# 强烈情感（感叹号）
|
||||
if has_exclamation and not has_question:
|
||||
# 检查是否是积极内容
|
||||
positive_words = emotion_keywords['happy'] + emotion_keywords['laughing'] + emotion_keywords['cool']
|
||||
if any(word in text for word in positive_words):
|
||||
return 'laughing'
|
||||
# 检查是否是消极内容
|
||||
negative_words = emotion_keywords['angry'] + emotion_keywords['sad'] + emotion_keywords['crying']
|
||||
if any(word in text for word in negative_words):
|
||||
return 'angry'
|
||||
return 'surprised'
|
||||
# 省略号（表示犹豫或思考）
|
||||
if has_ellipsis:
|
||||
return 'thinking'
|
||||
|
||||
# 关键词匹配（带权重）
|
||||
emotion_scores = {emotion: 0 for emotion in emoji_map.keys()}
|
||||
|
||||
# 给匹配到的关键词加分
|
||||
for emotion, keywords in emotion_keywords.items():
|
||||
for keyword in keywords:
|
||||
if keyword in text:
|
||||
emotion_scores[emotion] += 1
|
||||
|
||||
# 给长文本中的重复关键词额外加分
|
||||
if len(text) > 20: # 长文本
|
||||
for emotion, keywords in emotion_keywords.items():
|
||||
for keyword in keywords:
|
||||
emotion_scores[emotion] += text.count(keyword) * 0.5
|
||||
|
||||
# 根据分数选择最可能的情感
|
||||
max_score = max(emotion_scores.values())
|
||||
if max_score == 0:
|
||||
return 'happy' # 默认
|
||||
|
||||
# 可能有多个情感同分，根据上下文选择最合适的
|
||||
top_emotions = [e for e, s in emotion_scores.items() if s == max_score]
|
||||
|
||||
# 如果多个情感同分，使用以下优先级
|
||||
priority_order = [
|
||||
'laughing', 'crying', 'angry', 'surprised', 'shocked', # 强烈情感优先
|
||||
'loving', 'happy', 'funny', 'cool', # 积极情感
|
||||
'sad', 'embarrassed', 'confused', # 消极情感
|
||||
'thinking', 'winking', 'relaxed', # 中性情感
|
||||
'delicious', 'kissy', 'confident', 'sleepy', 'silly' # 特殊场景
|
||||
]
|
||||
|
||||
for emotion in priority_order:
|
||||
if emotion in top_emotions:
|
||||
return emotion
|
||||
|
||||
return top_emotions[0] # 如果都不在优先级列表里，返回第一个
|
||||
@@ -27,18 +27,6 @@ class WebSocketServer:
|
||||
host = server_config.get("ip", "0.0.0.0")
|
||||
port = int(server_config.get("port", 8000))
|
||||
|
||||
self.logger.bind(tag=TAG).info(
|
||||
"Server is running at ws://{}:{}/xiaozhi/v1/", get_local_ip(), port
|
||||
)
|
||||
self.logger.bind(tag=TAG).info(
|
||||
"=======上面的地址是websocket协议地址，请勿用浏览器访问======="
|
||||
)
|
||||
self.logger.bind(tag=TAG).info(
|
||||
"如想测试websocket请用谷歌浏览器打开test目录下的test_page.html"
|
||||
)
|
||||
self.logger.bind(tag=TAG).info(
|
||||
"=============================================================\n"
|
||||
)
|
||||
async with websockets.serve(self._handle_connection, host, port):
|
||||
await asyncio.Future()
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ services:
|
||||
ports:
|
||||
# ws服务端
|
||||
- "8000:8000"
|
||||
# ota服务端
|
||||
- "8002:8002"
|
||||
volumes:
|
||||
# 配置文件目录
|
||||
- ./data:/opt/xiaozhi-esp32-server/data
|
||||
|
||||
@@ -153,21 +153,17 @@ def parse_weather_info(soup):
|
||||
def get_weather(conn, location: str = None, lang: str = "zh_CN"):
|
||||
api_key = conn.config["plugins"]["get_weather"]["api_key"]
|
||||
default_location = conn.config["plugins"]["get_weather"]["default_location"]
|
||||
print("用户设置的default_location为:", default_location)
|
||||
client_ip = conn.client_ip
|
||||
print("用户提供的location为:", location)
|
||||
# 优先使用用户提供的location参数
|
||||
if not location:
|
||||
# 通过客户端IP解析城市
|
||||
if client_ip:
|
||||
# 动态解析IP对应的城市信息
|
||||
ip_info = get_ip_info(client_ip, logger)
|
||||
print("解析用户IP后的城市为：", ip_info)
|
||||
location = ip_info.get("city") if ip_info and "city" in ip_info else None
|
||||
else:
|
||||
# 若IP解析失败或无IP，使用默认位置
|
||||
location = default_location
|
||||
print("解析失败，将使用默认地址", location)
|
||||
|
||||
city_info = fetch_city_info(location, api_key)
|
||||
if not city_info:
|
||||
@@ -179,8 +175,6 @@ def get_weather(conn, location: str = None, lang: str = "zh_CN"):
|
||||
return ActionResponse(Action.REQLLM, None, "请求失败")
|
||||
city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
|
||||
|
||||
print(f"当前查询的城市名称是: {city_name}")
|
||||
|
||||
weather_report = f"您查询的位置是：{city_name}\n\n当前天气: {current_abstract}\n"
|
||||
|
||||
# 添加有效的当前天气参数
|
||||
|
||||
@@ -10,7 +10,10 @@ class ToolType(Enum):
|
||||
NONE = (1, "调用完工具后，不做其他操作")
|
||||
WAIT = (2, "调用工具，等待函数返回")
|
||||
CHANGE_SYS_PROMPT = (3, "修改系统提示词，切换角色性格或职责")
|
||||
SYSTEM_CTL = (4, "系统控制，影响正常的对话流程，如退出、播放音乐等，需要传递conn参数")
|
||||
SYSTEM_CTL = (
|
||||
4,
|
||||
"系统控制，影响正常的对话流程，如退出、播放音乐等，需要传递conn参数",
|
||||
)
|
||||
IOT_CTL = (5, "IOT设备控制，需要传递conn参数")
|
||||
MCP_CLIENT = (6, "MCP客户端")
|
||||
|
||||
@@ -30,12 +33,14 @@ class Action(Enum):
|
||||
self.code = code
|
||||
self.message = message
|
||||
|
||||
|
||||
class ActionResponse:
|
||||
def __init__(self, action: Action, result, response):
|
||||
self.action = action # 动作类型
|
||||
self.result = result # 动作产生的结果
|
||||
self.response = response # 直接回复的内容
|
||||
|
||||
|
||||
class FunctionItem:
|
||||
def __init__(self, name, description, func, type):
|
||||
self.name = name
|
||||
@@ -43,40 +48,49 @@ class FunctionItem:
|
||||
self.func = func
|
||||
self.type = type
|
||||
|
||||
|
||||
class DeviceTypeRegistry:
|
||||
"""设备类型注册表，用于管理IOT设备类型及其函数"""
|
||||
|
||||
def __init__(self):
|
||||
self.type_functions = {} # type_signature -> {func_name: FunctionItem}
|
||||
|
||||
|
||||
def generate_device_type_id(self, descriptor):
|
||||
"""通过设备能力描述生成类型ID"""
|
||||
properties = sorted(descriptor["properties"].keys())
|
||||
methods = sorted(descriptor["methods"].keys())
|
||||
# 使用属性和方法的组合作为设备类型的唯一标识
|
||||
type_signature = f"{descriptor['name']}:{','.join(properties)}:{','.join(methods)}"
|
||||
type_signature = (
|
||||
f"{descriptor['name']}:{','.join(properties)}:{','.join(methods)}"
|
||||
)
|
||||
return type_signature
|
||||
|
||||
|
||||
def get_device_functions(self, type_id):
|
||||
"""获取设备类型对应的所有函数"""
|
||||
return self.type_functions.get(type_id, {})
|
||||
|
||||
|
||||
def register_device_type(self, type_id, functions):
|
||||
"""注册设备类型及其函数"""
|
||||
if type_id not in self.type_functions:
|
||||
self.type_functions[type_id] = functions
|
||||
|
||||
|
||||
# 初始化函数注册字典
|
||||
all_function_registry = {}
|
||||
device_type_registry = DeviceTypeRegistry()
|
||||
|
||||
|
||||
def register_function(name, desc, type=None):
|
||||
"""注册函数到函数注册字典的装饰器"""
|
||||
|
||||
def decorator(func):
|
||||
all_function_registry[name] = FunctionItem(name, desc, func, type)
|
||||
logger.bind(tag=TAG).debug(f"函数 '{name}' 已加载，可以注册使用")
|
||||
return func
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
class FunctionRegistry:
|
||||
def __init__(self):
|
||||
self.function_registry = {}
|
||||
@@ -89,9 +103,9 @@ class FunctionRegistry:
|
||||
self.logger.bind(tag=TAG).error(f"函数 '{name}' 未找到")
|
||||
return None
|
||||
self.function_registry[name] = func
|
||||
self.logger.bind(tag=TAG).info(f"函数 '{name}' 注册成功")
|
||||
self.logger.bind(tag=TAG).debug(f"函数 '{name}' 注册成功")
|
||||
return func
|
||||
|
||||
|
||||
def unregister_function(self, name):
|
||||
# 注销函数，检测是否存在
|
||||
if name not in self.function_registry:
|
||||
@@ -103,9 +117,9 @@ class FunctionRegistry:
|
||||
|
||||
def get_function(self, name):
|
||||
return self.function_registry.get(name)
|
||||
|
||||
|
||||
def get_all_functions(self):
|
||||
return self.function_registry
|
||||
|
||||
|
||||
def get_all_function_desc(self):
|
||||
return [func.description for _, func in self.function_registry.items()]
|
||||
return [func.description for _, func in self.function_registry.items()]
|
||||
|
||||
@@ -446,7 +446,7 @@
|
||||
<h2>
|
||||
设备配置
|
||||
<span class="device-info">
|
||||
<span>MAC: <strong id="displayMac">00:11:22:33:44:55</strong></span>
|
||||
<span>MAC: <strong id="displayMac"></strong></span>
|
||||
<span>客户端: <strong id="displayClient">web_test_client</strong></span>
|
||||
</span>
|
||||
<button class="toggle-button" id="toggleConfig">编辑</button>
|
||||
@@ -455,7 +455,7 @@
|
||||
<div class="control-panel">
|
||||
<div class="config-item">
|
||||
<label for="deviceMac">设备MAC:</label>
|
||||
<input type="text" id="deviceMac" value="00:11:22:33:44:55" placeholder="设备MAC地址">
|
||||
<input type="text" id="deviceMac" placeholder="设备MAC地址">
|
||||
</div>
|
||||
<div class="config-item">
|
||||
<label for="deviceName">设备名称:</label>
|
||||
@@ -1582,6 +1582,19 @@
|
||||
}
|
||||
}
|
||||
|
||||
// 生成随机MAC地址
|
||||
function generateRandomMac() {
|
||||
const hexDigits = '0123456789ABCDEF';
|
||||
let mac = '';
|
||||
for (let i = 0; i < 6; i++) {
|
||||
if (i > 0) mac += ':';
|
||||
for (let j = 0; j < 2; j++) {
|
||||
mac += hexDigits.charAt(Math.floor(Math.random() * 16));
|
||||
}
|
||||
}
|
||||
return mac;
|
||||
}
|
||||
|
||||
// 初始化事件监听器
|
||||
function initEventListeners() {
|
||||
connectButton.addEventListener('click', connectToServer);
|
||||
@@ -1595,10 +1608,22 @@
|
||||
const displayMac = document.getElementById('displayMac');
|
||||
const displayClient = document.getElementById('displayClient');
|
||||
|
||||
// 从localStorage加载MAC地址，如果没有则生成新的
|
||||
let savedMac = localStorage.getItem('deviceMac');
|
||||
if (!savedMac) {
|
||||
savedMac = generateRandomMac();
|
||||
localStorage.setItem('deviceMac', savedMac);
|
||||
}
|
||||
deviceMacInput.value = savedMac;
|
||||
displayMac.textContent = savedMac;
|
||||
|
||||
// 更新显示的值
|
||||
function updateDisplayValues() {
|
||||
displayMac.textContent = deviceMacInput.value;
|
||||
const newMac = deviceMacInput.value;
|
||||
displayMac.textContent = newMac;
|
||||
displayClient.textContent = clientIdInput.value;
|
||||
// 保存MAC地址到localStorage
|
||||
localStorage.setItem('deviceMac', newMac);
|
||||
}
|
||||
|
||||
// 监听输入变化
|
||||
|
||||
Reference in New Issue
Block a user