update:test迁移重命名为digital-human
@@ -1,8 +1,8 @@
|
|||||||
# Test 页面语音唤醒
|
# 数字人唤醒词教程
|
||||||
|
|
||||||
## 概述
|
## 概述
|
||||||
|
|
||||||
Test 页面集成了基于 **Sherpa-ONNX** 的高精度语音唤醒功能,支持自定义唤醒词和实时检测。使用轻量级关键词检测模型,提供毫秒级响应速度。
|
测试页面集成了基于 **Sherpa-ONNX** 的高精度语音唤醒功能,支持自定义唤醒词和实时检测。使用轻量级关键词检测模型,提供毫秒级响应速度。
|
||||||
|
|
||||||
## 唤醒词模型
|
## 唤醒词模型
|
||||||
|
|
||||||
@@ -21,7 +21,7 @@ Test 页面集成了基于 **Sherpa-ONNX** 的高精度语音唤醒功能,支
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 方法1:直接下载(推荐)
|
# 方法1:直接下载(推荐)
|
||||||
cd main/xiaozhi-server/test/wakeword_runtime/
|
cd main/digital-human/wakeword_runtime/
|
||||||
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/kws-models/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01.tar.bz2
|
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/kws-models/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01.tar.bz2
|
||||||
|
|
||||||
# 解压
|
# 解压
|
||||||
@@ -42,13 +42,13 @@ snapshot_download('pkufool/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01
|
|||||||
```
|
```
|
||||||
sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01/
|
sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01/
|
||||||
├── encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
|
├── encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
|
||||||
├── encoder-epoch-12-avg-2-chunk-16-left-64.onnx #
|
├── encoder-epoch-12-avg-2-chunk-16-left-64.onnx
|
||||||
├── encoder-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
|
├── encoder-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
|
||||||
├── encoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
├── encoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
||||||
├── decoder-epoch-12-avg-2-chunk-16-left-64.onnx #
|
├── decoder-epoch-12-avg-2-chunk-16-left-64.onnx
|
||||||
├── decoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
├── decoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
||||||
├── joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
|
├── joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
|
||||||
├── joiner-epoch-12-avg-2-chunk-16-left-64.onnx #
|
├── joiner-epoch-12-avg-2-chunk-16-left-64.onnx
|
||||||
├── joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
|
├── joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
|
||||||
├── joiner-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
├── joiner-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
|
||||||
├── tokens.txt # Token映射表(必需)
|
├── tokens.txt # Token映射表(必需)
|
||||||
@@ -71,12 +71,12 @@ mkdir -p ../models
|
|||||||
|
|
||||||
# 复制精度优先的epoch-99 fp32三件套
|
# 复制精度优先的epoch-99 fp32三件套
|
||||||
cp encoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/encoder.onnx
|
cp encoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/encoder.onnx
|
||||||
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
|
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
|
||||||
cp joiner-epoch-99-avg-1-chunk-16-left-64.onnx ../models/joiner.onnx
|
cp joiner-epoch-99-avg-1-chunk-16-left-64.onnx ../models/joiner.onnx
|
||||||
|
|
||||||
# 复制配套文件
|
# 复制配套文件
|
||||||
cp tokens.txt ../models/tokens.txt
|
cp tokens.txt ../models/tokens.txt
|
||||||
# keywords_raw.txt 如果模型包里附带,可自行保留;test runtime 不依赖它
|
# keywords_raw.txt 如果模型包里附带,可自行保留;runtime 不依赖它
|
||||||
```
|
```
|
||||||
|
|
||||||
**方案二:速度优先**
|
**方案二:速度优先**
|
||||||
@@ -92,42 +92,42 @@ cp encoder-epoch-99-avg-1-chunk-16-left-64.int8.onnx ../models/encoder.onnx
|
|||||||
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
|
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
|
||||||
cp joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx ../models/joiner.onnx
|
cp joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx ../models/joiner.onnx
|
||||||
|
|
||||||
# 复制配套文件
|
# 复制配套文件
|
||||||
cp tokens.txt ../models/tokens.txt
|
cp tokens.txt ../models/tokens.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
**注意事项**:
|
**注意事项**:
|
||||||
|
|
||||||
- **不要混用fp32与int8**:三个模型文件必须保持一致的精度
|
- **不要混用 fp32 与 int8**:三个模型文件必须保持一致的精度
|
||||||
- **优先选择epoch-99**:比epoch-12训练更充分,精度更高
|
- **优先选择 epoch-99**:比 epoch-12 训练更充分,精度更高
|
||||||
- **必需文件**:`encoder.onnx` + `decoder.onnx` + `joiner.onnx` + `tokens.txt` + `keywords.txt`
|
- **必需文件**:`encoder.onnx` + `decoder.onnx` + `joiner.onnx` + `tokens.txt` + `keywords.txt`
|
||||||
|
|
||||||
### 最终模型文件结构
|
### 最终模型文件结构
|
||||||
|
|
||||||
配置完成后,模型文件应放在 `wakeword_runtime/models/` 目录下(完整路径:`main/xiaozhi-server/test/wakeword_runtime/models/`):
|
配置完成后,模型文件应放在 `wakeword_runtime/models/` 目录下,完整路径为 `main/digital-human/wakeword_runtime/models/`:
|
||||||
|
|
||||||
```
|
```
|
||||||
wakeword_runtime/models/
|
wakeword_runtime/models/
|
||||||
├── encoder.onnx # 编码器模型(重命名后)
|
├── encoder.onnx # 编码器模型(重命名后)
|
||||||
├── decoder.onnx # 解码器模型(重命名后)
|
├── decoder.onnx # 解码器模型(重命名后)
|
||||||
├── joiner.onnx # 连接器模型(重命名后)
|
├── joiner.onnx # 连接器模型(重命名后)
|
||||||
├── tokens.txt # 拼音Token映射表(228行版本)
|
├── tokens.txt # 拼音 Token 映射表(228行版本)
|
||||||
├── keywords.txt # 关键词配置文件(首次启动自动生成)
|
├── keywords.txt # 关键词配置文件(首次启动自动生成)
|
||||||
└── keywords_raw.txt # 可选,runtime 不依赖
|
└── keywords_raw.txt # 可选,runtime 不依赖
|
||||||
```
|
```
|
||||||
|
|
||||||
## 启动方式
|
## 启动方式
|
||||||
|
|
||||||
在 `main/xiaozhi-server/test` 目录执行:
|
在 `main/digital-human` 目录执行:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -r wakeword_runtime/requirements.txt
|
pip install -r wakeword_runtime/requirements.txt
|
||||||
python start_test_runtime.py
|
python start.py
|
||||||
```
|
```
|
||||||
|
|
||||||
启动后默认地址:
|
启动后默认地址:
|
||||||
|
|
||||||
- 页面地址:`http://127.0.0.1:8006/test_page.html`
|
- 页面地址:`http://127.0.0.1:8006/index.html`
|
||||||
- 事件桥地址:`ws://127.0.0.1:8006/wakeword-ws`
|
- 事件桥地址:`ws://127.0.0.1:8006/wakeword-ws`
|
||||||
- 健康检查:`http://127.0.0.1:8006/health`
|
- 健康检查:`http://127.0.0.1:8006/health`
|
||||||
|
|
||||||
@@ -136,38 +136,37 @@ python start_test_runtime.py
|
|||||||
- 在运行终端按 `Ctrl+C`
|
- 在运行终端按 `Ctrl+C`
|
||||||
- 会同时停止静态页面服务、事件桥和唤醒词检测流程
|
- 会同时停止静态页面服务、事件桥和唤醒词检测流程
|
||||||
|
|
||||||
|
|
||||||
## 配置文件说明
|
## 配置文件说明
|
||||||
|
|
||||||
配置文件位于 [config.json](./config.json)。
|
配置文件位于 [main/digital-human/wakeword_runtime/config.json](../main/digital-human/wakeword_runtime/config.json)。
|
||||||
|
|
||||||
当前主要配置项:
|
当前主要配置项:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"wakeword": {
|
"wakeword": {
|
||||||
"enabled": true
|
"enabled": true
|
||||||
},
|
},
|
||||||
"model_dir": "models",
|
"model_dir": "models",
|
||||||
"audio": {
|
"audio": {
|
||||||
"input_device": null,
|
"input_device": null,
|
||||||
"sample_rate": 16000,
|
"sample_rate": 16000,
|
||||||
"channels": 1
|
"channels": 1
|
||||||
},
|
},
|
||||||
"detector": {
|
"detector": {
|
||||||
"num_threads": 4,
|
"num_threads": 4,
|
||||||
"provider": "cpu",
|
"provider": "cpu",
|
||||||
"max_active_paths": 2,
|
"max_active_paths": 2,
|
||||||
"keywords_score": 1.8,
|
"keywords_score": 1.8,
|
||||||
"keywords_threshold": 0.1,
|
"keywords_threshold": 0.1,
|
||||||
"num_trailing_blanks": 1,
|
"num_trailing_blanks": 1,
|
||||||
"cooldown_seconds": 1.5
|
"cooldown_seconds": 1.5
|
||||||
},
|
},
|
||||||
"logging": {
|
"logging": {
|
||||||
"level": "INFO",
|
"level": "INFO",
|
||||||
"dir": "logs",
|
"dir": "logs",
|
||||||
"file": "wakeword-runtime.log"
|
"file": "wakeword-runtime.log"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -197,13 +196,13 @@ python start_test_runtime.py
|
|||||||
|
|
||||||
1. 准备 `models/` 目录下的模型文件和 `tokens.txt`
|
1. 准备 `models/` 目录下的模型文件和 `tokens.txt`
|
||||||
2. 确认 `models/keywords.txt` 存在
|
2. 确认 `models/keywords.txt` 存在
|
||||||
3. 在 `test` 目录运行 `python start_test_runtime.py`
|
3. 在 `digital-human` 目录运行 `python start.py`
|
||||||
4. 浏览器打开 `http://127.0.0.1:8006/test_page.html`
|
4. 浏览器打开 `http://127.0.0.1:8006/index.html`
|
||||||
5. 进入设置页检查“唤醒词”配置
|
5. 进入设置页检查“唤醒词”配置
|
||||||
|
|
||||||
### 修改唤醒词
|
### 修改唤醒词
|
||||||
|
|
||||||
1. 打开 test 页面设置
|
1. 打开数字人页面设置
|
||||||
2. 切到“唤醒词”页签
|
2. 切到“唤醒词”页签
|
||||||
3. 修改启用状态或唤醒词列表
|
3. 修改启用状态或唤醒词列表
|
||||||
4. 点击“应用唤醒词”
|
4. 点击“应用唤醒词”
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
本文档是开发类文档,如需部署小智服务端,[点击这里查看部署教程](../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3)
|
||||||
|
|
||||||
|
如需查看唤醒词模型下载、运行时配置和详细使用说明,[点击这里查看专题文档](../../docs/digital-human-wakeword.md)
|
||||||
|
|
||||||
|
# 项目介绍
|
||||||
|
|
||||||
|
digital-human 是独立的数字人测试模块,负责提供本地测试页面、前端交互资源、唤醒词运行时和事件桥能力,用于联调整个数字人交互链路。
|
||||||
|
|
||||||
|
# 快速启动
|
||||||
|
|
||||||
|
安装依赖:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r wakeword_runtime/requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
启动模块:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python start.py
|
||||||
|
```
|
||||||
|
|
||||||
|
# 访问地址
|
||||||
|
|
||||||
|
启动后可访问:
|
||||||
|
|
||||||
|
- 页面地址:http://127.0.0.1:8006/index.html
|
||||||
|
- 事件桥地址:ws://127.0.0.1:8006/wakeword-ws
|
||||||
|
- 健康检查:http://127.0.0.1:8006/health
|
||||||
|
|
||||||
|
# 目录说明
|
||||||
|
|
||||||
|
- `start.py`:模块启动入口
|
||||||
|
- `index.html`:数字人测试页面入口
|
||||||
|
- `wakeword_runtime`:本地唤醒词运行时与配置目录
|
||||||
|
- `js`、`css`:页面前端脚本与样式
|
||||||
|
- `images`、`resources`:页面资源文件
|
||||||
|
|
||||||
|
# 相关文档
|
||||||
|
|
||||||
|
唤醒词专题文档:[../../docs/digital-human-wakeword.md](../../docs/digital-human-wakeword.md)
|
||||||
|
Before Width: | Height: | Size: 185 B After Width: | Height: | Size: 185 B |
|
Before Width: | Height: | Size: 4.2 KiB After Width: | Height: | Size: 4.2 KiB |
|
Before Width: | Height: | Size: 136 KiB After Width: | Height: | Size: 136 KiB |
|
Before Width: | Height: | Size: 321 KiB After Width: | Height: | Size: 321 KiB |
|
Before Width: | Height: | Size: 290 KiB After Width: | Height: | Size: 290 KiB |
@@ -24,11 +24,11 @@
|
|||||||
<p>您可以使用nginx映射启动测试页面,也可以请按照以下步骤启动测试运行时:</p>
|
<p>您可以使用nginx映射启动测试页面,也可以请按照以下步骤启动测试运行时:</p>
|
||||||
<ol>
|
<ol>
|
||||||
<li>打开命令行终端</li>
|
<li>打开命令行终端</li>
|
||||||
<li>命令行进入到 xiaozhi-server/test 目录</li>
|
<li>命令行进入到 digital-human 目录</li>
|
||||||
<li>执行以下命令启动测试页面运行时:</li>
|
<li>执行以下命令启动测试页面运行时:</li>
|
||||||
</ol>
|
</ol>
|
||||||
<pre>python start_test_runtime.py</pre>
|
<pre>python start.py</pre>
|
||||||
<p>然后在浏览器中访问:<strong>http://localhost:8006/test_page.html</strong></p>
|
<p>然后在浏览器中访问:<strong>http://localhost:8006/index.html</strong></p>
|
||||||
`;
|
`;
|
||||||
document.body.appendChild(warningDiv);
|
document.body.appendChild(warningDiv);
|
||||||
});
|
});
|
||||||
|
Before Width: | Height: | Size: 1.7 MiB After Width: | Height: | Size: 1.7 MiB |
|
Before Width: | Height: | Size: 2.4 MiB After Width: | Height: | Size: 2.4 MiB |
|
Before Width: | Height: | Size: 6.6 MiB After Width: | Height: | Size: 6.6 MiB |
@@ -25,7 +25,7 @@ class TestRuntimeHttpServer:
|
|||||||
|
|
||||||
@property
|
@property
|
||||||
def page_url(self) -> str:
|
def page_url(self) -> str:
|
||||||
return f"http://127.0.0.1:{self.port}/test_page.html"
|
return f"http://127.0.0.1:{self.port}/index.html"
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def bridge_url(self) -> str:
|
def bridge_url(self) -> str:
|
||||||