Compare commits

...
69 Commits
Author SHA1 Message Date
欣南科技andGitHub b1e1a29145 Merge pull request #2446 from xinnan-tech/update_version
Bump to 0.8.6
2025-11-01 11:16:33 +08:00
hrz e59df53c99 Bump to 0.8.6 2025-11-01 11:15:58 +08:00
hrzandGitHub cdf0f91251 Merge pull request #2443 from xinnan-tech/py_fix_prompt_template
fix:修复全模块时缺少prompt_template配置的问题
2025-10-31 21:23:50 +08:00
3030332422 32ecbe55fc fix:修复全模块时缺少prompt_template配置的问题 2025-10-31 20:36:55 +08:00
hrzandGitHub cc9d5306e6 Merge pull request #2437 from xinnan-tech/py_mcp_server_fix
update:服务端MCP新增支持Streamable HTTP传输协议
2025-10-31 10:09:02 +08:00
3030332422 5149a8049a update:服务端MCP新增支持Streamable HTTP传输协议 2025-10-29 23:14:42 +08:00
欣南科技andGitHub cde093dd20 Merge pull request #2430 from xinnan-tech/mac_safe
update:默认版本不提供设备ID敏感信息,按需开启
2025-10-28 15:03:19 +08:00
hrz 455b3df5ae update:默认版本不提供设备ID敏感信息,按需开启 2025-10-28 15:02:01 +08:00
hrzandGitHub 4d14caf0d4 Merge pull request #2416 from xinnan-tech/test-page-add-mcp
添加MCP工具测试页面
2025-10-25 09:25:16 +08:00
Chingfeng Li 47b44bb124 修复未定义的方法 2025-10-24 16:01:12 +08:00
Chingfeng Li eb743ab577 从json中加载默认工具,方便自定义tools 2025-10-24 15:58:01 +08:00
Chingfeng Li d6344bde5e 添加MCP测试功能,支持自定义添加编辑MCP Tools 2025-10-24 10:17:32 +08:00
Chingfeng Li 7ce1672d98 Merge branch 'main' of https://github.com/xinnan-tech/xiaozhi-esp32-server 2025-10-24 09:02:09 +08:00
欣南科技andGitHub 4e3701e62c Merge pull request #2410 from xinnan-tech/fix-xunfei-asr
修复:讯飞asr最后一个字丢失的bug
2025-10-23 17:05:06 +08:00
hrz a035ed525a 修复:讯飞asr最后一个字丢失的bug 2025-10-23 17:04:33 +08:00
hrzandGitHub dbea7ae11e Merge pull request #2403 from xinnan-tech/main-aliyun-nls-taskid
阿里云NLS协议中message_id每次发送都必须唯一;task_id是会话id,整个请求中需要保持一致。
2025-10-23 15:12:19 +08:00
hrzandGitHub 26127e9b4a Merge pull request #2401 from xinnan-tech/py_fix_huoshan
fix: 未等待服务端残留状态问题
2025-10-23 14:32:17 +08:00
Chingfeng Li 7035a57cf1 Merge branch 'main' of https://github.com/xinnan-tech/xiaozhi-esp32-server 2025-10-23 13:05:10 +08:00
hrzandGitHub aee9bdc514 Merge pull request #2399 from xinnan-tech/main-agent-base-prompt
常量可配置:允许自定义agent-base-prompt.txt文件
2025-10-23 12:26:08 +08:00
欣南科技andGitHub 9018422ecb Merge pull request #2405 from xinnan-tech/hot-fix
update:优化文档
2025-10-23 12:24:52 +08:00
hrz da8d8c53c0 update:优化文档 2025-10-23 12:24:16 +08:00
欣南科技andGitHub 2c428a380e Merge pull request #2404 from xinnan-tech/hot-fix
update:该功能易被坏人滥用,违背开发者初心
2025-10-23 11:47:22 +08:00
hrz bde50e661d update:改功能易引发坏人滥用,违背开发者初心 2025-10-23 11:43:49 +08:00
Chingfeng Li 2beeb825cf 阿里云NLS协议中message_id每次发送都必须唯一;task_id是会话id,整个请求中需要保持一致。 2025-10-22 18:21:34 +08:00
Sakura-RanChen 371dab1282 fix: 未等待服务端残留状态问题 2025-10-22 17:26:36 +08:00
Chingfeng Li 3ba437c937 常量可配置:允许自定义agent-base-prompt.txt文件 2025-10-22 14:53:05 +08:00
hrzandGitHub 4b2d7da4e5 Merge pull request #2357 from kkroid/feature_tts-audio-send-delay
feat: 添加可配置的TTS音频发送延迟功能
2025-10-22 11:48:34 +08:00
hrzandGitHub d00592020e Update config.yaml 2025-10-22 11:47:54 +08:00
hrzandGitHub e0b24f4e59 Merge pull request #2383 from xinnan-tech/docker-server-base
构建base镜像,只有当依赖发生变化时再进行重新构建,否则只使用之前的base镜像进行构建,能够使本地使用缓存的镜像
2025-10-22 11:41:35 +08:00
欣南科技andGitHub 4b7be99837 Merge pull request #2378 from xinnan-tech/py_update_vad
update: 优化vad判断逻辑
2025-10-22 11:19:33 +08:00
hrz 99db948b96 update:恢复vad判断 2025-10-22 10:46:57 +08:00
欣南科技andGitHub 2b3e205cda Merge pull request #2397 from xinnan-tech/pip_version
Pip version
2025-10-22 09:17:29 +08:00
hrz c9b7d77094 updte:python3.10最高只支持torch==2.2.2、torchaudio==2.2.2 2025-10-22 09:16:33 +08:00
hrzandGitHub f3818167f3 Merge pull request #2380 from KurobaKaitou/main
chore: 补充在 Linux 环境下进行源码部署时出现缺失动态库的解决方案以及保留检测 ffmpeg 环境时的原有报错信息
2025-10-21 17:42:52 +08:00
hrzandGitHub 55d6c2a193 Merge pull request #2387 from xinnan-tech/dependabot/pip/main/xiaozhi-server/ormsgpack-1.11.0
build(deps): bump ormsgpack from 1.7.0 to 1.11.0 in /main/xiaozhi-server
2025-10-21 17:42:09 +08:00
hrzandGitHub 6cfb6a8f43 Merge pull request #2386 from xinnan-tech/dependabot/pip/main/xiaozhi-server/dashscope-1.24.6
build(deps): bump dashscope from 1.23.1 to 1.24.6 in /main/xiaozhi-server
2025-10-21 17:41:38 +08:00
hrzandGitHub 396c3eb18b Merge pull request #2388 from xinnan-tech/dependabot/pip/main/xiaozhi-server/openai-2.5.0
build(deps): bump openai from 1.107.0 to 2.5.0 in /main/xiaozhi-server
2025-10-21 17:41:06 +08:00
hrzandGitHub d8c4c1f207 Merge branch 'pip_version' into dependabot/pip/main/xiaozhi-server/openai-2.5.0 2025-10-21 17:40:47 +08:00
hrzandGitHub 268ce9fff3 Merge pull request #2389 from xinnan-tech/dependabot/pip/main/xiaozhi-server/torchaudio-2.9.0
build(deps): bump torchaudio from 2.2.2 to 2.9.0 in /main/xiaozhi-server
2025-10-21 17:39:12 +08:00
hrzandGitHub bbc3b44336 Merge pull request #2390 from xinnan-tech/dependabot/pip/main/xiaozhi-server/mem0ai-1.0.0
build(deps): bump mem0ai from 0.1.62 to 1.0.0 in /main/xiaozhi-server
2025-10-21 17:38:48 +08:00
HanabiandGitHub fb223984d3 Merge branch 'xinnan-tech:main' into main 2025-10-21 00:52:25 +08:00
欣南科技andGitHub 9f21711cae Merge pull request #2392 from xinnan-tech/hot-fix
update:修复manager-api错误码重复问题
2025-10-20 23:39:43 +08:00
hrz c139701468 update:修复manager-api错误码重复问题 2025-10-20 23:38:59 +08:00
欣南科技andGitHub 66d3bcfa13 Merge pull request #2379 from xinnan-tech/main-fix-auth_key
如果已经配置了auth_key,则不使用uuid生成; 适配单模块启动,防止每次服务重启设备都无法连接问题
2025-10-20 22:19:10 +08:00
hrzandGitHub ff048797a1 auth_key优先级:配置文件server.auth_key > manager-api.secret > 自动生成 2025-10-20 21:25:37 +08:00
dependabot[bot]andGitHub ad6607b7f5 build(deps): bump mem0ai from 0.1.62 to 1.0.0 in /main/xiaozhi-server
Bumps [mem0ai](https://github.com/mem0ai/mem0) from 0.1.62 to 1.0.0.
- [Release notes](https://github.com/mem0ai/mem0/releases)
- [Changelog](https://github.com/mem0ai/mem0/blob/main/docs/changelog.mdx)
- [Commits](https://github.com/mem0ai/mem0/compare/0.1.62...v1.0.0)

---
updated-dependencies:
- dependency-name: mem0ai
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-20 13:19:42 +00:00
dependabot[bot]andGitHub c992ed48da build(deps): bump torchaudio from 2.2.2 to 2.9.0 in /main/xiaozhi-server
Bumps [torchaudio](https://github.com/pytorch/audio) from 2.2.2 to 2.9.0.
- [Release notes](https://github.com/pytorch/audio/releases)
- [Commits](https://github.com/pytorch/audio/compare/v2.2.2...v2.9.0)

---
updated-dependencies:
- dependency-name: torchaudio
  dependency-version: 2.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-20 13:19:35 +00:00
HanabiandGitHub bdf9cda4c5 Merge branch 'xinnan-tech:main' into main 2025-10-20 20:23:54 +08:00
dependabot[bot]andGitHub c85ffcc181 build(deps): bump openai from 1.107.0 to 2.5.0 in /main/xiaozhi-server
Bumps [openai](https://github.com/openai/openai-python) from 1.107.0 to 2.5.0.
- [Release notes](https://github.com/openai/openai-python/releases)
- [Changelog](https://github.com/openai/openai-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/openai/openai-python/compare/v1.107.0...v2.5.0)

---
updated-dependencies:
- dependency-name: openai
  dependency-version: 2.5.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-20 11:56:58 +00:00
dependabot[bot]andGitHub c53e3ed965 build(deps): bump ormsgpack from 1.7.0 to 1.11.0 in /main/xiaozhi-server
Bumps [ormsgpack](https://github.com/aviramha/ormsgpack) from 1.7.0 to 1.11.0.
- [Release notes](https://github.com/aviramha/ormsgpack/releases)
- [Changelog](https://github.com/aviramha/ormsgpack/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aviramha/ormsgpack/compare/1.7.0...1.11.0)

---
updated-dependencies:
- dependency-name: ormsgpack
  dependency-version: 1.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-20 11:51:52 +00:00
dependabot[bot]andGitHub d374b77baa build(deps): bump dashscope in /main/xiaozhi-server
Bumps [dashscope](https://dashscope.aliyun.com/) from 1.23.1 to 1.24.6.

---
updated-dependencies:
- dependency-name: dashscope
  dependency-version: 1.24.6
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-20 11:51:47 +00:00
hrzandGitHub f3b2339803 Merge pull request #2384 from xinnan-tech/fix-dependabot
修正检测周期
2025-10-20 18:47:30 +08:00
hrzandGitHub 27ed540113 Merge pull request #2385 from xinnan-tech/fixmqtt
fix:修复单模块mqtt下发错误
2025-10-20 18:47:05 +08:00
FAN-yeB 434fba55f7 fix:修复单模块下发逻辑 2025-10-20 18:21:12 +08:00
Chingfeng Li 280c04e753 修正检测周期 2025-10-20 15:54:50 +08:00
Chingfeng Li 24832822a5 默认main分支 2025-10-20 15:11:23 +08:00
Chingfeng Li a357f0818d 依赖baseAction 2025-10-20 15:10:19 +08:00
Chingfeng Li e4bf5d2d21 构建base镜像,只有当依赖发生变化时再进行重新构建,否则只使用之前的base镜像进行构建,能够使本地使用缓存的镜像 2025-10-20 15:07:02 +08:00
HanabiandGitHub 8b261d473a fix: 补充README文档 334行的 ffmpeg 单词缺失
补充README文档 334行的 ffmpeg 单词缺失
2025-10-20 12:53:40 +08:00
HanabiandGitHub bba84382ec docs: 补充源码部署时安装 ffmpeg 环境可能会出现缺失动态库的解决方案
补充说明
2025-10-20 12:32:09 +08:00
HanabiandGitHub d081803434 chore(util): 优化项目启动时检测ffmpeg环境时在某些场景下的提示
此次变动主要针对 Linux (Ubuntu 22.04) 环境下根据官方提供的README文档以及conda安装 ffmpeg 环境后出现缺失动态库libiconv.so.2的问题 报错完整信息如下:
`ffmpeg: error while loading shared libraries: libiconv.so.2: cannot open shared object file: No such file or directory` 基于以上报错信息 优化了util.py中的check_ffmpeg_installed函数
- 保留了原有命令行的提示信息 [定位问题更加友好]
- 抓取报错中的 libiconv.so.2 文本内容 进行提示用户通过conda安装 libiconv 动态库
2025-10-20 12:28:22 +08:00
Chingfeng Li 9194ffa6b7 如果已经配置了auth_key,则不使用uuid生成; 适配单模块启动,防止每次服务重启设备都无法连接问题 2025-10-20 11:57:18 +08:00
Sakura-RanChen c06a7b1db6 update: 优化vad判断逻辑 2025-10-20 11:45:50 +08:00
欣南科技andGitHub 1736e10189 Merge pull request #2374 from xinnan-tech/myifeng-patch-1
更新 dependabot.yml
2025-10-20 11:37:55 +08:00
hrzandGitHub 43b79c50a0 每个月更新组件版本 2025-10-20 11:35:31 +08:00
欣南科技andGitHub 2e6aced1bd Merge pull request #2377 from xinnan-tech/update-doc
fix:克隆教程文字错误
2025-10-20 10:17:55 +08:00
hrz 2dc80c8432 fix:克隆教程文字错误 2025-10-20 10:16:10 +08:00
myifengandGitHub 01e7a7d3b8 更新 dependabot.yml 2025-10-19 16:06:44 +08:00
will.zhang 3b8bbb5c5f feat: 添加可配置的TTS音频发送延迟功能
- 修改 sendAudio() 函数,新增 tts_audio_send_delay 配置参数(毫秒)
- 0: 使用原有逻辑
- 大于0: 使用固定延迟发送(降低首包延迟,提升响应速度)
2025-10-14 18:30:17 +08:00
47 changed files with 1431 additions and 2108 deletions
+6
View File
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: pip
directory: /main/xiaozhi-server
schedule:
interval: weekly
+49
View File
@@ -0,0 +1,49 @@
name: Build Base Image
on:
push:
branches:
- main
paths:
- 'main/xiaozhi-server/requirements.txt'
- 'Dockerfile-server-base'
- '.github/workflows/build-base-image.yml'
jobs:
build-base:
name: Build and push server base image
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.TOKEN }}
- name: Build and push server-base
uses: docker/build-push-action@v6
with:
context: .
file: Dockerfile-server-base
push: true
tags: ghcr.io/${{ github.repository }}:server-base
platforms: linux/amd64
cache-from: type=gha,scope=server-base
cache-to: type=gha,mode=max,scope=server-base
build-args: |
BUILDKIT_PROGRESS=plain
- name: Output image info
run: |
echo "✅ Base image built and pushed successfully!"
echo "📦 Tag: ghcr.io/${{ github.repository }}:server-base"
+4
View File
@@ -5,6 +5,10 @@ on:
tags: tags:
- 'v*.*.*' # 只在以 v 开头的标签推送时触发,例如 v1.0.0 - 'v*.*.*' # 只在以 v 开头的标签推送时触发,例如 v1.0.0
workflow_dispatch: workflow_dispatch:
workflow_run:
workflows: ["Build Base Image"]
types:
- completed
jobs: jobs:
release: release:
+2 -32
View File
@@ -1,36 +1,6 @@
# 第一阶段:构建Python依赖 # 生产镜像,仅包含应用代码
FROM python:3.10-slim AS builder FROM ghcr.io/xinnan-tech/xiaozhi-esp32-server:server-base
WORKDIR /app
# 配置pip使用国内镜像源(阿里云)并设置超时和重试
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && \
pip config set global.trusted-host mirrors.aliyun.com && \
pip config set global.timeout 120 && \
pip config set install.retries 5
COPY main/xiaozhi-server/requirements.txt .
# 安装Python依赖,使用并行下载
RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \
pip install --no-cache-dir -r requirements.txt --default-timeout=120 --retries 5
# 第二阶段:生产镜像
FROM python:3.10-slim
WORKDIR /opt/xiaozhi-esp32-server
# 安装系统依赖
RUN apt-get update && \
apt-get install -y --no-install-recommends libopus0 ffmpeg && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 从构建阶段复制Python包和前端构建产物
COPY --from=builder /usr/local/lib/python3.10/site-packages /usr/local/lib/python3.10/site-packages
COPY --from=builder /usr/local/bin/mcp-proxy /usr/local/bin/mcp-proxy
# 复制应用代码
COPY main/xiaozhi-server . COPY main/xiaozhi-server .
# 启动应用 # 启动应用
+24
View File
@@ -0,0 +1,24 @@
# Dockerfile-server-base
# 基础镜像,包含系统依赖和Python包
FROM python:3.10-slim
# 安装系统依赖
RUN apt-get update && \
apt-get install -y --no-install-recommends libopus0 ffmpeg && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 配置pip使用国内镜像源(阿里云)并设置超时和重试
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && \
pip config set global.trusted-host mirrors.aliyun.com && \
pip config set global.timeout 120 && \
pip config set install.retries 5
WORKDIR /opt/xiaozhi-esp32-server
# 复制requirements.txt
COPY main/xiaozhi-server/requirements.txt .
# 安装Python依赖
RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \
pip install --no-cache-dir -r requirements.txt --default-timeout=120 --retries 5
+11 -5
View File
@@ -139,6 +139,9 @@ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/
conda install libopus -y conda install libopus -y
conda install ffmpeg -y conda install ffmpeg -y
# 在 Linux 环境下进行部署时,如出现类似缺失 libiconv.so.2 动态库的报错 请通过以下命令进行安装
conda install libiconv -y
``` ```
请注意,以上命令,不是一股脑执行就成功的,你需要一步步执行,每一步执行完后,都检查一下输出的日志,查看是否成功。 请注意,以上命令,不是一股脑执行就成功的,你需要一步步执行,每一步执行完后,都检查一下输出的日志,查看是否成功。
@@ -267,7 +270,8 @@ LLM:
6、[我说话很慢,停顿时小智老是抢话](./FAQ.md)<br/> 6、[我说话很慢,停顿时小智老是抢话](./FAQ.md)<br/>
## 部署相关教程 ## 部署相关教程
1、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/> 1、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/>
2、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/> 2、[如何部署MQTT网关开启MQTT+UDP协议](./mqtt-gateway-integration.md)<br/>
3、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/>
## 拓展相关教程 ## 拓展相关教程
1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/> 1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/>
2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/> 2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/>
@@ -275,11 +279,13 @@ LLM:
4、[如何部署MCP接入点](./mcp-endpoint-enable.md)<br/> 4、[如何部署MCP接入点](./mcp-endpoint-enable.md)<br/>
5、[如何接入MCP接入点](./mcp-endpoint-integration.md)<br/> 5、[如何接入MCP接入点](./mcp-endpoint-integration.md)<br/>
6、[如何开启声纹识别](./voiceprint-integration.md)<br/> 6、[如何开启声纹识别](./voiceprint-integration.md)<br/>
10、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/> 7、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/>
8、[天气插件使用指南](./weather-integration.md)<br/>
## 语音克隆、本地语音部署相关教程 ## 语音克隆、本地语音部署相关教程
1、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/> 1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/>
2、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/> 2、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/>
3、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/> 3、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/>
4、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/>
## 性能测试教程 ## 性能测试教程
1、[各组件速度测试指南](./performance_tester.md)<br/> 1、[各组件速度测试指南](./performance_tester.md)<br/>
2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/> 2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/>
+9 -4
View File
@@ -355,6 +355,9 @@ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/
conda install libopus -y conda install libopus -y
conda install ffmpeg -y conda install ffmpeg -y
# 在 Linux 环境下进行部署时,如出现类似缺失 libiconv.so.2 动态库的报错 请通过以下命令进行安装
conda install libiconv -y
``` ```
请注意,以上命令,不是一股脑执行就成功的,你需要一步步执行,每一步执行完后,都检查一下输出的日志,查看是否成功。 请注意,以上命令,不是一股脑执行就成功的,你需要一步步执行,每一步执行完后,都检查一下输出的日志,查看是否成功。
@@ -468,7 +471,8 @@ ws://你电脑局域网的ip:8000/xiaozhi/v1/
6、[我说话很慢,停顿时小智老是抢话](./FAQ.md)<br/> 6、[我说话很慢,停顿时小智老是抢话](./FAQ.md)<br/>
## 部署相关教程 ## 部署相关教程
1、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/> 1、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/>
2、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/> 2、[如何部署MQTT网关开启MQTT+UDP协议](./mqtt-gateway-integration.md)<br/>
3、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/>
## 拓展相关教程 ## 拓展相关教程
1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/> 1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/>
2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/> 2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/>
@@ -479,9 +483,10 @@ ws://你电脑局域网的ip:8000/xiaozhi/v1/
7、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/> 7、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/>
8、[天气插件使用指南](./weather-integration.md)<br/> 8、[天气插件使用指南](./weather-integration.md)<br/>
## 语音克隆、本地语音部署相关教程 ## 语音克隆、本地语音部署相关教程
1、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/> 1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/>
2、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/> 2、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/>
3、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/> 3、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/>
4、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/>
## 性能测试教程 ## 性能测试教程
1、[各组件速度测试指南](./performance_tester.md)<br/> 1、[各组件速度测试指南](./performance_tester.md)<br/>
2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/> 2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/>
+10 -9
View File
@@ -62,30 +62,31 @@ VAD:
### 7、部署相关教程 ### 7、部署相关教程
1、[如何进行最简化部署](./Deployment.md)<br/> 1、[如何进行最简化部署](./Deployment.md)<br/>
2、[如何进行全模块部署](./Deployment_all.md)<br/> 2、[如何进行全模块部署](./Deployment_all.md)<br/>
2、[如何部署MQTT网关开启MQTT+UDP协议](./mqtt-gateway-integration.md)<br/> 3、[如何部署MQTT网关开启MQTT+UDP协议](./mqtt-gateway-integration.md)<br/>
3、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/> 4、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/>
4、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/> 5、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/>
### 8、编译固件相关教程 ### 9、编译固件相关教程
1、[如何自己编译小智固件](./firmware-build.md)<br/> 1、[如何自己编译小智固件](./firmware-build.md)<br/>
2、[如何基于虾哥编译好的固件修改OTA地址](./firmware-setting.md)<br/> 2、[如何基于虾哥编译好的固件修改OTA地址](./firmware-setting.md)<br/>
### 8、拓展相关教程 ### 10、拓展相关教程
1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/> 1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/>
2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/> 2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/>
3、[如何开启视觉模型实现拍照识物](./mcp-vision-integration.md)<br/> 3、[如何开启视觉模型实现拍照识物](./mcp-vision-integration.md)<br/>
4、[如何部署MCP接入点](./mcp-endpoint-enable.md)<br/> 4、[如何部署MCP接入点](./mcp-endpoint-enable.md)<br/>
5、[如何接入MCP接入点](./mcp-endpoint-integration.md)<br/> 5、[如何接入MCP接入点](./mcp-endpoint-integration.md)<br/>
6、[如何开启声纹识别](./voiceprint-integration.md)<br/> 6、[MCP方法如何获取设备信息](./mcp-get-device-info.md)<br/>
10、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/> 7、[如何开启声纹识别](./voiceprint-integration.md)<br/>
8、[新闻插件源配置指南](./newsnow_plugin_config.md)<br/>
### 9、语音克隆、本地语音部署相关教程 ### 11、语音克隆、本地语音部署相关教程
1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/> 1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/>
2、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/> 2、[如何部署集成index-tts本地语音](./index-stream-integration.md)<br/>
3、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/> 3、[如何部署集成fish-speech本地语音](./fish-speech-integration.md)<br/>
4、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/> 4、[如何部署集成PaddleSpeech本地语音](./paddlespeech-deploy.md)<br/>
### 10、性能测试教程 ### 12、性能测试教程
1、[各组件速度测试指南](./performance_tester.md)<br/> 1、[各组件速度测试指南](./performance_tester.md)<br/>
2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/> 2、[定期公开测试结果](https://github.com/xinnan-tech/xiaozhi-performance-research)<br/>
+2 -2
View File
@@ -1,6 +1,6 @@
# 智控台 火山双流式语音合成+音色克隆配置教程 # 智控台 火山双流式语音合成+音色克隆配置教程
本教程分为3个阶段:准备阶段、配置阶段、克隆阶段、使用阶段。主要是介绍通过智控台配置火山双流式语音合成+音色克隆的过程。 本教程分为4个阶段:准备阶段、配置阶段、克隆阶段、使用阶段。主要是介绍通过智控台配置火山双流式语音合成+音色克隆的过程。
## 第一阶段:准备阶段 ## 第一阶段:准备阶段
超级管理员先预先把火山引擎服务开通好,获取到App Id,Access Token。默认火上引擎会赠送一个音色资源。这个音色资源需要把它复制到本项目里。 超级管理员先预先把火山引擎服务开通好,获取到App Id,Access Token。默认火上引擎会赠送一个音色资源。这个音色资源需要把它复制到本项目里。
@@ -46,7 +46,7 @@
如果复刻成功,在列表里会看到对应的音色会变成“训练成功”状态。此时你可以点击【声音名称】栏的修改按钮,修改音色资源的名称,方便后期选择使用。 如果复刻成功,在列表里会看到对应的音色会变成“训练成功”状态。此时你可以点击【声音名称】栏的修改按钮,修改音色资源的名称,方便后期选择使用。
## 第阶段:使用阶段 ## 第阶段:使用阶段
点击顶部【智能体管理】,选择任意一个智能体,点击【配置角色】按钮。 点击顶部【智能体管理】,选择任意一个智能体,点击【配置角色】按钮。
+40
View File
@@ -0,0 +1,40 @@
# MCP 方法如何获取设备信息
本教程将指导你如何使用MCP方法获取设备信息。
第一步:自定义你的`agent-base-prompt.txt`文件
把xiaozhi-server目录的`agent-base-prompt.txt`文件内容复制到你的`data`目录下,并重命名为`.agent-base-prompt.txt`
第二步:修改`data/.agent-base-prompt.txt`文件,找到`<context>`标签,在标签内容中添加以下代码内容:
```
- **设备ID** {{device_id}}
```
添加完成后,你的`data/.agent-base-prompt.txt`文件的`<context>`标签内容大致如下:
```
<context>
【重要!以下信息已实时提供,无需调用工具查询,请直接使用:】
- **设备ID** {{device_id}}
- **当前时间:** {{current_time}}
- **今天日期:** {{today_date}} ({{today_weekday}})
- **今天农历:** {{lunar_date}}
- **用户所在城市:** {{local_address}}
- **当地未来7天天气:** {{weather_info}}
</context>
```
第三步:修改`data/.config.yaml`文件,找到`agent-base-prompt`配置,修改前内容如下:
```
prompt_template: agent-base-prompt.txt
```
修改成
```
prompt_template: data/.agent-base-prompt.txt
```
第四步:重启你的xiaozhi-server服务。
第五步:在你的mcp方法增加名称为`device_id`,类型为`string`,描述为`设备ID`的参数。
第六步:重新唤醒小智,让他调用mcp方法,查看你的mcp方法是否可以获取`设备ID`
@@ -294,7 +294,7 @@ public interface Constant {
/** /**
* 版本号 * 版本号
*/ */
public static final String VERSION = "0.8.5"; public static final String VERSION = "0.8.6";
/** /**
* 无效固件URL * 无效固件URL
@@ -82,14 +82,6 @@ public interface ErrorCode {
int DEVICE_ALREADY_ACTIVATED = 10063; int DEVICE_ALREADY_ACTIVATED = 10063;
// 默认模型删除错误 // 默认模型删除错误
int DEFAULT_MODEL_DELETE_ERROR = 10064; int DEFAULT_MODEL_DELETE_ERROR = 10064;
// 设备相关错误码
int MAC_ADDRESS_ALREADY_EXISTS = 10090; // Mac地址已存在
// 模型相关错误码
int MODEL_PROVIDER_NOT_EXIST = 10091; // 供应器不存在
int LLM_NOT_EXIST = 10092; // 设置的LLM不存在
int MODEL_REFERENCED_BY_AGENT = 10093; // 该模型配置已被智能体引用,无法删除
int LLM_REFERENCED_BY_INTENT = 10094; // 该LLM模型已被意图识别配置引用,无法删除
// 登录相关错误码 // 登录相关错误码
int ADD_DATA_FAILED = 10065; // 新增数据失败 int ADD_DATA_FAILED = 10065; // 新增数据失败
int UPDATE_DATA_FAILED = 10066; // 修改数据失败 int UPDATE_DATA_FAILED = 10066; // 修改数据失败
@@ -127,6 +119,14 @@ public interface ErrorCode {
int VOICEPRINT_UNREGISTER_PROCESS_ERROR = 10090; // 声纹注销处理失败 int VOICEPRINT_UNREGISTER_PROCESS_ERROR = 10090; // 声纹注销处理失败
int VOICEPRINT_IDENTIFY_REQUEST_ERROR = 10091; // 声纹识别请求失败 int VOICEPRINT_IDENTIFY_REQUEST_ERROR = 10091; // 声纹识别请求失败
// 设备相关错误码
int MAC_ADDRESS_ALREADY_EXISTS = 10161; // Mac地址已存在
// 模型相关错误码
int MODEL_PROVIDER_NOT_EXIST = 10162; // 供应器不存在
int LLM_NOT_EXIST = 10092; // 设置的LLM不存在
int MODEL_REFERENCED_BY_AGENT = 10093; // 该模型配置已被智能体引用,无法删除
int LLM_REFERENCED_BY_INTENT = 10094; // 该LLM模型已被意图识别配置引用,无法删除
// 服务端管理相关错误码 // 服务端管理相关错误码
int INVALID_SERVER_ACTION = 10095; // 无效服务端操作 int INVALID_SERVER_ACTION = 10095; // 无效服务端操作
int SERVER_WEBSOCKET_NOT_CONFIGURED = 10096; // 未配置服务端WebSocket地址 int SERVER_WEBSOCKET_NOT_CONFIGURED = 10096; // 未配置服务端WebSocket地址
@@ -93,7 +93,7 @@ public class DeviceController {
} }
@PostMapping("/bind/{agentId}") @PostMapping("/bind/{agentId}")
@Operation(summary = "转发POST请求到MQTT网关") @Operation(summary = "设备在线接口")
@RequiresPermissions("sys:role:normal") @RequiresPermissions("sys:role:normal")
public Result<String> forwardToMqttGateway(@PathVariable String agentId, @RequestBody String requestBody) { public Result<String> forwardToMqttGateway(@PathVariable String agentId, @RequestBody String requestBody) {
try { try {
@@ -208,63 +208,4 @@ public class DeviceController {
deviceService.manualAddDevice(user.getId(), dto); deviceService.manualAddDevice(user.getId(), dto);
return new Result<>(); return new Result<>();
} }
@PostMapping("/commands/{deviceId}")
@Operation(summary = "发送设备指令")
@RequiresPermissions("sys:role:normal")
public Result<String> sendDeviceCommand(@PathVariable String deviceId, @RequestBody String command) {
try {
// 从系统参数中获取MQTT网关地址
String mqttGatewayUrl = sysParamsService.getValue("server.mqtt_manager_api", true);
if (StringUtils.isBlank(mqttGatewayUrl) || "null".equals(mqttGatewayUrl)) {
return new Result<String>().error("MQTT网关地址未配置");
}
// 构建完整的URL
// 获取设备信息以构建mqttClientId
DeviceEntity deviceById = deviceService.selectById(deviceId);
if (!deviceById.getUserId().equals(SecurityUser.getUser().getId())) {
return new Result<String>().error("设备不存在");
}
String macAddress = deviceById != null ? deviceById.getMacAddress() : "unknown";
String groupId = deviceById != null ? deviceById.getBoard() : null;
if (groupId == null) {
groupId = "GID_default";
}
groupId = groupId.replace(":", "_");
macAddress = macAddress.replace(":", "_");
// 拼接为groupId@@@macAddress@@@deviceId格式
String mqttClientId = groupId + "@@@" + macAddress + "@@@" + macAddress;
String url = "http://" + mqttGatewayUrl + "/api/commands/" + mqttClientId;
// 设置请求头
HttpHeaders headers = new HttpHeaders();
headers.set("Content-Type", "application/json");
// 生成Bearer令牌
String dateStr = java.time.LocalDate.now()
.format(java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd"));
String signatureKey = sysParamsService.getValue("server.mqtt_signature_key", false);
if (StringUtils.isBlank(signatureKey)) {
return new Result<String>().error("MQTT签名密钥未配置");
}
String tokenContent = dateStr + signatureKey;
String token = org.apache.commons.codec.digest.DigestUtils.sha256Hex(tokenContent);
headers.set("Authorization", "Bearer " + token);
// 构建请求体
HttpEntity<String> requestEntity = new HttpEntity<>(command, headers);
// 发送POST请求
ResponseEntity<String> response = restTemplate.exchange(url, HttpMethod.POST, requestEntity, String.class);
// 返回响应
return new Result<String>().ok(response.getBody());
} catch (Exception e) {
return new Result<String>().error("发送指令失败: " + e.getMessage());
}
}
} }
@@ -167,3 +167,5 @@
10158=\u8BF7\u6C42\u5931\u8D25\uFF0C\u72B6\u6001\u7801{0} 10158=\u8BF7\u6C42\u5931\u8D25\uFF0C\u72B6\u6001\u7801{0}
10159=\u97F3\u8272ID\u5DF2\u5B58\u5728 10159=\u97F3\u8272ID\u5DF2\u5B58\u5728
10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u9519\u8BEF\uFF0C\u5FC5\u987B\u4EE5S_\u5F00\u5934 10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u9519\u8BEF\uFF0C\u5FC5\u987B\u4EE5S_\u5F00\u5934
10161=Mac\u5730\u5740\u5DF2\u5B58\u5728
10162=\u6A21\u578B\u4F9B\u5E94\u5668\u4E0D\u5B58\u5728
@@ -167,3 +167,5 @@
10158=Clone Voice: 10158=Clone Voice:
10159=Voice ID already exists 10159=Voice ID already exists
10160=Huoshan Engine voice ID format error, must start with S_ 10160=Huoshan Engine voice ID format error, must start with S_
10161=Mac address already exists
10162=Model provider does not exist
@@ -167,3 +167,5 @@
10158=\u514B\u9686\u8272\u97F3: 10158=\u514B\u9686\u8272\u97F3:
10159=\u97F3\u8272ID\u5DF2\u5B58\u5728 10159=\u97F3\u8272ID\u5DF2\u5B58\u5728
10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u9519\u8BEF\uFF0C\u5FC5\u987B\u4EE5S_\u5F00\u5934 10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u9519\u8BEF\uFF0C\u5FC5\u987B\u4EE5S_\u5F00\u5934
10161=Mac\u5730\u5740\u5DF2\u5B58\u5728
10162=\u6A21\u578B\u4F9B\u5E94\u5668\u4E0D\u5B58\u5728
@@ -167,3 +167,6 @@
10158=\u514B\u9686\u8A9E\u97F3: 10158=\u514B\u9686\u8A9E\u97F3:
10159=\u97F3\u8272ID\u5DF2\u5B58\u5728 10159=\u97F3\u8272ID\u5DF2\u5B58\u5728
10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u932F\u8AA4\uFF0C\u5FC5\u9808\u4EE5S_\u958B\u982D 10160=\u706B\u5C71\u5F15\u64CE\u97F3\u8272ID\u683C\u5F0F\u932F\u8AA4\uFF0C\u5FC5\u9808\u4EE5S_\u958B\u982D
10161=Mac\u5730\u5740\u5DF2\u5B58\u5728
10162=\u6A21\u578B\u63D0\u4F9B\u5546\u4E0D\u5B58\u5728
@@ -7,12 +7,12 @@
}</route> }</route>
<script lang="ts" setup> <script lang="ts" setup>
import { changeLanguage, getCurrentLanguage, getSupportedLanguages, t } from '@/i18n'
import type { Language } from '@/store/lang'
import { clearServerBaseUrlOverride, getEnvBaseUrl, getServerBaseUrlOverride, setServerBaseUrlOverride } from '@/utils' import { clearServerBaseUrlOverride, getEnvBaseUrl, getServerBaseUrlOverride, setServerBaseUrlOverride } from '@/utils'
import { isMp } from '@/utils/platform' import { isMp } from '@/utils/platform'
import { computed, onMounted, reactive, ref } from 'vue' import { computed, onMounted, reactive, ref } from 'vue'
import { useToast } from 'wot-design-uni' import { useToast } from 'wot-design-uni'
import { t, changeLanguage, getSupportedLanguages, getCurrentLanguage } from '@/i18n'
import type { Language } from '@/store/lang'
defineOptions({ defineOptions({
name: 'SettingsPage', name: 'SettingsPage',
@@ -235,7 +235,7 @@ function showAbout() {
title: t('settings.aboutApp', { appName: import.meta.env.VITE_APP_TITLE }), title: t('settings.aboutApp', { appName: import.meta.env.VITE_APP_TITLE }),
content: t('settings.aboutContent', { content: t('settings.aboutContent', {
appName: import.meta.env.VITE_APP_TITLE, appName: import.meta.env.VITE_APP_TITLE,
version: '0.8.5' version: '0.8.6'
}), }),
showCancel: false, showCancel: false,
confirmText: t('common.confirm'), confirmText: t('common.confirm'),
@@ -283,9 +283,9 @@ onMounted(async () => {
<view class="mb-[24rpx]"> <view class="mb-[24rpx]">
<view class="w-full rounded-[16rpx] border border-[#eeeeee] bg-[#f5f7fb] overflow-hidden"> <view class="w-full rounded-[16rpx] border border-[#eeeeee] bg-[#f5f7fb] overflow-hidden">
<wd-input v-model="baseUrlInput" type="text" clearable :maxlength="200" <wd-input v-model="baseUrlInput" type="text" clearable :maxlength="200"
:placeholder="t('settings.enterServerUrl')" :placeholder="t('settings.enterServerUrl')"
custom-class="!border-none !bg-transparent h-[64rpx] px-[24rpx] items-center" custom-class="!border-none !bg-transparent h-[64rpx] px-[24rpx] items-center"
input-class="text-[28rpx] text-[#232338]" @input="validateUrl" @blur="validateUrl" /> input-class="text-[28rpx] text-[#232338]" @input="validateUrl" @blur="validateUrl" />
</view> </view>
<text v-if="urlError" class="mt-[8rpx] block text-[24rpx] text-[#ff4d4f]"> <text v-if="urlError" class="mt-[8rpx] block text-[24rpx] text-[#ff4d4f]">
{{ urlError }} {{ urlError }}
@@ -323,11 +323,11 @@ onMounted(async () => {
class="flex items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx] transition-all active:bg-[#eef3ff]"> class="flex items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx] transition-all active:bg-[#eef3ff]">
<view> <view>
<text class="text-[28rpx] text-[#232338] font-medium"> <text class="text-[28rpx] text-[#232338] font-medium">
{{ t('settings.totalCacheSize') }} {{ t('settings.totalCacheSize') }}
</text> </text>
<text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]"> <text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]">
{{ t('settings.appDataSize') }} {{ t('settings.appDataSize') }}
</text> </text>
</view> </view>
<text class="text-[28rpx] text-[#65686f] font-semibold"> <text class="text-[28rpx] text-[#65686f] font-semibold">
{{ cacheInfo.storageSize }} {{ cacheInfo.storageSize }}
@@ -339,11 +339,11 @@ onMounted(async () => {
class="flex items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx]"> class="flex items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx]">
<view> <view>
<text class="text-[28rpx] text-[#232338] font-medium"> <text class="text-[28rpx] text-[#232338] font-medium">
{{ t('settings.cacheClear') }} {{ t('settings.cacheClear') }}
</text> </text>
<text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]"> <text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]">
{{ t('settings.clearAllCache') }} {{ t('settings.clearAllCache') }}
</text> </text>
</view> </view>
<view <view
class="cursor-pointer rounded-[24rpx] bg-[rgba(255,107,107,0.1)] px-[28rpx] py-[16rpx] text-[24rpx] text-[#ff6b6b] font-semibold transition-all duration-300 active:scale-95 active:bg-[#ff6b6b] active:text-white" class="cursor-pointer rounded-[24rpx] bg-[rgba(255,107,107,0.1)] px-[28rpx] py-[16rpx] text-[24rpx] text-[#ff6b6b] font-semibold transition-all duration-300 active:scale-95 active:bg-[#ff6b6b] active:text-white"
@@ -359,8 +359,8 @@ onMounted(async () => {
<view class="mb-[32rpx]"> <view class="mb-[32rpx]">
<view class="mb-[24rpx] flex items-center"> <view class="mb-[24rpx] flex items-center">
<text class="text-[32rpx] text-[#232338] font-bold"> <text class="text-[32rpx] text-[#232338] font-bold">
{{ t('settings.appInfo') }} {{ t('settings.appInfo') }}
</text> </text>
</view> </view>
<view class="border border-[#eeeeee] rounded-[24rpx] bg-[#fbfbfb] p-[32rpx]" <view class="border border-[#eeeeee] rounded-[24rpx] bg-[#fbfbfb] p-[32rpx]"
@@ -370,11 +370,11 @@ onMounted(async () => {
@click="showAbout"> @click="showAbout">
<view> <view>
<text class="text-[28rpx] text-[#232338] font-medium"> <text class="text-[28rpx] text-[#232338] font-medium">
{{ t('settings.aboutUs') }} {{ t('settings.aboutUs') }}
</text> </text>
<text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]"> <text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]">
{{ t('settings.appVersion') }} {{ t('settings.appVersion') }}
</text> </text>
</view> </view>
<wd-icon name="arrow-right" custom-class="text-[32rpx] text-[#9d9ea3]" /> <wd-icon name="arrow-right" custom-class="text-[32rpx] text-[#9d9ea3]" />
</view> </view>
@@ -385,24 +385,26 @@ onMounted(async () => {
<view class="mb-[32rpx]"> <view class="mb-[32rpx]">
<view class="mb-[24rpx] flex items-center"> <view class="mb-[24rpx] flex items-center">
<text class="text-[32rpx] text-[#232338] font-bold"> <text class="text-[32rpx] text-[#232338] font-bold">
{{ t('settings.languageSettings') }} {{ t('settings.languageSettings') }}
</text> </text>
</view> </view>
<view class="border border-[#eeeeee] rounded-[24rpx] bg-[#fbfbfb] p-[32rpx]" <view class="border border-[#eeeeee] rounded-[24rpx] bg-[#fbfbfb] p-[32rpx]"
style="box-shadow: 0 4rpx 20rpx rgba(0, 0, 0, 0.06);"> style="box-shadow: 0 4rpx 20rpx rgba(0, 0, 0, 0.06);">
<view class="flex cursor-pointer items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx] transition-all active:bg-[#eef3ff]" @click="showLanguageSheet = true"> <view
class="flex cursor-pointer items-center justify-between border border-[#eeeeee] rounded-[16rpx] bg-[#f5f7fb] p-[24rpx] transition-all active:bg-[#eef3ff]"
@click="showLanguageSheet = true">
<view> <view>
<text class="text-[32rpx] text-[#232338] font-medium"> <text class="text-[32rpx] text-[#232338] font-medium">
{{ t('settings.language') }} {{ t('settings.language') }}
</text> </text>
<text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]"> <text class="mt-[4rpx] block text-[24rpx] text-[#9d9ea3]">
{{ t('settings.selectLanguage') }} {{ t('settings.selectLanguage') }}
</text> </text>
</view> </view>
<view class="flex items-center"> <view class="flex items-center">
<text class="text-[32rpx] text-[#9d9ea3] font-semibold mr-[16rpx]"> <text class="text-[32rpx] text-[#9d9ea3] font-semibold mr-[16rpx]">
{{ supportedLanguages.find(lang => lang.code === currentLanguage)?.name }} {{supportedLanguages.find(lang => lang.code === currentLanguage)?.name}}
</text> </text>
<wd-icon name="arrow-right" custom-class="text-[32rpx] text-[#9d9ea3]" /> <wd-icon name="arrow-right" custom-class="text-[32rpx] text-[#9d9ea3]" />
</view> </view>
@@ -411,19 +413,11 @@ onMounted(async () => {
</view> </view>
<!-- 语言选择弹窗 --> <!-- 语言选择弹窗 -->
<wd-action-sheet <wd-action-sheet v-model="showLanguageSheet" :title="t('settings.selectLanguage')" :close-on-click-modal="true">
v-model="showLanguageSheet"
:title="t('settings.selectLanguage')"
:close-on-click-modal="true"
>
<view class="language-sheet"> <view class="language-sheet">
<scroll-view scroll-y class="language-list"> <scroll-view scroll-y class="language-list">
<view <view v-for="lang in supportedLanguages" :key="lang.code" class="language-item"
v-for="lang in supportedLanguages" @click="handleLanguageChange(lang.code)">
:key="lang.code"
class="language-item"
@click="handleLanguageChange(lang.code)"
>
<text class="language-name"> <text class="language-name">
{{ lang.name }} {{ lang.name }}
</text> </text>
@@ -446,17 +440,21 @@ onMounted(async () => {
.language-sheet { .language-sheet {
.language-list { .language-list {
max-height: 50vh; max-height: 50vh;
.language-item { .language-item {
padding: 30rpx 0; padding: 30rpx 0;
text-align: center; text-align: center;
border-bottom: 1rpx solid #f0f0f0; border-bottom: 1rpx solid #f0f0f0;
.language-name { .language-name {
font-size: 28rpx; font-size: 28rpx;
color: #333; color: #333;
} }
&:last-child { &:last-child {
border-bottom: none; border-bottom: none;
} }
&:active { &:active {
background-color: #f5f7fb; background-color: #f5f7fb;
} }
@@ -102,21 +102,4 @@ export default {
}); });
}).send(); }).send();
}, },
// 发送设备指令
sendDeviceCommand(deviceId, mcpData, callback) {
RequestService.sendRequest()
.url(`${getServiceUrl()}/device/commands/${deviceId}`)
.method('POST')
.data(mcpData)
.success((res) => {
RequestService.clearRequestTime();
callback(res);
})
.networkFail((err) => {
console.error('发送设备指令失败:', err);
RequestService.reAjaxFun(() => {
this.sendDeviceCommand(deviceId, mcpData, callback);
});
}).send();
},
} }
File diff suppressed because it is too large Load Diff
+16 -26
View File
@@ -64,9 +64,6 @@
<el-button size="mini" type="text" @click="handleUnbind(scope.row.device_id)"> <el-button size="mini" type="text" @click="handleUnbind(scope.row.device_id)">
{{ $t('device.unbind') }} {{ $t('device.unbind') }}
</el-button> </el-button>
<el-button v-if="scope.row.deviceStatus === 'online'" size="mini" type="text" @click="handleMcpToolCall(scope.row.device_id)">
{{ $t('device.toolCall') }}
</el-button>
</template> </template>
</el-table-column> </el-table-column>
</el-table> </el-table>
@@ -82,9 +79,9 @@
<el-button type="success" size="mini" class="add-device-btn" @click="handleManualAddDevice"> <el-button type="success" size="mini" class="add-device-btn" @click="handleManualAddDevice">
{{ $t('device.manualAdd') }} {{ $t('device.manualAdd') }}
</el-button> </el-button>
<el-button size="mini" type="danger" icon="el-icon-delete" @click="deleteSelected">{{ <el-button size="mini" type="danger" icon="el-icon-delete" @click="deleteSelected">
$t('device.unbind') {{ $t('device.unbind') }}
}}</el-button> </el-button>
</div> </div>
<div class="custom-pagination"> <div class="custom-pagination">
<el-select v-model="pageSize" @change="handlePageSizeChange" class="page-size-select"> <el-select v-model="pageSize" @change="handlePageSizeChange" class="page-size-select">
@@ -92,20 +89,22 @@
:label="$t('dictManagement.itemsPerPage').replace('{items}', item)" :value="item"> :label="$t('dictManagement.itemsPerPage').replace('{items}', item)" :value="item">
</el-option> </el-option>
</el-select> </el-select>
<button class="pagination-btn" :disabled="currentPage === 1" @click="goFirst">{{ <button class="pagination-btn" :disabled="currentPage === 1" @click="goFirst">
$t('dictManagement.firstPage') {{ $t('dictManagement.firstPage') }}
}}</button> </button>
<button class="pagination-btn" :disabled="currentPage === 1" @click="goPrev">{{ <button class="pagination-btn" :disabled="currentPage === 1" @click="goPrev">
$t('dictManagement.prevPage') {{ $t('dictManagement.prevPage') }}
}}</button> </button>
<button v-for="page in visiblePages" :key="page" class="pagination-btn" <button v-for="page in visiblePages" :key="page" class="pagination-btn"
:class="{ active: page === currentPage }" @click="goToPage(page)"> :class="{ active: page === currentPage }" @click="goToPage(page)">
{{ page }} {{ page }}
</button> </button>
<button class="pagination-btn" :disabled="currentPage === pageCount" @click="goNext">{{ <button class="pagination-btn" :disabled="currentPage === pageCount" @click="goNext">
$t('dictManagement.nextPage') }}</button> {{ $t('dictManagement.nextPage') }}
<span class="total-text">{{ $t('dictManagement.totalRecords').replace('{total}', deviceList.length) </button>
}}</span> <span class="total-text">
{{ $t('dictManagement.totalRecords').replace('{total}', deviceList.length) }}
</span>
</div> </div>
</div> </div>
</el-card> </el-card>
@@ -117,7 +116,6 @@
@refresh="fetchBindDevices(currentAgentId)" /> @refresh="fetchBindDevices(currentAgentId)" />
<ManualAddDeviceDialog :visible.sync="manualAddDeviceDialogVisible" :agent-id="currentAgentId" <ManualAddDeviceDialog :visible.sync="manualAddDeviceDialogVisible" :agent-id="currentAgentId"
@refresh="fetchBindDevices(currentAgentId)" /> @refresh="fetchBindDevices(currentAgentId)" />
<McpToolCallDialog :visible.sync="mcpToolCallDialogVisible" :device-id="selectedDeviceId" />
</div> </div>
</template> </template>
@@ -127,20 +125,17 @@ import Api from '@/apis/api';
import AddDeviceDialog from "@/components/AddDeviceDialog.vue"; import AddDeviceDialog from "@/components/AddDeviceDialog.vue";
import HeaderBar from "@/components/HeaderBar.vue"; import HeaderBar from "@/components/HeaderBar.vue";
import ManualAddDeviceDialog from "@/components/ManualAddDeviceDialog.vue"; import ManualAddDeviceDialog from "@/components/ManualAddDeviceDialog.vue";
import McpToolCallDialog from "@/components/McpToolCallDialog.vue";
export default { export default {
components: { components: {
HeaderBar, HeaderBar,
AddDeviceDialog, AddDeviceDialog,
ManualAddDeviceDialog, ManualAddDeviceDialog
McpToolCallDialog
}, },
data() { data() {
return { return {
addDeviceDialogVisible: false, addDeviceDialogVisible: false,
manualAddDeviceDialogVisible: false, manualAddDeviceDialogVisible: false,
mcpToolCallDialogVisible: false,
selectedDeviceId: '', selectedDeviceId: '',
searchKeyword: "", searchKeyword: "",
activeSearchKeyword: "", activeSearchKeyword: "",
@@ -280,11 +275,6 @@ export default {
handleManualAddDevice() { handleManualAddDevice() {
this.manualAddDeviceDialogVisible = true; this.manualAddDeviceDialogVisible = true;
}, },
handleMcpToolCall(deviceId) {
this.selectedDeviceId = deviceId;
this.mcpToolCallDialogVisible = true;
},
submitRemark(row) { submitRemark(row) {
if (row._submitting) return; if (row._submitting) return;
@@ -67,7 +67,6 @@
<context> <context>
【重要!以下信息已实时提供,无需调用工具查询,请直接使用:】 【重要!以下信息已实时提供,无需调用工具查询,请直接使用:】
- **设备ID** {{device_id}}
- **当前时间:** {{current_time}} - **当前时间:** {{current_time}}
- **今天日期:** {{today_date}} ({{today_weekday}}) - **今天日期:** {{today_date}} ({{today_weekday}})
- **今天农历:** {{lunar_date}} - **今天农历:** {{lunar_date}}
+11 -5
View File
@@ -46,12 +46,18 @@ async def main():
check_ffmpeg_installed() check_ffmpeg_installed()
config = load_config() config = load_config()
# 默认使用manager-apisecret作为auth_key # auth_key优先级:配置文件server.auth_key > manager-api.secret > 自动生成
# 如果secret为空,则生成随机密钥 # auth_key用于jwt认证,比如视觉分析接口的jwt认证、ota接口的token生成与websocket认证
# auth_key用于jwt认证,比如视觉分析接口的jwt认证 # 获取配置文件中的auth_key
auth_key = config.get("manager-api", {}).get("secret", "") auth_key = config["server"].get("auth_key", "")
# 验证auth_key,无效则尝试使用manager-api.secret
if not auth_key or len(auth_key) == 0 or "" in auth_key: if not auth_key or len(auth_key) == 0 or "" in auth_key:
auth_key = str(uuid.uuid4().hex) auth_key = config.get("manager-api", {}).get("secret", "")
# 验证secret,无效则生成随机密钥
if not auth_key or len(auth_key) == 0 or "" in auth_key:
auth_key = str(uuid.uuid4().hex)
config["server"]["auth_key"] = auth_key config["server"]["auth_key"] = auth_key
# 添加 stdin 监控任务 # 添加 stdin 监控任务
+9
View File
@@ -70,6 +70,12 @@ enable_stop_tts_notify: false
# 说完话是否开启提示音,音效地址 # 说完话是否开启提示音,音效地址
stop_tts_notify_voice: "config/assets/tts_notify.mp3" stop_tts_notify_voice: "config/assets/tts_notify.mp3"
# TTS音频发送延迟配置
# tts_audio_send_delay: 控制音频包发送间隔
# 0: 使用精确时间控制,严格匹配音频帧率(默认,运行时按音频帧率计算)
# > 0: 使用固定延迟(毫秒)发送,例如: 60
tts_audio_send_delay: 0
exit_commands: exit_commands:
- "退出" - "退出"
- "关闭" - "关闭"
@@ -173,6 +179,9 @@ prompt: |
- 长篇大论,叽叽歪歪 - 长篇大论,叽叽歪歪
- 长时间严肃对话 - 长时间严肃对话
# 默认系统提示词模板文件
prompt_template: agent-base-prompt.txt
# 结束语prompt # 结束语prompt
end_prompt: end_prompt:
enable: true # 是否开启结束语 enable: true # 是否开启结束语
@@ -68,6 +68,9 @@ def get_config_from_api(config):
"vision_explain": config["server"].get("vision_explain", ""), "vision_explain": config["server"].get("vision_explain", ""),
"auth_key": config["server"].get("auth_key", ""), "auth_key": config["server"].get("auth_key", ""),
} }
# 如果服务器没有prompt_template,则从本地配置读取
if not config_data.get("prompt_template"):
config_data["prompt_template"] = config.get("prompt_template")
return config_data return config_data
+1 -1
View File
@@ -5,7 +5,7 @@ from config.config_loader import load_config
from config.settings import check_config_file from config.settings import check_config_file
from datetime import datetime from datetime import datetime
SERVER_VERSION = "0.8.5" SERVER_VERSION = "0.8.6"
_logger_initialized = False _logger_initialized = False
+2
View File
@@ -23,3 +23,5 @@ manager-api:
url: http://127.0.0.1:8002/xiaozhi url: http://127.0.0.1:8002/xiaozhi
# 你的manager-api的token,就是刚才复制出来的server.secret # 你的manager-api的token,就是刚才复制出来的server.secret
secret: 你的server.secret值 secret: 你的server.secret值
# 默认系统提示词模板文件
prompt_template: agent-base-prompt.txt
+1 -1
View File
@@ -140,7 +140,7 @@ class OTAHandler(BaseHandler):
self.logger.bind(tag=TAG).warning("缺少MQTT签名密钥,密码留空") self.logger.bind(tag=TAG).warning("缺少MQTT签名密钥,密码留空")
# 构建MQTT配置(直接使用 mqtt_gateway 字符串) # 构建MQTT配置(直接使用 mqtt_gateway 字符串)
return_json["mqtt_gateway"] = { return_json["mqtt"] = {
"endpoint": mqtt_gateway_endpoint, "endpoint": mqtt_gateway_endpoint,
"client_id": mqtt_client_id, "client_id": mqtt_client_id,
"username": username, "username": username,
+1 -1
View File
@@ -115,9 +115,9 @@ class ConnectionHandler:
# vad相关变量 # vad相关变量
self.client_audio_buffer = bytearray() self.client_audio_buffer = bytearray()
self.client_have_voice = False self.client_have_voice = False
self.client_voice_window = deque(maxlen=5)
self.last_activity_time = 0.0 # 统一的活动时间戳(毫秒) self.last_activity_time = 0.0 # 统一的活动时间戳(毫秒)
self.client_voice_stop = False self.client_voice_stop = False
self.client_voice_window = deque(maxlen=5)
self.last_is_voice = False self.last_is_voice = False
# asr相关变量 # asr相关变量
@@ -33,7 +33,7 @@ async def handleAudioMessage(conn, audio):
async def resume_vad_detection(conn): async def resume_vad_detection(conn):
# 等待2秒后恢复VAD检测 # 等待2秒后恢复VAD检测
await asyncio.sleep(1) await asyncio.sleep(2)
conn.just_woken_up = False conn.just_woken_up = False
@@ -90,6 +90,9 @@ async def sendAudio(conn, audios, frame_duration=60):
if audios is None or len(audios) == 0: if audios is None or len(audios) == 0:
return return
# 获取发送延迟配置
send_delay = conn.config.get("tts_audio_send_delay", -1) / 1000.0
if isinstance(audios, bytes): if isinstance(audios, bytes):
if conn.client_abort: if conn.client_abort:
return return
@@ -107,16 +110,21 @@ async def sendAudio(conn, audios, frame_duration=60):
flow_control = conn.audio_flow_control flow_control = conn.audio_flow_control
current_time = time.perf_counter() current_time = time.perf_counter()
# 计算预期发送时间
expected_time = flow_control["start_time"] + ( if send_delay > 0:
flow_control["packet_count"] * frame_duration / 1000 # 使用固定延迟
) await asyncio.sleep(send_delay)
delay = expected_time - current_time
if delay > 0:
await asyncio.sleep(delay)
else: else:
# 纠正误差 # 计算预期发送时间
flow_control["start_time"] += abs(delay) expected_time = flow_control["start_time"] + (
flow_control["packet_count"] * frame_duration / 1000
)
delay = expected_time - current_time
if delay > 0:
await asyncio.sleep(delay)
else:
# 纠正误差
flow_control["start_time"] += abs(delay)
if conn.conn_from_mqtt_gateway: if conn.conn_from_mqtt_gateway:
# 计算时间戳和序列号 # 计算时间戳和序列号
@@ -164,12 +172,16 @@ async def sendAudio(conn, audios, frame_duration=60):
# 重置没有声音的状态 # 重置没有声音的状态
conn.last_activity_time = time.time() * 1000 conn.last_activity_time = time.time() * 1000
# 计算预期发送时间 if send_delay > 0:
expected_time = start_time + (play_position / 1000) # 固定延迟模式
current_time = time.perf_counter() await asyncio.sleep(send_delay)
delay = expected_time - current_time else:
if delay > 0: # 计算预期发送时间
await asyncio.sleep(delay) expected_time = start_time + (play_position / 1000)
current_time = time.perf_counter()
delay = expected_time - current_time
if delay > 0:
await asyncio.sleep(delay)
if conn.conn_from_mqtt_gateway: if conn.conn_from_mqtt_gateway:
# 计算时间戳和序列号(使用当前的数据包索引确保连续性) # 计算时间戳和序列号(使用当前的数据包索引确保连续性)
@@ -96,6 +96,8 @@ class ASRProvider(ASRProviderBase):
self.delete_audio_file = delete_audio_file self.delete_audio_file = delete_audio_file
self.expire_time = None self.expire_time = None
self.task_id = uuid.uuid4().hex
# Token管理 # Token管理
if self.access_key_id and self.access_key_secret: if self.access_key_id and self.access_key_secret:
self._refresh_token() self._refresh_token()
@@ -170,6 +172,10 @@ class ASRProvider(ASRProviderBase):
close_timeout=5, close_timeout=5,
) )
self.task_id = uuid.uuid4().hex
logger.bind(tag=TAG).info(f"WebSocket连接建立成功, task_id: {self.task_id}")
self.is_processing = True self.is_processing = True
self.server_ready = False # 重置服务器准备状态 self.server_ready = False # 重置服务器准备状态
self.forward_task = asyncio.create_task(self._forward_results(conn)) self.forward_task = asyncio.create_task(self._forward_results(conn))
@@ -180,8 +186,8 @@ class ASRProvider(ASRProviderBase):
"namespace": "SpeechTranscriber", "namespace": "SpeechTranscriber",
"name": "StartTranscription", "name": "StartTranscription",
"status": 20000000, "status": 20000000,
"message_id": ''.join(random.choices('0123456789abcdef', k=32)), "message_id": uuid.uuid4().hex,
"task_id": ''.join(random.choices('0123456789abcdef', k=32)), "task_id": self.task_id,
"status_text": "Gateway:SUCCESS:Success.", "status_text": "Gateway:SUCCESS:Success.",
"appkey": self.appkey "appkey": self.appkey
}, },
@@ -292,7 +298,8 @@ class ASRProvider(ASRProviderBase):
"namespace": "SpeechTranscriber", "namespace": "SpeechTranscriber",
"name": "StopTranscription", "name": "StopTranscription",
"status": 20000000, "status": 20000000,
"message_id": ''.join(random.choices('0123456789abcdef', k=32)), "message_id": uuid.uuid4().hex,
"task_id": self.task_id,
"status_text": "Client:Stop", "status_text": "Client:Stop",
"appkey": self.appkey "appkey": self.appkey
} }
@@ -68,7 +68,7 @@ class ASRProviderBase(ABC):
conn.asr_audio.clear() conn.asr_audio.clear()
conn.reset_vad_states() conn.reset_vad_states()
if len(asr_audio_task) > 15: if len(asr_audio_task) > 15 or conn.client_listen_mode == "manual":
await self.handle_voice_stop(conn, asr_audio_task) await self.handle_voice_stop(conn, asr_audio_task)
# 处理语音停止 # 处理语音停止
@@ -22,6 +22,7 @@ STATUS_FIRST_FRAME = 0 # 第一帧的标识
STATUS_CONTINUE_FRAME = 1 # 中间帧标识 STATUS_CONTINUE_FRAME = 1 # 中间帧标识
STATUS_LAST_FRAME = 2 # 最后一帧的标识 STATUS_LAST_FRAME = 2 # 最后一帧的标识
class ASRProvider(ASRProviderBase): class ASRProvider(ASRProviderBase):
def __init__(self, config, delete_audio_file): def __init__(self, config, delete_audio_file):
super().__init__() super().__init__()
@@ -35,6 +36,7 @@ class ASRProvider(ASRProviderBase):
self.server_ready = False self.server_ready = False
self.last_frame_sent = False # 标记是否已发送最终帧 self.last_frame_sent = False # 标记是否已发送最终帧
self.best_text = "" # 保存最佳识别结果 self.best_text = "" # 保存最佳识别结果
self.has_final_result = False # 标记是否收到最终识别结果
# 讯飞配置 # 讯飞配置
self.app_id = config.get("app_id") self.app_id = config.get("app_id")
@@ -50,11 +52,7 @@ class ASRProvider(ASRProviderBase):
"language": config.get("language", "zh_cn"), "language": config.get("language", "zh_cn"),
"accent": config.get("accent", "mandarin"), "accent": config.get("accent", "mandarin"),
"dwa": config.get("dwa", "wpgs"), "dwa": config.get("dwa", "wpgs"),
"result": { "result": {"encoding": "utf8", "compress": "raw", "format": "plain"},
"encoding": "utf8",
"compress": "raw",
"format": "plain"
}
} }
self.output_dir = config.get("output_dir", "tmp/") self.output_dir = config.get("output_dir", "tmp/")
@@ -62,7 +60,7 @@ class ASRProvider(ASRProviderBase):
def create_url(self) -> str: def create_url(self) -> str:
"""生成认证URL""" """生成认证URL"""
url = 'ws://iat.cn-huabei-1.xf-yun.com/v1' url = "ws://iat.cn-huabei-1.xf-yun.com/v1"
# 生成RFC1123格式的时间戳 # 生成RFC1123格式的时间戳
now = datetime.now() now = datetime.now()
date = format_date_time(mktime(now.timetuple())) date = format_date_time(mktime(now.timetuple()))
@@ -73,23 +71,30 @@ class ASRProvider(ASRProviderBase):
signature_origin += "GET " + "/v1 " + "HTTP/1.1" signature_origin += "GET " + "/v1 " + "HTTP/1.1"
# 进行hmac-sha256进行加密 # 进行hmac-sha256进行加密
signature_sha = hmac.new(self.api_secret.encode('utf-8'), signature_origin.encode('utf-8'), signature_sha = hmac.new(
digestmod=hashlib.sha256).digest() self.api_secret.encode("utf-8"),
signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8') signature_origin.encode("utf-8"),
digestmod=hashlib.sha256,
).digest()
signature_sha = base64.b64encode(signature_sha).decode(encoding="utf-8")
authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % ( authorization_origin = (
self.api_key, "hmac-sha256", "host date request-line", signature_sha) 'api_key="%s", algorithm="%s", headers="%s", signature="%s"'
authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8') % (self.api_key, "hmac-sha256", "host date request-line", signature_sha)
)
authorization = base64.b64encode(authorization_origin.encode("utf-8")).decode(
encoding="utf-8"
)
# 将请求的鉴权参数组合为字典 # 将请求的鉴权参数组合为字典
v = { v = {
"authorization": authorization, "authorization": authorization,
"date": date, "date": date,
"host": "iat.cn-huabei-1.xf-yun.com" "host": "iat.cn-huabei-1.xf-yun.com",
} }
# 拼接鉴权参数,生成url # 拼接鉴权参数,生成url
url = url + '?' + urlencode(v) url = url + "?" + urlencode(v)
return url return url
async def open_audio_channels(self, conn): async def open_audio_channels(self, conn):
@@ -100,7 +105,7 @@ class ASRProvider(ASRProviderBase):
await super().receive_audio(conn, audio, audio_have_voice) await super().receive_audio(conn, audio, audio_have_voice)
# 存储音频数据用于声纹识别 # 存储音频数据用于声纹识别
if not hasattr(conn, 'asr_audio_for_voiceprint'): if not hasattr(conn, "asr_audio_for_voiceprint"):
conn.asr_audio_for_voiceprint = [] conn.asr_audio_for_voiceprint = []
conn.asr_audio_for_voiceprint.append(audio) conn.asr_audio_for_voiceprint.append(audio)
@@ -146,8 +151,10 @@ class ASRProvider(ASRProviderBase):
# 发送首帧音频 # 发送首帧音频
if conn.asr_audio and len(conn.asr_audio) > 0: if conn.asr_audio and len(conn.asr_audio) > 0:
first_audio = conn.asr_audio[-1] if conn.asr_audio else b'' first_audio = conn.asr_audio[-1] if conn.asr_audio else b""
pcm_frame = self.decoder.decode(first_audio, 960) if first_audio else b'' pcm_frame = (
self.decoder.decode(first_audio, 960) if first_audio else b""
)
await self._send_audio_frame(pcm_frame, STATUS_FIRST_FRAME) await self._send_audio_frame(pcm_frame, STATUS_FIRST_FRAME)
self.server_ready = True self.server_ready = True
logger.bind(tag=TAG).info("已发送首帧,开始识别") logger.bind(tag=TAG).info("已发送首帧,开始识别")
@@ -176,23 +183,14 @@ class ASRProvider(ASRProviderBase):
if not self.asr_ws: if not self.asr_ws:
return return
audio_b64 = base64.b64encode(audio_data).decode('utf-8') audio_b64 = base64.b64encode(audio_data).decode("utf-8")
frame_data = { frame_data = {
"header": { "header": {"status": status, "app_id": self.app_id},
"status": status, "parameter": {"iat": self.iat_params},
"app_id": self.app_id
},
"parameter": {
"iat": self.iat_params
},
"payload": { "payload": {
"audio": { "audio": {"audio": audio_b64, "sample_rate": 16000, "encoding": "raw"}
"audio": audio_b64, },
"sample_rate": 16000,
"encoding": "raw"
}
}
} }
await self.asr_ws.send(json.dumps(frame_data, ensure_ascii=False)) await self.asr_ws.send(json.dumps(frame_data, ensure_ascii=False))
@@ -207,11 +205,13 @@ class ASRProvider(ASRProviderBase):
try: try:
while self.asr_ws and not conn.stop_event.is_set(): while self.asr_ws and not conn.stop_event.is_set():
# 获取当前连接的音频数据 # 获取当前连接的音频数据
audio_data = getattr(conn, 'asr_audio_for_voiceprint', []) audio_data = getattr(conn, "asr_audio_for_voiceprint", [])
try: try:
# 如果已发送最终帧,增加超时时间等待完整结果 # 如果已发送最终帧,增加超时时间等待完整结果
timeout = 3.0 if self.last_frame_sent else 30.0 timeout = 3.0 if self.last_frame_sent else 30.0
response = await asyncio.wait_for(self.asr_ws.recv(), timeout=timeout) response = await asyncio.wait_for(
self.asr_ws.recv(), timeout=timeout
)
result = json.loads(response) result = json.loads(response)
logger.bind(tag=TAG).debug(f"收到ASR结果: {result}") logger.bind(tag=TAG).debug(f"收到ASR结果: {result}")
@@ -221,7 +221,9 @@ class ASRProvider(ASRProviderBase):
status = header.get("status", 0) status = header.get("status", 0)
if code != 0: if code != 0:
logger.bind(tag=TAG).error(f"识别错误,错误码: {code}, 消息: {header.get('message', '')}") logger.bind(tag=TAG).error(
f"识别错误,错误码: {code}, 消息: {header.get('message', '')}"
)
if code in [10114, 10160]: # 连接问题 if code in [10114, 10160]: # 连接问题
break break
continue continue
@@ -231,52 +233,124 @@ class ASRProvider(ASRProviderBase):
text_data = payload["result"]["text"] text_data = payload["result"]["text"]
if text_data: if text_data:
# 解码base64文本 # 解码base64文本
decoded_text = base64.b64decode(text_data).decode('utf-8') decoded_text = base64.b64decode(text_data).decode("utf-8")
text_json = json.loads(decoded_text) text_json = json.loads(decoded_text)
# 提取文本内容 # 提取文本内容
text_ws = text_json.get('ws', []) text_ws = text_json.get("ws", [])
result_text = '' result_text = ""
for i in text_ws: for i in text_ws:
for j in i.get("cw", []): for j in i.get("cw", []):
w = j.get("w", "") w = j.get("w", "")
result_text += w result_text += w
# 更新识别文本 - 实时更新策略 # 更新识别文本 - 实时更新策略
if result_text and result_text.strip() not in ['', '', '.', ',', '']: # 只检查是否为空字符串,不再过滤任何标点符号
# 这样可以确保所有识别到的内容,包括标点符号都能被实时更新
if result_text and result_text.strip():
# 实时更新:正常情况下都更新,提高响应速度 # 实时更新:正常情况下都更新,提高响应速度
should_update = True should_update = True
# 保存最佳文本(最长的有意义文本) # 保存最佳文本
if (len(result_text) > len(self.best_text) and # 1. 如果是识别完成状态或最终帧后收到的结果,优先保存
result_text.strip() not in ['', '?', '', '.']): # 2. 否则保存最长的有意义文本
self.best_text = result_text # 取消对标点符号的过滤,只检查是否为空
logger.bind(tag=TAG).debug(f"保存最佳文本: {self.best_text}") # 这样可以保留所有识别到的内容,包括各种标点符号
is_valid_text = len(result_text.strip()) > 0
# 如果已发送最终帧,只过滤明显的无效结果 if (
self.last_frame_sent or status == 2
) and is_valid_text:
self.best_text = result_text
self.has_final_result = True # 标记已收到最终结果
logger.bind(tag=TAG).debug(
f"保存最终识别结果: {self.best_text}"
)
elif (
len(result_text) > len(self.best_text)
and is_valid_text
and not self.has_final_result
):
self.best_text = result_text
logger.bind(tag=TAG).debug(
f"保存中间最佳文本: {self.best_text}"
)
# 如果已发送最终帧,只过滤空文本
if self.last_frame_sent: if self.last_frame_sent:
# 最终帧后拒绝问号等明显错误的结果 # 只拒绝完全空的结果
if result_text.strip() in ['', '?', '', '.']: if not result_text.strip():
should_update = False should_update = False
logger.bind(tag=TAG).warning(f"最终帧后拒绝无效文本: {result_text}") logger.bind(tag=TAG).warning(
f"最终帧后拒绝空文本"
)
if should_update: if should_update:
self.text = result_text # 处理流式识别结果,避免简单替换导致内容丢失
logger.bind(tag=TAG).info(f"实时更新识别文本: {self.text} (最终帧已发送: {self.last_frame_sent})") # 1. 如果是中间状态(非最终帧后),可能需要替换为更完整的识别
# 2. 如果是最终帧后收到的结果,可能是对前面文本的补充
if self.last_frame_sent:
# 最终帧后收到的结果可能是标点符号等补充内容
# 检查是否需要合并文本而不是替换
# 如果当前文本是纯标点而前面已有内容,应该追加而不是替换
if len(
self.text
) > 0 and result_text.strip() in [
"",
".",
"?",
"",
"!",
"",
",",
"",
";",
"",
]:
# 对于标点符号,追加到现有文本后
self.text = (
self.text.rstrip().rstrip("。.")
+ result_text
)
else:
# 其他情况保持替换逻辑
self.text = result_text
else:
# 中间状态替换为新的识别结果
self.text = result_text
logger.bind(tag=TAG).info(
f"实时更新识别文本: {self.text} (最终帧已发送: {self.last_frame_sent})"
)
# 识别完成,但如果还没发送最终帧,继续等待 # 识别完成,但如果还没发送最终帧,继续等待
if status == 2: if status == 2:
logger.bind(tag=TAG).info(f"识别完成状态已到达,当前识别文本: {self.text}") logger.bind(tag=TAG).info(
f"识别完成状态已到达,当前识别文本: {self.text}"
)
# 如果还没发送最终帧,继续等待 # 如果还没发送最终帧,继续等待
if not self.last_frame_sent: if not self.last_frame_sent:
logger.bind(tag=TAG).info("识别完成但最终帧未发送,继续等待...") logger.bind(tag=TAG).info(
"识别完成但最终帧未发送,继续等待..."
)
continue continue
# 已发送最终帧且收到完成状态,使用最佳文本作为最终结果 # 已发送最终帧且收到完成状态,使用最佳策略选择最终结果
if self.best_text and len(self.best_text) > len(self.text): # 优先使用识别完成状态下的最新结果,而不是仅仅基于长度
logger.bind(tag=TAG).info(f"使用最佳文本作为最终结果: {self.text} -> {self.best_text}") if self.best_text:
self.text = self.best_text # 如果当前文本是在最终帧发送后或识别完成状态下收到的,优先使用
if (
self.last_frame_sent or status == 2
) and self.text.strip():
logger.bind(tag=TAG).info(
f"使用完成状态下的最新识别结果: {self.text}"
)
elif len(self.best_text) > len(self.text):
logger.bind(tag=TAG).info(
f"使用更长的最佳文本作为最终结果: {self.text} -> {self.best_text}"
)
self.text = self.best_text
logger.bind(tag=TAG).info(f"获取到最终完整文本: {self.text}") logger.bind(tag=TAG).info(f"获取到最终完整文本: {self.text}")
conn.reset_vad_states() conn.reset_vad_states()
@@ -289,9 +363,13 @@ class ASRProvider(ASRProviderBase):
if self.last_frame_sent: if self.last_frame_sent:
# 超时时也使用最佳文本 # 超时时也使用最佳文本
if self.best_text and len(self.best_text) > len(self.text): if self.best_text and len(self.best_text) > len(self.text):
logger.bind(tag=TAG).info(f"超时,使用最佳文本: {self.text} -> {self.best_text}") logger.bind(tag=TAG).info(
f"超时,使用最佳文本: {self.text} -> {self.best_text}"
)
self.text = self.best_text self.text = self.best_text
logger.bind(tag=TAG).info(f"最终帧后超时,使用结果: {self.text}") logger.bind(tag=TAG).info(
f"最终帧后超时,使用结果: {self.text}"
)
break break
# 如果还没发送最终帧,继续等待 # 如果还没发送最终帧,继续等待
continue continue
@@ -316,11 +394,11 @@ class ASRProvider(ASRProviderBase):
self.asr_ws = None self.asr_ws = None
self.is_processing = False self.is_processing = False
if conn: if conn:
if hasattr(conn, 'asr_audio_for_voiceprint'): if hasattr(conn, "asr_audio_for_voiceprint"):
conn.asr_audio_for_voiceprint = [] conn.asr_audio_for_voiceprint = []
if hasattr(conn, 'asr_audio'): if hasattr(conn, "asr_audio"):
conn.asr_audio = [] conn.asr_audio = []
if hasattr(conn, 'has_valid_voice'): if hasattr(conn, "has_valid_voice"):
conn.has_valid_voice = False conn.has_valid_voice = False
async def handle_voice_stop(self, conn, asr_audio_task: List[bytes]): async def handle_voice_stop(self, conn, asr_audio_task: List[bytes]):
@@ -330,7 +408,7 @@ class ASRProvider(ASRProviderBase):
if self.asr_ws and self.is_processing: if self.asr_ws and self.is_processing:
try: try:
# 取最后一个有效的音频帧作为最后一帧数据 # 取最后一个有效的音频帧作为最后一帧数据
last_frame = b'' last_frame = b""
if asr_audio_task: if asr_audio_task:
last_audio = asr_audio_task[-1] last_audio = asr_audio_task[-1]
last_frame = self.decoder.decode(last_audio, 960) last_frame = self.decoder.decode(last_audio, 960)
@@ -338,8 +416,7 @@ class ASRProvider(ASRProviderBase):
logger.bind(tag=TAG).info("已发送最后一帧") logger.bind(tag=TAG).info("已发送最后一帧")
# 发送最终帧后,给_forward_results适当时间处理最终结果 # 发送最终帧后,给_forward_results适当时间处理最终结果
# 减少等待时间,提高响应速度 await asyncio.sleep(0.25)
await asyncio.sleep(0.1) # 从3秒减少到1秒
logger.bind(tag=TAG).info(f"准备处理最终识别结果: {self.text}") logger.bind(tag=TAG).info(f"准备处理最终识别结果: {self.text}")
except Exception as e: except Exception as e:
@@ -350,6 +427,7 @@ class ASRProvider(ASRProviderBase):
except Exception as e: except Exception as e:
logger.bind(tag=TAG).error(f"处理语音停止失败: {e}") logger.bind(tag=TAG).error(f"处理语音停止失败: {e}")
import traceback import traceback
logger.bind(tag=TAG).debug(f"异常详情: {traceback.format_exc()}") logger.bind(tag=TAG).debug(f"异常详情: {traceback.format_exc()}")
def stop_ws_connection(self): def stop_ws_connection(self):
@@ -360,12 +438,14 @@ class ASRProvider(ASRProviderBase):
async def _cleanup(self, conn): async def _cleanup(self, conn):
"""清理资源""" """清理资源"""
logger.bind(tag=TAG).info(f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}") logger.bind(tag=TAG).info(
f"开始ASR会话清理 | 当前状态: processing={self.is_processing}, server_ready={self.server_ready}"
)
# 发送最后一帧 # 发送最后一帧
if self.asr_ws and self.is_processing: if self.asr_ws and self.is_processing:
try: try:
await self._send_audio_frame(b'', STATUS_LAST_FRAME) await self._send_audio_frame(b"", STATUS_LAST_FRAME)
await asyncio.sleep(0.1) await asyncio.sleep(0.1)
logger.bind(tag=TAG).info("已发送最后一帧") logger.bind(tag=TAG).info("已发送最后一帧")
except Exception as e: except Exception as e:
@@ -376,6 +456,7 @@ class ASRProvider(ASRProviderBase):
self.server_ready = False self.server_ready = False
self.last_frame_sent = False self.last_frame_sent = False
self.best_text = "" self.best_text = ""
self.has_final_result = False
logger.bind(tag=TAG).info("ASR状态已重置") logger.bind(tag=TAG).info("ASR状态已重置")
# 清理任务 # 清理任务
@@ -403,11 +484,11 @@ class ASRProvider(ASRProviderBase):
# 清理连接的音频缓存 # 清理连接的音频缓存
if conn: if conn:
if hasattr(conn, 'asr_audio_for_voiceprint'): if hasattr(conn, "asr_audio_for_voiceprint"):
conn.asr_audio_for_voiceprint = [] conn.asr_audio_for_voiceprint = []
if hasattr(conn, 'asr_audio'): if hasattr(conn, "asr_audio"):
conn.asr_audio = [] conn.asr_audio = []
if hasattr(conn, 'has_valid_voice'): if hasattr(conn, "has_valid_voice"):
conn.has_valid_voice = False conn.has_valid_voice = False
logger.bind(tag=TAG).info("ASR会话清理完成") logger.bind(tag=TAG).info("ASR会话清理完成")
@@ -432,11 +513,11 @@ class ASRProvider(ASRProviderBase):
self.forward_task = None self.forward_task = None
self.is_processing = False self.is_processing = False
# 清理所有连接的音频缓冲区 # 清理所有连接的音频缓冲区
if hasattr(self, '_connections'): if hasattr(self, "_connections"):
for conn in self._connections.values(): for conn in self._connections.values():
if hasattr(conn, 'asr_audio_for_voiceprint'): if hasattr(conn, "asr_audio_for_voiceprint"):
conn.asr_audio_for_voiceprint = [] conn.asr_audio_for_voiceprint = []
if hasattr(conn, 'asr_audio'): if hasattr(conn, "asr_audio"):
conn.asr_audio = [] conn.asr_audio = []
if hasattr(conn, 'has_valid_voice'): if hasattr(conn, "has_valid_voice"):
conn.has_valid_voice = False conn.has_valid_voice = False
@@ -349,11 +349,6 @@ async def call_mcp_endpoint_tool(
raise ValueError(f"参数处理失败: {str(e)}") raise ValueError(f"参数处理失败: {str(e)}")
raise e raise e
# 加入MAC地址
if mcp_client.conn and mcp_client.conn.device_id:
arguments['mac_address'] = mcp_client.conn.device_id
logger.bind(tag=TAG).info(f"已将设备MAC地址 {mcp_client.conn.device_id} 加入到MCP接入点工具调用参数中")
actual_name = mcp_client.name_mapping.get(tool_name, tool_name) actual_name = mcp_client.name_mapping.get(tool_name, tool_name)
payload = { payload = {
"jsonrpc": "2.0", "jsonrpc": "2.0",
@@ -363,7 +358,9 @@ async def call_mcp_endpoint_tool(
} }
message = json.dumps(payload) message = json.dumps(payload)
logger.bind(tag=TAG).info(f"发送MCP接入点工具调用请求: {actual_name},参数: {json.dumps(arguments, ensure_ascii=False)}") logger.bind(tag=TAG).info(
f"发送MCP接入点工具调用请求: {actual_name},参数: {json.dumps(arguments, ensure_ascii=False)}"
)
await mcp_client.send_message(message) await mcp_client.send_message(message)
try: try:
@@ -13,6 +13,7 @@ from typing import Optional, List, Dict, Any
from mcp import ClientSession, StdioServerParameters from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client from mcp.client.stdio import stdio_client
from mcp.client.sse import sse_client from mcp.client.sse import sse_client
from mcp.client.streamable_http import streamablehttp_client
from config.logger import setup_logging from config.logger import setup_logging
from core.utils.util import sanitize_tool_name from core.utils.util import sanitize_tool_name
@@ -172,10 +173,33 @@ class ServerMCPClient:
if "API_ACCESS_TOKEN" in self.config: if "API_ACCESS_TOKEN" in self.config:
headers["Authorization"] = f"Bearer {self.config['API_ACCESS_TOKEN']}" headers["Authorization"] = f"Bearer {self.config['API_ACCESS_TOKEN']}"
self.logger.bind(tag=TAG).warning(f"你正在使用旧过时的配置 API_ACCESS_TOKEN ,请在.mcp_server_settings.json中将API_ACCESS_TOKEN直接设置在headers中,例如 'Authorization': 'Bearer API_ACCESS_TOKEN'") self.logger.bind(tag=TAG).warning(f"你正在使用旧过时的配置 API_ACCESS_TOKEN ,请在.mcp_server_settings.json中将API_ACCESS_TOKEN直接设置在headers中,例如 'Authorization': 'Bearer API_ACCESS_TOKEN'")
sse_r, sse_w = await stack.enter_async_context(
sse_client(self.config["url"], headers=headers, timeout=self.config.get("timeout", 5), sse_read_timeout=self.config.get("sse_read_timeout", 60 * 5)) # 根据transport类型选择不同的客户端,默认为SSE
) transport_type = self.config.get("transport", "sse")
read_stream, write_stream = sse_r, sse_w
if transport_type == "streamable-http" or transport_type == "http":
# 使用 Streamable HTTP 传输
http_r, http_w, get_session_id = await stack.enter_async_context(
streamablehttp_client(
url=self.config["url"],
headers=headers,
timeout=self.config.get("timeout", 30),
sse_read_timeout=self.config.get("sse_read_timeout", 60 * 5),
terminate_on_close=self.config.get("terminate_on_close", True)
)
)
read_stream, write_stream = http_r, http_w
else:
# 使用传统的 SSE 传输
sse_r, sse_w = await stack.enter_async_context(
sse_client(
url=self.config["url"],
headers=headers,
timeout=self.config.get("timeout", 5),
sse_read_timeout=self.config.get("sse_read_timeout", 60 * 5)
)
)
read_stream, write_stream = sse_r, sse_w
else: else:
raise ValueError("MCP客户端配置必须包含'command''url'") raise ValueError("MCP客户端配置必须包含'command''url'")
@@ -1,3 +1,4 @@
import random
import uuid import uuid
import json import json
import hmac import hmac
@@ -131,7 +132,7 @@ class TTSProvider(TTSProviderBase):
self.last_active_time = None self.last_active_time = None
# 专属tts设置 # 专属tts设置
self.message_id = "" self.task_id = uuid.uuid4().hex
# 创建Opus编码器 # 创建Opus编码器
self.opus_encoder = opus_encoder_utils.OpusEncoderUtils( self.opus_encoder = opus_encoder_utils.OpusEncoderUtils(
@@ -185,7 +186,8 @@ class TTSProvider(TTSProviderBase):
current_time = time.time() current_time = time.time()
if self.ws and current_time - self.last_active_time < 10: if self.ws and current_time - self.last_active_time < 10:
# 10秒内才可以复用链接进行连续对话 # 10秒内才可以复用链接进行连续对话
logger.bind(tag=TAG).info(f"使用已有链接...") self.task_id = uuid.uuid4().hex
logger.bind(tag=TAG).info(f"使用已有链接..., task_id: {self.task_id}")
return self.ws return self.ws
logger.bind(tag=TAG).info("开始建立新连接...") logger.bind(tag=TAG).info("开始建立新连接...")
@@ -196,7 +198,8 @@ class TTSProvider(TTSProviderBase):
ping_timeout=10, ping_timeout=10,
close_timeout=10, close_timeout=10,
) )
logger.bind(tag=TAG).info("WebSocket连接建立成功") self.task_id = uuid.uuid4().hex
logger.bind(tag=TAG).info(f"WebSocket连接建立成功, task_id: {self.task_id}")
self.last_active_time = time.time() self.last_active_time = time.time()
return self.ws return self.ws
except Exception as e: except Exception as e:
@@ -224,18 +227,9 @@ class TTSProvider(TTSProviderBase):
if message.sentence_type == SentenceType.FIRST: if message.sentence_type == SentenceType.FIRST:
# 初始化参数 # 初始化参数
try: try:
if not getattr(self.conn, "sentence_id", None):
self.conn.sentence_id = uuid.uuid4().hex
logger.bind(tag=TAG).info(
f"自动生成新的 会话ID: {self.conn.sentence_id}"
)
# aliyunStream独有的参数生成
self.message_id = str(uuid.uuid4().hex)
logger.bind(tag=TAG).info("开始启动TTS会话...") logger.bind(tag=TAG).info("开始启动TTS会话...")
future = asyncio.run_coroutine_threadsafe( future = asyncio.run_coroutine_threadsafe(
self.start_session(self.conn.sentence_id), self.start_session(self.task_id),
loop=self.conn.loop, loop=self.conn.loop,
) )
future.result() future.result()
@@ -273,7 +267,7 @@ class TTSProvider(TTSProviderBase):
try: try:
logger.bind(tag=TAG).info("开始结束TTS会话...") logger.bind(tag=TAG).info("开始结束TTS会话...")
future = asyncio.run_coroutine_threadsafe( future = asyncio.run_coroutine_threadsafe(
self.finish_session(self.conn.sentence_id), self.finish_session(self.task_id),
loop=self.conn.loop, loop=self.conn.loop,
) )
future.result() future.result()
@@ -296,8 +290,8 @@ class TTSProvider(TTSProviderBase):
filtered_text = MarkdownCleaner.clean_markdown(text) filtered_text = MarkdownCleaner.clean_markdown(text)
run_request = { run_request = {
"header": { "header": {
"message_id": self.message_id, "message_id": uuid.uuid4().hex,
"task_id": self.conn.sentence_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "RunSynthesis", "name": "RunSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -318,8 +312,8 @@ class TTSProvider(TTSProviderBase):
self.ws = None self.ws = None
raise raise
async def start_session(self, session_id): async def start_session(self, task_id):
logger.bind(tag=TAG).info(f"开始会话~~{session_id}") logger.bind(tag=TAG).info("开始会话~~")
try: try:
# 会话开始时检测上个会话的监听状态 # 会话开始时检测上个会话的监听状态
if ( if (
@@ -340,8 +334,8 @@ class TTSProvider(TTSProviderBase):
start_request = { start_request = {
"header": { "header": {
"message_id": self.message_id, "message_id": uuid.uuid4().hex,
"task_id": self.conn.sentence_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "StartSynthesis", "name": "StartSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -365,14 +359,14 @@ class TTSProvider(TTSProviderBase):
await self.close() await self.close()
raise raise
async def finish_session(self, session_id): async def finish_session(self, task_id):
logger.bind(tag=TAG).info(f"关闭会话~~{session_id}") logger.bind(tag=TAG).info(f"关闭会话~~{task_id}")
try: try:
if self.ws: if self.ws:
stop_request = { stop_request = {
"header": { "header": {
"message_id": self.message_id, "message_id": uuid.uuid4().hex,
"task_id": self.conn.sentence_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "StopSynthesis", "name": "StopSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -484,8 +478,6 @@ class TTSProvider(TTSProviderBase):
loop = asyncio.new_event_loop() loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop) asyncio.set_event_loop(loop)
# 生成会话ID
session_id = uuid.uuid4().hex
# 存储音频数据 # 存储音频数据
audio_data = [] audio_data = []
@@ -504,11 +496,10 @@ class TTSProvider(TTSProviderBase):
) )
try: try:
# 发送StartSynthesis请求 # 发送StartSynthesis请求
start_message_id = str(uuid.uuid4().hex)
start_request = { start_request = {
"header": { "header": {
"message_id": start_message_id, "message_id": uuid.uuid4().hex,
"task_id": session_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "StartSynthesis", "name": "StartSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -549,11 +540,10 @@ class TTSProvider(TTSProviderBase):
# 发送文本合成请求 # 发送文本合成请求
filtered_text = MarkdownCleaner.clean_markdown(text) filtered_text = MarkdownCleaner.clean_markdown(text)
run_message_id = str(uuid.uuid4().hex)
run_request = { run_request = {
"header": { "header": {
"message_id": run_message_id, "message_id": uuid.uuid4().hex,
"task_id": session_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "RunSynthesis", "name": "RunSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -563,11 +553,10 @@ class TTSProvider(TTSProviderBase):
await ws.send(json.dumps(run_request)) await ws.send(json.dumps(run_request))
# 发送停止合成请求 # 发送停止合成请求
stop_message_id = str(uuid.uuid4().hex)
stop_request = { stop_request = {
"header": { "header": {
"message_id": stop_message_id, "message_id": uuid.uuid4().hex,
"task_id": session_id, "task_id": self.task_id,
"namespace": "FlowingSpeechSynthesizer", "namespace": "FlowingSpeechSynthesizer",
"name": "StopSynthesis", "name": "StopSynthesis",
"appkey": self.appkey, "appkey": self.appkey,
@@ -149,6 +149,7 @@ class TTSProvider(TTSProviderBase):
self.access_token = config.get("access_token") self.access_token = config.get("access_token")
self.cluster = config.get("cluster") self.cluster = config.get("cluster")
self.resource_id = config.get("resource_id") self.resource_id = config.get("resource_id")
self.activate_session = False
if config.get("private_voice"): if config.get("private_voice"):
self.voice = config.get("private_voice") self.voice = config.get("private_voice")
else: else:
@@ -180,7 +181,7 @@ class TTSProvider(TTSProviderBase):
raise raise
async def _ensure_connection(self): async def _ensure_connection(self):
"""建立新的WebSocket连接""" """建立新的WebSocket连接,并启动监听任务(仅第一次)"""
try: try:
if self.ws: if self.ws:
logger.bind(tag=TAG).info(f"使用已有链接...") logger.bind(tag=TAG).info(f"使用已有链接...")
@@ -196,6 +197,12 @@ class TTSProvider(TTSProviderBase):
self.ws_url, additional_headers=ws_header, max_size=1000000000 self.ws_url, additional_headers=ws_header, max_size=1000000000
) )
logger.bind(tag=TAG).info("WebSocket连接建立成功") logger.bind(tag=TAG).info("WebSocket连接建立成功")
# 连接建立成功后,启动监听任务
if self._monitor_task is None or self._monitor_task.done():
logger.bind(tag=TAG).info("启动监听任务...")
self._monitor_task = asyncio.create_task(self._start_monitor_tts_response())
return self.ws return self.ws
except Exception as e: except Exception as e:
logger.bind(tag=TAG).error(f"建立连接失败: {str(e)}") logger.bind(tag=TAG).error(f"建立连接失败: {str(e)}")
@@ -315,20 +322,22 @@ class TTSProvider(TTSProviderBase):
async def start_session(self, session_id): async def start_session(self, session_id):
logger.bind(tag=TAG).info(f"开始会话~~{session_id}") logger.bind(tag=TAG).info(f"开始会话~~{session_id}")
try: try:
# 会话开始时检测上个会话的监听状态 # 等待上一个会话结束,最多等待3次
if ( for _ in range(3):
self._monitor_task is not None if not self.activate_session:
and isinstance(self._monitor_task, Task) break
and not self._monitor_task.done() logger.bind(tag=TAG).debug(f"等待上一个会话结束...")
): await asyncio.sleep(0.1)
logger.bind(tag=TAG).info("检测到未完成的上个会话,关闭监听任务和连接...") else:
# 等待超时,强制清除连接状态
logger.bind(tag=TAG).debug("等待上一个会话超时,清除连接状态...")
await self.close() await self.close()
# 建立新连接 # 设置会话激活标志
await self._ensure_connection() self.activate_session = True
# 启动监听任务 # 确保连接建立
self._monitor_task = asyncio.create_task(self._start_monitor_tts_response()) await self._ensure_connection()
header = Header( header = Header(
message_type=FULL_CLIENT_REQUEST, message_type=FULL_CLIENT_REQUEST,
@@ -365,17 +374,6 @@ class TTSProvider(TTSProviderBase):
await self.send_event(self.ws, header, optional, payload) await self.send_event(self.ws, header, optional, payload)
logger.bind(tag=TAG).info("会话结束请求已发送") logger.bind(tag=TAG).info("会话结束请求已发送")
# 等待监听任务完成
if self._monitor_task:
try:
await self._monitor_task
except Exception as e:
logger.bind(tag=TAG).error(
f"等待监听任务完成时发生错误: {str(e)}"
)
finally:
self._monitor_task = None
except Exception as e: except Exception as e:
logger.bind(tag=TAG).error(f"关闭会话失败: {str(e)}") logger.bind(tag=TAG).error(f"关闭会话失败: {str(e)}")
# 确保清理资源 # 确保清理资源
@@ -405,6 +403,7 @@ class TTSProvider(TTSProviderBase):
async def close(self): async def close(self):
"""资源清理方法""" """资源清理方法"""
self.activate_session = False
# 取消监听任务 # 取消监听任务
if self._monitor_task: if self._monitor_task:
try: try:
@@ -424,9 +423,8 @@ class TTSProvider(TTSProviderBase):
self.ws = None self.ws = None
async def _start_monitor_tts_response(self): async def _start_monitor_tts_response(self):
"""监听TTS响应""" """监听TTS响应 - 长期运行"""
try: try:
session_finished = False # 标记会话是否正常结束
while not self.conn.stop_event.is_set(): while not self.conn.stop_event.is_set():
try: try:
# 确保 `recv()` 运行在同一个 event loop # 确保 `recv()` 运行在同一个 event loop
@@ -434,10 +432,17 @@ class TTSProvider(TTSProviderBase):
res = self.parser_response(msg) res = self.parser_response(msg)
self.print_response(res, "send_text res:") self.print_response(res, "send_text res:")
# 只处理当前活跃会话的响应
if res.optional.sessionId and self.conn.sentence_id != res.optional.sessionId:
# 如果是会话结束相关事件,即使会话ID不匹配也要重置状态
if res.optional.event in [EVENT_SessionCanceled, EVENT_SessionFailed, EVENT_SessionFinished]:
logger.bind(tag=TAG).debug(f"收到残余下行结束响应重置会话状态~~")
self.activate_session = False
continue
if res.optional.event == EVENT_SessionCanceled: if res.optional.event == EVENT_SessionCanceled:
logger.bind(tag=TAG).debug(f"释放服务端资源成功~~") logger.bind(tag=TAG).debug(f"释放服务端资源成功~~")
session_finished = True self.activate_session = False
break
elif res.optional.event == EVENT_TTSSentenceStart: elif res.optional.event == EVENT_TTSSentenceStart:
json_data = json.loads(res.payload.decode("utf-8")) json_data = json.loads(res.payload.decode("utf-8"))
self.tts_text = json_data.get("text", "") self.tts_text = json_data.get("text", "")
@@ -454,9 +459,8 @@ class TTSProvider(TTSProviderBase):
logger.bind(tag=TAG).info(f"句子语音生成成功:{self.tts_text}") logger.bind(tag=TAG).info(f"句子语音生成成功:{self.tts_text}")
elif res.optional.event == EVENT_SessionFinished: elif res.optional.event == EVENT_SessionFinished:
logger.bind(tag=TAG).debug(f"会话结束~~") logger.bind(tag=TAG).debug(f"会话结束~~")
self.activate_session = False
self._process_before_stop_play_files() self._process_before_stop_play_files()
session_finished = True
break
except websockets.ConnectionClosed: except websockets.ConnectionClosed:
logger.bind(tag=TAG).warning("WebSocket连接已关闭") logger.bind(tag=TAG).warning("WebSocket连接已关闭")
break break
@@ -466,8 +470,8 @@ class TTSProvider(TTSProviderBase):
) )
traceback.print_exc() traceback.print_exc()
break break
# 仅在连接异常时关闭 # 连接异常时关闭WebSocket
if not session_finished and self.ws: if self.ws:
try: try:
await self.ws.close() await self.ws.close()
except: except:
@@ -513,7 +517,7 @@ class TTSProvider(TTSProviderBase):
def read_res_content(self, res: bytes, offset: int): def read_res_content(self, res: bytes, offset: int):
content_size = int.from_bytes(res[offset : offset + 4], "big", signed=True) content_size = int.from_bytes(res[offset : offset + 4], "big", signed=True)
offset += 4 offset += 4
content = str(res[offset : offset + content_size]) content = res[offset : offset + content_size].decode('utf-8')
offset += content_size offset += content_size
return content, offset return content, offset
@@ -70,7 +70,9 @@ class VADProvider(VADProviderBase):
# 更新滑动窗口 # 更新滑动窗口
conn.client_voice_window.append(is_voice) conn.client_voice_window.append(is_voice)
client_have_voice = (conn.client_voice_window.count(True) >= self.frame_window_threshold) client_have_voice = (
conn.client_voice_window.count(True) >= self.frame_window_threshold
)
# 如果之前有声音,但本次没有声音,且与上次有声音的时间差已经超过了静默阈值,则认为已经说完一句话 # 如果之前有声音,但本次没有声音,且与上次有声音的时间差已经超过了静默阈值,则认为已经说完一句话
if conn.client_have_voice and not client_have_voice: if conn.client_have_voice and not client_have_voice:
@@ -66,7 +66,7 @@ class PromptManager:
def _load_base_template(self): def _load_base_template(self):
"""加载基础提示词模板""" """加载基础提示词模板"""
try: try:
template_path = "agent-base-prompt.txt" template_path = self.config.get("prompt_template", "agent-base-prompt.txt")
cache_key = f"prompt_template:{template_path}" cache_key = f"prompt_template:{template_path}"
# 先从缓存获取 # 先从缓存获取
@@ -88,7 +88,7 @@ class PromptManager:
self.base_prompt_template = template_content self.base_prompt_template = template_content
self.logger.bind(tag=TAG).debug("成功加载基础提示词模板并缓存") self.logger.bind(tag=TAG).debug("成功加载基础提示词模板并缓存")
else: else:
self.logger.bind(tag=TAG).warning("未找到agent-base-prompt.txt文件") self.logger.bind(tag=TAG).warning(f"未找到{template_path}文件")
except Exception as e: except Exception as e:
self.logger.bind(tag=TAG).error(f"加载提示词模板失败: {e}") self.logger.bind(tag=TAG).error(f"加载提示词模板失败: {e}")
@@ -225,6 +225,7 @@ class PromptManager:
weather_info=weather_info, weather_info=weather_info,
emojiList=EMOJI_List, emojiList=EMOJI_List,
device_id=device_id, device_id=device_id,
client_ip=client_ip,
*args, **kwargs *args, **kwargs
) )
device_cache_key = f"device_prompt:{device_id}" device_cache_key = f"device_prompt:{device_id}"
+51 -18
View File
@@ -176,31 +176,64 @@ def parse_string_to_list(value, separator=";"):
return [] return []
def check_ffmpeg_installed(): def check_ffmpeg_installed() -> bool:
ffmpeg_installed = False """
检查当前环境中是否已正确安装并可执行 ffmpeg。
Returns:
bool: 如果 ffmpeg 正常可用,返回 True;否则抛出 ValueError 异常。
Raises:
ValueError: 当检测到 ffmpeg 未安装或依赖缺失时,抛出详细的提示信息。
"""
try: try:
# 执行ffmpeg -version命令,并捕获输出 # 尝试执行 ffmpeg 命令
result = subprocess.run( result = subprocess.run(
["ffmpeg", "-version"], ["ffmpeg", "-version"],
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, text=True,
check=True, # 如果返回码非零则抛出异常 check=True, # 非零退出码会触发 CalledProcessError
) )
# 检查输出中是否包含版本信息(可选)
output = result.stdout + result.stderr output = (result.stdout + result.stderr).lower()
if "ffmpeg version" in output.lower(): if "ffmpeg version" in output:
ffmpeg_installed = True return True
return False
except (subprocess.CalledProcessError, FileNotFoundError): # 如果未检测到版本信息,也视为异常情况
# 命令执行失败或未找到 raise ValueError("未检测到有效的 ffmpeg 版本输出。")
ffmpeg_installed = False
if not ffmpeg_installed: except (subprocess.CalledProcessError, FileNotFoundError) as e:
error_msg = "您的电脑还没正确安装ffmpeg\n" # 提取错误输出
error_msg += "\n建议您:\n" stderr_output = ""
error_msg += "1、按照项目的安装文档,正确进入conda环境\n" if isinstance(e, subprocess.CalledProcessError):
error_msg += "2、查阅安装文档,如何在conda环境中安装ffmpeg\n" stderr_output = (e.stderr or "").strip()
raise ValueError(error_msg) else:
stderr_output = str(e).strip()
# 构建基础错误提示
error_msg = [
"❌ 检测到 ffmpeg 无法正常运行。\n",
"建议您:",
"1. 确认已正确激活 conda 环境;",
"2. 查阅项目安装文档,了解如何在 conda 环境中安装 ffmpeg。\n",
]
# 🎯 针对具体错误信息提供额外提示
if "libiconv.so.2" in stderr_output:
error_msg.append("⚠️ 发现缺少依赖库:libiconv.so.2")
error_msg.append("解决方法:在当前 conda 环境中执行:")
error_msg.append(" conda install -c conda-forge libiconv\n")
elif "no such file or directory" in stderr_output and "ffmpeg" in stderr_output.lower():
error_msg.append("⚠️ 系统未找到 ffmpeg 可执行文件。")
error_msg.append("解决方法:在当前 conda 环境中执行:")
error_msg.append(" conda install -c conda-forge ffmpeg\n")
else:
error_msg.append("错误详情:")
error_msg.append(stderr_output or "未知错误。")
# 抛出详细异常信息
raise ValueError("\n".join(error_msg)) from e
def extract_json_from_string(input_string): def extract_json_from_string(input_string):
+11 -2
View File
@@ -4,7 +4,7 @@
"后面不断测试补充好用的mcp服务,欢迎大家一起补充。", "后面不断测试补充好用的mcp服务,欢迎大家一起补充。",
"记得删除注释行,des属性仅为说明,不会被解析。", "记得删除注释行,des属性仅为说明,不会被解析。",
"des和link属性,仅为说明安装方式,方便大家查看原始链接,不是必须项。", "des和link属性,仅为说明安装方式,方便大家查看原始链接,不是必须项。",
"当前支持stdio/sse两种模式。" "当前支持三种传输模式:stdio(标准输入输出), sse(Server-Sent Events), streamable-http(流式HTTP)。"
], ],
"mcpServers": { "mcpServers": {
"Home Assistant": { "Home Assistant": {
@@ -41,7 +41,16 @@
"url": "http://localhost:8080/sse", "url": "http://localhost:8080/sse",
"headers": { "headers": {
"Authorization": "Bearer YOUR TOKEN" "Authorization": "Bearer YOUR TOKEN"
} },
"des": "使用SSE传输模式(默认)"
},
"streamable-http-mcp-server": {
"url": "http://localhost:8000/mcp",
"transport": "streamable-http",
"headers": {
"Authorization": "Bearer YOUR TOKEN"
},
"des": "使用Streamable HTTP传输模式,适用于生产环境的Web部署"
} }
} }
} }
@@ -38,7 +38,7 @@ class LLMPerformanceTester:
"""加载系统提示词""" """加载系统提示词"""
try: try:
prompt_file = os.path.join( prompt_file = os.path.join(
os.path.dirname(os.path.dirname(__file__)), "agent-base-prompt.txt" os.path.dirname(os.path.dirname(__file__)), self.config.get("prompt_template", "agent-base-prompt.txt")
) )
with open(prompt_file, "r", encoding="utf-8") as f: with open(prompt_file, "r", encoding="utf-8") as f:
content = f.read() content = f.read()
+5 -5
View File
@@ -7,25 +7,25 @@ numpy==1.26.4
pydub==0.25.1 pydub==0.25.1
funasr==1.2.3 funasr==1.2.3
torchaudio==2.2.2 torchaudio==2.2.2
openai==1.107.0 openai==2.5.0
google-generativeai==0.8.5 google-generativeai==0.8.5
edge_tts==7.0.0 edge_tts==7.0.0
httpx==0.27.2 httpx==0.27.2
aiohttp==3.12.15 aiohttp==3.12.15
aiohttp_cors==0.7.0 aiohttp_cors==0.7.0
ormsgpack==1.7.0 ormsgpack==1.11.0
ruamel.yaml==0.18.15 ruamel.yaml==0.18.15
loguru==0.7.3 loguru==0.7.3
requests==2.32.5 requests==2.32.5
cozepy==0.19.0 cozepy==0.19.0
mem0ai==0.1.62 mem0ai==1.0.0
bs4==0.0.2 bs4==0.0.2
modelscope==1.23.2 modelscope==1.23.2
sherpa_onnx==1.12.11 sherpa_onnx==1.12.11
mcp==1.13.1 mcp==1.13.1
cnlunar==0.2.0 cnlunar==0.2.0
PySocks==1.7.1 PySocks==1.7.1
dashscope==1.23.1 dashscope==1.24.6
baidu-aip==4.16.13 baidu-aip==4.16.13
chardet==5.2.0 chardet==5.2.0
aioconsole==0.8.1 aioconsole==0.8.1
@@ -35,4 +35,4 @@ PyJWT==2.8.0
psutil==7.0.0 psutil==7.0.0
portalocker==3.2.0 portalocker==3.2.0
Jinja2==3.1.6 Jinja2==3.1.6
vosk==0.3.45 vosk==0.3.44
@@ -0,0 +1,72 @@
[
{
"name": "self.get_device_status",
"description": "Provides the real-time information of the device, including the current status of the audio speaker, screen, battery, network, etc.\nUse this tool for: \n1. Answering questions about current condition (e.g. what is the current volume of the audio speaker?)\n2. As the first step to control the device (e.g. turn up / down the volume of the audio speaker, etc.)",
"inputSchema": {
"type": "object",
"properties": {}
},
"mockResponse": {
"audio_speaker": {
"volume": 50,
"muted": false
},
"screen": {
"brightness": 80,
"theme": "light"
},
"battery": {
"level": 85,
"charging": false
},
"network": {
"connected": true,
"type": "wifi"
}
}
},
{
"name": "self.audio_speaker.set_volume",
"description": "Set the volume of the audio speaker. If the current volume is unknown, you must call `self.get_device_status` tool first and then call this tool.",
"inputSchema": {
"type": "object",
"properties": {
"volume": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
},
"required": [
"volume"
]
},
"mockResponse": {
"success": true,
"volume": "${volume}",
"message": "音量已设置为 ${volume}"
}
},
{
"name": "self.screen.set_brightness",
"description": "Set the brightness of the screen.",
"inputSchema": {
"type": "object",
"properties": {
"brightness": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
},
"required": [
"brightness"
]
},
"mockResponse": {
"success": true,
"brightness": "${brightness}",
"message": "亮度已设置为 ${brightness}"
}
}
]
+3 -3
View File
@@ -21,9 +21,9 @@ h1 {
} }
.section { .section {
margin-bottom: 20px; margin-bottom: 5px;
padding: 15px; padding: 10px;
border-radius: 8px; border-radius: 10px;
background-color: #f9f9f9; background-color: #f9f9f9;
} }
+718 -67
View File
@@ -56,6 +56,163 @@
#fileProtocolWarning button:hover { #fileProtocolWarning button:hover {
background-color: #45a049; background-color: #45a049;
} }
/* MCP 工具管理样式 */
.mcp-tools-container {
display: grid;
gap: 12px;
margin-top: 10px;
}
.mcp-tool-card {
background-color: white;
border: 1px solid #e0e0e0;
border-radius: 8px;
padding: 15px;
transition: all 0.2s;
}
.mcp-tool-card:hover {
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
.mcp-tool-card.disabled {
opacity: 0.6;
pointer-events: none;
}
.mcp-tool-header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 10px;
}
.mcp-tool-name {
font-size: 15px;
font-weight: 600;
color: #333;
flex: 1;
}
.mcp-tool-actions {
display: flex;
gap: 6px;
}
.mcp-tool-description {
color: #666;
font-size: 13px;
line-height: 1.5;
margin-bottom: 8px;
}
.mcp-tool-info {
background-color: #f9f9f9;
border: 1px solid #e0e0e0;
border-radius: 4px;
padding: 8px;
font-size: 12px;
}
.mcp-tool-info-row {
display: flex;
gap: 15px;
margin-bottom: 4px;
}
.mcp-tool-info-label {
color: #999;
min-width: 60px;
}
.mcp-tool-info-value {
color: #333;
font-family: 'Courier New', monospace;
}
.mcp-property-item {
background-color: white;
border: 1px solid #e0e0e0;
border-radius: 5px;
padding: 12px;
margin-bottom: 10px;
}
.mcp-property-header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 10px;
}
.mcp-property-name {
font-weight: 600;
color: #333;
}
.mcp-property-row {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 10px;
margin-bottom: 8px;
}
.mcp-property-row-full {
margin-bottom: 8px;
}
.mcp-small-label {
display: block;
margin-bottom: 4px;
font-size: 12px;
color: #666;
}
.mcp-small-input {
width: 100%;
padding: 6px 8px;
border: 1px solid #ddd;
border-radius: 4px;
font-size: 13px;
}
.mcp-checkbox-label {
display: flex;
align-items: center;
gap: 6px;
font-size: 13px;
color: #666;
cursor: pointer;
}
.mcp-error {
background-color: #ffebee;
color: #c62828;
padding: 10px;
border-radius: 5px;
margin-bottom: 15px;
font-size: 14px;
}
.mcp-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 12px;
font-size: 11px;
font-weight: 500;
margin-left: 8px;
}
.mcp-badge-required {
background-color: #ffebee;
color: #c62828;
}
.mcp-badge-optional {
background-color: #e3f2fd;
color: #1976d2;
}
</style> </style>
<script> <script>
// 检测是否使用file://协议打开 // 检测是否使用file://协议打开
@@ -168,6 +325,83 @@
</div> </div>
</div> </div>
</div> </div>
<!-- MCP 工具管理区域 -->
<div class="section">
<h2>
MCP 工具管理
<span class="connection-status">
<span id="mcpToolsCount">0 个工具</span>
</span>
<button class="toggle-button" id="toggleMcpTools">展开</button>
</h2>
<div class="config-panel" id="mcpToolsPanel">
<div id="mcpToolsContainer" class="mcp-tools-container">
<!-- 工具列表将动态插入这里 -->
</div>
<div style="text-align: center; padding: 15px 0;">
<button class="btn" id="addMcpToolBtn" style="background-color: #4caf50;">
➕ 添加新工具
</button>
</div>
</div>
</div>
<!-- MCP 工具编辑模态框 -->
<div id="mcpToolModal" style="display: none; position: fixed; top: 0; left: 0; width: 100%; height: 100%; background-color: rgba(0, 0, 0, 0.5); z-index: 9999; overflow-y: auto;">
<div style="background-color: white; border-radius: 10px; padding: 25px; width: 90%; max-width: 700px; margin: 50px auto; max-height: 85vh; overflow-y: auto;">
<div style="display: flex; align-items: center; justify-content: space-between; margin-bottom: 20px; padding-bottom: 15px; border-bottom: 2px solid #e0e0e0;">
<h2 id="mcpModalTitle" style="font-size: 20px; font-weight: bold; color: #333; margin: 0;">添加工具</h2>
<button id="closeMcpModalBtn" style="background: none; border: none; font-size: 24px; cursor: pointer; color: #666; padding: 0; width: 30px; height: 30px;">&times;</button>
</div>
<div id="mcpErrorContainer"></div>
<form id="mcpToolForm">
<div style="margin-bottom: 20px;">
<label style="display: block; margin-bottom: 8px; font-weight: 500; color: #333; font-size: 14px;">工具名称 *</label>
<input type="text" id="mcpToolName" placeholder="例如: self.get_device_status" required
style="width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 5px; font-size: 14px;">
</div>
<div style="margin-bottom: 20px;">
<label style="display: block; margin-bottom: 8px; font-weight: 500; color: #333; font-size: 14px;">工具描述 *</label>
<textarea id="mcpToolDescription" placeholder="详细描述工具的功能和使用场景..." required
style="width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 5px; font-size: 14px; min-height: 80px; resize: vertical;"></textarea>
</div>
<div style="margin-bottom: 20px;">
<label style="display: block; margin-bottom: 8px; font-weight: 500; color: #333; font-size: 14px;">输入参数</label>
<div style="background-color: #f9f9f9; border: 1px solid #ddd; border-radius: 5px; padding: 15px;">
<div id="mcpPropertiesContainer">
<div style="text-align: center; padding: 20px; color: #999; font-size: 14px;">暂无参数,点击下方按钮添加参数</div>
</div>
<button type="button" id="addMcpPropertyBtn"
style="width: 100%; margin-top: 10px; padding: 8px 15px; border: none; border-radius: 5px; background-color: #2196f3; color: white; cursor: pointer; font-size: 14px;">
➕ 添加参数
</button>
</div>
</div>
<div style="margin-bottom: 20px;">
<label style="display: block; margin-bottom: 8px; font-weight: 500; color: #333; font-size: 14px;">
模拟返回结果 (JSON 格式,可选)
<span style="font-size: 12px; color: #999; font-weight: normal;">- 留空则返回默认成功消息</span>
</label>
<textarea id="mcpMockResponse" placeholder='{"success": true, "data": "执行成功"}'
style="width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 5px; font-size: 13px; font-family: 'Courier New', monospace; min-height: 100px; resize: vertical;"></textarea>
<div style="font-size: 12px; color: #666; margin-top: 4px;">
💡 提示:如果设置了模拟返回结果,工具调用时将返回这个 JSON 对象
</div>
</div>
<div style="display: flex; gap: 10px; justify-content: flex-end; margin-top: 25px;">
<button type="button" id="cancelMcpBtn" style="padding: 8px 15px; border: none; border-radius: 5px; background-color: #9e9e9e; color: white; cursor: pointer; font-size: 14px;">取消</button>
<button type="submit" style="padding: 8px 15px; border: none; border-radius: 5px; background-color: #4caf50; color: white; cursor: pointer; font-size: 14px;">保存</button>
</div>
</form>
</div>
</div>
</div> </div>
<!-- Opus解码库 --> <!-- Opus解码库 -->
@@ -810,74 +1044,63 @@
} else if (message.type === 'mcp') { } else if (message.type === 'mcp') {
const payload = message.payload || {}; const payload = message.payload || {};
log(`服务器下发: ${JSON.stringify(message)}`, 'info'); log(`服务器下发: ${JSON.stringify(message)}`, 'info');
if (payload) {
// 模拟小智客户端行为
if (payload.method === 'tools/list') {
const replay_message = JSON.stringify({
"session_id": "", "type": "mcp", "payload": {
"jsonrpc": "2.0", "id": 2, "result": {
"tools": [{
"name": "self.get_device_status",
"description": "Provides the real-time information of the device, including the current status of the audio speaker, screen, battery, network, etc.\nUse this tool for: \n1. Answering questions about current condition (e.g. what is the current volume of the audio speaker?)\n2. As the first step to control the device (e.g. turn up / down the volume of the audio speaker, etc.)",
"inputSchema": { "type": "object", "properties": {} }
}, {
"name": "self.audio_speaker.set_volume",
"description": "Set the volume of the audio speaker. If the current volume is unknown, you must call `self.get_device_status` tool first and then call this tool.",
"inputSchema": {
"type": "object",
"properties": {
"volume": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
},
"required": ["volume"]
}
}, {
"name": "self.screen.set_brightness",
"description": "Set the brightness of the screen.",
"inputSchema": {
"type": "object",
"properties": {
"brightness": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
},
"required": ["brightness"]
}
}, {
"name": "self.screen.set_theme",
"description": "Set the theme of the screen. The theme can be 'light' or 'dark'.",
"inputSchema": {
"type": "object",
"properties": { "theme": { "type": "string" } },
"required": ["theme"]
}
}]
}
}
})
websocket.send(replay_message);
log(`回复MCP消息: ${replay_message}`, 'info');
} else if (payload.method === 'tools/call') {
// 模拟回复
const replay_message = JSON.stringify({
"session_id": "9f261599",
"type": "mcp",
"payload": {
"jsonrpc": "2.0",
"id": payload.id,
"result": { "content": [{ "type": "text", "text": "true" }], "isError": false }
}
})
websocket.send(replay_message);
log(`回复MCP消息: ${replay_message}`, 'info');
}
}
if (payload.method === 'tools/list') {
// 返回工具列表
const tools = getMcpTools();
const replyMessage = JSON.stringify({
"session_id": message.session_id || "",
"type": "mcp",
"payload": {
"jsonrpc": "2.0",
"id": payload.id,
"result": {
"tools": tools
}
}
});
log(`客户端上报: ${replyMessage}`, 'info');
websocket.send(replyMessage);
log(`回复MCP工具列表: ${tools.length} 个工具`, 'info');
} else if (payload.method === 'tools/call') {
// 调用工具
const toolName = payload.params?.name;
const toolArgs = payload.params?.arguments;
log(`调用工具: ${toolName} 参数: ${JSON.stringify(toolArgs)}`, 'info');
// 执行工具
const result = executeMcpTool(toolName, toolArgs);
const replyMessage = JSON.stringify({
"session_id": message.session_id || "",
"type": "mcp",
"payload": {
"jsonrpc": "2.0",
"id": payload.id,
"result": {
"content": [
{
"type": "text",
"text": JSON.stringify(result)
}
],
"isError": false
}
}
});
log(`客户端上报: ${replyMessage}`, 'info');
websocket.send(replyMessage);
} else if(payload.method === 'initialize') {
log(`收到工具初始化请求: ${JSON.stringify(payload.params)}`, 'info');
// 目前仅记录日志
} else {
log(`未知的MCP方法: ${payload.method}`, 'warning');
}
} else { } else {
// 未知消息类型 // 未知消息类型
log(`未知消息类型: ${message.type}`, 'info'); log(`未知消息类型: ${message.type}`, 'info');
@@ -1566,7 +1789,435 @@
}; };
} }
// ==========================================
// MCP 工具管理逻辑
// ==========================================
// 默认工具数据
const defaultMcpTools = await fetch("default-mcp-tools.json").then(res => res.json());
// 全局变量
let mcpTools = [];
let mcpEditingIndex = null;
let mcpProperties = [];
// 初始化 MCP 工具
function initMcpTools() {
const savedTools = localStorage.getItem('mcpTools');
if (savedTools) {
try {
mcpTools = JSON.parse(savedTools);
} catch (e) {
log('加载MCP工具失败,使用默认工具', 'warning');
mcpTools = [...defaultMcpTools];
}
} else {
mcpTools = [...defaultMcpTools];
}
renderMcpTools();
setupMcpEventListeners();
}
// 渲染工具列表
function renderMcpTools() {
const container = document.getElementById('mcpToolsContainer');
const countSpan = document.getElementById('mcpToolsCount');
countSpan.textContent = `${mcpTools.length} 个工具`;
if (mcpTools.length === 0) {
container.innerHTML = '<div style="text-align: center; padding: 30px; color: #999;">暂无工具,点击下方按钮添加新工具</div>';
return;
}
container.innerHTML = mcpTools.map((tool, index) => {
const paramCount = tool.inputSchema.properties ? Object.keys(tool.inputSchema.properties).length : 0;
const requiredCount = tool.inputSchema.required ? tool.inputSchema.required.length : 0;
const hasMockResponse = tool.mockResponse && Object.keys(tool.mockResponse).length > 0;
return `
<div class="mcp-tool-card">
<div class="mcp-tool-header">
<div class="mcp-tool-name">${tool.name}</div>
<div class="mcp-tool-actions">
<button onclick="editMcpTool(${index})"
style="padding: 4px 10px; border: none; border-radius: 4px; background-color: #2196f3; color: white; cursor: pointer; font-size: 12px;">
✏️ 编辑
</button>
<button onclick="deleteMcpTool(${index})"
style="padding: 4px 10px; border: none; border-radius: 4px; background-color: #f44336; color: white; cursor: pointer; font-size: 12px;">
🗑️ 删除
</button>
</div>
</div>
<div class="mcp-tool-description">${tool.description}</div>
<div class="mcp-tool-info">
<div class="mcp-tool-info-row">
<span class="mcp-tool-info-label">参数数量:</span>
<span class="mcp-tool-info-value">${paramCount}${requiredCount > 0 ? `(${requiredCount} 个必填)` : ''}</span>
</div>
<div class="mcp-tool-info-row">
<span class="mcp-tool-info-label">模拟返回:</span>
<span class="mcp-tool-info-value">${hasMockResponse ? '✅ 已配置: ' + JSON.stringify(tool.mockResponse) : '⚪ 使用默认'}</span>
</div>
</div>
</div>
`;
}).join('');
}
// 渲染参数列表
function renderMcpProperties() {
const container = document.getElementById('mcpPropertiesContainer');
if (mcpProperties.length === 0) {
container.innerHTML = '<div style="text-align: center; padding: 20px; color: #999; font-size: 14px;">暂无参数,点击下方按钮添加参数</div>';
return;
}
container.innerHTML = mcpProperties.map((prop, index) => `
<div class="mcp-property-item">
<div class="mcp-property-header">
<span class="mcp-property-name">${prop.name}</span>
<button type="button" onclick="deleteMcpProperty(${index})"
style="padding: 3px 8px; border: none; border-radius: 3px; background-color: #f44336; color: white; cursor: pointer; font-size: 11px;">
删除
</button>
</div>
<div class="mcp-property-row">
<div>
<label class="mcp-small-label">参数名称 *</label>
<input type="text" class="mcp-small-input" value="${prop.name}"
onchange="updateMcpProperty(${index}, 'name', this.value)" required>
</div>
<div>
<label class="mcp-small-label">数据类型 *</label>
<select class="mcp-small-input" onchange="updateMcpProperty(${index}, 'type', this.value)">
<option value="string" ${prop.type === 'string' ? 'selected' : ''}>字符串</option>
<option value="integer" ${prop.type === 'integer' ? 'selected' : ''}>整数</option>
<option value="number" ${prop.type === 'number' ? 'selected' : ''}>数字</option>
<option value="boolean" ${prop.type === 'boolean' ? 'selected' : ''}>布尔值</option>
<option value="array" ${prop.type === 'array' ? 'selected' : ''}>数组</option>
<option value="object" ${prop.type === 'object' ? 'selected' : ''}>对象</option>
</select>
</div>
</div>
${(prop.type === 'integer' || prop.type === 'number') ? `
<div class="mcp-property-row">
<div>
<label class="mcp-small-label">最小值</label>
<input type="number" class="mcp-small-input" value="${prop.minimum !== undefined ? prop.minimum : ''}"
placeholder="可选" onchange="updateMcpProperty(${index}, 'minimum', this.value ? parseFloat(this.value) : undefined)">
</div>
<div>
<label class="mcp-small-label">最大值</label>
<input type="number" class="mcp-small-input" value="${prop.maximum !== undefined ? prop.maximum : ''}"
placeholder="可选" onchange="updateMcpProperty(${index}, 'maximum', this.value ? parseFloat(this.value) : undefined)">
</div>
</div>
` : ''}
<div class="mcp-property-row-full">
<label class="mcp-small-label">参数描述</label>
<input type="text" class="mcp-small-input" value="${prop.description || ''}"
placeholder="可选" onchange="updateMcpProperty(${index}, 'description', this.value)">
</div>
<label class="mcp-checkbox-label">
<input type="checkbox" ${prop.required ? 'checked' : ''}
onchange="updateMcpProperty(${index}, 'required', this.checked)">
必填参数
</label>
</div>
`).join('');
}
// 添加参数
function addMcpProperty() {
mcpProperties.push({
name: `param_${mcpProperties.length + 1}`,
type: 'string',
required: false,
description: ''
});
renderMcpProperties();
}
// 更新参数
window.updateMcpProperty = function(index, field, value) {
if (field === 'name') {
const isDuplicate = mcpProperties.some((p, i) => i !== index && p.name === value);
if (isDuplicate) {
alert('参数名称已存在,请使用不同的名称');
renderMcpProperties();
return;
}
}
mcpProperties[index][field] = value;
if (field === 'type' && value !== 'integer' && value !== 'number') {
delete mcpProperties[index].minimum;
delete mcpProperties[index].maximum;
renderMcpProperties();
}
};
// 删除参数
window.deleteMcpProperty = function(index) {
mcpProperties.splice(index, 1);
renderMcpProperties();
};
// 设置事件监听
function setupMcpEventListeners() {
const toggleBtn = document.getElementById('toggleMcpTools');
const panel = document.getElementById('mcpToolsPanel');
const addBtn = document.getElementById('addMcpToolBtn');
const modal = document.getElementById('mcpToolModal');
const closeBtn = document.getElementById('closeMcpModalBtn');
const cancelBtn = document.getElementById('cancelMcpBtn');
const form = document.getElementById('mcpToolForm');
const addPropertyBtn = document.getElementById('addMcpPropertyBtn');
toggleBtn.addEventListener('click', () => {
const isExpanded = panel.classList.contains('expanded');
panel.classList.toggle('expanded');
toggleBtn.textContent = isExpanded ? '展开' : '收起';
});
addBtn.addEventListener('click', () => openMcpModal());
closeBtn.addEventListener('click', closeMcpModal);
cancelBtn.addEventListener('click', closeMcpModal);
addPropertyBtn.addEventListener('click', addMcpProperty);
modal.addEventListener('click', (e) => {
if (e.target === modal) closeMcpModal();
});
form.addEventListener('submit', handleMcpSubmit);
}
// 打开模态框
function openMcpModal(index = null) {
const isConnected = websocket && websocket.readyState === WebSocket.OPEN;
if (isConnected) {
alert('WebSocket 已连接,无法编辑工具');
return;
}
mcpEditingIndex = index;
const errorContainer = document.getElementById('mcpErrorContainer');
errorContainer.innerHTML = '';
if (index !== null) {
document.getElementById('mcpModalTitle').textContent = '编辑工具';
const tool = mcpTools[index];
document.getElementById('mcpToolName').value = tool.name;
document.getElementById('mcpToolDescription').value = tool.description;
document.getElementById('mcpMockResponse').value = tool.mockResponse ? JSON.stringify(tool.mockResponse, null, 2) : '';
mcpProperties = [];
const schema = tool.inputSchema;
if (schema.properties) {
Object.keys(schema.properties).forEach(key => {
const prop = schema.properties[key];
mcpProperties.push({
name: key,
type: prop.type || 'string',
minimum: prop.minimum,
maximum: prop.maximum,
description: prop.description || '',
required: schema.required && schema.required.includes(key)
});
});
}
} else {
document.getElementById('mcpModalTitle').textContent = '添加工具';
document.getElementById('mcpToolForm').reset();
mcpProperties = [];
}
renderMcpProperties();
document.getElementById('mcpToolModal').style.display = 'block';
}
// 关闭模态框
function closeMcpModal() {
document.getElementById('mcpToolModal').style.display = 'none';
mcpEditingIndex = null;
document.getElementById('mcpToolForm').reset();
mcpProperties = [];
document.getElementById('mcpErrorContainer').innerHTML = '';
}
// 处理表单提交
function handleMcpSubmit(e) {
e.preventDefault();
const errorContainer = document.getElementById('mcpErrorContainer');
errorContainer.innerHTML = '';
const name = document.getElementById('mcpToolName').value.trim();
const description = document.getElementById('mcpToolDescription').value.trim();
const mockResponseText = document.getElementById('mcpMockResponse').value.trim();
// 检查名称重复
const isDuplicate = mcpTools.some((tool, index) =>
tool.name === name && index !== mcpEditingIndex
);
if (isDuplicate) {
showMcpError('工具名称已存在,请使用不同的名称');
return;
}
// 解析模拟返回结果
let mockResponse = null;
if (mockResponseText) {
try {
mockResponse = JSON.parse(mockResponseText);
} catch (e) {
showMcpError('模拟返回结果不是有效的 JSON 格式: ' + e.message);
return;
}
}
// 构建 inputSchema
const inputSchema = {
type: "object",
properties: {},
required: []
};
mcpProperties.forEach(prop => {
const propSchema = { type: prop.type };
if (prop.description) {
propSchema.description = prop.description;
}
if ((prop.type === 'integer' || prop.type === 'number')) {
if (prop.minimum !== undefined && prop.minimum !== '') {
propSchema.minimum = prop.minimum;
}
if (prop.maximum !== undefined && prop.maximum !== '') {
propSchema.maximum = prop.maximum;
}
}
inputSchema.properties[prop.name] = propSchema;
if (prop.required) {
inputSchema.required.push(prop.name);
}
});
if (inputSchema.required.length === 0) {
delete inputSchema.required;
}
const tool = { name, description, inputSchema, mockResponse };
if (mcpEditingIndex !== null) {
mcpTools[mcpEditingIndex] = tool;
log(`已更新工具: ${name}`, 'success');
} else {
mcpTools.push(tool);
log(`已添加工具: ${name}`, 'success');
}
saveMcpTools();
renderMcpTools();
closeMcpModal();
}
// 显示错误
function showMcpError(message) {
const errorContainer = document.getElementById('mcpErrorContainer');
errorContainer.innerHTML = `<div class="mcp-error">${message}</div>`;
}
// 编辑工具
window.editMcpTool = function(index) {
openMcpModal(index);
};
// 删除工具
window.deleteMcpTool = function(index) {
const isConnected = websocket && websocket.readyState === WebSocket.OPEN;
if (isConnected) {
alert('WebSocket 已连接,无法编辑工具');
return;
}
if (confirm(`确定要删除工具 "${mcpTools[index].name}" 吗?`)) {
const toolName = mcpTools[index].name;
mcpTools.splice(index, 1);
saveMcpTools();
renderMcpTools();
log(`已删除工具: ${toolName}`, 'info');
}
};
// 保存工具
function saveMcpTools() {
localStorage.setItem('mcpTools', JSON.stringify(mcpTools));
}
// 获取工具列表
function getMcpTools() {
return mcpTools.map(tool => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema
}));
}
// 执行工具调用
function executeMcpTool(toolName, toolArgs) {
const tool = mcpTools.find(t => t.name === toolName);
if (!tool) {
log(`未找到工具: ${toolName}`, 'error');
return {
success: false,
error: `未知工具: ${toolName}`
};
}
// 如果有模拟返回结果,使用它
if (tool.mockResponse) {
// 替换模板变量
let responseStr = JSON.stringify(tool.mockResponse);
// 替换 ${paramName} 格式的变量
if (toolArgs) {
Object.keys(toolArgs).forEach(key => {
const regex = new RegExp(`\\$\\{${key}\\}`, 'g');
responseStr = responseStr.replace(regex, toolArgs[key]);
});
}
try {
const response = JSON.parse(responseStr);
log(`工具 ${toolName} 执行成功,返回模拟结果: ${responseStr}`, 'success');
return response;
} catch (e) {
log(`解析模拟返回结果失败: ${e.message}`, 'error');
return tool.mockResponse;
}
}
// 没有模拟返回结果,返回默认成功消息
log(`工具 ${toolName} 执行成功,返回默认结果`, 'success');
return {
success: true,
message: `工具 ${toolName} 执行成功`,
tool: toolName,
arguments: toolArgs
};
}
initApp(); initApp();
initMcpTools();
</script> </script>
</body> </body>