Hass/xiaozhi-esp32-server
1
0
Fork 0
mirror of https://github.com/xinnan-tech/xiaozhi-esp32-server.git synced 2026-07-22 07:03:53 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'xinnan-tech:main' into main

Browse Source
This commit is contained in:
GoodyHao
2025-04-14 18:51:15 +08:00
committed by GitHub
parent c76f28824c 2bab917b3e
commit 978cd8ba8e
16 changed files with 226 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+45 -69
View File
@@ -145,42 +145,24 @@ server:
---
## 部署方式 🚀
## 部署文档
[![Banners](docs/images/banner2.png)](./docs/Deployment.md)
本项目提供两种部署方式，请根据您的具体需求选择：
### 部署文档
#### 🚀 部署方式选择
本项目支持`最简化`安装和`全模块`安装。请根据具体需求选择安装方式。
| 部署方式 | 特点 | 适用场景 | Docker部署文档 | 源码部署文档 |
|---------|------|---------|---------|---------|
| **最简化安装** | 智能对话、IOT功能，数据存储在配置文件 | 低配置环境，无需数据库 | [Docker只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) | [本地源码只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)|
| **全模块安装** | 智能对话、IOT、OTA、智控台，数据存储在数据库 | 完整功能体验 |[Docker运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) | [本地源码运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) |
`最简化`适合在较低资源配置中运行，无需数据库。[请参考-最简化安装架构图](./docs/images/deploy1.png)
`全模块`适合想体验完整功能，过程复杂。[请参考-全模块安装架构图](./docs/images/deploy2.png)
#### 1、 最简化安装
`最简化`安装包含：智能对话、IOT等功能，数据存储在`配置文件`。我们提供了两种安装教程，如果你决定使用最简化安装，可选择`Docker只运行Server`或`本地源码只运行Server`任意一种方式。
##### 1.1. Docker只运行Server
你可以使用编译好的docker镜像安装，[点击Docker只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)。优点：减少环境配置的烦恼；缺点：版本可能有点旧，镜像文件较大。
##### 2.2. 本地源码只运行Server
你可以以本地源码的方式运行，[点击本地源码只运行Server](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)。优点：可以随时更新最新代码；缺点：安装Conda环境较为复杂。
#### 2、 全模块安装
`全模块`安装包含：智能对话、IOT、OTA、智控台，数据存储在`数据库`中。我们提供了两种安装教程，如果你决定使用全模块安装，可选择`Docker运行全模块`或`本地源码运行全模块`任意一种方式。
##### 2.1. Docker运行全模块
你可以使用编译好的docker镜像安装，[点击Docker运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97)。优点：减少环境配置的烦恼；缺点：版本可能有点旧，镜像文件较大。
##### 2.2. 本地源码运行全模块
你可以以本地源码的方式运行，[点击本地源码运行全模块](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97)。优点：可以随时更新最新代码；缺点：安装Java、Node.js、Conda环境较为复杂。
> 💡 提示：以下是按最新代码部署后的测试平台，有需要可烧录测试，并发为6个，每天会清空数据
```
智控台地址: https://2662r3426b.vicp.fun
OTA接口地址: htts://2662r3426b.vicp.fun/xiaozhi/ota/
Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
```
---
## 常见问题 ❓
@@ -188,54 +170,41 @@ server:
如遇到问题或产品建议反馈[点这里](docs/FAQ.md)。
---
## 产品生态 👬
小智是一个生态，当你使用这个产品时，也可以看看其他在这个生态圈的优秀项目
- [小智安卓客户端](https://github.com/TOM88812/xiaozhi-android-client)
一个基于xiaozhi-server的Android、IOS语音对话应用,支持实时语音交互和文字对话。现在是flutter版本，打通IOS、Android端。
- [小智电脑客户端](https://github.com/Huang-junsen/py-xiaozhi)
该项目提供了一个基于 Python 实现的小白 AI 客户端，使得在不具备实体硬件条件的情况下，依然能够体过代码体验小智 AI 的功能。主要功能包括 AI 语音交互、视觉多模态识别、IoT 设备集成、联网音乐播放、语音唤醒、自动对话模式、图形化界面、命令行模式、跨平台支持、音量控制、会话管理、加密音频传输、自动验证码处理等。
- [小智Java服务端](https://github.com/joey-zhou/xiaozhi-esp32-server-java)
小智开源后端服务 Java 版本是一个基于 Java 的开源项目，它包括前后端的服务，旨在为用户提供一个完整的后端服务解决方案。
---
## 功能清单 ✨
### 已实现 ✅
- **通信协议**
基于 `xiaozhi-esp32` 协议，通过 WebSocket 实现数据交互。
- **对话交互**
支持唤醒对话、手动对话及实时打断。长时间无对话时自动休眠
- **意图识别**
支持使用LLM意图识别、function call函数调用，减少硬编码意图判断
- **多语言识别**
支持国语、粤语、英语、日语、韩语（默认使用 FunASR）。
- **LLM 模块**
支持灵活切换 LLM 模块，默认使用 ChatGLMLLM，也可选用阿里百炼、DeepSeek、Ollama 等接口。
- **TTS 模块**
支持 EdgeTTS（默认）、火山引擎豆包 TTS 等多种 TTS 接口，满足语音合成需求。
- **记忆功能**
支持超长记忆、本地总结记忆、无记忆三种模式，满足不同场景需求。
- **IOT功能**
支持管理注册设备IOT功能，支持基于对话上下文语境下的智能物联网控制。
- **智控台**
提供Web管理界面，支持智能体管理、用户管理、系统配置等功能，方便管理员和用户进行管理。
| 功能模块 | 描述 |
|---------|------|
| 通信协议 | 基于 `xiaozhi-esp32` 协议，通过 WebSocket 实现数据交互 |
| 对话交互 | 支持唤醒对话、手动对话及实时打断。长时间无对话时自动休眠 |
| 意图识别 | 支持使用LLM意图识别、function call函数调用，减少硬编码意图判断 |
| 多语言识别 | 支持国语、粤语、英语、日语、韩语（默认使用 FunASR） |
| LLM 模块 | 支持灵活切换 LLM 模块，默认使用 ChatGLMLLM，也可选用阿里百炼、DeepSeek、Ollama 等接口 |
| TTS 模块 | 支持 EdgeTTS（默认）、火山引擎豆包 TTS 等多种 TTS 接口，满足语音合成需求 |
| 记忆功能 | 支持超长记忆、本地总结记忆、无记忆三种模式，满足不同场景需求 |
| IOT功能 | 支持管理注册设备IOT功能，支持基于对话上下文语境下的智能物联网控制 |
| 智控台 | 提供Web管理界面，支持智能体管理、用户管理、系统配置等功能，方便管理员和用户进行管理 |
### 正在开发 🚧
- 多种心情模式
想了解具体开发进度，[请点击这里](https://github.com/users/xinnan-tech/projects/3)
想了解具体开发计划进度，[请点击这里](https://github.com/users/xinnan-tech/projects/3)
如果你是一名软件开发者，这里有一份[《致开发者的公开信》](docs/contributor_open_letter.md)，欢迎加入！
---
## 产品生态 👬
小智是一个生态，当你使用这个产品时，也可以看看其他在这个生态圈的优秀项目
| 项目名称 | 项目地址 | 项目描述 |
|:---------------------|:--------|:--------|
| 小智安卓客户端 | [xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) | 一个基于xiaozhi-server的Android、IOS语音对话应用,支持实时语音交互和文字对话。<br/>现在是flutter版本，打通IOS、Android端。 |
| 小智电脑客户端 | [py-xiaozhi](https://github.com/Huang-junsen/py-xiaozhi) | 该项目提供了一个基于 Python 实现的小白 AI 客户端，使得在不具备实体硬件条件的情况下，<br/>依然能够体过代码体验小智 AI 的功能。 |
| 小智Java服务端 | [xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) | 小智开源后端服务 Java 版本是一个基于 Java 的开源项目。<br/>它包括前后端的服务，旨在为用户提供一个完整的后端服务解决方案。 |
---
## 本项目支持的平台/组件列表 📋
### LLM 语言模型
@@ -298,10 +267,17 @@ server:
## 鸣谢 🙏
- 本项目受 [百聆语音对话机器人](https://github.com/wwbin2017/bailing) 启发，并在其基础上实现。
- 感谢 [十方融海](https://www.tenclass.com/) 对小智通讯协议提供的详尽文档支持。
| Logo | 项目/公司 | 说明 |
|:---:|:---:|:---|
| <img src="./docs/images/logo_bailing.png" width="160"> | [百聆语音对话机器人](https://github.com/wwbin2017/bailing) | 本项目受[百聆语音对话机器人](https://github.com/wwbin2017/bailing)启发，并在其基础上实现 |
| <img src="./docs/images/logo_tenclass.png" width="160"> | [十方融海](https://www.tenclass.com/) | 感谢[十方融海](https://www.tenclass.com/)为小智生态制定了标准的通讯协议、多设备兼容性方案及高并发场景实践示范；为本项目提供了全链路技术文档支持 |
| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [玄凤科技](https://github.com/Eric0308) | 感谢[玄凤科技](https://github.com/Eric0308)贡献函数调用框架、MCP通信协议及插件化调用机制的实现代码，通过标准化的指令调度体系与动态扩展能力，显著提升了前端设备(IoT)的交互效率和功能延展性 |
| <img src="./docs/images/logo_huiyuan.png" width="160"> | [汇远设计](http://ui.kwd988.net/) | 感谢[汇远设计](http://ui.kwd988.net/)为本项目提供专业视觉解决方案，用其服务超千家企业的设计实战经验，赋能本项目产品用户体验 |
| <img src="./docs/images/logo_qinren.png" width="160"> | [西安勤人信息科技](https://www.029app.com/) | 感谢[西安勤人信息科技](https://www.029app.com/)深化本项目视觉体系，确保整体设计风格在多场景应用中的一致性和扩展性 |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
docs/images/logo_bailing.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

docs/images/logo_huiyuan.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

docs/images/logo_qinren.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

docs/images/logo_tenclass.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.1 KiB

docs/images/logo_xuanfeng.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

main/manager-api/src/main/resources/db/changelog/202504131543.sql
+18
View File
@@ -0,0 +1,18 @@
-- 对0.3.0版本之前的参数进行修改
update `sys_params` set param_value = '.mp3;.wav;.p3' where param_code = 'plugins.play_music.music_ext';
update `ai_model_config` set config_json = '{\"type\": \"intent_llm\", \"llm\": \"LLM_ChatGLMLLM\"}' where id = 'Intent_intent_llm';
-- 添加edge音色
delete from `ai_tts_voice` where tts_model_id = 'TTS_EdgeTTS';
INSERT INTO `ai_tts_voice` VALUES
('TTS_EdgeTTS0001', 'TTS_EdgeTTS', 'EdgeTTS女声-晓晓', 'zh-CN-XiaoxiaoNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0002', 'TTS_EdgeTTS', 'EdgeTTS男声-云扬', 'zh-CN-YunyangNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0003', 'TTS_EdgeTTS', 'EdgeTTS女声-晓伊', 'zh-CN-XiaoyiNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0004', 'TTS_EdgeTTS', 'EdgeTTS男声-云健', 'zh-CN-YunjianNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0005', 'TTS_EdgeTTS', 'EdgeTTS男声-云希', 'zh-CN-YunxiNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0006', 'TTS_EdgeTTS', 'EdgeTTS男声-云夏', 'zh-CN-YunxiaNeural', '普通话', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0007', 'TTS_EdgeTTS', 'EdgeTTS女声-辽宁小贝', 'zh-CN-liaoning-XiaobeiNeural', '辽宁', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0008', 'TTS_EdgeTTS', 'EdgeTTS女声-陕西小妮', 'zh-CN-shaanxi-XiaoniNeural', '陕西', NULL, NULL, 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0009', 'TTS_EdgeTTS', 'EdgeTTS女声-香港海佳', 'zh-HK-HiuGaaiNeural', '粤语', 'General', 'Friendly, Positive', 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0010', 'TTS_EdgeTTS', 'EdgeTTS女声-香港海曼', 'zh-HK-HiuMaanNeural', '粤语', 'General', 'Friendly, Positive', 1, NULL, NULL, NULL, NULL),
('TTS_EdgeTTS0011', 'TTS_EdgeTTS', 'EdgeTTS男声-香港万龙', 'zh-HK-WanLungNeural', '粤语', 'General', 'Friendly, Positive', 1, NULL, NULL, NULL, NULL);
main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml
+2 -2
View File
@@ -52,9 +52,9 @@ databaseChangeLog:
encoding: utf8
path: classpath:db/changelog/202504112058.sql
- changeSet:
id: 202504131542
id: 202504131543
author: John
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202504131542.sql
path: classpath:db/changelog/202504131543.sql
main/xiaozhi-server/core/providers/tts/cozecn.py
+12 -4
View File
@@ -12,14 +12,20 @@ class TTSProvider(TTSProviderBase):
super().__init__(config, delete_audio_file)
self.model = config.get("model")
self.access_token = config.get("access_token")
self.voice = config.get("voice")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice")
self.response_format = config.get("response_format")
self.host = "api.coze.cn"
self.api_url = f"https://{self.host}/v1/audio/speech"
def generate_filename(self, extension=".wav"):
return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
request_json = {
@@ -30,9 +36,11 @@ class TTSProvider(TTSProviderBase):
}
headers = {
"Authorization": f"Bearer {self.access_token}",
"Content-Type": "application/json"
"Content-Type": "application/json",
}
response = requests.request("POST", self.api_url, json=request_json, headers=headers)
response = requests.request(
"POST", self.api_url, json=request_json, headers=headers
)
data = response.content
file_to_save = open(output_file, "wb")
file_to_save.write(data)
main/xiaozhi-server/core/providers/tts/doubao.py
+6 -1
View File
@@ -18,7 +18,12 @@ class TTSProvider(TTSProviderBase):
self.appid = config.get("appid")
self.access_token = config.get("access_token")
self.cluster = config.get("cluster")
self.voice = config.get("voice")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice")
self.api_url = config.get("api_url")
self.authorization = config.get("authorization")
self.header = {"Authorization": f"{self.authorization}{self.access_token}"}
main/xiaozhi-server/core/providers/tts/edge.py
+11 -5
View File
@@ -8,20 +8,26 @@ from core.providers.tts.base import TTSProviderBase
class TTSProvider(TTSProviderBase):
def __init__(self, config, delete_audio_file):
super().__init__(config, delete_audio_file)
self.voice = config.get("voice")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice")
def generate_filename(self, extension=".mp3"):
return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
communicate = edge_tts.Communicate(text, voice=self.voice)
# 确保目录存在并创建空文件
os.makedirs(os.path.dirname(output_file), exist_ok=True)
with open(output_file, 'wb') as f:
with open(output_file, "wb") as f:
pass
# 流式写入音频数据
with open(output_file, 'ab') as f: # 改为追加模式避免覆盖
with open(output_file, "ab") as f: # 改为追加模式避免覆盖
async for chunk in communicate.stream():
if chunk["type"] == "audio": # 只处理音频数据块
f.write(chunk["data"])
main/xiaozhi-server/core/providers/tts/minimax.py
+28 -16
View File
@@ -12,28 +12,33 @@ class TTSProvider(TTSProviderBase):
self.group_id = config.get("group_id")
self.api_key = config.get("api_key")
self.model = config.get("model")
self.voice_id = config.get("voice_id")
if config.get("private_voice"):
self.voice_id = config.get("private_voice")
else:
self.voice_id = config.get("voice_id")
default_voice_setting = {
"voice_id": "female-shaonv",
"speed": 1,
"vol": 1,
"pitch": 0,
"emotion": "happy"
}
default_pronunciation_dict = {
"tone": [
"处理/(chu3)(li3)", "危险/dangerous"
]
"emotion": "happy",
}
default_pronunciation_dict = {"tone": ["处理/(chu3)(li3)", "危险/dangerous"]}
defult_audio_setting = {
"sample_rate": 32000,
"bitrate": 128000,
"format": "mp3",
"channel": 1
"channel": 1,
}
self.voice_setting = {
**default_voice_setting,
**config.get("voice_setting", {}),
}
self.pronunciation_dict = {
**default_pronunciation_dict,
**config.get("pronunciation_dict", {}),
}
self.voice_setting = {**default_voice_setting, **config.get("voice_setting", {})}
self.pronunciation_dict = {**default_pronunciation_dict, **config.get("pronunciation_dict", {})}
self.audio_setting = {**defult_audio_setting, **config.get("audio_setting", {})}
self.timber_weights = config.get("timber_weights", [])
@@ -44,11 +49,14 @@ class TTSProvider(TTSProviderBase):
self.api_url = f"https://{self.host}/v1/t2a_v2?GroupId={self.group_id}"
self.header = {
"Content-Type": "application/json",
"Authorization": f"Bearer {self.api_key}"
"Authorization": f"Bearer {self.api_key}",
}
def generate_filename(self, extension=".mp3"):
return os.path.join(self.output_file, f"tts-{__name__}{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{__name__}{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
request_json = {
@@ -65,13 +73,17 @@ class TTSProvider(TTSProviderBase):
request_json["voice_setting"]["voice_id"] = ""
try:
resp = requests.post(self.api_url, json.dumps(request_json), headers=self.header)
resp = requests.post(
self.api_url, json.dumps(request_json), headers=self.header
)
# 检查返回请求数据的status_code是否为0
if resp.json()["base_resp"]["status_code"] == 0:
data = resp.json()['data']['audio']
data = resp.json()["data"]["audio"]
file_to_save = open(output_file, "wb")
file_to_save.write(bytes.fromhex(data))
else:
raise Exception(f"{__name__} status_code: {resp.status_code} response: {resp.content}")
raise Exception(
f"{__name__} status_code: {resp.status_code} response: {resp.content}"
)
except Exception as e:
raise Exception(f"{__name__} error: {e}")
raise Exception(f"{__name__} error: {e}")
main/xiaozhi-server/core/providers/tts/openai.py
+4 -1
View File
@@ -16,7 +16,10 @@ class TTSProvider(TTSProviderBase):
self.api_key = config.get("api_key")
self.api_url = config.get("api_url", "https://api.openai.com/v1/audio/speech")
self.model = config.get("model", "tts-1")
self.voice = config.get("voice", "alloy")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice", "alloy")
self.response_format = "wav"
self.speed = config.get("speed", 1.0)
self.output_file = config.get("output_dir", "tmp/")
main/xiaozhi-server/core/providers/tts/siliconflow.py
+12 -4
View File
@@ -10,7 +10,10 @@ class TTSProvider(TTSProviderBase):
super().__init__(config, delete_audio_file)
self.model = config.get("model")
self.access_token = config.get("access_token")
self.voice = config.get("voice")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice")
self.response_format = config.get("response_format")
self.sample_rate = config.get("sample_rate")
self.speed = config.get("speed")
@@ -20,7 +23,10 @@ class TTSProvider(TTSProviderBase):
self.api_url = f"https://{self.host}/v1/audio/speech"
def generate_filename(self, extension=".wav"):
return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
request_json = {
@@ -31,9 +37,11 @@ class TTSProvider(TTSProviderBase):
}
headers = {
"Authorization": f"Bearer {self.access_token}",
"Content-Type": "application/json"
"Content-Type": "application/json",
}
response = requests.request("POST", self.api_url, json=request_json, headers=headers)
response = requests.request(
"POST", self.api_url, json=request_json, headers=headers
)
data = response.content
file_to_save = open(output_file, "wb")
file_to_save.write(data)
main/xiaozhi-server/core/providers/tts/tencent.py
+52 -37
View File
@@ -16,7 +16,10 @@ class TTSProvider(TTSProviderBase):
self.appid = config.get("appid")
self.secret_id = config.get("secret_id")
self.secret_key = config.get("secret_key")
self.voice = config.get("voice")
if config.get("private_voice"):
self.voice = config.get("private_voice")
else:
self.voice = config.get("voice")
self.api_url = "https://tts.tencentcloudapi.com" # 正确的API端点
self.region = config.get("region")
self.output_file = config.get("output_dir")
@@ -25,35 +28,36 @@ class TTSProvider(TTSProviderBase):
"""生成鉴权请求头"""
# 获取当前UTC时间戳
timestamp = int(time.time())
# 使用UTC时间计算日期
utc_date = datetime.fromtimestamp(timestamp, tz=timezone.utc).strftime('%Y-%m-%d')
utc_date = datetime.fromtimestamp(timestamp, tz=timezone.utc).strftime(
"%Y-%m-%d"
)
# 服务名称必须是 "tts"
service = "tts"
# 拼接凭证范围
credential_scope = f"{utc_date}/{service}/tc3_request"
# 使用TC3-HMAC-SHA256签名方法
algorithm = "TC3-HMAC-SHA256"
# 构建规范请求字符串
http_request_method = "POST"
canonical_uri = "/"
canonical_querystring = ""
# 请求头必须包含host和content-type，且按字典序排列
canonical_headers = (
f"content-type:application/json\n"
f"host:tts.tencentcloudapi.com\n"
f"content-type:application/json\n" f"host:tts.tencentcloudapi.com\n"
)
signed_headers = "content-type;host"
# 请求体哈希值
payload = json.dumps(request_body)
payload_hash = hashlib.sha256(payload.encode('utf-8')).hexdigest()
payload_hash = hashlib.sha256(payload.encode("utf-8")).hexdigest()
# 构建规范请求字符串
canonical_request = (
f"{http_request_method}\n"
@@ -63,10 +67,12 @@ class TTSProvider(TTSProviderBase):
f"{signed_headers}\n"
f"{payload_hash}"
)
# 计算规范请求的哈希值
hashed_canonical_request = hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
hashed_canonical_request = hashlib.sha256(
canonical_request.encode("utf-8")
).hexdigest()
# 构建待签名字符串
string_to_sign = (
f"{algorithm}\n"
@@ -74,19 +80,19 @@ class TTSProvider(TTSProviderBase):
f"{credential_scope}\n"
f"{hashed_canonical_request}"
)
# 计算签名密钥
secret_date = self._hmac_sha256(f"TC3{self.secret_key}".encode('utf-8'), utc_date)
secret_date = self._hmac_sha256(
f"TC3{self.secret_key}".encode("utf-8"), utc_date
)
secret_service = self._hmac_sha256(secret_date, service)
secret_signing = self._hmac_sha256(secret_service, "tc3_request")
# 计算签名
signature = hmac.new(
secret_signing,
string_to_sign.encode('utf-8'),
hashlib.sha256
secret_signing, string_to_sign.encode("utf-8"), hashlib.sha256
).hexdigest()
# 构建授权头
authorization = (
f"{algorithm} "
@@ -94,7 +100,7 @@ class TTSProvider(TTSProviderBase):
f"SignedHeaders={signed_headers}, "
f"Signature={signature}"
)
# 构建请求头
headers = {
"Content-Type": "application/json",
@@ -104,19 +110,22 @@ class TTSProvider(TTSProviderBase):
"X-TC-Timestamp": str(timestamp),
"X-TC-Version": "2019-08-23",
"X-TC-Region": self.region,
"X-TC-Language": "zh-CN"
"X-TC-Language": "zh-CN",
}
return headers
def _hmac_sha256(self, key, msg):
"""HMAC-SHA256加密"""
if isinstance(msg, str):
msg = msg.encode('utf-8')
msg = msg.encode("utf-8")
return hmac.new(key, msg, hashlib.sha256).digest()
def generate_filename(self, extension=".wav"):
return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
# 构建请求体
@@ -129,19 +138,23 @@ class TTSProvider(TTSProviderBase):
try:
# 获取请求头（每次请求都重新生成，以确保时间戳和签名是最新的）
headers = self._get_auth_headers(request_json)
# 发送请求
resp = requests.post(self.api_url, json.dumps(request_json), headers=headers)
resp = requests.post(
self.api_url, json.dumps(request_json), headers=headers
)
# 检查响应
if resp.status_code == 200:
response_data = resp.json()
# 检查是否成功
if response_data.get("Response", {}).get("Error") is not None:
error_info = response_data["Response"]["Error"]
raise Exception(f"API返回错误: {error_info['Code']}: {error_info['Message']}")
raise Exception(
f"API返回错误: {error_info['Code']}: {error_info['Message']}"
)
# 提取音频数据
audio_data = response_data["Response"].get("Audio")
if audio_data:
@@ -151,6 +164,8 @@ class TTSProvider(TTSProviderBase):
else:
raise Exception(f"{__name__}: 没有返回音频数据: {response_data}")
else:
raise Exception(f"{__name__} status_code: {resp.status_code} response: {resp.content}")
raise Exception(
f"{__name__} status_code: {resp.status_code} response: {resp.content}"
)
except Exception as e:
raise Exception(f"{__name__} error: {e}")
raise Exception(f"{__name__} error: {e}")
main/xiaozhi-server/core/providers/tts/ttson.py
+36 -22
View File
@@ -10,8 +10,14 @@ from core.providers.tts.base import TTSProviderBase
class TTSProvider(TTSProviderBase):
def __init__(self, config, delete_audio_file):
super().__init__(config, delete_audio_file)
self.url = config.get("url", "https://u95167-bd74-2aef8085.westx.seetacloud.com:8443/flashsummary/tts?token=")
self.voice_id = config.get("voice_id", 1695)
self.url = config.get(
"url",
"https://u95167-bd74-2aef8085.westx.seetacloud.com:8443/flashsummary/tts?token=",
)
if config.get("private_voice"):
self.voice_id = int(config.get("private_voice"))
else:
self.voice_id = int(config.get("voice_id", 1695))
self.token = config.get("token")
self.to_lang = config.get("to_lang")
self.volume_change_dB = config.get("volume_change_dB", 0)
@@ -21,37 +27,45 @@ class TTSProvider(TTSProviderBase):
self.pitch_factor = config.get("pitch_factor", 0)
self.format = config.get("format", "mp3")
self.emotion = config.get("emotion", 1)
self.header = {
"Content-Type": "application/json"
}
self.header = {"Content-Type": "application/json"}
def generate_filename(self, extension=".mp3"):
return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}")
return os.path.join(
self.output_file,
f"tts-{datetime.now().date()}@{uuid.uuid4().hex}{extension}",
)
async def text_to_speak(self, text, output_file):
url = f'{self.url}{self.token}'
url = f"{self.url}{self.token}"
result = "firefly"
payload = json.dumps({
"to_lang": self.to_lang,
"text": text,
"emotion": self.emotion,
"format": self.format,
"volume_change_dB": self.volume_change_dB,
"voice_id": self.voice_id,
"pitch_factor": self.pitch_factor,
"speed_factor": self.speed_factor,
"token": self.token
})
payload = json.dumps(
{
"to_lang": self.to_lang,
"text": text,
"emotion": self.emotion,
"format": self.format,
"volume_change_dB": self.volume_change_dB,
"voice_id": self.voice_id,
"pitch_factor": self.pitch_factor,
"speed_factor": self.speed_factor,
"token": self.token,
}
)
resp = requests.request("POST", url, data=payload)
if resp.status_code != 200:
return None
resp_json = resp.json()
try:
result = resp_json['url'] + ':' + str(
resp_json[
'port']) + '/flashsummary/retrieveFileData?stream=True&token=' + self.token + '&voice_audio_path=' + \
resp_json['voice_path']
result = (
resp_json["url"]
+ ":"
+ str(resp_json["port"])
+ "/flashsummary/retrieveFileData?stream=True&token="
+ self.token
+ "&voice_audio_path="
+ resp_json["voice_path"]
)
except Exception as e:
print("error:", e)