diff --git a/README.md b/README.md index b1bf6dae..4b3215f6 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@

-English +English · 常见问题 · 反馈问题 · 部署文档 diff --git a/README_en.md b/README_en.md new file mode 100644 index 00000000..9b4ed8ce --- /dev/null +++ b/README_en.md @@ -0,0 +1,273 @@ +[![Banners](docs/images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server) + +

Xiaozhi Backend Service xiaozhi-esp32-server

+ +

+This project provides backend services for the open-source smart hardware project +xiaozhi-esp32
+Implemented using Python, Java, and Vue according to the Xiaozhi Communication Protocol
+Helping you quickly set up your Xiaozhi server +

+ +

+中文 +· FAQ +· Report Issues +· Deployment Guide +· Release Notes +

+

+ + GitHub Contributors + + + GitHub Contributors + + + Issues + + + GitHub pull requests + + + GitHub pull requests + + + stars + +

+ +--- + +## Target Users 👥 + +This project requires ESP32 hardware devices. If you have purchased ESP32-related hardware, successfully connected to Brother Xia's backend service, and want to set up your own `xiaozhi-esp32` backend service, then this project is perfect for you. + +Want to see it in action? Check out these videos 🎥 + + + + + + + + + + + + + + + + +
+ + + Xiaozhi esp32 connecting to custom backend model + + + + + + Custom voice + + + + + + Cantonese communication + + + + + + Home appliance control + + + + + + Lowest cost configuration + + +
+ + + Custom voice + + + + + + Music playback + + + + + + Weather plugin + + + + + + IOT device control + + + + + + News broadcast + + +
+ +--- + +## Warning ⚠️ + +1. This project is open-source software. This software has no commercial relationship with any third-party API service providers (including but not limited to speech recognition, large models, speech synthesis, and other platforms) and does not provide any form of guarantee for their service quality or financial security. +It is recommended that users prioritize service providers with relevant business licenses and carefully read their service agreements and privacy policies. This software does not host any account keys, does not participate in fund transfers, and does not bear the risk of recharge fund losses. + +2. This project's functionality is not complete and has not passed network security testing. Please do not use it in production environments. If you deploy this project for learning in a public network environment, please ensure necessary protection measures are in place. + +--- + +## Deployment Documentation + +![Banners](docs/images/banner2.png) + +This project offers two deployment methods. Please choose based on your specific needs: + +#### 🚀 Deployment Method Selection + +| Deployment Method | Features | Use Case | Docker Deployment Guide | Source Code Deployment Guide | +|---------|------|---------|---------|---------| +| **Simplified Installation** | Smart dialogue, IOT functionality, data stored in configuration files | Low-configuration environment, no database required | [Docker Server Only](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) | [Local Source Code Server Only](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| +| **Full Module Installation** | Smart dialogue, IOT, OTA, Control Panel, data stored in database | Complete functionality experience |[Docker Full Module](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) | [Local Source Code Full Module](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) | + +> 💡 Note: Below is the test platform deployed with the latest code. You can flash and test if needed. Concurrent users: 6, data cleared daily + +``` +Control Panel: https://2662r3426b.vicp.fun + +Service Test Tool: https://2662r3426b.vicp.fun/test/ +OTA Interface: https://2662r3426b.vicp.fun/xiaozhi/ota/ +Websocket Interface: wss://2662r3426b.vicp.fun/xiaozhi/v1/ +``` + +--- +## Feature List ✨ + +### Implemented ✅ + +| Feature Module | Description | +|---------|------| +| Communication Protocol | Based on `xiaozhi-esp32` protocol, implements data interaction through WebSocket | +| Dialogue Interaction | Supports wake-up dialogue, manual dialogue, and real-time interruption. Auto-sleep after long periods of inactivity | +| Intent Recognition | Supports LLM intent recognition, function call, reducing hard-coded intent judgment | +| Multi-language Recognition | Supports Mandarin, Cantonese, English, Japanese, Korean (default using FunASR) | +| LLM Module | Supports flexible LLM module switching, default using ChatGLMLLM, also supports Ali Bailing, DeepSeek, Ollama, etc. | +| TTS Module | Supports EdgeTTS (default), Volcano Engine Doubao TTS, and other TTS interfaces for speech synthesis | +| Memory Function | Supports ultra-long memory, local summary memory, and no memory modes for different scenarios | +| IOT Function | Supports managing registered device IOT functionality, intelligent IoT control based on dialogue context | +| Control Panel | Provides web management interface, supports agent management, user management, system configuration, etc. | + +### In Development 🚧 + +To learn about specific development progress, [click here](https://github.com/users/xinnan-tech/projects/3) + +If you're a software developer, here's an [Open Letter to Developers](docs/contributor_open_letter.md). Welcome to join! + +--- + +## Product Ecosystem 👬 +Xiaozhi is an ecosystem. When using this product, you might want to check out other excellent projects in this ecosystem: + +| Project Name | Project Link | Description | +|:---------------------|:--------|:--------| +| Xiaozhi Android Client | [xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) | A Flutter-based Android and iOS voice dialogue application supporting real-time voice interaction and text dialogue | +| Xiaozhi PC Client | [py-xiaozhi](https://github.com/Huang-junsen/py-xiaozhi) | A Python-based AI client that allows you to experience Xiaozhi AI functionality through code without physical hardware | +| Xiaozhi Java Server | [xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) | A Java-based open-source project providing complete backend service solutions | + +--- + +## Supported Platforms/Components 📋 + +### LLM Language Models + +| Usage Method | Supported Platforms | Free Platforms | +|:---:|:---:|:---:| +| openai API | Ali Bailing, Volcano Engine Doubao, DeepSeek, ChatGLM, Gemini | ChatGLM, Gemini | +| ollama API | Ollama | - | +| dify API | Dify | - | +| fastgpt API | Fastgpt | - | +| coze API | Coze | - | + +Actually, any LLM supporting openai API calls can be integrated. + +--- + +### TTS Speech Synthesis + +| Usage Method | Supported Platforms | Free Platforms | +|:---:|:---:|:---:| +| API Calls | EdgeTTS, Volcano Engine Doubao TTS, Tencent Cloud, Aliyun TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS | EdgeTTS, CosyVoiceSiliconflow(partial) | +| Local Service | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, MinimaxTTS | + +--- + +### VAD Voice Activity Detection + +| Type | Platform Name | Usage Method | Pricing | Notes | +|:---:|:---------:|:----:|:----:|:--:| +| VAD | SileroVAD | Local Use | Free | | + +--- + +### ASR Speech Recognition + +| Usage Method | Supported Platforms | Free Platforms | +|:---:|:---:|:---:| +| Local Use | FunASR, SherpaASR | FunASR, SherpaASR | +| API Calls | DoubaoASR, FunASRServer, TencentASR, AliyunASR | FunASRServer | + +--- + +### Memory Storage + +| Type | Platform Name | Usage Method | Pricing | Notes | +|:------:|:---------------:|:----:|:---------:|:--:| +| Memory | mem0ai | API Calls | 1000 calls/month quota | | +| Memory | mem_local_short | Local Summary | Free | | + +--- + +### Intent Recognition + +| Type | Platform Name | Usage Method | Pricing | Notes | +|:------:|:-------------:|:----:|:-------:|:---------------------:| +| Intent | intent_llm | API Calls | Based on LLM pricing | Uses large model for intent recognition, highly versatile | +| Intent | function_call | API Calls | Based on LLM pricing | Uses large model function calls for intent, fast and effective | + +--- + +## Acknowledgments 🙏 + +| Logo | Project/Company | Description | +|:---:|:---:|:---| +| | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | This project was inspired by [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) and implemented based on it | +| | [Tenclass](https://www.tenclass.com/) | Thanks to [Tenclass](https://www.tenclass.com/) for developing standard communication protocols, multi-device compatibility solutions, and high-concurrency scenario practices for the Xiaozhi ecosystem; providing comprehensive technical documentation support for this project | +| | [Xuanfeng Technology](https://github.com/Eric0308) | Thanks to [Xuanfeng Technology](https://github.com/Eric0308) for contributing function call framework, MCP communication protocol, and plugin call mechanism implementation code, significantly improving front-end device (IoT) interaction efficiency and functional extensibility through standardized instruction scheduling system and dynamic expansion capabilities | +| | [Huiyuan Design](http://ui.kwd988.net/) | Thanks to [Huiyuan Design](http://ui.kwd988.net/) for providing professional visual solutions for this project, empowering product user experience with their design experience serving over a thousand enterprises | +| | [Xi'an Qinren Information Technology](https://www.029app.com/) | Thanks to [Xi'an Qinren Information Technology](https://www.029app.com/) for deepening this project's visual system, ensuring consistency and extensibility of overall design style in multi-scenario applications | + + + + + + + + Star History Chart + + \ No newline at end of file diff --git a/docs/homeassistant-integration.md b/docs/homeassistant-integration.md index d2dc4f9d..3142244e 100644 --- a/docs/homeassistant-integration.md +++ b/docs/homeassistant-integration.md @@ -1,4 +1,8 @@ -# ESP32设备与HomeAssistant集成指南 +# 小智ESP32-开源服务端与HomeAssistant集成指南 + +[TOC] + +----- ## 简介 @@ -7,26 +11,58 @@ ## 前提条件 - 已安装并配置好`HomeAssistant` -- 已经在HomeAssistant中集成`Xiaomi Home`，并将米家的设备导入进`HomeAssistant` -- `xiaozhi-esp32-server智控台`能正常使用 - 本次我选择的模型是：免费的ChatGLM，它支持functioncall函数调用 -## 网络环境 +## 开始前的操作（必要） + +### 1. 获取HA的网络网络地址信息 + +请访问你Home Assistant的网络地址，例如，我的HA的地址是192.168.4.7，端口则是默认的8123，则在浏览器打开 -1、我的`HomeAssistant`部署在下面，版本是`2025.3.4` ``` http://192.168.4.7:8123 ``` -2、我的`xiaozhi-esp32-server智控台`和`HomeAssistant`部署在同一台机器的另一个端口，版本是`0.3.10` +> 手动查询 HA 的 IP 地址方法**（仅限小智esp32-server和HA部署在同一个网络设备[例如同一个wifi]下）**： +> +> 1. 进入 Home Assistant（前端）。 +> +> 2. 点击左下角 **设置（Settings）** → **系统（System）** → **网络（Network）**。 +> +> 3. 滑到最底部`Home Assistant 网址(Home Assistant website)`区域，在`本地网络(local network)`中，点击`眼睛`按钮，可以看到当前使用的 IP 地址（如 `192.168.1.10`）和网络接口。点击`复制连接(copy link)`可以直接复制。 +> +> ![image-20250504051716417](images/image-ha-integration-01.png) + +或，您已经设置了直接可以访问的Home Assistant的OAuth地址，您也可以在浏览器内直接访问 + ``` -http://192.168.4.7:8002 +http://homeassistant.local:8123 ``` +### 2. 登录`Home Assistant`拿到开发密钥 -## 配置步骤 +登录`HomeAssistant`，点击`左下角头像 -> 个人`，切换`安全`导航栏，划到底部`长期访问令牌`生成api_key，并复制保存，后续的方法都需要使用这个api key且仅出现一次（小tips: 您可以保存生成的二维码图像，后续可以扫描二维码再此提取api key）。 -### 1. 登录`HomeAssistant`整理需要控制的设备清单 +## 方法1：小智社区共建的HA调用功能 + +### 功能描述 + +- 如您后续需要增加新的设备，该方法需要手动重启`xiaozhi-esp32-server服务端`以此更新设备信息**（重要**）。 + +- 需要您确保已经在HomeAssistant中集成`Xiaomi Home`，并将米家的设备导入进`HomeAssistant`。 + +- 需要您确保`xiaozhi-esp32-server智控台`能正常使用。 + +- 我的`xiaozhi-esp32-server智控台`和`HomeAssistant`部署在同一台机器的另一个端口，版本是`0.3.10` + + ``` + http://192.168.4.7:8002 + ``` + + +### 配置步骤 + +#### 1. 登录`HomeAssistant`整理需要控制的设备清单 登录`HomeAssistant`，点击`左下角的设置`，然后进入`设备与服务`，再点击顶部的`实体`。 @@ -56,12 +92,7 @@ http://192.168.4.7:8002 这段字符，我们成为“设备清单字符”需要保存好，等一下有用。 -### 2. 登录`HomeAssistant`拿到开发密钥 - -登录`HomeAssistant`，点击`左下角个人`，切换`安全`导航栏，划到底部`长期访问令牌`生成api_key。 - - -### 3. 登录`智控台` +#### 2. 登录`智控台` 使用管理员账号，登录`智控台`。点击顶部菜单`参数管理`，搜索`plugins.home_assistant.`，会有三条结果出来 @@ -77,7 +108,7 @@ http://192.168.4.7:8123 编辑`plugins.home_assistant.api_key`，把你从`HomeAssistant`复制过来的密钥，粘贴进去 -### 4. 设置`意图识别`函数 +#### 3. 设置`意图识别`函数 在智控台，点击顶部菜单“模型配置”，在左侧栏，找到“意图识别”，找到id为`Intent_function_call`的意图，点击编辑 @@ -94,18 +125,124 @@ change_role;get_weather;get_news;play_music change_role;get_weather;get_news;play_music;hass_get_state;hass_set_state ``` -### 6. 手动重启xiaozhi-server +#### 4. 手动重启xiaozhi-server 重启xiaozhi-server程序 -### 5. 确认角色配置是否设置了函数意图识别 +#### 5. 确认角色配置是否设置了函数意图识别 在智控台，点击顶部菜单“智能体管理”，找到设备所在的智能体，点击“配置角色” 确认意图识别(Intent)，是否选择“函数调用意图识别” -### 6. 唤醒设别进行控制 +#### 6. 唤醒设别进行控制 -尝试和esp32说，“打开XXX灯” \ No newline at end of file +尝试和esp32说，“打开XXX灯” + +## 方法2：小智将Home Assistant的语音助手作为LLM工具 + +### 功能描述 + +- 该方法有一个比较严重的缺点——**该方法无法使用小智开源生态的function_call插件功能的能力**，因为使用Home Assistant作为小智的LLM工具会将意图识别能力转让给Home Assistant。但是**这个方法是能体验到原生的Home Assistant操作功能，且小智的聊天能力不变**。如实在介意可以使用同样是Home Assistant支持的[方法3](##方法3：使用Home Assistant的MCP服务（推荐）)，能够最大程度体验到Home Assistant的功能。 + +### 配置步骤： + +#### 1. 配置Home Assistant的大模型语音助手。 + +**需要您提前配置好Home Assistant的语音助手或大模型工具。** + +#### 2. 获取Home Assistant的语言助手的Agent ID. + +1. 进入Home Assistant页面内。左侧点击`开发者助手`。 +2. 在打开的`开发者助手`内，点击`动作`选项卡（如图示操作1），在页面内的选项栏`动作`中，找到或输入`conversation.process（对话-处理）`并选择`对话（conversation）: 处理`（如图示操作2）。 + +![image-20250504043539343](images/image-ha-integration-02.png) + +3. 在页面内勾选`代理(agent)`选项，在变成常亮的`对话代理(conversation agent)`内选择您步骤一配置好的语音助手名称，如图示，我这边配置好的是`ZhipuAi`并选择。 + +![image-20250504043854760](images/image-ha-integration-03.png) + +4. 选中后，点击表单左下方的`进入YAML模式`。 + +![image-20250504043951126](images/image-ha-integration-04.png) + +5. 复制其中的agent-id的值，例如图示中我的是`01JP2DYMBDF7F4ZA2DMCF2AGX2`(仅供参考)。 + +![image-20250504044046466](images/image-ha-integration-05.png) + +6. 切换到小智开源服务端`xiaozhi-esp32-server`的`config.yaml`文件内，在LLM配置中，找到Home Assistant，设置您的Home Assistant的网络地址，Api key和刚刚查询到的agent_id。 +7. 修改`config.yaml`文件内的`selected_module`属性的`LLM`为`HomeAssistant`，`Intent`为`nointent`。 +8. 重启小智开源服务端`xiaozhi-esp32-server`即可正常使用。 + +## 方法3：使用Home Assistant的MCP服务（推荐） + +### 功能描述 + +- 需要您提前在Home Assistant内集成并安装好HA集成——[Model Context Protocol Server](https://www.home-assistant.io/integrations/mcp_server/)。 + +- 这个方法与方法2都是HA官方提供的解决方法，与方法2不同的是，您可以正常使用小智开源服务端`xiaozhi-esp32-server`的开源共建的插件，同时允许您随意使用任何一个支持function_call功能的LLM大模型。 + +### 配置步骤 + +#### 1. 安装Home Assistant的MCP服务集成。 + +集成官方网址——[Model Context Protocol Server](https://www.home-assistant.io/integrations/mcp_server/)。。 + +或跟随以下手动操作。 + +> - 前往Home Assistant页面的**[设置 > 设备和服务（Settings > Devices & Services.）](https://my.home-assistant.io/redirect/integrations)**。 +> +> - 在右下角，选择 **[添加集成（Add Integration）](https://my.home-assistant.io/redirect/config_flow_start?domain=mcp_server)**按钮。 +> +> - 从列表中选择**模型上下文协议服务器（Model Context Protocol Server）**。 +> +> - 按照屏幕上的说明完成设置。 + +#### 2. 配置小智开源服务端MCP配置信息 + +切换到小智开源服务端`xiaozhi-esp32-server`的`mcp_server_settings.json`文件内，在`"mcpServers"`的括号内添加以下内容： + +```json +"Home Assistant": { + "command": "mcp-proxy", + "args": [ + "http://YOUR_HA_HOST/mcp_server/sse" + ], + "env": { + "API_ACCESS_TOKEN": "YOUR_API_ACCESS_TOKEN" + } +}, +``` + +注意： + +1. **替换配置：** + - 替换`args`内的`YOUR_HA_HOST`为您的HA服务地址，如果你的服务地址已经包含了https/http字样（例如`http://192.168.1.101:8123`)，则只需要填入`192.168.1.101:8123`即可。 + - 将`env`内`API_ACCESS_TOKEN`的`YOUR_API_ACCESS_TOKEN`替换成您之前获取到的开发密钥api key。 +2. **如果你添加配置是在`"mcpServers"`的括号内后续没有新的`mcpServers`的配置时，需要把最后的逗号`,`移除**，否则可能会解析失败。 + +**最后效果参考以下（参考如下）**： + +```json + "mcpServers": { + "Home Assistant": { + "command": "mcp-proxy", + "args": [ + "http://192.168.1.101:8123/mcp_server/sse" + ], + "env": { + "API_ACCESS_TOKEN": "abcd.efghi.jkl" + } + } + } +``` + +#### 3. 配置小智开源服务端的系统配置 + +1. **选择任意一款支持function_call的LLM大模型作为小智的LLM聊天助手（但不要选择Home Assistant作为LLM工具）**，本次我选择的模型是：免费的ChatGLM，它支持functioncall函数调用，但部分时候调用不太稳定，如果像追求稳定建议把LLM设置成：DoubaoLLM，使用的具体model_name是：doubao-1-5-pro-32k-250115。 + +2. 切换到小智开源服务端`xiaozhi-esp32-server`的`config.yaml`文件内，设置您的LLM大模型配置，并且将`selected_module`配置的`Intent`调整为`function_call`。 + +3. 重启小智开源服务端`xiaozhi-esp32-server`即可正常使用。 \ No newline at end of file diff --git a/docs/images/image-ha-integration-01.png b/docs/images/image-ha-integration-01.png new file mode 100644 index 00000000..eb05461a Binary files /dev/null and b/docs/images/image-ha-integration-01.png differ diff --git a/docs/images/image-ha-integration-02.png b/docs/images/image-ha-integration-02.png new file mode 100644 index 00000000..7431defa Binary files /dev/null and b/docs/images/image-ha-integration-02.png differ diff --git a/docs/images/image-ha-integration-03.png b/docs/images/image-ha-integration-03.png new file mode 100644 index 00000000..3f1aa596 Binary files /dev/null and b/docs/images/image-ha-integration-03.png differ diff --git a/docs/images/image-ha-integration-04.png b/docs/images/image-ha-integration-04.png new file mode 100644 index 00000000..29b402e5 Binary files /dev/null and b/docs/images/image-ha-integration-04.png differ diff --git a/docs/images/image-ha-integration-05.png b/docs/images/image-ha-integration-05.png new file mode 100644 index 00000000..1aa8ffc2 Binary files /dev/null and b/docs/images/image-ha-integration-05.png differ diff --git a/main/README.md b/main/README.md index a91c9fdc..4255ab14 100644 --- a/main/README.md +++ b/main/README.md @@ -16,4 +16,4 @@ https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh # manager-web 、manager-api接口协议 -https://2662r3426b.vicp.fun/xiaozhi/v1/doc.html +https://2662r3426b.vicp.fun/xiaozhi/doc.html diff --git a/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java b/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java index 78154b5c..ab6766c7 100644 --- a/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java +++ b/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java @@ -1,5 +1,7 @@ package xiaozhi.common.constant; +import lombok.Getter; + /** * 常量 * Copyright (c) 人人开源 All rights reserved. @@ -174,6 +176,21 @@ public interface Constant { } } + @Getter + enum ChatHistoryConfEnum { + IGNORE(0, "不记录"), + RECORD_TEXT(1, "记录文本"), + RECORD_TEXT_AUDIO(2, "文本音频都记录"); + + private final int code; + private final String name; + + ChatHistoryConfEnum(int code, String name) { + this.code = code; + this.name = name; + } + } + /** * 版本号 */ diff --git a/main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java b/main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java index 9368cf6b..f72a1ba3 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java @@ -109,6 +109,7 @@ public class AgentController { entity.setMemModelId(template.getMemModelId()); entity.setIntentModelId(template.getIntentModelId()); entity.setSystemPrompt(template.getSystemPrompt()); + entity.setChatHistoryConf(template.getChatHistoryConf()); entity.setLangCode(template.getLangCode()); entity.setLanguage(template.getLanguage()); } @@ -166,6 +167,9 @@ public class AgentController { if (dto.getSystemPrompt() != null) { existingEntity.setSystemPrompt(dto.getSystemPrompt()); } + if (dto.getChatHistoryConf() != null) { + existingEntity.setChatHistoryConf(dto.getChatHistoryConf()); + } if (dto.getLangCode() != null) { existingEntity.setLangCode(dto.getLangCode()); } diff --git a/main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentUpdateDTO.java b/main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentUpdateDTO.java index d54685d8..e3d026e3 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentUpdateDTO.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/agent/dto/AgentUpdateDTO.java @@ -45,6 +45,9 @@ public class AgentUpdateDTO implements Serializable { @Schema(description = "角色设定参数", example = "你是一个专业的客服助手，负责回答用户问题并提供帮助", required = false) private String systemPrompt; + @Schema(description = "聊天记录配置（0不记录 1仅记录文本 2记录文本和语音）", example = "3", required = false) + private Integer chatHistoryConf; + @Schema(description = "语言编码", example = "zh_CN", required = false) private String langCode; diff --git a/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentEntity.java b/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentEntity.java index 917fe6d2..82a20489 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentEntity.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentEntity.java @@ -48,6 +48,9 @@ public class AgentEntity { @Schema(description = "意图模型标识") private String intentModelId; + @Schema(description = "聊天记录配置（0不记录 1仅记录文本 2记录文本和语音）") + private Integer chatHistoryConf; + @Schema(description = "角色设定参数") private String systemPrompt; diff --git a/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentTemplateEntity.java b/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentTemplateEntity.java index 6d83e4fe..1604a27b 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentTemplateEntity.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/agent/entity/AgentTemplateEntity.java @@ -69,6 +69,11 @@ public class AgentTemplateEntity implements Serializable { */ private String intentModelId; + /** + * 聊天记录配置（0不记录 1仅记录文本 2记录文本和语音） + */ + private Integer chatHistoryConf; + /** * 角色设定参数 */ diff --git a/main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java b/main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java index 21e645de..bf6c188b 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz/impl/AgentChatHistoryBizServiceImpl.java @@ -2,12 +2,14 @@ package xiaozhi.modules.agent.service.biz.impl; import java.util.Base64; import java.util.Date; +import java.util.Objects; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; +import xiaozhi.common.constant.Constant; import xiaozhi.common.redis.RedisKeys; import xiaozhi.common.redis.RedisUtils; import xiaozhi.modules.agent.dto.AgentChatHistoryReportDTO; @@ -47,8 +49,33 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic Byte chatType = report.getChatType(); log.info("小智设备聊天上报请求: macAddress={}, type={}", macAddress, chatType); - // 1. base64解码report.getOpusDataBase64(),存入ai_agent_chat_audio表 + // 根据设备MAC地址查询对应的默认智能体，判断是否需要上报 + AgentEntity agentEntity = agentService.getDefaultAgentByMacAddress(macAddress); + if (agentEntity == null) { + return Boolean.FALSE; + } + + Integer chatHistoryConf = agentEntity.getChatHistoryConf(); + String agentId = agentEntity.getId(); + + if (Objects.equals(chatHistoryConf, Constant.ChatHistoryConfEnum.RECORD_TEXT.getCode())) { + saveChatText(report, agentId, macAddress, null); + } else if (Objects.equals(chatHistoryConf, Constant.ChatHistoryConfEnum.RECORD_TEXT_AUDIO.getCode())) { + String audioId = saveChatAudio(report); + saveChatText(report, agentId, macAddress, audioId); + } + + // 更新设备最后对话时间 + redisUtils.set(RedisKeys.getAgentDeviceLastConnectedAtById(agentId), new Date()); + return Boolean.TRUE; + } + + /** + * base64解码report.getOpusDataBase64(),存入ai_agent_chat_audio表 + */ + private String saveChatAudio(AgentChatHistoryReportDTO report) { String audioId = null; + if (report.getAudioBase64() != null && !report.getAudioBase64().isEmpty()) { try { byte[] audioData = Base64.getDecoder().decode(report.getAudioBase64()); @@ -56,20 +83,18 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic log.info("音频数据保存成功，audioId={}", audioId); } catch (Exception e) { log.error("音频数据保存失败", e); - return false; + return null; } } + return audioId; + } - // 2. 组装上报数据 - // 2.1 根据设备MAC地址查询对应的默认智能体，判断是否需要上报 - AgentEntity agentEntity = agentService.getDefaultAgentByMacAddress(macAddress); - if (agentEntity == null) { - return false; - } - String agentId = agentEntity.getId(); - log.info("设备 {} 对应智能体 {} 上报", macAddress, agentEntity.getId()); + /** + * 组装上报数据 + */ + private void saveChatText(AgentChatHistoryReportDTO report, String agentId, String macAddress, String audioId) { - // 2.2 构建聊天记录实体 + // 构建聊天记录实体 AgentChatHistoryEntity entity = AgentChatHistoryEntity.builder() .macAddress(macAddress) .agentId(agentId) @@ -79,10 +104,9 @@ public class AgentChatHistoryBizServiceImpl implements AgentChatHistoryBizServic .audioId(audioId) .build(); - // 3. 保存数据 + // 保存数据 agentChatHistoryService.save(entity); - // 4. 更新设备最后对话时间 - redisUtils.set(RedisKeys.getAgentDeviceLastConnectedAtById(agentId), new Date()); - return Boolean.TRUE; + + log.info("设备 {} 对应智能体 {} 上报成功", macAddress, agentId); } } diff --git a/main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java b/main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java index a15df002..ee42c934 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/config/service/impl/ConfigServiceImpl.java @@ -257,7 +257,8 @@ public class ConfigServiceImpl implements ConfigService { if (intentLLMModelId != null && intentLLMModelId.equals(llmModelId)) { intentLLMModelId = null; } - } else if ("function_call".equals(map.get("type"))) { + } + if (map.get("functions") != null) { String functionStr = (String) map.get("functions"); if (StringUtils.isNotBlank(functionStr)) { String[] functions = functionStr.split("\\;"); diff --git a/main/manager-api/src/main/java/xiaozhi/modules/device/dao/DeviceDao.java b/main/manager-api/src/main/java/xiaozhi/modules/device/dao/DeviceDao.java index 6570b89b..f3fdb7a3 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/device/dao/DeviceDao.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/device/dao/DeviceDao.java @@ -1,18 +1,18 @@ package xiaozhi.modules.device.dao; +import java.util.Date; + import org.apache.ibatis.annotations.Mapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper; import xiaozhi.modules.device.entity.DeviceEntity; -import java.util.Date; -import java.util.List; - @Mapper public interface DeviceDao extends BaseMapper { /** * 获取此智能体全部设备的最后连接时间 + * * @param agentId 智能体id * @return */ diff --git a/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java b/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java index 515aea7f..19ab7178 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java @@ -118,7 +118,8 @@ public class DeviceServiceImpl extends BaseServiceImpl DeviceEntity deviceById = getDeviceByMacAddress(macAddress); - if (deviceById == null || deviceById.getAutoUpdate() != 0) { + // 只有在设备已绑定且autoUpdate不为0的情况下才返回固件升级信息 + if (deviceById != null && deviceById.getAutoUpdate() != 0) { String type = deviceReport.getBoard() == null ? null : deviceReport.getBoard().getType(); DeviceReportRespDTO.Firmware firmware = buildFirmwareInfo(type, deviceReport.getApplication() == null ? null : deviceReport.getApplication().getVersion()); diff --git a/main/manager-api/src/main/java/xiaozhi/modules/sys/dto/SysParamsDTO.java b/main/manager-api/src/main/java/xiaozhi/modules/sys/dto/SysParamsDTO.java index ad816b35..a6f27b72 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/sys/dto/SysParamsDTO.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/sys/dto/SysParamsDTO.java @@ -39,7 +39,7 @@ public class SysParamsDTO implements Serializable { @Schema(description = "值类型") @NotBlank(message = "{sysparams.valuetype.require}", groups = DefaultGroup.class) - @Pattern(regexp = "^(string|number|boolean|array)$", message = "{sysparams.valuetype.pattern}", groups = DefaultGroup.class) + @Pattern(regexp = "^(string|number|boolean|array|json)$", message = "{sysparams.valuetype.pattern}", groups = DefaultGroup.class) private String valueType; @Schema(description = "备注") diff --git a/main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl/SysParamsServiceImpl.java b/main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl/SysParamsServiceImpl.java index 15f62816..2187a8b4 100644 --- a/main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl/SysParamsServiceImpl.java +++ b/main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl/SysParamsServiceImpl.java @@ -127,6 +127,12 @@ public class SysParamsServiceImpl extends BaseServiceImpl { RequestService.clearRequestTime(); @@ -61,7 +61,7 @@ export default { .fail((err) => { console.error('获取配置失败:', err); RequestService.reAjaxFun(() => { - this.getDeviceConfig(deviceId, callback); + this.getDeviceConfig(agentId, callback); }); }).send(); }, diff --git a/main/manager-web/src/assets/login/login-person.png b/main/manager-web/src/assets/login/login-person.png index 27477f2f..081eff3e 100644 Binary files a/main/manager-web/src/assets/login/login-person.png and b/main/manager-web/src/assets/login/login-person.png differ diff --git a/main/manager-web/src/assets/model/model.png b/main/manager-web/src/assets/model/model.png index f08ecb86..21132b76 100644 Binary files a/main/manager-web/src/assets/model/model.png and b/main/manager-web/src/assets/model/model.png differ diff --git a/main/manager-web/src/components/AddDeviceDialog.vue b/main/manager-web/src/components/AddDeviceDialog.vue index 51f5426b..b273a8dc 100644 --- a/main/manager-web/src/components/AddDeviceDialog.vue +++ b/main/manager-web/src/components/AddDeviceDialog.vue @@ -1,5 +1,5 @@