update:homeassistant插件化改造

update:homeassistant插件化改造

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,34 @@
+---
+name: Bug 报告（Bug Report）
+about: 反馈项目中的缺陷或问题
+title: "[Bug] 简短描述问题"
+labels: bug
+assignees: ''
+---
+
+## 🐛 问题描述
+<!-- 清晰简洁地描述问题是什么 -->
+
+## 🔍 复现步骤
+<!-- 详细描述复现问题的步骤 -->
+1. 打开 '...'
+2. 点击 '...'
+3. 滚动到 '...'
+4. 看到错误
+
+## 🤔 预期行为
+<!-- 简要描述预期的正确行为 -->
+
+## 😯 截图
+<!-- 如果适用，添加问题的截图 -->
+
+## 🖥️ 环境信息
+- 操作系统: [例如 Windows 10]
+- 浏览器: [例如 Chrome 89]
+- 项目版本: [例如 1.0.0]
+- Python版本: [例如 3.9.13]
+- Jdk版本：[例如 java 21.0.5 2024-10-15 LTS]
+- Nodejs版本：[例如 v20.14.0]
+
+## 📋 其他信息
+<!-- 在此添加关于此问题的任何其他上下文信息 -->

.github/ISSUE_TEMPLATE/code_improvement.md

@@ -0,0 +1,19 @@
+---
+name: 代码优化建议（Code Improvement）
+about: 提出对现有代码的优化或改进建议
+title: "[Improvement] 简短描述改进内容"
+labels: refactor
+assignees: ''
+---
+
+## 💡 改进描述
+<!-- 描述需要改进的代码部分以及存在的问题 -->
+
+## 🌟 改进建议
+<!-- 提出具体的改进方案或思路 -->
+
+## 🛠️ 相关代码
+<!-- 如果可能，提供相关代码片段或链接 -->
+
+## 📋 其他信息
+<!-- 在此添加任何其他相关信息 -->

.github/ISSUE_TEMPLATE/documentation_improvement.md

@@ -0,0 +1,16 @@
+---
+name: 文档改进建议（Documentation Improvement）
+about: 提出对项目文档的改进或补充建议
+title: "[Docs] 简短描述改进内容"
+labels: documentation
+assignees: ''
+---
+
+## 📚 改进描述
+<!-- 描述需要改进的文档部分以及存在的问题 -->
+
+## ✨ 改进建议
+<!-- 提出具体的改进方案或内容 -->
+
+## 📋 其他信息
+<!-- 在此添加任何其他相关信息 -->

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: 功能请求（Feature Request）
+about: 提出新的功能或改进建议
+title: "[Feature] 简短描述功能"
+labels: enhancement
+assignees: ''
+---
+
+## 🚀 需求描述
+<!-- 清晰简洁地描述你希望发生什么 -->
+
+## 🎯 解决方案
+<!-- 描述你认为可行的解决方案或实现方式 -->
+
+## 📝 备选方案
+<!-- 描述你考虑过的替代方案或功能 -->
+
+## 📋 其他信息
+<!-- 在此添加关于需求的任何其他上下文信息 -->

.gitignore

@@ -141,8 +141,8 @@ music/
 # Cython debug symbols
 cython_debug/
 *.iml
-model.pt
 tmp
+.history
 .DS_Store
 main/xiaozhi-server/data
 main/manager-web/node_modules
@@ -151,3 +151,6 @@ main/manager-web/node_modules
 .private_config.yaml
 .env.development
 
+# model files
+main/xiaozhi-server/models/SenseVoiceSmall/model.pt
+main/xiaozhi-server/models/sherpa-onnx*

README.md

@@ -51,12 +51,19 @@
         </a>
     </td>
     <td>
-        <a href="https://www.bilibili.com/video/av114036381327149" target="_blank">
+        <a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
          <picture>
            <img alt="控制家电开关" src="docs/images/demo5.png" />
          </picture>
         </a>
     </td>
+    <td>
+        <a href="https://www.bilibili.com/video/BV1kgA2eYEQ9" target="_blank">
+         <picture>
+           <img alt="成本最低配置" src="docs/images/demo4.png" />
+         </picture>
+        </a>
+    </td>
   </tr>
   <tr>
     <td>
@@ -74,13 +81,25 @@
         </a>
     </td>
     <td>
-        <a href="https://www.bilibili.com/video/BV1kgA2eYEQ9" target="_blank">
+        <a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
          <picture>
-           <img alt="成本最低配置" src="docs/images/demo4.png" />
+           <img alt="天气插件" src="docs/images/demo8.png" />
          </picture>
         </a>
     </td>
     <td>
+        <a href="https://www.bilibili.com/video/BV178XuYfEpi" target="_blank">
+         <picture>
+           <img alt="IOT指令控制设备" src="docs/images/demo9.png" />
+         </picture>
+        </a>
+    </td>
+    <td>
+        <a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
+         <picture>
+           <img alt="播报新闻" src="docs/images/demo0.png" />
+         </picture>
+        </a>
     </td>
   </tr>
 </table>
@@ -92,9 +111,11 @@
 - **硬件**：一套兼容 `xiaozhi-esp32`
   的硬件设备（具体型号请参考 [此处](https://rcnv1t9vps13.feishu.cn/wiki/DdgIw4BUgivWDPkhMj1cGIYCnRf)）。
 
-- **电脑或服务器**：至少 4 核 CPU、8G 内存的电脑。
+- **电脑或服务器**：建议 4 核 CPU、8G 内存的电脑。如果开启ASR也使用API，可运行在2核CPU、2G内存的服务器中。
 - **固件编译**：请将本后端服务的接口地址更新至 `xiaozhi-esp32` 项目中，再重新编译`xiaozhi-esp32`固件并烧录到设备上。
 
+![图片](docs/images/deploy.png)
+
 如果你没有esp32相关的硬件设备，但是非常想体验该项目，可以使用以下的项目让你的电脑、手机模拟成esp32设备。
 
 - [小智安卓端](https://github.com/TOM88812/xiaozhi-android-client)
@@ -141,12 +162,13 @@ server:
   支持 EdgeTTS（默认）、火山引擎豆包 TTS 等多种 TTS 接口，满足语音合成需求。
 - **记忆功能**  
   支持超长记忆、本地总结记忆、无记忆三种模式，满足不同场景需求。
+- **IOT功能**  
+  支持管理注册设备IOT功能，支持基于对话上下文语境下的智能物联网控制。
 
 ### 正在开发 🚧
 
 - 多种心情模式
 - 智控台webui
-- iot功能
 
 想了解具体开发进度，[请点击这里](https://github.com/users/xinnan-tech/projects/3)
 
@@ -168,7 +190,6 @@ server:
 | LLM |     FastgptLLM     |     fastgpt 接口调用      | 免费/消耗 token |                                              本地化部署，注意配置提示词需在 Fastgpt 控制台设置                                              |
 | LLM |     GeminiLLM      |      gemini 接口调用      |     免费      |                                      [点击申请密钥](https://aistudio.google.com/apikey)                                       |
 | LLM |      CozeLLM       |       coze 接口调用       |  消耗 token   |                                                需提供 bot_id、user_id 及个人令牌                                                 |
-| LLM |   Home Assistant   | homeassistant语音助手接口调用 |     免费      |                                                   需提供home assistant令牌                                                   |
 
 实际上，任何支持 openai 接口调用的 LLM 均可接入使用。
 
@@ -184,6 +205,7 @@ server:
 | TTS |  CosyVoiceSiliconflow  | 接口调用 | 消耗 token |                         需申请硅基流动 API 密钥；输出格式为 wav                          |
 | TTS |        TTS302AI        | 接口调用 | 消耗 token |                  [点击创建密钥](https://dash.302.ai/apis/list)                  |
 | TTS |       CozeCnTTS        | 接口调用 | 消耗 token |                        需提供 Coze API key；输出格式为 wav                         |
+| TTS |       GizwitsTTS       | 接口调用 | 消耗 token |                  [点击创建密钥](https://agentrouter.gizwitsapi.com)                  |
 | TTS |        ACGNTTS         | 接口调用 | 消耗 token |                        [联系网站管理员购买密钥](www.ttson.cn)                        |
 | TTS |       OpenAITTS        | 接口调用 | 消耗 token |                                 境外使用，境外购买                                 |
 | TTS |       FishSpeech       | 接口调用 |  免费/自定义  |                         本地启动 TTS 服务；启动方法见配置文件内说明                          |
@@ -206,16 +228,18 @@ server:
 | 类型  |   平台名称    | 使用方式 | 收费模式 | 备注 |
 |:---:|:---------:|:----:|:----:|:--:|
 | ASR |  FunASR   | 本地使用 |  免费  |    |
+| ASR | SherpaASR | 本地使用 |  免费  |    |
 | ASR | DoubaoASR | 接口调用 |  收费  |    |
 
+
 ---
 
 ### Memory 记忆存储
 
-|   类型   |      平台名称       | 使用方式 |   收费模式   | 备注 |
+|   类型   |      平台名称       | 使用方式 |   收费模式    | 备注 |
-|:------:|:---------------:|:----:|:--------:|:--:|
+|:------:|:---------------:|:----:|:---------:|:--:|
-| Memory |     mem0ai      | 接口调用 | 100次/月额度 |    |
+| Memory |     mem0ai      | 接口调用 | 1000次/月额度 |    |
-| Memory | mem_local_short | 本地总结 |    免费    |    |
+| Memory | mem_local_short | 本地总结 |    免费     |    |
 
 ---
 
@@ -350,7 +374,41 @@ VAD:
 
 ### 6、我想通过小智控制电灯、空调、远程开关机等操作 💡
 
-建议：在配置文件中将 `LLM` 设置为 `HomeAssistant`，通过 调用`HomeAssistant`接口实现相关控制。
+本项目，支持以工具调用的方式控制HomeAssistant设备
+
+1、首先选择一款支持function call支持的LLM，例如`ChatGLMLLM`。
+
+2、在配置文件中，将 `selected_module.Intent` 设置为 `function_call`。
+
+3、登录`HomeAssistant`，点击`左下角个人`，切换`安全`导航栏，划到底部`长期访问令牌`生成api_key。
+
+在配置文件中，配置好你的home assistant的`devices`（被控制的设备）和`api_key`和`base_url`等信息。例如：
+
+``` yaml 
+plugins
+  home_assistant:
+    devices:
+      - 客厅,玩具灯,switch.cuco_cn_460494544_cp1_on_p_2_1
+      - 卧室,台灯,switch.iot_cn_831898993_socn1_on_p_2_1
+    base_url: http://你的homeassistant地址:8123
+    api_key: 你的home assistant api访问令牌
+```
+
+最后，允许function_call 插件在配置文件中启用`hass_get_state`(必须)、`hass_set_state`(必须)、`hass_play_music`(不想用ha听音乐可以不启动)，例如：
+
+``` yaml 
+Intent:
+  ...
+  function_call:
+    type: nointent
+    functions:
+      - change_role
+      - get_weather
+      - get_news
+      - hass_get_state
+      - hass_set_state
+      - hass_play_music
+```
 
 ### 7、更多问题，可联系我们反馈 💬
 

README_en.md

@@ -165,6 +165,7 @@ In fact, any LLM that supports OpenAI API calls can be integrated.
 | Type |    Platform Name    | Usage Method | Pricing Model | Remarks |
 |:----:|:-------------------:|:------------:|:-------------:|:-------:|
 | ASR  | FunASR              | Local        | Free          |         |
+| ASR  | SherpaASR           | Local        | Free          |         |
 | ASR  | DoubaoASR           | API call     | Paid          |         |
 
 ---

docs/Deployment.md

@@ -1,3 +1,5 @@
+# 部署方案参考
+![图片](images/deploy.png)
 # 方式一：docker快速部署
 
 docker镜像已支持x86架构、arm64架构的CPU，支持在国产操作系统上运行。

docs/contributor_open_letter.md

@@ -35,7 +35,7 @@ John，2025.3.11，广州
 
 ### 1、成为普通贡献者
 
-Fork 项目，，提交 PR，由开发者审核后合入主分支。
+Fork 项目，提交 PR，由开发者审核后合入主分支。
 
 ### 2、成为开发者
 

main/README.md

@@ -18,6 +18,6 @@ xiaozhi-esp32-server
 
 # manager-web 、manager-api接口协议
 
-[manager前后端接口协议](https://app.apifox.com/invite/project?token=H_8qhgfjUeaAL0wybghgU)
+[manager前后端接口协议](https://app.apifox.com/invite/project?token=eXg2_tUv85q-gc3ZRowmn)
 
 [前端页面设计图](https://codesign.qq.com/app/s/526108506410828)

main/manager-api/src/main/java/xiaozhi/common/aspect/DataFilterAspect.java

@@ -1,99 +0,0 @@
-package xiaozhi.common.aspect;
-
-import cn.hutool.core.collection.CollUtil;
-import xiaozhi.common.annotation.DataFilter;
-import xiaozhi.common.constant.Constant;
-import xiaozhi.common.exception.ErrorCode;
-import xiaozhi.common.exception.RenException;
-import xiaozhi.common.interceptor.DataScope;
-import xiaozhi.common.user.UserDetail;
-import xiaozhi.modules.security.user.SecurityUser;
-import xiaozhi.modules.sys.enums.SuperAdminEnum;
-import org.apache.commons.lang3.StringUtils;
-import org.aspectj.lang.JoinPoint;
-import org.aspectj.lang.annotation.Aspect;
-import org.aspectj.lang.annotation.Before;
-import org.aspectj.lang.annotation.Pointcut;
-import org.aspectj.lang.reflect.MethodSignature;
-import org.springframework.stereotype.Component;
-
-import java.lang.reflect.Method;
-import java.util.List;
-import java.util.Map;
-
-/**
- * 数据过滤，切面处理类
- * Copyright (c) 人人开源 All rights reserved.
- * Website: https://www.renren.io
- */
-@Aspect
-@Component
-public class DataFilterAspect {
-
-    @Pointcut("@annotation(xiaozhi.common.annotation.DataFilter)")
-    public void dataFilterCut() {
-
-    }
-
-    @Before("dataFilterCut()")
-    public void dataFilter(JoinPoint point) {
-        Object params = point.getArgs()[0];
-        if (params != null && params instanceof Map) {
-            UserDetail user = SecurityUser.getUser();
-
-            //如果是超级管理员，则不进行数据过滤
-            if (user.getSuperAdmin() == SuperAdminEnum.YES.value()) {
-                return;
-            }
-
-            try {
-                //否则进行数据过滤
-                Map map = (Map) params;
-                String sqlFilter = getSqlFilter(user, point);
-                map.put(Constant.SQL_FILTER, new DataScope(sqlFilter));
-            } catch (Exception e) {
-
-            }
-
-            return;
-        }
-
-        throw new RenException(ErrorCode.DATA_SCOPE_PARAMS_ERROR);
-    }
-
-    /**
-     * 获取数据过滤的SQL
-     */
-    private String getSqlFilter(UserDetail user, JoinPoint point) throws Exception {
-        MethodSignature signature = (MethodSignature) point.getSignature();
-        Method method = point.getTarget().getClass().getDeclaredMethod(signature.getName(), signature.getParameterTypes());
-        DataFilter dataFilter = method.getAnnotation(DataFilter.class);
-
-        //获取表的别名
-        String tableAlias = dataFilter.tableAlias();
-        if (StringUtils.isNotBlank(tableAlias)) {
-            tableAlias += ".";
-        }
-
-        StringBuilder sqlFilter = new StringBuilder();
-        sqlFilter.append(" (");
-
-        //部门ID列表
-        List<Long> deptIdList = user.getDeptIdList();
-        if (CollUtil.isNotEmpty(deptIdList)) {
-            sqlFilter.append(tableAlias).append(dataFilter.deptId());
-
-            sqlFilter.append(" in(").append(StringUtils.join(deptIdList, ",")).append(")");
-        }
-
-        //查询本人数据
-        if (CollUtil.isNotEmpty(deptIdList)) {
-            sqlFilter.append(" or ");
-        }
-        sqlFilter.append(tableAlias).append(dataFilter.userId()).append("=").append(user.getId());
-
-        sqlFilter.append(")");
-
-        return sqlFilter.toString();
-    }
-}

main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java

@@ -73,10 +73,6 @@ public interface Constant {
      * 排序方式
      */
     String ORDER = "order";
-    /**
-     * token header
-     */
-    String TOKEN_HEADER = "token";
 
     String AUTHORIZATION = "Authorization";
 

main/manager-api/src/main/java/xiaozhi/common/handler/FieldMetaObjectHandler.java

@@ -34,9 +34,6 @@ public class FieldMetaObjectHandler implements MetaObjectHandler {
         //创建时间
         strictInsertFill(metaObject, CREATE_DATE, Date.class, date);
 
-        //创建者所属部门
-        strictInsertFill(metaObject, DEPT_ID, Long.class, user.getDeptId());
-
         //更新者
         strictInsertFill(metaObject, UPDATER, Long.class, user.getId());
         //更新时间

main/manager-api/src/main/java/xiaozhi/common/user/UserDetail.java

@@ -3,7 +3,6 @@ package xiaozhi.common.user;
 import lombok.Data;
 
 import java.io.Serializable;
-import java.util.List;
 
 /**
  * 登录用户信息
@@ -14,19 +13,7 @@ import java.util.List;
 public class UserDetail implements Serializable {
     private Long id;
     private String username;
-    private String realName;
-    private String headUrl;
-    private Integer gender;
-    private String email;
-    private String mobile;
-    private Long deptId;
-    private String password;
-    private Integer status;
     private Integer superAdmin;
     private String token;
-    /**
+    private Integer status;
-     * 部门数据权限
-     */
-    private List<Long> deptIdList;
-
 }

main/manager-api/src/main/java/xiaozhi/common/utils/HttpContextUtils.java

@@ -1,6 +1,5 @@
 package xiaozhi.common.utils;
 
-import xiaozhi.common.constant.Constant;
 import jakarta.servlet.http.HttpServletRequest;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.http.HttpHeaders;
@@ -8,6 +7,8 @@ import org.springframework.util.DigestUtils;
 import org.springframework.web.context.request.RequestAttributes;
 import org.springframework.web.context.request.RequestContextHolder;
 import org.springframework.web.context.request.ServletRequestAttributes;
+import xiaozhi.common.exception.ErrorCode;
+import xiaozhi.common.exception.RenException;
 
 import java.util.Date;
 import java.util.Enumeration;
@@ -30,13 +31,14 @@ public class HttpContextUtils {
         return ((ServletRequestAttributes) requestAttributes).getRequest();
     }
 
-    public static String getToken() {
+    public static String getToken(String authorization) {
-        HttpServletRequest httpRequest = getHttpServletRequest();
+        String token;
-        String token = httpRequest.getHeader(Constant.TOKEN_HEADER);
+        if (StringUtils.isBlank(authorization) && authorization.contains("Bearer ")) {
-
+            throw new RenException(ErrorCode.UNAUTHORIZED);
-        //如果header中不存在token，则从参数中获取token
+        }
+        token = authorization.replace("Bearer ", "");
         if (StringUtils.isBlank(token)) {
-            token = httpRequest.getParameter(Constant.TOKEN_HEADER);
+            throw new RenException(ErrorCode.TOKEN_NOT_EMPTY);
         }
         return token;
     }

main/manager-api/src/main/java/xiaozhi/common/utils/Result.java

@@ -1,16 +1,10 @@
 package xiaozhi.common.utils;
 
-import xiaozhi.common.exception.ErrorCode;
-import xiaozhi.common.page.PageData;
-import xiaozhi.modules.security.user.SecurityUser;
 import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.Data;
-import org.apache.commons.lang3.StringUtils;
+import xiaozhi.common.exception.ErrorCode;
 
 import java.io.Serializable;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
 
 /**
  * 响应数据
@@ -42,9 +36,6 @@ public class Result<T> implements Serializable {
         return this;
     }
 
-    public boolean success() {
-        return code == 0;
-    }
 
     public Result<T> error() {
         this.code = ErrorCode.INTERNAL_SERVER_ERROR;

main/manager-api/src/main/java/xiaozhi/modules/device/controller/DeviceController.java

@@ -25,11 +25,10 @@ import xiaozhi.modules.security.user.SecurityUser;
 import java.util.List;
 import java.util.Map;
 
+@Tag(name = "设备管理")
 @AllArgsConstructor
 @RestController
 @RequestMapping("/device")
-@Tag(name = "设备管理")
-
 public class DeviceController {
     private final DeviceService deviceService;
     private final RedisUtils redisUtils;

main/manager-api/src/main/java/xiaozhi/modules/security/config/ShiroConfig.java

@@ -70,7 +70,9 @@ public class ShiroConfig {
         filterMap.put("/v3/api-docs/**", "anon");
         filterMap.put("/doc.html", "anon");
         filterMap.put("/favicon.ico", "anon");
-        filterMap.put("/user/**", "anon");
+        filterMap.put("/user/captcha", "anon");
+        filterMap.put("/user/login", "anon");
+        filterMap.put("/user/register", "anon");
         filterMap.put("/**", "oauth2");
         shiroFilter.setFilterChainDefinitionMap(filterMap);
 

main/manager-api/src/main/java/xiaozhi/modules/security/controller/LoginController.java

@@ -4,20 +4,21 @@ import io.swagger.v3.oas.annotations.Operation;
 import io.swagger.v3.oas.annotations.tags.Tag;
 import jakarta.servlet.http.HttpServletResponse;
 import lombok.AllArgsConstructor;
-import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.web.bind.annotation.*;
 import xiaozhi.common.exception.ErrorCode;
 import xiaozhi.common.exception.RenException;
 import xiaozhi.common.page.TokenDTO;
+import xiaozhi.common.user.UserDetail;
 import xiaozhi.common.utils.Result;
 import xiaozhi.common.validator.AssertUtils;
-import xiaozhi.modules.security.dao.SysUserTokenDao;
 import xiaozhi.modules.security.dto.LoginDTO;
 import xiaozhi.modules.security.password.PasswordUtils;
 import xiaozhi.modules.security.service.CaptchaService;
 import xiaozhi.modules.security.service.SysUserTokenService;
+import xiaozhi.modules.security.user.SecurityUser;
+import xiaozhi.modules.sys.dto.PasswordDTO;
 import xiaozhi.modules.sys.dto.SysUserDTO;
 import xiaozhi.modules.sys.service.SysUserService;
 
@@ -26,7 +27,6 @@ import java.io.IOException;
 /**
  * 登录控制层
  */
-@Tag(name = "登录管理")
 @AllArgsConstructor
 @RestController
 @RequestMapping("/user")
@@ -36,7 +36,6 @@ public class LoginController {
     private final SysUserTokenService sysUserTokenService;
     private final CaptchaService captchaService;
 
-    private static final Logger logger = LoggerFactory.getLogger(LoginController.class);
 
     @GetMapping("/captcha")
     @Operation(summary = "验证码")
@@ -79,32 +78,31 @@ public class LoginController {
         }
         // 按照用户名获取用户
         SysUserDTO userDTO = sysUserService.getByUsername(login.getUsername());
-        if (userDTO != null){
+        if (userDTO != null) {
             throw new RenException("此手机号码已经注册过");
         }
         userDTO = new SysUserDTO();
         userDTO.setUsername(login.getUsername());
         userDTO.setPassword(login.getPassword());
         sysUserService.save(userDTO);
-        return new Result<Void>();
+        return new Result<>();
 
     }
 
     @GetMapping("/info")
     @Operation(summary = "用户信息获取")
-    public Result<SysUserDTO> info(@RequestHeader("Authorization")String authorization) {
+    public Result<UserDetail> info() {
-        logger.info("the authorization:{}", authorization);
+        UserDetail user = SecurityUser.getUser();
+        Result<UserDetail> result = new Result<>();
+        result.setData(user);
+        return result;
+    }
 
-        String token;
+    @PutMapping("/change-password")
-        if (StringUtils.isBlank(authorization) && authorization.contains("Bearer ")) {
+    @Operation(summary = "修改用户密码")
-            throw new RenException(ErrorCode.UNAUTHORIZED);
+    public Result<?> changePassword(@RequestBody PasswordDTO passwordDTO) {
-        }
+        Long userId = SecurityUser.getUserId();
-        token = authorization.replace("Bearer ", "");
+        sysUserTokenService.changePassword(userId, passwordDTO);
-        if (StringUtils.isBlank(token)) {
+        return new Result<>();
-            throw new RenException(ErrorCode.UNAUTHORIZED);
-        }
-        SysUserDTO sysUserDTO = sysUserTokenService.getUserByToken(token);
-        Result result = new Result<SysUserDTO>();
-        return result.ok(sysUserDTO);
     }
 }

main/manager-api/src/main/java/xiaozhi/modules/security/oauth2/Oauth2Filter.java

@@ -11,7 +11,6 @@ import org.apache.shiro.web.filter.authc.AuthenticatingFilter;
 import org.springframework.web.bind.annotation.RequestMethod;
 import xiaozhi.common.constant.Constant;
 import xiaozhi.common.exception.ErrorCode;
-import xiaozhi.common.exception.RenException;
 import xiaozhi.common.utils.HttpContextUtils;
 import xiaozhi.common.utils.JsonUtils;
 import xiaozhi.common.utils.Result;
@@ -90,21 +89,10 @@ public class Oauth2Filter extends AuthenticatingFilter {
      * 获取请求的token
      */
     private String getRequestToken(HttpServletRequest httpRequest) {
-        String token;
+        String token = null;
         //从header中获取token
         String authorization = httpRequest.getHeader(Constant.AUTHORIZATION);
-        if (StringUtils.isBlank(authorization) && authorization.contains("Bearer ")) {
+        if (StringUtils.isNotBlank(authorization) && authorization.startsWith("Bearer ")) {
-            throw new RenException(ErrorCode.UNAUTHORIZED);
-        }
-        token = authorization.replace("Bearer ", "");
-
-        //如果header中不存在token，则从参数中获取token
-        if (StringUtils.isBlank(token)) {
-            authorization = httpRequest.getParameter(Constant.AUTHORIZATION);
-
-            if (StringUtils.isBlank(authorization) && authorization.contains("Bearer ")) {
-                throw new RenException(ErrorCode.UNAUTHORIZED);
-            }
             token = authorization.replace("Bearer ", "");
         }
         return token;

main/manager-api/src/main/java/xiaozhi/modules/security/oauth2/Oauth2Realm.java

@@ -14,12 +14,12 @@ import xiaozhi.common.exception.ErrorCode;
 import xiaozhi.common.user.UserDetail;
 import xiaozhi.common.utils.ConvertUtils;
 import xiaozhi.common.utils.MessageUtils;
-import xiaozhi.modules.security.controller.LoginController;
 import xiaozhi.modules.security.entity.SysUserTokenEntity;
 import xiaozhi.modules.security.service.ShiroService;
 import xiaozhi.modules.sys.entity.SysUserEntity;
 import xiaozhi.modules.sys.enums.SuperAdminEnum;
 
+import java.util.HashSet;
 import java.util.Set;
 
 /**
@@ -48,7 +48,7 @@ public class Oauth2Realm extends AuthorizingRealm {
         UserDetail user = (UserDetail) principals.getPrimaryPrincipal();
 
         //用户权限列表
-        Set<String> permsSet = shiroService.getUserPermissions(user);
+        Set<String> permsSet = new HashSet<>();
 
         if (user.getSuperAdmin() == SuperAdminEnum.YES.value()) {
             permsSet.add("sys:role:superAdmin");
@@ -82,8 +82,6 @@ public class Oauth2Realm extends AuthorizingRealm {
         //转换成UserDetail对象
         UserDetail userDetail = ConvertUtils.sourceToTarget(userEntity, UserDetail.class);
 
-        //获取用户对应的部门数据权限
-        userDetail.setDeptIdList(null);
         userDetail.setToken(accessToken);
 
         //账号锁定

main/manager-api/src/main/java/xiaozhi/modules/security/service/ShiroService.java

@@ -1,22 +1,14 @@
 package xiaozhi.modules.security.service;
 
-import xiaozhi.common.user.UserDetail;
 import xiaozhi.modules.security.entity.SysUserTokenEntity;
 import xiaozhi.modules.sys.entity.SysUserEntity;
 
-import java.util.List;
-import java.util.Set;
-
 /**
  * shiro相关接口
  * Copyright (c) 人人开源 All rights reserved.
  * Website: https://www.renren.io
  */
 public interface ShiroService {
-    /**
-     * 获取用户权限列表
-     */
-    Set<String> getUserPermissions(UserDetail user);
 
     SysUserTokenEntity getByToken(String token);
 

main/manager-api/src/main/java/xiaozhi/modules/security/service/SysUserTokenService.java

@@ -1,14 +1,12 @@
 package xiaozhi.modules.security.service;
 
-import xiaozhi.common.page.PageData;
 import xiaozhi.common.page.TokenDTO;
 import xiaozhi.common.service.BaseService;
 import xiaozhi.common.utils.Result;
 import xiaozhi.modules.security.entity.SysUserTokenEntity;
+import xiaozhi.modules.sys.dto.PasswordDTO;
 import xiaozhi.modules.sys.dto.SysUserDTO;
 
-import java.util.Map;
-
 /**
  * 用户Token
  * Copyright (c) 人人开源 All rights reserved.
@@ -32,4 +30,12 @@ public interface SysUserTokenService extends BaseService<SysUserTokenEntity> {
      */
     void logout(Long userId);
 
+    /**
+     * 修改密码
+     *
+     * @param userId
+     * @param passwordDTO
+     */
+    void changePassword(Long userId, PasswordDTO passwordDTO);
+
 }

main/manager-api/src/main/java/xiaozhi/modules/security/service/impl/ShiroServiceImpl.java

@@ -19,22 +19,6 @@ public class ShiroServiceImpl implements ShiroService {
     private final SysUserDao sysUserDao;
     private final SysUserTokenDao sysUserTokenDao;
 
-    @Override
-    public Set<String> getUserPermissions(UserDetail user) {
-        //系统管理员，拥有最高权限
-        // TODO: 暂时写死，后续改成从数据库查询
-        List<String> permissionsList = new ArrayList<>();
-        //用户权限列表
-        Set<String> permsSet = new HashSet<>();
-        for (String permissions : permissionsList) {
-            if (StringUtils.isBlank(permissions)) {
-                continue;
-            }
-            permsSet.addAll(Arrays.asList(permissions.trim().split(",")));
-        }
-
-        return permsSet;
-    }
 
     @Override
     public SysUserTokenEntity getByToken(String token) {

main/manager-api/src/main/java/xiaozhi/modules/security/service/impl/SysUserTokenServiceImpl.java

@@ -2,6 +2,7 @@ package xiaozhi.modules.security.service.impl;
 
 import cn.hutool.core.date.DateUtil;
 import lombok.AllArgsConstructor;
+import org.springframework.stereotype.Service;
 import xiaozhi.common.exception.ErrorCode;
 import xiaozhi.common.exception.RenException;
 import xiaozhi.common.page.TokenDTO;
@@ -12,7 +13,7 @@ import xiaozhi.modules.security.dao.SysUserTokenDao;
 import xiaozhi.modules.security.entity.SysUserTokenEntity;
 import xiaozhi.modules.security.oauth2.TokenGenerator;
 import xiaozhi.modules.security.service.SysUserTokenService;
-import org.springframework.stereotype.Service;
+import xiaozhi.modules.sys.dto.PasswordDTO;
 import xiaozhi.modules.sys.dto.SysUserDTO;
 import xiaozhi.modules.sys.service.SysUserService;
 
@@ -81,6 +82,9 @@ public class SysUserTokenServiceImpl extends BaseServiceImpl<SysUserTokenDao, Sy
     @Override
     public SysUserDTO getUserByToken(String token) {
         SysUserTokenEntity userToken = baseDao.getByToken(token);
+        if (null == userToken) {
+            throw new RenException(ErrorCode.TOKEN_INVALID);
+        }
 
         Date now = new Date();
         if (userToken.getExpireDate().before(now)) {
@@ -97,4 +101,14 @@ public class SysUserTokenServiceImpl extends BaseServiceImpl<SysUserTokenDao, Sy
         Date expireDate = DateUtil.offsetMinute(new Date(), -1);
         baseDao.logout(userId, expireDate);
     }
+
+    @Override
+    public void changePassword(Long userId, PasswordDTO passwordDTO) {
+        // 修改密码
+        sysUserService.changePassword(userId, passwordDTO);
+
+        // 使 token 失效，后需要重新登录
+        Date expireDate = DateUtil.offsetMinute(new Date(), -1);
+        baseDao.logout(userId, expireDate);
+    }
 }

main/manager-api/src/main/java/xiaozhi/modules/security/user/SecurityUser.java

@@ -46,11 +46,4 @@ public class SecurityUser {
     public static Long getUserId() {
         return getUser().getId();
     }
-
-    /**
-     * 获取部门ID
-     */
-    public static Long getDeptId() {
-        return getUser().getDeptId();
-    }
 }

main/manager-api/src/main/java/xiaozhi/modules/sys/entity/SysUserEntity.java

@@ -3,9 +3,9 @@ package xiaozhi.modules.sys.entity;
 import com.baomidou.mybatisplus.annotation.FieldFill;
 import com.baomidou.mybatisplus.annotation.TableField;
 import com.baomidou.mybatisplus.annotation.TableName;
-import xiaozhi.common.entity.BaseEntity;
 import lombok.Data;
 import lombok.EqualsAndHashCode;
+import xiaozhi.common.entity.BaseEntity;
 
 import java.util.Date;
 
@@ -32,11 +32,6 @@ public class SysUserEntity extends BaseEntity {
      * 状态  0：停用   1：正常
      */
     private Integer status;
-    /**
-     * 更新者
-     */
-    @TableField(fill = FieldFill.INSERT)
-    private Date createDate;
     /**
      * 更新者
      */
@@ -47,10 +42,5 @@ public class SysUserEntity extends BaseEntity {
      */
     @TableField(fill = FieldFill.INSERT_UPDATE)
     private Date updateDate;
-    /**
-     * 部门名称
-     */
-    @TableField(exist = false)
-    private String deptName;
 
 }

main/manager-api/src/main/java/xiaozhi/modules/sys/service/SysUserService.java

@@ -1,6 +1,7 @@
 package xiaozhi.modules.sys.service;
 
 import xiaozhi.common.service.BaseService;
+import xiaozhi.modules.sys.dto.PasswordDTO;
 import xiaozhi.modules.sys.dto.SysUserDTO;
 import xiaozhi.modules.sys.entity.SysUserEntity;
 
@@ -18,4 +19,5 @@ public interface SysUserService extends BaseService<SysUserEntity> {
 
     void delete(Long[] ids);
 
+    void changePassword(Long userId, PasswordDTO passwordDTO);
 }

main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl/SysUserServiceImpl.java

@@ -10,6 +10,7 @@ import xiaozhi.common.service.impl.BaseServiceImpl;
 import xiaozhi.common.utils.ConvertUtils;
 import xiaozhi.modules.security.password.PasswordUtils;
 import xiaozhi.modules.sys.dao.SysUserDao;
+import xiaozhi.modules.sys.dto.PasswordDTO;
 import xiaozhi.modules.sys.dto.SysUserDTO;
 import xiaozhi.modules.sys.entity.SysUserEntity;
 import xiaozhi.modules.sys.enums.SuperAdminEnum;
@@ -81,6 +82,31 @@ public class SysUserServiceImpl extends BaseServiceImpl<SysUserDao, SysUserEntit
         baseDao.deleteBatchIds(Arrays.asList(ids));
     }
 
+    @Override
+    public void changePassword(Long userId, PasswordDTO passwordDTO) {
+        SysUserEntity sysUserEntity = sysUserDao.selectById(userId);
+
+        if (null == sysUserEntity) {
+            throw new RenException(ErrorCode.TOKEN_INVALID);
+        }
+
+        // 判断旧密码是否正确
+        if (!PasswordUtils.matches(passwordDTO.getPassword(), sysUserEntity.getPassword())) {
+            throw new RenException("旧密码输入错误");
+        }
+
+        //新密码强度
+        if (!isStrongPassword(passwordDTO.getNewPassword())) {
+            throw new RenException(ErrorCode.PASSWORD_WEAK_ERROR);
+        }
+
+        //密码加密
+        String password = PasswordUtils.encode(passwordDTO.getNewPassword());
+        sysUserEntity.setPassword(password);
+
+        updateById(sysUserEntity);
+    }
+
     private Long getUserCount() {
         QueryWrapper<SysUserEntity> queryWrapper = new QueryWrapper<>();
         return baseDao.selectCount(queryWrapper);

main/manager-api/src/main/java/xiaozhi/modules/timbre/controller/TimbreController.java

@@ -0,0 +1,85 @@
+package xiaozhi.modules.timbre.controller;
+
+import io.swagger.v3.oas.annotations.Operation;
+import io.swagger.v3.oas.annotations.Parameter;
+import io.swagger.v3.oas.annotations.Parameters;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import lombok.AllArgsConstructor;
+import org.apache.shiro.authz.annotation.RequiresPermissions;
+import org.springframework.web.bind.annotation.*;
+import xiaozhi.common.constant.Constant;
+import xiaozhi.common.page.PageData;
+import xiaozhi.common.utils.Result;
+import xiaozhi.common.validator.ValidatorUtils;
+import xiaozhi.modules.timbre.dto.TimbreDataDTO;
+import xiaozhi.modules.timbre.dto.TimbrePageDTO;
+import xiaozhi.modules.timbre.service.TimbreService;
+import xiaozhi.modules.timbre.vo.TimbreDetailsVO;
+
+import java.util.Map;
+
+/**
+ * 音色控制层
+ *
+ * @author zjy
+ * @since 2025-3-21
+ */
+@AllArgsConstructor
+@RestController
+@RequestMapping("/ttsVoice")
+@Tag(name = "音色管理")
+public class TimbreController {
+    private final TimbreService timbreService;
+
+    @GetMapping
+    @Operation(summary = "分页查找")
+    @RequiresPermissions("sys:role:superAdmin")
+    @Parameters({
+            @Parameter(name = "ttsModelId", description = "对应 TTS 模型主键", required = true),
+            @Parameter(name = "name", description = "音色名称"),
+            @Parameter(name = Constant.PAGE, description = "当前页码，从1开始", required = true),
+            @Parameter(name = Constant.LIMIT, description = "每页显示记录数", required = true),
+    })
+    public Result<PageData<TimbreDetailsVO>> page(
+            @Parameter(hidden = true) @RequestParam Map<String, Object> params) {
+        TimbrePageDTO dto = new TimbrePageDTO();
+        dto.setTtsModelId((String) params.get("ttsModelId"));
+        dto.setName((String) params.get("name"));
+        dto.setLimit((String) params.get(Constant.LIMIT));
+        dto.setPage((String) params.get(Constant.PAGE));
+
+        ValidatorUtils.validateEntity(dto);
+        PageData<TimbreDetailsVO> page = timbreService.page(dto);
+        return new Result<PageData<TimbreDetailsVO>>().ok(page);
+    }
+
+    @PostMapping
+    @Operation(summary = "音色保存")
+    @RequiresPermissions("sys:role:superAdmin")
+    public Result<Void> save(@RequestBody TimbreDataDTO dto) {
+        ValidatorUtils.validateEntity(dto);
+        timbreService.save(dto);
+        return new Result<>();
+    }
+
+    @PutMapping("/{id}")
+    @Operation(summary = "音色修改")
+    @RequiresPermissions("sys:role:superAdmin")
+    public Result<Void> update(
+            @PathVariable Long id,
+            @RequestBody TimbreDataDTO dto) {
+        ValidatorUtils.validateEntity(dto);
+        timbreService.update(id, dto);
+        return new Result<>();
+    }
+
+    @DeleteMapping("/{id}")
+    @Operation(summary = "音色删除")
+    @RequiresPermissions("sys:role:superAdmin")
+    public Result<Void> delete(@PathVariable Long id) {
+        timbreService.delete(new Long[]{id});
+        return new Result<>();
+    }
+
+
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/dao/TimbreDao.java

@@ -0,0 +1,14 @@
+package xiaozhi.modules.timbre.dao;
+
+import com.baomidou.mybatisplus.core.mapper.BaseMapper;
+import org.apache.ibatis.annotations.Mapper;
+import xiaozhi.modules.timbre.entity.TimbreEntity;
+
+/**
+ * 音色持久层定义
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Mapper
+public interface TimbreDao extends BaseMapper<TimbreEntity> {
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/dto/TimbreDataDTO.java

@@ -0,0 +1,45 @@
+package xiaozhi.modules.timbre.dto;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.validation.constraints.Min;
+import jakarta.validation.constraints.NotBlank;
+import lombok.Data;
+
+
+/**
+ * 音色表数据DTO
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Data
+@Schema(description = "音色表信息")
+public class TimbreDataDTO {
+
+    @Schema(description = "语言")
+    @NotBlank(message = "{timbre.languages.require}")
+    private String languages;
+
+    @Schema(description = "音色名称")
+    @NotBlank(message = "{timbre.name.require}")
+    private String name;
+
+    @Schema(description = "备注")
+    @NotBlank(message = "{timbre.remark.require}")
+    private String remark;
+
+    @Schema(description = "排序")
+    @Min(value = 0, message = "{sort.number}")
+    private long sort;
+
+    @Schema(description = "对应 TTS 模型主键")
+    @NotBlank(message = "{timbre.ttsModelId.require}")
+    private String ttsModelId;
+
+    @Schema(description = "音色编码")
+    @NotBlank(message = "{timbre.ttsVoice.require}")
+    private String ttsVoice;
+
+    @Schema(description = "音频播放地址")
+    @NotBlank(message = "{timbre.voiceDemo.require}")
+    private String voiceDemo;
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/dto/TimbrePageDTO.java

@@ -0,0 +1,28 @@
+package xiaozhi.modules.timbre.dto;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import jakarta.validation.constraints.NotBlank;
+import lombok.Data;
+
+/**
+ * 音色分页参数DTO
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Data
+@Schema(description = "音色分页参数")
+public class TimbrePageDTO {
+
+    @Schema(description = "对应 TTS 模型主键")
+    @NotBlank(message = "{timbre.ttsModelId.require}")
+    private String ttsModelId;
+
+    @Schema(description = "音色名称")
+    private String name;
+
+    @Schema(description = "页数")
+    private String page;
+
+    @Schema(description = "显示列数")
+    private String limit;
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/entity/TimbreEntity.java

@@ -0,0 +1,48 @@
+package xiaozhi.modules.timbre.entity;
+
+import com.baomidou.mybatisplus.annotation.TableName;
+import io.swagger.v3.oas.annotations.media.Schema;
+import lombok.Data;
+import lombok.EqualsAndHashCode;
+import xiaozhi.common.entity.BaseEntity;
+
+import java.util.Date;
+
+/**
+ * 音色表实体类
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Data
+@EqualsAndHashCode(callSuper = false)
+@TableName("ai_tts_voice")
+@Schema(description = "音色信息")
+public class TimbreEntity extends BaseEntity {
+
+    @Schema(description = "语言")
+    private String languages;
+
+    @Schema(description = "音色名称")
+    private String name;
+
+    @Schema(description = "备注")
+    private String remark;
+
+    @Schema(description = "排序")
+    private long sort;
+
+    @Schema(description = "对应 TTS 模型主键")
+    private String ttsModelId;
+
+    @Schema(description = "音色编码")
+    private String ttsVoice;
+
+    @Schema(description = "音频播放地址")
+    private String voiceDemo;
+
+    @Schema(description = "更新者")
+    private Long updater;
+
+    @Schema(description = "更新时间")
+    private Date updateDate;
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/service/TimbreService.java

@@ -0,0 +1,49 @@
+package xiaozhi.modules.timbre.service;
+
+import xiaozhi.common.page.PageData;
+import xiaozhi.common.service.BaseService;
+import xiaozhi.modules.timbre.dto.TimbreDataDTO;
+import xiaozhi.modules.timbre.dto.TimbrePageDTO;
+import xiaozhi.modules.timbre.entity.TimbreEntity;
+import xiaozhi.modules.timbre.vo.TimbreDetailsVO;
+
+
+/**
+ * 音色的业务层的定义
+ * @author zjy
+ * @since 2025-3-21
+ */
+public interface TimbreService extends BaseService<TimbreEntity> {
+    /**
+     * 分页获取音色指定tts的下的音色
+     * @param dto 分页查找参数
+     * @return 音色列表分页数据
+     */
+    PageData<TimbreDetailsVO> page(TimbrePageDTO dto);
+
+    /**
+     * 获取音色指定id的详情信息
+     * @param timbreId 音色表id
+     * @return 音色信息
+     */
+    TimbreDetailsVO get(Long timbreId);
+
+    /**
+     * 保存音色信息
+     * @param dto 需要保存数据
+     */
+    void save(TimbreDataDTO dto);
+
+    /**
+     * 保存音色信息
+     * @param timbreId 需要修改的id
+     * @param dto 需要修改的数据
+     */
+    void update(Long timbreId,TimbreDataDTO dto);
+
+    /**
+     * 批量删除音色
+     * @param ids 需要被删除的音色id列表
+     */
+    void delete(Long[] ids);
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/service/impl/TimbreServiceImpl.java

@@ -0,0 +1,85 @@
+package xiaozhi.modules.timbre.service.impl;
+
+import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
+import com.baomidou.mybatisplus.core.metadata.IPage;
+import org.apache.commons.lang3.StringUtils;
+import org.springframework.stereotype.Service;
+import org.springframework.transaction.annotation.Transactional;
+import xiaozhi.common.constant.Constant;
+import xiaozhi.common.page.PageData;
+import xiaozhi.common.service.impl.BaseServiceImpl;
+import xiaozhi.common.utils.ConvertUtils;
+import xiaozhi.modules.timbre.dao.TimbreDao;
+import xiaozhi.modules.timbre.dto.TimbreDataDTO;
+import xiaozhi.modules.timbre.dto.TimbrePageDTO;
+import xiaozhi.modules.timbre.service.TimbreService;
+import xiaozhi.modules.timbre.entity.TimbreEntity;
+import xiaozhi.modules.timbre.vo.TimbreDetailsVO;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * 音色的业务层的实现
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Service
+public class TimbreServiceImpl extends BaseServiceImpl<TimbreDao, TimbreEntity> implements TimbreService {
+
+
+    @Override
+    public PageData<TimbreDetailsVO> page(TimbrePageDTO dto) {
+        Map<String, Object> params = new HashMap<String, Object>();
+        params.put(Constant.PAGE, dto.getPage());
+        params.put(Constant.LIMIT,dto.getLimit());
+        IPage<TimbreEntity> page = baseDao.selectPage(
+                getPage(params, "sort", true),
+                //定义查询条件
+                new QueryWrapper<TimbreEntity>()
+                        //必须按照ttsID查找
+                        .eq("tts_model_id",dto.getTtsModelId())
+                        //如果有音色名字，按照音色名模糊查找
+                        .like(StringUtils.isNotBlank(dto.getName()),"name",dto.getName())
+        );
+
+        return getPageData(page, TimbreDetailsVO.class);
+    }
+
+    @Override
+    public TimbreDetailsVO get(Long timbreId) {
+        TimbreEntity entity = baseDao.selectById(timbreId);
+        return ConvertUtils.sourceToTarget(entity, TimbreDetailsVO.class);
+    }
+
+    @Override
+    @Transactional(rollbackFor = Exception.class)
+    public void save(TimbreDataDTO dto) {
+        isTtsModelId(dto.getTtsModelId());
+        TimbreEntity timbreEntity = ConvertUtils.sourceToTarget(dto, TimbreEntity.class);
+        baseDao.insert(timbreEntity);
+    }
+
+    @Override
+    @Transactional(rollbackFor = Exception.class)
+    public void update(Long timbreId, TimbreDataDTO dto) {
+        isTtsModelId(dto.getTtsModelId());
+        TimbreEntity timbreEntity = ConvertUtils.sourceToTarget(dto, TimbreEntity.class);
+        timbreEntity.setId(timbreId);
+        baseDao.updateById(timbreEntity);
+    }
+
+    @Override
+    @Transactional(rollbackFor = Exception.class)
+    public void delete(Long[] ids) {
+        baseDao.deleteBatchIds(Arrays.asList(ids));
+    }
+
+    /**
+     * 处理是不是tts模型的id
+     */
+    private void isTtsModelId(String ttsModelId){
+        //等模型配置那边写好调用方法判断
+    }
+}

main/manager-api/src/main/java/xiaozhi/modules/timbre/vo/TimbreDetailsVO.java

@@ -0,0 +1,39 @@
+package xiaozhi.modules.timbre.vo;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+import lombok.Data;
+
+import java.io.Serializable;
+
+/**
+ * 音色详情展示VO
+ * @author zjy
+ * @since 2025-3-21
+ */
+@Data
+public class TimbreDetailsVO implements Serializable {
+    @Schema(description = "音色id")
+    private Long id;
+
+    @Schema(description = "语言")
+    private String languages;
+
+    @Schema(description = "音色名称")
+    private String name;
+
+    @Schema(description = "备注")
+    private String remark;
+
+    @Schema(description = "排序")
+    private long sort;
+
+    @Schema(description = "对应 TTS 模型主键")
+    private String ttsModelId;
+
+    @Schema(description = "音色编码")
+    private String ttsVoice;
+
+    @Schema(description = "音频播放地址")
+    private String voiceDemo;
+
+}

main/manager-api/src/main/resources/i18n/validation.properties

@@ -20,3 +20,12 @@ sysuser.deptId.require=\u90E8\u95E8\u4E0D\u80FD\u4E3A\u7A7A
 sysuser.status.range=\u72B6\u6001\u53D6\u503C\u8303\u56F40~1
 sysuser.captcha.require=\u9A8C\u8BC1\u7801\u4E0D\u80FD\u4E3A\u7A7A
 sysuser.uuid.require=\u552F\u4E00\u6807\u8BC6\u4E0D\u80FD\u4E3A\u7A7A
+
+
+timbre.languages.require=\u97F3\u8272\u7684\u8BED\u8A00\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.name.require=\u97F3\u8272\u7684\u540D\u79F0\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.remark.require=\u97F3\u8272\u7684\u5907\u6CE8\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.ttsModelId.require=\u97F3\u8272\u7684tts\u4E3B\u952E\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.ttsVoice.require=\u97F3\u8272\u7684\u7F16\u7801\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.voiceDemo.require=\u97F3\u8272\u7684\u64AD\u653EDemo\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+

main/manager-api/src/main/resources/i18n/validation_en_US.properties

@@ -20,3 +20,11 @@ sysuser.deptId.require=Departments cannot be empty
 sysuser.status.range=State ranges from 0 to 1
 sysuser.captcha.require=The captcha cannot be empty
 sysuser.uuid.require=The unique identifier cannot be empty
+
+timbre.languages.require=The language of the timbre cannot be empty
+timbre.name.require=The name of the timbre cannot be empty
+timbre.remark.require=The remark of the timbre cannot be empty
+timbre.ttsModelId.require=The TTS model ID of the timbre cannot be empty
+timbre.ttsVoice.require=The TTS voice of the timbre cannot be empty
+timbre.voiceDemo.require=The voice demo of the timbre cannot be empty
+

main/manager-api/src/main/resources/i18n/validation_zh_CN.properties

@@ -21,3 +21,10 @@ sysuser.status.range=\u72B6\u6001\u53D6\u503C\u8303\u56F40~1
 sysuser.captcha.require=\u9A8C\u8BC1\u7801\u4E0D\u80FD\u4E3A\u7A7A
 sysuser.uuid.require=\u552F\u4E00\u6807\u8BC6\u4E0D\u80FD\u4E3A\u7A7A
 
+timbre.languages.require=\u97F3\u8272\u7684\u8A9E\u8A00\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+timbre.name.require=\u97F3\u8272\u7684\u540D\u7A31\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+timbre.remark.require=\u97F3\u8272\u7684\u5099\u6CE8\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+timbre.ttsModelId.require=\u97F3\u8272\u7684tts\u4E3B\u9375\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+timbre.ttsVoice.require=\u97F3\u8272\u7684\u7DE8\u78BC\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+timbre.voiceDemo.require=\u97F3\u8272\u7684\u64AD\u653E\u5730\u5740\u4E0D\u53EF\u4EE5\u70BA\u7A7A
+

main/manager-api/src/main/resources/i18n/validation_zh_TW.properties

@@ -20,3 +20,10 @@ sysuser.deptId.require=\u90E8\u95E8\u4E0D\u80FD\u4E3A\u7A7A
 sysuser.status.range=\u72B6\u6001\u53D6\u503C\u8303\u56F40~1
 sysuser.captcha.require=\u9A8C\u8BC1\u7801\u4E0D\u80FD\u4E3A\u7A7A
 sysuser.uuid.require=\u552F\u4E00\u6807\u8BC6\u4E0D\u80FD\u4E3A\u7A7A
+
+timbre.languages.require=\u97F3\u8272\u7684\u8BED\u8A00\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.name.require=\u97F3\u8272\u7684\u540D\u79F0\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.remark.require=\u97F3\u8272\u7684\u5907\u6CE8\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.ttsModelId.require=\u97F3\u8272\u7684tts\u4E3B\u952E\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.ttsVoice.require=\u97F3\u8272\u7684\u7F16\u7801\u4E0D\u53EF\u4EE5\u4E3A\u7A7A
+timbre.voiceDemo.require=\u97F3\u8272\u7684\u64AD\u653EDemo\u4E0D\u53EF\u4EE5\u4E3A\u7A7A

main/manager-web/src/apis/module/user.js

@@ -18,20 +18,6 @@ export default {
                 })
             }).send()
     },
-    // 获取用户信息
-    getUserInfo(callback) {
-        RequestService.sendRequest().url(`${getServiceUrl()}/api/v1/user/info`)
-            .method('GET')
-            .success((res) => {
-                RequestService.clearRequestTime()
-                callback(res)
-            })
-            .fail(() => {
-                RequestService.reAjaxFun(() => {
-                    this.getUserInfo()
-                })
-            }).send()
-    },
     // 获取设备信息
     getHomeList(callback) {
         RequestService.sendRequest().url(`${getServiceUrl()}/api/v1/user/device/bind`)
@@ -173,5 +159,52 @@ export default {
                 });
             }).send();
     },
+    // 获取智能体列表
+    getAgentList(callback) {
+        RequestService.sendRequest()
+            .url(`${getServiceUrl()}/api/v1/user/agent`)
+            .method('GET')
+            .success((res) => {
+                RequestService.clearRequestTime();
+                callback(res);
+            })
+            .fail(() => {
+                RequestService.reAjaxFun(() => {
+                    this.getAgentList(callback);
+                });
+            }).send();
+        },
+
+    getUserInfo(callback) {
+        RequestService.sendRequest()
+            .url(`${getServiceUrl()}/api/v1/user/info`)
+            .method('GET')
+            .success((res) => {
+                RequestService.clearRequestTime()
+                callback(res)
+            })
+            .fail((err) => {
+                console.error('接口请求失败:', err)
+                RequestService.reAjaxFun(() => {
+                    this.getUserInfo(callback)
+                })
+            }).send()
+    },
+    // 添加智能体
+    addAgent(agentName, callback) {
+        RequestService.sendRequest()
+            .url(`${getServiceUrl()}/api/v1/user/agent`)
+            .method('POST')
+            .data({ name: agentName })
+            .success((res) => {
+                RequestService.clearRequestTime();
+                callback(res);
+            })
+            .fail(() => {
+                RequestService.reAjaxFun(() => {
+                    this.addAgent(agentName, callback);
+                });
+            }).send();
+    },
 
 }

main/manager-web/src/components/AddDeviceDialog.vue

@@ -1,21 +1,22 @@
 <template>
-  <el-dialog :visible.sync="visible" width="480px" center>
+  <el-dialog :visible.sync="visible" width="400px" center >
-    <div style="margin: 0 20px 20px;display: flex;align-items: center;gap: 10px;font-weight: 700;font-size: 20px;text-align: left;color: #3d4566;">
+    <div style="margin: 0 10px 10px;display: flex;align-items: center;gap: 10px;font-weight: 700;font-size: 20px;text-align: left;color: #3d4566;">
-      <div style="width: 36px;height: 36px;border-radius: 50%;background: #5778ff;display: flex;align-items: center;justify-content: center;">
+      <div style="width: 40px;height: 40px;border-radius: 50%;background: #5778ff;display: flex;align-items: center;justify-content: center;">
-        <img src="@/assets/home/equipment.png" alt="" style="width: 16px;height: 14px;" />
+        <img src="@/assets/home/equipment.png" alt="" style="width: 18px;height: 15px;" />
       </div>
       添加设备
     </div>
     <div style="height: 1px;background: #e8f0ff;" />
-    <div style="margin: 30px 20px;">
+    <div style="margin: 22px 15px;">
       <div style="font-weight: 400;font-size: 14px;text-align: left;color: #3d4566;">
-        <div style="color: red;display: inline-block;">*</div>验证码：
+        <div style="color: red;display: inline-block;">*</div>
+        <span style="font-size: 11px"> 验证码：</span>
       </div>
-      <div class="input-46" style="margin-top: 10px;">
+      <div class="input-46" style="margin-top: 12px;">
         <el-input placeholder="请输入设备播报的6位数验证码.." v-model="deviceCode" />
       </div>
     </div>
-    <div style="display: flex;margin: 0 20px;gap: 10px;">
+    <div style="display: flex;margin: 15px 15px;gap: 7px;">
       <div class="dialog-btn" @click="confirm">
         确定
       </div>
@@ -53,7 +54,6 @@ export default {
 
 <style scoped>
 
-
 .input-46 {
   border: 1px solid #e4e6ef;
   background: #f6f8fb;
@@ -65,11 +65,26 @@ export default {
   flex: 1;
   border-radius: 23px;
   background: #5778ff;
-  height: 46px;
+  height: 40px;
   font-weight: 500;
-  font-size: 14px;
+  font-size: 12px;
   color: #fff;
-  line-height: 46px;
+  line-height: 40px;
   text-align: center;
 }
+
+::v-deep .el-dialog {
+  border-radius: 15px;
+  box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
+}
+::v-deep .el-dialog__headerbtn {
+  display: none;
+}
+::v-deep .el-dialog__body {
+  padding: 4px 6px;
+}
+::v-deep .el-dialog__header{
+  padding: 10px;
+}
+
 </style>

main/manager-web/src/components/AddWisdomBodyDialog.vue

@@ -0,0 +1,97 @@
+<template>
+  <el-dialog :visible.sync="visible" width="400px" center>
+    <div style="margin: 0 10px 10px;display: flex;align-items: center;gap: 10px;font-weight: 700;font-size: 20px;text-align: left;color: #3d4566;">
+      <div style="width: 40px;height: 40px;border-radius: 50%;background: #5778ff;display: flex;align-items: center;justify-content: center;">
+        <img src="@/assets/home/equipment.png" alt="" style="width: 18px;height: 15px;" />
+      </div>
+      添加智慧体
+    </div>
+    <div style="height: 1px;background: #e8f0ff;" />
+    <div style="margin: 22px 15px;">
+      <div style="font-weight: 400;font-size: 14px;text-align: left;color: #3d4566;">
+        <div style="color: red;display: inline-block;">*</div> 智慧体名称：
+      </div>
+      <div class="input-46" style="margin-top: 12px;">
+        <el-input placeholder="请输入智慧体名称.." v-model="wisdomBodyName" />
+      </div>
+    </div>
+    <div style="display: flex;margin: 15px 15px;gap: 7px;">
+      <div class="dialog-btn" @click="confirm">
+        确定
+      </div>
+      <div class="dialog-btn"
+           style="background: #e6ebff;border: 1px solid #adbdff;color: #5778ff;"
+           @click="cancel">
+        取消
+      </div>
+    </div>
+  </el-dialog>
+</template>
+
+<script>
+import userApi from '@/apis/module/user';
+
+
+export default {
+  name: 'AddWisdomBodyDialog',
+  props: {
+    visible: { type: Boolean, required: true }
+  },
+  data() {
+    return { wisdomBodyName: "" }
+  },
+  methods: {
+    confirm() {
+      if (!this.wisdomBodyName.trim()) {
+        this.$message.error('请输入智慧体名称');
+        return;
+      }
+      userApi.addAgent(this.wisdomBodyName, (res) => {
+        this.$message.success('添加成功');
+        this.$emit('confirm', res);
+        this.$emit('update:visible', false);
+        this.wisdomBodyName = "";
+      });
+    },
+    cancel() {
+      this.$emit('update:visible', false)
+      this.wisdomBodyName = ""
+    }
+  }
+}
+</script>
+
+<style scoped>
+
+.input-46 {
+  border: 1px solid #e4e6ef;
+  background: #f6f8fb;
+  border-radius: 15px;
+}
+
+.dialog-btn {
+  cursor: pointer;
+  flex: 1;
+  border-radius: 23px;
+  background: #5778ff;
+  height: 40px;
+  font-weight: 500;
+  font-size: 12px;
+  color: #fff;
+  line-height: 40px;
+  text-align: center;
+}
+::v-deep .el-dialog {
+  border-radius: 15px;
+  box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
+}
+::v-deep .el-dialog__headerbtn {
+  display: none;
+}
+::v-deep .el-dialog__body {
+  padding: 4px 6px;
+}
+::v-deep .el-dialog__header{
+  padding: 10px;
+}
+</style>

main/manager-web/src/components/DeviceItem.vue

@@ -1,17 +1,20 @@
 <template>
   <div class="device-item">
     <div style="display: flex;justify-content: space-between;">
-      <div style="font-weight: 700;font-size: 24px;text-align: left;color: #3d4566;">
+      <div style="font-weight: 700;font-size: 18px;text-align: left;color: #3d4566;">
-        {{ device.mac }}
+         {{ device.agentName }}
       </div>
       <div>
         <img src="@/assets/home/delete.png" alt=""
-             style="width: 24px;height: 24px;margin-right: 10px;" />
+             style="width: 18px;height: 18px;margin-right: 10px;" />
-        <img src="@/assets/home/info.png" alt="" style="width: 24px;height: 24px;" />
+        <img src="@/assets/home/info.png" alt="" style="width: 18px;height: 18px;" />
       </div>
     </div>
     <div class="device-name">
-      设备型号：{{ device.model }}
+      设备型号：{{ device.ttsModelName }}
+    </div>
+    <div class="device-name">
+      音色模型：{{ device.ttsVoiceName }}
     </div>
     <div style="display: flex;gap: 10px;align-items: center;">
       <div class="settings-btn" @click="$emit('configure')">
@@ -23,12 +26,12 @@
       <div class="settings-btn">
         历史对话
       </div>
-      <el-switch v-model="switchValue" inactive-text="OTA升级:" :width="42"
+      <div class="settings-btn"  @click="$emit('deviceManage')">
-                 style="margin-left: auto;" />
+        设备管理
+      </div>
     </div>
     <div class="version-info">
-      <div>最近对话：{{ device.lastConversation }}</div>
+      <div>最近对话：{{ device.lastConnectedAt }}</div>
-      <div>APP版本：{{ device.appVersion }}</div>
     </div>
   </div>
 </template>
@@ -46,28 +49,28 @@ export default {
 </script>
 <style scoped>
 .device-item {
-  width: 455px;
+  width: 342px;
   border-radius: 20px;
   background: #fafcfe;
-  padding: 30px;
+  padding: 22px;
   box-sizing: border-box;
 }
 .device-name {
-  margin: 10px 0 14px;
+  margin: 7px 0 10px;
   font-weight: 400;
-  font-size: 14px;
+  font-size: 11px;
   color: #3d4566;
   text-align: left;
 }
 
 .settings-btn {
   font-weight: 500;
-  font-size: 14px;
+  font-size: 10px;
   color: #5778ff;
   background: #e6ebff;
-  width: 76px;
+  width: 57px;
-  height: 28px;
+  height: 21px;
-  line-height: 28px;
+  line-height: 21px;
   cursor: pointer;
   border-radius: 14px;
 }
@@ -75,9 +78,10 @@ export default {
 .version-info {
   display: flex;
   justify-content: space-between;
-  margin-top: 20px;
+  margin-top: 15px;
-  font-size: 14px;
+  font-size: 10px;
   color: #979db1;
   font-weight: 400;
 }
+
 </style>

main/manager-web/src/components/HeaderBar.vue

@@ -1,49 +1,98 @@
 <template>
   <el-header class="header">
-    <div style="display: flex;justify-content: space-between;">
+    <div style="display: flex;justify-content: space-between;margin-top: 6px; ">
       <div style="display: flex;align-items: center;gap: 10px;">
-        <img src="@/assets/xiaozhi-logo.png" alt="" style="width: 56px;height: 56px;" />
+        <img alt="" src="@/assets/xiaozhi-logo.png" style="width: 42px;height: 42px;"/>
-        <img src="@/assets/xiaozhi-ai.png" alt="" style="width: 78px;height: 16px;" />
+        <img alt="" src="@/assets/xiaozhi-ai.png" style="width: 58px;height: 12px;"/>
         <div class="equipment-management" @click="goHome">
-          <img src="@/assets/home/equipment.png" alt="" style="width: 16px;height: 14px;" />
+          <img alt="" src="@/assets/home/equipment.png" style="width: 12px;height: 10px;"/>
-          设备管理
+          智能体管理
         </div>
         <div class="console">
-          <i class="el-icon-s-grid" style="font-size: 14px;color: #979db1;" />
+          <i class="el-icon-s-grid" style="font-size: 10px;color: #979db1;"/>
           控制台
         </div>
         <div class="equipment-management2">
           设备管理
-          <img src="@/assets/home/close.png" alt="" style="width: 8px;height: 8px;" />
+          <img alt="" src="@/assets/home/close.png" style="width: 6px;height: 6px;"/>
         </div>
       </div>
-      <div style="display: flex;align-items: center;gap: 10px;">
+      <div style="display: flex;align-items: center;gap: 7px; margin-top: 2px;">
         <div class="serach-box">
-          <el-input placeholder="输入名称搜索.." v-model="serach" />
+          <el-input v-model="serach" placeholder="输入名称搜索.." style="border: none; background: transparent;"
-          <img src="@/assets/home/search.png" alt=""
+                    @keyup.enter.native="handleSearch"/>
-               style="width: 16px;height: 16px;margin-right: 15px;cursor: pointer;" />
+          <img alt="" src="@/assets/home/search.png"
-        </div>
+               style="width: 14px;height: 14px;margin-right: 11px;cursor: pointer;" @click="handleSearch"/>
-        <img src="@/assets/home/avatar.png" alt="" style="width: 28px;height: 28px;" />
-        <div class="user-info">
-          158 3632 4642
         </div>
+        <img alt="" src="@/assets/home/avatar.png" style="width: 21px;height: 21px;"/>
+        <el-dropdown trigger="click">
+          <span class="el-dropdown-link">
+             {{ userInfo.username || '加载中...' }}<i class="el-icon-arrow-down el-icon--right"></i>
+          </span>
+          <el-dropdown-menu slot="dropdown">
+            <el-dropdown-item icon="el-icon-plus" @click.native="">个人中心</el-dropdown-item>
+            <el-dropdown-item icon="el-icon-circle-plus" @click.native="">修改密码</el-dropdown-item>
+            <el-dropdown-item icon="el-icon-circle-plus-outline" @click.native="">退出登录</el-dropdown-item>
+          </el-dropdown-menu>
+        </el-dropdown>
       </div>
     </div>
   </el-header>
 </template>
 
 <script>
+import userApi from '@/apis/module/user'
+
+
 export default {
   name: 'HeaderBar',
+  props: ['devices'],  // 接收父组件设备列表
   data() {
-    return { serach: '' }
+    return {
+      serach: '',
+      userInfo: {
+        username: '',
+        mobile: ''
+      }
+    }
+  },
+  mounted() {
+    this.fetchUserInfo()
   },
   methods: {
     goHome() {
       // 跳转到首页
       this.$router.push('/')
+    },
+    // 获取用户信息
+    fetchUserInfo() {
+      userApi.getUserInfo(({data}) => {
+        this.userInfo = data.data
+      })
+    },
+
+    // 处理搜索
+    handleSearch() {
+      const searchValue = this.serach.trim();
+      let filteredDevices;
+
+      if (!searchValue) {
+        // 当搜索内容为空时，显示原始完整列表
+        filteredDevices = this.$parent.originalDevices;
+      } else {
+        // 过滤逻辑
+        filteredDevices = this.devices.filter(device => {
+          return device.agentName.includes(searchValue) ||
+              device.ttsModelName.includes(searchValue) ||
+              device.ttsVoiceName.includes(searchValue);
+        });
+      }
+
+      this.$emit('search-result', filteredDevices);
     }
+
   }
+
 }
 </script>
 
@@ -54,69 +103,93 @@ export default {
 }
 
 .equipment-management {
-  width: 110px;
+  width: 82px;
-  height: 32px;
+  height: 24px;
-  border-radius: 16px;
+  border-radius: 12px;
   background: #5778ff;
   display: flex;
   align-items: center;
   justify-content: center;
-  gap: 10px;
+  gap: 7px;
   font-weight: 500;
   color: #fff;
-  font-size: 14px;
+  font-size: 10px;
 }
 
 .equipment-management2 {
-  width: 116px;
+  width: 87px;
-  height: 30px;
+  height: 22px;
-  border-radius: 15px;
+  border-radius: 11px;
   background: #fff;
   display: flex;
   justify-content: center;
-  font-size: 12px;
+  font-size: 9px;
-  color: #979db1;
   font-weight: 400;
-  gap: 10px;
+  gap: 7px;
   color: #3d4566;
-  margin-left: 20px;
+  margin-left: 2px;
   align-items: center;
 }
 
 .header {
   background: #f6fcfe66;
   border: 1px solid #fff;
+  height: 53px !important;
 }
 
 .serach-box {
   display: flex;
-  width: 306px;
+  width: 220px;
-  height: 40px;
+  height: 30px;
-  border-radius: 20px;
+  border-radius: 15px;
-  background-color: #e2f5f7;
+  background-color: #f6fcfe66;
+  border: 1px solid #e4e6ef;
   align-items: center;
+  padding: 0 7px;
+  box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
+  margin-right: 15px;
 }
 
+.serach-box /deep/ .el-input__inner {
+  border-radius: 15px;
+  height: 100%;
+  width: 100%;
+  border: 0;
+  background: transparent;
+  padding-left: 12px;
+}
+
+
 .user-info {
   font-weight: 600;
-  font-size: 16px;
+  font-size: 12px;
   letter-spacing: -0.02px;
   text-align: left;
   color: #3d4566;
 }
+
 .console {
-  width: 120px;
+  width: 90px;
-  height: 30px;
+  height: 22px;
-  border-radius: 15px;
+  border-radius: 11px;
   background: radial-gradient(50% 50% at 50% 50%, #fff 0%, #e8f0ff 100%);
   display: flex;
   align-items: center;
   justify-content: center;
-  font-size: 12px;
+  font-size: 9px;
   color: #979db1;
   font-weight: 400;
-  gap: 10px;
+  gap: 7px;
-  color: #979db1;
+  margin-left: 15px;
-  margin-left: 20px;
 }
-</style>
+
+.el-dropdown-link {
+  cursor: pointer;
+  color: #5778ff;
+}
+
+.el-icon-arrow-down {
+  font-size: 12px;
+}
+
+</style>

main/manager-web/src/router/index.js

@@ -39,6 +39,15 @@ const routes = [
       return import('../views/register.vue')
     }
   },
+      // 新增设备管理页面路由
+  {
+    path: '/device-management',
+    name: 'DeviceManagement',
+    component: function () {
+      return import('../views/DeviceManagement.vue')
+    }
+
+  }
 ]
 
 const router = new VueRouter({

main/manager-web/src/views/DeviceManagement.vue

@@ -0,0 +1,169 @@
+<template>
+  <div class="welcome">
+        <HeaderBar />
+    <el-main style="padding: 20px; display: flex; flex-direction: column;">
+      <div class="table-container">
+         <h3 class="device-list-title">设备列表</h3>
+        <el-button type="primary" class="add-device-btn" @click="handleAddDevice">
+          + 添加设备
+        </el-button>
+        <el-table :data="deviceList" style="width: 100%; margin-top: 20px" border stripe>
+          <el-table-column label="设备型号" prop="model" flex></el-table-column>
+          <el-table-column label="固件版本" prop="firmwareVersion" width="140"></el-table-column>
+          <el-table-column label="Mac地址" prop="macAddress" width="220"></el-table-column>
+          <el-table-column label="绑定时间" prop="bindTime" width="260"></el-table-column>
+          <el-table-column label="最近对话" prop="lastConversation" width="100"></el-table-column>
+          <el-table-column label="备注" width="220">
+             <template slot-scope="scope">
+              <el-input v-if="scope.row.isEdit" v-model="scope.row.remark" size="mini" @blur="stopEditRemark(scope.$index)"></el-input>
+              <span v-else>
+                <i v-if="!scope.row.remark" class="el-icon-edit" @click="startEditRemark(scope.$index, scope.row)"></i>
+                <span v-else @click="startEditRemark(scope.$index, scope.row)">
+                  {{ scope.row.remark }}
+                </span>
+              </span>
+            </template>
+          </el-table-column>
+          <el-table-column label="OTA升级" width="120">
+            <template slot-scope="scope">
+              <el-switch v-model="scope.row.otaSwitch" size="mini" active-color="#13ce66" inactive-color="#ff4949"></el-switch>
+            </template>
+          </el-table-column>
+          <el-table-column label="操作" width="80">
+            <template slot-scope="scope">
+              <el-button size="mini" type="text" @click="handleUnbind(scope.row)" style="color: #ff4949">
+                解绑
+              </el-button>
+            </template>
+          </el-table-column>
+        </el-table>
+      </div>
+      <div style="font-size: 12px; font-weight: 400; margin-top: auto; padding-top: 30px; color: #979db1;">
+        ©2025 xiaozhi-esp32-server
+      </div>
+      <AddDeviceDialog :visible.sync="addDeviceDialogVisible" @added="handleDeviceAdded" />
+    </el-main>
+  </div>
+</template>
+
+<script>
+import HeaderBar from "@/components/HeaderBar.vue";
+import AddDeviceDialog from "@/components/AddDeviceDialog.vue";
+
+export default {
+  components: {HeaderBar, AddDeviceDialog },
+  data() {
+    return {
+      addDeviceDialogVisible: false,
+      deviceList: [
+        {
+          model: 'xingzhi-cube-0.96oled-wifi',
+          firmwareVersion: '1.4.6',
+          macAddress: 'fc:01:2c:c5:d5:7c',
+          bindTime: '2025-03-10 18:16:21',
+          lastConversation: '6 天前',
+          remark: '',
+          isEdit: false,
+          otaSwitch: false
+        },
+        {
+          model: 'xingzhi-board-1.3tft-ble',
+          firmwareVersion: '2.1.0',
+          macAddress: 'ac:12:3d:e7:f8:9a',
+          bindTime: '2025-03-12 09:30:15',
+          lastConversation: '4 天前',
+          remark: '测试设备',
+          isEdit: false,
+          otaSwitch: true
+      },
+      {
+        model: 'xingzhi-kit-0.91oled-4g',
+        firmwareVersion: '1.8.3',
+        macAddress: 'bc:45:6f:1e:2d:3c',
+        bindTime: '2025-03-15 14:22:08',
+        lastConversation: '2 天前',
+        remark: '生产环境设备',
+        isEdit: false,
+        otaSwitch: false
+      }
+      ]
+    };
+  },
+  methods: {
+    handleAddDevice() {
+      // 添加设备逻辑
+      this.addDeviceDialogVisible = true;
+    },
+    startEditRemark(index, row) {
+      this.deviceList[index].isEdit = true;
+    },
+    stopEditRemark(index) {
+      this.deviceList[index].isEdit = false;
+    },
+    handleUnbind(device) {
+      // 解绑逻辑
+      console.log('解绑设备', device);
+    },
+    handleDeviceAdded(deviceCode) {
+      console.log('添加的智慧体名称：', deviceCode);
+    },
+  }
+};
+</script>
+
+<style scoped>
+.welcome {
+  min-width: 900px;
+  min-height: 506px;
+  height: 100vh;
+  display: flex;
+  flex-direction: column;
+  background-image: url("@/assets/home/background.png");
+  background-size: cover;
+  background-position: center;
+  -webkit-background-size: cover;
+  -o-background-size: cover;
+}
+
+.table-container {
+  background: #f9fafc;
+  padding: 20px;
+  border-radius: 20px;
+  box-shadow: 0 2px 12px rgba(0, 0, 0, 0.1);
+  margin-top: 15px;
+}
+
+.add-device-btn {
+  float: right;
+  background: #409eff;
+  border: none;
+  border-radius: 10px;
+  width: 105px;
+  height: 32px;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  font-size: 14px;
+  gap: 8px;
+  margin-bottom: 15px;
+  &:hover {
+    background: #3a8ee6;
+  }
+}
+
+.device-list-title {
+  float: left;
+  font-size: 18px;
+  font-weight: 700;
+  margin: 5px;
+  color: #2c3e50;
+}
+
+.el-icon-edit {
+  color: #409eff;
+  cursor: pointer;
+  font-size: 14px;
+  vertical-align: middle;
+}
+
+</style>

main/manager-web/src/views/home.vue

@@ -1,7 +1,7 @@
 <template>
   <div class="welcome">
       <!-- 公共头部 -->
-      <HeaderBar />
+      <HeaderBar :devices="devices" @search-result="handleSearchResult" />
       <el-main style="padding: 20px;display: flex;flex-direction: column;">
         <div>
           <!-- 首页内容 -->
@@ -21,23 +21,23 @@
               </div>
               <div class="add-device-btn" @click="showAddDialog">
                 <div class="left-add">
-                  添加设备
+                  添加智能体
                 </div>
                 <div style="width: 23px;height: 13px;background: #5778ff;margin-left: -10px;" />
                 <div class="right-add">
-                  <i class="el-icon-right" style="font-size: 30px;color: #fff;" />
+                  <i class="el-icon-right" style="font-size: 20px;color: #fff;" />
                 </div>
               </div>
             </div>
           </div>
-          <div style="display: flex;flex-wrap: wrap;margin-top: 20px;gap: 20px;justify-content: space-between;box-sizing: border-box;">
+          <div style="display: flex;flex-wrap: wrap;margin-top: 20px;gap: 20px;justify-content: flex-start;box-sizing: border-box;">
-            <DeviceItem v-for="(item,index) in devices" :key="index" :device="item" @configure="goToRoleConfig" />
+            <DeviceItem v-for="(item,index) in devices" :key="index" :device="item" @configure="goToRoleConfig" @deviceManage="handleDeviceManage" />
           </div>
         </div>
         <div style="font-size: 12px;font-weight: 400;margin-top: auto;padding-top: 30px;color: #979db1;">
           ©2025 xiaozhi-esp32-server
         </div>
-        <AddDeviceDialog :visible.sync="addDeviceDialogVisible" @added="handleDeviceAdded" />
+        <AddWisdomBodyDialog :visible.sync="addDeviceDialogVisible" @confirm="handleWisdomBodyAdded" />
       </el-main>
   </div>
 
@@ -45,24 +45,24 @@
 
 <script>
 import DeviceItem from '@/components/DeviceItem.vue'
-import AddDeviceDialog from '@/components/AddDeviceDialog.vue'
+import AddWisdomBodyDialog from '@/components/AddWisdomBodyDialog.vue'
 import HeaderBar from '@/components/HeaderBar.vue'
+
 export default {
   name: 'HomePage',
-  components: { DeviceItem, AddDeviceDialog, HeaderBar },
+  components: { DeviceItem, AddWisdomBodyDialog, HeaderBar },
   data() {
     return {
       addDeviceDialogVisible: false,
-      // 此处模拟设备列表（10条数据）
+      devices: [],
-      devices: Array.from({ length: 10 }, (_, i) => ({
+      originalDevices: [],
-        id: i,
-        mac: 'CC:ba:97:11:a6:ac',
-        model: 'esp32-s3-touch-amoled-1.8',
-        lastConversation: '6天前',
-        appVersion: '1.1.0'
-      }))
     }
   },
+
+  mounted() {
+    this.fetchAgentList();
+  },
+
   methods: {
     showAddDialog() {
       this.addDeviceDialogVisible = true
@@ -71,19 +71,39 @@ export default {
       // 点击配置角色后跳转到角色配置页
       this.$router.push('/role-config')
     },
-    handleDeviceAdded(deviceCode) {
+    handleWisdomBodyAdded(res) {
-      // 根据需要处理添加设备后逻辑，比如刷新设备列表等
+      console.log('新增智能体响应：', res);
-      console.log('设备验证码：', deviceCode)
+      this.fetchAgentList();
+      this.addDeviceDialogVisible = false;
+    },
+    handleDeviceManage() {
+      this.$router.push('/device-management');
+    },
+    // 获取智能体列表
+    fetchAgentList() {
+      import('@/apis/module/user').then(({ default: userApi }) => {
+        userApi.getAgentList(({data}) => {
+          this.originalDevices = data.data;
+          this.devices = data.data;
+        });
+      });
+    },
+    // 搜索更新智能体列表
+    handleSearchResult(filteredList) {
+      this.devices = filteredList; // 更新设备列表
     }
+
   }
 }
 </script>
 
 <style scoped>
 .welcome {
-  min-width: 1200px;
+  min-width: 900px;
-  min-height: 675px;
+  min-height: 506px;
   height: 100vh;
+  display: flex;
+  flex-direction: column;
   background-image: url("@/assets/home/background.png");
   background-size: cover;
   /* 确保背景图像覆盖整个元素 */
@@ -95,8 +115,8 @@ export default {
   /* 兼容老版本Opera浏览器 */
 }
 .add-device {
-  height: 260px;
+  height: 195px;
-  border-radius: 20px;
+  border-radius: 15px;
   position: relative;
   overflow: hidden;
   background: linear-gradient(
@@ -122,48 +142,48 @@ export default {
   box-sizing: border-box;
   /* 兼容老版本Opera浏览器 */
   .hellow-text {
-    margin-left: 100px;
+    margin-left: 75px;
     color: #3d4566;
-    font-size: 44px;
+    font-size: 33px;
     font-weight: 700;
     letter-spacing: 0;
   }
 
   .hi-hint {
     font-weight: 400;
-    font-size: 12px;
+    font-size: 10px;
     text-align: left;
     color: #818cae;
-    margin-left: 100px;
+    margin-left: 75px;
-    margin-top: 7px;
+    margin-top: 5px;
   }
 }
 
 .add-device-btn {
   display: flex;
   align-items: center;
-  margin-left: 100px;
+  margin-left: 75px;
-  margin-top: 20px;
+  margin-top: 15px;
   cursor: pointer;
 
   .left-add {
-    width: 140px;
+    width: 105px;
-    height: 46px;
+    height: 34px;
-    border-radius: 23px;
+    border-radius: 17px;
     background: #5778ff;
     color: #fff;
-    font-size: 14px;
+    font-size: 10px;
     font-weight: 500;
     text-align: center;
-    line-height: 46px;
+    line-height: 34px;
   }
 
   .right-add {
-    width: 46px;
+    width: 34px;
-    height: 46px;
+    height: 34px;
     border-radius: 50%;
     background: #5778ff;
-    margin-left: -8px;
+    margin-left: -6px;
     display: flex;
     justify-content: center;
     align-items: center;

main/manager-web/src/views/login.vue

@@ -4,15 +4,15 @@
       <el-header>
         <div
             style="display: flex;align-items: center;margin-top: 15px;margin-left: 10px;gap: 10px;">
-          <img src="@/assets/xiaozhi-logo.png" alt="" style="width: 45px;height: 45px;"/>
+          <img alt="" src="@/assets/xiaozhi-logo.png" style="width: 45px;height: 45px;"/>
-          <img src="@/assets/xiaozhi-ai.png" alt="" style="width: 70px;height: 13px;"/>
+          <img alt="" src="@/assets/xiaozhi-ai.png" style="width: 70px;height: 13px;"/>
         </div>
       </el-header>
       <el-main style="position: relative;">
         <div class="login-box">
           <div
               style="display: flex;align-items: center;gap: 20px;margin-bottom: 39px;padding: 0 30px;">
-            <img src="@/assets/login/hi.png" alt="" style="width: 34px;height: 34px;"/>
+            <img alt="" src="@/assets/login/hi.png" style="width: 34px;height: 34px;"/>
             <div class="login-text">登录</div>
             <div class="login-welcome">
               WELCOME TO LOGIN
@@ -20,16 +20,16 @@
           </div>
           <div style="padding: 0 30px;">
             <div class="input-box">
-              <img src="@/assets/login/username.png" alt="" class="input-icon"/>
+              <img alt="" class="input-icon" src="@/assets/login/username.png"/>
               <el-input v-model="form.username" placeholder="请输入用户名"/>
             </div>
             <div class="input-box">
-              <img src="@/assets/login/password.png" alt="" class="input-icon"/>
+              <img alt="" class="input-icon" src="@/assets/login/password.png"/>
-              <el-input v-model="form.password" type="password" placeholder="请输入密码"/>
+              <el-input v-model="form.password" placeholder="请输入密码" type="password"/>
             </div>
             <div style="display: flex; align-items: center; margin-top: 20px; width: 100%; gap: 10px;">
               <div class="input-box" style="width: calc(100% - 130px); margin-top: 0;">
-                <img src="@/assets/login/shield.png" alt="" class="input-icon"/>
+                <img alt="" class="input-icon" src="@/assets/login/shield.png"/>
                 <el-input v-model="form.captcha" placeholder="请输入验证码" style="flex: 1;"/>
               </div>
               <img v-if="captchaUrl"
@@ -105,33 +105,43 @@ export default {
       }
     },
 
+    // 封装输入验证逻辑
+    validateInput(input, message) {
+      if (!input.trim()) {
+        showDanger(message);
+        return false;
+      }
+      return true;
+    },
+
     async login() {
-      if (!this.form.username.trim()) {  // 替换isNull校验
+      // 验证用户名
-        showDanger('用户名不能为空')
+      if (!this.validateInput(this.form.username, '用户名不能为空')) {
-        return
+        return;
       }
-      if (!this.form.password.trim()) {  // 替换isNull校验
+      // 验证密码
-        showDanger('密码不能为空')
+      if (!this.validateInput(this.form.password, '密码不能为空')) {
-        return
+        return;
       }
-      if (!this.form.captcha.trim()) {  // 替换isNull校验
+      // 验证验证码
-        showDanger('验证码不能为空')
+      if (!this.validateInput(this.form.captcha, '验证码不能为空')) {
-        return
+        return;
       }
 
       this.form.captchaId = this.captchaUuid
       Api.user.login(this.form, ({data}) => {
         console.log(data)
         showSuccess('登陆成功！')
-        
+        // 将令牌存储到 Vuex 中
         this.$store.commit('setToken', JSON.stringify(data.data))
 
         goToPage('/home')
       })
 
+      // 重新获取验证码
       setTimeout(() => {
-        this.fetchCaptcha()
+        this.fetchCaptcha();
-      }, 1000)
+      }, 1000);
     },
 
     goToRegister() {
@@ -140,6 +150,6 @@ export default {
   }
 }
 </script>
-<style scoped lang="scss">
+<style lang="scss" scoped>
 @import './auth.scss'; // 添加这行引用
 </style>

main/manager-web/src/views/register.vue

@@ -4,8 +4,8 @@
       <!-- 保持相同的头部 -->
       <el-header>
         <div style="display: flex;align-items: center;margin-top: 15px;margin-left: 10px;gap: 10px;">
-          <img src="@/assets/xiaozhi-logo.png" alt="" style="width: 45px;height: 45px;"/>
+          <img alt="" src="@/assets/xiaozhi-logo.png" style="width: 45px;height: 45px;"/>
-          <img src="@/assets/xiaozhi-ai.png" alt="" style="width: 70px;height: 13px;"/>
+          <img alt="" src="@/assets/xiaozhi-ai.png" style="width: 70px;height: 13px;"/>
         </div>
       </el-header>
 
@@ -13,7 +13,7 @@
         <div class="login-box">
           <!-- 修改标题部分 -->
           <div style="display: flex;align-items: center;gap: 20px;margin-bottom: 39px;padding: 0 30px;">
-            <img src="@/assets/login/hi.png" alt="" style="width: 34px;height: 34px;"/>
+            <img alt="" src="@/assets/login/hi.png" style="width: 34px;height: 34px;"/>
             <div class="login-text">注册</div>
             <div class="login-welcome">
               WELCOME TO REGISTER
@@ -23,33 +23,33 @@
           <div style="padding: 0 30px;">
             <!-- 用户名输入框 -->
             <div class="input-box">
-              <img src="@/assets/login/username.png" alt="" class="input-icon"/>
+              <img alt="" class="input-icon" src="@/assets/login/username.png"/>
               <el-input v-model="form.username" placeholder="请输入用户名"/>
             </div>
 
             <!-- 密码输入框 -->
             <div class="input-box">
-              <img src="@/assets/login/password.png" alt="" class="input-icon"/>
+              <img alt="" class="input-icon" src="@/assets/login/password.png"/>
-              <el-input v-model="form.password" type="password" placeholder="请输入密码"/>
+              <el-input v-model="form.password" placeholder="请输入密码" type="password"/>
             </div>
 
             <!-- 新增确认密码 -->
             <div class="input-box">
-              <img src="@/assets/login/password.png" alt="" class="input-icon"/>
+              <img alt="" class="input-icon" src="@/assets/login/password.png"/>
-              <el-input v-model="form.confirmPassword" type="password" placeholder="请确认密码"/>
+              <el-input v-model="form.confirmPassword" placeholder="请确认密码" type="password"/>
             </div>
 
             <!-- 验证码部分保持相同 -->
             <div style="display: flex; align-items: center; margin-top: 20px; width: 100%; gap: 10px;">
               <div class="input-box" style="width: calc(100% - 130px); margin-top: 0;">
-                <img src="@/assets/login/shield.png" alt="" class="input-icon"/>
+                <img alt="" class="input-icon" src="@/assets/login/shield.png"/>
                 <el-input v-model="form.captcha" placeholder="请输入验证码" style="flex: 1;"/>
               </div>
               <img v-if="captchaUrl"
-                  :src="captchaUrl"
+                   :src="captchaUrl"
-                  alt="验证码"
+                   alt="验证码"
-                  style="width: 150px; height: 40px; cursor: pointer;"
+                   style="width: 150px; height: 40px; cursor: pointer;"
-                  @click="fetchCaptcha"
+                   @click="fetchCaptcha"
               />
             </div>
 
@@ -119,24 +119,33 @@ export default {
       });
     },
 
+    // 封装输入验证逻辑
+    validateInput(input, message) {
+      if (!input.trim()) {
+        showDanger(message);
+        return false;
+      }
+      return true;
+    },
     // 注册逻辑
     register() {
-      if (!this.form.username.trim()) {
+      // 验证用户名
-        showDanger('用户名不能为空')
+      if (!this.validateInput(this.form.username, '用户名不能为空')) {
-        return
+        return;
       }
-      if (!this.form.password.trim()) {
+      // 验证密码
-        showDanger('密码不能为空')
+      if (!this.validateInput(this.form.password, '密码不能为空')) {
-        return
+        return;
       }
       if (this.form.password !== this.form.confirmPassword) {
         showDanger('两次输入的密码不一致')
         return
       }
-      if (!this.form.captcha.trim()) {
+      // 验证验证码
-        showDanger('验证码不能为空')
+      if (!this.validateInput(this.form.captcha, '验证码不能为空')) {
-        return
+        return;
       }
+
       Api.user.register(this.form, ({data}) => {
         console.log(data)
         if (data.code === 0) {
@@ -159,6 +168,6 @@ export default {
 }
 </script>
 
-<style scoped lang="scss">
+<style lang="scss" scoped>
 @import './auth.scss'; // 修改为导入新建的SCSS文件
-</style>
+</style>

main/manager-web/src/views/roleConfig.vue

@@ -2,45 +2,33 @@
   <div class="welcome">
     <!-- 公共头部 -->
     <HeaderBar/>
-    <el-main style="padding: 20px;display: flex;flex-direction: column;">
+    <el-main style="padding: 16px;display: flex;flex-direction: column;">
-      <div style="border-radius: 20px;background: #fafcfe;">
+      <div style="border-radius: 16px;background: #fafcfe; border: 1px solid #e8f0ff;">
         <div
-            style="padding: 19px 30px;font-weight: 700;font-size: 24px;text-align: left;color: #3d4566;display: flex;gap: 16px;align-items: center;">
+            style="padding: 15px 24px;font-weight: 700;font-size: 19px;text-align: left;color: #3d4566;display: flex;gap: 13px;align-items: center;">
           <div
-              style="width: 46px;height: 46px;background: #5778ff;border-radius: 50%;display: flex;align-items: center;justify-content: center;">
+              style="width: 37px;height: 37px;background: #5778ff;border-radius: 50%;display: flex;align-items: center;justify-content: center;">
-            <img src="@/assets/home/setting-user.png" alt="" style="width: 24px;height: 24px;"/>
+            <img src="@/assets/home/setting-user.png" alt="" style="width: 19px;height: 19px;"/>
           </div>
           {{ deviceMac }}
         </div>
         <div style="height: 1px;background: #e8f0ff;"/>
-        <el-form ref="form" :model="form" label-width="90px">
+        <el-form ref="form" :model="form" label-width="72px">
-          <div style="padding: 20px 30px;max-width: 990px;">
+          <div style="padding: 16px 24px;max-width: 792px;">
             <el-form-item label="助手昵称：">
-              <div class="input-46">
+              <div class="input-46" style="width: 100%; max-width: 412px;">
                 <el-input v-model="form.name"/>
               </div>
             </el-form-item>
             <el-form-item label="角色模版：">
-              <div style="display: flex;gap: 10px;">
+              <div style="display: flex;gap: 8px;">
-                <div class="template-item">
+                <div v-for="template in templates" :key="template" class="template-item" @click="selectTemplate(template)">
-                  台湾女友
+                  {{ template }}
-                </div>
-                <div class="template-item">
-                  土豆子
-                </div>
-                <div class="template-item">
-                  英语老师
-                </div>
-                <div class="template-item">
-                  好奇小男孩
-                </div>
-                <div class="template-item">
-                  汪汪队队长
                 </div>
               </div>
             </el-form-item>
             <el-form-item label="角色音色：">
-              <div style="display: flex;gap: 10px;align-items: center;">
+              <div style="display: flex;gap: 8px;align-items: center;">
                 <div class="input-46" style="flex:1.4;">
                   <el-select v-model="form.timbre" placeholder="请选择" style="width: 100%;">
                     <el-option v-for="item in options" :key="item.value" :label="item.label"
@@ -56,45 +44,39 @@
             </el-form-item>
             <el-form-item label="角色介绍：">
               <div class="textarea-box">
-                <el-input type="textarea" rows="6" resize="none" placeholder="请输入内容"
+                <el-input type="textarea" rows="5" resize="none" placeholder="请输入内容"
                           v-model="form.introduction" maxlength="2000" show-word-limit/>
               </div>
             </el-form-item>
             <el-form-item label="记忆体：">
               <div class="textarea-box">
-                <el-input type="textarea" rows="6" resize="none" placeholder="请输入内容"
+                <el-input type="textarea" rows="5" resize="none" placeholder="请输入内容"
                           v-model="form.prompt" maxlength="1000"/>
                 <div class="prompt-bottom">
-                  <div style="display: flex;gap: 10px;align-items: center;">
+                  <div style="display: flex;gap: 8px;align-items: center;">
-                    <div style="color: #979db1;font-size: 14px;">当前记忆（每次对话后重新生成）</div>
+                    <div style="color: #979db1;font-size: 11px;">当前记忆（每次对话后重新生成）</div>
                     <div class="clear-btn">
-                      <i class="el-icon-delete-solid" style="font-size: 14px;"/>
+                      <i class="el-icon-delete-solid" style="font-size: 11px;"/>
                       清除
                     </div>
                   </div>
-                  <div style="color: #979db1;font-size:14px;">{{ form.prompt.length }}/1000</div>
+                  <div style="color: #979db1;font-size:11px;">{{ form.prompt.length }}/1000</div>
                 </div>
               </div>
             </el-form-item>
-            <el-form-item label="语言模型（内测）：" class="lh-form-item">
+            <el-form-item v-for="model in models" :key="model.label" :label="model.label" class="model-item">
-              <div style="display: flex;gap: 10px;">
+              <el-select v-model="form.model[model.key]" filterable placeholder="请选择" class="select-field">
-                <div class="input-46" style="width: 100%;">
+                <el-option v-for="item in options" :key="item.value" :label="item.label" :value="item.value"/>
-                  <el-select v-model="form.model" placeholder="请选择" style="width: 100%;">
+              </el-select>
-                    <el-option v-for="item in options" :key="item.value" :label="item.label"
-                               :value="item.value">
-                    </el-option>
-                  </el-select>
-                </div>
-              </div>
             </el-form-item>
-            <el-form-item label="" class="lh-form-item">
+            <el-form-item label="" class="lh-form-item" style="margin-top: -25px;">
               <div style="color: #979db1;text-align: left;">除了“Qwen
                 实时”，其他模型通常会增加约1秒的延迟。改变模型后，建议清空记忆体，以免影响体验。
               </div>
             </el-form-item>
           </div>
         </el-form>
-        <div style="display: flex;padding: 20px;gap: 10px;align-items: center;">
+        <div style="display: flex;padding: 16px;gap: 8px;align-items: center;">
           <div class="save-btn" @click="saveConfig">
             保存配置
           </div>
@@ -102,12 +84,12 @@
             重制
           </div>
           <div class="clear-text">
-            <img src="@/assets/home/red-info.png" alt="" style="width: 24px;height: 24px;"/>
+            <img src="@/assets/home/red-info.png" alt="" style="width: 19px;height: 19px;"/>
             保存配置后，需要重启设备，新的配置才会生效。
           </div>
         </div>
       </div>
-      <div style="font-size: 12px;font-weight: 400;margin-top: auto;padding-top: 30px;color: #979db1;">
+      <div style="font-size: 12px;font-weight: 400;margin-top: auto;padding-top: 24px;color: #979db1;">
         ©2025 xiaozhi-esp32-server
       </div>
     </el-main>
@@ -128,15 +110,29 @@ export default {
         timbre: "",
         introduction: "",
         prompt: "",
-        model: ""
+        model: {
+          tts: "",
+          vad: "",
+          asr: "",
+          llm: "",
+          memory:"",
+          intent:""
+        }
       },
-      options: [{
+    options: [
-        value: '选项1',
+      { value: '选项1', label: '黄金糕' },
-        label: '黄金糕'
+      { value: '选项2', label: '双皮奶' }
-      }, {
+    ],
-        value: '选项2',
+      models: [
-        label: '双皮奶'
+      { label: '大语言模型(LLM)', key: 'llm' },
-      }]
+      { label: '语音转文本模型(ASR)', key: 'asr' },
+      { label: '语音活动检测模型(VAD)', key: 'vad' },
+      { label: '语音生成模型(TTS)', key: 'tts' },
+      { label: '意图分类模型(Intent)', key: 'intent' },
+      { label: '记忆增强模型(Memory)', key: 'memory' }
+    ],
+      templates: ['台湾女友', '土豆子', '英语老师', '好奇小男孩', '汪汪队队长']
+
     }
   },
   methods: {
@@ -161,6 +157,11 @@ export default {
         this.$message.success('配置已重置')
       }).catch(() => {
       })
+    },
+    // 处理选择模板的逻辑
+    selectTemplate(template) {
+      this.form.name = template;
+      this.$message.success(`已选择模板：${template}`);
     }
   }
 }
@@ -168,9 +169,11 @@ export default {
 
 <style scoped>
 .welcome {
-  min-width: 1200px;
+  min-width: 900px;
-  min-height: 675px;
+  min-height: 506px;
   height: 100vh;
+  display: flex;
+  flex-direction: column;
   background-image: url("@/assets/home/background.png");
   background-size: cover;
   /* 确保背景图像覆盖整个元素 */
@@ -181,71 +184,97 @@ export default {
   -o-background-size: cover;
   /* 兼容老版本Opera浏览器 */
 }
+
+.el-form-item ::v-deep .el-form-item__label {
+  font-size: 10px !important;
+  color: #3d4566 !important;
+  font-weight: 400;
+  line-height: 22px;
+  padding-bottom: 2px;
+}
+
+.select-field{
+  width: 100%;
+  max-width: 720px;
+  border: 1px solid #e4e6ef;
+  background: #f6f8fb;
+  border-radius: 8px;
+  height: 36px !important;
+}
+
 .audio-box {
   flex: 1;
-  height: 46px;
+  height: 37px;
-  border-radius: 10px;
+  border-radius: 20px;
   border: 1px solid #e4e6ef;
 }
 
 .clear-btn {
-  width: 60px;
+  width: 48px;
-  height: 24px;
+  height: 19px;
   background: #fd8383;
-  border-radius: 12px;
+  border-radius: 10px;
-  line-height: 24px;
+  line-height: 19px;
-  font-size: 14px;
+  font-size: 11px;
   color: #fff;
   cursor: pointer;
 }
 
 .clear-text {
   color: #979db1;
-  font-size: 14px;
+  font-size: 11px;
   display: flex;
   align-items: center;
-  gap: 10px;
+  gap: 8px;
-  margin-left: 20px;
+  margin-left: 16px;
 }
 
 .template-item {
-  height: 46px;
+  height: 37px;
-  width: 100px;
+  width: 76px;
-  border-radius: 10px;
+  border-radius: 8px;
   background: #e6ebff;
-  line-height: 46px;
+  line-height: 37px;
   font-weight: 400;
-  font-size: 14px;
+  font-size: 11px;
   text-align: center;
   color: #5778ff;
+  cursor: pointer;
+  transition: background-color 0.3s ease;
+}
+
+.template-item:hover {
+  background-color: #d0d8ff;
 }
 
 .prompt-bottom {
-  margin-bottom: 5px;
+  margin-bottom: 4px;
   display: flex;
   justify-content: space-between;
-  padding: 0 20px;
+  padding: 0 16px;
   align-items: center;
 }
 
 .input-46 {
   border: 1px solid #e4e6ef;
   background: #f6f8fb;
-  border-radius: 10px;
+  border-radius: 8px;
+  height: 36px !important;
 }
 
 .save-btn,
 .reset-btn {
-  width: 140px;
+  width: 112px;
-  height: 46px;
+  height: 37px;
-  border-radius: 23px;
+  border-radius: 18px;
-  line-height: 46px;
+  line-height: 37px;
   box-sizing: border-box;
   cursor: pointer;
+  font-size: 11px
 }
 
 .save-btn {
-  border-radius: 23px;
+  border-radius: 18px;
   background: #5778ff;
   color: #fff;
 }
@@ -258,7 +287,8 @@ export default {
 
 .textarea-box {
   border: 1px solid #e4e6ef;
-  border-radius: 10px;
+  border-radius: 8px;
   background: #f6f8fb;
 }
-</style>
+</style>
+

main/manager-web/vue.config.js

@@ -7,6 +7,7 @@ dotenv.config();
 module.exports = defineConfig({
     devServer: {
       // Bug 修复：将代理配置为环境变量中定义的 API 基础 URL
+      port: 8001, // 指定端口为 8001
       proxy: {
         '/xiaozhi-esp32-api': {
           target: process.env.VUE_APP_API_BASE_URL || 'http://localhost:8002', // 后端 API 的基础 URL

main/xiaozhi-server/config.yaml

@@ -35,10 +35,7 @@ log:
   log_file: "server.log"
   # 设置数据文件路径
   data_dir: data
-iot:
+
-  Speaker:
-    # 设置esp32的音量，范围0-100
-    volume: 80
 xiaozhi:
   type: hello
   version: 1
@@ -52,7 +49,7 @@ prompt: |
   你是一个叫小智/小志的台湾女孩，说话机车，声音好听，习惯简短表达，爱用网络梗。
   请注意，要像一个人一样说话，请不要回复表情符号、代码、和xml标签。
   现在我正在和你进行语音聊天，我们开始吧。
-  如果用户希望结束对话，请在最后说“拜拜”或“再见”。
+
 # 使用完声音文件后删除文件(Delete the sound file when you are done using it)
 delete_audio: true
 
@@ -77,11 +74,11 @@ selected_module:
   TTS: EdgeTTS
   # 记忆模块，默认不开启记忆；如果想使用超长记忆，推荐使用mem0ai；如果注重隐私，请使用本地的mem_local_short
   Memory: nomem
-  # 意图识别模块，默认不开启。开启后，可以播放音乐、控制音量、识别退出指令
+  # 意图识别模块，默认使用function_call。开启后，可以播放音乐、控制音量、识别退出指令
   # 意图识别使用intent_llm，优点：通用性强，缺点：增加串行前置意图识别模块，会增加处理时间
   # 意图识别使用function_call，缺点：需要所选择的LLM支持function_call，优点：按需调用工具、速度快
-  # 如果意图识别设置成 function_call，建议把LLM设置成：DoubaoLLM，使用的具体model_name是：doubao-pro-32k-functioncall-241028
+  # 默认免费的ChatGLMLLM就已经支持function_call，但是如果像追求稳定建议把LLM设置成：DoubaoLLM，使用的具体model_name是：doubao-pro-32k-functioncall-241028
-  Intent: nointent
+  Intent: function_call
 
 # 意图识别，是用于理解用户意图的模块，例如：播放音乐
 Intent:
@@ -97,10 +94,40 @@ Intent:
     type: nointent
     # plugins_func/functions下的模块，可以通过配置，选择加载哪个模块，加载后对话支持相应的function调用
     # 系统默认已经记载“handle_exit_intent(退出识别)”、“play_music(音乐播放)”插件，请勿重复加载
-    # 下面是加载查天气、角色切换的插件示例
+    # 下面是加载查天气、角色切换、加载查新闻的插件示例
     functions:
       - change_role
       - get_weather
+      - get_news
+      # play_music是服务器自带的音乐播放，hass_play_music是通过home assistant控制的独立外部程序音乐播放
+      # 如果用了hass_play_music，就不要开启play_music，两者只留一个
+      - play_music
+      #- hass_get_state
+      #- hass_set_state
+      #- hass_play_music
+
+# 插件的基础配置
+plugins:
+  # 获取天气插件的配置，这里填写你的api_key
+  # 这个密钥是项目共用的key，用多了可能会被限制
+  # 想稳定一点就自行申请替换，每天有1000次免费调用
+  # 申请地址：https://console.qweather.com/#/apps/create-key/over
+  get_weather: { "api_key": "a861d0d5e7bf4ee1a83d9a9e4f96d4da", "default_location": "广州" }
+  # 获取新闻插件的配置，这里根据需要的新闻类型传入对应的url链接，默认支持社会、科技、财经新闻
+  # 更多类型的新闻列表查看 https://www.chinanews.com.cn/rss/
+  get_news:
+    default_rss_url: "https://www.chinanews.com.cn/rss/society.xml"
+    category_urls:
+      society: "https://www.chinanews.com.cn/rss/society.xml"
+      world: "https://www.chinanews.com.cn/rss/world.xml"
+      finance: "https://www.chinanews.com.cn/rss/finance.xml"
+  home_assistant:
+    devices:
+      - 客厅,玩具灯,switch.cuco_cn_460494544_cp1_on_p_2_1
+      - 卧室,台灯,switch.iot_cn_831898993_socn1_on_p_2_1
+    base_url: http://homeassistant.local:8123
+    api_key: 你的home assistant api访问令牌
+
 
 Memory:
   mem0ai:
@@ -109,10 +136,10 @@ Memory:
     # 每月有1000次免费调用
     api_key: 你的mem0ai api key
   nomem:
-    # 不想使用记忆功能，可以使用nomem 
+    # 不想使用记忆功能，可以使用nomem
     type: nomem
   mem_local_short:
-    # 本地记忆功能，通过selected_module的llm总结，数据保存在本地，不会上传到服务器 
+    # 本地记忆功能，通过selected_module的llm总结，数据保存在本地，不会上传到服务器
     type: mem_local_short
 
 ASR:
@@ -120,6 +147,10 @@ ASR:
     type: fun_local
     model_dir: models/SenseVoiceSmall
     output_dir: tmp/
+  SherpaASR:
+    type: sherpa_onnx_local
+    model_dir: models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17
+    output_dir: tmp/
   DoubaoASR:
     type: doubao
     appid: 你的火山引擎语音合成服务appid
@@ -133,6 +164,7 @@ VAD:
     min_silence_duration_ms: 700  # 如果说话停顿比较长，可以把这个值设置大一些
 
 LLM:
+  # 所有openai类型均可以修改超参，以AliLLM为例
   # 当前支持的type为openai、dify、ollama，可自行适配
   AliLLM:
     # 定义LLM API类型
@@ -141,6 +173,23 @@ LLM:
     base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
     model_name: qwen-turbo
     api_key: 你的deepseek web key
+    temperature: 0.7  # 温度值
+    max_tokens: 500   # 最大生成token数
+    top_p: 1         
+    top_k: 50        
+    frequency_penalty: 0  # 频率惩罚
+  AliAppLLM:
+    # 定义LLM API类型
+    type: AliBL
+    base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
+    app_id: 你的app_id
+    # 可在这里找到你的 api_key https://bailian.console.aliyun.com/?apiKey=1#/api-key
+    api_key: 你的api_key
+    # 是否不使用本地prompt：true|false (默不用请在百练应用中设置prompt) 
+    is_no_prompt: true
+    # Ali_memory_id：false（不使用）|你的memory_id（请在百练应用中设置中获取）
+    # Tips！：Ali_memory未实现多用户存储记忆(记忆按id调用)
+    ali_memory_id: false
   DoubaoLLM:
     # 定义LLM API类型
     type: openai
@@ -176,8 +225,12 @@ LLM:
     type: dify
     # 建议使用本地部署的dify接口，国内部分区域访问dify公有云接口可能会受限
     # 如果使用DifyLLM，配置文件里prompt(提示词)是无效的，需要在dify控制台设置提示词
-    base_url: https://api.dify.cn/v1
+    base_url: https://api.dify.ai/v1
     api_key: 你的DifyLLM web key
+    # 使用的对话模式 可以选择工作流 workflows/run 对话模式 chat-messages  文本生成 completion-messages
+    # 使用workflows进行返回的时候输入参数为 query 返回参数的名字要设置为 answer
+    # 文本生成的默认输入参数也是query
+    mode: chat-messages
   GeminiLLM:
     type: gemini
     # 谷歌Gemini API，需要先在Google Cloud控制台创建API密钥并获取api_key
@@ -185,7 +238,9 @@ LLM:
     # token申请地址： https://aistudio.google.com/apikey
     # 若部署地无法访问接口，需要开启科学上网
     api_key: 你的gemini web key
-    model_name: "gemini-1.5-pro"  #  gemini-1.5-pro 是免费的
+    model_name: "gemini-2.0-flash"
+    http_proxy: ""  #"http://127.0.0.1:10808"
+    https_proxy: "" #http://127.0.0.1:10808"
   CozeLLM:
     # 定义LLM API类型
     type: coze
@@ -199,12 +254,6 @@ LLM:
     model_name: deepseek-r1-distill-llama-8b@q4_k_m # 使用的模型名称，需要预先在社区下载
     url: http://localhost:1234/v1 # LM Studio服务地址
     api_key: lm-studio # LM Studio服务的固定API Key
-  HomeAssistant:
-    # 定义LLM API类型
-    type: homeassistant
-    base_url: http://homeassistant.local:8123
-    agent_id: conversation.chatgpt
-    api_key: 你的home assistant api访问令牌
   FastgptLLM:
     # 定义LLM API类型
     type: fastgpt
@@ -220,17 +269,18 @@ TTS:
     # 定义TTS API类型
     type: edge
     voice: zh-CN-XiaoxiaoNeural
-    output_file: tmp/
+    output_dir: tmp/
   DoubaoTTS:
     # 定义TTS API类型
     type: doubao
     # 火山引擎语音合成服务，需要先在火山引擎控制台创建应用并获取appid和access_token
     # 山引擎语音一定要购买花钱，起步价30元，就有100并发了。如果用免费的只有2个并发，会经常报tts错误
     # 购买服务后，购买免费的音色后，可能要等半小时左右，才能使用。
-    # 地址：https://console.volcengine.com/speech/service/8
+    # 普通音色在这里开通：https://console.volcengine.com/speech/service/8
+    # 湾湾小何音色在这里开通：https://console.volcengine.com/speech/service/10007，开通后将下面的voice设置成zh_female_wanwanxiaohe_moon_bigtts
     api_url: https://openspeech.bytedance.com/api/v1/tts
     voice: BV001_streaming
-    output_file: tmp/
+    output_dir: tmp/
     authorization: "Bearer;"
     appid: 你的火山引擎语音合成服务appid
     access_token: 你的火山引擎语音合成服务access_token
@@ -241,7 +291,7 @@ TTS:
     # token申请地址 https://cloud.siliconflow.cn/account/ak
     model: FunAudioLLM/CosyVoice2-0.5B
     voice: FunAudioLLM/CosyVoice2-0.5B:alex
-    output_file: tmp/
+    output_dir: tmp/
     access_token: 你的硅基流动API密钥
     response_format: wav
   CozeCnTTS:
@@ -249,7 +299,7 @@ TTS:
     # COZECN TTS
     # token申请地址 https://www.coze.cn/open/oauth/pats
     voice: 7426720361733046281
-    output_file: tmp/
+    output_dir: tmp/
     access_token: 你的coze web key
     response_format: wav
   FishSpeech:
@@ -262,7 +312,7 @@ TTS:
     #--decoder-config-name firefly_gan_vq
     #--compile
     type: fishspeech
-    output_file: tmp/
+    output_dir: tmp/
     response_format: wav
     reference_id: null
     reference_audio: ["/tmp/test.wav",]
@@ -286,7 +336,7 @@ TTS:
     #python api_v2.py -a 127.0.0.1 -p 9880 -c GPT_SoVITS/configs/caixukun.yaml
     type: gpt_sovits_v2
     url: "http://127.0.0.1:9880/tts"
-    output_file: tmp/
+    output_dir: tmp/
     text_lang: "auto"
     ref_audio_path: "caixukun.wav"
     prompt_text: ""
@@ -306,20 +356,24 @@ TTS:
     repetition_penalty: 1.35
     aux_ref_audio_paths: []
   GPT_SOVITS_V3:
+    # 定义TTS API类型 GPT-SoVITS-v3lora-20250228
+    #启动tts方法：
+    #python api.py
     type: gpt_sovits_v3
-    url: "http://127.0.0.1:9880/tts"
+    url: "http://127.0.0.1:9880"
-    output_file: tmp/
+    output_dir: tmp/
-    text_lang: "auto"
+    text_language: "auto"
-    ref_audio_path: "caixukun.wav"
+    refer_wav_path: "caixukun.wav"
-    prompt_lang: "zh"
+    prompt_language: "zh"
     prompt_text: ""
-    top_k: 5
+    top_k: 15
-    top_p: 1
+    top_p: 1.0
-    temperature: 1
+    temperature: 1.0
-    sample_steps: 16
+    cut_punc: ""
-    media_type: "wav"
+    speed: 1.0
-    streaming_mode: false
+    inp_refs: []
-    threshold: 30
+    sample_steps: 32
+    if_sr: false
   MinimaxTTS:
     # Minimax语音合成服务，需要先在minimax平台创建账户充值，并获取登录信息
     # 平台地址：https://platform.minimaxi.com/
@@ -328,7 +382,7 @@ TTS:
     # api_key地址：https://platform.minimaxi.com/user-center/basic-information/interface-key
     # 定义TTS API类型
     type: minimax
-    output_file: tmp/
+    output_dir: tmp/
     group_id: 你的minimax平台groupID
     api_key: 你的minimax平台接口密钥
     model: "speech-01-turbo"
@@ -365,7 +419,7 @@ TTS:
     # token地址：https://nls-portal.console.aliyun.com/overview
     # 定义TTS API类型
     type: aliyun
-    output_file: tmp/
+    output_dir: tmp/
     appkey: 你的阿里云智能语音交互服务项目Appkey
     token: 你的阿里云智能语音交互服务AccessToken，临时的24小时，要长期用下方的access_key_id，access_key_secret
     voice: xiaoyun
@@ -387,9 +441,21 @@ TTS:
     type: doubao
     api_url: https://api.302ai.cn/doubao/tts_hd
     authorization: "Bearer "
+    # 湾湾小何音色
     voice: "zh_female_wanwanxiaohe_moon_bigtts"
-    output_file: tmp/
+    output_dir: tmp/
     access_token: "你的302API密钥"
+  GizwitsTTS:
+    type: doubao
+    # 火山引擎作为基座，可以完全使用企业级火山引擎语音合成服务
+    # 前一万名注册的用户，将送5元体验金额
+    # 获取API Key地址：https://agentrouter.gizwitsapi.com/panel/token
+    api_url: https://bytedance.gizwitsapi.com/api/v1/tts
+    authorization: "Bearer "
+    # 湾湾小何音色
+    voice: "zh_female_wanwanxiaohe_moon_bigtts"
+    output_dir: tmp/
+    access_token: "你的机智云API key"
   ACGNTTS:
     #在线网址：https://acgn.ttson.cn/
     #token购买：www.ttson.cn
@@ -405,7 +471,7 @@ TTS:
     to_lang: ZH
     url: https://u95167-bd74-2aef8085.westx.seetacloud.com:8443/flashsummary/tts?token=
     format: mp3
-    output_file: tmp/
+    output_dir: tmp/
     emotion: 1
   OpenAITTS:
     # openai官方文本转语音服务，可支持全球大多数语种
@@ -419,7 +485,7 @@ TTS:
     voice: onyx
     # 语速范围0.25-4.0
     speed: 1
-    output_file: tmp/
+    output_dir: tmp/
   CustomTTS:
     # 自定义的TTS接口服务，请求参数可自定义
     # 要求接口使用GET方式请求，并返回音频文件
@@ -434,27 +500,10 @@ TTS:
     headers: # 自定义请求头
       # Authorization: Bearer xxxx
     format: wav # 接口返回的音频格式
-    output_file: tmp/
+    output_dir: tmp/
 # 模块测试配置
 module_test:
   test_sentences:  # 自定义测试语句
     - "你好，请介绍一下你自己"
     - "What's the weather like today?"
     - "请用100字概括量子计算的基本原理和应用前景"
-
-# 本地音乐播放配置
-music:
-  music_dir: "./music"  # 音乐文件存放路径，将从该目录及子目录下搜索音乐文件
-  music_ext: # 音乐文件类型，p3格式效率最高
-    - ".mp3"
-    - ".wav"
-    - ".p3"
-  refresh_time: 300 # 刷新音乐列表的时间间隔，单位为秒
-
-# 以下配置在小于等于0.0.9版本中的docker容器中可用
-# 0.0.9以后的新版本源码部署已经无法奏效
-manager:
-  enabled: false
-  ip: 0.0.0.0
-  port: 8002
-use_private_config: false

main/xiaozhi-server/config/settings.py

@@ -7,9 +7,48 @@ from core.utils.util import read_config, get_project_dir
 default_config_file = "config.yaml"
 
 
+def ensure_directories(config):
+    """确保所有配置路径存在"""
+    dirs_to_create = set()
+    project_dir = get_project_dir()  # 获取项目根目录
+    # 日志文件目录
+    log_dir = config.get('log', {}).get('log_dir', 'tmp')
+    dirs_to_create.add(os.path.join(project_dir, log_dir))
+
+    # ASR/TTS模块输出目录
+    for module in ['ASR', 'TTS']:
+        for provider in config.get(module, {}).values():
+            output_dir = provider.get('output_dir', '')
+            if output_dir:
+                dirs_to_create.add(output_dir)
+
+    # 根据selected_module创建模型目录
+    selected_modules = config.get('selected_module', {})
+    for module_type in ['ASR', 'LLM', 'TTS']:
+        selected_provider = selected_modules.get(module_type)
+        if not selected_provider:
+            continue
+        provider_config = config.get(module_type, {}).get(selected_provider, {})
+        output_dir = provider_config.get('output_dir')
+        if output_dir:
+            full_model_dir = os.path.join(project_dir, output_dir)
+            dirs_to_create.add(full_model_dir)
+
+    # 统一创建目录（保留原data目录创建）
+    for dir_path in dirs_to_create:
+        try:
+            os.makedirs(dir_path, exist_ok=True)
+        except PermissionError:
+            print(f"警告：无法创建目录 {dir_path}，请检查写入权限")
+
+
 def get_config_file():
     global default_config_file
-    # 判断是否存在私有的配置文件
+    """获取配置文件路径，优先使用私有配置文件（若存在）。
+
+    Returns:
+       str: 配置文件路径（相对路径或默认路径）
+    """
     config_file = default_config_file
     if os.path.exists(get_project_dir() + "data/." + default_config_file):
         config_file = "data/." + default_config_file
@@ -20,9 +59,13 @@ def load_config():
     """加载配置文件"""
     parser = argparse.ArgumentParser(description="Server configuration")
     config_file = get_config_file()
+
     parser.add_argument("--config_path", type=str, default=config_file)
     args = parser.parse_args()
-    return read_config(args.config_path)
+    config = read_config(args.config_path)
+    # 初始化目录
+    ensure_directories(config)
+    return config
 
 
 def update_config(config):
@@ -67,7 +110,7 @@ def find_missing_keys(new_config, old_config, parent_key=''):
 def check_config_file():
     old_config_file = get_config_file()
     global default_config_file
-    if not old_config_file.startswith('data'):
+    if not 'data' in old_config_file:
         return
     old_config = read_config(get_project_dir() + old_config_file)
     new_config = read_config(get_project_dir() + default_config_file)

main/xiaozhi-server/core/connection.py

@@ -5,13 +5,15 @@ import time
 import queue
 import asyncio
 import traceback
-from config.logger import setup_logging
+
 import threading
 import websockets
 from typing import Dict, Any
+import plugins_func.loadplugins
+from config.logger import setup_logging
 from core.utils.dialogue import Message, Dialogue
 from core.handle.textHandle import handleTextMessage
-from core.utils.util import get_string_no_punctuation_or_emoji, extract_json_from_string
+from core.utils.util import get_string_no_punctuation_or_emoji, extract_json_from_string, get_ip_info
 from concurrent.futures import ThreadPoolExecutor, TimeoutError
 from core.handle.sendAudioHandle import sendAudioMessage
 from core.handle.receiveAudioHandle import handleAudioMessage
@@ -20,7 +22,6 @@ from plugins_func.register import Action
 from config.private_config import PrivateConfig
 from core.auth import AuthMiddleware, AuthenticationError
 from core.utils.auth_code_gen import AuthCodeGenerator
-import plugins_func.loadplugins  
 
 TAG = __name__
 
@@ -30,13 +31,15 @@ class TTSException(RuntimeError):
 
 
 class ConnectionHandler:
-    def __init__(self, config: Dict[str, Any], _vad, _asr, _llm, _tts, _music, _memory, _intent):
+    def __init__(self, config: Dict[str, Any], _vad, _asr, _llm, _tts, _memory, _intent):
         self.config = config
         self.logger = setup_logging()
         self.auth = AuthMiddleware(config)
 
         self.websocket = None
         self.headers = None
+        self.client_ip = None
+        self.client_ip_info = {}
         self.session_id = None
         self.prompt = None
         self.welcome_msg = None
@@ -91,21 +94,18 @@ class ConnectionHandler:
         self.private_config = None
         self.auth_code_gen = AuthCodeGenerator.get_instance()
         self.is_device_verified = False  # 添加设备验证状态标志
-        self.music_handler = _music
         self.close_after_chat = False  # 是否在聊天结束后关闭连接
         self.use_function_call_mode = False
         if self.config["selected_module"]["Intent"] == 'function_call':
             self.use_function_call_mode = True
-        
-        self.func_handler = FunctionHandler(self.config)
 
     async def handle_connection(self, ws):
         try:
             # 获取并验证headers
             self.headers = dict(ws.request.headers)
             # 获取客户端ip地址
-            client_ip = ws.remote_address[0]
+            self.client_ip = ws.remote_address[0]
-            self.logger.bind(tag=TAG).info(f"{client_ip} conn - Headers: {self.headers}")
+            self.logger.bind(tag=TAG).info(f"{self.client_ip} conn - Headers: {self.headers}")
 
             # 进行认证
             await self.auth.authenticate(self.headers)
@@ -149,6 +149,7 @@ class ConnectionHandler:
             self.welcome_msg["session_id"] = self.session_id
             await self.websocket.send(json.dumps(self.welcome_msg))
 
+            # 异步初始化
             await self.loop.run_in_executor(None, self._initialize_components)
 
             # tts 消化线程
@@ -189,8 +190,14 @@ class ConnectionHandler:
         self.prompt = self.config["prompt"]
         if self.private_config:
             self.prompt = self.private_config.private_config.get("prompt", self.prompt)
+
+        self.client_ip_info = get_ip_info(self.client_ip)
+        self.logger.bind(tag=TAG).info(f"Client ip info: {self.client_ip_info}")
+        self.prompt = self.prompt + f"\n我在:{self.client_ip_info}"
         self.dialogue.put(Message(role="system", content=self.prompt))
-    
+
+        self.func_handler = FunctionHandler(self)
+
     def change_system_prompt(self, prompt):
         self.prompt = prompt
         # 找到原来的role==system，替换原来的系统提示
@@ -296,7 +303,7 @@ class ConnectionHandler:
         self.logger.bind(tag=TAG).debug(json.dumps(self.dialogue.get_llm_dialogue(), indent=4, ensure_ascii=False))
         return True
 
-    def chat_with_function_calling(self, query, tool_call = False):
+    def chat_with_function_calling(self, query, tool_call=False):
         self.logger.bind(tag=TAG).debug(f"Chat with function calling start: {query}")
         """Chat with function calling for intent detection using streaming"""
         if self.isNeedAuth():
@@ -304,7 +311,7 @@ class ConnectionHandler:
             future = asyncio.run_coroutine_threadsafe(self._check_and_broadcast_auth_code(), self.loop)
             future.result()
             return True
-        
+
         if not tool_call:
             self.dialogue.put(Message(role="user", content=query))
 
@@ -313,7 +320,7 @@ class ConnectionHandler:
 
         response_message = []
         processed_chars = 0  # 跟踪已处理的字符位置
-   
+
         try:
             start_time = time.time()
 
@@ -321,7 +328,7 @@ class ConnectionHandler:
             future = asyncio.run_coroutine_threadsafe(self.memory.query_memory(query), self.loop)
             memory_str = future.result()
 
-            #self.logger.bind(tag=TAG).info(f"对话记录: {self.dialogue.get_llm_dialogue_with_memory(memory_str)}")
+            # self.logger.bind(tag=TAG).info(f"对话记录: {self.dialogue.get_llm_dialogue_with_memory(memory_str)}")
 
             # 使用支持functions的streaming接口
             llm_responses = self.llm.response_with_functions(
@@ -344,8 +351,8 @@ class ConnectionHandler:
         content_arguments = ""
         for response in llm_responses:
             content, tools_call = response
-            if content is not None and len(content)>0:
+            if content is not None and len(content) > 0:
-                if len(response_message)<=0 and (content=="```" or "<tool_call>" in content):
+                if len(response_message) <= 0 and (content == "```" or "<tool_call>" in content):
                     tool_call_flag = True
 
             if tools_call is not None:
@@ -359,7 +366,7 @@ class ConnectionHandler:
 
             if content is not None and len(content) > 0:
                 if tool_call_flag:
-                    content_arguments+=content
+                    content_arguments += content
                 else:
                     response_message.append(content)
 
@@ -415,16 +422,17 @@ class ConnectionHandler:
                 else:
                     function_arguments = json.loads(function_arguments)
             if not bHasError:
-                self.logger.bind(tag=TAG).info(f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}")
+                self.logger.bind(tag=TAG).info(
+                    f"function_name={function_name}, function_id={function_id}, function_arguments={function_arguments}")
                 function_call_data = {
                     "name": function_name,
                     "id": function_id,
                     "arguments": function_arguments
                 }
                 result = self.func_handler.handle_llm_function_call(self, function_call_data)
-                self._handle_function_result(result, function_call_data, text_index+1)
+                self._handle_function_result(result, function_call_data, text_index + 1)
 
-         # 处理最后剩余的文本
+        # 处理最后剩余的文本
         full_text = "".join(response_message)
         remaining_text = full_text[processed_chars:]
         if remaining_text:
@@ -436,7 +444,7 @@ class ConnectionHandler:
                 self.tts_queue.put(future)
 
         # 存储对话内容
-        if len(response_message)>0:
+        if len(response_message) > 0:
             self.dialogue.put(Message(role="assistant", content="".join(response_message)))
 
         self.llm_finish_task = True
@@ -445,31 +453,40 @@ class ConnectionHandler:
         return True
 
     def _handle_function_result(self, result, function_call_data, text_index):
-        if result.action == Action.RESPONSE: # 直接回复前端
+        if result.action == Action.RESPONSE:  # 直接回复前端
             text = result.response
             self.recode_first_last_text(text, text_index)
             future = self.executor.submit(self.speak_and_play, text, text_index)
             self.tts_queue.put(future)
             self.dialogue.put(Message(role="assistant", content=text))
-        if result.action == Action.REQLLM: # 调用函数后再请求llm生成回复
+        elif result.action == Action.REQLLM:  # 调用函数后再请求llm生成回复
-            
+
             text = result.result
             if text is not None and len(text) > 0:
                 function_id = function_call_data["id"]
                 function_name = function_call_data["name"]
                 function_arguments = function_call_data["arguments"]
                 self.dialogue.put(Message(role='assistant',
-                                            tool_calls=[{"id": function_id, 
+                                          tool_calls=[{"id": function_id,
-                                                        "function": {"arguments": function_arguments,"name": function_name},
+                                                       "function": {"arguments": function_arguments,
-                                                        "type": 'function', 
+                                                                    "name": function_name},
-                                                        "index": 0}]))
+                                                       "type": 'function',
+                                                       "index": 0}]))
 
                 self.dialogue.put(Message(role="tool", tool_call_id=function_id, content=text))
                 self.chat_with_function_calling(text, tool_call=True)
-        if result.action == Action.NOTFOUND:
+        elif result.action == Action.NOTFOUND:
-            text = result.response
+            text = result.result
-
+            self.recode_first_last_text(text, text_index)
-        
+            future = self.executor.submit(self.speak_and_play, text, text_index)
+            self.tts_queue.put(future)
+            self.dialogue.put(Message(role="assistant", content=text))
+        else:
+            text = result.result
+            self.recode_first_last_text(text, text_index)
+            future = self.executor.submit(self.speak_and_play, text, text_index)
+            self.tts_queue.put(future)
+            self.dialogue.put(Message(role="assistant", content=text))
 
     def _tts_priority_thread(self):
         while not self.stop_event.is_set():

main/xiaozhi-server/core/handle/functionHandler.py

@@ -1,16 +1,16 @@
-import asyncio
-from enum import Enum
-
 from config.logger import setup_logging
 import json
 from plugins_func.register import FunctionRegistry, ActionResponse, Action, ToolType
+from plugins_func.functions.hass_init import append_devices_to_prompt
+
 TAG = __name__
 logger = setup_logging()
 
 
 class FunctionHandler:
-    def __init__(self, config):
+    def __init__(self, conn):
-        self.config = config
+        self.conn = conn
+        self.config = conn.config
         self.function_registry = FunctionRegistry()
         self.register_nessary_functions()
         self.register_config_functions()
@@ -26,9 +26,10 @@ class FunctionHandler:
         func_names = ",".join(surport_plugins)
         for function_desc in self.functions_desc:
             if function_desc["function"]["name"] == "plugin_loader":
-                function_desc["function"]["description"] = function_desc["function"]["description"].replace("[plugins]", func_names)
+                function_desc["function"]["description"] = function_desc["function"]["description"].replace("[plugins]",
+                                                                                                            func_names)
                 break
-        
+
     def upload_functions_desc(self):
         self.functions_desc = self.function_registry.get_all_function_desc()
 
@@ -47,15 +48,18 @@ class FunctionHandler:
     def register_nessary_functions(self):
         """注册必要的函数"""
         self.function_registry.register_function("handle_exit_intent")
-        self.function_registry.register_function("play_music")
         self.function_registry.register_function("plugin_loader")
         self.function_registry.register_function("get_time")
+        self.function_registry.register_function("raise_and_lower_the_volume")
 
     def register_config_functions(self):
         """注册配置中的函数,可以不同客户端使用不同的配置"""
         for func in self.config["Intent"]["function_call"].get("functions", []):
             self.function_registry.register_function(func)
 
+        """home assistant需要初始化提示词"""
+        append_devices_to_prompt(self.conn)
+
     def get_function(self, name):
         return self.function_registry.get_function(name)
 
@@ -69,7 +73,7 @@ class FunctionHandler:
             arguments = function_call_data["arguments"]
             arguments = json.loads(arguments) if arguments else {}
             logger.bind(tag=TAG).info(f"调用函数: {function_name}, 参数: {arguments}")
-            if funcItem.type == ToolType.SYSTEM_CTL:
+            if funcItem.type == ToolType.SYSTEM_CTL or funcItem.type == ToolType.IOT_CTL:
                 return func(conn, **arguments)
             elif funcItem.type == ToolType.WAIT:
                 return func(**arguments)
@@ -80,4 +84,4 @@ class FunctionHandler:
         except Exception as e:
             logger.bind(tag=TAG).error(f"处理function call错误: {e}")
 
-        return None
+        return None

main/xiaozhi-server/core/handle/intentHandler.py

@@ -1,5 +1,6 @@
 from config.logger import setup_logging
 import json
+import uuid
 from core.handle.sendAudioHandle import send_stt_message
 from core.utils.util import remove_punctuation_and_length
 
@@ -8,30 +9,16 @@ logger = setup_logging()
 
 
 async def handle_user_intent(conn, text):
-    """
-    Handle user intent before starting chat
-    
-    Args:
-        conn: Connection object
-        text: User's text input
-    
-    Returns:
-        bool: True if intent was handled, False if should proceed to chat
-    """
     # 检查是否有明确的退出命令
     if await check_direct_exit(conn, text):
         return True
-
     if conn.use_function_call_mode:
         # 使用支持function calling的聊天方法,不再进行意图分析
         return False
-
     # 使用LLM进行意图分析
     intent = await analyze_intent_with_llm(conn, text)
-
     if not intent:
         return False
-
     # 处理各种意图
     return await process_intent_result(conn, intent, text)
 
@@ -58,7 +45,6 @@ async def analyze_intent_with_llm(conn, text):
     dialogue = conn.dialogue
     try:
         intent_result = await conn.intent.detect_intent(conn, dialogue.dialogue, text)
-
         # 尝试解析JSON结果
         try:
             intent_data = json.loads(intent_result)
@@ -79,9 +65,6 @@ async def process_intent_result(conn, intent, original_text):
     # 处理退出意图
     if "结束聊天" in intent:
         logger.bind(tag=TAG).info(f"识别到退出意图: {intent}")
-
-        # 如果正在播放音乐，可以关了 TODO
-
         # 如果是明确的离别意图，发送告别语并关闭连接
         await send_stt_message(conn, original_text)
         conn.executor.submit(conn.chat_and_close, original_text)
@@ -90,10 +73,37 @@ async def process_intent_result(conn, intent, original_text):
     # 处理播放音乐意图
     if "播放音乐" in intent:
         logger.bind(tag=TAG).info(f"识别到音乐播放意图: {intent}")
-        await conn.music_handler.handle_music_command(conn, intent)
+        # 调用play_music函数来播放音乐
+        song_name = extract_text_in_brackets(intent)
+        function_id = str(uuid.uuid4().hex)
+        function_name = "play_music"
+        function_arguments = '{ "song_name": "' + song_name + '" }'
+
+        function_call_data = {
+            "name": function_name,
+            "id": function_id,
+            "arguments": function_arguments
+        }
+        conn.func_handler.handle_llm_function_call(conn, function_call_data)
         return True
 
     # 其他意图处理可以在这里扩展
 
     # 默认返回False，表示继续常规聊天流程
     return False
+
+
+def extract_text_in_brackets(s):
+    """
+    从字符串中提取中括号内的文字
+
+    :param s: 输入字符串
+    :return: 中括号内的文字，如果不存在则返回空字符串
+    """
+    left_bracket_index = s.find('[')
+    right_bracket_index = s.find(']')
+
+    if left_bracket_index != -1 and right_bracket_index != -1 and left_bracket_index < right_bracket_index:
+        return s[left_bracket_index + 1:right_bracket_index]
+    else:
+        return ""

main/xiaozhi-server/core/handle/iotHandle.py

@@ -1,24 +1,120 @@
 import json
+import asyncio
 from config.logger import setup_logging
+from plugins_func.register import device_type_registry, register_function, ActionResponse, Action, ToolType
 
 TAG = __name__
 logger = setup_logging()
 
 
+def wrap_async_function(async_func):
+    """包装异步函数为同步函数"""
+
+    def wrapper(*args, **kwargs):
+        try:
+            # 获取连接对象（第一个参数）
+            conn = args[0]
+            if not hasattr(conn, 'loop'):
+                logger.bind(tag=TAG).error("Connection对象没有loop属性")
+                return ActionResponse(Action.ERROR, "Connection对象没有loop属性",
+                                      "执行操作时出错: Connection对象没有loop属性")
+
+            # 使用conn对象中的事件循环
+            loop = conn.loop
+            # 在conn的事件循环中运行异步函数
+            future = asyncio.run_coroutine_threadsafe(async_func(*args, **kwargs), loop)
+            # 等待结果返回
+            return future.result()
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"运行异步函数时出错: {e}")
+            return ActionResponse(Action.ERROR, str(e), f"执行操作时出错: {e}")
+
+    return wrapper
+
+
+def create_iot_function(device_name, method_name, method_info):
+    """
+    根据IOT设备描述生成通用的控制函数
+    """
+
+    async def iot_control_function(conn, response_success=None, response_failure=None, **params):
+        try:
+            # 打印响应参数
+            logger.bind(tag=TAG).info(
+                f"控制函数接收到的响应参数: success='{response_success}', failure='{response_failure}'")
+
+            # 发送控制命令
+            await send_iot_conn(conn, device_name, method_name, params)
+            # 等待一小段时间让状态更新
+            await asyncio.sleep(0.1)
+
+            # 生成结果信息
+            result = f"{device_name}的{method_name}操作执行成功"
+
+
+            # 处理响应中可能的占位符
+            response = response_success
+            # 替换{value}占位符
+            for param_name, param_value in params.items():
+                # 先尝试直接替换参数值
+                if "{" + param_name + "}" in response:
+                    response = response.replace("{" + param_name + "}", str(param_value))
+
+                # 如果有{value}占位符，用相关参数替换
+                if "{value}" in response:
+                    response = response.replace("{value}", str(param_value))
+                    break
+
+            return ActionResponse(Action.RESPONSE, result, response)
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"执行{device_name}的{method_name}操作失败: {e}")
+
+            # 操作失败时使用大模型提供的失败响应
+            response = response_failure
+
+            return ActionResponse(Action.ERROR, str(e), response)
+
+    return wrap_async_function(iot_control_function)
+
+
+def create_iot_query_function(device_name, prop_name, prop_info):
+    """
+    根据IOT设备属性创建查询函数
+    """
+
+    async def iot_query_function(conn, response_success=None, response_failure=None):
+        try:
+            # 打印响应参数
+            logger.bind(tag=TAG).info(
+                f"查询函数接收到的响应参数: success='{response_success}', failure='{response_failure}'")
+
+            value = await get_iot_status(conn, device_name, prop_name)
+
+            # 查询成功，生成结果
+            if value is not None:
+                # 使用大模型提供的成功响应，并替换其中的占位符
+                response = response_success.replace("{value}", str(value))
+
+                return ActionResponse(Action.RESPONSE, str(value), response)
+            else:
+                # 查询失败，使用大模型提供的失败响应
+                response = response_failure
+
+                return ActionResponse(Action.ERROR, f"属性{prop_name}不存在", response)
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"查询{device_name}的{prop_name}时出错: {e}")
+
+            # 查询出错时使用大模型提供的失败响应
+            response = response_failure
+
+            return ActionResponse(Action.ERROR, str(e), response)
+
+    return wrap_async_function(iot_query_function)
+
+
 class IotDescriptor:
     """
     A class to represent an IoT descriptor.
-    Attributes:
-    ----------
-    name : str
-        The name of the IoT descriptor.
-    description : str
-        A brief description of the IoT descriptor.
-    properties : dict
-        A dictionary containing properties of the IoT descriptor.
-    methods : dict
-        A dictionary containing methods of the IoT descriptor.
-    -------
     """
 
     def __init__(self, name, description, properties, methods):
@@ -29,17 +125,7 @@ class IotDescriptor:
 
         # 根据描述创建属性
         for key, value in properties.items():
-            # "volume":{"description":"当前音量 值","type":"number"}
+            property_item = globals()[key] = {}
-            """
-            等价于
-            {
-                'name': 名字,
-                'description': 描述,
-                'value': 0
-            }
-            """
-            # setattr(self, key, {}) # 创建一个空字典, 名字是属性名
-            property_item = globals()[key] = {}  # 创建一个空字典, 名字是属性名
             property_item['name'] = key
             property_item["description"] = value["description"]
             if value["type"] == "number":
@@ -52,23 +138,10 @@ class IotDescriptor:
 
         # 根据描述创建方法
         for key, value in methods.items():
-            # "SetVolume": {"description":"设置音量","parameters":{"volume":{"description":"0到100之间的整数","type":"number"}}}
+            method = globals()[key] = {}
-            """
-            等价于
-            SetVolume = {
-                `description`: 描述,
-                `volume`: {
-                    `description`: 描述,
-                    `value`: 0
-                }
-            }
-            """
-            # setattr(self, key, {}) # 创建一个空字典, 名字是方法名
-            method = globals()[key] = {}  # 创建一个空字典, 名字是方法名
             method["description"] = value["description"]
             method['name'] = key
             for k, v in value["parameters"].items():
-                # 不同的参数解析
                 method[k] = {}
                 method[k]["description"] = v["description"]
                 if v["type"] == "number":
@@ -77,58 +150,136 @@ class IotDescriptor:
                     method[k]["value"] = False
                 else:
                     method[k]["value"] = ""
-
             self.methods.append(method)
 
 
-async def handleIotDescriptors(conn, descriptors):
+def register_device_type(descriptor):
-    """
+    """注册设备类型及其功能"""
-    处理物联网描述
+    device_name = descriptor["name"]
-    示例: [{
+    type_id = device_type_registry.generate_device_type_id(descriptor)
-        "name":"Speaker",
+
-        "description":"当前 AI 机器人的扬声器",
+    # 如果该类型已注册，直接返回类型ID
-        "properties":{
+    if type_id in device_type_registry.type_functions:
-            "volume":{"description":"当前音量 值","type":"number"}  可以有boolean, number, string三种类型
+        return type_id
-        },
+
-        "methods":{
+    functions = {}
-            "SetVolume":{
+
-                "description":"设置音量","parameters":{"volume":{"description":"0到100之间的整数","type":"number"}}
+    # 为每个属性创建查询函数
+    for prop_name, prop_info in descriptor["properties"].items():
+        func_name = f"get_{device_name.lower()}_{prop_name.lower()}"
+        func_desc = {
+            "type": "function",
+            "function": {
+                "name": func_name,
+                "description": f"查询{descriptor['description']}的{prop_info['description']}",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "response_success": {
+                            "type": "string",
+                            "description": f"查询成功时的友好回复，必须使用{{value}}作为占位符表示查询到的值"
+                        },
+                        "response_failure": {
+                            "type": "string",
+                            "description": f"查询失败时的友好回复，例如：'无法获取{device_name}的{prop_info['description']}'"
+                        }
+                    },
+                    "required": ["response_success", "response_failure"]
+                }
             }
         }
-    }]
+        query_func = create_iot_query_function(device_name, prop_name, prop_info)
-    descriptors: 描述列表
+        decorated_func = register_function(func_name, func_desc, ToolType.IOT_CTL)(query_func)
-    """
+        functions[func_name] = decorated_func
+
+    # 为每个方法创建控制函数
+    for method_name, method_info in descriptor["methods"].items():
+        func_name = f"{device_name.lower()}_{method_name.lower()}"
+
+        # 创建参数字典，添加原有参数
+        parameters = {
+            param_name: {
+                "type": param_info["type"],
+                "description": param_info["description"]
+            }
+            for param_name, param_info in method_info["parameters"].items()
+        }
+
+        # 添加响应参数
+        parameters.update({
+            "response_success": {
+                "type": "string",
+                "description": "操作成功时的友好回复,关于该设备的操作结果，设备名称尽量使用description中的名称"
+            },
+            "response_failure": {
+                "type": "string",
+                "description": "操作失败时的友好回复,关于该设备的操作结果，设备名称尽量使用description中的名称"
+            }
+        })
+
+        # 构建必须参数列表（原有参数 + 响应参数）
+        required_params = list(method_info["parameters"].keys())
+        required_params.extend(["response_success", "response_failure"])
+
+        func_desc = {
+            "type": "function",
+            "function": {
+                "name": func_name,
+                "description": f"{descriptor['description']} - {method_info['description']}",
+                "parameters": {
+                    "type": "object",
+                    "properties": parameters,
+                    "required": required_params
+                }
+            }
+        }
+        control_func = create_iot_function(device_name, method_name, method_info)
+        decorated_func = register_function(func_name, func_desc, ToolType.IOT_CTL)(control_func)
+        functions[func_name] = decorated_func
+
+    device_type_registry.register_device_type(type_id, functions)
+    return type_id
+
+
+# 用于接受前端设备推送的搜索iot描述
+async def handleIotDescriptors(conn, descriptors):
+    """处理物联网描述"""
+    functions_changed = False
+
     for descriptor in descriptors:
+        # 创建IOT设备描述符
         iot_descriptor = IotDescriptor(descriptor["name"], descriptor["description"], descriptor["properties"],
                                        descriptor["methods"])
         conn.iot_descriptors[descriptor["name"]] = iot_descriptor
 
-    # 暂时从配置文件中设置音量，后期通过意图识别控制音量
+        if conn.use_function_call_mode:
-    default_iot_volume = 100
+            # 注册或获取设备类型
-    if "iot" in conn.config:
+            type_id = register_device_type(descriptor)
-        default_iot_volume = conn.config["iot"]["Speaker"]["volume"]
+            device_functions = device_type_registry.get_device_functions(type_id)
-    logger.bind(tag=TAG).info(f"服务端设置音量为{default_iot_volume}")
+
-    await send_iot_conn(conn, "Speaker", "SetVolume", {"volume": default_iot_volume})
+            # 在连接级注册设备函数
+            if hasattr(conn, 'func_handler'):
+                for func_name in device_functions:
+                    conn.func_handler.function_registry.register_function(func_name)
+                    logger.bind(tag=TAG).info(f"注册IOT函数到function handler: {func_name}")
+                    functions_changed = True
+
+    # 如果注册了新函数，更新function描述列表
+    if functions_changed and hasattr(conn, 'func_handler'):
+        conn.func_handler.upload_functions_desc()
+        func_names = conn.func_handler.current_support_functions()
+        logger.bind(tag=TAG).info(f"设备类型: {type_id}")
+        logger.bind(tag=TAG).info(f"更新function描述列表完成，当前支持的函数: {func_names}")
+
+
 async def handleIotStatus(conn, states):
-    """
+    """处理物联网状态"""
-    处理物联网状态
-    示例: [{
-        "name":"Speaker",
-        "state":{
-            "volume":100
-        }
-    }]
-    states: 状态列表
-    """
     for state in states:
         for key, value in conn.iot_descriptors.items():
             if key == state["name"]:
                 for property_item in value.properties:
-                    # properties为字典列表, 记录各种属性
                     for k, v in state["state"].items():
-                        # state为字典, 记录各种属性的值, 是需要记录的信息
                         if property_item["name"] == k:
-                            # 检查一下属性是不是相同的
                             if type(v) != type(property_item["value"]):
                                 logger.bind(tag=TAG).error(f"属性{property_item['name']}的值类型不匹配")
                                 break
@@ -138,41 +289,35 @@ async def handleIotStatus(conn, states):
                             break
                 break
 
+
 async def get_iot_status(conn, name, property_name):
-    """
+    """获取物联网状态"""
-    获取物联网状态
-    name: 设备名称 "Speaker"
-    property_name: 属性名称 "volume"
-    返回值: 属性值, 实际的属性有int, bool和str三种类型
-    """
     for key, value in conn.iot_descriptors.items():
         if key == name:
             for property_item in value.properties:
                 if property_item["name"] == property_name:
                     return property_item["value"]
+    logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
     return None
 
-async def send_iot_conn(conn, name, method_name, parameters):
-    """
-    发送物联网指令
-    name: 设备名称 "Speaker"
-    method: 方法 "SetVolume"
-    parameters: 参数, 是一个字典 {"volume": 100}
-    发送示例:
-    {
-        "type": "iot",
-        "commands": [
-            {
-                "name" :  "Speaker",
-                "method": "SetVolume",
-                "parameters": {
-                    "volume": 100
-                    }
-            }
-        ]
-    }
-    """
 
+async def set_iot_status(conn, name, property_name, value):
+    """设置物联网状态"""
+    for key, iot_descriptor in conn.iot_descriptors.items():
+        if key == name:
+            for property_item in iot_descriptor.properties:
+                if property_item["name"] == property_name:
+                    if type(value) != type(property_item["value"]):
+                        logger.bind(tag=TAG).error(f"属性{property_item['name']}的值类型不匹配")
+                        return
+                    property_item["value"] = value
+                    logger.bind(tag=TAG).info(f"物联网状态更新: {name} , {property_name} = {value}")
+                    return
+    logger.bind(tag=TAG).warning(f"未找到设备 {name} 的属性 {property_name}")
+
+
+async def send_iot_conn(conn, name, method_name, parameters):
+    """发送物联网指令"""
     for key, value in conn.iot_descriptors.items():
         if key == name:
             # 找到了设备

main/xiaozhi-server/core/handle/musicHandler.py

@@ -1,139 +0,0 @@
-from config.logger import setup_logging
-import os
-import random
-import difflib
-import re
-import traceback
-from pathlib import Path
-import time
-from core.handle.sendAudioHandle import send_stt_message
-from core.utils import p3
-
-TAG = __name__
-logger = setup_logging()
-
-
-def _extract_song_name(text):
-    """从用户输入中提取歌名"""
-    for keyword in ["播放音乐"]:
-        if keyword in text:
-            parts = text.split(keyword)
-            if len(parts) > 1:
-                return parts[1].strip()
-    return None
-
-
-def _find_best_match(potential_song, music_files):
-    """查找最匹配的歌曲"""
-    best_match = None
-    highest_ratio = 0
-
-    for music_file in music_files:
-        song_name = os.path.splitext(music_file)[0]
-        ratio = difflib.SequenceMatcher(None, potential_song, song_name).ratio()
-        if ratio > highest_ratio and ratio > 0.4:
-            highest_ratio = ratio
-            best_match = music_file
-    return best_match
-
-
-class MusicManager:
-    def __init__(self, music_dir, music_ext):
-        self.music_dir = Path(music_dir)
-        self.music_ext = music_ext
-
-    def get_music_files(self):
-        music_files = []
-        for file in self.music_dir.rglob("*"):
-            # 判断是否是文件
-            if file.is_file():
-                # 获取文件扩展名
-                ext = file.suffix.lower()
-                # 判断扩展名是否在列表中
-                if ext in self.music_ext:
-                    # music_files.append(str(file.resolve()))  # 添加绝对路径
-                    # 添加相对路径
-                    music_files.append(str(file.relative_to(self.music_dir)))
-        return music_files
-
-
-class MusicHandler:
-    def __init__(self, config):
-        self.config = config
-
-        if "music" in self.config:
-            self.music_config = self.config["music"]
-            self.music_dir = os.path.abspath(
-                self.music_config.get("music_dir", "./music")  # 默认路径修改
-            )
-            self.music_ext = self.music_config.get("music_ext", (".mp3", ".wav", ".p3"))
-            self.refresh_time = self.music_config.get("refresh_time", 60)
-        else:
-            self.music_dir = os.path.abspath("./music")
-            self.music_ext = (".mp3", ".wav", ".p3")
-            self.refresh_time = 60
-
-        # 获取音乐文件列表
-        self.music_files = MusicManager(self.music_dir, self.music_ext).get_music_files()
-        self.scan_time = time.time()
-        logger.bind(tag=TAG).debug(f"找到的音乐文件: {self.music_files}")
-
-    async def handle_music_command(self, conn, text):
-        """处理音乐播放指令"""
-        clean_text = re.sub(r'[^\w\s]', '', text).strip()
-        logger.bind(tag=TAG).debug(f"检查是否是音乐命令: {clean_text}")
-
-        # 尝试匹配具体歌名
-        if os.path.exists(self.music_dir):
-            if time.time() - self.scan_time > self.refresh_time:
-                # 刷新音乐文件列表
-                self.music_files = MusicManager(self.music_dir, self.music_ext).get_music_files()
-                self.scan_time = time.time()
-                logger.bind(tag=TAG).debug(f"刷新的音乐文件: {self.music_files}")
-
-            potential_song = _extract_song_name(clean_text)
-            if potential_song:
-                best_match = _find_best_match(potential_song, self.music_files)
-                if best_match:
-                    logger.bind(tag=TAG).info(f"找到最匹配的歌曲: {best_match}")
-                    await self.play_local_music(conn, specific_file=best_match)
-                    return True
-        # 检查是否是通用播放音乐命令
-        await self.play_local_music(conn)
-        return True
-
-    async def play_local_music(self, conn, specific_file=None):
-        """播放本地音乐文件"""
-        try:
-            if not os.path.exists(self.music_dir):
-                logger.bind(tag=TAG).error(f"音乐目录不存在: {self.music_dir}")
-                return
-
-            # 确保路径正确性
-            if specific_file:
-                selected_music = specific_file
-                music_path = os.path.join(self.music_dir, specific_file)
-            else:
-                if not self.music_files:
-                    logger.bind(tag=TAG).error("未找到MP3音乐文件")
-                    return
-                selected_music = random.choice(self.music_files)
-                music_path = os.path.join(self.music_dir, selected_music)
-
-            if not os.path.exists(music_path):
-                logger.bind(tag=TAG).error(f"选定的音乐文件不存在: {music_path}")
-                return
-            text = f"正在播放{selected_music}"
-            await send_stt_message(conn, text)
-            conn.tts_first_text_index = 0
-            conn.tts_last_text_index = 0
-            conn.llm_finish_task = True
-            if music_path.endswith(".p3"):
-                opus_packets, duration = p3.decode_opus_from_file(music_path)
-            else:
-                opus_packets, duration = conn.tts.audio_to_opus_data(music_path)
-            conn.audio_play_queue.put((opus_packets, selected_music, 0))
-
-        except Exception as e:
-            logger.bind(tag=TAG).error(f"播放音乐失败: {str(e)}")
-            logger.bind(tag=TAG).error(f"详细错误: {traceback.format_exc()}")

main/xiaozhi-server/core/handle/textHandle.py

@@ -2,7 +2,7 @@ from config.logger import setup_logging
 import json
 from core.handle.abortHandle import handleAbortMessage
 from core.handle.helloHandle import handleHelloMessage
-from core.handle.receiveAudioHandle import startToChat
+from core.handle.receiveAudioHandle import startToChat, handleAudioMessage
 from core.handle.iotHandle import handleIotDescriptors, handleIotStatus
 
 TAG = __name__
@@ -31,6 +31,8 @@ async def handleTextMessage(conn, message):
             elif msg_json["state"] == "stop":
                 conn.client_have_voice = True
                 conn.client_voice_stop = True
+                if len(conn.asr_audio) > 0:
+                    await handleAudioMessage(conn, b'')
             elif msg_json["state"] == "detect":
                 conn.asr_server_receive = False
                 conn.client_have_voice = False
@@ -41,6 +43,6 @@ async def handleTextMessage(conn, message):
             if "descriptors" in msg_json:
                 await handleIotDescriptors(conn, msg_json["descriptors"])
             if "states" in msg_json:
-                await handleIotStatus(conn, msg_json["states"])  
+                await handleIotStatus(conn, msg_json["states"])
     except json.JSONDecodeError:
         await conn.websocket.send(message)

main/xiaozhi-server/core/providers/asr/sherpa_onnx_local.py

@@ -0,0 +1,164 @@
+import time
+import wave
+import os
+import sys
+import io
+from config.logger import setup_logging
+from typing import Optional, Tuple, List
+import uuid
+import opuslib_next
+from core.providers.asr.base import ASRProviderBase
+
+import numpy as np
+import sherpa_onnx
+
+from modelscope.hub.file_download import model_file_download
+
+TAG = __name__
+logger = setup_logging()
+
+
+# 捕获标准输出
+class CaptureOutput:
+    def __enter__(self):
+        self._output = io.StringIO()
+        self._original_stdout = sys.stdout
+        sys.stdout = self._output
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        sys.stdout = self._original_stdout
+        self.output = self._output.getvalue()
+        self._output.close()
+
+        # 将捕获到的内容通过 logger 输出
+        if self.output:
+            logger.bind(tag=TAG).info(self.output.strip())
+
+
+class ASRProvider(ASRProviderBase):
+    def __init__(self, config: dict, delete_audio_file: bool):
+        self.model_dir = config.get("model_dir")
+        self.output_dir = config.get("output_dir")
+        self.delete_audio_file = delete_audio_file
+
+        # 确保输出目录存在
+        os.makedirs(self.output_dir, exist_ok=True)
+        
+        # 初始化模型文件路径
+        model_files = {
+            "model.int8.onnx": os.path.join(self.model_dir, "model.int8.onnx"),
+            "tokens.txt": os.path.join(self.model_dir, "tokens.txt")
+        }
+
+        # 下载并检查模型文件
+        try:
+            for file_name, file_path in model_files.items():
+                if not os.path.isfile(file_path):
+                    logger.bind(tag=TAG).info(f"正在下载模型文件: {file_name}")
+                    model_file_download(
+                        model_id="pengzhendong/sherpa-onnx-sense-voice-zh-en-ja-ko-yue",
+                        file_path=file_name,
+                        local_dir=self.model_dir
+                    )
+                    
+                    if not os.path.isfile(file_path):
+                        raise FileNotFoundError(f"模型文件下载失败: {file_path}")
+                        
+            self.model_path = model_files["model.int8.onnx"]
+            self.tokens_path = model_files["tokens.txt"]
+            
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"模型文件处理失败: {str(e)}")
+            raise
+
+        with CaptureOutput():
+            self.model = sherpa_onnx.OfflineRecognizer.from_sense_voice(
+                model=self.model_path,
+                tokens=self.tokens_path,
+                num_threads=2,
+                sample_rate=16000,
+                feature_dim=80,
+                decoding_method="greedy_search",
+                debug=False,
+                use_itn=True,
+            )
+
+    def save_audio_to_file(self, opus_data: List[bytes], session_id: str) -> str:
+        """将Opus音频数据解码并保存为WAV文件"""
+        file_name = f"asr_{session_id}_{uuid.uuid4()}.wav"
+        file_path = os.path.join(self.output_dir, file_name)
+
+        decoder = opuslib_next.Decoder(16000, 1)  # 16kHz, 单声道
+        pcm_data = []
+
+        for opus_packet in opus_data:
+            try:
+                pcm_frame = decoder.decode(opus_packet, 960)  # 960 samples = 60ms
+                pcm_data.append(pcm_frame)
+            except opuslib_next.OpusError as e:
+                logger.bind(tag=TAG).error(f"Opus解码错误: {e}", exc_info=True)
+
+        with wave.open(file_path, "wb") as wf:
+            wf.setnchannels(1)
+            wf.setsampwidth(2)  # 2 bytes = 16-bit
+            wf.setframerate(16000)
+            wf.writeframes(b"".join(pcm_data))
+
+        return file_path
+
+    def read_wave(self, wave_filename: str) -> Tuple[np.ndarray, int]:
+        """
+        Args:
+        wave_filename:
+            Path to a wave file. It should be single channel and each sample should
+            be 16-bit. Its sample rate does not need to be 16kHz.
+        Returns:
+        Return a tuple containing:
+        - A 1-D array of dtype np.float32 containing the samples, which are
+        normalized to the range [-1, 1].
+        - sample rate of the wave file
+        """
+
+        with wave.open(wave_filename) as f:
+            assert f.getnchannels() == 1, f.getnchannels()
+            assert f.getsampwidth() == 2, f.getsampwidth()  # it is in bytes
+            num_samples = f.getnframes()
+            samples = f.readframes(num_samples)
+            samples_int16 = np.frombuffer(samples, dtype=np.int16)
+            samples_float32 = samples_int16.astype(np.float32)
+
+            samples_float32 = samples_float32 / 32768
+            return samples_float32, f.getframerate()
+
+    async def speech_to_text(self, opus_data: List[bytes], session_id: str) -> Tuple[Optional[str], Optional[str]]:
+        """语音转文本主处理逻辑"""
+        file_path = None
+        try:
+            # 保存音频文件
+            start_time = time.time()
+            file_path = self.save_audio_to_file(opus_data, session_id)
+            logger.bind(tag=TAG).debug(f"音频文件保存耗时: {time.time() - start_time:.3f}s | 路径: {file_path}")
+
+            # 语音识别
+            start_time = time.time()
+            s = self.model.create_stream()
+            samples, sample_rate = self.read_wave(file_path)
+            s.accept_waveform(sample_rate, samples)
+            self.model.decode_stream(s)
+            text = s.result.text
+            logger.bind(tag=TAG).debug(f"语音识别耗时: {time.time() - start_time:.3f}s | 结果: {text}")
+
+            return text, file_path
+
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"语音识别失败: {e}", exc_info=True)
+            return "", None
+
+        finally:
+            # 文件清理逻辑
+            if self.delete_audio_file and file_path and os.path.exists(file_path):
+                try:
+                    os.remove(file_path)
+                    logger.bind(tag=TAG).debug(f"已删除临时音频文件: {file_path}")
+                except Exception as e:
+                    logger.bind(tag=TAG).error(f"文件删除失败: {file_path} | 错误: {e}")

main/xiaozhi-server/core/providers/intent/intent_llm/intent_llm.py

@@ -1,12 +1,13 @@
 from typing import List, Dict
 from ..base import IntentProviderBase
+from plugins_func.functions.play_music import initialize_music_handler
 from config.logger import setup_logging
-import re 
+import re
+
 TAG = __name__
 logger = setup_logging()
 
 
-
 class IntentProvider(IntentProviderBase):
     def __init__(self, config):
         super().__init__(config)
@@ -73,8 +74,8 @@ class IntentProvider(IntentProviderBase):
             "你现在可以使用的音乐的名称如下(使用<start>和<end>标志):\n"
         )
         return prompt
-    
+
-    async def detect_intent(self, conn, dialogue_history: List[Dict], text:str) -> str:
+    async def detect_intent(self, conn, dialogue_history: List[Dict], text: str) -> str:
         if not self.llm:
             raise ValueError("LLM provider not set")
 
@@ -89,7 +90,9 @@ class IntentProvider(IntentProviderBase):
 
         msgStr += f"User: {text}\n"
         user_prompt = f"当前的对话如下：\n{msgStr}"
-        prompt_music = f"{self.promot}\n<start>{conn.music_handler.music_files}\n<end>"
+        music_config = initialize_music_handler(conn)
+        music_file_names = music_config["music_file_names"]
+        prompt_music = f"{self.promot}\n<start>{music_file_names}\n<end>"
         logger.bind(tag=TAG).debug(f"User prompt: {prompt_music}")
         # 使用LLM进行意图识别
         intent = self.llm.response_no_stream(
@@ -100,10 +103,9 @@ class IntentProvider(IntentProviderBase):
         # 使用正则表达式提取 {} 中的内容
         match = re.search(r'\{.*?\}', intent)
         if match:
-            result = match.group(0)  # 获取匹配到的内容（包含 {}）
+            result = match.group(0)
-            print(result)  # 输出：{intent: '播放音乐 [中秋月]'}
             intent = result
         else:
             intent = "{intent: '继续聊天'}"
         logger.bind(tag=TAG).info(f"Detected intent: {intent}")
-        return intent.strip()
+        return intent.strip()

main/xiaozhi-server/core/providers/llm/AliBL/AliBL.py

@@ -0,0 +1,52 @@
+from config.logger import setup_logging
+from http import HTTPStatus
+from dashscope import Application
+from core.providers.llm.base import LLMProviderBase
+
+TAG = __name__
+logger = setup_logging()
+
+class LLMProvider(LLMProviderBase):
+    def __init__(self, config):
+        self.api_key = config["api_key"]
+        self.app_id = config["app_id"]
+        self.base_url = config.get("base_url")
+        self.is_No_prompt = config.get("is_no_prompt")
+        self.memory_id = config.get("ali_memory_id")
+
+    def response(self, session_id, dialogue):
+        try:
+            # 处理dialogue
+            if self.is_No_prompt:
+                dialogue.pop(0)
+                logger.bind(tag=TAG).debug(f"【阿里百练API服务】处理后的dialogue: {dialogue}")
+
+            # 构造调用参数
+            call_params = {
+                "api_key": self.api_key,
+                "app_id": self.app_id,
+                "session_id": session_id,
+                "messages": dialogue
+            }
+            if self.memory_id != False:
+                # 百练memory需要prompt参数
+                prompt = dialogue[-1].get("content")
+                call_params["memory_id"] = self.memory_id
+                call_params["prompt"] = prompt
+                logger.bind(tag=TAG).debug(f"【阿里百练API服务】处理后的prompt: {prompt}")
+
+            responses = Application.call(**call_params)
+            if responses.status_code != HTTPStatus.OK:
+                logger.bind(tag=TAG).error(
+                    f"code={responses.status_code}, "
+                    f"message={responses.message}, "
+                    f"请参考文档：https://help.aliyun.com/zh/model-studio/developer-reference/error-code"
+                )
+                yield "【阿里百练API服务响应异常】"
+            else:
+                logger.bind(tag=TAG).debug(f"【阿里百练API服务】构造参数: {call_params}")
+                yield responses.output.text
+
+        except Exception as e:
+            logger.bind(tag=TAG).error(f"【阿里百练API服务】响应异常: {e}")
+            yield "【LLM服务响应异常】"

main/xiaozhi-server/core/providers/llm/dify/dify.py

@@ -9,6 +9,7 @@ logger = setup_logging()
 class LLMProvider(LLMProviderBase):
     def __init__(self, config):
         self.api_key = config["api_key"]
+        self.mode = config.get("mode", "chat-messages")
         self.base_url = config.get("base_url", "https://api.dify.ai/v1").rstrip('/')
         self.session_conversation_map = {}  # 存储session_id和conversation_id的映射
 
@@ -19,27 +20,59 @@ class LLMProvider(LLMProviderBase):
             conversation_id = self.session_conversation_map.get(session_id)
 
             # 发起流式请求
-            with requests.post(
+            if self.mode == "chat-messages":
-                    f"{self.base_url}/chat-messages",
+                request_json = {
-                    headers={"Authorization": f"Bearer {self.api_key}"},
-                    json={
                         "query": last_msg["content"],
                         "response_mode": "streaming",
                         "user": session_id,
                         "inputs": {},
                         "conversation_id": conversation_id
-                    },
+                    }
+            elif self.mode == "workflows/run":
+                request_json = {
+                    "inputs": {"query": last_msg["content"]},
+                    "response_mode": "streaming",
+                    "user": session_id
+                }
+            elif self.mode == "completion-messages":
+                request_json = {
+                    "inputs": {"query": last_msg["content"]},
+                    "response_mode": "streaming",
+                    "user": session_id
+                }
+
+            with requests.post(
+                    f"{self.base_url}/{self.mode}",
+                    headers={"Authorization": f"Bearer {self.api_key}"},
+                    json=request_json,
                     stream=True
             ) as r:
-                for line in r.iter_lines():
+                if self.mode == "chat-messages":
-                    if line.startswith(b'data: '):
+                    for line in r.iter_lines():
-                        event = json.loads(line[6:])
+                        if line.startswith(b'data: '):
-                        # 如果没有找到conversation_id，则获取此次conversation_id
+                            event = json.loads(line[6:])
-                        if not conversation_id:
+                            # 如果没有找到conversation_id，则获取此次conversation_id
-                            conversation_id = event.get('conversation_id')
+                            if not conversation_id:
-                            self.session_conversation_map[session_id] = conversation_id  # 更新映射
+                                conversation_id = event.get('conversation_id')
-                        if event.get('answer'):
+                                self.session_conversation_map[session_id] = conversation_id  # 更新映射
-                            yield event['answer']
+                            if event.get('answer'):
+                                yield event['answer']
+                elif self.mode == "workflows/run":
+                    for line in r.iter_lines():
+                        # logger.bind(tag=TAG).info(f"chat message response: {line}")
+                        if line.startswith(b'data: '):
+                            event = json.loads(line[6:])
+                            if event.get('event') == "workflow_finished":
+                                if event['data']['status'] == "succeeded":
+                                    yield event['data']['outputs']['answer']
+                                else:
+                                    yield "【服务响应异常】"
+                elif self.mode == "completion-messages":
+                    for line in r.iter_lines():
+                        if line.startswith(b'data: '):
+                            event = json.loads(line[6:])
+                            if event.get('answer'):
+                                yield event['answer'] 
 
         except Exception as e:
             logger.bind(tag=TAG).error(f"Error in response generation: {e}")

main/xiaozhi-server/core/providers/llm/gemini/gemini.py

@@ -1,14 +1,19 @@
 import google.generativeai as genai
 from core.utils.util import check_model_key
 from core.providers.llm.base import LLMProviderBase
-
+from config.logger import setup_logging
+import requests
+import json
+TAG = __name__
+logger = setup_logging()
 
 class LLMProvider(LLMProviderBase):
     def __init__(self, config):
         """初始化Gemini LLM Provider"""
         self.model_name = config.get("model_name", "gemini-1.5-pro")
         self.api_key = config.get("api_key")
-
+        self.http_proxy=config.get("http_proxy")
+        self.https_proxy = config.get("https_proxy")
         have_key = check_model_key("LLM", self.api_key)
 
         if not have_key:
@@ -16,6 +21,19 @@ class LLMProvider(LLMProviderBase):
 
         try:
             # 初始化Gemini客户端
+            # 配置代理（如果提供了代理配置）
+            self.proxies=None
+            if self.http_proxy is not "" or self.https_proxy is not "":
+
+                self.proxies = {
+                    "http": self.http_proxy,
+                    "https": self.https_proxy,
+                }
+                logger.bind(tag=TAG).info(f"Gemini set proxys:{self.proxies}")
+                # 使用猴子补丁修改 google-generativeai 库的请求会话
+
+                # 使用 session 对象配置 genai
+
             genai.configure(api_key=self.api_key)
             self.model = genai.GenerativeModel(self.model_name)
 
@@ -46,26 +64,54 @@ class LLMProvider(LLMProviderBase):
                 if content:
                     chat_history.append({
                         "role": role,
-                        "parts": [content]
+                        "parts": [{"text":content}]
+
                     })
 
             # 获取当前消息
             current_msg = dialogue[-1]["content"]
 
-            # 创建新的聊天会话
+            # 构建请求体
-            chat = self.model.start_chat(history=chat_history)
+            request_body = {
+                "contents": chat_history + [{"role": "user", "parts": [{"text":current_msg}]}],
+                "generationConfig": self.generation_config
+            }
 
-            # 发送消息并获取流式响应
+            # 构建请求URL
-            response = chat.send_message(
+            url = f"https://generativelanguage.googleapis.com/v1beta/models/{self.model_name}:generateContent?key={self.api_key}"
-                current_msg,
-                stream=True,
-                generation_config=self.generation_config
-            )
 
-            # 处理流式响应
+            # 构建请求头
-            for chunk in response:
+            headers = {
-                if hasattr(chunk, 'text') and chunk.text:
+                "Content-Type": "application/json",
-                    yield chunk.text
+            }
+
+            # 发送POST请求,经测试手动 request 无法使用 stream 模式
+            if self.proxies:
+                response = requests.post(url, headers=headers, json=request_body, stream=False, proxies=self.proxies)
+                try:
+                    data = response.json()  # 直接解析JSON
+                    if 'candidates' in data and data['candidates']:
+                        yield data['candidates'][0]['content']['parts'][0]['text']
+                    else:
+                        yield "未找到候选回复。"
+                except json.JSONDecodeError as e:
+                    yield f"JSON解码错误：{e}"
+                except Exception as e:
+                    yield f"发生错误：{e}"
+            else:
+                logger.bind(tag=TAG).info(f"Gemini stream mode ")
+                chat = self.model.start_chat(history=chat_history)
+
+                # 发送消息并获取流式响应
+                response = chat.send_message(
+                    current_msg,
+                    stream=True,
+                    generation_config=self.generation_config
+                )
+                # 处理流式响应
+                for chunk in response:
+                    if hasattr(chunk, 'text') and chunk.text:
+                        yield chunk.text
 
         except Exception as e:
             error_msg = str(e)
@@ -78,3 +124,13 @@ class LLMProvider(LLMProviderBase):
                 yield "【Gemini API key无效】"
             else:
                 yield f"【Gemini服务响应异常: {error_msg}】"
+
+
+
+
+        except requests.exceptions.RequestException as e:
+            yield f"请求失败：{e}"
+        except json.JSONDecodeError as e:
+            yield f"JSON解码错误：{e}"
+        except Exception as e:
+            yield f"发生错误：{e}"

main/xiaozhi-server/core/providers/llm/homeassistant/homeassistant.py

@@ -1,62 +0,0 @@
-import requests
-from requests.exceptions import RequestException
-from config.logger import setup_logging
-from core.providers.llm.base import LLMProviderBase
-
-TAG = __name__
-logger = setup_logging()
-
-
-class LLMProvider(LLMProviderBase):
-    def __init__(self, config):
-        self.agent_id = config.get("agent_id")  # 对应 agent_id
-        self.api_key = config.get("api_key")
-        self.base_url = config.get("base_url", config.get("url"))  # 默认使用 base_url
-        self.api_url = f"{self.base_url}/api/conversation/process"  # 拼接完整的 API URL
-
-    def response(self, session_id, dialogue):
-        print(dialogue)
-        try:
-            # home assistant语音助手自带意图，无需使用xiaozhi ai自带的，只需要把用户说的话传递给home assistant即可
-
-            # 提取最后一个 role 为 'user' 的 content
-            input_text = None
-            if isinstance(dialogue, list):  # 确保 dialogue 是一个列表
-                # 逆序遍历，找到最后一个 role 为 'user' 的消息
-                for message in reversed(dialogue):
-                    if message.get("role") == "user":  # 找到 role 为 'user' 的消息
-                        input_text = message.get("content", "")
-                        break  # 找到后立即退出循环
-
-            # 构造请求数据
-            payload = {
-                "text": input_text,
-                "agent_id": self.agent_id,
-                "conversation_id": session_id  # 使用 session_id 作为 conversation_id
-            }
-            # 设置请求头
-            headers = {
-                "Authorization": f"Bearer {self.api_key}",
-                "Content-Type": "application/json"
-            }
-
-            # 发起 POST 请求
-            response = requests.post(self.api_url, json=payload, headers=headers)
-
-            # 检查请求是否成功
-            response.raise_for_status()
-
-            # 解析返回数据
-            data = response.json()
-            speech = data.get("response", {}).get("speech", {}).get("plain", {}).get("speech", "")
-
-            # 返回生成的内容
-            if speech:
-                yield speech
-            else:
-                logger.bind(tag=TAG).warning("API 返回数据中没有 speech 内容")
-
-        except RequestException as e:
-            logger.bind(tag=TAG).error(f"HTTP 请求错误: {e}")
-        except Exception as e:
-            logger.bind(tag=TAG).error(f"生成响应时出错: {e}")

main/xiaozhi-server/core/providers/llm/ollama/ollama.py

@@ -12,10 +12,10 @@ class LLMProvider(LLMProviderBase):
         self.model_name = config.get("model_name")
         self.base_url = config.get("base_url", "http://localhost:11434")
         # Initialize OpenAI client with Ollama base URL
-        #如果没有v1，增加v1
+        # 如果没有v1，增加v1
         if not self.base_url.endswith("/v1"):
             self.base_url = f"{self.base_url}/v1"
-            
+
         self.client = OpenAI(
             base_url=self.base_url,
             api_key="ollama"  # Ollama doesn't need an API key but OpenAI client requires one
@@ -28,13 +28,20 @@ class LLMProvider(LLMProviderBase):
                 messages=dialogue,
                 stream=True
             )
-            
+            is_active=True
             for chunk in responses:
                 try:
                     delta = chunk.choices[0].delta if getattr(chunk, 'choices', None) else None
                     content = delta.content if hasattr(delta, 'content') else ''
                     if content:
-                        yield content
+                        if '<think>' in content:
+                            is_active = False
+                            content = content.split('<think>')[0]
+                        if '</think>' in content:
+                            is_active = True
+                            content = content.split('</think>')[-1]
+                        if is_active:
+                            yield content
                 except Exception as e:
                     logger.bind(tag=TAG).error(f"Error processing chunk: {e}")
 
@@ -50,10 +57,10 @@ class LLMProvider(LLMProviderBase):
                 stream=True,
                 tools=functions,
             )
-            
+
             for chunk in stream:
                 yield chunk.choices[0].delta.content, chunk.choices[0].delta.tool_calls
 
         except Exception as e:
             logger.bind(tag=TAG).error(f"Error in Ollama function call: {e}")
-            yield {"type": "content", "content": f"【Ollama服务响应异常: {str(e)}】"}
+            yield {"type": "content", "content": f"【Ollama服务响应异常: {str(e)}】"}

main/xiaozhi-server/core/providers/llm/openai/openai.py

@@ -1,7 +1,11 @@
 import openai
+from config.logger import setup_logging
 from core.utils.util import check_model_key
 from core.providers.llm.base import LLMProviderBase
 
+TAG = __name__
+logger = setup_logging()
+
 
 class LLMProvider(LLMProviderBase):
     def __init__(self, config):
@@ -11,6 +15,8 @@ class LLMProvider(LLMProviderBase):
             self.base_url = config.get("base_url")
         else:
             self.base_url = config.get("url")
+        self.max_tokens = config.get("max_tokens", 500)
+
         check_model_key("LLM", self.api_key)
         self.client = openai.OpenAI(api_key=self.api_key, base_url=self.base_url)
 
@@ -19,9 +25,10 @@ class LLMProvider(LLMProviderBase):
             responses = self.client.chat.completions.create(
                 model=self.model_name,
                 messages=dialogue,
-                stream=True
+                stream=True,
+                max_tokens=self.max_tokens,
             )
-            
+
             is_active = True
             for chunk in responses:
                 try:
@@ -50,12 +57,12 @@ class LLMProvider(LLMProviderBase):
                 model=self.model_name,
                 messages=dialogue,
                 stream=True,
-                tools=functions,
+                tools=functions
             )
-            
+
             for chunk in stream:
                 yield chunk.choices[0].delta.content, chunk.choices[0].delta.tool_calls
-                
+
         except Exception as e:
-            self.logger.bind(tag=TAG).error(f"Error in function call streaming: {e}")
+            logger.bind(tag=TAG).error(f"Error in function call streaming: {e}")
-            yield {"type": "content", "content": f"【OpenAI服务响应异常: {e}】"}
+            yield {"type": "content", "content": f"【OpenAI服务响应异常: {e}】"}

main/xiaozhi-server/core/providers/tts/base.py

@@ -5,6 +5,7 @@ import numpy as np
 import opuslib_next
 from pydub import AudioSegment
 from abc import ABC, abstractmethod
+from core.utils.tts import MarkdownCleaner
 
 TAG = __name__
 logger = setup_logging()
@@ -13,7 +14,7 @@ logger = setup_logging()
 class TTSProviderBase(ABC):
     def __init__(self, config, delete_audio_file):
         self.delete_audio_file = delete_audio_file
-        self.output_file = config.get("output_file")
+        self.output_file = config.get("output_dir")
 
     @abstractmethod
     def generate_filename(self):
@@ -23,6 +24,7 @@ class TTSProviderBase(ABC):
         tmp_file = self.generate_filename()
         try:
             max_repeat_time = 5
+            text = MarkdownCleaner.clean_markdown(text)
             while not os.path.exists(tmp_file) and max_repeat_time > 0:
                 asyncio.run(self.text_to_speak(text, tmp_file))
                 if not os.path.exists(tmp_file):
@@ -47,7 +49,8 @@ class TTSProviderBase(ABC):
         file_type = os.path.splitext(audio_file_path)[1]
         if file_type:
             file_type = file_type.lstrip('.')
-        audio = AudioSegment.from_file(audio_file_path, format=file_type)
+        # 读取音频文件，-nostdin 参数：不要从标准输入读取数据，否则FFmpeg会阻塞
+        audio = AudioSegment.from_file(audio_file_path, format=file_type, parameters=["-nostdin"])
 
         # 转换为单声道/16kHz采样率/16位小端编码（确保与编码器匹配）
         audio = audio.set_channels(1).set_frame_rate(16000).set_sample_width(2)

main/xiaozhi-server/core/providers/tts/custom.py

@@ -15,7 +15,7 @@ class TTSProvider(TTSProviderBase):
         self.headers = config.get("headers", {})
         self.params = config.get("params")
         self.format = config.get("format", "wav")
-        self.output_file = config.get("output_file", "tmp/")
+        self.output_file = config.get("output_dir", "tmp/")
 
     def generate_filename(self):
         return os.path.join(self.output_file, f"tts-{datetime.now().date()}@{uuid.uuid4().hex}.{self.format}")

main/xiaozhi-server/core/providers/tts/gpt_sovits_v3.py

@@ -12,17 +12,18 @@ class TTSProvider(TTSProviderBase):
     def __init__(self, config, delete_audio_file):
         super().__init__(config, delete_audio_file)
         self.url = config.get("url")
-        self.text_lang = config.get("text_lang", "audo")
+        self.refer_wav_path = config.get("refer_wav_path")
-        self.ref_audio_path = config.get("ref_audio_path")
-        self.prompt_lang = config.get("prompt_lang")
         self.prompt_text = config.get("prompt_text")
-        self.top_k = config.get("top_k", 5)
+        self.prompt_language = config.get("prompt_language")
-        self.top_p = config.get("top_p", 1)
+        self.text_language = config.get("text_language", "audo")
-        self.temperature = config.get("temperature", 1)
+        self.top_k = config.get("top_k", 15)
-        self.sample_steps = config.get("sample_steps", 16)
+        self.top_p = config.get("top_p", 1.0)
-        self.media_type = config.get("media_type", "wav")
+        self.temperature = config.get("temperature", 1.0)
-        self.streaming_mode = config.get("streaming_mode", False)
+        self.cut_punc = config.get("cut_punc","")
-        self.threshold = config.get("threshold", 30)
+        self.speed = config.get("speed", 1.0)
+        self.inp_refs = config.get("inp_refs",[])
+        self.sample_steps = config.get("sample_steps",32)
+        self.if_sr = config.get("if_sr",False)
 
 
     def generate_filename(self, extension=".wav"):
@@ -30,18 +31,19 @@ class TTSProvider(TTSProviderBase):
 
     async def text_to_speak(self, text, output_file):
         request_params = {
-            "text": text,
+            "refer_wav_path": self.refer_wav_path,
-            "text_lang": self.text_lang,
-            "ref_audio_path": self.ref_audio_path,
-            "prompt_lang": self.prompt_lang,
             "prompt_text": self.prompt_text,
+            "prompt_language": self.prompt_language,
+            "text": text,
+            "text_language": self.text_language,
             "top_k": self.top_k,
             "top_p": self.top_p,
             "temperature": self.temperature,
+            "cut_punc": self.cut_punc,
+            "speed": self.speed,
+            "inp_refs": self.inp_refs,
             "sample_steps": self.sample_steps,
-            "media_type": self.media_type,
+            "if_sr": self.if_sr,
-            "streaming_mode": self.streaming_mode,
-            "threshold": self.threshold,
         }
 
         resp = requests.get(self.url, params=request_params)

main/xiaozhi-server/core/providers/tts/openai.py

@@ -14,7 +14,7 @@ class TTSProvider(TTSProviderBase):
         self.voice = config.get("voice", "alloy")
         self.response_format = "wav"
         self.speed = config.get("speed", 1.0)
-        self.output_file = config.get("output_file", "tmp/")
+        self.output_file = config.get("output_dir", "tmp/")
         check_model_key("TTS", self.api_key)
 
     def generate_filename(self, extension=".wav"):

main/xiaozhi-server/core/providers/tts/ttson.py

@@ -17,7 +17,7 @@ class TTSProvider(TTSProviderBase):
         self.volume_change_dB = config.get("volume_change_dB", 0)
         self.speed_factor = config.get("speed_factor", 1)
         self.stream = config.get("stream", False)
-        self.output_file = config.get("output_file")
+        self.output_file = config.get("output_dir")
         self.pitch_factor = config.get("pitch_factor", 0)
         self.format = config.get("format", "mp3")
         self.emotion = config.get("emotion", 1)

main/xiaozhi-server/core/utils/dialogue.py

@@ -35,6 +35,15 @@ class Dialogue:
             self.getMessages(m, dialogue)
         return dialogue
 
+    def update_system_message(self, new_content: str):
+        """更新或添加系统消息"""
+        # 查找第一个系统消息
+        system_msg = next((msg for msg in self.dialogue if msg.role == "system"), None)
+        if system_msg:
+            system_msg.content = new_content
+        else:
+            self.put(Message(role="system", content=new_content))
+
     def get_llm_dialogue_with_memory(self, memory_str: str = None) -> List[Dict[str, str]]:
         if memory_str is None or len(memory_str) == 0:
             return self.get_llm_dialogue()

main/xiaozhi-server/core/utils/tts.py

@@ -1,4 +1,5 @@
 import os
+import re
 import sys
 from config.logger import setup_logging
 import importlib
@@ -14,4 +15,98 @@ def create_instance(class_name, *args, **kwargs):
             sys.modules[lib_name] = importlib.import_module(f'{lib_name}')
         return sys.modules[lib_name].TTSProvider(*args, **kwargs)
 
-    raise ValueError(f"不支持的TTS类型: {class_name}，请检查该配置的type是否设置正确")
+    raise ValueError(f"不支持的TTS类型: {class_name}，请检查该配置的type是否设置正确")
+
+
+class MarkdownCleaner:
+    """
+    封装 Markdown 清理逻辑：直接用 MarkdownCleaner.clean_markdown(text) 即可
+    """
+    # 公式字符
+    NORMAL_FORMULA_CHARS = re.compile(r'[a-zA-Z\\^_{}\+\-\(\)\[\]=]')
+
+    @staticmethod
+    def _replace_inline_dollar(m: re.Match) -> str:
+        """
+        只要捕获到完整的 "$...$":
+          - 如果内部有典型公式字符 => 去掉两侧 $
+          - 否则 (纯数字/货币等) => 保留 "$...$"
+        """
+        content = m.group(1)
+        if MarkdownCleaner.NORMAL_FORMULA_CHARS.search(content):
+            return content
+        else:
+            return m.group(0)
+
+    @staticmethod
+    def _replace_table_block(match: re.Match) -> str:
+        """
+        当匹配到一个整段表格块时，回调该函数。
+        """
+        block_text = match.group('table_block')
+        lines = block_text.strip('\n').split('\n')
+
+        parsed_table = []
+        for line in lines:
+            line_stripped = line.strip()
+            if re.match(r'^\|\s*[-:]+\s*(\|\s*[-:]+\s*)+\|?$', line_stripped):
+                continue
+            columns = [col.strip() for col in line_stripped.split('|') if col.strip() != '']
+            if columns:
+                parsed_table.append(columns)
+
+        if not parsed_table:
+            return ""
+
+        headers = parsed_table[0]
+        data_rows = parsed_table[1:] if len(parsed_table) > 1 else []
+
+        lines_for_tts = []
+        if len(parsed_table) == 1:
+            # 只有一行
+            only_line_str = ", ".join(parsed_table[0])
+            lines_for_tts.append(f"单行表格：{only_line_str}")
+        else:
+            lines_for_tts.append(f"表头是：{', '.join(headers)}")
+            for i, row in enumerate(data_rows, start=1):
+                row_str_list = []
+                for col_index, cell_val in enumerate(row):
+                    if col_index < len(headers):
+                        row_str_list.append(f"{headers[col_index]} = {cell_val}")
+                    else:
+                        row_str_list.append(cell_val)
+                lines_for_tts.append(f"第 {i} 行：{', '.join(row_str_list)}")
+
+        return "\n".join(lines_for_tts) + "\n"
+
+    # 预编译所有正则表达式（按执行频率排序）
+    # 这里要把 replace_xxx 的静态方法放在最前定义，以便在列表里能正确引用它们。
+    REGEXES = [
+        (re.compile(r'```.*?```', re.DOTALL), ''),  # 代码块
+        (re.compile(r'^#+\s*', re.MULTILINE), ''),  # 标题
+        (re.compile(r'(\*\*|__)(.*?)\1'), r'\2'),  # 粗体
+        (re.compile(r'(\*|_)(?=\S)(.*?)(?<=\S)\1'), r'\2'),  # 斜体
+        (re.compile(r'!\[.*?\]\(.*?\)'), ''),  # 图片
+        (re.compile(r'\[(.*?)\]\(.*?\)'), r'\1'),  # 链接
+        (re.compile(r'^\s*>+\s*', re.MULTILINE), ''),  # 引用
+        (
+            re.compile(r'(?P<table_block>(?:^[^\n]*\|[^\n]*\n)+)', re.MULTILINE),
+            _replace_table_block
+        ),
+        (re.compile(r'^\s*[*+-]\s*', re.MULTILINE), '- '),  # 列表
+        (re.compile(r'\$\$.*?\$\$', re.DOTALL), ''),  # 块级公式
+        (
+            re.compile(r'(?<![A-Za-z0-9])\$([^\n$]+)\$(?![A-Za-z0-9])'),
+            _replace_inline_dollar
+        ),
+        (re.compile(r'\n{2,}'), '\n'),  # 多余空行
+    ]
+
+    @staticmethod
+    def clean_markdown(text: str) -> str:
+        """
+        主入口方法：依序执行所有正则，移除或替换 Markdown 元素
+        """
+        for regex, replacement in MarkdownCleaner.REGEXES:
+            text = regex.sub(replacement, text)
+        return text.strip()

main/xiaozhi-server/core/utils/util.py

@@ -1,11 +1,11 @@
 import os
 import json
-from datetime import datetime
 import yaml
 import socket
 import subprocess
 import logging
 import re
+import requests
 
 
 def get_project_dir():
@@ -24,6 +24,64 @@ def get_local_ip():
     except Exception as e:
         return "127.0.0.1"
 
+def is_private_ip(ip_addr):
+    """
+    Check if an IP address is a private IP address (compatible with IPv4 and IPv6).
+
+    @param {string} ip_addr - The IP address to check.
+    @return {bool} True if the IP address is private, False otherwise.
+    """
+    try:
+        # Validate IPv4 or IPv6 address format
+        if not re.match(r"^(\d{1,3}\.){3}\d{1,3}$|^([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}$", ip_addr):
+            return False  # Invalid IP address format
+
+        # IPv4 private address ranges
+        if '.' in ip_addr:  # IPv4 address
+            ip_parts = list(map(int, ip_addr.split('.')))
+            if ip_parts[0] == 10:
+                return True  # 10.0.0.0/8 range
+            elif ip_parts[0] == 172 and 16 <= ip_parts[1] <= 31:
+                return True  # 172.16.0.0/12 range
+            elif ip_parts[0] == 192 and ip_parts[1] == 168:
+                return True  # 192.168.0.0/16 range
+            elif ip_addr == '127.0.0.1':
+                return True  # Loopback address
+            elif ip_parts[0] == 169 and ip_parts[1] == 254:
+                return True # Link-local address 169.254.0.0/16
+            else:
+                return False  # Not a private IPv4 address
+        else:  # IPv6 address
+            ip_addr = ip_addr.lower()
+            if ip_addr.startswith('fc00:') or ip_addr.startswith('fd00:'):
+                return True  # Unique Local Addresses (FC00::/7)
+            elif ip_addr == '::1':
+                return True  # Loopback address
+            elif ip_addr.startswith('fe80:'):
+                return True # Link-local unicast addresses (FE80::/10)
+            else:
+                return False  # Not a private IPv6 address
+
+    except (ValueError, IndexError):
+        return False  # IP address format error or insufficient segments
+
+def get_ip_info(ip_addr):
+    try:
+        base_url = "https://freeipapi.com/api/json"
+        url = base_url if is_private_ip(ip_addr) else f"{base_url}/{ip_addr}"
+
+        resp = requests.get(url).json()
+
+        ip_info = {
+            "city": resp.get("cityName"),
+            "region": resp.get("regionName"), 
+            "country": resp.get("countryName")
+        }
+        return ip_info
+    except Exception as e:
+        logging.error(f"Error getting client ip info: {e}")
+        return {}
+
 
 def read_config(config_path):
     with open(config_path, "r", encoding="utf-8") as file:

main/xiaozhi-server/core/websocket_server.py

@@ -2,7 +2,6 @@ import asyncio
 import websockets
 from config.logger import setup_logging
 from core.connection import ConnectionHandler
-from core.handle.musicHandler import MusicHandler
 from core.utils.util import get_local_ip
 from core.utils import asr, vad, llm, tts, memory, intent
 
@@ -13,7 +12,7 @@ class WebSocketServer:
     def __init__(self, config: dict):
         self.config = config
         self.logger = setup_logging()
-        self._vad, self._asr, self._llm, self._tts, self._music, self._memory, self.intent = self._create_processing_instances()
+        self._vad, self._asr, self._llm, self._tts, self._memory, self.intent = self._create_processing_instances()
         self.active_connections = set()  # 添加全局连接记录
 
     def _create_processing_instances(self):
@@ -50,7 +49,6 @@ class WebSocketServer:
                 self.config["TTS"][self.config["selected_module"]["TTS"]],
                 self.config["delete_audio"]
             ),
-            MusicHandler(self.config),
             memory.create_instance(memory_cls_name, memory_cfg),
             intent.create_instance(
                 self.config["selected_module"]["Intent"]
@@ -80,7 +78,7 @@ class WebSocketServer:
     async def _handle_connection(self, websocket):
         """处理新连接，每次创建独立的ConnectionHandler"""
         # 创建ConnectionHandler时传入当前server实例
-        handler = ConnectionHandler(self.config, self._vad, self._asr, self._llm, self._tts, self._music, self._memory, self.intent)
+        handler = ConnectionHandler(self.config, self._vad, self._asr, self._llm, self._tts, self._memory, self.intent)
         self.active_connections.add(handler)
         try:
             await handler.handle_connection(websocket)

main/xiaozhi-server/docker-compose.yml

@@ -21,6 +21,7 @@ services:
       - ./data:/opt/xiaozhi-esp32-server/data
       # 模型文件挂接，很重要
       - ./models/SenseVoiceSmall/model.pt:/opt/xiaozhi-esp32-server/models/SenseVoiceSmall/model.pt
+
 #  #智控台还没开发好，还不能完全使用，会报很多错误，如果是非技术人员，请不要启用智控台服务
 #  xiaozhi-esp32-server-web:
 #    image: ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:web_latest

main/xiaozhi-server/plugins_func/functions/get_news.py

@@ -0,0 +1,205 @@
+import random
+import requests
+import xml.etree.ElementTree as ET
+from bs4 import BeautifulSoup
+from config.logger import setup_logging
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+
+TAG = __name__
+logger = setup_logging()
+
+GET_NEWS_FUNCTION_DESC = {
+    "type": "function",
+    "function": {
+        "name": "get_news",
+        "description": (
+            "获取最新新闻，随机选择一条新闻进行播报。"
+            "用户可以指定新闻类型，如社会新闻、科技新闻、国际新闻等。"
+            "如果没有指定，默认播报社会新闻。"
+            "用户可以要求获取详细内容，此时会获取新闻的详细内容。"
+        ),
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "category": {
+                    "type": "string",
+                    "description": "新闻类别，例如社会、科技、国际。可选参数，如果不提供则使用默认类别"
+                },
+                "detail": {
+                    "type": "boolean",
+                    "description": "是否获取详细内容，默认为false。如果为true，则获取上一条新闻的详细内容"
+                },
+                "lang": {
+                    "type": "string",
+                    "description": "返回用户使用的语言code，例如zh_CN/zh_HK/en_US/ja_JP等，默认zh_CN"
+                }
+            },
+            "required": ["lang"]
+        }
+    }
+}
+
+
+def fetch_news_from_rss(rss_url):
+    """从RSS源获取新闻列表"""
+    try:
+        response = requests.get(rss_url)
+        response.raise_for_status()
+        
+        # 解析XML
+        root = ET.fromstring(response.content)
+        
+        # 查找所有item元素（新闻条目）
+        news_items = []
+        for item in root.findall('.//item'):
+            title = item.find('title').text if item.find('title') is not None else "无标题"
+            link = item.find('link').text if item.find('link') is not None else "#"
+            description = item.find('description').text if item.find('description') is not None else "无描述"
+            pubDate = item.find('pubDate').text if item.find('pubDate') is not None else "未知时间"
+            
+            news_items.append({
+                'title': title,
+                'link': link,
+                'description': description,
+                'pubDate': pubDate
+            })
+        
+        return news_items
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"获取RSS新闻失败: {e}")
+        return []
+
+
+def fetch_news_detail(url):
+    """获取新闻详情页内容并总结"""
+    try:
+        response = requests.get(url)
+        response.raise_for_status()
+        
+        soup = BeautifulSoup(response.content, 'html.parser')
+        
+        # 尝试提取正文内容 (这里的选择器需要根据实际网站结构调整)
+        content_div = soup.select_one('.content_desc, .content, article, .article-content')
+        if content_div:
+            paragraphs = content_div.find_all('p')
+            content = '\n'.join([p.get_text().strip() for p in paragraphs if p.get_text().strip()])
+            return content
+        else:
+            # 如果找不到特定的内容区域，尝试获取所有段落
+            paragraphs = soup.find_all('p')
+            content = '\n'.join([p.get_text().strip() for p in paragraphs if p.get_text().strip()])
+            return content[:2000]  # 限制长度
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"获取新闻详情失败: {e}")
+        return "无法获取详细内容"
+
+
+def map_category(category_text):
+    """将用户输入的中文类别映射到配置文件中的类别键"""
+    if not category_text:
+        return None
+        
+    # 类别映射字典，目前支持社会、国际、财经新闻，如需更多类型，参见配置文件
+    category_map = {
+        # 社会新闻
+        "社会": "society",
+        "社会新闻": "society",
+        # 国际新闻
+        "国际": "world",
+        "国际新闻": "world",
+        # 财经新闻
+        "财经": "finance",
+        "财经新闻": "finance",
+        "金融": "finance",
+        "经济": "finance"
+    }
+    
+    # 转换为小写并去除空格
+    normalized_category = category_text.lower().strip()
+    
+    # 返回映射结果，如果没有匹配项则返回原始输入
+    return category_map.get(normalized_category, category_text)
+
+
+@register_function('get_news', GET_NEWS_FUNCTION_DESC, ToolType.SYSTEM_CTL)
+def get_news(conn, category: str = None, detail: bool = False, lang: str = "zh_CN"):
+    """获取新闻并随机选择一条进行播报，或获取上一条新闻的详细内容"""
+    try:
+        # 如果detail为True，获取上一条新闻的详细内容
+        if detail:
+            if not hasattr(conn, 'last_news_link') or not conn.last_news_link or 'link' not in conn.last_news_link:
+                return ActionResponse(Action.REQLLM, "抱歉，没有找到最近查询的新闻，请先获取一条新闻。", None)
+            
+            link = conn.last_news_link.get('link')
+            title = conn.last_news_link.get('title', '未知标题')
+            
+            if link == '#':
+                return ActionResponse(Action.REQLLM, "抱歉，该新闻没有可用的链接获取详细内容。", None)
+            
+            logger.bind(tag=TAG).debug(f"获取新闻详情: {title}, URL={link}")
+            
+            # 获取新闻详情
+            detail_content = fetch_news_detail(link)
+            
+            if not detail_content or detail_content == "无法获取详细内容":
+                return ActionResponse(Action.REQLLM, f"抱歉，无法获取《{title}》的详细内容，可能是链接已失效或网站结构发生变化。", None)
+            
+            # 构建详情报告
+            detail_report = (
+                f"根据下列数据，用{lang}回应用户的新闻详情查询请求：\n\n"
+                f"新闻标题: {title}\n"
+                f"详细内容: {detail_content}\n\n"
+                f"(请对上述新闻内容进行总结，提取关键信息，以自然、流畅的方式向用户播报，"
+                f"不要提及这是总结，就像是在讲述一个完整的新闻故事)"
+            )
+            
+            return ActionResponse(Action.REQLLM, detail_report, None)
+        
+        # 否则，获取新闻列表并随机选择一条
+        # 从配置中获取RSS URL
+        rss_config = conn.config["plugins"]["get_news"]
+        default_rss_url = rss_config.get("default_rss_url", "https://www.chinanews.com.cn/rss/society.xml")
+        
+        # 将用户输入的类别映射到配置中的类别键
+        mapped_category = map_category(category)
+        
+        # 如果提供了类别，尝试从配置中获取对应的URL
+        rss_url = default_rss_url
+        if mapped_category and mapped_category in rss_config.get("category_urls", {}):
+            rss_url = rss_config["category_urls"][mapped_category]
+        
+        logger.bind(tag=TAG).info(f"获取新闻: 原始类别={category}, 映射类别={mapped_category}, URL={rss_url}")
+        
+        # 获取新闻列表
+        news_items = fetch_news_from_rss(rss_url)
+        
+        if not news_items:
+            return ActionResponse(Action.REQLLM, "抱歉，未能获取到新闻信息，请稍后再试。", None)
+        
+        # 随机选择一条新闻
+        selected_news = random.choice(news_items)
+        
+        # 保存当前新闻链接到连接对象，以便后续查询详情
+        if not hasattr(conn, 'last_news_link'):
+            conn.last_news_link = {}
+        conn.last_news_link = {
+            'link': selected_news.get('link', '#'),
+            'title': selected_news.get('title', '未知标题')
+        }
+        
+        # 构建新闻报告
+        news_report = (
+            f"根据下列数据，用{lang}回应用户的新闻查询请求：\n\n"
+            f"新闻标题: {selected_news['title']}\n"
+            f"发布时间: {selected_news['pubDate']}\n"
+            f"新闻内容: {selected_news['description']}\n"
+            f"(请以自然、流畅的方式向用户播报这条新闻，可以适当总结内容，"
+            f"直接读出新闻即可，不需要额外多余的内容。"
+            f"如果用户询问更多详情，告知用户可以说'请详细介绍这条新闻'获取更多内容)"
+        )
+        
+        return ActionResponse(Action.REQLLM, news_report, None)
+    
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"获取新闻出错: {e}")
+        return ActionResponse(Action.REQLLM, "抱歉，获取新闻时发生错误，请稍后再试。", None)

main/xiaozhi-server/plugins_func/functions/get_time.py

@@ -6,7 +6,7 @@ get_time_function_desc = {
     "function": {
         "name": "get_time",
         "description": "获取当前时间、日期、星期几",
-        "parameters": {}
+        'parameters': {'type': 'object', 'properties': {}, 'required': []}
     }
 }
 

main/xiaozhi-server/plugins_func/functions/get_weather.py

@@ -1,42 +1,104 @@
 import requests
 from bs4 import BeautifulSoup
-from plugins_func.register import register_function,ToolType, ActionResponse, Action
+from config.logger import setup_logging
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
 
+TAG = __name__
+logger = setup_logging()
 
-get_weather_function_desc = {
+GET_WEATHER_FUNCTION_DESC = {
     "type": "function",
     "function": {
         "name": "get_weather",
-        "description": "获取某个地点的天气，用户应先提供一个位置，比如用户说杭州天气，参数为：zhejiang/hangzhou，比如用户说北京天气怎么样，参数为：beijing/beijing。如果用户只问天气怎么样，参数是:guangdong/guangzhou",
+        "description": (
+            "获取某个地点的天气，用户应提供一个位置，比如用户说杭州天气，参数为：杭州。"
+            "如果用户说的是省份，默认用省会城市。如果用户说的不是省份或城市而是一个地名，"
+            "默认用该地所在省份的省会城市。"
+        ),
         "parameters": {
             "type": "object",
             "properties": {
-                "city": {
+                "location": {
                     "type": "string",
-                    "description": "城市，zhejiang/hangzhou"
+                    "description": "地点名，例如杭州。可选参数，如果不提供则不传"
+                },
+                "lang": {
+                    "type": "string",
+                    "description": "返回用户使用的语言code，例如zh_CN/zh_HK/en_US/ja_JP等，默认zh_CN"
                 }
             },
-            "required": [
+            "required": ["lang"]
-                "city"
-            ]
         }
     }
 }
 
-@register_function('get_weather', get_weather_function_desc, ToolType.WAIT)
+HEADERS = {
-def get_weather(city: str):
+    'User-Agent': (
-    """
+        'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
-    "获取某个地点的天气，用户应先提供一个位置，\n比如用户说杭州天气，参数为：zhejiang/hangzhou，\n\n比如用户说北京天气怎么样，参数为：beijing/beijing",
+        '(KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36'
-    city : 城市，zhejiang/hangzhou
+    )
-    """
+}
-    url = "https://tianqi.moji.com/weather/china/"+city
+
-    headers = {
+
-        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36'
+def fetch_city_info(location, api_key):
-    }
+    url = f"https://geoapi.qweather.com/v2/city/lookup?key={api_key}&location={location}&lang=zh"
-    response = requests.get(url, headers=headers)
+    response = requests.get(url, headers=HEADERS).json()
-    if response.status_code!=200:
+    return response.get('location', [])[0] if response.get('location') else None
+
+
+def fetch_weather_page(url):
+    response = requests.get(url, headers=HEADERS)
+    return BeautifulSoup(response.text, "html.parser") if response.ok else None
+
+
+def parse_weather_info(soup):
+    city_name = soup.select_one("h1.c-submenu__location").get_text(strip=True)
+
+    current_abstract = soup.select_one(".c-city-weather-current .current-abstract")
+    current_abstract = current_abstract.get_text(strip=True) if current_abstract else "未知"
+
+    current_basic = {}
+    for item in soup.select(".c-city-weather-current .current-basic .current-basic___item"):
+        parts = item.get_text(strip=True, separator=" ").split(" ")
+        if len(parts) == 2:
+            key, value = parts[1], parts[0]
+            current_basic[key] = value
+
+    temps_list = []
+    for row in soup.select(".city-forecast-tabs__row")[:7]:  # 取前7天的数据
+        date = row.select_one(".date-bg .date").get_text(strip=True)
+        temps = [span.get_text(strip=True) for span in row.select(".tmp-cont .temp")]
+        high_temp, low_temp = (temps[0], temps[-1]) if len(temps) >= 2 else (None, None)
+        temps_list.append((date, high_temp, low_temp))
+
+    return city_name, current_abstract, current_basic, temps_list
+
+
+@register_function('get_weather', GET_WEATHER_FUNCTION_DESC, ToolType.SYSTEM_CTL)
+def get_weather(conn, location: str = None, lang: str = "zh_CN"):
+    api_key = conn.config["plugins"]["get_weather"]["api_key"]
+    default_location = conn.config["plugins"]["get_weather"]["default_location"]
+    location = location or conn.client_ip_info.get("city") or default_location
+    logger.bind(tag=TAG).debug(f"获取天气: {location}")
+
+    city_info = fetch_city_info(location, api_key)
+    if not city_info:
+        return ActionResponse(Action.REQLLM, f"未找到相关的城市: {location}，请确认地点是否正确", None)
+
+    soup = fetch_weather_page(city_info['fxLink'])
+    if not soup:
         return ActionResponse(Action.REQLLM, None, "请求失败")
-    soup = BeautifulSoup(response.text, "html.parser")
+
-    weather = soup.find('meta', attrs={'name':'description'})["content"]
+    city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
-    weather = weather.replace("墨迹天气", "")
+    weather_report = f"根据下列数据，用{lang}回应用户的查询天气请求：\n{city_name}未来7天天气:\n"
-    return ActionResponse(Action.REQLLM, weather, None)
+    for i, (date, high, low) in enumerate(temps_list):
+        if high and low:
+            weather_report += f"{date}: {low}到{high}\n"
+    weather_report += (
+        f"当前天气: {current_abstract}\n"
+        f"当前天气参数: {current_basic}\n"
+        f"(确保只报告指定单日的气温范围，除非用户明确要求想要了解多日天气，如果未指定，默认报告今天的温度范围。"
+        "参数为0的值不需要报告给用户，每次都报告体感温度，根据语境选择合适的参数内容告知用户，并对参数给出相应评价)"
+    )
+
+    return ActionResponse(Action.REQLLM, weather_report, None)

main/xiaozhi-server/plugins_func/functions/hass_get_state.py

@@ -0,0 +1,56 @@
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+from plugins_func.functions.hass_init import initialize_hass_handler
+from config.logger import setup_logging
+import asyncio
+import requests
+
+TAG = __name__
+logger = setup_logging()
+
+hass_get_state_function_desc = {
+    "type": "function",
+    "function": {
+        "name": "hass_get_state",
+        "description": "获取homeassistant里设备的状态,包括灯光亮度,媒体播放器的音量,设备的暂停、继续操作",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "entity_id": {
+                    "type": "string",
+                    "description": "需要操作的设备id,homeassistant里的entity_id"
+                }
+            },
+            "required": ["entity_id"]
+        }
+    }
+}
+
+
+@register_function("hass_get_state", hass_get_state_function_desc, ToolType.SYSTEM_CTL)
+def hass_get_state(conn, entity_id=''):
+    try:
+
+        future = asyncio.run_coroutine_threadsafe(
+            handle_hass_get_state(conn, entity_id),
+            conn.loop
+        )
+        ha_response = future.result()
+        return ActionResponse(action=Action.REQLLM, result="执行成功", response=ha_response)
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"处理设置属性意图错误: {e}")
+
+
+async def handle_hass_get_state(conn, entity_id):
+    HASS_CACHE = initialize_hass_handler(conn)
+    api_key = HASS_CACHE['api_key']
+    base_url = HASS_CACHE['base_url']
+    url = f"{base_url}/api/states/{entity_id}"
+    headers = {
+        "Authorization": f"Bearer {api_key}",
+        "Content-Type": "application/json"
+    }
+    response = requests.get(url, headers=headers)
+    if response.status_code == 200:
+        return response.json()['state']
+    else:
+        return f"切换失败，错误码: {response.status_code}"

main/xiaozhi-server/plugins_func/functions/hass_init.py

@@ -0,0 +1,35 @@
+from config.logger import setup_logging
+from core.utils.util import check_model_key
+
+TAG = __name__
+logger = setup_logging()
+
+HASS_CACHE = {}
+
+
+def append_devices_to_prompt(conn):
+    if conn.use_function_call_mode:
+        funcs = conn.config["Intent"]["function_call"].get("functions", [])
+        if "hass_get_state" in funcs or "hass_get_state" in funcs:
+            prompt = "下面是我家智能设备，可以通过homeassistant控制\n"
+            devices = conn.config["plugins"]["home_assistant"].get("devices", [])
+            if len(devices) == 0:
+                return
+            for device in devices:
+                prompt += device + "\n"
+            conn.prompt += prompt
+            # 更新提示词
+            conn.dialogue.update_system_message(conn.prompt)
+
+
+def initialize_hass_handler(conn):
+    global HASS_CACHE
+    if HASS_CACHE == {}:
+        if conn.use_function_call_mode:
+            funcs = conn.config["Intent"]["function_call"].get("functions", [])
+            if "hass_get_state" in funcs or "hass_get_state" in funcs:
+                HASS_CACHE['base_url'] = conn.config["plugins"]["home_assistant"].get("base_url")
+                HASS_CACHE['api_key'] = conn.config["plugins"]["home_assistant"].get("api_key")
+
+                check_model_key("home_assistant", HASS_CACHE['api_key'])
+    return HASS_CACHE

main/xiaozhi-server/plugins_func/functions/hass_play_music.py

@@ -0,0 +1,64 @@
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+from plugins_func.functions.hass_init import initialize_hass_handler
+from config.logger import setup_logging
+import asyncio
+import requests
+
+TAG = __name__
+logger = setup_logging()
+
+hass_play_music_function_desc = {
+    "type": "function",
+    "function": {
+        "name": "hass_play_music",
+        "description": "用户想听音乐、有声书的时候使用，在房间的媒体播放器（media_player）里播放对应音频",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "media_content_id": {
+                    "type": "string",
+                    "description": "可以是音乐或有声书的专辑名称、歌曲名、演唱者,如果未指定就填random"
+                },
+                "entity_id": {
+                    "type": "string",
+                    "description": "需要操作的音箱的设备id,homeassistant里的entity_id,media_player开头"
+                }
+            },
+            "required": ["media_content_id", "entity_id"]
+        }
+    }
+}
+
+
+@register_function('hass_play_music', hass_play_music_function_desc, ToolType.SYSTEM_CTL)
+def hass_play_music(conn, entity_id='', media_content_id='random'):
+    try:
+        # 执行音乐播放命令
+        future = asyncio.run_coroutine_threadsafe(
+            handle_hass_play_music(conn, entity_id, media_content_id),
+            conn.loop
+        )
+        ha_response = future.result()
+        return ActionResponse(action=Action.RESPONSE, result="退出意图已处理", response=ha_response)
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"处理音乐意图错误: {e}")
+
+
+async def handle_hass_play_music(conn, entity_id, media_content_id):
+    HASS_CACHE = initialize_hass_handler(conn)
+    api_key = HASS_CACHE['api_key']
+    base_url = HASS_CACHE['base_url']
+    url = f"{base_url}/api/services/music_assistant/play_media"
+    headers = {
+        "Authorization": f"Bearer {api_key}",
+        "Content-Type": "application/json"
+    }
+    data = {
+        "entity_id": entity_id,
+        "media_id": media_content_id
+    }
+    response = requests.post(url, headers=headers, json=data)
+    if response.status_code == 200:
+        return f"正在播放{media_content_id}的音乐"
+    else:
+        return f"音乐播放失败，错误码: {response.status_code}"

main/xiaozhi-server/plugins_func/functions/hass_set_state.py

@@ -0,0 +1,159 @@
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+from plugins_func.functions.hass_init import initialize_hass_handler
+from config.logger import setup_logging
+import asyncio
+import requests
+
+TAG = __name__
+logger = setup_logging()
+
+hass_set_state_function_desc = {
+    "type": "function",
+    "function": {
+        "name": "hass_set_state",
+        "description": "设置homeassistant里设备的状态,包括开、关,调整灯光亮度,调整播放器的音量,设备的暂停、继续、静音操作",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "state": {
+                    "type": "object",
+                    "properties": {
+                        "type": {
+                            "type": "string",
+                            "description": "需要操作的动作,打开设备:turn_on,关闭设备:turn_off,增加亮度:brightness_up,降低亮度:brightness_down,设置亮度:brightness_value,增加>音量:,volume_up降低音量:volume_down,设置音量:volume_set,设备暂停:pause,设备继续:continue,静音/取消静音:volume_mute"
+                        },
+                        "input": {
+                            "type": "int",
+                            "description": "只有在设置音量,设置亮度时候才需要,有效值为1-100,对应音量和亮度的1%-100%"
+                        },
+                        "is_muted": {
+                            "type": "string",
+                            "description": "只有在设置静音操作时才需要,设置静音的时候该值为true,取消静音时该值为false"
+                        }
+                    },
+                    "required": ["type"]
+                },
+                "entity_id": {
+                    "type": "string",
+                    "description": "需要操作的设备id,homeassistant里的entity_id"
+                }
+            },
+            "required": ["state", "entity_id"]
+        }
+    }
+}
+
+
+@register_function('hass_set_state', hass_set_state_function_desc, ToolType.SYSTEM_CTL)
+def hass_set_state(conn, entity_id='', state={}):
+    try:
+        future = asyncio.run_coroutine_threadsafe(
+            handle_hass_set_state(conn, entity_id, state),
+            conn.loop
+        )
+        ha_response = future.result()
+        return ActionResponse(action=Action.REQLLM, result="执行成功", response=ha_response)
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"处理设置属性意图错误: {e}")
+
+
+async def handle_hass_set_state(conn, entity_id, state):
+    HASS_CACHE = initialize_hass_handler(conn)
+    api_key = HASS_CACHE['api_key']
+    base_url = HASS_CACHE['base_url']
+    '''
+    state = { "type":"brightness_up","input":"80","is_muted":"true"}
+    '''
+    domains = entity_id.split(".")
+    if len(domains) > 1:
+        domain = domains[0]
+    else:
+        return "执行失败，错误的设备id"
+    action = ''
+    arg = ''
+    value = ''
+    if state['type'] == 'turn_on':
+        description = "设备已打开"
+        if domain == "cover":
+            action = "open_cover"
+        elif domain == "vacuum":
+            action = "start"
+        else:
+            action = "turn_on"
+    elif state['type'] == 'turn_off':
+        description = "设备已关闭"
+        if domain == 'cover':
+            action = "close_cover"
+        elif domain == 'vacuum':
+            action = "stop"
+        else:
+            action = "turn_off"
+    elif state['type'] == 'brightness_up':
+        description = "灯光已调亮"
+        action = 'turn_on'
+        arg = 'brightness_step_pct'
+        value = 10
+    elif state['type'] == 'brightness_down':
+        description = "灯光已调暗"
+        action = 'turn_on'
+        arg = 'brightness_step_pct'
+        value = -10
+    elif state['type'] == 'brightness_value':
+        description = f"亮度已调整到{state['input']}"
+        action = 'turn_on'
+        arg = 'brightness_pct'
+        value = state['input']
+    elif state['type'] == 'volume_up':
+        description = "音量已调大"
+        action = state['type']
+    elif state['type'] == 'volume_down':
+        description = "音量已调小"
+        action = state['type']
+    elif state['type'] == 'volume_set':
+        description = f"音量已调整到{state['input']}"
+        action = state['type']
+        arg = 'volume_level'
+        value = state['input']
+    elif state['type'] == 'volume_mute':
+        description = f"设备已静音"
+        action = state['type']
+        arg = 'is_volume_muted'
+        value = state['is_muted']
+    elif state['type'] == 'pause':
+        description = f"设备已暂停"
+        action = state['type']
+        if domain == 'media_player':
+            action = 'media_pause'
+        if domain == 'cover':
+            action = 'stop_cover'
+        if domain == 'vacuum':
+            action = 'pause'
+    elif state['type'] == 'continue':
+        description = f"设备已继续"
+        if domain == 'media_player':
+            action = 'media_play'
+        if domain == 'vacuum':
+            action = 'start'
+    else:
+        return f"{domain} {state.type}功能尚未支持"
+
+    if arg == '':
+        data = {
+            "entity_id": entity_id,
+        }
+    else:
+        data = {
+            "entity_id": entity_id,
+            arg: value
+        }
+    url = f"{base_url}/api/services/{domain}/{action}"
+    headers = {
+        "Authorization": f"Bearer {api_key}",
+        "Content-Type": "application/json"
+    }
+    response = requests.post(url, headers=headers, json=data)
+    logger.bind(tag=TAG).info(f"设置状态:url:{url},return_code:{response.status_code}")
+    if response.status_code == 200:
+        return description
+    else:
+        return f"设置失败，错误码: {response.status_code}"

main/xiaozhi-server/plugins_func/functions/play_music.py

@@ -1,10 +1,22 @@
-from plugins_func.register import register_function,ToolType, ActionResponse, Action
 from config.logger import setup_logging
+import os
+import re
+import time
+import random
 import asyncio
+import difflib
+import traceback
+from pathlib import Path
+from core.utils import p3
+from core.handle.sendAudioHandle import send_stt_message
+from plugins_func.register import register_function,ToolType, ActionResponse, Action
+
 
 TAG = __name__
 logger = setup_logging()
 
+MUSIC_CACHE = {}
+
 play_music_function_desc = {
                 "type": "function",
                 "function": {
@@ -29,12 +41,156 @@ def play_music(conn, song_name: str):
     try:
         music_intent = f"播放音乐 {song_name}" if song_name != "random" else "随机播放音乐"
 
-        # 执行音乐播放命令
+        # 检查事件循环状态
+        if not conn.loop.is_running():
+            logger.bind(tag=TAG).error("事件循环未运行，无法提交任务")
+            return ActionResponse(action=Action.RESPONSE, result="系统繁忙", response="请稍后再试")
+
+        # 提交异步任务
         future = asyncio.run_coroutine_threadsafe(
-            conn.music_handler.handle_music_command(conn, music_intent),
+            handle_music_command(conn, music_intent),
             conn.loop
         )
-        future.result()
+
-        return ActionResponse(action=Action.RESPONSE, result="退出意图已处理", response="还想听什么歌？")
+        # 非阻塞回调处理
+        def handle_done(f):
+            try:
+                f.result()  # 可在此处理成功逻辑
+                logger.bind(tag=TAG).info("播放完成")
+            except Exception as e:
+                logger.bind(tag=TAG).error(f"播放失败: {e}")
+
+        future.add_done_callback(handle_done)
+
+        return ActionResponse(action=Action.RESPONSE, result="指令已接收", response="正在为您播放音乐")
     except Exception as e:
-        logger.bind(tag=TAG).error(f"处理音乐意图错误: {e}")
+        logger.bind(tag=TAG).error(f"处理音乐意图错误: {e}")
+        return ActionResponse(action=Action.RESPONSE, result=str(e), response="播放音乐时出错了")
+
+
+def _extract_song_name(text):
+    """从用户输入中提取歌名"""
+    for keyword in ["播放音乐"]:
+        if keyword in text:
+            parts = text.split(keyword)
+            if len(parts) > 1:
+                return parts[1].strip()
+    return None
+
+
+def _find_best_match(potential_song, music_files):
+    """查找最匹配的歌曲"""
+    best_match = None
+    highest_ratio = 0
+
+    for music_file in music_files:
+        song_name = os.path.splitext(music_file)[0]
+        ratio = difflib.SequenceMatcher(None, potential_song, song_name).ratio()
+        if ratio > highest_ratio and ratio > 0.4:
+            highest_ratio = ratio
+            best_match = music_file
+    return best_match
+
+
+def get_music_files(music_dir, music_ext):
+    music_dir = Path(music_dir)
+    music_files = []
+    music_file_names = []
+    for file in music_dir.rglob("*"):
+        # 判断是否是文件
+        if file.is_file():
+            # 获取文件扩展名
+            ext = file.suffix.lower()
+            # 判断扩展名是否在列表中
+            if ext in music_ext:
+                # 添加相对路径
+                music_files.append(str(file.relative_to(music_dir)))
+                music_file_names.append(os.path.splitext(str(file.relative_to(music_dir)))[0])
+    return music_files, music_file_names
+
+
+def initialize_music_handler(conn):
+    global MUSIC_CACHE
+    if MUSIC_CACHE == {}:
+        if "music" in conn.config:
+            MUSIC_CACHE["music_config"] = conn.config["music"]
+            MUSIC_CACHE["music_dir"] = os.path.abspath(
+                MUSIC_CACHE["music_config"].get("music_dir", "./music")  # 默认路径修改
+            )
+            MUSIC_CACHE["music_ext"] = MUSIC_CACHE["music_config"].get("music_ext", (".mp3", ".wav", ".p3"))
+            MUSIC_CACHE["refresh_time"] = MUSIC_CACHE["music_config"].get("refresh_time", 60)
+        else:
+            MUSIC_CACHE["music_dir"] = os.path.abspath("./music")
+            MUSIC_CACHE["music_ext"] = (".mp3", ".wav", ".p3")
+            MUSIC_CACHE["refresh_time"] = 60
+        # 获取音乐文件列表
+        MUSIC_CACHE["music_files"], MUSIC_CACHE["music_file_names"] = get_music_files(MUSIC_CACHE["music_dir"],
+                                                                                      MUSIC_CACHE["music_ext"])
+        MUSIC_CACHE["scan_time"] = time.time()
+    return MUSIC_CACHE
+
+
+async def handle_music_command(conn, text):
+    initialize_music_handler(conn)
+    global MUSIC_CACHE
+
+    """处理音乐播放指令"""
+    clean_text = re.sub(r'[^\w\s]', '', text).strip()
+    logger.bind(tag=TAG).debug(f"检查是否是音乐命令: {clean_text}")
+
+    # 尝试匹配具体歌名
+    if os.path.exists(MUSIC_CACHE["music_dir"]):
+        if time.time() - MUSIC_CACHE["scan_time"] > MUSIC_CACHE["refresh_time"]:
+            # 刷新音乐文件列表
+            MUSIC_CACHE["music_files"], MUSIC_CACHE["music_file_names"] = get_music_files(MUSIC_CACHE["music_dir"],
+                                                                                          MUSIC_CACHE["music_ext"])
+            MUSIC_CACHE["scan_time"] = time.time()
+
+        potential_song = _extract_song_name(clean_text)
+        if potential_song:
+            best_match = _find_best_match(potential_song, MUSIC_CACHE["music_files"])
+            if best_match:
+                logger.bind(tag=TAG).info(f"找到最匹配的歌曲: {best_match}")
+                await play_local_music(conn, specific_file=best_match)
+                return True
+    # 检查是否是通用播放音乐命令
+    await play_local_music(conn)
+    return True
+
+
+async def play_local_music(conn, specific_file=None):
+    global MUSIC_CACHE
+    """播放本地音乐文件"""
+    try:
+        if not os.path.exists(MUSIC_CACHE["music_dir"]):
+            logger.bind(tag=TAG).error(f"音乐目录不存在: " + MUSIC_CACHE["music_dir"])
+            return
+
+        # 确保路径正确性
+        if specific_file:
+            selected_music = specific_file
+            music_path = os.path.join(MUSIC_CACHE["music_dir"], specific_file)
+        else:
+            if not MUSIC_CACHE["music_files"]:
+                logger.bind(tag=TAG).error("未找到MP3音乐文件")
+                return
+            selected_music = random.choice(MUSIC_CACHE["music_files"])
+            music_path = os.path.join(MUSIC_CACHE["music_dir"], selected_music)
+
+        if not os.path.exists(music_path):
+            logger.bind(tag=TAG).error(f"选定的音乐文件不存在: {music_path}")
+            return
+        text = f"正在播放{selected_music}"
+        await send_stt_message(conn, text)
+        conn.tts_first_text_index = 0
+        conn.tts_last_text_index = 0
+        conn.llm_finish_task = True
+        if music_path.endswith(".p3"):
+            opus_packets, duration = p3.decode_opus_from_file(music_path)
+        else:
+            opus_packets, duration = conn.tts.audio_to_opus_data(music_path)
+        conn.audio_play_queue.put((opus_packets, selected_music, 0))
+
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"播放音乐失败: {str(e)}")
+        logger.bind(tag=TAG).error(f"详细错误: {traceback.format_exc()}")

main/xiaozhi-server/plugins_func/functions/raise_and_lower_the_volume.py

@@ -0,0 +1,64 @@
+from config.logger import setup_logging
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+from core.handle.iotHandle import get_iot_status, send_iot_conn
+import asyncio
+
+TAG = __name__
+logger = setup_logging()
+
+raise_and_lower_the_volume_function_desc = {
+    "type": "function",
+    "function": {
+        "name": "raise_and_lower_the_volume",
+        "description": "用户觉得声音过高或过低，或者用户想提高或降低音量。比如用户说太大声了，参数为：lower，比如用户说提高音量，参数为：raise",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "action": {
+                    "type": "string",
+                    "description": "动作名称，要么是raise，要么是lower"
+                }
+            },
+            "required": ["action"]
+        }
+    }
+}
+
+
+@register_function('raise_and_lower_the_volume', raise_and_lower_the_volume_function_desc, ToolType.IOT_CTL)
+def raise_and_lower_the_volume(conn, action: str):
+    """
+    获取当前设备音量
+    """
+
+    future = asyncio.run_coroutine_threadsafe(
+        _raise_and_lower_the_volume(conn, action),
+        conn.loop
+    )
+
+    try:
+        new_volume = future.result()  # 同步等待异步操作完成
+        logger.bind(tag=TAG).info(f"音量操作完成: {new_volume}")
+        response = f"音量已调整到{new_volume}"
+    except Exception as e:
+        logger.bind(tag=TAG).error(f"音量操作失败: {e}")
+        response = f"音量调整失败: {e}"
+
+    return ActionResponse(action=Action.RESPONSE, result="指令已接收", response=response)
+
+
+async def _raise_and_lower_the_volume(conn, action):
+    volume = await get_iot_status(conn, "Speaker", "volume")
+    if volume is None:
+        raise Exception("你的设备不支持音量控制")
+    if action == 'raise':
+        volume += 10
+    elif action == 'lower':
+        volume -= 10
+    # 限制音量范围在0到100之间
+    if volume < 0:
+        volume = 0
+    elif volume > 100:
+        volume = 100
+    await send_iot_conn(conn, "Speaker", "SetVolume", {"volume": volume})
+    return volume

main/xiaozhi-server/plugins_func/register.py

@@ -11,6 +11,7 @@ class ToolType(Enum):
     WAIT = (2, "调用工具，等待函数返回")
     CHANGE_SYS_PROMPT = (3, "修改系统提示词，切换角色性格或职责")
     SYSTEM_CTL = (4, "系统控制，影响正常的对话流程，如退出、播放音乐等，需要传递conn参数")
+    IOT_CTL = (5, "IOT设备控制，需要传递conn参数")
 
     def __init__(self, code, message):
         self.code = code
@@ -18,6 +19,7 @@ class ToolType(Enum):
 
 
 class Action(Enum):
+    ERROR = (-1, "错误")
     NOTFOUND = (0, "没有找到函数")
     NONE = (1, "啥也不干")
     RESPONSE = (2, "直接回复")
@@ -40,8 +42,31 @@ class FunctionItem:
         self.func = func
         self.type = type
 
+class DeviceTypeRegistry:
+    """设备类型注册表，用于管理IOT设备类型及其函数"""
+    def __init__(self):
+        self.type_functions = {}  # type_signature -> {func_name: FunctionItem}
+    
+    def generate_device_type_id(self, descriptor):
+        """通过设备能力描述生成类型ID"""
+        properties = sorted(descriptor["properties"].keys())
+        methods = sorted(descriptor["methods"].keys())
+        # 使用属性和方法的组合作为设备类型的唯一标识
+        type_signature = f"{descriptor['name']}:{','.join(properties)}:{','.join(methods)}"
+        return type_signature
+    
+    def get_device_functions(self, type_id):
+        """获取设备类型对应的所有函数"""
+        return self.type_functions.get(type_id, {})
+    
+    def register_device_type(self, type_id, functions):
+        """注册设备类型及其函数"""
+        if type_id not in self.type_functions:
+            self.type_functions[type_id] = functions
+
 # 初始化函数注册字典
 all_function_registry = {}
+device_type_registry = DeviceTypeRegistry()
 
 def register_function(name, desc, type=None):
     """注册函数到函数注册字典的装饰器"""

main/xiaozhi-server/requirements.txt

@@ -19,4 +19,6 @@ loguru==0.7.3
 requests==2.32.3
 cozepy==0.12.0
 mem0ai==0.1.62
-bs4==0.0.2
+bs4==0.0.2
+modelscope==1.23.2
+sherpa_onnx==1.11.0