diff --git a/README.md b/README.md index eb718c24..17e2b886 100644 --- a/README.md +++ b/README.md @@ -242,7 +242,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/ | 智能对话 | 支持多种LLM(大语言模型),实现智能对话 | | 视觉感知 | 支持多种VLLM(视觉大模型),实现多模态交互 | | 意图识别 | 支持外挂的大模型意图识别、大模型自主函数调用,提供插件化意图处理机制 | -| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆,具备记忆总结功能 | +| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆、PowerMem智能记忆,具备记忆总结功能 | | 知识库 | 支持RAGFlow知识库,让大模型判断需要调度知识库后再回答 | | 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 | | 指令下发 | 依托MQTT协议,支持从智控台将MCP指令下发到ESP32设备 | @@ -330,6 +330,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/ | 类型 | 平台名称 | 使用方式 | 收费模式 | 备注 | |:------:|:---------------:|:----:|:---------:|:--:| | Memory | mem0ai | 接口调用 | 1000次/月额度 | | +| Memory | [powermem](./docs/powermem-integration.md) | 本地总结 | 取决于LLM和DB | OceanBase开源,支持智能检索 | | Memory | mem_local_short | 本地总结 | 免费 | | | Memory | nomem | 无记忆模式 | 免费 | | diff --git a/README_de.md b/README_de.md index 0e4c74d5..403fc921 100644 --- a/README_de.md +++ b/README_de.md @@ -240,7 +240,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun | Intelligenter Dialog | Unterstützt mehrere LLM (große Sprachmodelle), implementiert intelligenten Dialog | | Visuelle Wahrnehmung | Unterstützt mehrere VLLM (Vision Large Models), implementiert multimodale Interaktion | | Absichtserkennung | Unterstützt LLM-Absichtserkennung, Function Call-Funktionsaufruf, bietet plugin-basierten Absichtsverarbeitungsmechanismus | -| Gedächtnissystem | Unterstützt lokales Kurzzeitgedächtnis, mem0ai-Schnittstellengedächtnis, mit Gedächtniszusammenfassungsfunktion | +| Gedächtnissystem | Unterstützt lokales Kurzzeitgedächtnis, mem0ai-Schnittstellengedächtnis, PowerMem intelligentes Gedächtnis, mit Gedächtniszusammenfassungsfunktion | | Wissensdatenbank | Unterstützt RAGFlow-Wissensdatenbank, ermöglicht großem Modell die Bewertung, ob Wissensdatenbank benötigt wird, bevor geantwortet wird | | Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Endpunktprotokoll, benutzerdefinierte Werkzeugfunktionen | | Befehlsübermittlung | Basierend auf MQTT-Protokoll, unterstützt die Übermittlung von MCP-Befehlen von der intelligenten Steuerkonsole an ESP32-Geräte | @@ -328,6 +328,7 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int | Typ | Plattformname | Verwendungsmethode | Preismodell | Hinweise | |:------:|:---------------:|:----:|:---------:|:--:| | Memory | mem0ai | Schnittstellenaufrufe | 1000 Mal/Monat Kontingent | | +| Memory | [powermem](./docs/powermem-integration.md) | Lokale Zusammenfassung | Abhängig von LLM und DB | OceanBase Open Source, unterstützt intelligente Abfrage | | Memory | mem_local_short | Lokale Zusammenfassung | Kostenlos | | | Memory | nomem | Kein Gedächtnismodus | Kostenlos | | diff --git a/README_en.md b/README_en.md index 196cf16f..e071e7fa 100644 --- a/README_en.md +++ b/README_en.md @@ -238,7 +238,7 @@ This project provides the following testing tools to help you verify the system | Intelligent Dialogue | Supports multiple LLM(large language models), implements intelligent dialogue | | Visual Perception | Supports multiple VLLM(vision large models), implements multimodal interaction | | Intent Recognition | Supports LLM intent recognition, Function Call function calling, provides plugin-based intent processing mechanism | -| Memory System | Supports local short-term memory, mem0ai interface memory, with memory summarization functionality | +| Memory System | Supports local short-term memory, mem0ai interface memory, PowerMem intelligent memory, with memory summarization functionality | | Knowledge Base | Supports RAGFlow knowledge base, enabling LLM to judge whether to schedule the knowledge base after receiving the user's question, and then answer the question | | Command Delivery | Supports MCP command delivery to ESP32 devices via MQTT protocol from Smart Console | | Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP endpoint protocol, custom tool functions | @@ -330,7 +330,9 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use | Type | Platform Name | Usage Method | Pricing Model | Notes | |:------:|:---------------:|:----:|:---------:|:--:| | Memory | mem0ai | Interface calls | 1000 times/month quota | | +| Memory | [powermem](./docs/powermem-integration.md) | Local summarization | Depends on LLM and DB | OceanBase open source, supports intelligent retrieval | | Memory | mem_local_short | Local summarization | Free | | +| Memory | nomem | No memory mode | Free | | --- diff --git a/README_vi.md b/README_vi.md index e6d5f489..4a2fbb6b 100644 --- a/README_vi.md +++ b/README_vi.md @@ -241,7 +241,7 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m | Đối thoại thông minh | Hỗ trợ nhiều LLM(Mô hình ngôn ngữ lớn), thực hiện đối thoại thông minh | | Cảm nhận thị giác | Hỗ trợ nhiều VLLM(Mô hình lớn thị giác), thực hiện tương tác đa phương thức | | Nhận dạng ý định | Hỗ trợ nhận dạng ý định mô hình lớn gắn ngoài, gọi hàm tự chủ mô hình lớn, cung cấp cơ chế xử lý ý định dạng plugin | -| Hệ thống bộ nhớ | Hỗ trợ bộ nhớ ngắn hạn cục bộ, bộ nhớ giao diện mem0ai, có chức năng tóm tắt bộ nhớ | +| Hệ thống bộ nhớ | Hỗ trợ bộ nhớ ngắn hạn cục bộ, bộ nhớ giao diện mem0ai, bộ nhớ thông minh PowerMem, có chức năng tóm tắt bộ nhớ | | Kho tri thức | Hỗ trợ kho tri thức RAGFlow, cho phép mô hình lớn đánh giá cần gọi kho tri thức trước khi trả lời | | Gọi công cụ | Hỗ trợ giao thức IOT phía client, giao thức MCP phía client, giao thức MCP phía server, giao thức điểm truy cập MCP, hàm công cụ tùy chỉnh | | Gửi lệnh | Dựa vào giao thức MQTT, hỗ trợ gửi lệnh MCP từ bảng điều khiển thông minh xuống thiết bị ESP32 | @@ -329,6 +329,7 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ | Loại | Tên nền tảng | Phương pháp sử dụng | Mô hình thu phí | Ghi chú | |:------:|:---------------:|:----:|:---------:|:--:| | Memory | mem0ai | Gọi giao diện | Hạn mức 1000 lần/tháng | | +| Memory | [powermem](./docs/powermem-integration.md) | Tóm tắt cục bộ | Phụ thuộc vào LLM và DB | OceanBase mã nguồn mở, hỗ trợ tìm kiếm thông minh | | Memory | mem_local_short | Tóm tắt cục bộ | Miễn phí | | | Memory | nomem | Chế độ không có bộ nhớ | Miễn phí | | diff --git a/docs/FAQ.md b/docs/FAQ.md index 69494c3d..b6bec471 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -82,6 +82,7 @@ VAD: 8、[新闻插件源配置指南](./newsnow_plugin_config.md)
9、[知识库ragflow集成指南](./ragflow-integration.md)
10、[如何部署上下文源](./context-provider-integration.md)
+11、[如何集成PowerMem智能记忆](./powermem-integration.md)
### 11、语音克隆、本地语音部署相关教程 1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)
diff --git a/docs/powermem-integration.md b/docs/powermem-integration.md new file mode 100644 index 00000000..b9a77ed5 --- /dev/null +++ b/docs/powermem-integration.md @@ -0,0 +1,345 @@ +# PowerMem 记忆组件集成指南 + +## 简介 + +[PowerMem](https://www.powermem.ai/) 是由 OceanBase 开源的 Agent 记忆组件,通过本地 LLM 进行记忆总结和智能检索,为 AI 代理提供高效的记忆管理功能。 + +费用说明:PowerMem 本身开源免费,实际费用取决于您选择的 LLM 和数据库: +- 使用 SQLite + 免费 LLM(如智谱 glm-4-flash)= **完全免费** +- 使用云端 LLM 或云端数据库 = 按对应服务收费 + +> 💡 **最佳性能提示**:PowerMem 配合 OceanBase 使用可实现最大性能释放,SQLite 仅建议在资源不足的情况下使用。 + +- **GitHub**: https://github.com/oceanbase/powermem +- **官网**: https://www.powermem.ai/ +- **使用示例**: https://github.com/oceanbase/powermem/tree/main/examples + +## 功能特性 + +- **本地总结**:通过 LLM 在本地进行记忆总结和提取 +- **用户画像**:通过 `UserMemory` 自动提取用户信息(姓名、职业、兴趣等),持续更新用户画像 +- **智能遗忘**:基于艾宾浩斯遗忘曲线,自动"遗忘"过时噪声信息 +- **多种存储后端**:支持 OceanBase(推荐,最佳性能)、SeekDB(推荐,AI应用存储一体)、PostgreSQL、SQLite(轻量备选) +- **多种 LLM 支持**:通义千问、智谱(glm-4-flash 免费)、OpenAI 等 +- **智能检索**:基于向量搜索的语义检索能力 +- **私有部署**:完全支持本地私有化部署 +- **异步操作**:高效的异步记忆管理 + +## 安装 + +PowerMem 已添加到项目依赖中,如果需要手动安装: + +```bash +pip install powermem +``` + +## 配置说明 + +### 基础配置 + +在 `config.yaml` 中配置 PowerMem: + +```yaml +selected_module: + Memory: powermem + +Memory: + powermem: + type: powermem + # 是否启用用户画像功能 + # 用户画像支持: oceanbase、seekdb、sqlite (powermem 0.3.0+) + enable_user_profile: true + + # ========== LLM 配置 ========== + llm: + provider: openai # 可选: qwen, openai, zhipu 等 + config: + api_key: 你的LLM API密钥 + model: qwen-plus + # openai_base_url: https://api.openai.com/v1 # 可选,自定义服务地址 + + # ========== Embedding 配置 ========== + embedder: + provider: openai # 可选: qwen, openai 等 + config: + api_key: 你的嵌入模型API密钥 + model: text-embedding-v4 + openai_base_url: https://dashscope.aliyuncs.com/compatible-mode/v1 + # embedding_dims: 1024 # 向量维度,非1536时需配置 + + # ========== Database 配置 ========== + vector_store: + provider: sqlite # 可选: oceanbase(推荐), seekdb(推荐), postgres, sqlite(轻量) + config: {} # SQLite 无需额外配置 +``` + +### 配置参数详解 + +#### LLM 配置 + +| 参数 | 说明 | 可选值 | +|------|------|--------| +| `llm.provider` | LLM 提供商 | `qwen`, `openai`, `zhipu` 等 | +| `llm.config.api_key` | API 密钥 | - | +| `llm.config.model` | 模型名称 | 根据提供商选择 | +| `llm.config.openai_base_url` | 自定义服务地址(可选) | - | + +#### Embedding 配置 + +| 参数 | 说明 | 可选值 | +|------|------|--------| +| `embedder.provider` | 嵌入模型提供商 | `qwen`, `openai` 等 | +| `embedder.config.api_key` | API 密钥 | - | +| `embedder.config.model` | 模型名称 | 根据提供商选择 | +| `embedder.config.openai_base_url` | 自定义服务地址(可选) | - | + +#### Database 配置 + +| 参数 | 说明 | 可选值 | +|------|------|--------| +| `vector_store.provider` | 存储后端类型 | `oceanbase`(推荐), `seekdb`(推荐), `postgres`, `sqlite`(轻量) | +| `vector_store.config` | 数据库连接配置 | 根据 provider 设置 | + +### 记忆模式说明 + +PowerMem 支持两种记忆模式: + +| 模式 | 配置 | 功能 | 存储要求 | +|------|------|------|----------| +| **普通记忆** | `enable_user_profile: false` | 对话记忆存储与检索 | 支持所有数据库 | +| **用户画像** | `enable_user_profile: true` | 记忆 + 自动提取用户画像 | oceanbase、seekdb、sqlite | + +> 📌 **版本说明**:PowerMem 0.3.0+ 版本,用户画像功能支持 OceanBase、SeekDB、SQLite 三种存储后端。 + +### 使用通义千问(推荐) + +1. 访问 [阿里云百炼平台](https://bailian.console.aliyun.com/) 注册账号 +2. 在 [API Key 管理](https://bailian.console.aliyun.com/?apiKey=1#/api-key) 页面获取 API 密钥 +3. 配置如下: + +```yaml +Memory: + powermem: + type: powermem + enable_user_profile: true + llm: + provider: qwen + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: qwen-plus + embedder: + provider: openai + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: text-embedding-v4 + openai_base_url: https://dashscope.aliyuncs.com/compatible-mode/v1 + vector_store: + provider: sqlite + config: {} +``` + +### 使用智谱免费 LLM(完全免费方案) + +智谱提供免费的 glm-4-flash 模型,配合 SQLite 可实现完全免费使用: + +1. 访问 [智谱AI开放平台](https://bigmodel.cn/) 注册账号 +2. 在 [API Keys](https://bigmodel.cn/usercenter/proj-mgmt/apikeys) 页面获取 API 密钥 +3. 配置如下: + +```yaml +Memory: + powermem: + type: powermem + enable_user_profile: true + llm: + provider: openai # 使用 openai 兼容模式 + config: + api_key: xxxxxxxxxxxxxxxx.xxxxxxxxxxxxxxxx + model: glm-4-flash + openai_base_url: https://open.bigmodel.cn/api/paas/v4/ + embedder: + provider: openai + config: + api_key: xxxxxxxxxxxxxxxx.xxxxxxxxxxxxxxxx + model: embedding-3 + openai_base_url: https://open.bigmodel.cn/api/paas/v4/ + vector_store: + provider: sqlite + config: {} +``` + +### 使用 OpenAI + +```yaml +Memory: + powermem: + type: powermem + enable_user_profile: true + llm: + provider: openai + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: gpt-4o-mini + openai_base_url: https://api.openai.com/v1 + embedder: + provider: openai + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: text-embedding-3-small + openai_base_url: https://api.openai.com/v1 + vector_store: + provider: sqlite + config: {} +``` + +### 使用 OceanBase(最佳性能方案) + +OceanBase 是 PowerMem 的最佳搭档,可实现最大性能释放: + +1. 部署 OceanBase 数据库(支持开源本地部署或使用云服务) + - 开源部署:https://github.com/oceanbase/oceanbase + - 云服务:https://www.oceanbase.com/ +2. 配置如下: + +```yaml +Memory: + powermem: + type: powermem + enable_user_profile: true + llm: + provider: qwen + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: qwen-plus + embedder: + provider: openai + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: text-embedding-v4 + openai_base_url: https://dashscope.aliyuncs.com/compatible-mode/v1 + vector_store: + provider: oceanbase + config: + host: 127.0.0.1 + port: 2881 + user: root@test + password: your_password + db_name: powermem + collection_name: memories # 默认值 + embedding_model_dims: 1536 # 嵌入向量维度,必需参数 +``` + +## 设备记忆隔离 + +PowerMem 会自动使用设备 ID(`device_id`)作为 `user_id` 进行记忆隔离。这意味着: + +- 每个设备拥有独立的记忆空间 +- 不同设备之间的记忆完全隔离 +- 同一设备的多次对话可以共享记忆上下文 + +## 用户画像(UserMemory) + +PowerMem 提供 `UserMemory` 类,可自动从对话中提取用户画像信息。 + +> 📌 **版本说明**:PowerMem 0.3.0+ 版本,用户画像功能支持 OceanBase、SeekDB、SQLite 三种存储后端。 + +### 启用用户画像 + +在配置中设置 `enable_user_profile: true` 即可启用: + +```yaml +Memory: + powermem: + type: powermem + enable_user_profile: true # 启用用户画像 + llm: + provider: qwen + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: qwen-plus + embedder: + provider: openai + config: + api_key: sk-xxxxxxxxxxxxxxxx + model: text-embedding-v4 + openai_base_url: https://dashscope.aliyuncs.com/compatible-mode/v1 + vector_store: + provider: sqlite # 用户画像支持: oceanbase、seekdb、sqlite + config: {} +``` + +### 用户画像能力 + +| 能力 | 说明 | +|------|------| +| **信息提取** | 自动从对话中提取姓名、年龄、职业、兴趣等 | +| **持续更新** | 随着对话进行,不断完善用户画像 | +| **画像检索** | 将用户画像与记忆搜索结合,提升检索相关性 | +| **智能遗忘** | 基于艾宾浩斯遗忘曲线,淡化过时信息 | + +### 工作原理 + +启用用户画像后,小智在查询记忆时会自动返回: +1. **用户画像**:用户的基本信息、兴趣爱好等 +2. **相关记忆**:与当前对话相关的历史记忆 + +> ✅ **版本说明**:PowerMem 0.3.0+ 版本,用户画像功能支持 OceanBase、SeekDB、SQLite 三种存储后端。 + +## 与其他记忆组件的对比 + +| 特性 | PowerMem | mem0ai | mem_local_short | +|------|----------|--------|-----------------| +| 工作方式 | 本地总结 | 云端接口 | 本地总结 | +| 存储位置 | 本地/云端DB | 云端 | 本地YAML | +| 费用 | 取决于LLM和DB | 1000次/月免费 | 完全免费 | +| 智能检索 | ✅ 向量搜索 | ✅ 向量搜索 | ❌ 全量返回 | +| 用户画像 | ✅ UserMemory | ❌ | ❌ | +| 智能遗忘 | ✅ 遗忘曲线 | ❌ | ❌ | +| 私有部署 | ✅ 支持 | ❌ 仅云端 | ✅ 支持 | +| 数据库支持 | OceanBase(推荐)/SeekDB/PostgreSQL/SQLite | - | YAML 文件 | + +## 常见问题 + +### 1. API 密钥错误 + +如果出现 `API key is required` 错误,请检查: +- `llm_api_key` 和 `embedding_api_key` 是否正确填写 +- API 密钥是否有效 + +### 2. 模型不存在 + +如果出现模型不存在的错误,请确认: +- `llm_model` 和 `embedding_model` 名称是否正确 +- 对应的模型服务是否已开通 + +### 3. 连接超时 + +如果出现连接超时,可以尝试: +- 检查网络连接 +- 如果使用代理,配置 `llm_base_url` 和 `embedding_base_url` + +## 测试验证 + +可以在虚拟环境中测试 PowerMem 是否正常工作: + +```bash +# 激活虚拟环境 +source .venv/bin/activate + +# 测试 PowerMem 导入 +python -c "from powermem import AsyncMemory; print('PowerMem 导入成功')" + +# 测试 UserMemory 导入(用户画像功能) +python -c "from powermem import UserMemory; print('UserMemory 导入成功')" +``` + +## 更多资源 + +- [PowerMem 官方文档](https://www.powermem.ai/) +- [PowerMem GitHub 仓库](https://github.com/oceanbase/powermem) +- [PowerMem 使用示例](https://github.com/oceanbase/powermem/tree/main/examples) +- [OceanBase 官网](https://www.oceanbase.com/) +- [OceanBase GitHub](https://github.com/oceanbase/oceanbase) +- [SeekDB GitHub](https://github.com/oceanbase/seekdb)(AI原生搜索数据库) +- [阿里云百炼平台](https://bailian.console.aliyun.com/) + diff --git a/main/manager-api/src/main/resources/db/changelog/202602051017.sql b/main/manager-api/src/main/resources/db/changelog/202602051017.sql new file mode 100644 index 00000000..c29347c1 --- /dev/null +++ b/main/manager-api/src/main/resources/db/changelog/202602051017.sql @@ -0,0 +1,115 @@ +-- 新增powermem记忆模型供应器 +INSERT INTO `ai_model_provider` (`id`, `model_type`, `provider_code`, `name`, `fields`, `sort`, `creator`, `create_date`, `updater`, `update_date`) +VALUES ('SYSTEM_Memory_powermem', 'Memory', 'powermem', 'PowerMem记忆', '[ + {"key":"enable_user_profile","label":"启用用户画像","type":"boolean"}, + {"key":"llm_provider","label":"LLM提供商","type":"string"}, + {"key":"llm_api_key","label":"LLM API密钥","type":"string"}, + {"key":"llm_model","label":"LLM模型","type":"string"}, + {"key":"openai_base_url","label":"OpenAI基础URL","type":"string"}, + {"key":"embedding_provider","label":"Embedding提供商","type":"string"}, + {"key":"embedding_api_key","label":"Embedding API密钥","type":"string"}, + {"key":"embedding_model","label":"Embedding模型","type":"string"}, + {"key":"embedding_openai_base_url","label":"Embedding OpenAI基础URL","type":"string"}, + {"key":"embedding_dims","label":"Embedding维度","type":"integer"}, + {"key":"vector_store","label":"向量存储配置(JSON)","type":"dict"} +]', 4, 1, NOW(), 1, NOW()); + +-- 新增PowerMem记忆模型配置 +INSERT INTO `ai_model_config` VALUES ( + 'Memory_powermem', + 'Memory', + 'powermem', + 'PowerMem记忆', + 0, + 1, + '{\"type\": \"powermem\", \"enable_user_profile\": true, \"llm_provider\": \"openai\", \"llm_api_key\": \"你的LLM API密钥\", \"llm_model\": \"qwen-plus\", \"openai_base_url\": \"\", \"embedding_provider\": \"openai\", \"embedding_api_key\": \"你的嵌入模型API密钥\", \"embedding_model\": \"text-embedding-v4\", \"embedding_openai_base_url\": \"https://api.openai.com/v1\", \"embedding_dims\": \"\", \"vector_store\": {\"provider\": \"sqlite\", \"config\": {}}}', + NULL, + NULL, + 4, + NULL, + NULL, + NULL, + NULL +); + + +-- PowerMem记忆配置说明 +UPDATE `ai_model_config` SET +`doc_link` = 'https://github.com/oceanbase/powermem', +`remark` = 'PowerMem是OceanBase开源的agent记忆组件,通过本地LLM进行记忆总结 +GitHub: https://github.com/oceanbase/powermem +官网: https://www.powermem.ai/ +使用示例: https://github.com/oceanbase/powermem/tree/main/examples + +【费用说明】 +PowerMem本身免费,实际费用取决于所选LLM和数据库: +- 使用sqlite + 免费LLM(如glm-4-flash) = 完全免费 +- 使用云端LLM或云端数据库 = 按对应服务收费 + +【enable_user_profile】用户画像功能 +- false: 使用普通记忆模式(AsyncMemory) +- true: 使用用户画像模式(UserMemory),自动提取用户信息 +- 用户画像功能支持: oceanbase、seekdb、sqlite (powermem 0.3.0+) + +【llm】LLM配置 - 用于记忆总结和用户画像提取 + provider: LLM提供商,可选值: + - qwen: 通义千问 (https://bailian.console.aliyun.com/?apiKey=1#/api-key) + - openai: OpenAI兼容接口 + - zhipu: 智谱AI (https://bigmodel.cn/usercenter/proj-mgmt/apikeys) - 推荐使用免费的glm-4-flash + config: LLM配置参数 + - api_key: API密钥 (必填) + - model: 模型名称,如 qwen-plus、glm-4-flash 等 + - openai_base_url: 自定义服务地址 (可选),如 https://api.openai.com/v1 + 示例: + {"provider": "zhipu", "config": {"api_key": "your_key", "model": "glm-4-flash"}} + {"provider": "qwen", "config": {"api_key": "your_key", "model": "qwen-plus"}} + +【embedder】Embedding配置 - 用于向量化记忆内容 + provider: 嵌入模型提供商,可选值: + - qwen: 通义千问 + - openai: OpenAI兼容接口 + config: Embedding配置参数 + - api_key: API密钥 (必填) + - model: 模型名称,如 text-embedding-v4、text-embedding-3-small 等 + - openai_base_url: 自定义服务地址 (可选) + - embedding_dims: 向量维度 (可选),非1536时需配置 + 示例: + {"provider": "openai", "config": {"api_key": "your_key", "model": "text-embedding-v4", "openai_base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1"}} + +【vector_store】数据库存储配置 - 用于存储向量化的记忆 + provider: 数据库类型,可选值: + - sqlite: 轻量级本地数据库 (推荐入门使用,无需额外配置) + - oceanbase: OceanBase数据库 (推荐生产使用,最佳性能) + - seekdb: SeekDB (推荐,AI应用存储一体) + - postgres: PostgreSQL数据库 + + SQLite配置 (无需额外配置): + {"provider": "sqlite", "config": {}} + + OceanBase配置示例: + {"provider": "oceanbase", "config": { + "host": "127.0.0.1", + "port": 2881, + "user": "root@test", + "password": "your_password", + "db_name": "powermem", + "collection_name": "memories", + "embedding_model_dims": 1024 + }} + 注意: + - collection_name: 默认表名,如创建维度错误请删除此表或更改名称 + - embedding_model_dims: 嵌入向量维度,需与embedder的模型维度匹配 + 例如智谱:embedding-2维度是1024,embedding-3维度是2048 + +【推荐配置组合】 +1. 完全免费方案: + - LLM: zhipu + glm-4-flash (免费) + - Embedder: 通义千问 text-embedding-v4 + - Database: sqlite + +2. 生产环境方案: + - LLM: qwen-plus 或其他商业模型 + - Embedder: text-embedding-v4 + - Database: oceanbase 或 seekdb +' +WHERE `id` = 'Memory_powermem'; diff --git a/main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml b/main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml index 2e9ecfd2..ded39ce1 100755 --- a/main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml +++ b/main/manager-api/src/main/resources/db/changelog/db.changelog-master.yaml @@ -515,6 +515,13 @@ databaseChangeLog: - sqlFile: encoding: utf8 path: classpath:db/changelog/202602021555.sql + - changeSet: + id: 202602051017 + author: RanChen + changes: + - sqlFile: + encoding: utf8 + path: classpath:db/changelog/202602051017.sql - changeSet: id: 202602051125 author: DaGou12138 diff --git a/main/xiaozhi-server/config.yaml b/main/xiaozhi-server/config.yaml index c52d38ca..51701d15 100644 --- a/main/xiaozhi-server/config.yaml +++ b/main/xiaozhi-server/config.yaml @@ -279,6 +279,54 @@ Memory: # https://app.mem0.ai/dashboard/api-keys # 每月有1000次免费调用 api_key: 你的mem0ai api key + powermem: + # PowerMem是OceanBase开源的agent记忆组件,通过本地LLM进行记忆总结 + # 费用说明:PowerMem本身免费,实际费用取决于所选LLM和数据库 + # - 使用sqlite + 免费LLM(如glm-4-flash) = 完全免费 + # - 使用云端LLM或云端数据库 = 按对应服务收费 + # GitHub: https://github.com/oceanbase/powermem + # 官网: https://www.powermem.ai/ + # 使用示例: https://github.com/oceanbase/powermem/tree/main/examples + type: powermem + # 是否启用用户画像功能 + # - false: 使用普通记忆模式(AsyncMemory) + # - true: 使用用户画像模式(UserMemory),自动提取用户信息 + # 用户画像功能支持: oceanbase、seekdb、sqlite (powermem 0.3.0+) + enable_user_profile: true + # ========== LLM 配置 ========== + # 通义千问: https://bailian.console.aliyun.com/?apiKey=1#/api-key + # 智谱AI(免费): https://bigmodel.cn/usercenter/proj-mgmt/apikeys + llm: + provider: openai # 可选: qwen, openai, zhipu 等 + config: + api_key: 你的LLM API密钥 + model: qwen-plus + # openai_base_url: https://api.openai.com/v1 # 可选,自定义服务地址 + # ========== Embedding 配置 ========== + embedder: + provider: openai # 可选: qwen, openai 等 + config: + api_key: 你的嵌入模型API密钥 + model: text-embedding-v4 + openai_base_url: https://dashscope.aliyuncs.com/compatible-mode/v1 + # embedding_dims: 1024 # 向量维度,非1536时需配置 + + # ========== Database 配置 ========== + # oceanbase(推荐,最佳性能), seekdb(推荐,AI应用存储一体), postgres, sqlite(轻量备选) + # 用户画像功能支持: oceanbase、seekdb、sqlite + vector_store: + provider: sqlite + config: {} # SQLite 无需额外配置 + # OceanBase 配置示例: + # provider: oceanbase + # config: + # host: 127.0.0.1 + # port: 2881 + # user: root@test + # password: your_password + # db_name: powermem + # collection_name: memories # 默认表名,如创建维度错误请删除此表或更改名称 + # embedding_model_dims: 1024 # 嵌入向量维度,注意跟使用模型适配,以智普为例:embedding-2的维度是1024,embedding-3的维度是2048 nomem: # 不想使用记忆功能,可以使用nomem type: nomem diff --git a/main/xiaozhi-server/core/connection.py b/main/xiaozhi-server/core/connection.py index ecb3b0cd..6e8f5a8b 100644 --- a/main/xiaozhi-server/core/connection.py +++ b/main/xiaozhi-server/core/connection.py @@ -841,7 +841,8 @@ class ConnectionHandler: try: # 使用带记忆的对话 memory_str = None - if self.memory is not None: + # 仅当query非空(代表用户询问)时查询记忆 + if self.memory is not None and query: future = asyncio.run_coroutine_threadsafe( self.memory.query_memory(query), self.loop ) diff --git a/main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py b/main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py index 7156ab72..80dbfeb5 100644 --- a/main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py +++ b/main/xiaozhi-server/core/providers/memory/mem0ai/mem0ai.py @@ -1,3 +1,4 @@ +import json import traceback from ..base import MemoryProviderBase, logger @@ -36,11 +37,26 @@ class MemoryProvider(MemoryProviderBase): try: # Format the content as a message list for mem0 - messages = [ - {"role": message.role, "content": message.content} - for message in msgs - if message.role != "system" - ] + messages = [] + for message in msgs: + if message.role == "system": + continue + + content = message.content + + # Extract content from JSON format if present (for ASR with emotion/language tags) + # Same logic as in query_memory method + try: + if content and content.strip().startswith("{") and content.strip().endswith("}"): + data = json.loads(content) + if "content" in data: + content = data["content"] + except (json.JSONDecodeError, KeyError, TypeError): + # If parsing fails, use original content + pass + + messages.append({"role": message.role, "content": content}) + result = self.client.add(messages, user_id=self.role_id) logger.bind(tag=TAG).debug(f"Save memory result: {result}") except Exception as e: @@ -56,7 +72,16 @@ class MemoryProvider(MemoryProviderBase): filters = {"user_id": self.role_id} - results = self.client.search(query, filters=filters) + search_query = query + try: + if query.strip().startswith("{") and query.strip().endswith("}"): + data = json.loads(query) + if "content" in data: + search_query = data["content"] + except (json.JSONDecodeError, KeyError): + pass + + results = self.client.search(search_query, filters=filters) if not results or "results" not in results: return "" diff --git a/main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py b/main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py index 2d0820f0..936b3b41 100644 --- a/main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py +++ b/main/xiaozhi-server/core/providers/memory/mem_local_short/mem_local_short.py @@ -149,10 +149,22 @@ class MemoryProvider(MemoryProviderBase): msgStr = "" for msg in msgs: + content = msg.content + + # Extract content from JSON format if present (for ASR with emotion/language tags) + try: + if content and content.strip().startswith("{") and content.strip().endswith("}"): + data = json.loads(content) + if "content" in data: + content = data["content"] + except (json.JSONDecodeError, KeyError, TypeError): + # If parsing fails, use original content + pass + if msg.role == "user": - msgStr += f"User: {msg.content}\n" + msgStr += f"User: {content}\n" elif msg.role == "assistant": - msgStr += f"Assistant: {msg.content}\n" + msgStr += f"Assistant: {content}\n" if self.short_memory and len(self.short_memory) > 0: msgStr += "历史记忆:\n" msgStr += self.short_memory diff --git a/main/xiaozhi-server/core/providers/memory/powermem/powermem.py b/main/xiaozhi-server/core/providers/memory/powermem/powermem.py new file mode 100644 index 00000000..5c453c07 --- /dev/null +++ b/main/xiaozhi-server/core/providers/memory/powermem/powermem.py @@ -0,0 +1,346 @@ +#!/usr/bin/env python +# -*- coding: UTF-8 -*- +""" +@time: 2026/01/08 +@file: powermem.py +@desc: PowerMem memory provider for xiaozhi-esp32-server + PowerMem is an open-source agent memory component from OceanBase + GitHub: https://github.com/oceanbase/powermem + Website: https://www.powermem.ai/ +@Author: wayyoungboy +""" + +import asyncio +import json +import traceback +from typing import Optional, Dict, Any + +from ..base import MemoryProviderBase, logger + +TAG = __name__ + + +class MemoryProvider(MemoryProviderBase): + """ + PowerMem memory provider implementation. + + PowerMem is an open-source agent memory component that provides + efficient memory management for AI agents. + + Supports multiple storage backends (sqlite, oceanbase, postgres), + LLM providers (qwen, openai, etc.) and embedding providers. + + Config options: + - enable_user_profile: bool - Enable UserMemory for user profiling (requires OceanBase) + - database_provider: str - Storage backend (sqlite, oceanbase, postgres) + - llm_provider: str - LLM provider (qwen, openai, etc.) + - embedding_provider: str - Embedding provider (qwen, openai, etc.) + """ + + def __init__(self, config: Dict[str, Any], summary_memory: Optional[str] = None): + super().__init__(config) + self.use_powermem = False + self.memory_client = None + self.enable_user_profile = False + self.last_profile_content = "" # Cache for user profile from UserMemory + + try: + # Check if user profile mode is enabled + self.enable_user_profile = config.get("enable_user_profile", False) + + # Get configuration parameters + database_provider = config.get("database_provider", "sqlite") + llm_provider = config.get("llm_provider", "qwen") + embedding_provider = config.get("embedding_provider", "qwen") + + # Build powermem configuration dict + # PowerMem supports two config styles: + # 1. powermem style: database, llm, embedding + # 2. mem0 style: vector_store, llm, embedder + powermem_config = {} + + # Configure vector store / database + if "vector_store" in config: + powermem_config["vector_store"] = config["vector_store"] + elif "database" in config: + powermem_config["database"] = config["database"] + else: + powermem_config["vector_store"] = { + "provider": database_provider, + "config": {} + } + + # Configure LLM + if "llm" in config: + powermem_config["llm"] = config["llm"] + else: + llm_config = {} + if "llm_api_key" in config: + llm_config["api_key"] = config["llm_api_key"] + if "llm_model" in config: + llm_config["model"] = config["llm_model"] + # Handle base_url based on provider type + # - qwen provider uses dashscope_base_url + # - openai provider uses openai_base_url + if "llm_base_url" in config: + if llm_provider == "qwen": + llm_config["dashscope_base_url"] = config["llm_base_url"] + else: + llm_config["openai_base_url"] = config["llm_base_url"] + if "openai_base_url" in config: + llm_config["openai_base_url"] = config["openai_base_url"] + if "dashscope_base_url" in config: + llm_config["dashscope_base_url"] = config["dashscope_base_url"] + powermem_config["llm"] = { + "provider": llm_provider, + "config": llm_config + } + + # Configure embedder + if "embedder" in config: + powermem_config["embedder"] = config["embedder"] + else: + embedder_config = {} + if "embedding_api_key" in config: + embedder_config["api_key"] = config["embedding_api_key"] + if "embedding_model" in config: + embedder_config["model"] = config["embedding_model"] + # Handle base_url based on provider type + # - qwen provider uses dashscope_base_url + # - openai provider uses openai_base_url + # Priority: embedding_xxx_base_url > embedding_base_url > xxx_base_url + if "embedding_base_url" in config: + if embedding_provider == "qwen": + embedder_config["dashscope_base_url"] = config["embedding_base_url"] + else: + embedder_config["openai_base_url"] = config["embedding_base_url"] + # Embedding-specific base_url (higher priority) + if "embedding_openai_base_url" in config: + embedder_config["openai_base_url"] = config["embedding_openai_base_url"] + if "embedding_dashscope_base_url" in config: + embedder_config["dashscope_base_url"] = config["embedding_dashscope_base_url"] + powermem_config["embedder"] = { + "provider": embedding_provider, + "config": embedder_config + } + + # Initialize memory client based on mode + if self.enable_user_profile: + from powermem import UserMemory + self.memory_client = UserMemory(config=powermem_config) + memory_mode = "UserMemory (用户画像模式)" + else: + from powermem import AsyncMemory + self.memory_client = AsyncMemory(config=powermem_config) + memory_mode = "AsyncMemory (普通记忆模式)" + + self.use_powermem = True + + logger.bind(tag=TAG).info( + f"PowerMem initialized successfully: mode={memory_mode}, " + f"database={powermem_config['vector_store']['provider']}, llm={powermem_config['llm']['provider']}, embedding={powermem_config['embedder']['provider']}" + ) + + except ImportError as e: + logger.bind(tag=TAG).error( + f"PowerMem not installed. Please install with: pip install powermem. Error: {e}" + ) + self.use_powermem = False + except Exception as e: + logger.bind(tag=TAG).error(f"Failed to initialize PowerMem: {str(e)}") + logger.bind(tag=TAG).debug(f"Detailed error: {traceback.format_exc()}") + self.use_powermem = False + + async def save_memory(self, msgs, session_id=None): + """ + Save conversation messages to PowerMem. + + Args: + msgs: List of message objects with 'role' and 'content' attributes + + session_id: Session identifier (optional, for compatibility) + + Returns: + Result from PowerMem API or None if failed + """ + if not self.use_powermem or self.memory_client is None: + logger.bind(tag=TAG).warning("PowerMem is not available, skipping save_memory") + return None + + if len(msgs) < 2: + logger.bind(tag=TAG).debug("Not enough messages to save (need at least 2)") + return None + + try: + # Format the content as a message list for PowerMem + messages = [] + for message in msgs: + if message.role == "system": + continue + + content = message.content + + # Extract content from JSON format if present (for ASR with emotion/language tags) + # Same logic as in query_memory method + try: + if content and content.strip().startswith("{") and content.strip().endswith("}"): + data = json.loads(content) + if "content" in data: + content = data["content"] + except (json.JSONDecodeError, KeyError, TypeError): + # If parsing fails, use original content + pass + + messages.append({"role": message.role, "content": content}) + + # Add memory using PowerMem SDK + result = self.memory_client.add( + messages=messages, + user_id=self.role_id + ) + # Handle both sync and async returns + if asyncio.iscoroutine(result): + result = await result + + logger.bind(tag=TAG).debug(f"Save memory result: {result}") + + # Cache user profile if UserMemory mode and profile was extracted + if self.enable_user_profile and result: + if result.get('profile_extracted'): + self.last_profile_content = result.get('profile_content', '') + logger.bind(tag=TAG).debug(f"User profile extracted: {self.last_profile_content}") + + return result + + except Exception as e: + logger.bind(tag=TAG).error(f"Error saving memory: {str(e)}") + logger.bind(tag=TAG).debug(f"Detailed error: {traceback.format_exc()}") + return None + + async def query_memory(self, query: str) -> str: + """ + Query memories from PowerMem based on similarity search. + + Args: + query: The search query string (may be JSON format with metadata) + + Returns: + Formatted string of relevant memories or empty string if none found + """ + if not self.use_powermem or self.memory_client is None: + logger.bind(tag=TAG).warning("PowerMem is not available, skipping query_memory") + return "" + + try: + if not getattr(self, "role_id", None): + logger.bind(tag=TAG).debug("No role_id set, returning empty memory") + return "" + + # Extract content from JSON format if present (for ASR with emotion/language tags) + search_query = query + try: + if query.strip().startswith("{") and query.strip().endswith("}"): + data = json.loads(query) + if "content" in data: + search_query = data["content"] + except (json.JSONDecodeError, KeyError): + # If parsing fails, use original query + pass + + result_parts = [] + + # If user profile mode is enabled, include user profile in results + if self.enable_user_profile: + profile = await self.get_user_profile() + if profile: + result_parts.append(f"【用户画像】\n{profile}") + + # Search memories using PowerMem SDK + if self.enable_user_profile: + # UserMemory uses sync search + results = await asyncio.to_thread( + self.memory_client.search, + query=search_query, + user_id=self.role_id, + limit=30 + ) + else: + # AsyncMemory uses async search + results = await self.memory_client.search( + query=search_query, + user_id=self.role_id, + limit=30 + ) + + if results and "results" in results: + # Format each memory entry with its update time + memories = [] + for entry in results.get("results", []): + # Get timestamp from updated_at or created_at + timestamp = "" + if "updated_at" in entry and entry["updated_at"]: + timestamp = str(entry["updated_at"]) + elif "created_at" in entry and entry["created_at"]: + timestamp = str(entry["created_at"]) + + if timestamp: + try: + # Parse and reformat the timestamp (remove milliseconds if present) + if "." in timestamp: + dt = timestamp.split(".")[0] + else: + dt = timestamp + formatted_time = dt.replace("T", " ") + except Exception: + formatted_time = timestamp + else: + formatted_time = "" + + memory = entry.get("memory", "") or entry.get("content", "") + if memory: + if formatted_time: + # Store tuple of (timestamp, formatted_string) for sorting + memories.append((timestamp, f"[{formatted_time}] {memory}")) + else: + memories.append(("", memory)) + + # Sort by timestamp in descending order (newest first) + memories.sort(key=lambda x: x[0], reverse=True) + + # Extract only the formatted strings + if memories: + memories_str = "\n".join(f"- {memory[1]}" for memory in memories) + result_parts.append(f"【相关记忆】\n{memories_str}") + + final_result = "\n\n".join(result_parts) + logger.bind(tag=TAG).debug(f"Query results: {final_result}") + return final_result + + except Exception as e: + logger.bind(tag=TAG).error(f"Error querying memory: {str(e)}") + logger.bind(tag=TAG).debug(f"Detailed error: {traceback.format_exc()}") + return "" + + async def get_user_profile(self) -> str: + """ + Get user profile from PowerMem (only available in UserMemory mode). + + In PowerMem 0.3.0+, user profile is automatically extracted during add() + and cached in last_profile_content. + + Returns: + Formatted user profile string or empty string if not available + """ + if not self.use_powermem or self.memory_client is None: + return "" + + if not self.enable_user_profile: + logger.bind(tag=TAG).debug("User profile mode is not enabled") + return "" + + # Return cached profile content from last add() operation + if self.last_profile_content: + return self.last_profile_content + + return "" + diff --git a/main/xiaozhi-server/requirements.txt b/main/xiaozhi-server/requirements.txt index cc4cd8fe..b2b2565e 100644 --- a/main/xiaozhi-server/requirements.txt +++ b/main/xiaozhi-server/requirements.txt @@ -23,6 +23,7 @@ loguru==0.7.3 requests==2.32.5 cozepy==0.20.0 mem0ai==1.0.0 +powermem>=0.3.1 bs4==0.0.2 modelscope==1.32.0 sherpa_onnx==1.12.17