fix: 修复启动脚本等待时长

加入联网等待延迟

README.md

@@ -26,9 +26,9 @@
 
 ## 你的声音 + 小爱音箱 = 无限可能
 
-👉 [小爱音箱接入小智 AI 演示视频](https://www.bilibili.com/video/BV1NBXWYSEvX)
+👉 [小爱音箱接入小智 AI 演示视频](https://www.bilibili.com/video/BV1TxJhzvEhz)
 
-[![](./docs/images/xiaozhi.jpg)](https://www.bilibili.com/video/BV1NBXWYSEvX)
+[![](./docs/images/xiaozhi.jpg)](https://www.bilibili.com/video/BV1TxJhzvEhz)
 
 👉 [小爱音箱自定义唤醒词演示视频](https://www.bilibili.com/video/BV1YfVUz5EMj)
 
@@ -53,7 +53,7 @@
    - 👉 [小爱音箱接入 MiGPT（完美版）](examples/migpt/README.md)
    - 👉 [小爱音箱接入 Gemini Live API](examples/gemini/README.md)
 
-以上皆为抛砖引玉，你也可以亲手编写自己想要的功能（比如：Airplay、DLNA、开启 DTS 音效等），一切由你定义！
+以上皆为抛砖引玉，你也可以亲手编写自己想要的功能，一切由你定义！
 
 ## 相关项目
 

examples/gemini/README.md

@@ -18,6 +18,12 @@
 > 请先确认你已经将小爱音箱刷机成功，并安装运行了 Rust 补丁程序 [👉 教程](../../packages/client-rust/README.md)，否则该项目启动后收不到音频输入，将无法正常工作。
 
 ```bash
+# 克隆代码
+git clone https://github.com/idootop/open-xiaoai.git
+
+# 进入当前项目根目录
+cd examples/gemini
+
 # 安装 Python 依赖
 uv sync --locked
 

examples/kws/README.md

@@ -146,4 +146,12 @@ curl -sSfL https://gitee.com/idootop/artifacts/releases/download/open-xiaoai-kws
 
 由于小爱音箱 client 端算力和存储空间有限，默认只使用了一个较小的[语音识别模型](https://k2-fsa.github.io/sherpa/onnx/kws/pretrained_models/index.html#sherpa-onnx-kws-pre-trained-models)（5MB 左右），需要占用大约 20 MB 的存储空间。其他型号的小爱音箱存储空间和算力可能不足以运行该模型，请自行测试。
 
-你也可以参考 [open-xiaoai](https://github.com/idootop/open-xiaoai) 里的演示程序，在 server 端运行更大规模、更先进的 AI 模型，来进行唤醒词识别，达到相同目的。推荐使用 [FunASR](https://github.com/modelscope/FunASR) 和 [Sherpa-ONNX](https://github.com/k2-fsa/sherpa-onnx)，可以参考 [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) 项目。
+你也可以参考 [open-xiaoai](https://github.com/idootop/open-xiaoai) 里的演示程序，在 server 端运行更大规模、更先进的 AI 模型，来进行唤醒词识别，达到相同目的。推荐使用 [FunASR](https://github.com/modelscope/FunASR) 和 [Sherpa-ONNX](https://github.com/k2-fsa/sherpa-onnx)，可以参考 [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) 项目。
+
+### Q：我是小爱音箱 Pro（LX06）为什么启动失败显示 Killed？
+
+目前只在新款小米智能音箱 Pro（OH2P）和旧款带 DTS 功能的小爱音箱 Pro（256 MB 内存）上测试可以正常运行。
+
+有些新版的小爱音箱 Pro（不带 DTS 版本）内存仅为 128 MB，无法提供充足的内存来加载模型文件，故无法在 Client 端运行该 KWS 服务。
+
+建议参考 [xiaozhi](../xiaozhi/README.md) 演示，在 Server 端运行 KWS 服务。

examples/kws/boot.sh

@@ -1,7 +1,24 @@
 #! /bin/sh
 
+exec > /dev/null 2>&1
+
+cat << 'EOF'
+
+▄▖      ▖▖▘    ▄▖▄▖
+▌▌▛▌█▌▛▌▚▘▌▀▌▛▌▌▌▐ 
+▙▌▙▌▙▖▌▌▌▌▌█▌▙▌▛▌▟▖
+  ▌                 
+
+v1.0.0  by: https://del.wang
+
+EOF
+
 set -e
 
+echo "🤫 等待网络连接中..."
+
+sleep 5
+
 MIN_SPACE_MB=32
 DOWNLOAD_BASE_URL="https://gitee.com/idootop/artifacts/releases/download/open-xiaoai-kws"
 

examples/kws/debug.sh

@@ -1,5 +1,16 @@
 #! /bin/sh
 
+cat << 'EOF'
+
+▄▖      ▖▖▘    ▄▖▄▖
+▌▌▛▌█▌▛▌▚▘▌▀▌▛▌▌▌▐ 
+▙▌▙▌▙▖▌▌▌▌▌█▌▙▌▛▌▟▖
+  ▌                 
+
+v1.0.0  by: https://del.wang
+
+EOF
+
 set -e
 
 MIN_SPACE_MB=32

examples/kws/init.sh

@@ -1,5 +1,16 @@
 #! /bin/sh
 
+cat << 'EOF'
+
+▄▖      ▖▖▘    ▄▖▄▖
+▌▌▛▌█▌▛▌▚▘▌▀▌▛▌▌▌▐ 
+▙▌▙▌▙▖▌▌▌▌▌█▌▙▌▛▌▟▖
+  ▌                 
+
+v1.0.0  by: https://del.wang
+
+EOF
+
 set -e
 
 MIN_SPACE_MB=32

examples/xiaozhi/README.md

@@ -2,9 +2,12 @@
 
 [Open-XiaoAI](https://github.com/idootop/open-xiaoai) 的 Python 版 Server 端，用来演示小爱音箱接入[小智 AI](https://github.com/78/xiaozhi-esp32)。
 
+> [!IMPORTANT]
+> 本项目只是一个简单的演示程序，抛砖引玉。诸如一些音频压缩、加密传输、多账号管理等功能并未提供，建议只在局域网内测试运行，不推荐部署在公网服务器上（消耗流量 100kb/s），请自行评估相关风险，合理使用。
+
 - 小爱音箱接入小智 AI
-- 自定义唤醒词（中英文）和提示语
 - 支持连续对话和中途打断
+- 自定义唤醒词（中英文）和提示语
 - 支持自定义消息处理，方便个人定制
 
 ## 快速开始
@@ -63,8 +66,11 @@ docker run -it --rm -p 4399:4399 -v $(pwd)/config.py:/app/config.py idootop/open
 # 安装 Python 依赖
 uv sync --locked
 
-# 编译运行
+# 编译运行（GUI 模式，不支持唤醒词唤醒）
 uv run main.py
+
+# 或者设置环境变量 CLI=true，开启 CLI 模式（支持自定义唤醒词）
+CLI=true uv run main.py
 ```
 
 如果你只是想体验一下小智 AI，请使用以下命令启动：
@@ -75,9 +81,6 @@ uv run main.py --mode xiaozhi
 
 该模式下使用电脑的麦克风和扬声器作为音频输入输出设备，无需连接小爱音箱。
 
-> [!NOTE]
-> 本项目只是一个简单的演示程序，抛砖引玉。诸如一些音频压缩、加密传输、多账号管理等功能并未提供，请自行评估用途和使用风险。
-
 ## 常见问题
 
 ### Q：回答太长了，如何打断小智 AI 的回答？
@@ -131,10 +134,6 @@ APP_CONFIG = {
 
 文字识别结果取决于你的小智 AI 服务器端的语音识别方案，与本项目无关。
 
-不过需要注意的是，在唤醒后或连续对话时，由于 VAD 激活阈值的关系，开始的一小段语音可能会被丢弃识别不到，使用时需要多注意。
-
-比如：“你知道我是谁吗” 有可能会被识别为“知道我是谁吗”（丢掉了最前面一个字）
-
 ### Q：唤醒词一直没有反应？
 
 由于小爱音箱远场拾音音量较小，有时可能会识别不清，你可以调大 `config.py` 配置文件里的 `boost` 参数，然后重启应用 / Docker 试试看。
@@ -157,6 +156,10 @@ APP_CONFIG = {
 
 PS：如果还是不行，建议更换其他更易识别的唤醒词。
 
+### Q: 我想自己编译运行，模型文件在哪里下载？
+
+由于 ASR 相关模型文件体积较大，并未直接提交在 git 仓库中，你可以在 release 中下载 [VAD + KWS 相关模型](https://github.com/idootop/open-xiaoai/releases/tag/vad-kws-models)，然后解压到 `xiaozhi/models` 路径下即可。
+
 ## 鸣谢
 
 该演示使用的 Python 版小智 AI 客户端基于 [py-xiaozhi](https://github.com/Huang-junsen/py-xiaozhi) 项目，特此鸣谢。

examples/xiaozhi/config.py

@@ -9,19 +9,12 @@ async def before_wakeup(speaker, text, source):
         - 'kws': 关键字唤醒
         - 'xiaoai': 小爱同学收到用户指令
     """
-    if source == "kws" and text == "你好小智":
+    if source == "kws":
         # 播放唤醒提示语
         await speaker.play(text="你好主人，我是小智，请问有什么吩咐？")
         # 返回 True 唤醒小智 AI
         return True
 
-    if source == "kws" and text == "hi siri":
-        # 播放唤醒提示语
-        await speaker.play(text="I'm Siri, how can I help you?")
-        # 唤醒小爱
-        await speaker.wake_up()
-        return False
-
     if source == "xiaoai" and text == "召唤小智":
         # 打断原来的小爱同学
         await speaker.abort_xiaoai()

examples/xiaozhi/xiaozhi/event.py

@@ -1,8 +1,17 @@
 import asyncio
 
 from config import APP_CONFIG
-from xiaozhi.ref import get_audio_codec, get_speaker, get_vad, get_xiaoai, get_xiaozhi
+from xiaozhi.ref import (
+    get_audio_codec,
+    get_kws,
+    get_speaker,
+    get_vad,
+    get_xiaoai,
+    get_xiaozhi,
+    set_speech_frames,
+)
 from xiaozhi.services.protocols.typing import AbortReason, DeviceState, ListeningMode
+from xiaozhi.utils.base import get_env
 
 
 class Step:
@@ -22,7 +31,7 @@ class __EventManager:
         self.next_step_future = None
 
     def update_step(self, step: Step, step_data=None):
-        if get_xiaoai().mode == "xiaozhi":
+        if not get_env("CLI"):
             return
 
         self.current_step = step
@@ -32,7 +41,7 @@ class __EventManager:
             )
             self.next_step_future = None
 
-    async def wait_next_step(self, timeout=None) -> Step | None:
+    async def wait_next_step(self, timeout=None):
         current_session = self.session_id
 
         self.next_step_future = get_xiaoai().async_loop.create_future()
@@ -70,16 +79,16 @@ class __EventManager:
         self.update_step(Step.on_wakeup)
         self.start_session()
 
-    def on_tts_end(self):
+    def on_tts_end(self, session_id):
         """TTS结束"""
-        if self.current_step == Step.on_interrupt:
+        if self.current_step in [Step.on_interrupt, Step.on_tts_end]:
             # 当前 session 已经被打断了，不再处理
             return
         self.session_id = self.session_id + 1
         self.update_step(Step.on_tts_end)
         self.start_session()
 
-    def on_tts_start(self):
+    def on_tts_start(self, session_id):
         """TTS结束"""
         self.update_step(Step.on_tts_start)
 
@@ -97,7 +106,7 @@ class __EventManager:
         )
 
     async def __start_session(self):
-        if get_xiaoai().mode == "xiaozhi":
+        if not get_env("CLI"):
             return
 
         vad = get_vad()
@@ -113,11 +122,6 @@ class __EventManager:
         if self.current_step == Step.on_interrupt:
             return
 
-        # 尝试打开音频通道
-        if not xiaozhi.protocol.is_audio_channel_opened():
-            xiaozhi.set_device_state(DeviceState.CONNECTING)
-            await xiaozhi.protocol.open_audio_channel()
-
         # 等待 TTS 余音结束
         if self.current_step in [Step.on_tts_end]:
             vad.resume("silence")
@@ -141,8 +145,9 @@ class __EventManager:
             return
 
         # 开始说话
+        set_speech_frames(speech_buffer)
+        codec.input_stream.start_stream()  # 开启录音
         await xiaozhi.protocol.send_start_listening(ListeningMode.MANUAL)
-        codec.input_stream.input(speech_buffer)  # 追加音频输入片段
         xiaozhi.set_device_state(DeviceState.LISTENING)
 
         # 等待说话结束
@@ -155,5 +160,13 @@ class __EventManager:
         await xiaozhi.protocol.send_stop_listening()
         xiaozhi.set_device_state(DeviceState.IDLE)
 
+    async def wakeup(self, text, source):
+        before_wakeup = APP_CONFIG["wakeup"]["before_wakeup"]
+        get_kws().pause()  # 暂停 KWS 检测
+        wakeup = await before_wakeup(get_speaker(), text, source)
+        get_kws().resume()  # 恢复 KWS 检测
+        if wakeup:
+            self.on_wakeup()
+
 
 EventManager = __EventManager()

examples/xiaozhi/xiaozhi/ref.py

@@ -49,3 +49,11 @@ def set_kws(kws: Any):
 
 def get_kws() -> Any:
     return GLOBAL_STATES.get("kws")
+
+
+def set_speech_frames(speech_frames: Any):
+    GLOBAL_STATES["speech_frames"] = speech_frames
+
+
+def get_speech_frames() -> Any:
+    return GLOBAL_STATES.get("speech_frames")

examples/xiaozhi/xiaozhi/services/audio/codec.py

@@ -1,8 +1,14 @@
 import opuslib_next as opuslib
 
-from xiaozhi.ref import set_audio_codec
+from xiaozhi.ref import (
+    get_speech_frames,
+    get_xiaozhi,
+    set_audio_codec,
+    set_speech_frames,
+)
 from xiaozhi.services.audio.stream import MyAudio
 from xiaozhi.services.protocols.typing import AudioConfig
+from xiaozhi.utils.base import get_env
 
 
 class AudioCodec:
@@ -36,11 +42,11 @@ class AudioCodec:
 
         # 初始化音频输出流
         self.output_stream = self.audio.open(
+            output=True,
             format=AudioConfig.FORMAT,
             channels=AudioConfig.CHANNELS,
-            rate=AudioConfig.SAMPLE_RATE,
-            output=True,
-            frames_per_buffer=AudioConfig.FRAME_SIZE,
+            rate=get_xiaozhi().protocol.server_sample_rate,
+            frames_per_buffer=get_xiaozhi().protocol.server_frame_size,
             output_device_index=MyAudio.get_output_device_index(self.audio),
         )
 
@@ -53,18 +59,36 @@ class AudioCodec:
 
         # 初始化Opus解码器
         self.opus_decoder = opuslib.Decoder(
-            fs=AudioConfig.SAMPLE_RATE, channels=AudioConfig.CHANNELS
+            fs=get_xiaozhi().protocol.server_sample_rate,
+            channels=AudioConfig.CHANNELS,
         )
+        self.temp_frames = bytes([])
 
     def read_audio(self):
         """读取音频输入数据并编码"""
         try:
+            speech_frames = get_speech_frames()
+
+            # 加入语音片段
+            if speech_frames:
+                self.temp_frames = speech_frames
+                set_speech_frames([])
+
+            # 读取音频输入数据
             data = self.input_stream.read(
-                AudioConfig.FRAME_SIZE, exception_on_overflow=False
+                num_frames=None if get_env("CLI") else AudioConfig.FRAME_SIZE,
+                exception_on_overflow=False,
             )
             if not data:
                 return None
-            return self.opus_encoder.encode(data, AudioConfig.FRAME_SIZE)
+
+            self.temp_frames += data
+            if len(self.temp_frames) < AudioConfig.FRAME_SIZE * 2:
+                return None
+
+            opus_frames, remain_frames = self.encode_audio(self.temp_frames)
+            self.temp_frames = remain_frames
+            return opus_frames
         except Exception:
             return None
 
@@ -76,24 +100,29 @@ class AudioCodec:
         except Exception:
             pass
 
-    def decode_audio(self, opus_data, frame_size=AudioConfig.FRAME_SIZE):
+    def decode_audio(self, opus_data):
         """解码音频数据"""
-        return self.opus_decoder.decode(opus_data, frame_size, decode_fec=False)
+        return self.opus_decoder.decode(
+            opus_data,
+            frame_size=get_xiaozhi().protocol.server_frame_size,
+            decode_fec=False,
+        )
 
-    def encode_audio(self, buffer, frame_size=AudioConfig.FRAME_SIZE):
+    def encode_audio(self, buffer: bytes, frame_size=AudioConfig.FRAME_SIZE):
         """编码音频数据"""
+        opus_frames = []
+        remain_frames = bytes([])
         try:
-            opus_frames = []
             for i in range(0, len(buffer), frame_size * 2):
                 chunk = buffer[i : i + frame_size * 2]
                 if len(chunk) < frame_size * 2:
-                    # 如果 buffer 长度不是 FRAME_SIZE 的 2 倍，需要补齐
-                    chunk += b"\x00" * (frame_size * 2 - len(chunk))
+                    remain_frames = chunk
+                    break
                 opus_frame = self.opus_encoder.encode(chunk, frame_size)
                 opus_frames.append(opus_frame)
-            return opus_frames
+            return opus_frames, remain_frames
         except Exception:
-            return None
+            return None, remain_frames
 
     def start_streams(self):
         """启动音频流"""

examples/xiaozhi/xiaozhi/services/audio/kws/__init__.py

@@ -31,6 +31,7 @@ class _KWS:
         )
 
         # 启动 KWS 服务
+        self.paused = False
         self.thread = threading.Thread(target=self._detection_loop, daemon=True)
         self.thread.start()
 
@@ -38,6 +39,12 @@ class _KWS:
         current_dir = os.path.dirname(os.path.abspath(__file__))
         return os.path.join(current_dir, "../../../models", file_name)
 
+    def pause(self):
+        self.paused = True
+
+    def resume(self):
+        self.paused = False
+
     def _detection_loop(self):
         SherpaOnnx.start()
         self.stream.start_stream()
@@ -46,10 +53,15 @@ class _KWS:
             frames = self.stream.read()
 
             # 在说话和监听状态时，暂停 KWS
-            if not frames or get_xiaozhi().device_state in [
-                DeviceState.LISTENING,
-                DeviceState.SPEAKING,
-            ]:
+            if (
+                not frames
+                or self.paused
+                or get_xiaozhi().device_state
+                in [
+                    DeviceState.LISTENING,
+                    DeviceState.SPEAKING,
+                ]
+            ):
                 time.sleep(0.01)
                 continue
 
@@ -60,14 +72,9 @@ class _KWS:
 
     def on_message(self, text: str):
         asyncio.run_coroutine_threadsafe(
-            self._on_message(text), get_xiaoai().async_loop
+            EventManager.wakeup(text, "kws"),
+            get_xiaoai().async_loop,
         )
 
-    async def _on_message(self, text: str):
-        before_wakeup = APP_CONFIG["wakeup"]["before_wakeup"]
-        wakeup = await before_wakeup(get_speaker(), text, "kws")
-        if wakeup:
-            EventManager.on_wakeup()
-
 
 KWS = _KWS()

examples/xiaozhi/xiaozhi/services/audio/stream.py

@@ -74,7 +74,7 @@ class MyStream:
             self._is_active = False
             if self._is_input:
                 GlobalStream.unregister_reader(self)
-                self.input_bytes = []
+                self.input_bytes.clear()
 
     def write(self, frames: bytes) -> None:
         # 发送输出音频流到扬声器
@@ -96,7 +96,7 @@ class MyStream:
     def read(self, num_frames=None, exception_on_overflow=False) -> bytes:
         if num_frames is None:
             data = bytes(self.input_bytes)
-            self.input_bytes = []
+            self.input_bytes.clear()
             return data
 
         num_frames = num_frames * 2

examples/xiaozhi/xiaozhi/services/audio/vad/__init__.py

@@ -1,20 +1,16 @@
 import threading
 import time
 
-import numpy as np
-
 from config import APP_CONFIG
 from xiaozhi.event import EventManager
 from xiaozhi.ref import set_vad
 from xiaozhi.services.audio.stream import MyAudio
-from xiaozhi.services.audio.vad.silero import VAD_MODEL
+from xiaozhi.services.audio.vad.silero import Silero
 from xiaozhi.services.protocols.typing import AudioConfig
 from xiaozhi.utils.base import get_env
 
 
 class _VAD:
-    """基于WebRTC VAD的语音活动检测器，用于检测用户打断"""
-
     def __init__(self):
         set_vad(self)
 
@@ -33,36 +29,30 @@ class _VAD:
         self.speech_count = 0
         self.silence_count = 0
 
-        # 创建独立的PyAudio实例和流，避免与主音频流冲突
         self.audio = None
         self.stream = None
 
         # 暂存的语音片段
-        self.temp_frames = []
-        self.speech_buffer = []
+        self.silence_frames = []  # 静音片段
+        self.speech_frames = []  # 语音片段
         self.target = None  # 检测目标 speech/silence
 
     def _reset_state(self):
         """重置状态"""
         self.speech_count = 0
         self.silence_count = 0
-        self.speech_buffer = []
-        self.temp_frames = []
+        self.speech_frames = []
+        self.silence_frames = []
 
     def start(self):
         """启动VAD检测器"""
         if not get_env("CLI"):
             return
 
-        if self.thread and self.thread.is_alive():
-            return
-
-        self.paused = False
-
-        # 初始化PyAudio和流
         self._initialize_audio_stream()
 
         # 启动检测线程
+        self.paused = False
         self.thread = threading.Thread(target=self._detection_loop, daemon=True)
         self.thread.start()
 
@@ -88,9 +78,16 @@ class _VAD:
         """处理语音帧"""
         self.speech_count += len(frames)
         self.silence_count = 0
-        self.speech_buffer.extend(frames)
 
-        speech_bytes = bytes(self.speech_buffer)
+        if self.target == "speech":
+            if not self.speech_frames:
+                # 加入静音片段（潜在的语音片段）
+                self.speech_frames.extend(self.silence_frames)
+
+        # 加入语音片段
+        self.speech_frames.extend(frames)
+
+        speech_bytes = bytes(self.speech_frames)
 
         if (
             self.target == "speech"
@@ -103,7 +100,18 @@ class _VAD:
         """处理静音帧"""
         self.silence_count += len(frames)
         self.speech_count = 0
-        self.speech_buffer = []
+
+        if self.target == "speech":
+            if not self.speech_frames:
+                # 如果之前没有语音片段，则将当前帧加入静音片段
+                self.silence_frames.extend(frames)
+                # 确保静音片段长度不超过 1s
+                self.silence_frames = self.silence_frames[
+                    -1 * 1 * 2 * self.sample_rate :
+                ]
+            else:
+                # 如果之前有语音片段，则将当前帧加入语音片段
+                self.speech_frames.extend(frames)
 
         if (
             self.target == "silence"
@@ -112,17 +120,6 @@ class _VAD:
             self.pause()
             EventManager.on_silence()
 
-    def _detect_speech(self, frames):
-        """检测是否是语音"""
-        try:
-            audio_int16 = np.frombuffer(frames, dtype=np.int16)
-            audio_float32 = audio_int16.astype(np.float32) / 32768.0
-            speech_prob = VAD_MODEL(audio_float32, 16000).item()
-            is_speech = speech_prob >= self.threshold
-            return is_speech
-        except Exception:
-            return False
-
     def _initialize_audio_stream(self):
         """初始化独立的音频流"""
         try:
@@ -171,7 +168,8 @@ class _VAD:
                 continue
 
             # 检测是否是语音
-            is_speech = self._detect_speech(frames)
+            speech_prob = Silero.vad(frames, self.sample_rate) or 0
+            is_speech = speech_prob >= self.threshold
             if is_speech:
                 self._handle_speech_frame(frames)
             else:

examples/xiaozhi/xiaozhi/services/audio/vad/silero.py

@@ -84,6 +84,19 @@ class OnnxWrapper:
         return out
 
 
-VAD_MODEL = OnnxWrapper(
-    path=get_model_file_path("silero_vad.onnx"),
-)
+class _Silero:
+    def __init__(self) -> None:
+        self.model = OnnxWrapper(
+            path=get_model_file_path("silero_vad.onnx"),
+        )
+
+    def vad(self, frames, sample_rate):
+        try:
+            audio_int16 = np.frombuffer(frames, dtype=np.int16)
+            audio_float32 = audio_int16.astype(np.float32) / 32768.0
+            return self.model(audio_float32, sample_rate).item()
+        except Exception:
+            return None
+
+
+Silero = _Silero()

examples/xiaozhi/xiaozhi/services/protocols/typing.py

@@ -24,7 +24,7 @@ class EventType:
 class AudioConfig:
     """音频配置"""
     FORMAT = 8
-    SAMPLE_RATE = 24000
+    SAMPLE_RATE = 16000
     CHANNELS = 1
     FRAME_DURATION = 60  # ms
     FRAME_SIZE = int(SAMPLE_RATE * (FRAME_DURATION / 1000))

examples/xiaozhi/xiaozhi/services/protocols/websocket_protocol.py

@@ -3,7 +3,9 @@ import json
 
 import websockets
 
+from xiaozhi.ref import get_xiaozhi
 from xiaozhi.services.protocols.protocol import Protocol
+from xiaozhi.services.protocols.typing import DeviceState
 from xiaozhi.utils.config import ConfigManager
 
 
@@ -13,7 +15,11 @@ class WebsocketProtocol(Protocol):
         # 获取配置管理器实例
         self.config = ConfigManager.instance()
         self.websocket = None
-        self.server_sample_rate = 16000
+        self.server_sample_rate = 24000
+        self.server_frame_duration = 60
+        self.server_frame_size = int(
+            self.server_sample_rate * (self.server_frame_duration / 1000)
+        )
         self.connected = False
         self.hello_received = None  # 初始化时先设为 None
         self.WEBSOCKET_URL = self.config.get_config("NETWORK.WEBSOCKET_URL")
@@ -23,9 +29,20 @@ class WebsocketProtocol(Protocol):
         self.CLIENT_ID = self.config.get_client_id()
         self.DEVICE_ID = self.config.get_device_id()
 
+    async def _close_websocket(self):
+        if self.websocket:
+            try:
+                await self.websocket.close()
+                self.websocket = None
+                self.connected = False
+            except Exception:
+                pass
+
     async def connect(self) -> bool:
         """连接到WebSocket服务器"""
         try:
+            await self._close_websocket()
+
             # 在连接时创建 Event，确保在正确的事件循环中
             self.hello_received = asyncio.Event()
 
@@ -68,7 +85,6 @@ class WebsocketProtocol(Protocol):
                 if self.on_network_error:
                     self.on_network_error("等待响应超时")
                 return False
-
         except Exception as e:
             if self.on_network_error:
                 self.on_network_error(f"无法连接服务: {str(e)}")
@@ -91,25 +107,21 @@ class WebsocketProtocol(Protocol):
                         pass
                 elif self.on_incoming_audio:
                     self.on_incoming_audio(message)
-        except websockets.ConnectionClosed:
+        except Exception:
             self.connected = False
             if self.on_audio_channel_closed:
                 await self.on_audio_channel_closed()
-        except Exception as e:
-            self.connected = False
-            if self.on_network_error:
-                self.on_network_error(f"连接错误: {str(e)}")
 
-    async def send_audio(self, data: bytes):
+    async def send_audio(self, frames: list[bytes]):
         """发送音频数据"""
         if not self.is_audio_channel_opened():  # 使用已有的 is_connected 方法
             return
 
         try:
-            await self.websocket.send(data)
-        except Exception as e:
-            if self.on_network_error:
-                self.on_network_error(f"发送音频失败: {str(e)}")
+            for frame in frames:
+                await self.websocket.send(frame)
+        except Exception:
+            pass
 
     async def send_text(self, message: str):
         """发送文本消息"""
@@ -117,24 +129,18 @@ class WebsocketProtocol(Protocol):
             try:
                 await self.websocket.send(message)
             except Exception as e:
-                await self.close_audio_channel()
-                if self.on_network_error:
-                    self.on_network_error(f"发送消息失败: {str(e)}")
-
+                raise e
     def is_audio_channel_opened(self) -> bool:
         """检查音频通道是否打开"""
         return self.websocket is not None and self.connected
 
-    async def open_audio_channel(self) -> bool:
-        """建立 WebSocket 连接
+    _is_heartbeat_running = False
 
-        如果尚未连接,则创建新的 WebSocket 连接
-        Returns:
-            bool: 连接是否成功
-        """
-        if not self.connected:
-            return await self.connect()
-        return True
+    async def open_audio_channel(self):
+        if not self._is_heartbeat_running:
+            self._is_heartbeat_running = True
+            asyncio.create_task(self.heartbeat())
+        await self.connect()
 
     async def _handle_server_hello(self, data: dict):
         """处理服务器的 hello 消息
@@ -150,13 +156,22 @@ class WebsocketProtocol(Protocol):
             if not transport or transport != "websocket":
                 return
 
+            # TODO 使用默认的 24k 采样率
+            # xiaozhi-esp32-server 返回的参数是 16k 采样率，但实际用的是 24k 采样率
+
             # 获取音频参数
-            audio_params = data.get("audio_params")
-            if audio_params:
-                # 获取服务器的采样率
-                sample_rate = audio_params.get("sample_rate")
-                if sample_rate:
-                    self.server_sample_rate = sample_rate
+            # audio_params = data.get("audio_params")
+            # if audio_params:
+            #     # 获取服务器的采样率
+            #     sample_rate = audio_params.get("sample_rate")
+            #     if sample_rate:
+            #         self.server_sample_rate = sample_rate
+            #     frame_duration = audio_params.get("frame_duration")
+            #     if frame_duration:
+            #         self.server_frame_duration = frame_duration
+            #     self.server_frame_size = int(
+            #         self.server_sample_rate * (self.server_frame_duration / 1000)
+            #     )
 
             # 设置 hello 接收事件
             self.hello_received.set()
@@ -180,3 +195,15 @@ class WebsocketProtocol(Protocol):
                     await self.on_audio_channel_closed()
             except Exception:
                 pass
+
+    async def heartbeat(self):
+        while True:
+            if self.websocket and get_xiaozhi().device_state == DeviceState.IDLE:
+                try:
+                    await self.send_text(
+                        json.dumps({"session_id": "", "type": "abort"})
+                    )
+                except Exception:
+                    # 发送心跳失败，重新连接
+                    await self.open_audio_channel()
+            await asyncio.sleep(1)

examples/xiaozhi/xiaozhi/xiaoai.py

@@ -5,7 +5,6 @@ import threading
 import numpy as np
 import open_xiaoai_server
 
-from config import APP_CONFIG
 from xiaozhi.event import EventManager
 from xiaozhi.ref import get_speaker, set_xiaoai
 from xiaozhi.services.audio.stream import GlobalStream
@@ -69,12 +68,6 @@ class XiaoAI:
         event_data = event_json.get("data", {})
         event_type = event_json.get("event")
 
-        async def on_wakeup(text, source):
-            before_wakeup = APP_CONFIG["wakeup"]["before_wakeup"]
-            wakeup = await before_wakeup(get_speaker(), text, source)
-            if wakeup:
-                EventManager.on_wakeup()
-
         if not event_json.get("event"):
             return
 
@@ -87,11 +80,11 @@ class XiaoAI:
             ):
                 text = line.get("payload", {}).get("results")[0].get("text")
                 if not text and not line.get("payload", {}).get("is_vad_begin"):
-                    print(f"🔥 唤醒小爱: {line}")
+                    print("🔥 唤醒小爱")
                     EventManager.on_interrupt()
                 elif text and line.get("payload", {}).get("is_final"):
                     print(f"🔥 收到指令: {text}")
-                    await on_wakeup(text, "xiaoai")
+                    await EventManager.wakeup(text, "xiaoai")
         elif event_type == "playing":
             get_speaker().status = event_data.lower()
 

examples/xiaozhi/xiaozhi/xiaozhi.py

@@ -10,7 +10,6 @@ from xiaozhi.services.audio.kws import KWS
 from xiaozhi.services.audio.vad import VAD
 from xiaozhi.services.protocols.typing import (
     AbortReason,
-    AudioConfig,
     DeviceState,
     EventType,
     ListeningMode,
@@ -88,9 +87,9 @@ class XiaoZhi:
         # 等待事件循环准备就绪
         time.sleep(0.1)
 
-        # 初始化应用程序（移除自动连接）
+        # 初始化应用程序
         asyncio.run_coroutine_threadsafe(XiaoAI.init_xiaoai(), self.loop)
-        asyncio.run_coroutine_threadsafe(self._initialize_without_connect(), self.loop)
+        asyncio.run_coroutine_threadsafe(self._initialize_xiaozhi(), self.loop)
 
         # 启动主循环线程
         main_loop_thread = threading.Thread(target=self._main_loop)
@@ -109,8 +108,8 @@ class XiaoZhi:
         asyncio.set_event_loop(self.loop)
         self.loop.run_forever()
 
-    async def _initialize_without_connect(self):
-        """初始化应用程序组件（不建立连接）"""
+    async def _initialize_xiaozhi(self):
+        """初始化应用程序组件"""
 
         # 初始化音频编解码器
         self._initialize_audio()
@@ -122,8 +121,9 @@ class XiaoZhi:
         self.protocol.on_audio_channel_opened = self._on_audio_channel_opened
         self.protocol.on_audio_channel_closed = self._on_audio_channel_closed
 
-        # 设置设备状态为待命
-        self.set_device_state(DeviceState.IDLE)
+        # 打开音频通道
+        self.device_state = DeviceState.CONNECTING
+        await self.protocol.open_audio_channel()
 
     def _initialize_audio(self):
         """初始化音频设备和编解码器"""
@@ -134,6 +134,8 @@ class XiaoZhi:
         except Exception as e:
             self.alert("错误", f"初始化音频设备失败: {e}")
 
+        threading.Thread(target=self._audio_input_event_trigger, daemon=True).start()
+
     def _initialize_display(self):
         """初始化显示界面"""
         if get_env("CLI"):
@@ -220,50 +222,6 @@ class XiaoZhi:
                     self.protocol.close_audio_channel(), self.loop
                 )
 
-    def _attempt_reconnect(self):
-        """尝试重新连接服务器"""
-        if self.device_state != DeviceState.CONNECTING:
-            self.set_device_state(DeviceState.CONNECTING)
-
-            # 关闭现有连接
-            if self.protocol:
-                asyncio.run_coroutine_threadsafe(
-                    self.protocol.close_audio_channel(), self.loop
-                )
-
-            # 延迟一秒后尝试重新连接
-            def delayed_reconnect():
-                time.sleep(1)
-                asyncio.run_coroutine_threadsafe(self._reconnect(), self.loop)
-
-            threading.Thread(target=delayed_reconnect, daemon=True).start()
-
-    async def _reconnect(self):
-        """重新连接到服务器"""
-
-        # 设置协议回调
-        self.protocol.on_network_error = self._on_network_error
-        self.protocol.on_incoming_audio = self._on_incoming_audio
-        self.protocol.on_incoming_json = self._on_incoming_json
-        self.protocol.on_audio_channel_opened = self._on_audio_channel_opened
-        self.protocol.on_audio_channel_closed = self._on_audio_channel_closed
-
-        # 连接到服务器
-        retry_count = 0
-        max_retries = 3
-
-        while retry_count < max_retries:
-            if await self.protocol.connect():
-                self.set_device_state(DeviceState.IDLE)
-                return True
-
-            retry_count += 1
-            await asyncio.sleep(2)  # 等待2秒后重试
-
-        self.schedule(lambda: self.alert("连接错误", "无法重新连接到服务器"))
-        self.set_device_state(DeviceState.IDLE)
-        return False
-
     def _on_incoming_audio(self, data):
         """接收音频数据回调"""
         if self.device_state == DeviceState.SPEAKING:
@@ -296,20 +254,22 @@ class XiaoZhi:
         """处理TTS消息"""
         state = data.get("state", "")
         if state == "start":
-            EventManager.on_tts_start()
+            EventManager.on_tts_start(data.get("session_id"))
             self.schedule(lambda: self._handle_tts_start())
         elif state == "stop":
-            EventManager.on_tts_end()
+            EventManager.on_tts_end(data.get("session_id"))
             self.schedule(lambda: self._handle_tts_stop())
         elif state == "sentence_start":
             text = data.get("text", "")
             if text:
                 print(f"🤖 小智：{text}")
 
-                verification_code = re.findall(r"验证码.*\d+", text)
+                verification_code = re.search(r"验证码.*?(\d+)", text) or re.search(
+                    r"控制面板.*?(\d+)", text
+                )
                 if verification_code:
                     self.config.update_config_file(
-                        "VERIFICATION_CODE", verification_code[0]
+                        "VERIFICATION_CODE", verification_code.group(1)
                     )
 
                 self.schedule(lambda: self.set_chat_message("assistant", text))
@@ -341,45 +301,14 @@ class XiaoZhi:
 
     async def _on_audio_channel_opened(self):
         """音频通道打开回调"""
-        self.schedule(lambda: self._start_audio_streams())
-
-    def _start_audio_streams(self):
-        """启动音频流"""
-        try:
-            # 确保流已关闭后再重新打开
-            if self.audio_codec.input_stream.is_active():
-                self.audio_codec.input_stream.stop_stream()
-
-            # 重新打开流
-            self.audio_codec.input_stream.start_stream()
-
-            if self.audio_codec.output_stream.is_active():
-                self.audio_codec.output_stream.stop_stream()
-
-            # 重新打开流
-            self.audio_codec.output_stream.start_stream()
-
-            # 设置事件触发器
-            threading.Thread(
-                target=self._audio_input_event_trigger, daemon=True
-            ).start()
-
-        except Exception:
-            pass
+        self.set_device_state(DeviceState.IDLE)
 
     def _audio_input_event_trigger(self):
         """音频输入事件触发器"""
         while self.running:
-            try:
-                if self.audio_codec.input_stream.is_active():
-                    self.events[EventType.AUDIO_INPUT_READY_EVENT].set()
-            except OSError as e:
-                if "Stream not open" in str(e):
-                    break
-            except Exception:
-                pass
-
-            time.sleep(AudioConfig.FRAME_DURATION / 1000)  # 按帧时长触发
+            if self.audio_codec.input_stream.is_active():
+                self.events[EventType.AUDIO_INPUT_READY_EVENT].set()
+            time.sleep(0.01)
 
     async def _on_audio_channel_closed(self):
         """音频通道关闭回调"""
@@ -492,26 +421,10 @@ class XiaoZhi:
             self.protocol.send_abort_speaking(AbortReason.ABORT),
             self.loop,
         )
-
-        # 尝试打开音频通道
-        if not self.protocol.is_audio_channel_opened():
-            self.set_device_state(DeviceState.CONNECTING)  # 设置设备状态为连接中
-            try:
-                # 等待异步操作完成
-                future = asyncio.run_coroutine_threadsafe(
-                    self.protocol.open_audio_channel(), self.loop
-                )
-                # 等待操作完成并获取结果
-                assert future.result(timeout=10.0)  # 添加超时时间
-            except Exception as e:
-                self.alert("错误", f"打开音频通道失败: {str(e)}")
-                self.set_device_state(DeviceState.IDLE)
-                return
-
         asyncio.run_coroutine_threadsafe(
             self.protocol.send_start_listening(ListeningMode.MANUAL), self.loop
         )
-        self.set_device_state(DeviceState.LISTENING)  # 设置设备状态为监听中
+        self.set_device_state(DeviceState.LISTENING)
 
     def stop_listening(self):
         """停止监听"""
@@ -519,8 +432,8 @@ class XiaoZhi:
 
     def _stop_listening_impl(self):
         """停止监听的实现"""
-        self.set_device_state(DeviceState.IDLE)
         asyncio.run_coroutine_threadsafe(self.protocol.send_stop_listening(), self.loop)
+        self.set_device_state(DeviceState.IDLE)
 
     def abort_speaking(self, reason):
         """中止语音输出"""

packages/client-patch/README.md

@@ -86,6 +86,12 @@ docker run -it --rm --env-file $(pwd)/.env -v $(pwd)/assets:/app/assets -v $(pwd
 - Node.js 22.x: https://nodejs.org/zh-cn/download
 
 ```bash
+# 克隆代码
+git clone https://github.com/idootop/open-xiaoai.git
+
+# 进入当前项目根目录
+cd packages/client-patch
+
 # 安装依赖
 npm install
 

packages/client-rust/README.md

@@ -48,6 +48,12 @@ reboot
 为了构建能够在小爱音箱上运行的 ARMv7 应用，你还需要安装 `cross` 👉 [传送门](https://github.com/cross-rs/cross)
 
 ```shell
+# 克隆代码
+git clone https://github.com/idootop/open-xiaoai.git
+
+# 进入当前项目根目录
+cd packages/client-rust
+
 # 交叉编译
 cross build --release --target armv7-unknown-linux-gnueabihf
 ```

packages/client-rust/boot.sh

@@ -1,7 +1,23 @@
 #! /bin/sh
 
+exec > /dev/null 2>&1
+
+cat << 'EOF'
+
+▄▖      ▖▖▘    ▄▖▄▖
+▌▌▛▌█▌▛▌▚▘▌▀▌▛▌▌▌▐ 
+▙▌▙▌▙▖▌▌▌▌▌█▌▙▌▛▌▟▖
+  ▌                 
+
+v1.0.0  by: https://del.wang
+
+EOF
+
 set -e
 
+echo "🤫 等待网络连接中..."
+
+sleep 5
 
 DOWNLOAD_BASE_URL="https://gitee.com/idootop/artifacts/releases/download/open-xiaoai-client"
 

packages/client-rust/init.sh

@@ -1,5 +1,16 @@
 #! /bin/sh
 
+cat << 'EOF'
+
+▄▖      ▖▖▘    ▄▖▄▖
+▌▌▛▌█▌▛▌▚▘▌▀▌▛▌▌▌▐ 
+▙▌▙▌▙▖▌▌▌▌▌█▌▙▌▛▌▟▖
+  ▌                 
+
+v1.0.0  by: https://del.wang
+
+EOF
+
 set -e
 
 

packages/flash-tool/README.md

@@ -20,6 +20,12 @@ brew install libusb-compat
 ## 使用方法
 
 ```shell
+# 克隆代码
+git clone https://github.com/idootop/open-xiaoai.git
+
+# 进入当前项目根目录
+cd packages/flash-tool
+
 # 授予可运行权限
 chmod +x ./flash