如要透過 Gemini Live API 獲得更佳成效,請著重於下列最佳做法:
設計清楚的系統指令
為充分發揮 Gemini Live API 的效能,建議您依序定義清楚的系統指令 (SI),包括定義代理程式角色、對話規則和防護措施。
為獲得最佳結果,請將每個代理程式分別歸入不同的 SI。
指定代理程式角色:提供代理程式的名稱、角色和任何偏好特徵的詳細資料。如要指定口音,請務必同時指定偏好的輸出語言 (例如,英文使用者偏好英國口音)。
指定對話規則:請按照您希望模型遵循的順序,區分對話的一次性元素和對話迴圈。例如:
- 一次性元素:收集顧客詳細資料一次 (例如姓名、地點、會員卡號)。
- 對話迴圈:使用者可以討論建議、價格、退貨和運送事宜,並可能想從一個主題轉到另一個主題。讓模型知道只要使用者願意,就可以持續進行這類對話。
在流程中以不同句子指定工具呼叫:舉例來說,如果收集顧客詳細資料的一次性步驟需要呼叫
get_user_info函式,你可以說:第一步是收集使用者資訊。首先,請使用者提供姓名、地點和會員卡號碼。然後使用這些詳細資料呼叫get_user_info。新增任何必要的防護措施:提供任何一般對話防護措施,避免模型做出您不希望的行為。您可以提供具體範例,說明如果發生 x,您希望模型執行 y。如果模型仍未達到您偏好的精確度,請使用「unmistakably」一詞引導模型提高精確度。
精確定義工具
使用 Gemini Live API 搭配工具時,請明確定義工具。請務必告訴 Gemini 應在哪些情況下呼叫工具。詳情請參閱「工具定義」。
撰寫有效的提示
使用清楚的提示:在提示中提供模型應執行的動作和不應執行的動作範例,並盡量一次只為一個角色或職務提供提示。請考慮使用提示鏈結,而非冗長的多頁提示。模型最擅長執行單一函式呼叫的任務。
# Prompt chaining example. chainable_long_prompt = """ You need to perform a sequence of tasks. First, you should do task1; after that, task2; later, task3; and finally, task4. """ # New initial prompt """ You need to perform a sequence of tasks. Once you finish the current task, call the `get_next_prompt` function to get instructions for the next task. """ PROMPT_LIST = ["Now, do task1", "Now, do task2", "Now, do task3", "Now, do task 4", "all tasks done"] def get_next_prompt(): # Provide this function as a tool to the model. for prompt in PROMPT_LIST: yield prompt # Catch and execute tool call `get_next_prompt` and send the new prompt back to the model.提供起始指令和資訊:Gemini Live API 會先等待使用者輸入內容,再進行回覆。如要讓 Gemini Live API 發起對話,請加入提示,要求該 API 向使用者問候或開始對話。加入使用者資訊,讓 Gemini Live API 提供個人化的問候語。
繼續工作階段
- 使用透明工作階段續傳:
在
genai.types.LiveConnectConfig中使用SessionResumptionConfig(transparent=True)設定連線。這表示用戶端打算順暢處理工作階段續傳作業,以便在重新連線時重播未使用的訊息等功能。
from google.genai import types
session_handle: str | None = None
live_config = types.LiveConnectConfig(
session_resumption=types.SessionResumptionConfig(
handle=session_handle,
transparent=True,
),
)
維護及更新工作階段控制代碼: 監聽伺服器傳送的
session_resumption_update訊息。如果resumable為 true 且提供new_handle,請儲存這個控制代碼。如果連線中斷,這個控制代碼對於重新連線至相同的工作階段狀態至關重要。緩衝處理傳送的訊息,並修剪已確認的訊息: 為確保用戶端訊息不會在連線中斷期間遺失,請維護傳送至 Gemini Live API 的訊息緩衝區。如果啟用透明工作階段續傳功能,
session_resumption_update訊息會包含last_consumed_client_message_index,指出伺服器處理的最後一則訊息。使用這個索引從緩衝區移除已確認的訊息。如要正確追蹤訊息,使用者管理的索引必須從 1 開始,因為索引 0 表示the session is not resumable。後續傳送給模型的每則訊息,這個索引都應增加 1。每次恢復工作階段時,請務必將索引重設為 1,以便透過新連線傳輸初始訊息。順利處理連線中斷:
- GoAway 信號:伺服器會在預期中斷連線 (例如逾時) 前傳送
go_away訊息。管理員應監聽這項訊息,然後使用最新控制代碼主動重新連線。 - API 錯誤:網路問題可能會導致
genai_errors.APIError(例如 WebSocket 錯誤的代碼 1000 或 1006)。管理員應在傳送和接收迴圈中擷取這些錯誤,並觸發工作階段更新或重新連線程序。
- GoAway 信號:伺服器會在預期中斷連線 (例如逾時) 前傳送
實作訊息重播的重新連線機制: 發生連線中斷時,請使用
client.aio.live.connect和最新的工作階段控制代碼建立新的工作階段。建立新連線後,請重新傳送緩衝區中所有未在連線中斷前獲得伺服器確認的訊息。緩衝區中傳送的第一則訊息應標示為新連線的索引 1。
啟用背景資訊視窗壓縮功能
使用 ContextWindowCompressionConfig設定工作階段的內容視窗,因為原生音訊符記會快速累積 (每秒音訊約 25 個符記),因此適用於長時間工作階段。
警告:壓縮對話內容會導致對話記錄遺失。
from google.genai import types
live_config = types.LiveConnectConfig(
context_window_compression=types.ContextWindowCompressionConfig(
trigger_tokens=100_000, # For better clarity
sliding_window=types.SlidingWindow(target_tokens=4_000),
),
)
詞元用量計算
如要瞭解 Gemini Live API 的詳細計費結構,請參閱價格頁面。在每個回合中,API 會針對所有情境權杖計費,包括對話記錄和使用者提供的系統指令。開發人員可以擷取模型回覆中提供的 usage_metadata 欄位,監控及計算這些費用。
# Example code to get token usage
from google.genai import live
session: live.AsyncSession
async for response in session.receive():
if response.usage_metadata is not None:
print("Token usage:", response.usage_metadata)
語音活動偵測 (VAD)
根據預設,Gemini Live API 會使用 Gemini 提供的 VAD。
使用 Gemini Live API VAD 時,您可以設定模型明確傳回 VAD 事件。在設定中啟用 explicit_vad_signal 後,您就能直接從模型的相關回應監控及擷取這些事件。
from google.genai import types
from google.genai import live
live_config = types.LiveConnectConfig(
explicit_vad_signal=True
)
session: live.AsyncSession
# In receive loop
async for response in session.receive():
if response.voice_activity is not None:
print("Get VAD event", response.voice_activity)
如要使用自訂活動偵測系統,請停用預設的語音活動偵測 (VAD),並手動向 Gemini 模型發出使用者輪流說話的信號。方法是傳輸 ActivityStart 或 ActivityEnd 事件,定義互動界線。
from google.genai import live
from google.genai import types
# Disable VAD in config
live_config = types.LiveConnectConfig(
realtime_input_config=types.RealtimeInputConfig(
automatic_activity_detection=types.AutomaticActivityDetection(
disabled=True
),
),
)
session: live.AsyncSession
await session.send_realtime_input( # Send activity start
activity_start=types.ActivityStart()
)
for audio_bytes in bytes_to_send_queue: # Send user data
await session.send_realtime_input(
audio=types.Blob(
data=audio_bytes,
mime_type=f"audio/pcm;rate=16000",
)
)
await session.send_realtime_input(activity_end=types.ActivityEnd()) # Send activity end
設定音訊語言代碼
建議您在設定中明確設定語言和語音代碼,以維持一致性;如果沒有這項定義,Gemini 可能會根據提供的脈絡變更對話語言。
from google.genai import types
config = types.LiveConnectConfig(
speech_config=types.SpeechConfig(
language_code="en-US",
),
)
此外,請在系統指令中提及下列事項:
RESPOND IN {OUTPUT_LANGUAGE}. YOU MUST RESPOND UNMISTAKABLY IN {OUTPUT_LANGUAGE}.
如果是 gemini-live-2.5-flash-native-audio 等原生音訊模型,您可以在工作階段設定中提供語言提示,提升多語言自動語音辨識 (ASR) 的轉錄品質。詳情請參閱「為工作階段啟用音訊轉錄功能」。
設定轉錄語言代碼
使用 BCP-47 語言代碼格式指定轉錄語言代碼,提高轉錄準確度。
注意:啟用轉錄功能會產生更多權杖。
from google.genai import types
config = types.LiveConnectConfig(
input_audio_transcription=types.AudioTranscriptionConfig(
language_codes=['en-US'] # This supports multiple language codes.
),
output_audio_transcription=types.AudioTranscriptionConfig(
language_codes=['en-US']
),
)
用戶端緩衝
請勿在傳送前大幅緩衝輸入音訊 (例如 1 秒)。 傳送小區塊 (20 毫秒到 40 毫秒之間),盡量縮短延遲時間。
重新取樣
請確保用戶端應用程式在傳輸前,將麥克風輸入內容 (通常為 44.1 kHz 或 48 kHz) 重新取樣為 16 kHz。
範例
這個範例結合了最佳做法和系統指令設計指南,引導模型扮演職涯教練的角色。
**Persona:**
You are Laura, a career coach from Brooklyn, NY. You specialize in providing
data-driven advice to give your clients a fresh perspective on the career
questions they're navigating. Your special sauce is providing quantitative,
data-driven insights to help clients think about their issues in a different
way. You leverage statistics, research, and psychology as much as possible.
You only speak to your clients in English, no matter what language they speak
to you in.
**Conversational Rules:**
1. **Introduce yourself:** Warmly greet the client.
2. **Intake:** Ask for your client's full name, date of birth, and state they're
calling in from. Call `create_client_profile` to create a new patient profile.
3. **Discuss the client's issue:** Get a sense of what the client wants to
cover in the session. DO NOT repeat what the client is saying back to them in
your response. Don't ask more than a few questions here.
4. **Reframe the client's issue with real data:** NO PLATITUDES. Start providing
data-driven insights for the client, but embed these as general facts within
conversation. This is what they're coming to you for: your unique thinking on
the subjects that are stressing them out. Show them a new way of thinking about
something. Let this step go on for as long as the client wants. As part of this,
if the client mentions wanting to take any actions, update
`add_action_items_to_profile` to remind the client later.
5. **Next appointment:** Call `get_next_appointment` to see if another
appointment has already been scheduled for the client. If so, then share the
date and time with the client and confirm if they'll be able to attend. If
there is no appointment, then call `get_available_appointments` to see openings.
Share the list of openings with the client and ask what they would prefer. Save
their preference with `schedule_appointment`. If the client prefers to schedule
offline, then let them know that's perfectly fine and to use the patient portal.
**General Guidelines:** You're meant to be a witty, snappy conversational
partner. Keep your responses short and progressively disclose more information
if the client requests it. Don't repeat what the client says back to them.
Each of your responses should add to the conversation, not just recap what
the client said. Be relatable by bringing in your own background
growing up professionally in Brooklyn, NY. If a client tries to get you off
track, gently bring them back to the workflow articulated above.
**Guardrails:** If the client is being hard on themselves, never encourage that.
Remember that your ultimate goal is to create a supportive environment for your
clients to thrive.
工具定義
這個 JSON 會定義職業教練範例中呼叫的相關函式。如要獲得最佳結果,請在定義函式時加入函式名稱、說明、參數和叫用條件。
[
{
"name": "create_client_profile",
"description": "Creates a new client profile with their personal details. Returns a unique client ID. \n**Invocation Condition:** Invoke this tool *only after* the client has provided their full name, date of birth, AND state. This should only be called once at the beginning of the 'Intake' step.",
"parameters": {
"type": "object",
"properties": {
"full_name": {
"type": "string",
"description": "The client's full name."
},
"date_of_birth": {
"type": "string",
"description": "The client's date of birth in YYYY-MM-DD format."
},
"state": {
"type": "string",
"description": "The 2-letter postal abbreviation for the client's state (e.g., 'NY', 'CA')."
}
},
"required": ["full_name", "date_of_birth", "state"]
}
},
{
"name": "add_action_items_to_profile",
"description": "Adds a list of actionable next steps to a client's profile using their client ID. \n**Invocation Condition:** Invoke this tool *only after* a list of actionable next steps has been discussed and agreed upon with the client during the 'Actions' step. Requires the `client_id` obtained from the start of the session.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client, obtained from create_client_profile."
},
"action_items": {
"type": "array",
"items": {
"type": "string"
},
"description": "A list of action items for the client (e.g., ['Update resume', 'Research three companies'])."
}
},
"required": ["client_id", "action_items"]
}
},
{
"name": "get_next_appointment",
"description": "Checks if a client has a future appointment already scheduled using their client ID. Returns the appointment details or null. \n**Invocation Condition:** Invoke this tool at the *start* of the 'Next Appointment' workflow step, immediately after the 'Actions' step is complete. This is used to check if an appointment *already exists*.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client."
}
},
"required": ["client_id"]
}
},
{
"name": "get_available_appointments",
"description": "Fetches a list of the next available appointment slots. \n**Invocation Condition:** Invoke this tool *only if* the `get_next_appointment` tool was called and it returned `null` (or an empty response), indicating no future appointment is scheduled.",
"parameters": {
"type": "object",
"properties": {}
}
},
{
"name": "schedule_appointment",
"description": "Books a new appointment for a client at a specific date and time. \n**Invocation Condition:** Invoke this tool *only after* `get_available_appointments` has been called, a list of openings has been presented to the client, and the client has *explicitly confirmed* which specific date and time they want to book.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client."
},
"appointment_datetime": {
"type": "string",
"description": "The chosen appointment slot in ISO 8601 format (e.g., '2025-10-30T14:30:00')."
}
},
"required": ["client_id", "appointment_datetime"]
}
}
]
更多資訊
如要進一步瞭解如何使用 Gemini Live API,請參閱: