Gemini Robotics ER 2 可編寫及執行 Python 程式碼，在回答前處理圖片並套用邏輯。本頁面提供執行程式碼範例：縮放及裁剪物件偵測、儀器讀數、液體測量、電路板讀數和圖片註解。
如需完整的可執行程式碼，請參閱機器人技術錦囊妙計。
思考程度
您可以控制思考等級，在延遲時間和準確度之間取得平衡。物件偵測等空間任務，在低思考等級下表現良好。計數或重量估算等複雜工作，則適合較高的解析度。
以下範例將思考等級設為高，用於複雜的計數工作：
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
# Load your image
with open('scene.jpeg', 'rb') as f:
image_bytes = f.read()
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
"Identify and count all objects on the table."
],
config = types.GenerateContentConfig(
thinking_config=types.ThinkingConfig(
# Use MINIMAL or LOW for faster spatial tasks.
thinking_level=types.ThinkingLevel.HIGH
),
)
)
print(response.text)
詳情請參閱「思考」一節。
物件偵測 (縮放及裁剪)
以下範例說明如何使用執行程式碼功能，在偵測物件並傳回定界框時，縮放及裁剪圖片，以便更清楚地查看。
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
# Load your image
with open('sorting.jpeg', 'rb') as f:
image_bytes = f.read()
prompt = """
Return JSON in the format {label: val, y: val, x: val, y2: val, x2: val} for
the compostable objects in this scene. Please Zoom and crop the image for a
clearer view. Return an annotated image of the final result with the bounding
boxes drawn on it to the API caller as a part of your process.
"""
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
prompt
],
config = types.GenerateContentConfig(
tools=[types.Tool(code_execution=types.ToolCodeExecution)],
)
)
print(response.text)
模型輸出內容大致如下：
[
{"label": "compostable", "y": 256, "x": 482, "y2": 295, "x2": 546},
{"label": "compostable", "y": 317, "x": 478, "y2": 350, "x2": 542},
{"label": "compostable", "y": 586, "x": 556, "y2": 668, "x2": 595},
{"label": "compostable", "y": 463, "x": 669, "y2": 511, "x2": 718},
{"label": "compostable", "y": 178, "x": 565, "y2": 250, "x2": 609}
]
以下顯示模型傳回的方塊。
讀取類比儀表並套用邏輯
以下範例說明如何使用模型讀取類比儀表，並執行時間計算。使用系統指令強制輸出 JSON。
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
with open('gauge.jpeg', 'rb') as f:
image_bytes = f.read()
system_instruction = "Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block)."
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
"""Read the current value from this gauge. Then, calculate how long
it will take at the current rate for the value to reach maximum.
Reply in JSON: {"current_value": val, "max_value": val,
"time_to_max_minutes": val}"""
],
config = types.GenerateContentConfig(
system_instruction=system_instruction,
tools=[types.Tool(code_execution=types.ToolCodeExecution)],
)
)
print(response.text)
測量容器中的液體
以下範例說明如何使用執行程式碼功能，測量容器中的液體量。
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
with open('fluid.jpeg', 'rb') as f:
image_bytes = f.read()
system_instruction = "Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block)."
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
"""Measure the amount of fluid in the container. Reply in JSON:
{"fluid_level_ml": val, "container_capacity_ml": val,
"percentage_full": val}"""
],
config = types.GenerateContentConfig(
system_instruction=system_instruction,
tools=[types.Tool(code_execution=types.ToolCodeExecution)],
)
)
print(response.text)
讀取電路板上的標記
以下範例說明如何使用執行程式碼功能，讀取電路板上的標記。
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
with open('circuit_board.jpeg', 'rb') as f:
image_bytes = f.read()
system_instruction = "Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block)."
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
"""Read all visible component labels and markings on this circuit
board. Reply in JSON: {"components": [{"label": val,
"location": [y, x]}]}"""
],
config = types.GenerateContentConfig(
system_instruction=system_instruction,
tools=[types.Tool(code_execution=types.ToolCodeExecution)],
)
)
print(response.text)
圖片註解
以下範例說明如何使用執行程式碼功能，在圖片中加入註解 (例如繪製箭頭來指示如何丟棄物品)，並傳回修改後的圖片。
Python
from google import genai
from google.genai import types
client = genai.Client(
enterprise=True,
project="your-project-id",
location="global",
)
# Load your image
with open('sorting.jpeg', 'rb') as f:
image_bytes = f.read()
prompt = """
Look at this image and return it as an annotated version using arrows of
different colors to represent which items should go in which bins for
disposal. You must return the final image to the API caller.
"""
response = client.models.generate_content(
model="gemini-robotics-er-2-preview",
contents=[
types.Part.from_bytes(
data=image_bytes,
mime_type='image/jpeg',
),
prompt
],
config = types.GenerateContentConfig(
tools=[types.Tool(code_execution=types.ToolCodeExecution)],
)
)
print(response.text)
以下是圖片輸入範例。
模型輸出內容大致如下：
The annotated image shows the suggested disposal locations for the items on the table:
- **Green bin (Compost/Organic)**: Green chili, red chili, grapes, and cherries.
- **Blue bin (Recycling)**: Yellow crushed can and plastic container.
- **Black bin (Trash)**: Chocolate bar wrapper, Welch's packet, and white tissue.
後續步驟
如要進一步瞭解 Gemini Robotics ER 2，請參閱下列資源：
- 工作自動化調度管理：使用自訂機器人 API 執行長程工作。
- 空間推論：指向、追蹤、定界框和軌跡。
- 影片理解：尋找特定時刻和進度分類。
- Gemini Robotics ER 2 總覽：功能和快速入門指南。
- Gemini Robotics ER 2 模型資訊：技術規格、權杖限制和支援功能。