教學課程：使用控制台執行評估

瞭解如何透過 Google Cloud 控制台開始使用 Gen AI 評估服務。 Google Cloud

事前準備

  1. 登入 Google Cloud 帳戶。如果您是 Google Cloud新手，歡迎 建立帳戶，親自評估產品在實際工作環境中的成效。新客戶還能獲得價值 $300 美元的免費抵免額，可用於執行、測試及部署工作負載。

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  3. Verify that billing is enabled for your Google Cloud project.

  4. Make sure that you have the following role or roles on the project: Storage Admin

    Check for the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.

    3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

    4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

    Grant the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. Click Grant access.

    4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

    5. Click Select a role, then search for the role.
    6. To grant additional roles, click Add another role and add each additional role.
    7. Click Save.

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

    Go to project selector

  6. Verify that billing is enabled for your Google Cloud project.

  7. Make sure that you have the following role or roles on the project: Storage Admin

    Check for the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.

    3. In the Principal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

    4. For all rows that specify or include you, check the Role column to see whether the list of roles includes the required roles.

    Grant the roles

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM
    2. Select the project.
    3. Click Grant access.

    4. In the New principals field, enter your user identifier. This is typically the email address for a Google Account.

    5. Click Select a role, then search for the role.
    6. To grant additional roles, click Add another role and add each additional role.
    7. Click Save.

評估模型

如要評估模型，請按照下列步驟操作：

  1. 前往 Google Cloud 控制台的「Gen AI Evaluation」頁面。

    前往「Evaluation」(評估)

  2. 按一下「New evaluation」(新增評估) 開啟評估頁面。

  3. 選取來源，載入評估用的資料集：

    • 如要上傳本機 CSV 或 JSONL 檔案，請選取「上傳檔案」。資料集必須包含提示詞或記錄，才能在提示詞範本中使用，也可以選擇是否包含模型回應。最多 200 列。

    • 如要從提示詞範本生成提示，請選取「生成資料」。建立資料集時，Gen AI Evaluation Service 會產生並填入您在提示詞範本中定義的變數。如要進一步瞭解如何撰寫提示範本，請參閱「使用提示範本」。

      1. 在「提示詞範本」欄位中輸入提示詞範本和變數。

      2. 如要為每個變數新增說明或指定要產生的樣本數，請展開「定義變數和樣本大小」。

      3. 按一下「產生資料集」即可產生提示。

  4. 根據提示生成及評估回覆：

    1. 在「評估候選項目」部分中，按一下「新增評估候選項目」，或按一下 「編輯」 (如果已有候選項目)，定義要評估的提示和回應。舉例來說，您可以指定上傳檔案或生成資料中的提示或回覆。

    2. 如要比較多個候選項目，請按一下「新增候選比較項目」。

    3. 在「指標」部分，至少新增一項指標，為候選人的回覆品質評分。如要進一步瞭解指標類型，請參閱 Gen AI Evaluation Service 總覽頁面的「評估指標」一節。

    對於部分適應性評量表，您可以展開「進階」並提供自訂指示 (例如 Evaluate the dataset on cultural sensitivity)，引導系統根據每個提示生成評量表。

    1. 在「名稱和儲存空間設定」部分，指定評估名稱和 Cloud Storage bucket，評估結果會儲存在該 bucket 中。

  5. 按一下「評估」。

查看評估結果

如要查看評估結果，請按照下列步驟操作：

  1. 前往 Google Cloud 控制台的「GenAI Evaluation」頁面。

    前往「Evaluation」(評估)

  2. 按一下「Evaluation name」(評估作業名稱)。

    評估資料集中的每個提示都會顯示回應和評估結果。

評估合作夥伴模型

您可以使用 Gen AI Evaluation Service 評估下列合作夥伴模型：

  • Anthropic
  • Llama

Gemini Enterprise Agent Platform Model Garden 支援合作夥伴模型。您必須先在Model Garden中啟用合作夥伴模型，才能選取模型進行評估。如要評估合作夥伴模型，請在評估設定期間，從模型選取選單中選取該模型。

定價

評估第三方模型的費用，取決於在 Gemini Enterprise Agent Platform Model Garden 中進行模型推論所產生的費用。請參閱「Agent Platform 的生成式 AI 定價」頁面。

後續步驟