Autogenerating embeddings for Agent Retrieval

You can use Agent Retrieval (formerly Vector Search 2.0) to autogenerate embeddings for your Collections. This lets you build new embeddings and deploy them instantly, which streamlines the path from raw data to a live production-scale search engine.

Supported Gemini Enterprise embedding models

Agent Retrieval (formerly Vector Search 2.0) supports the following embedding models:

• Gemini — Provides state-of-the-art performance for embedding text (English-only and multilingual).

• Text — Specializes in (English-only and multilingual) and source code data.

The following table provides details on each supported model.

Creating Collections with autogenerated embeddings

When creating a Collection, specify the embedding model in the model_id field of vertex_embedding_config. This model is used whenever a Data Object is created without genre_embedding data defined.

The following code demonstrates how to specify the embedding model to use when autogenerating embeddings.

request = vectorsearch.CreateCollectionRequest(
   parent=f"projects/{PROJECT_ID}/locations/{LOCATION}",
   collection_id=collection_id,
   collection={
       "data_schema": {
           "type": "object",
           "properties": {
               "year": {"type": "number"},
               "genre": {"type": "string"},
               "director": {"type": "string"},
               "title": {"type": "string"},
           },
       },
       "vector_schema": {
           "plot_embedding": {"dense_vector": {"dimensions": 3} },
           "soundtrack_embedding": {"dense_vector": {"dimensions": 5} },
           "genre_embedding": {
               "dense_vector": {
                   "dimensions": 4,
                   "vertex_embedding_config": {
                       # If a data object is created without a supplied value for genre_embedding, it will be
                       # auto-generated based on this config.
                       "model_id": "text-embedding-004",
                       "text_template": ("Movie: {title} Genre: {genre} Year: {year}"),
                       "task_type": "RETRIEVAL_DOCUMENT",
                   },
               }
           },
           "sparse_embedding": {"sparse_vector": { } },
       },
   },
)
operation = vector_search_service_client.create_collection(request=request)
operation.result()

In the example code, a new Collection is created with the model_id field set to text-embedding-004. See Supported Gemini Enterprise embedding models for which embedding models can be specified for model_id.

Quotas

Autogenerated embeddings rely on on customer quotas for the underlying Gemini Enterprise embedding models. This is primarily constrained by two main quotas:

• Embed content input tokens per minute per region per base_model.

• Online prediction requests per base model per minute per region per base_model.

Make sure you have enough quota before creating Data Objects or running an import job.

See Manage your quota using the console for information on how to request larger quotas.

See also

• Collections
• Data Objects