Managing Collections

Collections serve as a fundamental organizational tool in vector storage, allowing you to group vectors by both index and dimension. It's important to bear in mind that once a collection's name and dimension are established, they cannot be altered.

To effectively harness the power of vector storage, start with creating a collection in the UI or via the SDK.

Creating collections

Creating a new collection is done with the following simple command:

from qwak.vector_store import VectorStoreClient

client = VectorStoreClient()
catalog_collection = client.create_collection(name="product_catalog",
                                              description="Indexing a product catalog of fashion items",
                                              dimension=384,
                                              metric="cosine",
                                              vectorizer="<optional>")

Name

The collection name must be unique across your account.

Dimension

This parameter determines the length of vectors supported within this collection. Please note that the dimension size remains immutable once the collection is created.

Metric type

Choose the index type that aligns with your specific search and retrieval requirements to maximize the effectiveness of your vector-based operations:

• Cosine similarity - ideal for semantic and conceptual similarity searches
• L2 distance - best for geometric distance calculations involving both direction and magnitude.

Vectorizer

A vectorizer is an embeddings model deployed on Qwak, and is an optional feature of the collection.

🚧

To use a vectorizer, you have to deploy an embedding models as a real-time model.

Use this example model or make sure that your model supports the following structure to automatically use the vector store conversion abilities.

# The model receives a dataframe with a single key named "input"
input_ = DataFrame(
  [{
    "input": "Why does it matter if a Central Bank has a negative rather than 0% interest rate?"
  }]
).to_json()

# The model returns a list of objects with a single key named "embeddings"
[
    {
        "embeddings": [
            0.0517567955,
            0.0195365045
        ]
    }
]

Deleting collections

Deleting a collection will delete all the data and vectors associated with the collection. Deleting a collection is irreversible, and deleted data cannot be restored.

Deleting a collection is done by using the collection name, which is unique across the account.

from qwak.vector_store import VectorStoreClient

client = VectorStoreClient()
client.delete_collection(name="product_catalog")

Getting all collections

To get a list of all the collections in your account, use the following code snippets:

from qwak.vector_store import VectorStoreClient

client = VectorStoreClient()
collections = client.list_collections()