Weaviate: Vektor-Datenbank angetrieben von GraphQL

Warum Weaviate ?

• GraphQL-API (flexible Abfragen)
• Integrierte Vectorisierungs-Module
• Hybride Suche (vector + BM25)
• Generative Suche (integriertes RAG)
• Open-Source + verwalteter Cloud

Docker-Konfiguration

DEVELOPERbash
docker run -p 8080:8080 -p 50051:50051 semitechnologies/weaviate:latest

Oder mit docker-compose :

DEVELOPERyaml
version: '3.8'
services:
  weaviate:
    image: semitechnologies/weaviate:1.24.6
    ports:
      - "8080:8080"
      - "50051:50051"
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
      ENABLE_MODULES: 'text2vec-openai,generative-openai'
      OPENAI_APIKEY: ${OPENAI_API_KEY}

Python-Client

DEVELOPERpython
import weaviate

client = weaviate.Client("http://localhost:8080")

# Create schema
schema = {
    "class": "Document",
    "vectorizer": "text2vec-openai",
    "moduleConfig": {
        "text2vec-openai": {
            "model": "text-embedding-3-small"
        }
    },
    "properties": [
        {
            "name": "content",
            "dataType": ["text"],
            "moduleConfig": {
                "text2vec-openai": {
                    "skip": False,
                    "vectorizePropertyName": False
                }
            }
        },
        {
            "name": "title",
            "dataType": ["text"]
        },
        {
            "name": "category",
            "dataType": ["text"]
        }
    ]
}

client.schema.create_class(schema)

Dokumenteinfügung

DEVELOPERpython
# Auto-vectorization
client.data_object.create(
    class_name="Document",
    data_object={
        "content": "Weaviate is a vector database...",
        "title": "Introduction to Weaviate",
        "category": "tutorial"
    }
)

# Batch import (faster)
with client.batch as batch:
    batch.batch_size = 100

    for doc in documents:
        batch.add_data_object(
            class_name="Document",
            data_object={
                "content": doc['text'],
                "title": doc['title'],
                "category": doc['category']
            }
        )

Semantische Suche (GraphQL)

DEVELOPERpython
# nearText search
result = (
    client.query
    .get("Document", ["content", "title", "category"])
    .with_near_text({"concepts": ["vector database tutorial"]})
    .with_limit(5)
    .do()
)

print(result["data"]["Get"]["Document"])

Hybride Suche

Kombinieren vector + Stichwortsuche :

DEVELOPERpython
result = (
    client.query
    .get("Document", ["content", "title"])
    .with_hybrid(
        query="machine learning models",
        alpha=0.5  # 0=BM25, 1=vector, 0.5=ausgewogen
    )
    .with_limit(10)
    .do()
)

Filterung

DEVELOPERpython
# Filter by category
result = (
    client.query
    .get("Document", ["content", "title"])
    .with_near_text({"concepts": ["python tutorial"]})
    .with_where({
        "path": ["category"],
        "operator": "Equal",
        "valueText": "programming"
    })
    .with_limit(5)
    .do()
)

Generative Suche (integriertes RAG)

DEVELOPERpython
# Generate answer from retrieved documents
result = (
    client.query
    .get("Document", ["content", "title"])
    .with_near_text({"concepts": ["how to use embeddings"]})
    .with_generate(
        single_prompt="Résume ce document : {content}"
    )
    .with_limit(3)
    .do()
)

# Access generated text
for doc in result["data"]["Get"]["Document"]:
    print(doc["_additional"]["generate"]["singleResult"])

Mehrmandantenfähigkeit

DEVELOPERpython
# Create tenants
client.schema.add_class_tenants(
    class_name="Document",
    tenants=[
        {"name": "tenant_a"},
        {"name": "tenant_b"}
    ]
)

# Query specific tenant
result = (
    client.query
    .get("Document", ["content"])
    .with_tenant("tenant_a")
    .with_near_text({"concepts": ["query"]})
    .do()
)

Replikation

DEVELOPERyaml
# docker-compose.yml with 3 nodes
services:
  weaviate-node1:
    image: semitechnologies/weaviate:latest
    environment:
      CLUSTER_HOSTNAME: 'node1'
      CLUSTER_GOSSIP_BIND_PORT: '7100'
      CLUSTER_DATA_BIND_PORT: '7101'

  weaviate-node2:
    image: semitechnologies/weaviate:latest
    environment:
      CLUSTER_HOSTNAME: 'node2'
      CLUSTER_JOIN: 'weaviate-node1:7100'

RAG-Pipeline in Python

DEVELOPERpython
def weaviate_rag(query):
    # Retrieve with generative search
    result = (
        client.query
        .get("Document", ["content", "title"])
        .with_near_text({"concepts": [query]})
        .with_generate(
            grouped_task=f"Réponds à cette question : {query}",
            grouped_properties=["content"]
        )
        .with_limit(5)
        .do()
    )

    # Extract answer
    answer = result["data"]["Get"]["Document"][0]["_additional"]["generate"]["groupedResult"]

    return answer

# Usage
answer = weaviate_rag("What is machine learning?")
print(answer)

Die GraphQL-Oberfläche von Weaviate und das integrierte RAG machen es ideal für schnelles Prototyping.