Diagramme und Schemata : Visuelle Informationen extrahieren

Diagramme, technische Schemata und Infografiken enthalten eine enorme Informationsdichte. Eine Systemarchitektur, ein Flowchart oder ein Schaltplan kodiert Stunden an Dokumentation in einem einzigen Bild. Dieser Leitfaden zeigt, wie Sie diese visuellen Inhalte in Ihrem RAG-System durchsuchbar machen.

Die Herausforderung der Diagramme

Warum das anders ist als Fotos

Anwendungsfälle im Unternehmen

• IT : Systemarchitekturen, UML-Diagramme, Netzwerkschemata
• Industrie : Technische Pläne, elektrische Schaltpläne, P&ID
• Business : Organigramme, Prozessmaps, Flowcharts
• Data : ERD-Diagramme, Data Lineage, Pipelines
• Marketing : Infografiken, Präsentationen, visuelle Reports

ROI der Indexierung

• -80% Suchzeit in der technischen Dokumentation
• +50% Verständnis komplexer Systeme
• Nachvollziehbarkeit : Ursprung einer Architekturentscheidung wiederfinden

Typen von Diagrammen und Strategien

Strategie nach Typ

Extraction avec vision models

Prompt engineering pour diagrammes

DEVELOPERpython
from openai import OpenAI
import base64

def extract_diagram_info(
    image_path: str,
    diagram_type: str = "auto",
    client: OpenAI = None
) -> dict:
    """
    Extrahiert die strukturierten Informationen eines Diagramms.
    """
    if client is None:
        client = OpenAI()

    with open(image_path, "rb") as f:
        img_base64 = base64.b64encode(f.read()).decode("utf-8")

    prompts = {
        "flowchart": """Analyse ce flowchart:
1. Etapes principales dans l'ordre
2. Points de decision et branches
3. Flux de donnees principal et alternatives
4. Points d'entree/sortie""",

        "architecture": """Analyse cette architecture:
1. Composants (services, BDD, APIs)
2. Connexions et protocoles
3. Layers (frontend, backend, data, infra)
4. Technologies mentionnees""",

        "auto": """Analyse ce diagramme:
1. Type de diagramme
2. Elements visibles
3. Relations entre elements
4. Texte visible
5. Contexte d'usage"""
    }

    prompt = prompts.get(diagram_type, prompts["auto"])

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}", "detail": "high"}}
            ]
        }],
        max_tokens=2000
    )

    return {"extraction": response.choices[0].message.content}

Generation de code Mermaid

DEVELOPERpython
def diagram_to_mermaid(image_path: str, client: OpenAI) -> str:
    """Konvertiert ein Diagramm in Mermaid-Code."""
    with open(image_path, "rb") as f:
        img_base64 = base64.b64encode(f.read()).decode("utf-8")

    prompt = """Convertis ce diagramme en code Mermaid.
Exemple:
```mermaid
graph TD
    A[Start] --> B{Decision}
    B -->|Yes| C[Action 1]
    B -->|No| D[Action 2]
```"""

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}", "detail": "high"}}
            ]
        }],
        max_tokens=2000
    )

    return response.choices[0].message.content

Indexation pour le RAG

Structure de donnees

DEVELOPERpython
from dataclasses import dataclass, field
from typing import Optional, List

@dataclass
class DiagramDocument:
    diagram_id: str
    source_file: str
    diagram_type: str
    description: str
    extracted_text: str
    components: List[str]
    relationships: List[dict]
    mermaid_code: Optional[str]
    domain: str
    technologies: List[str] = field(default_factory=list)
    keywords: List[str] = field(default_factory=list)

    def to_embedding_text(self) -> str:
        parts = [
            f"Type: {self.diagram_type}",
            f"Domain: {self.domain}",
            f"Description: {self.description}",
        ]
        if self.components:
            parts.append(f"Components: {', '.join(self.components)}")
        if self.technologies:
            parts.append(f"Technologies: {', '.join(self.technologies)}")
        return "\n".join(parts)

Pipeline d'indexation

DEVELOPERpython
from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, PointStruct

class DiagramRAGPipeline:
    def __init__(self):
        self.qdrant = QdrantClient(url="http://localhost:6333")
        self.openai = OpenAI()
        self.collection_name = "diagram_rag"

    def process_diagram(self, image_path: str, diagram_type: str = "auto") -> DiagramDocument:
        extraction = extract_diagram_info(image_path, diagram_type, self.openai)
        mermaid = diagram_to_mermaid(image_path, self.openai)

        return DiagramDocument(
            diagram_id=hashlib.md5(image_path.encode()).hexdigest(),
            source_file=image_path,
            diagram_type=diagram_type,
            description=extraction['extraction'],
            extracted_text="",
            components=[],
            relationships=[],
            mermaid_code=mermaid,
            domain="general"
        )

    def index_diagram(self, doc: DiagramDocument):
        text = doc.to_embedding_text()
        response = self.openai.embeddings.create(
            model="text-embedding-3-small",
            input=text
        )

        point = PointStruct(
            id=hash(doc.diagram_id) % (2**63),
            vector=response.data[0].embedding,
            payload={
                "diagram_id": doc.diagram_id,
                "source_file": doc.source_file,
                "content": text,
                "mermaid_code": doc.mermaid_code
            }
        )

        self.qdrant.upsert(collection_name=self.collection_name, points=[point])

Kosten und Performance

Kosten pro Diagramm

Genauigkeit nach Typ

Integration mit Ailog

Ailog unterstützt die Indexierung von Diagrammen :

1. Upload : PNG, JPG, SVG, PDF
2. Detection automatique : Type de diagramme
3. Extraction intelligente : Composants et relations
4. Code genere : Mermaid reproductible

Essayez le Diagram RAG sur Ailog

Verwandte Leitfäden

• Guide complet du RAG multimodal - Article pilier
• RAG sur images : Vision models
• Parsing de tableaux pour le RAG
• OCR et documents scannes