Diagrammes et schemas : Extraire l'information visuelle

Les diagrammes, schemas techniques et infographies contiennent une densite d'information enorme. Une architecture systeme, un flowchart ou un schema electrique encode des heures de documentation en une seule image. Ce guide vous montre comment rendre ces contenus visuels cherchables dans votre systeme RAG.

Le defi des diagrammes

Pourquoi c'est different des photos

Cas d'usage en entreprise

• IT : Architectures systeme, diagrammes UML, schemas reseau
• Industrie : Plans techniques, schemas electriques, P&ID
• Business : Organigrammes, process maps, flowcharts
• Data : Diagrammes ERD, data lineage, pipelines
• Marketing : Infographies, presentations, rapports visuels

ROI de l'indexation

• -80% temps de recherche dans la documentation technique
• +50% comprehension des systemes complexes
• Tracabilite : Retrouver l'origine d'une decision d'architecture

Types de diagrammes et strategies

Strategie par type

Extraction avec vision models

Prompt engineering pour diagrammes

DEVELOPERpython
from openai import OpenAI
import base64

def extract_diagram_info(
    image_path: str,
    diagram_type: str = "auto",
    client: OpenAI = None
) -> dict:
    """
    Extrait les informations structurees d'un diagramme.
    """
    if client is None:
        client = OpenAI()

    with open(image_path, "rb") as f:
        img_base64 = base64.b64encode(f.read()).decode("utf-8")

    prompts = {
        "flowchart": """Analyse ce flowchart:
1. Etapes principales dans l'ordre
2. Points de decision et branches
3. Flux de donnees principal et alternatives
4. Points d'entree/sortie""",

        "architecture": """Analyse cette architecture:
1. Composants (services, BDD, APIs)
2. Connexions et protocoles
3. Layers (frontend, backend, data, infra)
4. Technologies mentionnees""",

        "auto": """Analyse ce diagramme:
1. Type de diagramme
2. Elements visibles
3. Relations entre elements
4. Texte visible
5. Contexte d'usage"""
    }

    prompt = prompts.get(diagram_type, prompts["auto"])

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}", "detail": "high"}}
            ]
        }],
        max_tokens=2000
    )

    return {"extraction": response.choices[0].message.content}

Generation de code Mermaid

DEVELOPERpython
def diagram_to_mermaid(image_path: str, client: OpenAI) -> str:
    """Convertit un diagramme en code Mermaid."""
    with open(image_path, "rb") as f:
        img_base64 = base64.b64encode(f.read()).decode("utf-8")

    prompt = """Convertis ce diagramme en code Mermaid.
Exemple:
```mermaid
graph TD
    A[Start] --> B{Decision}
    B -->|Yes| C[Action 1]
    B -->|No| D[Action 2]
```"""

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}", "detail": "high"}}
            ]
        }],
        max_tokens=2000
    )

    return response.choices[0].message.content

Indexation pour le RAG

Structure de donnees

DEVELOPERpython
from dataclasses import dataclass, field
from typing import Optional, List

@dataclass
class DiagramDocument:
    diagram_id: str
    source_file: str
    diagram_type: str
    description: str
    extracted_text: str
    components: List[str]
    relationships: List[dict]
    mermaid_code: Optional[str]
    domain: str
    technologies: List[str] = field(default_factory=list)
    keywords: List[str] = field(default_factory=list)

    def to_embedding_text(self) -> str:
        parts = [
            f"Type: {self.diagram_type}",
            f"Domain: {self.domain}",
            f"Description: {self.description}",
        ]
        if self.components:
            parts.append(f"Components: {', '.join(self.components)}")
        if self.technologies:
            parts.append(f"Technologies: {', '.join(self.technologies)}")
        return "\n".join(parts)

Pipeline d'indexation

DEVELOPERpython
from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, PointStruct

class DiagramRAGPipeline:
    def __init__(self):
        self.qdrant = QdrantClient(url="http://localhost:6333")
        self.openai = OpenAI()
        self.collection_name = "diagram_rag"

    def process_diagram(self, image_path: str, diagram_type: str = "auto") -> DiagramDocument:
        extraction = extract_diagram_info(image_path, diagram_type, self.openai)
        mermaid = diagram_to_mermaid(image_path, self.openai)

        return DiagramDocument(
            diagram_id=hashlib.md5(image_path.encode()).hexdigest(),
            source_file=image_path,
            diagram_type=diagram_type,
            description=extraction['extraction'],
            extracted_text="",
            components=[],
            relationships=[],
            mermaid_code=mermaid,
            domain="general"
        )

    def index_diagram(self, doc: DiagramDocument):
        text = doc.to_embedding_text()
        response = self.openai.embeddings.create(
            model="text-embedding-3-small",
            input=text
        )

        point = PointStruct(
            id=hash(doc.diagram_id) % (2**63),
            vector=response.data[0].embedding,
            payload={
                "diagram_id": doc.diagram_id,
                "source_file": doc.source_file,
                "content": text,
                "mermaid_code": doc.mermaid_code
            }
        )

        self.qdrant.upsert(collection_name=self.collection_name, points=[point])

Couts et performance

Couts par diagramme

Precision par type

Integration avec Ailog

Ailog supporte l'indexation de diagrammes :

1. Upload : PNG, JPG, SVG, PDF
2. Detection automatique : Type de diagramme
3. Extraction intelligente : Composants et relations
4. Code genere : Mermaid reproductible

Essayez le Diagram RAG sur Ailog

Guides connexes

• Guide complet du RAG multimodal - Article pilier
• RAG sur images : Vision models
• Parsing de tableaux pour le RAG
• OCR et documents scannes