Wazuh + AWS Bedrock: RAG with Titan Embeddings (Part 3)

Introduction

Ask the chat agent from Part 1 what the team runbook prescribes for an SSH brute force against web-server-01, or ask the MCP sidecar from Part 2 whether a file hash appeared in a threat intelligence feed last month, and the answer is whatever Claude remembers about the world in general. Both tools query alerts well. Neither of them can read a document, and the runbook, the manual and the feed never reach the model.

This part puts those documents into the Wazuh Indexer as one k-NN index, soc-knowledge, and lets the agent and the sidecar search it. The index holds 7,118 chunks from four sources: MITRE ATT&CK as the Wazuh manager serves it (1,031 documents), the Wazuh 4.14 user manual (3,854 chunks from 316 pages), four SOC playbooks (24 chunks) and 200 pinned MISP OSINT events (2,209 chunks). Amazon Titan Text Embeddings V2 computes the vectors through an ML Commons connector, so a client sends plain text and the Indexer stores the 1,024-float vector itself. The Part 1 agent gets two tools, a VectorDBTool that searches the index and a SearchIndexTool that reads one playbook from its first section to its last. The Part 2 sidecar needs no change.

This series has four parts.

• Part 1 - ML Commons, the Bedrock Claude connector, and the Dashboard chat agent
• Part 2 - opensearch-mcp-server-py as a sidecar, and Claude Desktop over MCP
• Part 3 (this article) - Titan Embeddings V2, a lucene k-NN index over four sources, and hybrid retrieval
• Part 4 - the same corpus in Amazon S3 Vectors through Bedrock Knowledge Bases, compared against this in-Indexer index (upcoming)

All of this ran on one installation. The index was built on 2026-09-03; the agent run, the query examples, the two negative controls and the cost figures are from 2026-09-04. The faiss crash and the first hybrid query for an indicator are older, from 2026-09-02, on an earlier build of the same index: I did not repeat the crash once the index was in use, and I repeated the hybrid query with a different hash. The MITRE, documentation and playbook counts rest on pinned inputs and should repeat on another installation; the MISP count rests on a pinned UUID list whose events are fetched live, so it moves if an event is edited or withdrawn; the scores depend on the embeddings of the day and may not repeat.

Prerequisites

The stack is the Part 1 stack: Docker and Docker Compose v2, the wazuh-docker single-node checkout with generated Indexer certificates, and at least 8 GB of RAM allocated to Docker. If Part 1 is running, this part adds a second connector, one index and two pipelines.

Part 1 ran on Wazuh 4.14.3. This part was measured on 4.14.7 with OpenSearch 2.19.5 and every ML plugin at 2.19.5.0, so the plugin versions in the Part 1 download commands change. On the Indexer side that means opensearch-skills and opensearch-flow-framework 2.19.5.0 from Maven; the image itself already ships opensearch-knn, opensearch-ml and opensearch-neural-search at the same version.

mkdir -p config/wazuh_indexer/plugins
cd config/wazuh_indexer/plugins

curl -L -O https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-flow-framework/2.19.5.0/opensearch-flow-framework-2.19.5.0.zip
curl -L -O https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-skills/2.19.5.0/opensearch-skills-2.19.5.0.zip

mkdir -p opensearch-flow-framework opensearch-skills
unzip opensearch-flow-framework-2.19.5.0.zip -d opensearch-flow-framework/
unzip opensearch-skills-2.19.5.0.zip         -d opensearch-skills/
rm -f *.zip

cd ../../..

The three Dashboard plugins come from the matching OpenSearch Dashboards 2.19.5 bundle.

curl https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.19.5/opensearch-dashboards-2.19.5-linux-x64.tar.gz -o opensearch-dashboards.tar.gz
tar -xzf opensearch-dashboards.tar.gz

mkdir -p config/wazuh_dashboard/plugins
cp -r opensearch-dashboards-2.19.5/plugins/observabilityDashboards config/wazuh_dashboard/plugins/
cp -r opensearch-dashboards-2.19.5/plugins/mlCommonsDashboards     config/wazuh_dashboard/plugins/
cp -r opensearch-dashboards-2.19.5/plugins/assistantDashboards     config/wazuh_dashboard/plugins/

rm -rf opensearch-dashboards-2.19.5* opensearch-dashboards.tar.gz

On the AWS side you need Bedrock model access to amazon.titan-embed-text-v2:0 and to us.anthropic.claude-sonnet-4-5-20250929-v1:0 in your region, plus bedrock:InvokeModel on both model ARNs. The keys come from a named AWS CLI profile through aws configure export-credentials and reach a single process; the connector body that carries them is written to a temporary file for the one request and deleted as soon as that process exits. The region throughout is us-east-1.

Why the Vector Index Lives in the Indexer

There were two places to keep the vectors: a separate service next to Wazuh, or the Indexer itself. I chose the Indexer, mainly because of where the embedding is computed. The ingest pipeline holds the model id, so a client writes a document with a text field and the Indexer calls Bedrock and stores text_embedding on its own. A neural query works the same way: it carries query_text and a model_id, and the Indexer embeds the question before searching. The loader needs no AWS credentials, no SDK and no embedding library, and any client that can send JSON to the Indexer can use the index. That is why the Part 2 sidecar searches the knowledge base with no new code.

It also meant installing nothing for search itself. The Wazuh Indexer image already carries opensearch-knn for the vector field, opensearch-ml for the remote model and opensearch-neural-search for the neural and hybrid queries, and a second service would have brought its own backups, its own access control and its own copy of the data.

One index serves all four sources. source is a keyword field, so a question that belongs to one corpus gets a term filter and a question that does not is run unscoped, and a single mapping keeps the chunk size, the vector dimension and the search pipeline identical for MITRE entries, manual pages, playbooks and MISP events.

The vector field’s engine was the first thing I got wrong. A knn_vector field declared with engine: faiss accepts the mapping with HTTP 200 and kills the node on the first document write. curl reports HTTP 000 for that write, since the connection dies in the middle of the request, and the container log explains why.

java.lang.UnsatisfiedLinkError: no opensearchknn_faiss in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2458)
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916)
--
fatal error in thread [opensearch[wazuh.indexer][write][T#13]], exiting

One crash is not the end of it: translog recovery replays the same write at the next start and the process exits again. The first time this happened the node exited twice before I could delete the index. In the controlled reproduction on 2026-09-02, in a throwaway index, the node answered again about 15 seconds after the write, the index was deleted, and the restart count of the container moved from 2 to 3.

The k-NN engines page for 2.19 lists the libopensearchknn_faiss*.so libraries OpenSearch ships for Faiss. On 2026-09-04 I listed the plugin directory of wazuh/wazuh-indexer:4.14.7 from a throwaway container with its entrypoint replaced, on both linux/amd64 and linux/arm64: each holds opensearch-knn-2.19.5.0.jar with its helper jars and zero libopensearchknn* files, at least in this image at this version. Lucene needs no native library; the same page describes it as the vector implementation inside Lucene itself.

After that first write I replaced faiss with lucene everywhere in this series and kept the rest of the method as it was: hnsw, cosinesimil, ef_construction 128 (the documented default is 100) and m 16 (the default). A Wazuh build that ships the native libraries would put faiss back on the table.

Titan Embeddings Connector and Model

Before writing any cluster setting I read the ones ML Commons already had, and left them alone. On this build agent_framework_enabled, memory_feature_enabled and rag_pipeline_feature_enabled are true by default, and the default trusted_connector_endpoints_regex list holds 11 patterns with bedrock-runtime among them. Part 1 replaced that list with a single Bedrock pattern; on 2.19.5.0 the replacement is unnecessary. plugins.ml_commons.only_run_on_ml_node stayed at its default true as well, and the remote model deployed with it in place; Part 1 set it to false, and the default worked here.

The connector follows the ML Commons blueprint for amazon.titan-embed-text-v2:0, including the two bedrock.embedding process functions that translate between the ML Commons text-embedding format and the Titan request and response.

POST /_plugins/_ml/connectors/_create
{
  "name": "soc-titan-embed-v2",
  "description": "Amazon Titan Text Embeddings V2 for the SOC knowledge index",
  "version": 1,
  "protocol": "aws_sigv4",
  "credential": {
    "access_key": "<AWS_ACCESS_KEY_ID>",
    "secret_key": "<AWS_SECRET_ACCESS_KEY>"
  },
  "parameters": {
    "region": "us-east-1",
    "service_name": "bedrock",
    "model": "amazon.titan-embed-text-v2:0",
    "dimensions": 1024,
    "normalize": true,
    "embeddingTypes": ["float"]
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/invoke",
      "headers": {
        "content-type": "application/json",
        "x-amz-content-sha256": "required"
      },
      "request_body": "{ \"inputText\": \"${parameters.inputText}\", \"dimensions\": ${parameters.dimensions}, \"normalize\": ${parameters.normalize}, \"embeddingTypes\": ${parameters.embeddingTypes} }",
      "pre_process_function": "connector.pre_process.bedrock.embedding",
      "post_process_function": "connector.post_process.bedrock.embedding"
    }
  ]
}

Register and deploy the model against that connector and keep the model_id; the ingest pipeline, the agent tool and every neural clause reference it.

POST /_plugins/_ml/models/_register?deploy=true
{
  "name": "soc-titan-embed-v2",
  "function_name": "remote",
  "description": "Titan Text Embeddings V2 for the SOC knowledge index",
  "connector_id": "<titan_connector_id>"
}

The smoke test goes through the ML Commons text-embedding predict endpoint, the same format the ingest pipeline and the neural query use.

POST /_plugins/_ml/_predict/text_embedding/<titan_model_id>
{
  "text_docs": ["What does our runbook say for an SSH brute force against web-server-01?"],
  "return_number": true,
  "target_response": ["sentence_embedding"]
}

The response holds one sentence_embedding output whose data array has 1,024 elements, the dimensions value of the connector, and a probe document sent through the ingest pipeline comes back with a vector of the same width. The Claude connector and model from Part 1 are registered alongside it, unchanged, and their smoke test with the prompt parameter passes.

Chunk Size, Pipelines and Mapping

I measured the chunk size instead of taking it from a rule of thumb. The Amazon Bedrock model guide gives Titan V2 a limit of 8,192 tokens or 50,000 characters and quotes 4.7 characters per token for English prose. The Wazuh manual is reStructuredText with directives, tables and code blocks, and I had no idea how far it sits from English prose. So I sent growing prefixes of a 150 KB page of the manual, source/user-manual/reference/internal-options.rst, straight to bedrock-runtime and read inputTextTokenCount back from each answer.

Within a single page the ratio moves from 4.99 to 10.48 characters per token, and no prefix up to 48,000 characters was refused. An average would have been wrong for the dense pages by a factor of two, so the size stays an explicit parameter: 3,000 characters with a 200-character overlap, about 600 tokens at the tightest ratio measured on this one page, and a manual in another language or a feed of long JSON attributes gets its own probe before the size is reused. Before the index is created the loader checks that four chunks of that size still fit inside the largest accepted prefix.

The ingest pipeline is one text_embedding processor.

PUT /_ingest/pipeline/soc-knowledge-embed
{
  "description": "Embed the text field with Titan Text Embeddings V2",
  "processors": [
    {
      "text_embedding": {
        "model_id": "<titan_model_id>",
        "field_map": { "text": "text_embedding" }
      }
    }
  ]
}

The search pipeline combines a keyword clause with a neural clause. The normalization processor scales both score lists with min_max and combines them with a weighted arithmetic mean; the documentation requires as many weights as there are sub-queries and a sum of 1.0. The pipeline is attached to the index as index.search.default_pipeline, so a hybrid query needs no search_pipeline parameter and behaves the same from the Dashboard, from curl and from MCP.

PUT /_search/pipeline/soc-hybrid
{
  "description": "Hybrid: exact indicators plus semantic text",
  "phase_results_processors": [
    {
      "normalization-processor": {
        "normalization": { "technique": "min_max" },
        "combination": {
          "technique": "arithmetic_mean",
          "parameters": { "weights": [0.7, 0.3] }
        }
      }
    }
  ]
}

The index declares the vector field, the source marker and the metadata needed to cite a result.

PUT /soc-knowledge
{
  "settings": {
    "index": {
      "knn": true,
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "search.default_pipeline": "soc-hybrid"
    }
  },
  "mappings": {
    "properties": {
      "source": { "type": "keyword" },
      "doc_id": { "type": "keyword" },
      "chunk": { "type": "integer" },
      "title": { "type": "text" },
      "text": { "type": "text" },
      "text_embedding": {
        "type": "knn_vector",
        "dimension": 1024,
        "method": {
          "name": "hnsw",
          "engine": "lucene",
          "space_type": "cosinesimil",
          "parameters": { "ef_construction": 128, "m": 16 }
        }
      },
      "indicators": { "type": "keyword" },
      "url": { "type": "keyword" },
      "version": { "type": "keyword" },
      "ingested_at": { "type": "date" }
    }
  }
}

indicators is a keyword field, and it is what makes the hybrid query work later on. A MISP event stores its hashes, addresses and domains there, a MITRE entry stores its own T-number, and a playbook stores the Wazuh rule ids it covers as rule-5712 and similar. A term query on that field matches exactly, an embedding never does, and the pipeline above merges the two score lists.

Four Sources, One Index

Every source has a selection rule that can be repeated later, so the index can be rebuilt and compared against this one.

• MITRE ATT&CK. Read from the Wazuh manager API, so the documents match the dataset the manager maps alerts against: 750 techniques, 267 mitigations and 14 tactics at mitre_version 2.0. Each technique becomes one document with its id, name, description, tactics, detection guidance and mitigation names.
• Wazuh documentation. Every .rst file under source/user-manual/ of the 4.14 branch at one pinned commit, e3296dd. The file list and the files themselves come from that commit: 316 files, one per manual page, split into sections on RST title underlines and then into chunks.
• Playbooks. Four Markdown runbooks, split on H1 and H2 headings. I wrote them for this article; they are not from a real SOC.
• MISP. 200 event UUIDs pinned from the manifest of the CIRCL OSINT feed, which listed 1,680 events at the 2026-09-02 snapshot. The UUID list is fixed, and each event is fetched again by its UUID on the day of the run. An event contributes its info line and its comment and text attributes as prose, and its hashes, addresses, domains and URLs as indicators.

The manager API forced one change on the MITRE loader. On this manager 224 of the 267 mitigations come back with an external_id that is a technique-style T-number, so a mitigation and a technique would share a document id and the later bulk upsert would replace the earlier document without a trace. Mitigations and tactics carry a type prefix in doc_id (mitigation:T1081, tactic:TA0003) while techniques keep the plain T-number.

The embedding route forced another. ML Commons substitutes the input into request_body as it is, and when the agent once passed a Query DSL object as its search input, the Titan route answered HTTP 400 with Invalid payload and the JSON echoed back inside inputText. Since then every documentation and playbook chunk starts with its section title, a chunk that still parses as JSON gets an Excerpt: prefix, and the search tool description tells the model to pass one plain-text sentence.

The playbooks are the smallest source and the one you will replace first with your own documents. They are also the source I test retrieval against. MITRE and the Wazuh manual are public text, so a model that answers an ATT&CK question correctly may have answered from memory; a runbook written for this article is a document the model cannot have seen. The question I ask the agent is what the runbook prescribes for an SSH brute force against web-server-01 and which Wazuh rules it covers. A correct answer contains facts that exist only in that document: the 12-hour block, the two rule ids this host’s playbook applies to, and an escalation triggered by a successful login after the failed burst.

The document under test:

# Playbook: SSH brute force against web-server-01

Scope: agent web-server-01 (Ubuntu 24.04, internet-facing). Applies to Wazuh rule 5712 (SSH brute force against a non-existent user) and rule 5763 (SSH brute force with failed authentication). Owner: SOC tier 1. Escalate to tier 2 when containment fails or when a successful login follows the burst.

## Detection

Read the alert fields data.srcip, data.dstuser and rule.frequency. Eight matches from one source within 120 seconds trigger rule 5712 when the failures target a non-existent user, or rule 5763 for failed authentication. If a successful login from the same address follows either burst within two minutes, preserve the authentication evidence, contain the source and escalate to tier 2. Never close the sequence as benign based on the success alone.

## Triage

Check whether data.srcip belongs to the corporate VPN range 10.20.0.0/16 or to a known scanner. Query the alert store for the same source across all agents for the last 24 hours. If the address hits more than one host, open a campaign ticket instead of a single-host ticket.

## Containment

Block the source address on the host firewall of web-server-01 with the active response firewall-drop for 12 hours. Confirm the action in /var/ossec/logs/active-responses.log and verify the firewall state on the endpoint. A zero count of later alerts is supporting evidence only, because the source may have stopped sending traffic. Do not disable the sshd service.

## Escalation

Escalate to tier 2 if any of the following hold: a successful login from the blocked address before the block, more than three hosts targeted by the same address, or a user account that is not in the expected administrator list.

## Closure

Record the source address, the number of failures, the block time and the ticket id in the case. Close within one business day if no escalation criterion was met.

The rule mapping follows the Wazuh 4.14.7 SSH rule definitions, where 5712 and 5763 both fire on eight matches within 120 seconds, and the containment step follows the Wazuh SSH brute-force active response use case, which blocks with firewall-drop on rule 5763 for 180 seconds in its example. The 12-hour block and the escalation criteria are mine; a production SOC sets its own. The other three runbooks have the same five sections: one covers an unexpected change to sshd_config under rules 550 and 553, one covers a web server process that spawned a shell under rule 100450, and one covers any alert at level 12 or above and points at the other playbooks by name for the cases they own.

A full load on 2026-09-03 produced, by a per-source _count after the load, 1,031 MITRE documents, 3,854 documentation chunks from 316 files, 24 playbook chunks from 4 files and 2,209 MISP chunks from 200 events, 7,118 documents in soc-knowledge. I ran it twice in a row and compared the two id sets after a scroll over the index: 7,118 lines each, byte-identical, with no duplicate _id and no duplicate (source, doc_id, chunk) triple. That comparison covers the ids only; I did not compare the stored text or the vectors between the two runs. The first run took about 15 minutes and the second about 25 by the log timestamps. Both ran through a similar number of throttling retries, 174 and 175, each cleared on the first retry after a 2-second pause, so the pauses account for about six minutes of each run and not for the gap between them, and I did not measure where the rest of the second run went.

bulk: 13 items throttled (429), retry 1/6 after 2s

Throttling arrives in a shape that is easy to miss. A throttled _bulk call returns HTTP 200 with errors: true and individual items carrying status 429, so a client that only checks the response code records a successful ingest with documents missing. The loader reads every item, re-sends only the ones that answered 429 or 5xx, backs off through 2, 4, 8, 16, 32 and 64 seconds, and gives up after six rounds; an item that failed for any other reason ends the run at once, and an answer that says errors: true without naming a failed item is refused as unreadable.

Before anything is deleted, every id the run wrote is read back in batches of 500 through _mget with _source=false as a query parameter; the same flag beside ids in the request body gets a parsing_exception from the Indexer, which is what stopped one rerun on 2026-09-03. Stale ids are pruned only after a full snapshot of a source: the loader lists what the index still holds for that source and deletes the difference against the ids it just sent. A rerun over unchanged inputs deletes nothing; a rerun over a changed source removes exactly what the source stopped producing.

The Search Tool and the Read-Through Tool

The agent keeps the four alert tools of Part 1 and gains two tools over the knowledge index. Their shape was set by what happened on live requests.

A search alone was not enough. SocKnowledgeSearch is a VectorDBTool that returns the best-matching chunks, and for the runbook question the best match is the Scope section of the SSH playbook, chunk 0 at a score of 0.866 in the run quoted below. With the search alone the model received five hits, and the Scope section was the only one from the playbook. The containment step lives in chunk 3 of the same document, so it never reached the model, and the answer could not name the 12-hour block. SocPlaybookRead exists for that: a SearchIndexTool that reads the whole playbook the hit belongs to, filtered by source, doc_id and version and sorted by chunk. For that to work the search hits have to carry doc_id, chunk and version beside the text, which is what the source_field list of the search tool requests.

The read tool’s description is shaped by the errors the tool returned before it worked. The SearchIndexTool expects one JSON object with two top-level keys, index and query, where query is the complete search request body; the documented example shows size and _source inside it. A bool clause placed directly under the top-level query is sent to OpenSearch as the whole body, and the tool answered Unknown key for a START_OBJECT in [bool]. The body also has to carry track_scores: true: with a field sort and no computed scores the tool failed with NaN is not a valid double value as per JSON specification, the Gson message, on this installation. Both rules are written into the tool description as a literal to copy, and the two tools look like this in the register body.

[
  {
    "type": "VectorDBTool",
    "name": "SocKnowledgeSearch",
    "description": "Semantic search over the SOC knowledge base: MITRE ATT&CK techniques and mitigations, Wazuh 4.14 documentation, the team's playbooks for an SSH brute force against web-server-01, an unexpected change to sshd_config, a web server process that spawned a shell and alerts at level 12 or higher, and recent MISP threat intelligence events. INPUT RULE: the input is the question or topic as one plain-text sentence, for example: SSH brute force playbook for web-server-01. NEVER pass JSON, a Query DSL object, an index name or field names as the input: this tool embeds the input text as it is, and a JSON input is refused by the embedding model. Every hit carries source, doc_id, chunk, title, text, url and version. A playbook hit is one section of a document; pass its doc_id and version to SocPlaybookRead to read the whole document.",
    "parameters": {
      "model_id": "<titan_model_id>",
      "index": "soc-knowledge",
      "embedding_field": "text_embedding",
      "source_field": [
        "source",
        "doc_id",
        "chunk",
        "title",
        "text",
        "url",
        "version"
      ],
      "doc_size": 5,
      "k": 10,
      "input": "${parameters.question}"
    }
  },
  {
    "type": "SearchIndexTool",
    "name": "SocPlaybookRead",
    "description": "Reads one playbook document of the SOC knowledge base through, in section order. INPUT RULE: the input is one JSON object with exactly two top-level keys, index and query. index is always the string soc-knowledge. The value of query is the COMPLETE search request body, and that body has its own key named query holding the bool filter, beside sort, size and _source. Never put bool directly under the top-level query, and never put sort, size or _source at the top level. Copy this literal input and replace only DOC_ID and VERSION with the doc_id and version of the SocKnowledgeSearch hit: {\"index\": \"soc-knowledge\", \"query\": {\"query\": {\"bool\": {\"filter\": [{\"term\": {\"source\": \"playbook\"}}, {\"term\": {\"doc_id\": \"DOC_ID\"}}, {\"term\": {\"version\": \"VERSION\"}}]}}, \"sort\": [{\"chunk\": \"asc\"}], \"track_scores\": true, \"size\": 20, \"_source\": [\"source\", \"doc_id\", \"chunk\", \"title\", \"text\", \"version\"]}}. track_scores must be true: this tool serialises the score of every hit, and a sort without computed scores makes it fail. Use the doc_id and version of the hit, never a guessed name. Every returned hit is one section of that one document; read them all before answering.",
    "parameters": {
      "index": "soc-knowledge"
    }
  }
]

The agent’s instruction lives under llm.parameters as prompt.prefix. The ML Commons 2.19 chat agent assembles its prompt from prompt, prompt.prefix and prompt.suffix (MLChatAgentRunner and AgentUtils.addPrefixSuffixToPrompt), and the Part 1 connector body sends the model only ${parameters.prompt}. An instruction placed under system_instruction, as the earlier definition in this series did, never reached the model on this path, because that connector body has no system field; once it moved to prompt.prefix, the required answer structure appeared in both languages. A connector that maps a system field behaves differently. The instruction asks for five labelled parts, Rules covered, Owner, Containment actions, Escalation conditions and Sources quoted, with the owner as one clause taken from the owner line of the document and Owner: not found when the context has none; a Russian question gets the same five labels in Russian.

The Dashboard chat is bound to the agent only once both language runs have passed, and the write is conditional: the loader reads the os_chat document first and sends its _seq_no and _primary_term with the update, so a document changed in the meantime answers 409 instead of being overwritten; the result is then read back. The document is {"type": "os_chat_root_agent", "configuration": {"agent_id": "<agent_id>"}} in .plugins-ml-config/_doc/os_chat, written with the Indexer admin certificate as in Part 1.

The Playbook, Read End to End

I ran one _execute per language with verbose: true and read the trace before the answer: the read step follows the search, its request names the index and the doc_id and version of a hit the search returned, its output is exactly the chunk set the second load wrote for that document (chunks 0 to 5 in order, with the section headings of the file), and every fact the answer states is present in the chunks the tools returned. On 2026-09-04 both languages satisfied all four conditions. The English answer, quoted from the log with the rule mapping part left out:

[...]

Owner: SOC tier 1.

Containment actions: Block the source address on the host firewall of web-server-01 using the active response firewall-drop for 12 hours. Confirm the action in /var/ossec/logs/active-responses.log and verify the firewall state on the endpoint. Do not disable the sshd service.

Escalation conditions: Escalate to tier 2 when containment fails, when a successful login follows the burst, if a successful login from the blocked address occurred before the block, if more than three hosts are targeted by the same address, or if a user account is not in the expected administrator list.

Sources quoted: ssh-brute-force-web-server-01.md version playbook-2026-09, sections: Playbook: SSH brute force against web-server-01, Detection, Containment, and Escalation.

This is one question, one document and one run per language. The runs earlier that day failed on the shape of the tool calls quoted above, not on the ranking of the index, and a runbook with a different structure would need its own look at what the read step returns.

Queries Through Every Door

Outside the agent, the plain queries are what an integration will copy. A neural clause with a source filter searches one corpus.

POST /soc-knowledge/_search
{
  "size": 5,
  "_source": ["source", "title", "doc_id", "url"],
  "query": {
    "neural": {
      "text_embedding": {
        "query_text": "Which technique covers password guessing against SSH and what mitigates it?",
        "model_id": "<titan_model_id>",
        "k": 5,
        "filter": { "term": { "source": "mitre" } }
      }
    }
  }
}

The same body with a different question and filter was run once per source on 2026-09-04.

Three corpora answer from the intended source with the intended section on top. The MISP corpus does worse: it returns a flat list of daily digests from events that carry little prose, and that is why indicators go through the hybrid query instead.

An embedding does little with a hash on its own (the numbers are below), so the exact clause is a keyword term and the neural clause supplies the context around it. The hash below was picked from the ingested MISP events on the day of the run.

POST /soc-knowledge/_search
{
  "size": 5,
  "_source": ["source", "title", "doc_id", "indicators"],
  "query": {
    "hybrid": {
      "queries": [
        { "term": { "indicators": "0d2211b7e92fcc6a9f7c94d4adf8e47f6f97e31dacd3b2ffb6cce3c485fcef26" } },
        {
          "neural": {
            "text_embedding": {
              "query_text": "Is this file hash known malware and which campaign does it belong to?",
              "model_id": "<titan_model_id>",
              "k": 10
            }
          }
        }
      ]
    }
  }
}

On 2026-09-04 the MISP event carrying the hash, Malicious File Creates Network Socket and Contacts fdh32fsdfhs.shop, came first at exactly 0.7000, and the closest semantic neighbour, a manual section on detecting malware with file hashes in a CDB list, came second at exactly 0.3000, out of 11 hits in 574 ms. The round numbers are the pipeline arithmetic: min_max scales each clause’s scores to a 0 to 1 range, the only hit of the term clause gets 1.0 there and nothing from the neural list, the top neural hit gets 1.0 there and nothing from the term list, and with weights 0.7 and 0.3 the exact match lands at 0.7 and the neighbour at 0.3. That is why the exact clause carries the larger weight; with the weights swapped, the neighbour would come first.

The neural clause on its own, with the same question plus the hash appended, a source: misp filter and k 5, returned five Maltrail IOC digests between 0.6772 and 0.6610 and did not return the event at all. On 2026-09-02, with a different hash from a CERT-FR Sandworm event, the event came first at 0.7000 through the hybrid pipeline and only third at 0.6645 through the neural clause alone.

The same body goes through the Part 2 sidecar. The argument name comes from this build’s live tools/list answer, where the SearchIndexTool schema is format, index, query_dsl and size; the hybrid query above goes in as the value of query_dsl, unchanged.

{ "jsonrpc": "2.0", "id": 1, "method": "tools/call",
  "params": { "name": "SearchIndexTool",
              "arguments": { "index": "soc-knowledge", "query_dsl": { ...the hybrid query above... } } } }

The sidecar accepted the object, returned isError: false and produced the same event first at 0.7 with the hash in its indicators array, 11 hits in 503 ms. The embedding is still computed in the Indexer and the weights still come from the default pipeline of the index; the sidecar only forwards the body, and the Claude Desktop side is unchanged from Part 2.

When the Right Answer Is Outdated or Missing

A retrieval layer that only ever answers correctly in a demo tells you nothing about the day it answers wrongly, so I ran two controls in which the right answer is outdated or missing, each in a throwaway index with the same mapping and search pipeline as the real one.

Stale corpus. I loaded the same three manual pages twice, from branch 4.7 at commit 288e2ee into soc-knowledge-stale (10 chunks) and from branch 4.14 at commit e3296dd into soc-knowledge-fresh (13 chunks), and asked both indices what the manager XML block for vulnerability detection is called. The stale index returned five chunks, the top one containing the retired name once, and the model answered:

Based on the documentation excerpt, the XML block for vulnerability detection is called: **vulnerability-detector**

The fresh index also returned five chunks, the second one containing the current name twice, and the model answered:

Based on the documentation, the XML block for vulnerability detection is called: **vulnerability-detection** This is shown in the exact tag format: `<vulnerability-detection>`

An index loaded from outdated pages produces a confident answer with the retired name, nothing in the response marks it as stale, and the record holds only the ranked list and the answer text, not which chunk the model used.

Silent miss. I loaded the 48 pages of the 4.14 vulnerability detection and ossec-conf directories, listed and fetched at the same pinned commit, into soc-knowledge-miss, excluding every file that matches vulnerability-detect(or|ion). That removed 4 files and left 1,099 chunks, and a scan over the raw text of all 1,099 confirms that neither tag name is in the index. The scan uses raw text because an analyzed query_string phrase for the same tag matched 12 chunks: the standard analyzer splits on hyphens, and those chunks contain the two words in unhyphenated prose. Retrieval on the miss index returned five ranked chunks with scores from 0.7946 down to 0.7547, the same picture as when the answer is present, because k-NN returns the k nearest vectors whether or not any of them is relevant. This time the model declined, and the refusal is its output of that day, not a property of the index; a different day or a different prompt can produce a confident wrong name from the same five chunks:

Based on the documentation provided, the XML tag name for vulnerability detection is not explicitly shown in these excerpts. The documentation discusses the Vulnerability Detection module and its functionality, but does not include a section showing the XML configuration block name for vulnerability

The quote ends mid-sentence because I keep only the first 300 characters of each control answer.

Scores interleave. The stale top five (0.7852, 0.7739, 0.7596, 0.7539, 0.7454) and the fresh top five (0.7946, 0.7931, 0.7591, 0.7528, 0.7414) cross each other in both directions, so no score threshold separates a chunk that answers the question from one that does not.

The crash. The faiss reproduction was not repeated on 2026-09-04: it restarts the node, and the index and the chat binding were in use that day. The evidence is the log of 2026-09-02 and the plugin listing of 2026-09-04, both in the engine section.

The controls never write to soc-knowledge; it held 7,118 documents before and after the runs of 2026-09-04, and the chat binding was untouched.

Cost, Operations and Limits

A full load embeds 7,118 chunks holding 10,885,546 characters of stored text: 1,678,191 from MITRE, 2,929,684 from the manual, 5,763 from the playbooks and 6,271,908 from MISP, counted on 2026-09-04 with a scroll over the index. The ratios from the token probe bound the token count: 10,885,546 / 10.48 = 1.04 million and 10,885,546 / 4.99 = 2.18 million input tokens. On 2026-09-04 the AWS Pricing API listed Titan Text Embeddings V2 on-demand in us-east-1 at $0.00002 per 1,000 input tokens (usage type USE1-TitanEmbeddingV2-Text-input-tokens, effective 2026-08-01), so a full load costs between 1.04 M x 0.00002 / 1,000 = $0.021 and 2.18 M x 0.00002 / 1,000 = $0.044 at that list price. The range cannot be narrowed from the Indexer: the connector’s post-process function keeps only the embedding and drops inputTextTokenCount. I did not read the invoice, so this is a list-price estimate. Titan V2 also supports Bedrock Batch, and AWS prices batch inference 50% below on-demand for supported models, but batch is a separate S3 job and not a switch on this pipeline. A query costs one embedding, which is negligible next to the Claude call that follows it.

Re-ingest when a source changes, not on a schedule: a new MITRE dataset after a manager upgrade, a new documentation commit, an edited playbook or a fresh MISP snapshot. The ids are sha1(source, doc_id, chunk) and the prune follows the read-back, so a rerun is harmless over unchanged inputs and exact over changed ones: there is no need to delete and recreate the index to drop a retired file, and the version field finds a generation with a term query.

When an answer is wrong, debug it in this order.

• Read the chunks first. Run the neural clause on its own with the same question and read the five texts that come back. If the answer is not in them, the model is not the problem.
• Then check the index. If the chunks are irrelevant, ask whether the source document is there at all: a term query on doc_id or on version settles it in one call, and the silent-miss control above is what that failure looks like from the outside.
• Only then look at generation. If the answer is in the retrieved chunks and the response contradicts them, the prompt or the model is at fault, and only then is changing the agent instruction the right move.

The measurements behind this article, in one place:

Keeping the index inside the Indexer is the better choice when the corpus is this size and the readers already talk to OpenSearch: there is no second service to secure and back up, the embedding is computed server-side, and the alert store and the knowledge base share one access control and one endpoint. It is the worse choice when the vector engine matters (this image offers lucene only), when the corpus grows far beyond a single node, or when alert ingestion already claims the Indexer’s CPU and heap, since every embedding call and every k-NN search runs on that same node. Part 4 loads the same four sources into Amazon S3 Vectors through Bedrock Knowledge Bases and compares the two against the same questions.

Conclusion

A knowledge base of 7,118 chunks inside the Wazuh Indexer, queried the same way by the Dashboard chat, by curl and by the MCP sidecar, costs a few cents to load and one embedding per question. The agent reads a playbook end to end and names the rule ids, the owner, the containment step and the escalation conditions from the document, and an exact indicator lookup ranks the event carrying the hash first where a pure embedding search missed it on one day and ranked it third on the other. The controls draw the boundary: an outdated corpus answers with confidence, a missing document still returns five neighbours, and no score tells the two apart.

Series Navigation:

• Part 1: ML Commons + Bedrock Connector
• Part 2: OpenSearch MCP Server + Claude Desktop
• Part 3: RAG with Titan Embeddings and k-NN (you are here)
• Part 4: S3 Vectors and Bedrock Knowledge Bases (upcoming)

Related Reading

• Ollama in Wazuh Dashboard: AI Security Analysis
• Wazuh Documentation RAG: Part 2

See also

• Wazuh Rule Static Analysis: Linter Evolution
• Wazuh MCP Server: Claude Desktop + OpenSearch (Part 2)
• Wazuh + AWS Bedrock: AI Security in Docker (Part 1)
• From Wazuh Ambassador to AWS Community Builder
• Static Analysis Tool for Wazuh Decoder XML Files