Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
72 changes: 70 additions & 2 deletions assets/js/mermaid.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,77 @@
{{ if .enable }}
(function($) {
var needMermaid = false;
$('.language-mermaid').parent().replaceWith(function() {
function preprocessMermaid(text) {
var fm = text.match(/^\s*---\s*\n([\s\S]*?)\n---\s*\n?/);
var directive = '';
var body = text;
if (fm) {
var yaml = fm[1];
var obj = {};
var current = null;
yaml.split('\n').forEach(function(line) {
var l = line.trim();
if (!l) return;
if (/^[\w-]+\s*:\s*$/.test(l)) {
current = l.replace(/:\s*$/, '').trim();
obj[current] = obj[current] || {};
return;
}
var m = l.match(/^(\w[\w-]*)\s*:\s*(.*)$/);
if (m) {
var k = m[1];
var v = m[2].trim();
v = v.replace(/^['"]|['"]$/g, '');
if (current) {
obj[current][k] = v;
} else {
obj[k] = v;
}
}
});
var cfg = obj.config || obj;
directive = '%%{init: ' + JSON.stringify(cfg) + '}%%\n';
body = text.replace(fm[0], '');
}
body = body.replace(/<br\s*\/?>(?![^`]*`)/gi, '\\n');
body = body.split('\n').map(function(line) {
var m = line.match(/^\s*([A-Za-z0-9_]+)\s*@\{\s*([^}]*)\s*\}\s*$/);
if (m) {
var id = m[1];
var props = m[2];
var lm = props.match(/label\s*:\s*("[^"]*"|'[^']*'|[^,]+)/);
var label = lm ? lm[1].replace(/^['"]|['"]$/g, '') : id;
return id + '["' + label + '"]';
}
return line;
}).join('\n');
return directive + body;
}

function isMermaidLike(text) {
var t = text.trim();
return /^%%\{init:/.test(t) || /^---\s*/.test(t) || /^(graph|flowchart|sequenceDiagram|classDiagram|stateDiagram|gantt|pie)\b/.test(t);
}

var toReplace = [];
$('pre > code.language-mermaid').each(function() { toReplace.push($(this)); });
$('pre > code').each(function() {
var $c = $(this);
if ($c.hasClass('language-mermaid')) return;
if (isMermaidLike($c.text())) toReplace.push($c);
});

toReplace.forEach(function($code) {
needMermaid = true;
return $('<pre class="mermaid">').text($(this).text());
var raw = $code.text();
var processed = preprocessMermaid(raw);
var $wrapper = $code.closest('div[class*=language-]');
var $new = $('<pre class="mermaid">').text(processed);
if ($wrapper.length) {
$wrapper.replaceWith($new);
} else {
$code.parent().replaceWith($new);
}
});

if (!needMermaid) {
Expand Down
24 changes: 24 additions & 0 deletions content/en/docs/eino/Eino: Contribution Guidelines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
Description: ""
date: "2025-07-21"
lastmod: ""
tags: []
title: 'Eino: Contribution Guidelines'
weight: 7
---

Forms of code contribution for Eino:

1. Contributors submit code via a Fork + Pull Request.
1. When submitting a PR, please clearly describe the context and rationale of the change.
2. Eino maintainers regularly review incoming PRs and discuss the changes in PR conversations.
3. Once the code meets merge criteria, Eino maintainers trigger tests and merge the PR.

# Component Contributions

Component implementations are typically hosted in the `Eino-Ext` repository under dedicated directories, each as a standalone Go submodule for independent development and maintenance.

- Must include both Chinese and English `README` files.
- Must include comprehensive and accurate `Examples`.
- Must inject callback.Handler aspects (callback hook points) properly.

143 changes: 143 additions & 0 deletions content/en/docs/eino/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
---
Description: ""
date: "2025-12-01"
lastmod: ""
tags: []
title: FAQ
weight: 6
---

# Q: cannot use openapi3.TypeObject (untyped string constant "object") as *openapi3.Types value in struct literal; cannot use types (variable of type string) as *openapi3.Types value in struct literal

Ensure the `github.com/getkin/kin-openapi` dependency version does not exceed `v0.118.0`.

# Q: During agent streaming, it never reaches ToolsNode, or streaming is lost and appears non-streaming.

- First, update Eino to the latest version.

Different models output tool calls differently in streaming mode. Some models (e.g., OpenAI) emit tool calls directly; others (e.g., Claude) might emit text first and then the tool call. You therefore need different logic to determine whether a tool call is present in a streamed message.

The ReAct Agent `Config` has a `StreamToolCallChecker`. If omitted, the agent uses a default checker that determines a tool call by inspecting whether any non-empty early chunk contains tool calls:

```go
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()

for {
msg, err := sr.Recv()
if err == io.EOF {
return false, nil
}
if err != nil {
return false, err
}

if len(msg.ToolCalls) > 0 {
return true, nil
}

if len(msg.Content) == 0 { // skip empty chunks at the front
continue
}

return false, nil
}
}
```

This default works when the Tool Call message contains only a tool call.

It does not fit cases where a non-empty content chunk appears before the tool call. In such cases, provide a custom checker like this:

```go
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()
for {
msg, err := sr.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
// finish
break
}
return false, err
}
if len(msg.ToolCalls) > 0 {
return true, nil
}
}
return false, nil
}
```

Note: this custom `StreamToolCallChecker` checks all chunks for tool calls. When the model is outputting a normal answer, this may reduce “early streaming detection”, because it waits until all chunks are inspected. To preserve streaming responsiveness, try guiding the model with prompts:

> 💡
> Add prompt constraints such as: “If a tool is required, output only the tool call; do not output text.”
>
> Models vary in how much they adhere to such prompts. Tune and validate for your chosen model.

# Q: [github.com/bytedance/sonic/loader](http://github.com/bytedance/sonic/loader): invalid reference to runtime.lastmoduledatap

Older versions of `sonic` are incompatible with `go1.24`. Upgrade to `v1.13.2` or higher.

# Q: Tool input deserialization failed: `failed to invoke tool call {tool_call_id}: unmarshal input fail`

Models typically do not produce invalid JSON. Investigate the specific reason for deserialization failure; in most cases this is due to output truncation when the model’s response exceeds limits.

# Q: How can I implement batch processing nodes in Eino (like Coze’s batch nodes)?

Eino currently does not support batch processing. Two options:

1. Dynamically build the graph per request — the overhead is low. Note that `Chain Parallel` requires the number of parallel nodes to be greater than one.
2. Implement a custom batch node and handle batching inside the node.

# Q: Panic occurs in Fornax SDK or panic stack mentions Fornax SDK

Upgrade both the Fornax SDK and Eino to the latest versions and retry.

# Q: Does Eino support structured model outputs?

Yes, in two steps:

1. Ensure the model outputs structured data. Options:
- Some models support configuration for structured output (e.g., OpenAI response format).
- Use tool calls to obtain structured results.
- Prompt the model explicitly to output structured data.
2. After obtaining a structured message, use `schema.NewMessageJSONParser` to parse the message into your target struct.

# Q: In image recognition scenarios, error: `One or more parameters specified in the request are not valid`

Check whether the model supports image input (for Doubao models, only variants with `vision` support it).

# Q: How to access Reasoning Content / “thinking” output from a chat model?

If a model implementation supports Reasoning Content (deep thinking), it is stored in the `Extra` field of the output `Message`. The provider package usually offers helpers like `GetReasoningContent`/`GetThinking` to extract it.

# Q: Errors include `context deadline exceeded`, `timeout`, or `context canceled`

This indicates a timeout or that the `ctx` was canceled by a service/framework or manually. Adjust timeouts or investigate your code. If model/component latency seems abnormally high, contact the provider directly. Eino only passes through.

# Q: How to access parent graph `State` within a subgraph?

If both parent and subgraph have `state`, the subgraph’s state overrides the parent’s. Define a custom context key. Before invoking the subgraph, call `compose.ProcessState()` to retrieve the parent state and pass it via your custom context key.

# Q: How does `eino-ext` adapt multimodal input/output for supported models?

For multimodal support, see [https://www.cloudwego.io/en/docs/eino/ecosystem_integration/chat_model](https://www.cloudwego.io/en/docs/eino/ecosystem_integration/chat_model) and the corresponding examples for each model.

# Q: Using `UserInputMultiContent` for multimodal input, but the model side seems to miss the data or cannot read `multicontent`

Recent versions of Eino introduce `UserInputMultiContent` and `AssistantGenMultiContent` for multimodal user input and model output respectively. All `eino-ext` chat model implementations have been adapted. If the model does not receive the multimodal payload, upgrade the provider package to the latest version and try again.

# Q: After upgrading to `0.6.x`, there are breaking changes

Per the migration plan [Migration from OpenAPI 3.0 Schema Object to JSONSchema in Eino · Discussion #397](https://github.com/cloudwego/eino/discussions/397), Eino `v0.6.1` removed the dependency on `getkin/kin-openapi` and all OpenAPI 3.0-related code.

If `eino-ext` modules error with `undefined: schema.NewParamsOneOfByOpenAPIV3`, upgrade those modules to the latest versions.

If schema migration is complex, use the helper tooling in the [JSONSchema conversion guide](https://bytedance.larkoffice.com/wiki/ZMaawoQC4iIjNykzahwc6YOknXf).

# Q: `context canceled` / `context deadline exceeded`

The `ctx` passed to Eino was canceled or timed out; this is unrelated to the framework itself.

65 changes: 4 additions & 61 deletions content/en/docs/eino/_index.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,15 @@
---
Description: Eino is an AI application development framework based on Go
date: "2025-02-21"
Description: Eino is a Golang-based AI application development framework
date: "2025-11-20"
lastmod: ""
linktitle: Eino
menu:
main:
parent: Documentation
parent: Docs
weight: 6
tags: []
title: 'Eino: User Manual'
title: Eino User Manual
weight: 6
---

> Eino pronunciation: US / 'aino /, approximate sound: i know, with the hope that the application can achieve the vision of "i know"

## What is Eino

> 💡
> Eino:An AI Application Development Framework Built with Go

Eino aims to provide an AI application development framework built with Go. Eino refers to many excellent AI application development frameworks in the open-source community, such as LangChain, LangGraph, LlamaIndex, etc., and provides an AI application development framework that is more in line with the programming habits of Go.

Eino provides rich capabilities such as **atomic components**, **integrated components**, **component orchestration**, and **aspect extension** that assist in AI application development, which can help developers more simply and conveniently develop AI applications with a clear architecture, easy maintenance, and high availability.

Take ReactAgent as an example:

- It provides common components such as ChatModel, ToolNode, and PromptTemplate, and the business can be customized and extended.
- Flexible orchestration can be performed based on existing components to generate integrated components for use in business services.

<a href="/img/eino/en_eino_graph_2.png" target="_blank"><img src="/img/eino/en_eino_graph_2.png" width="100%" /></a>

## Eino Components

> One of Eino's goals is: to collect and improve the component system in the context of AI applications, so that the business can easily find some common AI components, facilitating the iteration of the business.
>
> Eino will provide components with a relatively good abstraction around the scenarios of AI applications, and provide some common implementations around this abstraction.

- The abstract definition of Eino components is in: [eino/components](https://github.com/cloudwego/eino/tree/main/components)
- The implementation of Eino components is in: [eino-ext/components](https://github.com/cloudwego/eino-ext/tree/main/components)

## Eino application scenarios

Thanks to the lightweight and in-field affinity properties of Eino, users can introduce powerful large model capabilities to their existing microservices with just a few lines of code, allowing traditional microservices to evolve with AI genes.

When people hear the term "Graph Orchestration", their first reaction might be to segment and layer the implementation logic of the entire application interface, and convert it into an orchestratable Node. The biggest problem encountered in this process is the issue of **long-distance context passing (variable passing across Node nodes)**, whether using the State of Graph/Chain or using Options for transparent passing, the entire orchestration process is extremely complex, far from being as simple as directly making function calls.

Based on the current Graph orchestration capabilities, the scenarios suitable for orchestration have the following characteristics:

- The overall focus is on the semantic processing-related capabilities of the model. Here, semantics are not limited to modalities.
- In orchestration output, there are very few nodes related to Session. Overall, the vast majority of nodes do not have processing logic at the granularity of unenumerable business entities such as users/devices.
- Whether through the State of Graph/Chain or through CallOptions, the methods for reading, writing, or transparently passing user/device granularity information are not convenient.
- Require common aspect capabilities, and build horizontal governance capabilities such as observation, rate limiting, and evaluation based on this.

What is the significance of orchestration: To aggregate, control, and present the context of long-distance orchestration elements in a fixed paradigm.

- Context of long-distance orchestration elements: Generally, orchestration elements are scattered throughout the system of the entire orchestration product, making it difficult for people to have a macroscopic and overall cognition. The role of orchestration is to gather and present this macroscopic information.
- Aggregation control and presentation: It is to easily organize and control the relationship between orchestration elements, facilitating adjustment and display.

Overall, the scenarios where "Graph Orchestration" is applicable are: business-customized AI integration components. That is, to flexibly orchestrate AI-related atomic capabilities and provide simple and easy-to-use scenario-based AI components. Moreover, in this AI component, there is a unified and complete horizontal governance capability.

- When generating the corresponding AI component, the core is to provide common cross-cutting capabilities.
- The logic inside a service interface: Generally speaking, the responsibilities are relatively single, the distribution is relatively concentrated, and the context is relatively cohesive. It does not match the applicable scenarios of "orchestration". The AI integration component can be used as an SDK in the business interface.
- Recommended usage methods

<a href="/img/eino/en_eino_recommend_using_graph.png" target="_blank"><img src="/img/eino/en_eino_recommend_using_graph.png" width="100%" /></a>

- A more challenging approach -- 【Node orchestration of the entire business process】
- Biz Handler typically focuses on business logic and has a relatively light focus on data flow, and is more suitable for development in a function stack call manner.
- If the logical division and combination are carried out in a graph orchestration manner, it will increase the difficulty of business logic development.

<a href="/img/eino/en_eino_not_recommend_of_biz.png" target="_blank"><img src="/img/eino/en_eino_not_recommend_of_biz.png" width="100%" /></a>
19 changes: 8 additions & 11 deletions content/en/docs/eino/core_modules/_index.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,23 @@
---
Description: ""
date: "2025-02-11"
date: "2025-07-21"
lastmod: ""
tags: []
title: 'Eino: Core Modules'
weight: 3
---

The core modules in Eino include the following parts:
Eino’s core modules include the following parts:

- **Components**: [Eino: Components](/docs/eino/core_modules/components)
- **Components**: [Eino: Components](/en/docs/eino/core_modules/components)

Eino abstracts commonly used components in LLM applications, such as `ChatModel`, `Embedding`, `Retriever`, etc. These are the building blocks for constructing an LLM application, forming the foundation of application capabilities and serving as atomic objects for complex logic orchestration.
Eino abstracts commonly used building blocks in LLM applicationssuch as `ChatModel`, `Embedding`, `Retriever` — which serve as the foundation for application capabilities and the atomic units for complex orchestration.

- **Chain/Graph Orchestration**: [Eino: Chain & Graph Orchestration](/docs/eino/core_modules/chain_and_graph_orchestration)
- **Chain/Graph Orchestration**: [Eino: Chain/Graph Orchestration](/en/docs/eino/core_modules/chain_and_graph_orchestration/chain_graph_introduction)

Using multiple components in combination to implement the chain of business logic, Eino provides orchestration methods through Chains/Graphs, encapsulating the complexity of linking business logic within Eino itself. It offers easy-to-understand business logic orchestration interfaces and provides a unified cross-sectional governance capability.
Real applications combine and connect multiple components to implement business logic. Eino provides Chain/Graph orchestration to encapsulate complexity, exposing intuitive APIs for logic composition with unified cross-cutting governance.

- **Flow Integration Tools (Agents)**: [Eino: Flow integration components](/docs/eino/core_modules/flow_integration_components)
- **Flow Integration (agents)**: [Eino: Flow Integration Components](/en/docs/eino/core_modules/flow_integration_components)

Eino packages the most commonly used LLM application modes into simple and easy-to-use tools, ultra-simplifying the development of LLM applications for generic scenarios. Currently, it provides `ReAct Agent` and `Host Multi Agent`.
Eino wraps common LLM application patterns into simple, ready-to-use tools, dramatically simplifying development for standard scenarios. Currently available: `ReAct Agent` and `Host Multi-Agent`.

- **EinoDev Development Assistant Tool**: [EinoDev: Devops tools](/docs/eino/core_modules/devops)

Eino is dedicated to making the development of large-scale model applications with full-code very simple, and EinoDev provides a `visual` and `interactive` development and debugging solution for Eino orchestration, which allows developers to see the results immediately, releasing their energy from the `debugging hell` and focusing on the scene logic.
Loading