StreamGraph

StreamGraph in Nstream AI is a dynamic, graph-based execution framework designed for real-time data processing and AI-driven decision-making in streaming applications. It enables users to define complex streaming workflows using a declarative YAML-based approach, integrating AI models, vector databases, and event-driven data sources to process, analyze, and act on high-throughput data streams. Built on a Kubernetes-native architecture, StreamGraph ensures scalability, fault tolerance, and adaptive data processing, making it ideal for applications such as fraud detection, market sentiment analysis, RAG pipelines, and predictive maintenance. By seamlessly combining streaming analytics with AI inference, StreamGraph transforms raw event streams into actionable intelligence in real-time.

Sample YAML Configuration

apiVersion: streams.nstream.ai/v1
kind: StreamGraph
metadata:
  name: "STREAMGRAPH_NAME"
spec:
  graphName: "GRAPH_NAME"
  graphConfig:
    graphNodes:
      - nodeName: "NODE_NAME"
        nodeInput:
          nodeConnector: "INPUT_CONNECTOR"
        nodeRef: "NODE_REFERENCE"
        nodeContext:
          nodeKnowledgeBase: "KNOWLEDGEBASE_NAME"
          knowledgeSearchLimit: "SEARCH_LIMIT"
        nodeInference:
          modelName: "MODEL_NAME"
          modelFinetuningName: "FINETUNER_NAME"
        nodePreprocessing:
          inputTableName: "INPUT_TABLE_NAME"
          query: "SELECT fields FROM {{INPUT_TABLE_NAME}} WHERE conditions"
        promptTemplate: "SYSTEM_CONTEXT: Contextual background. USER_QUESTION: Specific question or directive."
  outputDataConnector: "OUTPUT_CONNECTOR"

StreamGraph YAML Configuration Parameters

Key	Description	Example
apiVersion	Defines the API version for the configuration	streams.nstream.ai/v1
kind	Identifies the type of resource being configured	StreamGraph
name	Unique name for the StreamGraph instance	stream-graph
graphName	Designation for the StreamGraph instance	stream-graph
nodeName	Name of the node within the graph	analytic_node
nodeConnector	Specifies the connector for node input	event-source-connector
nodeKnowledgeBase	Knowledge base used for node context	demo-node-1-kb
modelName	Model used for inference at this node	llama3-8b-instruct
modelFinetuningName	Finetuner associated with the model	demo-node-1-finetuner
inputTableName	Table name for preprocessing queries	source_table
query	SQL query for data preprocessing	SELECT datetime, event_data FROM {{source_table}}
promptTemplate	Template providing context for user queries	"SYSTEM_CONTEXT: - USER_QUESTION: " -
outputDataConnector	Connector for outputting data from the graph	demo-sink-connector