Message Queue Trigger: KEDA

A message queue trigger invokes a function for each message published to a queue or stream. The KEDA-based message queue trigger is the recommended way to consume from message queues in Fission — it autoscales the connector that reads from your event source, scaling all the way down to zero when there are no messages and back up as load grows.

KEDA is the default and recommended kind for message queue triggers. --mqtkind defaults to keda, so you only need to set it explicitly when you want the legacy Fission-managed connector (--mqtkind=fission). The Fission kind is being phased out — prefer KEDA for all new triggers.

Supported event sources

Each connector is documented on its own page:

• Apache Kafka
• AWS SQS
• AWS Kinesis
• GCP Pub/Sub
• NATS JetStream
• NATS Streaming
• RabbitMQ
• Redis Lists

Architecture

flowchart TB
  source["Event Source"]:::user

  subgraph k8s["Kubernetes Cluster"]
    keda["KEDA"]:::user
    scaledobject["ScaledObject"]:::store
    connector["Connector Deployment"]:::fission
    router["Router"]:::fission
    fnPod["Function Pod"]:::pod
  end

  source -->|"<b>1.</b> messages"| keda
  keda -->|"<b>2.</b> scales 0..N"| connector
  scaledobject -.->|"defines scaling"| connector
  source -->|"<b>3.</b> consume"| connector
  connector -->|"<b>4.</b> POST message"| router
  router -->|"<b>5.</b> forwards request"| fnPod
  fnPod -->|"<b>6.</b> response"| connector
  classDef user fill:#ffffff,stroke:#94a3b8,color:#1f2a43
  classDef fission fill:#e8f0fe,stroke:#2d70de,color:#1f2a43
  classDef pod fill:#e6f7f1,stroke:#11a37f,color:#1f2a43,stroke-dasharray:5 3
  classDef store fill:#fff7e0,stroke:#dba514,color:#1f2a43,stroke-dasharray:5 3

1. When you create a KEDA message queue trigger, Fission creates a ScaledObject and a connector deployment that the ScaledObject references. The ScaledObject is KEDA’s way of encapsulating the connector deployment and the information needed to connect to an event source. If authentication is required, the trigger also creates a TriggerAuthentication referencing your secret.
2. KEDA creates an HPA for the deployment and scales it down to zero while there are no messages.
3. As messages arrive in the event source, KEDA scales the connector deployment from 0 to 1, and beyond as the backlog grows.
4. The connector consumes each message and POSTs it to the function through the router.
5. The function returns a response; the connector writes successful responses to the response topic and failures to the error topic, when those are configured.

Prerequisites

• KEDA must be installed on your cluster. Fission v1.25.0 is built against KEDA v2.20.
• The KEDA integration must be enabled in Fission. Set mqt_keda.enabled to true when installing or upgrading the Fission Helm chart.

Create a KEDA trigger

Create a message queue trigger with fission mqtrigger create (alias fission mqt create). The connection details for each event source go in repeatable --metadata key=value flags, and credentials are referenced by --secret.

fission mqt create --name <name> --function <function> \
    --mqtype <source-type> --mqtkind keda \
    --topic <input-topic> --resptopic <response-topic> --errortopic <error-topic> \
    --maxretries 3 \
    --metadata key1=value1 --metadata key2=value2 \
    --secret <secret-name>

Key flags:

The exact --metadata keys depend on the event source — see the connector page for each source.

Related

• Triggers overview
• KEDA scalers