For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Goose

Route Goose LLM traffic through agentgateway running in Kubernetes.

Route Goose LLM traffic through agentgateway running in Kubernetes to centralize credentials and capture audit logs for every agent call.

Before you begin

1. Set up an agentgateway proxy.
2. Set up access to the OpenAI LLM provider.

Install Goose

Install Goose by following the Goose installation guide.

Get the gateway URL

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80

Set up the OpenAI backend

1. Export your OpenAI API key.

   export OPENAI_API_KEY="sk-your-key-here"

2. Create a Kubernetes Secret for your API key.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: openai-secret
     namespace: agentgateway-system
   type: Opaque
   stringData:
     Authorization: $OPENAI_API_KEY
   EOF

3. Create an AgentgatewayBackend for OpenAI.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     ai:
       provider:
         openai: {}
     policies:
       auth:
         secretRef:
           name: openai-secret
   EOF

4. Create an HTTPRoute to forward traffic to the backend.

   kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     parentRefs:
       - name: agentgateway-proxy
         namespace: agentgateway-system
     rules:
       - matches:
         - path:
             type: PathPrefix
             value: /
         backendRefs:
         - name: openai
           namespace: agentgateway-system
           group: agentgateway.dev
           kind: AgentgatewayBackend
   EOF

Configure Goose

Point Goose at the agentgateway ingress address using environment variables.

export GOOSE_PROVIDER=openai
export GOOSE_MODEL=gpt-4o
export OPENAI_HOST=http://$INGRESS_GW_ADDRESS
export OPENAI_API_KEY=placeholder

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80 &

export GOOSE_PROVIDER=openai
export GOOSE_MODEL=gpt-4o
export OPENAI_HOST=http://localhost:8080
export OPENAI_API_KEY=placeholder

The following table describes each environment variable:

Verify the connection

1. Send a one-shot prompt to confirm requests flow through agentgateway.

   goose run --text "say hello"

2. Check the agentgateway proxy logs to confirm the request was routed through the gateway.

   kubectl logs deployment/agentgateway-proxy -n agentgateway-system --tail=5

   You should see a log entry showing the request was forwarded to the OpenAI endpoint with the configured model:

   info  request gateway=agentgateway-system/agentgateway-proxy listener=http route=agentgateway-system/openai endpoint=api.openai.com:443 http.method=POST http.path=/v1/chat/completions http.status=200 protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=openai gen_ai.request.model=gpt-4o gen_ai.usage.input_tokens=4569 gen_ai.usage.output_tokens=10 duration=2242ms

Next steps