Secondary Storage Plugin

Use various key-value store options for rate limiting counters and other high-frequency records.

Overview

The Secondary Storage plugin allows you to use various key-value store options such as in-memory, database, and custom implementations (e.g. Redis) for rate limiting counters and other high-frequency records. This is useful for offloading intensive operations from your main database to a fast storage layer.

Features

Multiple Storage Providers — In-memory, database, or Redis backends
Scalability — Offload high-churn data from your main database
Flexible Adapters — Implement custom storage adapters for Redis, Valkey, or any key-value store
Automatic Cleanup — Configurable cleanup intervals for expired entries
Plugin Dependency — Optionally used by various plugins for efficient storage and other transient/ephemeral data

Configuration

Standalone Mode:

[plugins.secondary_storage]
enabled = true
provider = "memory" # Options: "memory", "database", "redis"

[plugins.secondary_storage.memory]
cleanup_interval = "1m"

Library Mode:

import (
  "os"

  secondarystorageplugin "github.com/Authula/authula/plugins/secondary-storage"
)

secondarystorageplugin.New(secondarystorageplugin.SecondaryStoragePluginConfig{
  Enabled:  true,
  Provider: secondarystorageplugin.SecondaryStorageProviderRedis,
  Redis: &secondarystorageplugin.RedisStorageConfig{
    URL: os.Getenv("REDIS_URL"),
  },
})

NOTE: The secondary storage plugin MUST be registered before the rate-limit plugin if you plan to use the Redis provider. This is due to the Rate Limit plugin needing the secondary storage service as a dependency.

API Reference

This plugin doesn't expose any HTTP endpoints.

Database Schema

Table: `key_value_store`

Field	Type	Key	Description
`key`	`string`	`PK`	Storage key
`value`	`string`	-	Serialized value
`expires_at`	`timestamp?`	-	When the entry expires
`created_at`	`timestamp`	-	Record creation time
`updated_at`	`timestamp`	-	Record last update time

Migrations are automatically handled when the plugin is initialized.

Plugin Capabilities

This plugin doesn't provide any hooks and capabilities.

Storage Providers

The Secondary Storage plugin provides a storage abstraction used by other plugins for high-frequency data.

Provider	Performance	Persistence	Best For
Memory	Fastest	No	Single-instance, development
Database	Moderate	Yes	Persistent storage, audit trails
Redis	Fast	Yes	Multi-instance, distributed cache

Usage Example

The secondary storage plugin must be registered before plugins that depend on it (e.g. rate-limit plugin).

Database Provider:

Standalone Mode:

[plugins.secondary_storage]
enabled = true
provider = "database"

[plugins.secondary_storage.database]
cleanup_interval = "1m"

Library Mode:

import (
  "os"

  authula "github.com/Authula/authula"
  authulamodels "github.com/Authula/authula/models"
  authulaconfig "github.com/Authula/authula/config"
  secondarystorageplugin "github.com/Authula/authula/plugins/secondary-storage"
)

config := authulaconfig.NewConfig(/*...*/)
auth := authula.New(&authula.AuthConfig{
  Config: config,
  Plugins: []authulamodels.Plugin{
    secondarystorageplugin.New(secondarystorageplugin.SecondaryStoragePluginConfig{
      Enabled:  true,
      Provider: secondarystorageplugin.SecondaryStorageProviderDatabase,
      Database: &secondarystorageplugin.DatabaseStorageConfig{
        CleanupInterval: time.Minute,
      },
    }),
  },
})

Redis Provider:

Standalone Mode:

[plugins.secondary_storage]
enabled = true
provider = "redis"

[plugins.secondary_storage.redis]
# NOTE: It is highly recommended to set the Redis URL via the `REDIS_URL` environment variable rather than hardcoding it in the configuration file.
# url = "redis://localhost:6379"
# Optional: Specifies the maximum number of retries for failed Redis operations. Default is 3.
max_retries = 3
# Optional: Specifies the size of the Redis connection pool. Default is 10.
pool_size = 10
# Optional: Specifies the timeout for Redis connections. Default is 30s.
pool_timeout = "30s"

Library Mode:

import (
  "os"

  authula "github.com/Authula/authula"
  authulaconfig "github.com/Authula/authula/config"
  authulaenv "github.com/Authula/authula/env"
  secondarystorageplugin "github.com/Authula/authula/plugins/secondary-storage"
)

config := authulaconfig.NewConfig(/*...*/)
auth := authula.New(&authula.AuthConfig{
  Config: config,
  Plugins: []authulamodels.Plugin{
    secondarystorageplugin.New(secondarystorageplugin.SecondaryStoragePluginConfig{
      Enabled:  true,
      Provider: secondarystorageplugin.SecondaryStorageProviderRedis,
      Redis: &secondarystorageplugin.RedisStorageConfig{
        URL:  os.Getenv(authulaenv.EnvRedisURL),
        // Optional Redis configuration parameters...
      },
    }),
  },
})

Security Recommendations

Use Environment Variables for Secrets — Set Redis URLs and credentials via environment variables (e.g., REDIS_URL) rather than hardcoding them in configuration files.
Connection Pool Limits — Configure appropriate pool_size and pool_timeout values to prevent connection exhaustion.
Retry Configuration — Set max_retries to handle transient network failures gracefully.

Secondary Storage Plugin

On this page