Rate limiting for the API, and for scheduling and forecasting triggers in particular

[Flix6x]

Closes #306. Supersedes the tech spike in #587.

Scheduling is our most expensive operation, and nothing stopped a client from triggering it in a tight loop. This adds rate limiting via Flask-Limiter.

What it does

• A generous default limit (500 per minute) on the API as a whole, counted per user (per IP address if unauthenticated). The health endpoints are exempt, so that monitoring cannot lock itself out.
• A stricter trigger limit (10 per 5 minutes) on the endpoints which set expensive computation in motion, which share one budget:
  • POST /assets/<id>/schedules/trigger
  • POST /sensors/<id>/schedules/trigger (deprecated, but still callable)
  • POST /sensors/<id>/forecasts/trigger

Hitting a limit returns a 429 with our usual JSON error shape, plus Retry-After and X-RateLimit-* headers (which also gives clients something to poll on, cf. #645).

The two limits count differently. The default limit counts every request, including the ones we refuse — that is what bounds a client who keeps sending us requests we reject, including bad credentials. The trigger limit only counts triggers we accepted: it exists to protect the computation a trigger sets in motion, and a request we rejected (its payload did not validate, or the asset belongs to someone else) cost us no computation. So a client who made a mistake in their flex-model does not pay for it out of their scheduling budget.

Configuration

Setting	Default
RATELIMIT_ENABLED	True
FLEXMEASURES_API_DEFAULT_RATE_LIMIT	"500 per minute"
FLEXMEASURES_API_TRIGGER_RATE_LIMIT	"10 per 5 minutes"
FLEXMEASURES_API_RATE_LIMIT_KEY	"account"

How often it is reasonable to re-compute a schedule is a business decision, so the host decides what shares a budget: account (the default — one budget per account, which is how billing tends to work), account+asset (each asset gets its own budget), or user.

Plans

Per the discussion in #306, limits can be set per account by putting the account on a Plan: a bundle of the rate limits and quotas which apply to the accounts assigned to it, so it can be shared as a tier ("Free", "Pro"). A field left unset falls back to the server-wide config setting, so behaviour is unchanged for anyone without a plan. The special value "unlimited" exempts an account from a limit.

flexmeasures add plan --name Pro --trigger-rate-limit "60 per 5 minutes" --rate-limit-key account

Admins put an account on a plan from the account page, which also shows the plan the account is on:

• plan_id is admin-only on PATCH /api/v3_0/accounts/<id>, like consultancy_account_id.
• A plan usually reflects a contractual agreement, so rather than editing a plan which accounts are on, retire it: flexmeasures edit plan --name Pro --legacy. A legacy plan keeps applying to the accounts already on it, but is no longer offered when assigning a plan.

Plans also carry quotas (max_users, max_assets, max_clients). Those are not enforced yet — a quota is a cap on rows, checked at creation time, which is a separate enforcement path from a rate limit, and is left to a follow-up.

Counts are kept in the Redis we already connect to, so workers share them. If Redis is unreachable, requests are let through rather than taking the API down.

Notes for the reviewer

• The blocker from Rate limiting for scheduling triggers #587 is gone. That spike could not decorate a single flask-classful view method, so it decorated the whole SensorAPI and used a cost function which counted 0 for every method except the trigger. flask-classful 0.16 stacks per-method decorators fine, so @limit_triggers() now sits on each trigger endpoint and the workaround is dropped.
• Flask-Limiter buckets a limit per endpoint unless you tell it not to, which had two consequences worth naming, both fixed here: the trigger limit is now a shared_limit (otherwise a client could double their budget by alternating between the asset and the sensor trigger endpoint), and the default limit is now an application limit, i.e. one budget for the whole API rather than one per endpoint.
• Flask-Limiter[redis] pins redis<8, so this moves us from redis 8.0.0 to 7.4.1. It still satisfies our own redis>4.5 and works with rq and fakeredis. I preferred the supported version over forcing a combination limits explicitly excludes, but shout if you would rather keep redis 8 and drop the extra.
• Rate limiting stays initialized during tests, with limits set absurdly high, because Flask-Limiter returns early from init_app when disabled — meaning a test cannot turn it on afterwards, only off. The rate-limiting tests lower the limits themselves.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C2JTqYVFPmJgX7kN2DSvbE

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	redis@​8.0.0 ⏵ 7.4.1	+1
	flask-limiter@​4.1.1

View full report

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #33597210 | 📁 Comparing c58c3ea against latest (0409655)

  🔍 Preview build  

20 files changed · + 1 added · ± 19 modified

+ Added

• _autosummary/flexmeasures.api.common.rate_limiting.html

± Modified

• changelog.html
• configuration.html
• genindex.html
• py-modindex.html
• _autosummary/flexmeasures.api.common.html
• _autosummary/flexmeasures.api.common.responses.html
• _autosummary/flexmeasures.api.v3_0.accounts.html
• _autosummary/flexmeasures.api.v3_0.jobs.html
• _autosummary/flexmeasures.cli.utils.html
• _autosummary/flexmeasures.data.models.user.html
• and 9 more...

[nhoening]

I vote to make rate_limits its own account attribute.
We could do it later, as well.
But why not now?

It's a crucial attribute, and will be checked often I believe parsing attributes is a bit more expensive.

[Flix6x]

Agreed that it shouldn't stay buried in attributes — but I'd like to go one step further than an account column, because I think we'd regret that shape fairly quickly.

On the performance rationale: I don't think we'd actually gain much there. The cost isn't decoding the JSON — it's loading the Account row at all (current_user.account is a relationship, so it's a query on first touch; the JSONB decode on an already-loaded row is negligible next to that). Every shape pays that cost. So I'd rather justify this on the grounds you gave first in #306: it's a crucial, first-class field, and it deserves to be typed and discoverable rather than free-form. That argument I fully buy.

Why not scalar columns on Account: the moment we want a second budget — and I think we will: max users, max assets, maybe max sensors — each one is a new column plus a new migration. And a per-account column can't express the per-asset differentiation you asked for in #306 ("they might want to rate Assets differently within their account"), which is a natural next ask given that FLEXMEASURES_API_RATE_LIMIT_KEY already defaults to giving each asset its own counter.

So I'd propose the thing you actually suggested in #306: a plan table. Rate limits and quotas are both entitlements; a plan is the right noun for the bundle, and it gives us shared tiers with per-account overrides later without another migration.

I'd also put the key (what a trigger limit is counted against) on the plan, not just the limit value. My first instinct was to keep it as server config, on the grounds that keying is a hosting/protection decision rather than a customer entitlement — but that doesn't hold up: capping how many assets an account may create is just as much a hosting decision, and that's going on the plan too. And putting both on the plan makes it self-contained: "10 per 5 minutes, per asset" is a complete statement of what an account gets, whereas the value alone is only half an entitlement (the effective total is the value times the number of budgets).

Sketch:

class RateLimitKey(enum.Enum):
    ACCOUNT_ASSET = "account+asset"  # each asset gets its own budget
    ACCOUNT = "account"              # the account has one shared budget
    USER = "user"                    # each user gets their own budget


class Plan(db.Model):
    """A bundle of entitlements. Can be shared as a tier (e.g. "Free", "Pro")."""
    id = Column(Integer, primary_key=True)
    name = Column(String(80), unique=True)

    # Rate limits: counted per time window, enforced per request (Redis).
    # NULL means: fall back to the server config.
    default_rate_limit = Column(String(64), nullable=True)   # e.g. "500 per minute"
    trigger_rate_limit = Column(String(64), nullable=True)   # e.g. "10 per 5 minutes"

    # What the trigger limit is counted against. Together with the limit above,
    # this fully specifies the entitlement: "10 per 5 minutes, per asset".
    # Enum rather than free text: this value feeds the rate limiter's key function.
    rate_limit_key = Column(Enum(RateLimitKey), nullable=True)

    # Quotas: static caps on rows, enforced at creation time (SELECT count(*)).
    # NULL means: no cap.
    max_users = Column(Integer, nullable=True)
    max_assets = Column(Integer, nullable=True)


class Account(db.Model, AuthModelMixin):
    ...
    plan_id = Column(Integer, ForeignKey("plan.id"), nullable=True)
    plan = db.relationship("Plan", backref="accounts")

Three things worth flagging explicitly, because they're easy to get wrong:

1. Rate limits and quotas are not the same animal, even though they belong in the same bundle. A rate limit is a count per time window, checked on every request, answered with a 429. A quota is a cap on rows, checked when you create one, answered with a 403/422. Shared home, completely separate enforcement paths — worth keeping that clear in the code so nobody tries to run max_users through Flask-Limiter.

2. The key mode becomes user-supplied data feeding a key function. Today it's a config string the host writes; as a plan field it's a DB value. Hence the Enum rather than free text — and the key function should fall back to the config default if it ever sees something unexpected, rather than raise, so that one bad plan row can't turn into a 500 on every request for that account.

3. FLEXMEASURES_API_RATE_LIMIT_KEY doesn't disappear, it becomes the default for accounts without a plan — and for unauthenticated traffic, which has no account to read a plan from and stays keyed by IP. Same for the limit values: NULL → server config, so behaviour is unchanged for anyone without a plan.

Minor operational note for the docs: changing an account's rate_limit_key mid-window orphans its existing counters in Redis (the old keys simply expire), so a plan change can briefly hand out a fresh budget. Harmless, but surprising if unexplained.

Nothing is released yet, so the migration is trivial and there's no data to move. Happy to do it in this PR, or land the rate limiting as it stands and do the plan table as an immediate follow-up — your call on how much you want to grow this one.

[nhoening]

I agree with this design.

One thing that came to my mind: For a consultancy account, we might think about a max_clients setting. Also, a consultant should be limited in assigning plans to their clients.

This might need follow-up work, but for now I would make sure the host (admin users?) would be able to choose a default plan for clients (as a config setting, for all consultancy clients, or even per account - but probably the former, so we can get started). Just note that the host account (usually the first to be created on a new FM instance) would treat created consultancy accounts as its own clients.

[nhoening]

Good work. I have some smaller requests and questions.

Also, let's bring some of this to the UI. Hosts and users can not see plans, what plan an account is on, nor can they change the plan.
To begin with, I suggest:

• on the account page, show the plan the account is on
• admins see a plan dropdown in the account edit form, so they can change it.

Plan management has more details to it so maybe that is a follow-up. For instance, a plan usually represents a contractual agreement. Changing a plan should not be taken too lightly. Instead, the default choice might be to turn a plan into a legacy plan, which cannot be edited (we might decide to add that field in this PR already). Also, plans might have a time limitation, after which the account switches to a new one - but this needs to be discussed, it could be also done by each hosts in their own way.

[nhoening]

What if someone tries an endpoint for which they would not be authorized - will it still count towards their limit then, that they attempted to use the endpoint?

[Flix6x]

It did, and it no longer does — for the trigger limit.

The limiter runs before authentication, so as it stood, a 401/403 (and a 422) spent the trigger budget. That is now split by which limit we are talking about, because the two exist for different reasons:

• The trigger limit only counts triggers we accepted (deduct_when=lambda response: response.status_code < 400). It protects the expensive computation, and a request we refused cost us no computation. So a client whose flex-model does not validate, or who asked for an asset which is not theirs, keeps their scheduling budget.
• The default limit still counts every request, including the ones we refuse. That is what bounds a client hammering us with bad credentials — and it means we do not need the second counter you were worried about in your other comment.

Both are now spelled out in the docs.

[nhoening]

I believe the default business decision should be "account". That's how billing works.

[Flix6x]

Agreed, and changed: FLEXMEASURES_API_RATE_LIMIT_KEY now defaults to "account", and an unrecognized value falls back to "account" as well (rather than to "account+asset").

I also made the "account+asset" option honest while I was in there. It used to key on request.path, which meant a schedule triggered through the asset endpoint and one triggered through the deprecated sensor endpoint counted against different budgets, even for the same asset. It now resolves the sensor's asset, so the budget really is per asset. The docs note that "account+asset" multiplies the limit by the number of assets an account has — which is the argument for your default.

[nhoening]

We should use the AssetAPI for testing, rather than the deprecated endpoint.

[Flix6x]

Done — the tests now trigger through AssetAPI:trigger_schedule.

Worth flagging what that turned up. I kept one test on the deprecated sensor endpoint, and it failed: Flask-Limiter gives every endpoint its own bucket unless you tell it otherwise, so the three trigger endpoints each had their own budget, and a client could have doubled their allowance by alternating between the asset and the sensor endpoint. The trigger limit is now a shared_limit(scope="triggers"), and that test asserts the two endpoints draw on the same budget.

The same bug bit the default limit, which was really "500 per minute per endpoint" rather than "500 per minute to call the API". It is now an application limit, i.e. one budget for the whole API, which is what its docs already claimed.

[nhoening]

I'm not sure why this is misconfigured (422) - please add a short comment.

Also, as I said above - technically there is a difference between getting to the point of computation (which is what we are rate limiting) or not (due to auth or config failures).
A design choice could be to only count the ones where the user got to the good part.
Is there consensus on which approach is favored?

Pro: users who made some mistake in their auth or config are not punished.
Contra: We might need to have two counters, I think.

[Flix6x]

The 422s are gone, so there is nothing left to explain there: budget is now spent by triggers we accept, so the tests send valid payloads and assert on 200s.

On the design question, I went with your "pro", and I think the "contra" turns out not to bite. There is a difference between getting to the point of computation and not, and the trigger limit is exactly the limit that should care about it — it exists to protect the computation, and a request we refused cost us none. So it only counts triggers that were accepted (deduct_when on the response status).

We do not need two counters for it, though, because we already have a second limit doing that job: the default limit keeps counting every request, refused or not. So a client who fails auth in a loop is bounded (500/minute), while a client who got their flex-model wrong is not punished with a scheduling lockout. In other words: mistakes cost you out of the generous budget, not the scarce one.

There is a new test for it — test_rejected_triggers_do_not_spend_the_trigger_budget.

[nhoening]

a bit inconsistent that we use constants from RateLimitKey in line 217 but not here.

[Flix6x]

Fixed — RateLimitKey constants everywhere now, including in the parametrize list.

[Flix6x]

Thanks — all addressed in 18ac5d5, and your UI ask is in this PR.

Plans in the UI. The account page now shows the plan the account is on, and admins get a plan dropdown in the account edit form. Under it: plan_id is admin-only on PATCH /api/v3_0/accounts/<id>, exactly like consultancy_account_id, and plan changes land in the account audit log.

Legacy plans. I took you up on adding the field now, since nothing is released and the migration is free. Plan.legacy means: the plan keeps applying to the accounts already on it, but it is no longer offered when assigning a plan (the dropdown hides it, and the API refuses it — except that an account already on a legacy plan keeps showing it, marked as such). That gives us the "do not edit a contract, retire it" default you describe, without committing to the rest of plan management yet.

Creating plans. The dropdown needed something to offer, so plans are created and retired from the CLI for now:

flexmeasures add plan --name Pro --trigger-rate-limit "60 per 5 minutes" --rate-limit-key account
flexmeasures edit plan --name Pro --legacy

Limit strings are validated when the plan is created (via the limits parser Flask-Limiter itself uses), so a typo fails at creation rather than at request time.

Left for follow-up, per your read that plan management has more to it: a plans CRUD UI, enforcing the quotas (max_users/max_assets/max_clients — a different enforcement path from rate limits: a cap on rows, checked at creation, answered with a 403/422), a default plan for new client accounts, and time-limited plans. Happy to open an issue capturing those, unless you would rather grow this PR further.

Two bugs also came out of your review, which I have described in the threads: the trigger endpoints each had their own budget instead of sharing one (a client could double their allowance by alternating between the asset and the deprecated sensor endpoint), and the default limit was counted per endpoint rather than across the API. Both fixed.