Metered providers

Purpose

A metered provider is an outside service that the companion uses through an MCP server, and that costs money for each call. Joshu has two metered providers: fal.ai (fal), which makes images, video, audio, and 3D, and treg (treg), which sells calls to many data APIs, for example ScrapeCreators for the caption, the transcript, and the comments of one social media post. On a fleet box, the calls go through the control plane, which holds the provider keys and takes the cost of each call from the usage wallet of the owner. On a self-hosted box, the owner saves their own key in Connectors, and Hermes calls the provider directly.

This document covers the box side: the provider definitions, the local metered MCP relay, the Hermes configuration, the Connectors card, the skill for social posts, and what the social sources of Research reports do when the usage wallet is empty. The billing on the control plane is in the private documents of the control plane.

Motivation

These constraints shaped the design:

• A fleet box must never hold a provider key. A key on a box could leak to the companion, to a skill, or to a log, and it would let anyone with the box spend the money of Joshu.
• The owner pays for what the companion uses. The cost of a call is only known after the call, so the control plane checks the balance of the usage wallet before each billable call and takes the real cost after it.
• Hermes registers the tools of an MCP server once, when the gateway starts. A toolset that is on while the usage wallet is empty gives the companion tools that fail on each call, so the box turns the toolset off when the balance is zero.
• A self-hosted box has no control plane. The owner of a self-hosted box needs the same tools with their own key and no usage wallet.
• A second and a third provider should need a new definition, not new code. The first version of this code had fal in the name of each function.
• A login wall hides most of an Instagram or Facebook post from the shared browser. Treg returns the full post from its link, so the companion needs a rule for when to use treg and when to use the shared browser.

Architecture

Source of the diagram: diagrams/metered-providers.json.

The parts:

• Provider definitions (METERED_PROVIDER_DEFINITIONS in src/meteredProviders/config.ts). Each definition has the id, the names of its settings (the mode, the relay URL, the usage summary URL, and the owner switch), the key name and the MCP URL for a self-hosted box, the local port, the Hermes MCP server name, and the Hermes toolset. The functions of config.ts take a definition: the mode, the owner switch, the relay check, the key of a self-hosted box, the local relay URL, and whether the toolset is on.
• Usage summary. fetchUsageSummaryFromCp reads the balance of the usage wallet from the control plane with the agent token of the box, and keeps it in a cache for 15 seconds. One usage summary serves each provider.
• Local metered MCP relay (scripts/lib/metered-mcp-relay.mjs). One process for each provider in relay mode, on 127.0.0.1 and the port of the provider. The relay is an MCP server for Hermes. It forwards tools/list and tools/call as JSON-RPC to the metered MCP relay route of the control plane, with the bearer <instance id>.<agent token>. scripts/start-metered-mcp-relay.sh <provider id> <port> starts it, and scripts/start-fal-mcp-relay.sh is a wrapper for fal that the release image still names.
• MCP supervisor and health (src/mcpSupervisor.ts, src/mcpDependencyHealth.ts). For each provider in relay mode, the health check probes /health of the local relay, and the supervisor restarts the relay when the probe fails.
• Hermes configuration (src/meteredProviders/hermesConfig.ts, called by the gateway sync in src/hermesApi.ts). For each provider, the sync writes one MCP server into ~/.hermes/config.yaml (fal_ai, treg) and adds or removes its toolset (mcp-fal, mcp-treg). src/hermesMcpAllowlist.ts lists both server names as servers that Joshu owns.
• Connectors (src/connectors/meteredProviderRoutes.ts, apps/connectors/src/main.tsx). One card for each provider in section "AI providers": the on or off switch, and either the balance with a link to the usage billing page (fleet box) or the key field (self-hosted box).
• Skill joshu-social-posts (integrations/hermes/skills/research/joshu-social-posts/SKILL.md). The rule for a link to one post on Instagram, Facebook, TikTok, YouTube, Threads, or LinkedIn: which treg catalog entry to call for the post details, the transcript, and the comments, the limit of five calls for each link, and the fallback to the shared browser.
• Social sources (src/research/social/). The searches of Reddit, YouTube, TikTok, and Instagram for Research reports call the ScrapeCreators relay of the control plane. That relay is not an MCP server, but the control plane bills it to the usage wallet in the same way, so it can answer HTTP 402.

Theory of operation

Modes

Each provider has a mode in its own setting (JOSHU_FAL_MODE, JOSHU_TREG_MODE). The box reads the provision file first and the process env second.

When the toolset is on

At each gateway sync, the box first refreshes the usage summary when at least one provider is in relay mode. Then, for each provider, the toolset is on only when all of these are true:

1. The mode is not off.
2. The owner has not turned the provider off in Connectors. The owner switch is FAL_MCP_USER_ENABLED or TREG_MCP_USER_ENABLED in connectors-providers.env of the Joshu state folder. In relay mode, the default is on. In direct mode, the default follows whether a key is saved.
3. In relay mode: the relay URL is set, and the last usage summary shows a balance above zero. Before the first usage summary, the toolset is off.
4. In direct mode: a key is saved.

When a toolset is off, the sync sets enabled: false on its MCP server and removes the toolset from the toolset list. The toolset of one provider does not change the other provider.

One treg call on a fleet box

1. The owner sends a link to an Instagram reel and asks what the video says. Skill joshu-social-posts tells the companion to call mcp_treg_catalog_call_read with endpoint_id scrapecreators.x.v2-instagram-media-transcript and params {"url": "<link>"}.
2. Hermes sends the MCP call to the local metered MCP relay of treg on port 8798.
3. The relay sends tools/call as JSON-RPC to JOSHU_TREG_RELAY_URL, the treg path of the metered MCP relay route of the control plane, with the agent token of the box.
4. The control plane lets a box use only three treg tools: catalog_search, catalog_get, and catalog_call_read. It also refuses a treg catalog entry that uses an account of the Joshu treg team. It checks the balance of the usage wallet against the price of the entry, calls treg, and takes the cost that treg reports from the usage wallet, with no markup.
5. The relay gives the reply of treg to Hermes. When the control plane refuses the call, the relay gives Hermes an error result with the reason, for example tool_not_allowed or insufficient_balance.
6. The companion answers from the transcript. When treg is off, has no entry for the site, or returns no data, the companion opens the link in the shared browser instead.

The social sources when the usage wallet is empty

The control plane bills each call of the ScrapeCreators relay to the usage wallet. When the balance is lower than the price of a call, the relay answers HTTP 402. relayGet in src/research/social/relay.ts turns HTTP 402 into a RelayError of kind usage wallet empty, and makes no retry. The source gets the state "could not be reached" with engine status usage wallet empty, and the other sources of the run continue. sourceListHtml in src/research/template.ts writes the list "Sources searched" of a Research report and of a report update. For such a source, the list says that the usage wallet is empty and links to the usage billing page of the control plane (controlPlaneUsageDashboardUrl()).

Design decisions

2026-10-31: One set of functions for each metered provider

The functions of src/meteredProviders/config.ts take a provider definition, and each definition holds the names of its settings. The definition of treg is the only new code for treg in the config.

Alternatives: copy each fal function for treg. Rejected, because each later provider would add the same copies again, and the copies would drift.

2026-10-31: One start script for each provider

scripts/start-metered-mcp-relay.sh <provider id> <port> starts the local metered MCP relay of any provider, with one PID file and one log file for each provider in ~/.joshu/. The MCP supervisor passes the id and the port of each definition, so the port is written once, in the definition. scripts/start-fal-mcp-relay.sh stays as a one-line wrapper, because older release images and runbooks name it.

Alternatives: one start script for each provider. Rejected for the same reason as the functions.

2026-10-31: One usage summary for all providers

The usage summary of the control plane has one balance for the owner, and the balance pays for each provider. The box keeps one cache, and uses the first usage summary URL that is set. Each provider still has its own usage summary setting, so that each provider has the same three settings.

2026-10-31: A self-hosted box uses treg /mcp/v2/

A self-hosted box connects to https://treg.to/mcp/v2/ with the treg token of the owner. The fleet relay of the control plane uses the same treg surface, so skill joshu-social-posts works in the same way on both kinds of boxes. The /mcp/v2/ surface has the curated catalog only and keeps read calls apart from write calls.

Alternatives: https://treg.to/mcp/. Rejected, because it adds media generation and account tools that a box does not need.

2026-10-31: The rule for social post links is a factory skill

The rule goes into the factory skill joshu-social-posts, which reaches each box through the skills seed. The skill names the treg catalog entry of each site and job, so the companion does not search the catalog for each link, and it uses entries with a fixed price.

Alternatives: add the rule to a skill that a companion wrote on one box. Rejected, because such a skill is not in the repository and does not reach the other boxes.

2026-10-31: The social sources name an empty usage wallet

HTTP 402 from the ScrapeCreators relay becomes the failure kind usage wallet empty, and the source list of the report links to the usage billing page. The source state stays "could not be reached", so the code that ranks source states does not change.

Alternatives: leave HTTP 402 as a plain error. Rejected, because the owner then sees "could not be reached" and cannot know that money fixes it.

Limits and failure modes

• Empty usage wallet. The toolsets of the metered providers are off, and the social sources of Research reports return nothing. The rest of a Research report still works. The toolsets come back at the next gateway sync after the owner adds money.
• Stale balance. The usage summary cache is 15 seconds old at most, and the toolset changes only at a gateway sync. A call can still reach the control plane after the balance went to zero. The control plane then refuses the call with insufficient_balance before it calls the provider.
• Control plane not reachable. The usage summary keeps its last value. Calls through the relay fail, and the companion uses the shared browser for social post links.
• Relay process stops. The MCP supervisor restarts it within a minute. Until Hermes connects again, the tools of the provider fail.
• Large replies. A page of comments can be long. The relay gives it to Hermes unchanged, and the skill asks for comments only when the owner asks about them.
• Bad key on a self-hosted box. When the owner turns a provider on in direct mode, the box probes the MCP URL of the provider with the key and refuses to turn it on after HTTP 401 or 403.
• The relay of the control plane bills one call for each request. The box sends one tool call for each request.

Code map

Settings of a fleet box, which the control plane writes:

Keys of a self-hosted box: FAL_KEY (or FAL_API_KEY) and TREG_TOKEN, in Connectors or in the env.

To run a laptop as a fleet box against the production control plane, the control plane repository has scripts/issue-local-dev-relay-env.ts <box>. The script writes .local/instance.env, which Joshu reads through JOSHU_COMPOSE_ENV_FILE. The script makes a new agent token, so a live box with the same name stops working until it gets the new token.