Provider media and search

Embedding, generation, and web capabilities a provider plugin can register alongside text inference. Register each one inside register(api) next to your existing api.registerProvider(...) call. Part of the Building provider plugins guide.

Bundled runtime adapters can create deferred promises with createDeferred from the private apus/plugin-sdk/concurrency-runtime subpath. It returns promise, resolve, and reject without loading logging or provider auth; the adapter retains responsibility for cancellation and terminal settlement.

Media and search capabilities

Embeddings

```typescript
// fetchAcmeEmbedding is your plugin's own vendor API call, not an SDK export.
api.registerEmbeddingProvider({
  id: "acme-ai",
  defaultModel: "acme-embed",
  transport: "remote",
  authProviderId: "acme-ai",
  create: async ({ model }) => ({
    provider: {
      id: "acme-ai",
      model,
      dimensions: 1536,
      embed: async (input) => {
        const text = typeof input === "string" ? input : input.text;
        return fetchAcmeEmbedding(text);
      },
      embedBatch: async (inputs) =>
        Promise.all(
          inputs.map((input) =>
            fetchAcmeEmbedding(typeof input === "string" ? input : input.text),
          ),
        ),
    },
  }),
});
```

Declare the same id in `contracts.embeddingProviders`. This is the
general embedding contract for reusable vector generation, including
memory search. The retired memory-specific registrar and manifest
contract are no longer accepted.

OpenAI-compatible endpoints can use `createRemoteEmbeddingProvider`
from `apus/plugin-sdk/memory-core-host-engine-embeddings`. Its optional
`buildRequestFields(kind)` callback returns extra JSON fields for
`"query"` or `"document"` requests, such as `dimensions` or `input_type`.
The shared factory always supplies the client's `model` and the original
`input` array after those fields, preserving response-count validation.

Providers that accept model aliases can expose
`normalizeModel(options): string`. Memory uses this synchronous hook for
both creation options and cold index identity checks. Keep it configuration-only:
do not authenticate or access the network. Make normalization idempotent and
reuse it in `create`, which may receive an already-normalized model or be
called outside memory. Return an empty string only when the
model remains unknown until discovery; do not turn an invalid explicit
model into an omitted selection. For an exact pre-initialization identity,
`resolveIndexIdentity(options)` additionally supplies the required
`cacheKeyData` and any equivalent persisted aliases.

Image and video generation

Image and video capabilities use a **mode-aware** shape. Image
providers declare required `generate` and `edit` capability blocks;
video providers declare `generate`, `imageToVideo`, and
`videoToVideo`. Flat aggregate fields like `maxInputImages` /
`maxInputVideos` / `maxDurationSeconds` are not enough to advertise
transform-mode support or disabled modes cleanly. Music generation
follows the same `generate` / `edit` pattern.

Bundled providers can use `selectSupportedVideoDuration` from the private
`apus/plugin-sdk/video-generation` subpath to select the nearest value
from a nonempty list, preferring the longer duration on ties. Keep input
validation, rounding, bounds, and default durations in the provider.

```typescript
api.registerImageGenerationProvider({
  id: "acme-ai",
  label: "Acme Images",
  capabilities: {
    generate: { maxCount: 4, supportsSize: true },
    edit: { enabled: false },
  },
  generateImage: async (req) => ({
    images: [
      {
        buffer: await generateAcmeImageBytes(req),
        mimeType: "image/png",
        fileName: "acme-image.png",
      },
    ],
  }),
});

api.registerVideoGenerationProvider({
  id: "acme-ai",
  label: "Acme Video",
  defaultTimeoutMs: 600_000,
  models: ["acme-video", "acme-image-video"],
  capabilities: {
    generate: { maxVideos: 1, maxDurationSeconds: 10, supportsResolution: true },
    imageToVideo: {
      enabled: true,
      maxVideos: 1,
      maxInputImages: 1,
      maxInputImagesByModel: { "acme/reference-to-video": 9 },
      maxDurationSeconds: 5,
    },
    videoToVideo: { enabled: false },
  },
  catalogByModel: {
    "acme-image-video": {
      modes: ["imageToVideo"],
      capabilities: {
        imageToVideo: {
          enabled: true,
          maxVideos: 1,
          maxInputImages: 1,
          resolutions: ["480P", "720P", "1080P"],
          supportsResolution: true,
        },
        videoToVideo: { enabled: false },
      },
    },
  },
  generateVideo: async (req) => ({
    videos: [
      {
        url: await generateAcmeVideoUrl(req),
        mimeType: "video/mp4",
      },
    ],
  }),
});
```

The illustrative helpers stand in for provider calls: the image helper
returns non-empty encoded bytes, while the video helper returns a hosted
media URL. Video providers may return non-empty encoded bytes instead,
or both when the URL is a delivery fallback. Empty result arrays and
empty buffers are candidate failures, except that a video asset with a
usable URL ignores an empty buffer and continues with the URL.

`capabilities` is required on both provider types; `edit` and the
video transform blocks (`imageToVideo`, `videoToVideo`) always need an
explicit `enabled` flag.

Use `catalogByModel` when a listed model's static modes or capabilities
differ from the provider defaults. This metadata keeps
`video_generate action=list` and model catalogs accurate without
invoking provider code. Request-time capability lookup and enforcement
still belong in `resolveModelCapabilities` and `generateVideo`; reuse
the same capability constant for both paths when possible.

The following [private-local helpers](/plugins/sdk-subpaths#compatibility-and-private-local-helpers)
are supported only for bundled and separately published official plugins.

For asynchronous provider jobs, `pollProviderOperation` from
`apus/plugin-sdk/provider-http` shares the bounded polling loop while
the plugin supplies its request, completion/failure checks, and wait function.
`pollProviderOperationJson` adds the standard HTTP JSON transport.
Keep vendor authentication and deadline scope in the provider adapter.

Reuse `createProviderOperationTimeoutError(deadline)` when a custom body
reader exhausts that same deadline. It preserves the operation label and
optional timeout in the shared error format.

`readGeneratedVideoAsset` from `apus/plugin-sdk/media-generation-runtime`
reads a response under a byte cap and derives the asset's MIME type and filename.
Set `validateBinaryResponse` to reject non-video responses. An optional
`overflowUrl` provides delivery only when the body exceeds that cap; malformed
media and transport errors still fail. The caller owns response cleanup.
`downloadGeneratedVideoAsset` also owns fetching, deadlines, and cleanup.

Web fetch and search

```typescript
api.registerWebFetchProvider({
  id: "acme-ai-fetch",
  label: "Acme Fetch",
  hint: "Fetch pages through Acme's rendering backend.",
  envVars: ["ACME_FETCH_API_KEY"],
  placeholder: "acme-...",
  signupUrl: "https://acme.example.com/fetch",
  credentialPath: "plugins.entries.acme.config.webFetch.apiKey",
  getCredentialValue: (fetchConfig) => fetchConfig?.acme?.apiKey,
  setCredentialValue: (fetchConfigTarget, value) => {
    const acme = (fetchConfigTarget.acme ??= {});
    acme.apiKey = value;
  },
  createTool: () => ({
    description: "Fetch a page through Acme Fetch.",
    parameters: {},
    execute: async (args) => ({ content: [] }),
  }),
});

api.registerWebSearchProvider({
  id: "acme-ai-search",
  label: "Acme Search",
  hint: "Search the web through Acme's search backend.",
  envVars: ["ACME_SEARCH_API_KEY"],
  placeholder: "acme-...",
  signupUrl: "https://acme.example.com/search",
  credentialPath: "plugins.entries.acme.config.webSearch.apiKey",
  getCredentialValue: (searchConfig) => searchConfig?.acme?.apiKey,
  setCredentialValue: (searchConfigTarget, value) => {
    const acme = (searchConfigTarget.acme ??= {});
    acme.apiKey = value;
  },
  createTool: () => ({
    description: "Search the web through Acme Search.",
    parameters: {},
    execute: async (args) => ({ content: [] }),
  }),
});
```

Both provider types share the same credential-wiring shape:
`hint`, `envVars`, `placeholder`, `signupUrl`, `credentialPath`,
`getCredentialValue`, `setCredentialValue`, and `createTool` are all
required.

Search providers can declare `configPath` as a path relative to their own
plugin configuration for the Search settings page. It defaults to
`["webSearch"]`; use `null` when the provider has no inline settings.
Providers sharing a plugin can expose different settings without showing
fields that only apply to a sibling provider. Credentials remain described
by `credentialPath` and use the existing masked credential editor.

Search providers using `apus/plugin-sdk/provider-web-search` should
resolve `resolveSearchCacheTtlMs(searchConfig)` once per execution and
pass that value to both `readCachedSearchPayload(cacheKey, ttlMs)` and
`writeCachedSearchPayload(cacheKey, payload, ttlMs)`. A zero TTL bypasses
reads and writes; a positive TTL bounds entry age without extending its
original expiry. Reads return a payload marked `cached: true`, or
`undefined` on a miss. The reader's `ttlMs` argument is optional:
existing one-argument calls continue to use the stored expiry alone.

Both tool definitions accept `execute(args, context?)`, where the optional
context carries `signal?: AbortSignal`. Forward that signal to network
requests and check cancellation after asynchronous work. Existing
one-argument implementations remain valid; Apus rejects late fetch
results after cancellation before publishing them to its fetch cache.