MaRK: Contextual Modulation of Recurrent Parameters in Pretrained SSMs

14 November 20262 views

An approach is proposed that adapts the recurrent operator of a frozen state-space model to the context through low-rank parameter changes, affecting the input–output kernel at each generation step. A comparison of three adapter geometries shows that Chebyshev polynomials yield lower validation error, while the constrained parameterization allows for analytical stability verification.

MaRK: Contextual Modulation of Recurrent Parameters in Pretrained SSMs

Purpose and essence of MaRK

MaRK (Markov-adapted Recurrent Kernels) is a framework for dynamically conditioning operators. It maps context vectors to bounded modulations of recurrence operators.

  • In the LPV-SSM representation, MaRK defines a context-indexed family of Markov parameter sequences.
  • The approach is designed for pretrained SSMs.
  • Standard conditioning mechanisms leave the underlying temporal dynamics unchanged. These include feeding information into the input and modulating activations.

Practical criterion: MaRK is needed when the temporal dynamics of an SSM must change, rather than only its inputs or activations.

Which SSM parameters are modulated

MaRK modulates five groups of parameters in a frozen SSM:

  • A — recurrence;
  • B — input read-in;
  • C — output readout;
  • D — skip;
  • Δ — discretization.

Practical criterion: MaRK changes A, B, C, D, and Δ. If a task requires changing these parameters, the approach covers them.

Adapters and frozen-base training

MaRK uses a frozen Hydra SSM base model with 111 million parameters. The study covers three adapter geometries:

  • Hypernet;
  • Chebyshev polynomials;
  • Discrete Cosine Transform (DCT) kernels.

The adapters change the Markov parameter sequence through auxiliary low-rank mappings. The base model remains frozen.

Switching from a bidirectional objective to an iterative diffusion setting requires 6.3–11 million trainable auxiliary parameters. Practical criterion: only the auxiliary mappings need to be trained, while the base SSM remains frozen.

Experimental results and choice of geometry

In synthetic LPV recovery experiments and Markov operator diagnostics, MaRK recovers coordinate-invariant temporal operators under matched assumptions. The method produces distinct, time-step-conditioned stable memory profiles.

Adapter geometryValidation loss
Chebyshev2.55
DCT2.59
Hypernet3.77

Chebyshev achieved the best validation loss among the three options, at 2.55. DCT scored 2.59, and Hypernet 3.77. Practical criterion: based on validation loss in the described experiments, Chebyshev scored 2.55, DCT 2.59, and Hypernet 3.77. The choice of adapter geometry affects the result.

Stability and limitations

A bounded parameterization of recurrence provides an analytical Affine Quadratic Stability certificate for the modulated recurrence. MaRK is intended for iterative generation with pretrained SSMs. Each diffusion step can change the model’s input–output memory kernel.

Practical criterion: MaRK is appropriate when an analytical stability certificate is needed, along with a change to the memory kernel at each diffusion step.

Publication status and version

  • Paper authors: Syed Ibrahim Omer, Ginny Y. Wong, Xiangyu Zhao.
  • The arXiv paper 2610.09092 contains version v1 dated 6 Oct 2026.
  • The latest revision is dated 18 Oct 2026 (version v3).
  • The journal reference points to NeurIPS 2026 Submission.

Practical criterion: an accurate reference to the results includes version v3 dated 18 Oct 2026 and the status NeurIPS 2026 Submission.

Frequently asked questions

MaRK: Contextual Modulation of Recurrent Parameters in Pretrained SSMs