How to test a BeeAI ConditionalRequirement: onlyBefore is never read

Quick Answer (October 2026). On beeai-framework 0.1.31 the ConditionalRequirement option onlyBefore is accepted by the constructor, validated in five separate places, copied by clone() and cleared by reset(), and then never read by the function that decides whether your tool is offered to the model. The documentation says it will Disable target_tool after any of these tools are called. It disables nothing, in any combination of options. The same one-line omission has a second consequence: pair forceAfter with onlyAfter and the target tool is permanently disabled from the moment the named tool runs, which is the exact opposite of forcing it. Below is a seven-arm harness, three of them controls, that measures the tool set a RequirementAgent hands the model on every turn, with no API key and nothing calling a real model.

What a requirement is supposed to do

A RequirementAgent does not hand the model your whole toolbox every turn. Before each model call it runs every attached Requirement, collects the Rule objects they return, folds them together, and ships only the surviving set as the tools argument on the model request. That folded set is the single observable worth measuring, because it is the only thing the model can act on. Everything a requirement claims to do has to show up there or it did not happen.

ConditionalRequirement is the declarative front end to this. You name a target tool and attach conditions to it. Four of its options take other tools as their argument, and the vendor's parameter reference describes them like this, quoted from the TypeScript block on the Requirement Agent documentation page with inline code markup removed:

onlyBefore: Disable target_tool after any of these tools are called

onlyAfter: Disable target_tool before all these tools are called

forceAfter: Force target_tool execution immediately after any of these tools are called

Three positive claims about behaviour. The measurement below takes them one at a time.

Setup

Three things about this install are worth saying out loud, because none of them is in the quickstart and all three cost me a round trip.

First, a plain npm install beeai-framework fails with ERESOLVE on a brand new empty project. The package declares nineteen peer dependencies and marks seventeen of them optional, but npm still resolves optional peers when building the tree, and @langchain/community drags in a @getzep/zep-cloud that wants a @langchain/core in the 0.3.x range while the framework itself asks for ^1.1.49. Both of those are among the seventeen optional ones, so the conflict npm refuses to resolve is between two peers the package says you do not need. --legacy-peer-deps gets past it.

Second, zod is not declared anywhere in the package: not in dependencies, not in peerDependencies. Forty-three files in each of the two builds import it. If you install the current zod, which is 4.x, the first thing that touches a schema throws TypeError: Right-hand side of 'instanceof' is not an object, because the framework checks zod.ZodEffects and zod 4 removed that export. You need a zod 3. I pinned 3.25.76.

Third, uuid is also undeclared and is required by agents/requirement/runner, so importing RequirementAgent throws MODULE_NOT_FOUND until you add it yourself.

That gives a one-line install:

npm init -y
npm install --save-exact --legacy-peer-deps \
  beeai-framework@0.1.31 zod@3.25.76 uuid@11.1.0

The harness

Nothing here calls a model. DummyChatModel is the framework's own stub, and its _create throws by design, so the harness subclasses it and overrides _create to do two things: record the names in input.tools, which is exactly the folded set the requirement system produced, and then return a canned tool call from a fixed script. If the tool the script wants is not on offer, the stub answers instead and the run ends. That makes every arm deterministic and reproducible with no network, no key and no spend.

Each arm attaches exactly one ConditionalRequirement so there is nothing for rules to fold against, prints the documented expectation next to the measurement, and then prints the offered set and the toolChoice for every turn.

// Measures which tools the BeeAI RequirementAgent offers the model on each turn.
// No network, no API key: the model is a scripted stub that records input.tools.
const { RequirementAgent } = require('beeai-framework/agents/requirement/agent');
const { ConditionalRequirement } = require('beeai-framework/agents/requirement/requirements/conditional');
const { DynamicTool, StringToolOutput, ChatModelOutput } = require('beeai-framework');
const { DummyChatModel } = require('beeai-framework/adapters/dummy/backend/chat');
const { AssistantMessage } = require('beeai-framework/backend/message');
const { z } = require('zod');
const fs = require('fs');
const path = require('path');

const out = [];
function say(line) { out.push(line); }

function pkgVersion(name) {
  let dir = path.dirname(require.resolve(name));
  for (;;) {
    const f = path.join(dir, 'package.json');
    if (fs.existsSync(f)) {
      const j = JSON.parse(fs.readFileSync(f, 'utf8'));
      if (j.name === name) return j.version;
    }
    const up = path.dirname(dir);
    if (up === dir) return 'unknown';
    dir = up;
  }
}

function makeTool(name, failing) {
  return new DynamicTool({
    name,
    description: 'test tool ' + name,
    inputSchema: z.object({}),
    handler: async function () {
      if (failing) throw new Error('tool ' + name + ' always fails');
      return new StringToolOutput('result of ' + name);
    },
  });
}

// The workaround for arm A7: customChecks IS read by _run(), so a check that
// returns false once any recorded step used the named tool reproduces exactly
// what onlyBefore is documented to do.
function noStepUsed(toolName) {
  return function (state) {
    for (const step of state.steps) {
      if (step.tool && step.tool.name === toolName) {
        return false;
      }
    }
    return true;
  };
}

// A stub ChatModel. It records the tool set it was offered, then calls the next
// tool in its script if that tool is still on offer, otherwise it answers.
class ScriptedModel extends DummyChatModel {
  constructor(script) {
    super('scripted');
    this.script = script;
    this.turns = [];
    this.calls = 0;
  }
  async _create(input) {
    const offered = (input.tools || []).map(function (t) { return t.name; });
    const choice =
      typeof input.toolChoice === 'string'
        ? input.toolChoice
        : (input.toolChoice && input.toolChoice.name) || 'none';
    this.turns.push({ offered: offered, toolChoice: choice });
    const want = this.script[this.calls] || 'final_answer';
    this.calls = this.calls + 1;
    const pick = offered.includes(want) ? want : 'final_answer';
    const args = pick === 'final_answer' ? { response: 'done' } : {};
    return new ChatModelOutput([
      new AssistantMessage({ type: 'tool-call', toolCallId: 'call-' + this.calls, toolName: pick, input: args }),
    ]);
  }
  async *_createStream(input) { yield await this._create(input); }
}

async function arm(label, toolNames, requirementFor, options, script, expectation, optionsLabel) {
  const tools = toolNames.map(function (n) {
    return makeTool(n.name || n, Boolean(n.failing));
  });
  const byName = {};
  for (const t of tools) byName[t.name] = t;
  const model = new ScriptedModel(script);
  const agent = new RequirementAgent({
    llm: model,
    tools: tools,
    requirements: [new ConditionalRequirement(byName[requirementFor], options)],
  });
  say('');
  say('--- ' + label);
  const shown = optionsLabel || JSON.stringify(options);
  say('    requirement : ConditionalRequirement(' + requirementFor + ', ' + shown + ')');
  say('    script      : ' + JSON.stringify(script));
  say('    documented  : ' + expectation);
  try {
    await agent.run({ prompt: 'go' });
  } catch (e) {
    say('    THREW       : ' + (e && e.message ? String(e.message).split('\n')[0] : String(e)));
  }
  // A for...of loop, so the source carries no angle bracket anywhere.
  let n = 0;
  for (const t of model.turns) {
    n = n + 1;
    say(
      '    turn ' + n +
      '  offered=[' + t.offered.join(' ') + ']' +
      '  toolChoice=' + t.toolChoice
    );
  }
}

async function main() {
  say('beeai-framework ' + pkgVersion('beeai-framework') + '   zod ' + pkgVersion('zod') + '   node ' + process.version);
  say('Observable: input.tools on every model call, i.e. the tool set the');
  say('requirement system handed the model. final_answer is always present.');

  // A1. The headline. onlyBefore is documented as: disable beta after alpha runs.
  await arm('A1  onlyBefore alone',
    ['alpha', 'beta'], 'beta', { onlyBefore: ['alpha'] },
    ['alpha', 'beta'],
    'beta disabled from turn 2 onward, because alpha ran on turn 1');

  // A2. CONTROL. The mirror option. Proves target matching and state both work.
  await arm('A2  onlyAfter alone  (CONTROL)',
    ['alpha', 'beta'], 'beta', { onlyAfter: ['alpha'] },
    ['alpha', 'beta'],
    'beta disabled on turn 1, enabled from turn 2 once alpha has run');

  // A3. CONTROL. Proves invocation counting reaches the offered tool set.
  await arm('A3  maxInvocations 1  (CONTROL)',
    ['alpha', 'beta'], 'beta', { maxInvocations: 1 },
    ['beta', 'beta'],
    'beta disabled from turn 2 onward, because it ran once');

  // A4. CONTROL. forceAfter on its own.
  await arm('A4  forceAfter alone  (CONTROL)',
    ['alpha', 'beta'], 'beta', { forceAfter: ['alpha'] },
    ['alpha', 'beta'],
    'beta forced on turn 2, immediately after alpha');

  // A5. The same forceAfter, with onlyAfter added.
  await arm('A5  forceAfter plus onlyAfter',
    ['alpha', 'beta'], 'beta', { onlyAfter: ['alpha'], forceAfter: ['alpha'] },
    ['alpha', 'beta'],
    'beta forced on turn 2, immediately after alpha');

  // A6. Does pairing onlyBefore with onlyAfter rescue it?
  await arm('A6  onlyBefore plus onlyAfter',
    ['gamma', 'alpha', 'beta'], 'beta', { onlyBefore: ['alpha'], onlyAfter: ['gamma'] },
    ['gamma', 'alpha', 'beta'],
    'beta enabled on turn 2 once gamma ran, disabled on turn 3 once alpha ran');

  // A7. The workaround, measured rather than asserted.
  await arm('A7  customChecks instead of onlyBefore',
    ['alpha', 'beta'], 'beta', { customChecks: [noStepUsed('alpha')] },
    ['alpha', 'beta'],
    'beta disabled from turn 2 onward, because alpha ran on turn 1',
    '{ customChecks: [noStepUsed("alpha")] }');

  fs.writeSync(1, out.join('\n') + '\n');
}

main().then(
  function () { process.exit(0); },
  function (e) { fs.writeSync(1, out.join('\n') + '\nFATAL ' + (e && e.stack) + '\n'); process.exit(1); }
);

Run it:

node requirement-probe.js

The transcript

beeai-framework 0.1.31   zod 3.25.76   node v24.8.0
Observable: input.tools on every model call, i.e. the tool set the
requirement system handed the model. final_answer is always present.

--- A1  onlyBefore alone
    requirement : ConditionalRequirement(beta, {"onlyBefore":["alpha"]})
    script      : ["alpha","beta"]
    documented  : beta disabled from turn 2 onward, because alpha ran on turn 1
    turn 1  offered=[alpha beta final_answer]  toolChoice=required
    turn 2  offered=[alpha beta final_answer]  toolChoice=required
    turn 3  offered=[alpha beta final_answer]  toolChoice=required

--- A2  onlyAfter alone  (CONTROL)
    requirement : ConditionalRequirement(beta, {"onlyAfter":["alpha"]})
    script      : ["alpha","beta"]
    documented  : beta disabled on turn 1, enabled from turn 2 once alpha has run
    turn 1  offered=[alpha final_answer]  toolChoice=required
    turn 2  offered=[alpha beta final_answer]  toolChoice=required
    turn 3  offered=[alpha beta final_answer]  toolChoice=required

--- A3  maxInvocations 1  (CONTROL)
    requirement : ConditionalRequirement(beta, {"maxInvocations":1})
    script      : ["beta","beta"]
    documented  : beta disabled from turn 2 onward, because it ran once
    turn 1  offered=[alpha beta final_answer]  toolChoice=required
    turn 2  offered=[alpha final_answer]  toolChoice=required

--- A4  forceAfter alone  (CONTROL)
    requirement : ConditionalRequirement(beta, {"forceAfter":["alpha"]})
    script      : ["alpha","beta"]
    documented  : beta forced on turn 2, immediately after alpha
    turn 1  offered=[alpha beta final_answer]  toolChoice=required
    turn 2  offered=[beta]  toolChoice=beta
    turn 3  offered=[alpha beta final_answer]  toolChoice=required

--- A5  forceAfter plus onlyAfter
    requirement : ConditionalRequirement(beta, {"onlyAfter":["alpha"],"forceAfter":["alpha"]})
    script      : ["alpha","beta"]
    documented  : beta forced on turn 2, immediately after alpha
    turn 1  offered=[alpha final_answer]  toolChoice=required
    turn 2  offered=[alpha final_answer]  toolChoice=required

--- A6  onlyBefore plus onlyAfter
    requirement : ConditionalRequirement(beta, {"onlyBefore":["alpha"],"onlyAfter":["gamma"]})
    script      : ["gamma","alpha","beta"]
    documented  : beta enabled on turn 2 once gamma ran, disabled on turn 3 once alpha ran
    turn 1  offered=[gamma alpha final_answer]  toolChoice=required
    turn 2  offered=[gamma alpha beta final_answer]  toolChoice=required
    turn 3  offered=[gamma alpha beta final_answer]  toolChoice=required
    turn 4  offered=[gamma alpha beta final_answer]  toolChoice=required

--- A7  customChecks instead of onlyBefore
    requirement : ConditionalRequirement(beta, { customChecks: [noStepUsed("alpha")] })
    script      : ["alpha","beta"]
    documented  : beta disabled from turn 2 onward, because alpha ran on turn 1
    turn 1  offered=[alpha beta final_answer]  toolChoice=required
    turn 2  offered=[alpha final_answer]  toolChoice=required

Reading the six arms

A1 is the headline. onlyBefore: ['alpha'] on beta should, per the documentation, disable beta the moment alpha has been called. alpha is called on turn 1. On turns 2 and 3 beta is still in the offered set. The option changed nothing.

A2 is the control that makes A1 mean something. It is the mirror option, onlyAfter: ['alpha'], on the same tool, with the same tools, the same script and the same harness. beta is correctly absent on turn 1 and correctly present from turn 2 once alpha has run. So target matching works, the run state is being read, and the folded set really does reach the model. A1 is not a broken harness.

A3 is a second control on a different mechanism. maxInvocations: 1 removes beta after one successful call. Invocation counting also reaches the offered set. Two independent paths through the same requirement work; onlyBefore is the one that does not.

A4 is the third control and it sets up the second finding. forceAfter: ['alpha'] on its own does force: on turn 2 the offered set collapses to [beta] and toolChoice is beta. final_answer is gone from that turn too, which is forcePreventStop doing its job. On turn 3, with beta no longer the previous step, the force releases and the full set comes back. This is the documented behaviour, working.

A5 adds one option to A4 and inverts it. The forceAfter: ['alpha'] is byte-identical to A4. The only change is onlyAfter: ['alpha'] alongside it. Now beta is absent on turn 1, which is right, and still absent on turn 2, which is not. The combination that should read force beta immediately after alpha instead means beta can never run once alpha has run.

A6 rules out the obvious rescue. You might reasonably guess that onlyBefore is only consulted in combination, so A6 pairs it with onlyAfter: ['gamma'] and gives the agent a gamma to call. beta is correctly withheld until gamma runs, then becomes available on turn 2, and then stays available on turns 3 and 4 after alpha has run. Pairing does not rescue it.

A7 is the workaround, measured rather than recommended on faith. It drops onlyBefore entirely and passes a customChecks callback that returns false once any recorded step used alpha. beta is offered on turn 1 and gone on turn 2. That is the documented onlyBefore behaviour, produced by an option the decision function does read.

Why: one identifier, in one line, in the port

Both consequences come from the same line.

Open dist/agents/requirement/requirements/conditional.js in the installed package. The constructor stores the option on line 32:

this.before = extractTargets(options.onlyBefore);

this.before is then referenced seven more times in that file: three times inside checkInvariant(), which rejects self-references and rejects a tool that appears in both onlyBefore and onlyAfter; once in init(), which asserts the named tool actually exists; once in reset(); and once in clone(). Those references sit on lines 56, 63, 69, 81, 98, 106 and 167.

The decision function, _run(), runs from line 111 to line 164. Not one of those eight references falls inside it. The framework validates onlyBefore thoroughly and then never asks it anything.

The gap is visible if you put the two language bindings side by side. BeeAI ships a Python implementation and a TypeScript one, and the TypeScript RequirementAgent arrived later, in pull request 1357. Inside the block that walks the recorded steps, Python and TypeScript do not read the same field. Here are the two lines, abridged out of their enclosing loops because the surrounding guards are comparisons and this article carries no angle brackets anywhere:

Python   conditional.py line 185   if _target_seen_in(step_tool, self._before):
TypeScript  conditional.js line 146   if (targetSeenIn(stepTool, this.forceAfterSet)) {

Both lines sit at the same position inside the same loop, both are followed by the same early return that marks the tool as not allowed, and both loops are entered only when onlyAfter is non-empty. The TypeScript port reads forceAfterSet where the Python original reads _before. One identifier.

That single substitution produces both measurements. this.before loses its only reader, which is A1 and A6. And forceAfterSet gains a reader that turns it into a disable condition whenever onlyAfter is also set, which is A5. The Python line is in the repository at conditional.py, pinned to the commit I read on 9 October 2026.

Two details are worth not overstating.

The first is that Python is not fully correct either. Its _before check lives inside the same onlyAfter guard, so a Python only_before used on its own is also never consulted. The port did not introduce the narrowness; it widened it from works only when paired with only_after to never works at all, and it added the forceAfter inversion on the way. I read the Python source and did not run it, so that half is a source reading rather than a measurement, and I am not claiming a Python runtime behaviour.

The second is that the framework's own error message already treats before as a first-class role. When a named tool cannot be found, assertAllRulesFound throws with the text is specified as 'source', 'before', 'after' or 'force_after' but not found. The validation layer knows the role exists. The decision layer does not use it.

The second finding on its own terms

A5 deserves stating without the A1 framing, because it is the one that can bite a working agent.

forceAfter is the option you reach for when a tool must run immediately after another one: validate after fetch, log after write, re-rank after search. onlyAfter is the option you reach for when a tool must not run before its prerequisite. Reaching for both on the same tool is a natural thing to do, and on beeai-framework 0.1.31 the two together are not additive. The moment any tool named in forceAfter appears in the recorded steps, the target is marked not allowed and stays that way, because the check runs over the entire step history rather than the previous step.

Note the asymmetry in where each option is read. The forcing behaviour is computed from lastStepTool, the most recent step only. The disabling behaviour in A5 is computed by scanning every step. So the force is a one-turn event and the disable is permanent.

A4 and A5 differ by exactly one key in one object literal, which is why they are in the harness as a pair. If you only ran A5 you would reasonably conclude forceAfter is broken. It is not.

What to do instead

Until this is fixed, treat onlyBefore as decoration and write the constraint from the other side.

Express it as onlyAfter on the other tool. If you wanted beta only before alpha, what you usually mean operationally is alpha only after beta, which A2 shows works. The two are not strictly equivalent, because the first forbids beta and the second forbids alpha, but in most real pipelines the thing you care about is the ordering, and onlyAfter enforces ordering.

Or use customChecks, which is read. customChecks is one of the conditions _run() genuinely consults, every entry has to pass, and the callback is handed the full run state. The noStepUsed helper in the harness above is the whole workaround, and arm A7 is it running: beta offered on turn 1, gone on turn 2 once alpha has been called. That is what onlyBefore was supposed to do, in a ten-line helper, through an option that is actually consulted.

Do not combine forceAfter with onlyAfter on the same requirement. Use two requirements on the same target instead, or drop the onlyAfter and let forceAfter alone do the work, as in A4.

And check the folded set rather than trusting the configuration. This is the general lesson and it is why the harness is shaped the way it is. A declarative rule layer sits between your configuration and the model, and the only evidence that a rule did anything is the tool set that came out of it. Every arm above is one assertion about input.tools. That assertion is cheap, it needs no model, and it would have caught this on the day the option was added. The same shape caught a Mastra branch predicate that silently never matched: in both cases the declaration was accepted, nothing threw, and the behaviour was simply absent.

For contrast, a lower-level SDK makes you compute the restricted set yourself. The Vercel AI SDK's activeTools takes the array of tool names you want this call to see, and because you build that array in your own code there is no layer between your intent and the request. The AI SDK tool-calling documentation covers it. That is more work per call and it has no equivalent of forceAfter, which is the trade a declarative requirement layer is making. The trade is a good one. It just needs the rules to be read.

What I did not measure

Stating these so nobody reads more into the result than it carries.

I did not run the Python implementation. There is no pip or uv on the machine I measured from. The Python half of the side-by-side is a reading of the source file at a pinned commit, not a behavioural claim.

I did not test every option. forceAtStep, minInvocations, consecutiveAllowed, priority, enabled and onlySuccessInvocations are untouched here. One of them is worth flagging as an open question rather than a finding: onlySuccessInvocations defaults to true, and the filtered step list it produces is then used for the invocation count, the previous-step lookup and the step number, not only for forceAtStep as the parameter reference describes. I have not measured what that does, and I am not asserting anything about it.

I did not test multiple requirements folding together. Every arm attaches exactly one, deliberately, so that nothing else could explain the offered set. Priority ordering across several requirements on the same tool is a different surface.

I did not test a real model. The point of the stub is that the requirement system runs identically either way, because it builds the request before the request is sent. If your provider also drops or rewrites tools, that is a separate layer and this harness will not see it.

The onlyBefore behaviour is version-specific. Everything here is beeai-framework 0.1.31, published 28 September 2026 and the latest release at the time of writing. The structural lesson is not version-specific; the line numbers are.

Prior art

I searched the vendor's issue tracker before writing any of this, because the cheapest way to be wrong is to report something already filed.

A GitHub search across issues and pull requests on i-am-bee/beeai-framework returns zero results for onlyBefore. A search for only_before returns exactly one, pull request 1649, which changes target validation to use a union rather than an intersection and does not touch the run loop. ConditionalRequirement returns sixteen results, none of them about this. onlySuccessInvocations returns one, the TypeScript port itself.

On the open web, a generic search returns no indexed discussion of the option at all. Two attempts at an exact-phrase search came back with the quoted token dropped entirely and results about conditional green cards and college admissions, so those passes measure nothing and I am discarding them rather than scoring them. A search that did echo the query surfaced one close near-miss, the BeeAI Framework encyclopedia entry, updated four days before I read it, which mentions RequirementAgent eight times and ConditionalRequirement zero times; its headings are History, Architecture and design, Features and capabilities, Integration and ecosystem, Competitive landscape, and Significance and impact. The two other readable non-vendor results were about the A2A protocol and mention none of these identifiers.

The vendor's own documentation corpus is 443,165 bytes and mentions onlyBefore exactly once, in the parameter reference quoted at the top. That one occurrence is the whole reason this is a defect rather than an undocumented quirk: the option is not merely unimplemented, it is described as doing something specific.