Recursive Flint: All Four LogicMark Benchmarks, Scores and Failures

Recursive Flint is a new open-weight coding model with an unusually small active-parameter count for its total size. We tested it against every current LogicMark benchmark: Math and Coding, each in Tool-assisted and Quick mode. We started on OpenRouter and used Kilo Gateway’s same-named free route after reaching OpenRouter’s daily quota. These are original measurements from October 22, 2026, rather than the developer’s published benchmark scores. The cover is an AI-generated illustration.

OpenRouter’s Flint listing identifies the route as recursive/flint-v1.0:free, released October 21, 2026, with free input and output and a roughly 262K-token context. Its dashboard latency, throughput and availability are rolling service statistics. They change, so the figures in a copied listing should not be treated as the speed this evaluation achieved.

Recursive’s model card specifies a text-only mixture-of-experts Transformer with 39.8B total parameters and approximately 2.5B active per token, including the output head. It documents 49 layers, 256 routed experts per MoE layer with six selected per token, and a shared expert. The weights use the Apache 2.0 license; hosted service terms are separate.

Active parameters do not describe the full download: the card lists approximately 80 GB for BF16 weights. Quantized deployments differ. We tested hosted inference, not browser execution or local-hardware speed.

In its launch article, Recursive describes Eureka Machine as an automated research system that contributed to architecture, data, training, quantization and inference optimization. It reports fast local inference on selected Apple and NVIDIA hardware. Those are developer claims under their stated conditions, not measurements made in this review.

There is a context distinction worth preserving: the launch article advertises up to 512K tokens and recommends staying within 256K, while the hosted route tested here advertises 262,144. Use the limit of the actual endpoint and runtime you select. A maximum context length does not guarantee equal accuracy at every length.

The model card reports 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified. We did not reproduce those evaluations. They use different agent scaffolds, sampling and attempt policies, and should not be combined with our LogicMark scores.

LogicMark currently contains two version-1.0.0 suites: twelve exact-answer Math problems and ten Coding tasks, split evenly between JavaScript and Python. Coding has 55 checker cases across those tasks. We attempted all four suite/mode combinations and added a separately labeled larger-budget Coding diagnostic after a default-run failure. This is a small, inspectable workload, not a general intelligence score or a substitute for a long real-repository evaluation.

Tool-assisted Math starts a fresh conversation per problem and permits a standard-library Python action. Tool-assisted Coding provides a fresh starter repository and allows listing, reading and writing files. The model cannot run its coding submission or see checker results before submitting. Quick mode sends all twelve Math questions or all ten Coding repositories in a single request and accepts one response, with no tools or follow-up feedback.

The QA page imported the application’s actual task definitions, runner, Quick runner and sandbox bridge. Coding submissions were evaluated in the site’s existing isolated Pyodide and QuickJS runtimes. We recorded each Tool-assisted task separately using the same fresh-workspace runner, then applied LogicMark’s equal-task aggregation. This instrumentation preserved the prompts and grading rules. The run did not change the user’s equipped models, active chat model or personal leaderboard.

Requests went directly to the named free model through temporary local credential-holding bridges for OpenRouter and Kilo. API keys stayed server-side and are absent from the published artifacts. We used LogicMark’s fixed temperature of zero, an 8,192-token output cap and 180-second request timeout for default Tool-assisted mode; Quick used 32,768 tokens and 600 seconds. A separate Coding diagnostic increased its per-response cap to 32,768 and timeout to 600 seconds. Tools mode retained the 24-turn limit per task. Python and checker execution retained the existing 20-second limit per invocation.

This setup differs from Recursive’s recommendation of temperature 1, top-p 0.95, top-k 20 and at least 32,768 output tokens for difficult problems. We kept LogicMark’s common settings to make this a harness evaluation. The model-facing protocol uses JSON actions in message content, not OpenAI-native tool-call objects, and does not replay private reasoning fields. The results therefore describe compatibility and performance with this particular harness.

Math awards either 100 or zero for an exact answer, with rational-number normalization. Coding awards the percentage of deterministic cases passed for each task; all ten tasks then contribute equally to the suite mean. Cases within a task can provide partial credit. The checker compares outputs and verifies that submissions preserve their inputs. No AI judge assigns these scores.

Malformed submissions score zero under the existing protocol; provider or evaluator infrastructure failures are reported separately. There were no editorial repairs to a model answer or submitted file before grading. A Python error returned to the model is part of the permitted Math interaction; an unavailable evaluator is a different event. We did not collect repeated completed attempts under identical conditions, so there is no reliable estimate of run-to-run variance. Provider retries and altered-budget diagnostics remain separate records.

A dash means no suite score was recorded, not zero. The 0.0 Quick Coding score has a different meaning: the API returned text, but the response ended inside a JSON string. LogicMark’s strict parser could not extract the required results object, so every task received zero. We did not repair the JSON or salvage individual files. This measures delivery within the prescribed budget and protocol; it does not show that all ten proposed algorithms were incorrect.

Wall time includes request latency and any local tool/checker work. It is not a tokens-per-second benchmark. At most two retained inference requests overlapped while the gateway retry and Coding diagnostic ran. The initial OpenRouter requests exhausted a 50-request daily free quota; no credits were purchased. We also checked OpenCode’s free API, which rejected direct use outside its client, and did not attempt to bypass that restriction. Kilo accepted the same free model ID through its configured account.

The completed Tool-assisted Math run scored 66.7/100, or eight correct answers out of twelve, in 183.9 seconds including tool work. The completed Kilo Quick Math run scored 41.7/100, or five out of twelve, in 176.1 seconds. An earlier OpenRouter Quick Math attempt returned null content with a length-limit finish and received no suite score. Provider differences and single-attempt sampling prevent treating this as a controlled estimate of the benefit of Python.

The Tool-assisted misses were not all mathematical failures. The bounded-excursion and spanning-tree responses had malformed JSON, so their proposed Python actions never executed. On the recurrence task, Flint ran a list with an extra leading element, obtained the wrong value and submitted it. Python faithfully executing a flawed algorithm does not establish correctness. The congruence task received a wrong direct answer.

There were useful recovery behaviors too. On constrained subset counting, a slow brute-force attempt hit the execution limit, an intermediate rewrite had an indentation error, and a later dynamic-programming implementation produced a correct submission. On the determinant task, Flint recovered after trying an unavailable Python package. These interactions exercised the real Python tool rather than a simulated answer fixture.

The default 8,192-token Coding attempt stopped on the inventory task without a complete suite score. A larger-budget OpenRouter attempt then hit the daily quota. We continued diagnostic coverage through Kilo with 32,768 tokens per response. A development-page reload interrupted that sequential diagnostic after four inventory responses had been retained. This was a test-page interruption, not a model failure.

To finish inventory, we replayed its four recorded actions into the same starter workspace, verified that every cached request history matched exactly, and continued inside the original 24-turn limit. No file was repaired and no checker feedback was given to the model before submission. The remaining eight tasks ran independently, two at a time, with the original prompts and isolated checker. These diagnostic observations are not pooled into a completed default-suite or leaderboard score.

Six tasks passed all of their checks, one passed partially, and three failed at the action protocol. In the CSV and calendar tasks, Flint returned two separate read objects in one response. For template rendering, it returned a list action followed by two read objects. LogicMark permits exactly one object per turn. Those zeros reveal a real integration limitation in this harness, rather than demonstrating that the model cannot implement CSV parsing, templates or calendar logic.

The passing tasks show useful repository-level behavior: reading starter files, changing implementations and submitting working Python or JavaScript without executing coding tests itself. The inventory result also shows why producing plausible code is insufficient: one deterministic case still failed. We retained that result rather than fixing its file and presenting the repaired output as Flint’s work.

The run manifest records separate attempts, per-task diagnostics, settings, timing, API usage and SHA-256 hashes of the runner, task pack and evaluator source. The task prompts and starter files exclude expected answers and hidden checker cases. The original visible model responses preserve returned JSON actions, files and finish reasons; private reasoning fields and authentication headers are not published.

Download the evidence bundle. The retained inference responses reported zero cost wherever a usage cost field was present. That is provider-reported telemetry, not an independent billing audit. Readiness and availability probes are outside the retained benchmark-call records.

Some responses reported a reasoning-token counter larger than their completion-token counter. We preserve those inconsistent fields unchanged, rather than adding them together or claiming a precise hardware-compute measurement. In the successful Kilo Quick Math request, the API reported 1,381 prompt tokens and 25,647 completion tokens; its reasoning detail was 16,497. The output budget must cover enough work to deliver a usable final response, regardless of how short that final answer looks.

The checked-in evidence validator can be run with node scripts/check-flint-logicmark.mjs on the project’s supported Node version. It verifies source hashes, task coverage, score arithmetic and that exported prompts omit grading keys. Coding results themselves came from the isolated browser runtimes, not that metadata validator or an AI judge.

Flint solved eight of twelve Math tasks with Python available and five in the completed one-response Quick run. Its larger-budget Coding diagnostics included six complete task passes and a partial pass. We also observed strict-format failures, output-budget exhaustion and a free-provider quota interruption. These are all part of the result.

For this harness, instruction adherence and delivering a complete response mattered as much as writing an algorithm. Quick means one model response, not guaranteed low latency. Provider choice, budget and reasoning-history handling deserve attention before using these numbers to predict behavior in another agent. We did not benchmark local hardware, repeat the vendor’s published coding evaluations, estimate variance, or establish a ranking against other models.

To try it yourself, select OpenRouter in Model Organizer and use recursive/flint-v1.0:free, then equip it for LogicMark. Free access has provider limits and can change. Keep Quick and Tool-assisted results separate, and treat an incomplete run as incomplete. Our Model Organizer guide explains provider setup; our token-efficiency article explains why visible answer length, billed tokens and successful work need separate measurements.