feat: implicit argument derivation and structural record synthesis by Kamirus · Pull Request #5903 · caffeinelabs/motoko

Kamirus · 2026-03-11T12:35:24Z

Summary

Extends the Motoko compiler's implicit argument resolution with two new mechanisms:

1. Implicit derivation from functions with inner implicits

The compiler can now resolve implicit arguments by composing them from functions (possibly polymorphic) that themselves have implicit parameters. For example, an implicit compare for [Nat] is automatically derived from Array.compare<Nat> + Nat.compare, eliminating boilerplate wrapper modules. Works transitively (e.g., [[Nat]]). Depth-limited with --implicit-derivation-depth (default: 5).

Before:

module MyArray {
  public func compare(a : [Nat], b : [Nat]) : Order { Array.compare(a, b) };
};
let m = Map.empty<[Nat], Text>();
m.add([1, 2, 3], "abc"); // uses MyArray.compare

After:

let m = Map.empty<[Nat], Text>();
m.add([1, 2, 3], "abc"); // just works — derived from Array.compare<Nat> + Nat.compare

2. Structural derivation for records and tuples

A function whose sole explicit parameter is named __record (typed [(Text, T)] -> R) or __tuple (typed [T] -> R) acts as a structural combiner. When the compiler needs an implicit for a record or tuple type, it automatically decomposes the type, resolves a per-field/per-element implicit (using the same search label), and synthesizes a wrapper. Both unary (X -> R) and binary ((X, X) -> R) hole types are supported.

Example — generic JSON serialization for any record:

import Json "mo:json/Json";
import IntJson "mo:json/IntJson";
import TextJson "mo:json/TextJson";

type Person = { name : Text; age : Int };

let p : Person = { name = "Alice"; age = 30 };
let json = p.toJson();
// #obj([("name", #text "Alice"), ("age", #number 30)])

The Json package defines a single _toJson(__record : [(Text, Json)]) : Json combiner. That's all — the compiler handles every record type automatically, as long as each field type has a _toJson instance.

Resolution order

Direct matches are always preferred over derived ones. Within each tier, the most specific candidate (by subtyping) wins:

Direct local values
Direct module fields
Direct library fields (requires --implicit-package)
Derived from local values
Derived from module fields
Derived from library fields (requires --implicit-package)
Structural from local values (__record / __tuple combiner)
Structural from module fields
Structural from library fields (requires --implicit-package)

Test plan

Run tests

test/run/implicit-derivation.mo — basic derivation, monomorphic, polymorphic, multiple implicits, priority ordering, subtyping
test/run/implicit-derivation-transitive.mo — [[Nat]] through two derivation levels
test/run/implicit-derivation-recursive.mo — single recursion, mutual recursion, recursive trees
test/run/implicit-derivation-record-variant.mo — real-world record/variant comparison with compareBy
test/run/implicit-derivation-core.mo — integration with mo:core library (Array, Nat, Int, Text)
test/run/implicit-derivation-implicit-package.mo — derivation from unimported library modules
test/run/implicit-derivation-json.mo — structural derivation for JSON serialization (records, tuples, maps, lists)
test/run/implicit-derivation-structural.mo — basic structural record derivation (unary)
test/run/implicit-derivation-structural-compare.mo — binary structural derivation for record/tuple comparison
test/run/implicit-derivation-structural-dispatch.mo — __record vs __tuple dispatch and binary/unary disambiguation

Fail tests (14 files)

test/fail/implicit-derivation.mo — missing inner implicit, basic derivation errors
test/fail/implicit-derivation-ambiguous.mo — ambiguous derived candidates at the head level
test/fail/implicit-derivation-ambiguous-deep.mo — ambiguous inner implicit during derivation
test/fail/implicit-derivation-bimatch.mo — bi-matching limits during type instantiation
test/fail/implicit-derivation-deep.mo — derivation chain errors at multiple depths
test/fail/implicit-derivation-depth.mo — --implicit-derivation-depth limit hit
test/fail/implicit-derivation-no-backtracking.mo — no backtracking across derivation candidates
test/fail/implicit-derivation-json.mo — real-world JSON errors: missing leaves, expanded Map types, nested records
test/fail/implicit-derivation-structural-missing.mo — record with a field type that has no instance
test/fail/implicit-derivation-structural-ambiguous.mo — two structural combiners in scope
test/fail/implicit-derivation-structural-hint.mo — structural combiner in an unimported library (import hint)
test/fail/implicit-derivation-structural-depth.mo — structural derivation depth limit
test/fail/implicit-derivation-structural-record2-missing.mo — binary record derivation with missing field instance
test/fail/implicit-derivation-structural-tuple-missing.mo — tuple derivation with missing element instance

Test package

test/json-stub/ — minimal JSON package used by the structural derivation tests; demonstrates the __record convention and companion per-type modules (IntJson, TextJson, ArrayJson, ListJson, MapJson, Tuple2Json, Tuple3Json)

When no direct implicit match exists, the compiler now derives implicit arguments from functions that themselves have implicit parameters. For example, if `Array.compare<T>` has an implicit `(T, T) -> Order`, the compiler can automatically derive a `([Nat], [Nat]) -> Order` by instantiating `T := Nat` and recursively resolving `Nat.compare`. This works transitively (e.g. `[[Nat]]` derives through two levels) and with both polymorphic and monomorphic candidates. Resolution depth is bounded by `--implicit-derivation-depth` (default: 5). Error messages distinguish missing vs ambiguous inner implicits and report which candidate was tried and which inner implicit failed. Made-with: Cursor

github-actions · 2026-03-11T12:56:51Z

Comparing from 6e25ac1 to 4b1dda5:
In terms of gas, no changes are observed in 5 tests.
In terms of size, no changes are observed in 5 tests.

… ambiguous candidates

…rameters and enhance error reporting for ambiguous implicit arguments - Removed the `inst` parameter from `synthesize_derived_wrapper` as it was not utilized. - Improved error messages for ambiguous implicit arguments, updating error codes and notes to provide clearer context on the candidates involved. - Updated test cases to reflect changes in error reporting for ambiguous implicit derivations.

crusso · 2026-03-11T16:12:03Z

Cool, any ideas how handle record and variants, where you don't have a function predefined?

Kamirus · 2026-03-11T16:34:56Z

Cool, any ideas how handle record and variants, where you don't have a function predefined?

I think that records and variants need an explicit compare function to say

Which fields take part in the comparison
In which order should the fields be compared

Then we could have helper functions defined in core (or compiler-derived) and use like:

type Foo = {
  name : Text;
  id : Nat;
  subs : List.List<Bar>;
};
module FooCompareByNameAndId {
  public func compare(a : Foo, b : Foo) : Order.Order {
    a.compareBy(b, func (r) { (r.name, r.id) })
    // `compareBy` should be defined in `Order.mo`
  }
};

So that a record/variant comparison could be defined via tuple comparison.
And to support tuple comparison we could either rely on helpers in mo:core in the Tuples module (that would work for tuples up to size N).
Or have the compiler derive those.

- Updated the implicit derivation process to clarify the search order for candidates, prioritizing direct local values over derived ones. - Improved documentation in the language manual and implicit parameters guide to reflect changes in implicit derivation and resolution order. - Fixed minor typos and inconsistencies in comments and documentation across multiple files. - Adjusted test cases to ensure clarity and accuracy in expected outcomes for implicit derivation scenarios.

- Introduced a new function `display_typ_oneline` for concise type display. - Updated `render_derivation_tree` to improve handling of error messages, including clearer differentiation between failed and attempted implicit derivations. - Adjusted test cases to reflect changes in error reporting format, ensuring consistency in the output for ambiguous and not found cases.

- Removed the unused `~tldr` parameter from `render_derivation_tree` for clarity. - Simplified the handling of inner errors by always including the detailed type display. - Adjusted indentation logic for better readability in error reporting.

…e types Detect recursion during implicit derivation by tracking in-progress goals in a shared mutable list. When a cycle is found, return a VarE referencing the already-in-progress function. After resolution succeeds, wrap the result in a BlockE with mutually recursive LetD bindings so all generated $derived_implicit_N functions can reference each other. Made-with: Cursor

…riant helpers Demonstrates how Order.compareBy, Variant3, and Variant4 reduce compare boilerplate for records and variants. Uses mutable fields, optional List<Text> payloads, and two alternative sort orderings. Made-with: Cursor

Made-with: Cursor

- Use `is_matching_typ` in `matching_vals` and `matching_fields` instead of inline `T.sub` (eliminates dead helper) - Refactor `synthesize_derived_wrapper` from mutable refs to `List.fold_left`, making data flow explicit - Document `Local/Returns` restriction on derivation candidates - Update `render_derivation_tree` comment (kept pending full review) - Fix trailing whitespace - Add subtyping edge-case test: `Int.compare` satisfies inner `(Nat, Nat) -> Order` implicit via function contravariance - Add PR number (#5903) to changelog - Accept pre-existing tc-human indentation drift Made-with: Cursor

- Add comment explaining why `assert false` in `resolve_hole` is safe (depends on no-backtracking invariant) - Fix changelog to say "functions (possibly polymorphic)" instead of "polymorphic functions" since derivation also works for monomorphic functions with implicit parameters Made-with: Cursor

Allow the compiler to derive implicit arguments from functions that themselves have implicit parameters (e.g. `compare` for `[Nat]` from `Array.compare<Nat>` + `Nat.compare`). Works transitively and is depth-limited via `--implicit-derivation-depth`. Structural derivation (`__record`/`__tuple` combiners) will follow in a separate PR. Made-with: Cursor

crusso · 2026-04-27T21:07:13Z

+      | _ -> None in
+    match T.promote candidate_typ with
+    | T.Func (T.Local, T.Returns, [], [T.Named ("__record", inner_typ)], [ret_typ]) ->
+      (match T.promote inner_typ with


dubious promote on negative occurrence?

crusso · 2026-04-27T21:07:32Z

+        with_thunk_elem `Record thunk_typ ret_typ
+      | _ -> None)
+    | T.Func (T.Local, T.Returns, [], [T.Named ("__tuple", inner_typ)], [ret_typ]) ->
+      (match T.promote inner_typ with


Ditto: dubious promote on negative occurrence?

crusso · 2026-04-27T21:13:06Z

Is there any way to present this PR as a diff to part 1 #5966? I tried to open a PR with this branch and the branch of #5966 as a base but it didn't reduce the diff for me.

Kamirus · 2026-04-28T07:14:45Z

Is there any way to present this PR as a diff to part 1 #5966? I tried to open a PR with this branch and the branch of #5966 as a base but it didn't reduce the diff for me.

Not easily no, after we merge the part1 then I'll update this PR
Because after reviews and such the part 1 got updates but they were not reapplied on this branch

## Summary Part 1 of caffeinelabs#5903 — implicit argument derivation only, without structural derivation (`__record`/`__tuple` combiners). The compiler can now derive implicit arguments from functions that themselves have implicit parameters. For example, an implicit `compare : ([Nat], [Nat]) -> Order` is automatically derived from `Array.compare<Nat>` when `Nat.compare` is in scope. - Derivation works transitively (e.g. `compare` for `[[Nat]]` chains through `Array.compare<[Nat]>` → `Array.compare<Nat>` → `Nat.compare`) - Recursive derivation cycles are detected and handled via `rec_bindings` - Depth is bounded by `--implicit-derivation-depth` (default 100) - Refactored `resolve_hole` and `disambiguate_resolutions` to support the new derivation tiers while preserving existing direct-match behavior - Improved error messages: derivation failures report which inner implicits are missing and suggest imports Structural derivation (`__record`/`__tuple` combiners for records and tuples) will follow in a separate PR on top of this one.

christoph-dfinity

I'm still a bit worried about code size/perf degradation if this feature is applied liberally in a large-ish codebase. If that becomes a problem I'll rely on you to write the skills to guide the AI.

Some smaller change requests, but looks good otherwise.

christoph-dfinity · 2026-05-11T06:59:47Z

+        || hasPrefix "base-stub/" "${relPath}/"
+        || hasPrefix "json-stub/" "${relPath}/";


I don't think we should be adding to these. The top-level directories under test/ would ideally just be test suites.

I'd rather get rid of the stubs at some point and just replace them with our $MOTOKO_CORE that's around in the tests.

- Use T.normalize (not promote) on the structural combiner's argument type — it's a negative occurrence, so we shouldn't follow Abs bounds. - MapJson._toJson now takes a _toJson implicit for the key (not toText) and encodes entries as #array [k_json, v_json], matching Tuple2Json. Co-authored-by: Cursor <cursoragent@cursor.com>

Same negative-occurrence concern as the candidate-side fix: the hole's domain is the argument type of the function we're synthesizing, so we shouldn't follow Abs bounds. Otherwise structural derivation succeeds inside a polymorphic body on T's bound and silently drops fields of the concrete T at the call site. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-05-27T09:29:57Z

Cursor AI review

👍 APPROVE — looks safe to merge

Category	Assessment	Details
Summary	✅	Adds structural implicit derivation: `__record` / `__tuple` combiners decompose record/tuple hole types, resolve per-field/element implicits recursively, and synthesize lazy-thunk wrappers; integrated after direct and function-composition derivation tiers, with docs, changelog, and broad run/fail coverage.
Code Quality	✅	Reuses existing `ImplicitHoles`/`try_derive_with`/`SynthesizeWrapper` infrastructure cleanly; `Lib.Result.Syntax` added in `lib.ml`/`lib.mli` appears unused (minor dead code, not a defect).
Consistency	✅	Follows established implicit-resolution patterns (tier ordering, disambiguation via subtyping, depth limits, lib gating on `--implicit-package`).
Correctness	✅	Combiner matching (`__record`/`__tuple` + thunk types), hole-shape dispatch (unary/binary record vs tuple), lazy thunk synthesis, lib-derivation ambiguous fallthrough, and depth limiting all align with stated semantics and tests.
Tests	✅	Six new run tests, six new fail tests with matching `.tc*.ok` pairs, JSON/ambig stubs wired in `nix/tests.nix`; covers compare short-circuit, JSON serialization, dispatch, depth, ambiguity, missing leaves, and import hints.
Changelog	✅	User-visible structural derivation feature recorded in the unreleased section at top of `Changelog.md`.

Verdict

Decision: APPROVE
Risk: Low
Reason: This is a well-scoped typechecker extension with thorough test and documentation coverage, no codegen/RTS changes, and no correctness or regression issues identified relative to the base SHA. Residual risk is the usual kind for new implicit-resolution behavior (e.g., documented Map/container structural decomposition), which is explicitly tested and documented.

Generated for commit 4b1dda5

Language manual showed `[(Text, E)] -> R` / `[E] -> R` for `__record` / `__tuple` combiners but the implementation requires lazy thunks (`[(Text, () -> E)] -> R` / `[() -> E] -> R`). The fundamentals doc was already correct. Also drop unused `ImplicitHoles.partition` helper. Co-authored-by: Cursor <cursoragent@cursor.com>

Motoko has no type abstraction, so containers like Map/Set/Buffer that normalize to records can be structurally decomposed if no dedicated instance is in scope. Document the risk and the workaround. Co-authored-by: Cursor <cursoragent@cursor.com>

Replace per-arity Tuple2Json/Tuple3Json with a single TupleJson using the __tuple structural combiner, and move toJson<R> + the __record combiner out of Json.mo into a dedicated RecordJson.mo. Json.mo now only holds the Json type and toText utility. Co-authored-by: Cursor <cursoragent@cursor.com>

Local accept produced 8-space continuation indents because NO_COLOR was set in the dev shell, which strips the ANSI escapes around the note bullet that otherwise contribute to grace's prefix-length-based continuation alignment. CI doesn't set NO_COLOR, so it produces 17-space indents. Regenerate the two affected .ok files to match CI output. Co-authored-by: Cursor <cursoragent@cursor.com>

Move the user-facing toJson<R> entry point back to Json.mo so .toJson() on primitives or tuples doesn't require importing RecordJson. RecordJson now contains only the structural __record combiner. Co-authored-by: Cursor <cursoragent@cursor.com>

Kamirus · 2026-05-27T11:07:33Z

+import ArrayJson "mo:json/ArrayJson";
+import ListJson "mo:json/ListJson";
+import MapJson "mo:json/MapJson";
+import TupleJson "mo:json/TupleJson";


Currently this multiple-imports and flat-modules hierarchy is the recommended way because of how our import hints work.
Our hints look like this currently Did you mean to import mo:json/MapJson?
Nested modules won't show up in these hints so we either accept this style or redesign.

Cover the case where two unimported lib modules provide ambiguous derivation candidates for an implicit hole and a structural-record combiner uniquely resolves it. Without the fall-through behaviour this test would error out at the lib-derivation tier with M0231. Co-authored-by: Cursor <cursoragent@cursor.com>

Move the ambiguity-fallthrough stub package under test/run/ so it ships with the run/ suite via the existing nix sandbox filter, instead of adding another top-level test/ directory. Co-authored-by: Cursor <cursoragent@cursor.com>

Kamirus added 5 commits March 11, 2026 14:01

move as_implicit + refactor HoleAmbiguous avoiding closure to suggest…

c0bca34

… ambiguous candidates

rename

e60e027

disambiguate_resolutions return 3 cases; refactor

35f34bb

refactor

9c9208b

Kamirus and others added 21 commits March 12, 2026 09:01

no backtracking

9c1246f

implicit package direct lib candidates before derived

a6b12e4

HoleCandidates module refactor

7c053b5

refactor

c8ec7c3

refactor + derive from local

b66579d

depth-limited diagnostic

9a7d6d8

render_derivation_tree

5a68d9c

derivation_error

50509c0

Refactor render_derivation_tree function

e264523

- Removed the unused `~tldr` parameter from `render_derivation_tree` for clarity. - Simplified the handling of inner errors by always including the detailed type display. - Adjusted indentation logic for better readability in error reporting.

Merge branch 'master' into kamil/implicit-derivation

94024bf

Merge branch 'master' into kamil/implicit-derivation

1972013

test: group type definitions together in record-variant test

8f1b08d

Made-with: Cursor

test: add manual compare functions to contrast with implicit derivation

0bb47ec

Made-with: Cursor

comment

0d2a403

Merge branch 'master' into kamil/implicit-derivation

54a20ab

crusso reviewed Apr 27, 2026

View reviewed changes

Kamirus added 3 commits April 30, 2026 09:57

Merge branch 'master' into kamil/implicit-derivation

91fadd6

fix merge?

138662c

renames

5b32cf2

christoph-dfinity approved these changes May 11, 2026

View reviewed changes

Kamirus and others added 3 commits May 17, 2026 21:22

Merge branch 'master' into kamil/implicit-derivation

da12881

Kamirus and others added 5 commits May 27, 2026 11:46

Kamirus commented May 27, 2026

View reviewed changes

Kamirus and others added 4 commits May 27, 2026 13:09

fix changelog

42448d4

Merge branch 'master' into kamil/implicit-derivation

c37da18

test: keep lib-ambig stub local to test/run

4b1dda5

Move the ambiguity-fallthrough stub package under test/run/ so it ships with the run/ suite via the existing nix sandbox filter, instead of adding another top-level test/ directory. Co-authored-by: Cursor <cursoragent@cursor.com>

Kamirus added this pull request to the merge queue Jun 1, 2026

Merged via the queue into master with commit 9d0099a Jun 1, 2026
22 checks passed

Kamirus deleted the kamil/implicit-derivation branch June 1, 2026 14:36

github-actions Bot mentioned this pull request Jun 2, 2026

chore: Releasing 1.9.0 #6164

Merged

github-actions Bot mentioned this pull request Jul 27, 2026

feat: searches nested modules for implicits #6084

Open

Navigation Menu

Uh oh!

feat: implicit argument derivation and structural record synthesis - #5903

feat: implicit argument derivation and structural record synthesis#5903
Kamirus merged 70 commits into
masterfrom
kamil/implicit-derivation

Kamirus commented Mar 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 11, 2026 •

edited

Loading

Uh oh!

crusso commented Mar 11, 2026

Uh oh!

Kamirus commented Mar 11, 2026 •

edited

Loading

Uh oh!

crusso Apr 27, 2026

Uh oh!

crusso Apr 27, 2026

Uh oh!

crusso commented Apr 27, 2026

Uh oh!

Kamirus commented Apr 28, 2026

Uh oh!

christoph-dfinity left a comment

Uh oh!

christoph-dfinity May 11, 2026

Uh oh!

Uh oh!

github-actions Bot commented May 27, 2026 •

edited

Loading

Uh oh!

Kamirus May 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		\|\| hasPrefix "base-stub/" "${relPath}/"
		\|\| hasPrefix "json-stub/" "${relPath}/";

Uh oh!

Conversation

Kamirus commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

1. Implicit derivation from functions with inner implicits

2. Structural derivation for records and tuples

Resolution order

Test plan

Run tests

Fail tests (14 files)

Test package

Uh oh!

github-actions Bot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crusso commented Mar 11, 2026

Uh oh!

Kamirus commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crusso Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

crusso Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

crusso commented Apr 27, 2026

Uh oh!

Kamirus commented Apr 28, 2026

Uh oh!

christoph-dfinity left a comment

Choose a reason for hiding this comment

Uh oh!

christoph-dfinity May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Cursor AI review

Verdict

Uh oh!

Kamirus May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Kamirus commented Mar 11, 2026 •

edited

Loading

github-actions Bot commented Mar 11, 2026 •

edited

Loading

Kamirus commented Mar 11, 2026 •

edited

Loading

github-actions Bot commented May 27, 2026 •

edited

Loading