● Every operator works correctly. Issue #29 is implemented. #33

Merged

navicore merged 1 commit from issue-29 into main

2026-06-02 22:53:54 +00:00

navicore commented

2026-06-02 22:53:18 +00:00

Owner

#29

Summary:

Tokenizer (crates/patch-prolog-core/src/tokenizer.rs) — 9 new TokenKind variants with Display arms (lexemes
backticked per #20):

Pow (**) — b'*' arm peeks for second *.
Caret (^) — new b'^' arm.
Colon (:) — b':' arm's fallthrough now emits Colon (was an error).
ShiftLeft (<<), ShiftRight (>>) — b'<'/b'>' arms peek for repeated char (>> is tested before >= since both
start with > followed by different bytes).
BitAnd (/) — b'/' arm: longest-match order // → /\ → /.
BitOr (/) — b'\' arm: longest-match == → = → + → / → bare .
Div, Xor — added to the keyword table next to is/mod/rem.
Tokenizer unit tests verify each new token, plus regression coverage that longest-match order is preserved (*
not **, >= not >>, // not /, :- not :).

Parser (parser.rs):

New parse_expr_200 between parse_expr_400 and parse_primary. Handles three operators with three different
associativities at the same precedence level — ** (xfx, no chaining), ^ and : (xfy, right-associative via
self-recursion on RHS).
parse_expr_400 extended with Div, ShiftLeft, ShiftRight (yfx) — and its operand calls now route through
parse_expr_200.
parse_expr_500 extended with BitAnd, BitOr, Xor (yfx).
operator_as_atom_lookahead extended with all 9 new tokens so f(**), [<<, >>, xor], X = (div) still read as
bare atoms when in closing context.

Arithmetic evaluator (builtins.rs):

** — always float per ISO 9.3.1, even Int**Int. Goes through check_float.
^ — Int^Int with non-negative exponent uses checked_pow for int result; anything else (negative exponent,
float operand) falls through to powf.
<<, >> — int-only via checked_shl/checked_shr. Negative shift count or count ≥ 64 throws
evaluation_error(undefined).
/, /, xor — int-only bitwise via Rust's &/|/^ (no overflow possible).
div — ISO floor division. Truncating checked_div + checked_rem, then adjust by -1 when the remainder is
non-zero and has the opposite sign of the divisor. Disagrees with // (truncating) when signs differ; agrees on
the positive case.
: is parser-only — deliberately not added to eval_arith (matches SWI behavior when not in module mode).

Integration tests — 21 new tests in tests/integration.rs covering each operator's value, the right-associativity
of ^ and :, the precedence relationships (** < *, << < +, /\ left-to-right with +), atom-in-closing-context for
the new tokens, and the two error paths (negative shift, div 0). Plus regression coverage that 1 + 2 and the
existing + * / // mod rem family still work.

Out of scope per the issue body: declaration-class prefixes (dynamic/multifile/...), DCG, comparison-aliases
beyond what already exists. The "full prefix + infix" doc the issue mentioned can now be written accurately.

https://git.navicore.tech/navicore/patch-prolog/issues/29 Summary: Tokenizer (crates/patch-prolog-core/src/tokenizer.rs) — 9 new TokenKind variants with Display arms (lexemes backticked per #20): - Pow (**) — b'*' arm peeks for second *. - Caret (^) — new b'^' arm. - Colon (:) — b':' arm's fallthrough now emits Colon (was an error). - ShiftLeft (<<), ShiftRight (>>) — b'<'/b'>' arms peek for repeated char (>> is tested before >= since both start with > followed by different bytes). - BitAnd (/\) — b'/' arm: longest-match order // → /\ → /. - BitOr (\/) — b'\\' arm: longest-match \== → \= → \+ → \/ → bare \. - Div, Xor — added to the keyword table next to is/mod/rem. - Tokenizer unit tests verify each new token, plus regression coverage that longest-match order is preserved (* not **, >= not >>, // not /\, :- not :). Parser (parser.rs): - New parse_expr_200 between parse_expr_400 and parse_primary. Handles three operators with three different associativities at the same precedence level — ** (xfx, no chaining), ^ and : (xfy, right-associative via self-recursion on RHS). - parse_expr_400 extended with Div, ShiftLeft, ShiftRight (yfx) — and its operand calls now route through parse_expr_200. - parse_expr_500 extended with BitAnd, BitOr, Xor (yfx). - operator_as_atom_lookahead extended with all 9 new tokens so f(**), [<<, >>, xor], X = (div) still read as bare atoms when in closing context. Arithmetic evaluator (builtins.rs): - ** — always float per ISO 9.3.1, even Int**Int. Goes through check_float. - ^ — Int^Int with non-negative exponent uses checked_pow for int result; anything else (negative exponent, float operand) falls through to powf. - <<, >> — int-only via checked_shl/checked_shr. Negative shift count or count ≥ 64 throws evaluation_error(undefined). - /\, \/, xor — int-only bitwise via Rust's &/|/^ (no overflow possible). - div — ISO floor division. Truncating checked_div + checked_rem, then adjust by -1 when the remainder is non-zero and has the opposite sign of the divisor. Disagrees with // (truncating) when signs differ; agrees on the positive case. - : is parser-only — deliberately not added to eval_arith (matches SWI behavior when not in module mode). Integration tests — 21 new tests in tests/integration.rs covering each operator's value, the right-associativity of ^ and :, the precedence relationships (** < *, << < +, /\ left-to-right with +), atom-in-closing-context for the new tokens, and the two error paths (negative shift, div 0). Plus regression coverage that 1 + 2 and the existing + * / // mod rem family still work. Out of scope per the issue body: declaration-class prefixes (dynamic/multifile/...), DCG, comparison-aliases beyond what already exists. The "full prefix + infix" doc the issue mentioned can now be written accurately.

navicore added 1 commit

2026-06-02 22:53:19 +00:00

● Every operator works correctly. Issue #29 is implemented.

CI - Linux / CI - Linux x86_64 (pull_request) Successful in 21s

Details

56a63c173f

Summary:

  Tokenizer (crates/patch-prolog-core/src/tokenizer.rs) — 9 new TokenKind variants with Display arms (lexemes
  backticked per #20):
  - Pow (**) — b'*' arm peeks for second *.
  - Caret (^) — new b'^' arm.
  - Colon (:) — b':' arm's fallthrough now emits Colon (was an error).
  - ShiftLeft (<<), ShiftRight (>>) — b'<'/b'>' arms peek for repeated char (>> is tested before >= since both
  start with > followed by different bytes).
  - BitAnd (/\) — b'/' arm: longest-match order // → /\ → /.
  - BitOr (\/) — b'\\' arm: longest-match \== → \= → \+ → \/ → bare \.
  - Div, Xor — added to the keyword table next to is/mod/rem.
  - Tokenizer unit tests verify each new token, plus regression coverage that longest-match order is preserved (*
  not **, >= not >>, // not /\, :- not :).

  Parser (parser.rs):
  - New parse_expr_200 between parse_expr_400 and parse_primary. Handles three operators with three different
  associativities at the same precedence level — ** (xfx, no chaining), ^ and : (xfy, right-associative via
  self-recursion on RHS).
  - parse_expr_400 extended with Div, ShiftLeft, ShiftRight (yfx) — and its operand calls now route through
  parse_expr_200.
  - parse_expr_500 extended with BitAnd, BitOr, Xor (yfx).
  - operator_as_atom_lookahead extended with all 9 new tokens so f(**), [<<, >>, xor], X = (div) still read as
  bare atoms when in closing context.

  Arithmetic evaluator (builtins.rs):
  - ** — always float per ISO 9.3.1, even Int**Int. Goes through check_float.
  - ^ — Int^Int with non-negative exponent uses checked_pow for int result; anything else (negative exponent,
  float operand) falls through to powf.
  - <<, >> — int-only via checked_shl/checked_shr. Negative shift count or count ≥ 64 throws
  evaluation_error(undefined).
  - /\, \/, xor — int-only bitwise via Rust's &/|/^ (no overflow possible).
  - div — ISO floor division. Truncating checked_div + checked_rem, then adjust by -1 when the remainder is
  non-zero and has the opposite sign of the divisor. Disagrees with // (truncating) when signs differ; agrees on
  the positive case.
  - : is parser-only — deliberately not added to eval_arith (matches SWI behavior when not in module mode).

  Integration tests — 21 new tests in tests/integration.rs covering each operator's value, the right-associativity
  of ^ and :, the precedence relationships (** < *, << < +, /\ left-to-right with +), atom-in-closing-context for
  the new tokens, and the two error paths (negative shift, div 0). Plus regression coverage that 1 + 2 and the
  existing + * / // mod rem family still work.

  Out of scope per the issue body: declaration-class prefixes (dynamic/multifile/...), DCG, comparison-aliases
  beyond what already exists. The "full prefix + infix" doc the issue mentioned can now be written accurately.

navicore referenced this pull request

2026-06-02 22:53:34 +00:00

Parser: add missing standard infix operators (** ^ >> << xor div /\ \/ :) #29

navicore merged commit b6e06c517e into main

2026-06-02 22:53:54 +00:00

navicore deleted branch issue-29

2026-06-02 22:53:54 +00:00

navicore referenced this pull request from a commit

2026-06-02 22:53:55 +00:00

Merge pull request '● Every operator works correctly. Issue #29 is implemented.' (#33) from issue-29 into main