Skip to content

Commit cb27275

Browse files
committed
Rotom: axis roll endpoints, roll steps, and per-piece roll semantics
A roll endpoint is now either a piece -- a dims-list position, the original Rotom reading -- or a whole tensor axis, spelled 'axis N' and stored as -(axis+1) in the flat rolls array. An axis endpoint is legal only when the axis is packed as more than one piece; the piece spelling is canonical for an unsplit axis, where the two coincide. A piece FROM rewrites only its own mixed-radix digit -- the original per-piece semantics, now materialized correctly on split axes (no borrow crosses digits). An axis FROM rewrites the whole axis index modulo its full extent, each piece then taking its digit of the rolled index: the borrow across digits is what diagonal packings over a split axis need and no combination of piece rolls can express. A BY piece of a split axis shifts by that piece's digit of the axis's current (possibly already-rolled) expression. Rolls also take an optional step s, stored in a parallel rollSteps array (canonically omitted when all 1): the FROM index moves by -s * shift(by) mod its extent. A baby-step/giant-step pre-rotation's magnitude is expressible structurally (a piece FROM shifts its axis in strides of the piece), but its direction is not -- the kernel's shared baby rotations need the pre-rotation to run against the roll's subtraction convention, and the mirror-direction packing would need per-giant baby rotations, forfeiting the BSGS saving -- so the signed step stays explicit. Together these make the BSGS diagonal packing an ordinary layout: #rotom.layout<n = 16, rolls = [(axis 1, 2), (2, 0, -4)], dims = [[1:4:4], [1:4:1], [0:16:1]]> where roll 1 diagonalizes the whole split k against i (ciphertext (g, b) holds the digits of (k - i) mod 16) and roll 2 pre-rotates i by 4g, giant digit read from the rolled k.
1 parent 60ec4c7 commit cb27275

8 files changed

Lines changed: 739 additions & 119 deletions

File tree

lib/Dialect/Rotom/IR/RotomAttributes.cpp

Lines changed: 212 additions & 51 deletions
Large diffs are not rendered by default.

lib/Dialect/Rotom/IR/RotomAttributes.h

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,48 @@
1515

1616
namespace mlir::heir::rotom {
1717

18+
// One endpoint of a roll: either one piece of the layout (a position in its
19+
// dims list) or a whole tensor axis (spelled `axis N`; legal only when the
20+
// axis is packed as more than one piece -- the piece spelling is canonical
21+
// for an unsplit axis, where the two coincide).
22+
struct RollEndpoint {
23+
bool isAxis;
24+
int64_t index; // Dims-list position, or the tensor axis id when isAxis.
25+
bool operator==(const RollEndpoint& other) const {
26+
return isAxis == other.isAxis && index == other.index;
27+
}
28+
};
29+
30+
// Endpoint encoding in the flat rolls storage: a piece endpoint is its
31+
// non-negative dims-list position, an axis endpoint is -(axis + 1).
32+
inline int64_t encodeRollEndpoint(RollEndpoint e) {
33+
return e.isAxis ? -(e.index + 1) : e.index;
34+
}
35+
inline RollEndpoint decodeRollEndpoint(int64_t encoded) {
36+
return encoded < 0 ? RollEndpoint{true, -encoded - 1}
37+
: RollEndpoint{false, encoded};
38+
}
39+
40+
// One roll of a layout: FROM's index is rewritten to
41+
// (idx_from - step * shift(by)) mod extent(from), where a piece FROM
42+
// rewrites only its own mixed-radix digit and an axis FROM rewrites the
43+
// whole axis index. `step` is 1 unless the layout carries an explicit
44+
// rollSteps entry.
45+
struct RollSpec {
46+
RollEndpoint from;
47+
RollEndpoint by;
48+
int64_t step;
49+
};
50+
51+
// The layout's rolls with both endpoints decoded and their steps attached
52+
// (step 1 where the rollSteps storage is absent). Reading rolls this way
53+
// keeps a caller from copying roll pairs while silently dropping their
54+
// steps.
55+
llvm::SmallVector<RollSpec> getRollSpecs(LayoutAttr layout);
56+
57+
// Whether every roll of the layout has unit step (or there are no rolls).
58+
bool allRollStepsAreUnit(LayoutAttr layout);
59+
1860
enum class LayoutPieceKind { Traversal, Replication, Gap };
1961

2062
struct LayoutData {

lib/Dialect/Rotom/IR/RotomAttributes.td

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -57,26 +57,53 @@ def Rotom_LayoutAttr : Rotom_Attr<"Layout", "layout"> {
5757
ciphertext side of Rotom's `;` split (one piece); remaining entries are
5858
in-slot. See [Section 4.2 of the Rotom paper](https://eprint.iacr.org/2025/1319.pdf).
5959

60-
Optional **rolls** encode a `roll(i,j)` metadata object: each pair `(i, j)`
61-
indexes into the `dims` array (the flattened `ct_dims + slot_dims` list) and
62-
rewrites `dims[i]`'s index to `(idx_i - idx_j) mod size(dims[i])`. The two
63-
extents need not match: the shift reduces modulo the rolled dim's extent, so
64-
a smaller partner covers a prefix of the rotations and a larger one wraps.
60+
Optional **rolls** encode `roll(from, by)` metadata objects, applied left
61+
to right. Each endpoint is either a *piece* -- a position in the `dims`
62+
list, spelled as a bare integer -- or a whole tensor *axis*, spelled
63+
`axis N` (legal only when axis `N` is packed as more than one piece; the
64+
piece spelling is canonical for an unsplit axis, where the two coincide).
65+
66+
A piece FROM rewrites that piece's own mixed-radix digit in place,
67+
`digit(from) <- (digit(from) - s * shift(by)) mod extent(from)`, leaving
68+
the axis's other digits untouched. An `axis` FROM rewrites the whole axis
69+
index modulo its full extent, and each piece then takes its digit of the
70+
rolled index -- the shift borrows across digits, which no combination of
71+
piece rolls can express. The shift is the by endpoint's index: a
72+
traversal piece's digit (its whole index when the axis is unsplit), a
73+
whole axis's index via `axis`, a replication piece's replica index, or a
74+
gap piece's block index. The two extents need not match: the shift
75+
reduces modulo the rolled extent, so a smaller partner covers a prefix of
76+
the rotations and a larger one wraps.
77+
78+
The step `s` defaults to 1 and may be spelled as a third tuple element
79+
`(from, by, s)`. A baby-step/giant-step pre-rotation runs AGAINST the
80+
roll's subtraction direction (step `-B` shifts by `+B * shift(by)`);
81+
the magnitude of a step is often expressible structurally instead (a
82+
piece FROM shifts its axis in strides of the piece), but the direction
83+
is not, so the step stays explicit.
84+
85+
Optional **rollSteps** stores one step per roll pair; it must be omitted
86+
when every step is 1 (the canonical form the parser and builders produce).
6587
}];
6688

6789
let parameters = (ins
6890
"::mlir::ArrayAttr":$dims,
6991
"int64_t":$n,
70-
OptionalParameter<"::mlir::DenseI64ArrayAttr">:$rolls
92+
OptionalParameter<"::mlir::DenseI64ArrayAttr">:$rolls,
93+
OptionalParameter<"::mlir::DenseI64ArrayAttr">:$rollSteps
7194
);
7295

7396
let genVerifyDecl = 1;
7497
let hasCustomAssemblyFormat = 1;
7598

7699
let extraClassDeclaration = [{
77-
/// Layout with no `roll(i,j)` metadata (empty rolls storage).
100+
/// Layout with no `roll(from, by)` metadata (empty rolls storage).
78101
static ::mlir::heir::rotom::LayoutAttr get(::mlir::MLIRContext *context,
79102
::mlir::ArrayAttr dims, int64_t n);
103+
/// Layout with unit-step rolls (empty step storage).
104+
static ::mlir::heir::rotom::LayoutAttr get(::mlir::MLIRContext *context,
105+
::mlir::ArrayAttr dims, int64_t n,
106+
::mlir::DenseI64ArrayAttr rolls);
80107
}];
81108
}
82109

lib/Dialect/Rotom/Utils/BUILD

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ cc_test(
2929
"@heir//lib/Dialect/Rotom/IR:Dialect",
3030
"@heir//lib/Utils/Layout:Evaluate",
3131
"@heir//lib/Utils/Layout:IslConversion",
32+
"@heir//lib/Utils/Layout:Utils",
33+
"@llvm-project//mlir:Analysis",
3234
"@llvm-project//mlir:IR",
3335
"@llvm-project//mlir:Support",
3436
],

lib/Dialect/Rotom/Utils/RotomTensorExtLayoutLowering.cpp

Lines changed: 142 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -18,20 +18,40 @@
1818
namespace mlir::heir::rotom {
1919
namespace {
2020

21-
/// Maps a `#rotom.dim` from the layout's `dims` list to its iterator index `i*`
22-
/// after preprocessing (match logical axis, size, and stride).
23-
static FailureOr<int64_t> traversalIndexForRotomDim(
24-
const SmallVector<DimAttr>& traversalDims, DimAttr want) {
21+
/// Maps a tensor axis id to its iterator index `i*` after preprocessing.
22+
/// Preprocessing dedupes traversal dims per logical axis (a mixed-radix split
23+
/// contributes one variable), so the axis id identifies the variable.
24+
static FailureOr<int64_t> traversalIndexForAxis(
25+
const SmallVector<DimAttr>& traversalDims, int64_t axis) {
2526
for (int64_t i = 0; i < static_cast<int64_t>(traversalDims.size()); ++i) {
26-
if (traversalDims[i].getDim() == want.getDim() &&
27-
traversalDims[i].getSize() == want.getSize() &&
28-
traversalDims[i].getStride() == want.getStride()) {
29-
return i;
30-
}
27+
if (traversalDims[i].getDim() == axis) return i;
3128
}
3229
return failure();
3330
}
3431

32+
/// Product of the extents of `dim`'s pieces in the layout's dims list (the
33+
/// logical axis extent; a single whole-dim piece gives its own size).
34+
static int64_t fullExtentOfDim(ArrayAttr rotomDims, int64_t dim) {
35+
int64_t extent = 1;
36+
for (Attribute a : rotomDims) {
37+
auto d = cast<DimAttr>(a);
38+
if (!d.isGap() && !d.isReplicate() && d.getDim() == dim) {
39+
extent *= d.getSize();
40+
}
41+
}
42+
return extent;
43+
}
44+
45+
/// Whether `dim` is packed as more than one mixed-radix piece.
46+
static bool isSplitDim(ArrayAttr rotomDims, int64_t dim) {
47+
int64_t count = 0;
48+
for (Attribute a : rotomDims) {
49+
auto d = cast<DimAttr>(a);
50+
if (!d.isGap() && !d.isReplicate() && d.getDim() == dim) ++count;
51+
}
52+
return count > 1;
53+
}
54+
3555
static std::string modExpr(llvm::StringRef expr, int64_t mod) {
3656
std::string out;
3757
llvm::raw_string_ostream os(out);
@@ -84,7 +104,7 @@ static LogicalResult emitSegmentAddress(
84104
const SmallVector<DimAttr>& replicationDims,
85105
int64_t numActiveTraversalComponents, size_t segStart, size_t segEnd,
86106
const llvm::DenseSet<int64_t>& rolledGapIndices, ArrayRef<int64_t> rolls,
87-
ArrayAttr rotomDims) {
107+
ArrayRef<int64_t> rollSteps, ArrayAttr rotomDims) {
88108
// Effective extent of a piece = the range of its mixed-radix digit: modBy
89109
// when the piece reads i mod L (the slot/low part), else sz/divBy (the whole
90110
// dim when divBy 1, or the ct/high part sz/L of a straddling dim).
@@ -166,56 +186,117 @@ static LogicalResult emitSegmentAddress(
166186
traversalExprs.push_back("i" + std::to_string(i));
167187
}
168188

169-
// Apply roll(a,b) transforms left-to-right:
170-
// t_a <- (t_a - t_b) mod extent(a).
189+
// Apply rolls left-to-right. A piece FROM rewrites that piece's own
190+
// mixed-radix digit in place, (digit - step * shift) mod extent(piece),
191+
// leaving its axis's other digits untouched; an `axis` FROM rewrites the
192+
// whole axis index mod its full extent, so the shift borrows across digits
193+
// (each piece then takes its digit of the rolled index). The shift is the
194+
// by endpoint's index: a traversal piece's digit (its whole index when the
195+
// axis is unsplit), a whole axis via `axis`, a slot replication's replica
196+
// index, or a gap's block index. A step multiplies the shift -- the
197+
// baby-step/giant-step pre-rotation runs against the roll's subtraction
198+
// direction, so its sign cannot be folded into the endpoint structure.
171199
//
172-
// The rewrite lands wherever the FROM dimension sits -- the ciphertext
173-
// address, the slot address, or both when it straddles the boundary. BY is
174-
// another traversal dimension on either axis, or a slot replication/gap
175-
// block index, so a roll diagonalizes a ciphertext dimension against a slot
176-
// one (one ciphertext per diagonal) or two slot dimensions (a Halevi-Shoup
177-
// slot diagonal); a roll within the ciphertext axis is a free ciphertext
200+
// The rewrite lands wherever the FROM pieces sit -- the ciphertext
201+
// address, the slot address, or both when the axis straddles the boundary.
202+
// A roll diagonalizes a ciphertext dimension against a slot one (one
203+
// ciphertext per diagonal) or two slot dimensions (a Halevi-Shoup slot
204+
// diagonal); a roll within the ciphertext axis is a free ciphertext
178205
// relabeling that enumeration never generates.
179206
if (!rolls.empty()) {
180207
if (!rotomDims || rolls.size() % 2 != 0) return failure();
181208
for (size_t i = 0; i < rolls.size(); i += 2) {
182-
const int64_t fromIdx = rolls[i];
183-
const int64_t toIdx = rolls[i + 1];
184-
if (fromIdx < 0 || toIdx < 0 ||
185-
fromIdx >= static_cast<int64_t>(rotomDims.size()) ||
186-
toIdx >= static_cast<int64_t>(rotomDims.size())) {
209+
const RollEndpoint from = decodeRollEndpoint(rolls[i]);
210+
const RollEndpoint by = decodeRollEndpoint(rolls[i + 1]);
211+
const int64_t step = i / 2 < rollSteps.size() ? rollSteps[i / 2] : 1;
212+
if (!from.isAxis && from.index >= static_cast<int64_t>(rotomDims.size()))
187213
return failure();
188-
}
189-
auto fromDim = dyn_cast<DimAttr>(rotomDims[fromIdx]);
190-
auto toDim = dyn_cast<DimAttr>(rotomDims[toIdx]);
191-
if (!fromDim || !toDim) return failure();
214+
if (!by.isAxis && by.index >= static_cast<int64_t>(rotomDims.size()))
215+
return failure();
216+
DimAttr fromPiece =
217+
from.isAxis ? DimAttr() : dyn_cast<DimAttr>(rotomDims[from.index]);
218+
if (!from.isAxis && !fromPiece) return failure();
219+
const int64_t fromAxis = from.isAxis ? from.index : fromPiece.getDim();
192220
FailureOr<int64_t> maybeFromTrav =
193-
traversalIndexForRotomDim(traversalDims, fromDim);
221+
traversalIndexForAxis(traversalDims, fromAxis);
194222
if (failed(maybeFromTrav)) return failure();
195223
const int64_t fromTrav = *maybeFromTrav;
224+
196225
std::string toExpr;
197-
if (toDim.isGap()) {
198-
// Rolling by a gap dim: the shift is the gap's existential block
199-
// index, so block g holds the rolled dim cyclically shifted by g.
200-
toExpr = "g" + std::to_string(gapIndexForPosition(rotomDims, toIdx));
201-
} else if (toDim.isReplicate()) {
202-
// Rolling by a replication dim: the shift is the replica index, whose
203-
// existential variable is named after the piece's replication slot.
204-
int64_t replicationIndex = 0;
205-
for (int64_t k = 0; k < toIdx; ++k) {
206-
if (cast<DimAttr>(rotomDims[k]).isReplicate()) ++replicationIndex;
207-
}
208-
toExpr = "d" + std::to_string(numActiveTraversalComponents +
209-
replicationIndex);
210-
} else {
226+
if (by.isAxis) {
227+
// Rolling by a whole axis: the shift is the axis's (possibly
228+
// already-rolled) index expression.
211229
FailureOr<int64_t> maybeToTrav =
212-
traversalIndexForRotomDim(traversalDims, toDim);
230+
traversalIndexForAxis(traversalDims, by.index);
213231
if (failed(maybeToTrav)) return failure();
214232
toExpr = traversalExprs[*maybeToTrav];
233+
} else {
234+
auto toDim = dyn_cast<DimAttr>(rotomDims[by.index]);
235+
if (!toDim) return failure();
236+
if (toDim.isGap()) {
237+
// Rolling by a gap dim: the shift is the gap's existential block
238+
// index, so block g holds the rolled index cyclically shifted by g.
239+
toExpr =
240+
"g" + std::to_string(gapIndexForPosition(rotomDims, by.index));
241+
} else if (toDim.isReplicate()) {
242+
// Rolling by a replication dim: the shift is the replica index,
243+
// whose existential variable is named after the piece's replication
244+
// slot.
245+
int64_t replicationIndex = 0;
246+
for (int64_t k = 0; k < by.index; ++k) {
247+
if (cast<DimAttr>(rotomDims[k]).isReplicate()) ++replicationIndex;
248+
}
249+
toExpr = "d" + std::to_string(numActiveTraversalComponents +
250+
replicationIndex);
251+
} else {
252+
FailureOr<int64_t> maybeToTrav =
253+
traversalIndexForAxis(traversalDims, toDim.getDim());
254+
if (failed(maybeToTrav)) return failure();
255+
toExpr = traversalExprs[*maybeToTrav];
256+
if (isSplitDim(rotomDims, toDim.getDim())) {
257+
// The by piece's digit: (i / stride) mod extent, with the modulus
258+
// redundant on the most-significant digit.
259+
const int64_t fullTo = fullExtentOfDim(rotomDims, toDim.getDim());
260+
if (toDim.getStride() > 1) {
261+
toExpr = floorDivExpr(toExpr, toDim.getStride());
262+
}
263+
if (toDim.getStride() * toDim.getSize() < fullTo) {
264+
toExpr = modExpr(toExpr, toDim.getSize());
265+
}
266+
}
267+
}
268+
}
269+
270+
if (step != 1) {
271+
toExpr = std::to_string(step) + " * (" + toExpr + ")";
272+
}
273+
274+
if (from.isAxis || !isSplitDim(rotomDims, fromAxis)) {
275+
// Whole-axis rewrite (the piece spelling of an unsplit axis is the
276+
// same operation: its one piece's digit is the axis index).
277+
std::string diffExpr =
278+
"(" + traversalExprs[fromTrav] + " - " + toExpr + ")";
279+
traversalExprs[fromTrav] =
280+
modExpr(diffExpr, fullExtentOfDim(rotomDims, fromAxis));
281+
} else {
282+
// Piece rewrite on a split axis: replace this piece's digit with
283+
// (digit - step * shift) mod extent(piece), i.e. add
284+
// stride * (newDigit - digit) to the axis index. No borrow crosses
285+
// into the other digits.
286+
const int64_t stride = fromPiece.getStride();
287+
const int64_t extent = fromPiece.getSize();
288+
const std::string axisExpr = traversalExprs[fromTrav];
289+
std::string digit = axisExpr;
290+
if (stride > 1) digit = floorDivExpr(digit, stride);
291+
if (stride * extent < fullExtentOfDim(rotomDims, fromAxis)) {
292+
digit = modExpr(digit, extent);
293+
}
294+
const std::string newDigit =
295+
modExpr("(" + digit + " - " + toExpr + ")", extent);
296+
traversalExprs[fromTrav] =
297+
"(" + axisExpr + " + " + std::to_string(stride) + " * (" +
298+
newDigit + ") - " + std::to_string(stride) + " * (" + digit + "))";
215299
}
216-
std::string diffExpr =
217-
"(" + traversalExprs[fromTrav] + " - " + toExpr + ")";
218-
traversalExprs[fromTrav] = modExpr(diffExpr, fromDim.getSize());
219300
}
220301
}
221302

@@ -257,7 +338,7 @@ static FailureOr<std::string> emitSplitCtSlotIsl(
257338
const SmallVector<DimAttr>& replicationDims,
258339
const SmallVector<DimAttr>& gapDims, int64_t numTraversalComponents,
259340
int64_t numReplication, int64_t numGap, ArrayRef<int64_t> rolls,
260-
ArrayAttr rotomDims) {
341+
ArrayRef<int64_t> rollSteps, ArrayAttr rotomDims) {
261342
if (prefix > pieces.size()) return failure();
262343

263344
const llvm::DenseSet<int64_t> rolledGapIndices =
@@ -332,20 +413,21 @@ static FailureOr<std::string> emitSplitCtSlotIsl(
332413
emitAnd();
333414
bool firstTerm = true;
334415
os << "ct = ";
335-
if (failed(emitSegmentAddress(os, firstTerm, pieces, pieceIndex, pieceDivBy,
336-
pieceModBy, traversalDims, gapDims,
337-
replicationDims, numTraversalComponents, 0,
338-
prefix, rolledGapIndices, rolls, rotomDims)))
416+
if (failed(emitSegmentAddress(
417+
os, firstTerm, pieces, pieceIndex, pieceDivBy, pieceModBy,
418+
traversalDims, gapDims, replicationDims, numTraversalComponents, 0,
419+
prefix, rolledGapIndices, rolls, rollSteps, rotomDims)))
339420
return failure();
340421
if (firstTerm) os << "0";
341422

342423
emitAnd();
343424
firstTerm = true;
344425
os << "slot = ";
345-
if (failed(emitSegmentAddress(
346-
os, firstTerm, pieces, pieceIndex, pieceDivBy, pieceModBy,
347-
traversalDims, gapDims, replicationDims, numTraversalComponents,
348-
prefix, pieces.size(), rolledGapIndices, rolls, rotomDims)))
426+
if (failed(emitSegmentAddress(os, firstTerm, pieces, pieceIndex, pieceDivBy,
427+
pieceModBy, traversalDims, gapDims,
428+
replicationDims, numTraversalComponents, prefix,
429+
pieces.size(), rolledGapIndices, rolls,
430+
rollSteps, rotomDims)))
349431
return failure();
350432
if (firstTerm) os << "0";
351433

@@ -388,10 +470,14 @@ static FailureOr<std::string> lowerToIslImpl(LayoutAttr layout) {
388470
DenseI64ArrayAttr rollsAttr = layout.getRolls();
389471
ArrayRef<int64_t> rolls =
390472
rollsAttr ? rollsAttr.asArrayRef() : ArrayRef<int64_t>{};
473+
DenseI64ArrayAttr stepsAttr = layout.getRollSteps();
474+
ArrayRef<int64_t> rollSteps =
475+
stepsAttr ? stepsAttr.asArrayRef() : ArrayRef<int64_t>{};
391476
return emitSplitCtSlotIsl(
392477
data.n, data.ctPrefixLen, data.pieces, data.pieceIndex, data.pieceDivBy,
393478
data.pieceModBy, data.traversalDims, data.replicationDims, data.gapDims,
394-
numTraversalComponents, numReplication, numGap, rolls, layout.getDims());
479+
numTraversalComponents, numReplication, numGap, rolls, rollSteps,
480+
layout.getDims());
395481
}
396482

397483
} // namespace

0 commit comments

Comments
 (0)