Skip to content

Fix/range ring stencil overflow - #149

Open
M3RT1N99 wants to merge 10 commits into
FAForever:masterfrom
M3RT1N99:fix/range-ring-stencil-overflow
Open

M3RT1N99 wants to merge 10 commits into
FAForever:masterfrom
M3RT1N99:fix/range-ring-stencil-overflow

Conversation

@M3RT1N99

@M3RT1N99 M3RT1N99 commented Mar 31, 2026 •

Copy link
Copy Markdown

Fix range ring stencil overflow with 128+ overlapping units

Problem
When 128+ units overlap (e.g., 200 engineers in a cluster), their range ring overlays render as individual circles instead of merging into a single unified shape. This is caused by the 7-bit stencil counter in func_RenderRings (0x7EF5A0) wrapping from 0x7F to 0x00, creating holes in the fill area that the boundary Cast loop then fills with visible ring outlines.

Old:
image

New:
image

Fix
Insert intermediate RangeMask flushes in the first Cast loop of func_RenderRings. Batch size is reduced from 1000 to 30 units. After each batch, a RangeMask fullscreen pass converts accumulated stencil marks to bit 7 before overflow can occur. Subsequent Cast operations skip already-marked pixels via StencilFunc=notequal on bit 7, which also enables GPU stencil early-out for dense clusters.

Files
hooks/HRenderRingsFlush.cpp — 3 hooks: two data patches reducing batch size at 0x7EF77E/0x7EF785, one call hook at 0x7EF7EA
section/RenderRingsFlush.cpp — IntermediateRangeMask() function replicating the RangeMask sequence from 0x7EF802-0x7EF849
section.ld — external symbols 0x7F5DA0 (InitTransformedVerts) and 0x7F6030 (CRenFrame::Render)
Test instructions
Spawn 200+ engineers at the same location
Enable Build Range overlay — should show one merged fill area, no individual circles
Select a single unit — normal range ring, no artifacts
Compare FPS with large clusters before/after — should be equal or better due to stencil early-out

Checklist

  • Read all guides
  • Clear naming of variables and structs
  • Add new data into moho.h/global.h/Info.txt
  • Add description of changes to README.md
  • Add test hints/instructions

Summary by CodeRabbit

  • Bug Fixes
    • Fixed visual artifacts when rendering range rings with 128+ overlapping units, improving stability in densely populated scenarios.

Marti and others added 2 commits March 31, 2026 17:44
The 7-bit stencil counter in func_RenderRings overflows when 128+
units overlap, causing individual circle outlines instead of merged
fill. Fix: batch size 1000->30 with intermediate RangeMask flushes
in the first Cast loop. Enables GPU stencil early-out for dense
clusters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Info.txt: document CRenFrame::InitTransformedVerts and CRenFrame::Render addresses and calling conventions
- changelog.md: add entry under Optimizations

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 31, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Symbol entries are updated in Info.txt with a qualified label for RenderRings and three new public function entries added. A new changelog entry documents a fix for range ring rendering artifacts caused by 7-bit GPU stencil overflow when 128+ units overlap. A runtime hook file is introduced to patch the RenderRings function, reducing batch size from 1000 to 128 and enabling intermediate flushes.

Changes

Cohort / File(s) Summary
Symbol Updates
Info.txt
Updated RenderRings entry with qualified label func_RenderRings. Added three new symbol entries: CRenFrame::InitTransformedVerts, CRenFrame::Render, and std::string::string constructor, each with retn 8 annotations.
Documentation
changelog.md
Added technical-patch changelog entry describing fix for overlapping range ring rendering. Documents batch size reduction (1000 → 128), intermediate RangeMask flushes, and GPU stencil early-out optimization.
Runtime Patching
hooks/RenderRingsFlush.hook
New hook file targeting func_RenderRings to address 7-bit stencil overflow. Patches Loop 1's batch operands from 0x3E8 to 128, preventing counter wrap-around during dense ring overlaps.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A stencil in seven bits did break,
When rings all merged and started to shake!
Now batched by 128, flushed with grace,
Our rendering loops find their perfect space! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Fix/range ring stencil overflow' is concise and clearly describes the main technical change being addressed in the PR.
Description check ✅ Passed The PR description is comprehensive, covering the problem statement with visuals, the implemented fix, affected files, test instructions, and all checklist items marked as complete.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
section/RenderRingsFlush.cpp (2)

8-8: Consider adding AssignString to Info.txt for consistency.

InitTransformedVerts (0x7F5DA0) and CRenFrame::Render (0x7F6030) were added to Info.txt, but AssignString (0x4059E0) referenced here is not. For maintainability, consider adding it alongside the other entries.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@section/RenderRingsFlush.cpp` at line 8, Add the missing AssignString entry
to Info.txt to match the other symbols; locate the existing entries for
InitTransformedVerts and CRenFrame::Render and add a line for AssignString with
its signature and address ("AssignString (0x4059E0): this@<ecx>, (char* s,
size_t n), retn 8") so Info.txt consistently documents all referenced symbols.

1-1: Unused include.

The desync_fix_global.h header is included but none of its declarations (tick_num, tag_sent, num_clients, etc.) are used in IntermediateRangeMask(). Consider removing it or replacing with a more minimal include if only specific definitions are needed.

🔧 Suggested fix
-#include "desync_fix_global.h"
+#include "global.h"

Or if no includes are actually required:

-#include "desync_fix_global.h"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@section/RenderRingsFlush.cpp` at line 1, The include "desync_fix_global.h" at
the top of RenderRingsFlush.cpp is unused in the IntermediateRangeMask()
implementation; remove that include (or replace it with a more specific header
that provides only the symbols actually needed) and then rebuild to ensure no
unresolved symbols remain; specifically check the IntermediateRangeMask function
and any helper/inline functions in this translation unit for usages of tick_num,
tag_sent, num_clients, or other symbols from desync_fix_global.h before
removing, and if any are required, include only the minimal header that declares
those specific symbols.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@section/RenderRingsFlush.cpp`:
- Line 8: Add the missing AssignString entry to Info.txt to match the other
symbols; locate the existing entries for InitTransformedVerts and
CRenFrame::Render and add a line for AssignString with its signature and address
("AssignString (0x4059E0): this@<ecx>, (char* s, size_t n), retn 8") so Info.txt
consistently documents all referenced symbols.
- Line 1: The include "desync_fix_global.h" at the top of RenderRingsFlush.cpp
is unused in the IntermediateRangeMask() implementation; remove that include (or
replace it with a more specific header that provides only the symbols actually
needed) and then rebuild to ensure no unresolved symbols remain; specifically
check the IntermediateRangeMask function and any helper/inline functions in this
translation unit for usages of tick_num, tag_sent, num_clients, or other symbols
from desync_fix_global.h before removing, and if any are required, include only
the minimal header that declares those specific symbols.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 086a5596-9b2e-4ce4-9447-2ded30da7df3

📥 Commits

Reviewing files that changed from the base of the PR and between 8da86fe and 8027c74.

📒 Files selected for processing (4)
  • Info.txt
  • changelog.md
  • hooks/HRenderRingsFlush.cpp
  • section/RenderRingsFlush.cpp

Comment thread hooks/HRenderRingsFlush.cpp Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Convert to the .hook format. Maybe:

// === FIRST Cast loop (fill rings) ===
// Reduce batch size from 1000 to 30 for stencil early-out optimization
0x007EF77D:
    cmp eax, 0x1E
    jl .+0x9
    mov eax, 0x1E
    nop
    nop

// Hook after func_Draw_Rings: intermediate RangeMask flush
// Replaces 7-byte mov eax,[esp+0x84] with call + 2 NOPs
0x007EF7EA:
    call IntermediateRangeMask
    nop
    nop

Comment thread section/RenderRingsFlush.cpp Outdated
"movss dword ptr [esp], xmm0;"
);
asm(
"call -0x1000 +8351136;" // InitTransformedVerts

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call -0x1000 +8351136; -> call 0x007F5DA0;

Comment thread section/RenderRingsFlush.cpp Outdated
Address review feedback on PR FAForever#149:
- Move hook injection from hooks/HRenderRingsFlush.cpp (legacy .cpp form)
  to hooks/RenderRingsFlush.hook (canonical .hook format)
- Replace `call -0x1000 +N` address arithmetic with explicit hex addresses
  (call 0x007F5DA0 / 0x004059E0 / 0x007F6030)
- Consolidate the four split asm() blocks in IntermediateRangeMask into a
  single asm() to prevent the compiler from inserting code between them
- Document the custom register-based thiscall used by CRenFrame methods
  (this@ebx for InitTransformedVerts, this@edi for Render) and explain
  why a direct C++ call is impossible
- Add detailed header comment explaining the loop1/loop2 stencil-overflow
  analysis and why only loop 1 needs patching (loop 2 draws thin edge
  strips with negligible per-pixel overdraw)
- Drop unused desync_fix_global.h include
- Add std::string::string ctor (0x004059E0) entry to Info.txt

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
section/RenderRingsFlush.cpp (2)

86-90: Minor: consider documenting the hardcoded string address.

The address 0x00E3F8E8 for the "RangeMask" string literal works but could benefit from a brief comment noting where this address comes from (e.g., "string literal in .rdata at 0x00E3F8E8" or referencing the engine's existing usage).

📝 Suggested documentation
         // --- std::string::string(this=ecx, "RangeMask", 9) ---
         "push 9;"
-        "push 0x00E3F8E8;"                 // address of "RangeMask" string
+        "push 0x00E3F8E8;"                 // "RangeMask" literal in .rdata (used by engine at 0x7EF82C)
         "mov ecx, edi;"                    // this @ ecx
         "call 0x004059E0;"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@section/RenderRingsFlush.cpp` around lines 86 - 90, Document the hardcoded
string address 0x00E3F8E8 used when pushing the "RangeMask" literal (seen in the
push sequence before the call to 0x004059E0) by adding a brief comment
explaining its origin (for example: "string literal in .rdata at 0x00E3F8E8" or
referencing the engine global where "RangeMask" is defined/used), so future
readers can trace why this specific address is used and where the literal comes
from.

67-106: Consider using __attribute__((naked)) for explicit stack frame control.

The stack offset calculations assume esp = caller_esp - 4 at entry. While the extensive header comments show careful stack analysis, this is fragile if the compiler generates prologue code (push ebp; mov ebp, esp).

However, similar inline-asm-only functions in the codebase (e.g., selectionPriority() in section/selectionPriority.cpp) work without this attribute, suggesting the compiler does not generate prologue for such functions in the current build environment. To make the intent explicit and improve robustness across compiler versions, consider marking the function as naked:

-void IntermediateRangeMask()
+void __attribute__((naked)) IntermediateRangeMask()

This is not a confirmed issue but a defensive measure to prevent future fragility.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@section/RenderRingsFlush.cpp` around lines 67 - 106, Add the GCC/Clang naked
attribute to the IntermediateRangeMask function declaration to prevent the
compiler from emitting a prologue/epilogue (which would invalidate the current
esp-based offsets); update the function signature to include
__attribute__((naked)) so the inline asm remains the sole emitted code and stack
layout is stable, and verify the function body remains a single asm block that
performs its own push/pop and returns (keeping the existing
IntermediateRangeMask asm sequence and restoring registers as it does now).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@section/RenderRingsFlush.cpp`:
- Around line 86-90: Document the hardcoded string address 0x00E3F8E8 used when
pushing the "RangeMask" literal (seen in the push sequence before the call to
0x004059E0) by adding a brief comment explaining its origin (for example:
"string literal in .rdata at 0x00E3F8E8" or referencing the engine global where
"RangeMask" is defined/used), so future readers can trace why this specific
address is used and where the literal comes from.
- Around line 67-106: Add the GCC/Clang naked attribute to the
IntermediateRangeMask function declaration to prevent the compiler from emitting
a prologue/epilogue (which would invalidate the current esp-based offsets);
update the function signature to include __attribute__((naked)) so the inline
asm remains the sole emitted code and stack layout is stable, and verify the
function body remains a single asm block that performs its own push/pop and
returns (keeping the existing IntermediateRangeMask asm sequence and restoring
registers as it does now).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e69a21d4-189a-469e-a495-3a3bc2bcb2d9

📥 Commits

Reviewing files that changed from the base of the PR and between 8027c74 and fe4f13b.

📒 Files selected for processing (4)
  • Info.txt
  • changelog.md
  • hooks/RenderRingsFlush.hook
  • section/RenderRingsFlush.cpp
🚧 Files skipped from review as they are similar to previous changes (2)
  • changelog.md
  • Info.txt

@RutreD RutreD left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks satisfactory to me. The patch works and, in addition, increases FPS by about 15% (500 units)

@M3RT1N99

M3RT1N99 commented Apr 8, 2026

Copy link
Copy Markdown
Author

@RutreD I’m currently implementing a feature that displays health bars only for damaged units, as constantly rendering them—especially in dense unit clusters—may impact FPS.

@RutreD

RutreD commented Apr 9, 2026

Copy link
Copy Markdown

@RutreD I’m currently implementing a feature that displays health bars only for damaged units, as constantly rendering them—especially in dense unit clusters—may impact FPS.

Nice
Take a look at the shaders too. I’m no expert, calculating view * proj matrix once on the CPU and passing it to the shader might be more efficient than doing it per-vertex in the shader. See mul(viewMatrix, projMatrix) in some files under https://github.com/FAForever/fa/blob/develop/effects . We can use the float4x4 viewProjMatrix implementation from sky.fx as a reference and apply it to the rest of the shaders

@M3RT1N99

M3RT1N99 commented Apr 9, 2026

Copy link
Copy Markdown
Author

@RutreD

The real performance bottleneck with range rings is not the stencil overflow itself — it's the sheer number of rings being processed per frame. With all rings enabled and 600+ units crowded together, func_RenderRings (0x7EF5A0) drops FPS from ~110 to ~16 because every single unit renders its own ring even when fully hidden by neighbours.

I confirmed this by toggling ren_Ranges 0 in-game which instantly recovered FPS from 30 → 80. No other render toggle (ren_Shadows, ren_Fx, ren_MeshSkinned, ren_Decals) had any effect.

The stencil overflow fix (batch cap + intermediate flush) addresses the visual bug at 128+ overlapping rings but does not help with the underlying performance cost. A separate hull-cull patch on feature/range-ring-hull-cull tackles the FPS side by dropping interior rings whose outer circle is fully covered by neighbours, reducing 600 rings to ~50 with no visible change in the merged outline.

M3RT1N99 added a commit to M3RT1N99/FA-Binary-Patches that referenced this pull request Apr 9, 2026
The engine's range ring pipeline uses a 7-bit GPU stencil counter that
increments per overlapping ring fill. At 128+ rings the counter wraps,
breaking the stencil-based outline merge (individual broken circles
instead of a unified shape). This hard cap guarantees correct visuals
even for pathological formations. Makes PR FAForever#149 fully obsolete.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@hooks/RenderRingsFlush.hook`:
- Around line 19-25: The two patched immediate literals currently set to 128
(the .int entries shown at 0x007EF77E and 0x007EF785 that correspond to the
patched 'mov eax, imm32' for the loop/batch ceiling) contradict the intended
reduction to 30 and reintroduce the 7‑bit overflow; replace both .int 128 values
with .int 30 so the mov eax immediate becomes 30 (or any value <128) to stay
strictly under the GPU's 7‑bit stencil limit and match the design documented in
section/RenderRingsFlush.cpp.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3f1dfa38-a1a7-45a9-ada9-8bae05869a15

📥 Commits

Reviewing files that changed from the base of the PR and between fe4f13b and 542ebed.

📒 Files selected for processing (1)
  • hooks/RenderRingsFlush.hook

Comment thread hooks/RenderRingsFlush.hook Outdated
Comment on lines +19 to +25
0x007EF77E:
.int 128

// Loop 1 batch ceiling: mov eax, 0x3E8 -> mov eax, 30
// (patches the imm32 of the 5-byte 'mov eax, imm32' instruction at 0x007EF784)
0x007EF785:
.int 128

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: literal value 128 contradicts the stated batch size of 30 and reintroduces the overflow it is meant to fix.

The comments at Lines 14, 17, and 22, the PR description, and the AI summary all state the batch size is reduced from 1000 to 30 (replacing cmp eax, 0x3E8/mov eax, 0x3E8 with a 30 immediate). However, both data patches write .int 128, not 30.

This is not just a documentation mismatch — it defeats the fix. The GPU stencil counter is 7‑bit (max 127). Allowing up to 128 rings per batch means a fully overlapping batch can still wrap the counter from 0x7F to 0x00 before the intermediate RangeMask flush runs, producing exactly the individual‑circle artifact this PR targets. To stay strictly under the 7‑bit limit (and match the design rationale documented in section/RenderRingsFlush.cpp), the literal must be 30 (or at minimum < 128).

🐛 Proposed fix
 // Loop 1 batch cap: cmp eax, 0x3E8  ->  cmp eax, 30
 // (patches the imm32 of the 5-byte 'cmp eax, imm32' instruction at 0x007EF77D)
 0x007EF77E:
-    .int 128
+    .int 30

 // Loop 1 batch ceiling: mov eax, 0x3E8  ->  mov eax, 30
 // (patches the imm32 of the 5-byte 'mov eax, imm32' instruction at 0x007EF784)
 0x007EF785:
-    .int 128
+    .int 30
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
0x007EF77E:
.int 128
// Loop 1 batch ceiling: mov eax, 0x3E8 -> mov eax, 30
// (patches the imm32 of the 5-byte 'mov eax, imm32' instruction at 0x007EF784)
0x007EF785:
.int 128
0x007EF77E:
.int 30
// Loop 1 batch ceiling: mov eax, 0x3E8 -> mov eax, 30
// (patches the imm32 of the 5-byte 'mov eax, imm32' instruction at 0x007EF784)
0x007EF785:
.int 30
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@hooks/RenderRingsFlush.hook` around lines 19 - 25, The two patched immediate
literals currently set to 128 (the .int entries shown at 0x007EF77E and
0x007EF785 that correspond to the patched 'mov eax, imm32' for the loop/batch
ceiling) contradict the intended reduction to 30 and reintroduce the 7‑bit
overflow; replace both .int 128 values with .int 30 so the mov eax immediate
becomes 30 (or any value <128) to stay strictly under the GPU's 7‑bit stencil
limit and match the design documented in section/RenderRingsFlush.cpp.

@4z0t

4z0t commented Apr 26, 2026

Copy link
Copy Markdown
Member

Watch and learn :P

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
hooks/RenderRingsFlush.hook (1)

17-25: ⚠️ Potential issue | 🔴 Critical

Batch cap of 128 still wraps the 7‑bit counter — must be <128.

The 7‑bit stencil holds values 0..127. A batch of 128 maximally‑overlapping rings increments the counter 128 times, and the 128th increment wraps 0x7F → 0x00, which is exactly the failure mode this PR sets out to fix. The PR description and inline rationale both reference a target of 30; the literal here should match (or at minimum be <128).

This was raised on a previous revision and still applies.

🐛 Proposed fix
 0x007EF77E:
-    .int 128
+    .int 30
@@
 0x007EF785:
-    .int 128
+    .int 30
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@hooks/RenderRingsFlush.hook` around lines 17 - 25, The batch cap values
patched at the two immediates (the .int entries currently set at 0x007EF77E and
0x007EF785, corresponding to the "cmp eax, 0x3E8 -> cmp eax, 128" and "mov eax,
0x3E8 -> mov eax, 128" diffs) must be reduced below 128 to avoid wrapping the
7‑bit stencil; change both immediates from 128 to the intended target 30 (or any
value <128) so the 7‑bit counter cannot wrap on the Nth increment.
🧹 Nitpick comments (1)
section/RenderRingsFlush.cpp (1)

67-106: Add __attribute__((naked)) for robustness against future build flag changes.

The function relies on the compiler omitting prologue/epilogue, which is guaranteed by GCC's default -fomit-frame-pointer at standard optimization levels on x86. However, this implicit contract makes the code fragile: if build flags ever change (or this file is compiled with different settings than the rest), the stack offsets become invalid. The codebase currently doesn't use the naked attribute for other asm-only functions (e.g., HookHydroCondition in BuildTemplateSnap.cpp, functions in moho_input.cpp), but adding it here makes the intent explicit and future-proof.

Additionally, consider extern "C" if the binary patcher references this function by unmangled name; the current C++ mangling would prevent it from being found.

Suggested fix
-void IntermediateRangeMask()
+extern "C" __attribute__((naked)) void IntermediateRangeMask()
 {
     asm(
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@section/RenderRingsFlush.cpp` around lines 67 - 106, The
IntermediateRangeMask function is relying on implicit omission of
prologue/epilogue; mark it explicitly as naked and export it with unmangled
linkage so the patcher finds it: add the __attribute__((naked)) attribute to the
IntermediateRangeMask declaration and wrap its declaration/definition in extern
"C" (or add extern "C" linkage only if the patcher requires unmangled name) so
the assembler body remains intact and the symbol name is predictable; update any
corresponding forward declaration or header to match the naked extern "C"
signature so calls/patcher references still resolve.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@hooks/RenderRingsFlush.hook`:
- Around line 27-66: The comment and dead code mismatch: remove the unused
IntermediateRangeMask() function from section/RenderRingsFlush.cpp (it's never
called) and update the comment in hooks/RenderRingsFlush.hook to describe that
the hook contains an inline assembly block reproducing the full RangeMask logic
(instead of claiming a 5-byte call + 2 NOPs to IntermediateRangeMask); ensure no
references to IntermediateRangeMask() remain and keep the existing label
L_ExitRangeRender and the inline sequence (pushad → InitTransformedVerts →
RangeMask constructor → Render → popad → mov eax,[esp+0x88]) intact in the hook
comment wording.

---

Duplicate comments:
In `@hooks/RenderRingsFlush.hook`:
- Around line 17-25: The batch cap values patched at the two immediates (the
.int entries currently set at 0x007EF77E and 0x007EF785, corresponding to the
"cmp eax, 0x3E8 -> cmp eax, 128" and "mov eax, 0x3E8 -> mov eax, 128" diffs)
must be reduced below 128 to avoid wrapping the 7‑bit stencil; change both
immediates from 128 to the intended target 30 (or any value <128) so the 7‑bit
counter cannot wrap on the Nth increment.

---

Nitpick comments:
In `@section/RenderRingsFlush.cpp`:
- Around line 67-106: The IntermediateRangeMask function is relying on implicit
omission of prologue/epilogue; mark it explicitly as naked and export it with
unmangled linkage so the patcher finds it: add the __attribute__((naked))
attribute to the IntermediateRangeMask declaration and wrap its
declaration/definition in extern "C" (or add extern "C" linkage only if the
patcher requires unmangled name) so the assembler body remains intact and the
symbol name is predictable; update any corresponding forward declaration or
header to match the naked extern "C" signature so calls/patcher references still
resolve.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0e77fc43-a824-4c30-8a17-4becf29bda57

📥 Commits

Reviewing files that changed from the base of the PR and between 542ebed and 33311c0.

📒 Files selected for processing (3)
  • changelog.md
  • hooks/RenderRingsFlush.hook
  • section/RenderRingsFlush.cpp
✅ Files skipped from review due to trivial changes (1)
  • changelog.md

Comment thread hooks/RenderRingsFlush.hook Outdated
@4z0t

4z0t commented Apr 26, 2026

Copy link
Copy Markdown
Member

@RutreD can you review?

@4z0t

4z0t commented Apr 26, 2026

Copy link
Copy Markdown
Member

My change is basically moving code that was executed after the loop into it. Had to change some registers, since esi was used as increment.

Comment thread hooks/RenderRingsFlush.hook Outdated
// Fix: reduce Loop 1 batch size from 1000 to 30, and inject an intermediate
// RangeMask flush between batches that resets the stencil counter.

// Loop 1 batch cap: cmp eax, 0x3E8 -> cmp eax, 30

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 30? What's the math behind it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

007EEDC9 checks agains 1000 even if you changed that. is it intentional?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was testing and 128 was also fine.

@4z0t 4z0t added the fix pull request fixes an issue or a bug label Apr 28, 2026
@RutreD

RutreD commented May 1, 2026

Copy link
Copy Markdown

@4z0t I love it, even more fps! btw, you forget delete .cpp in section folder

@4z0t

4z0t commented May 1, 2026

Copy link
Copy Markdown
Member

Yeah, I was hoping that @M3RT1N99 finalizes it, so comments are all in place :D

- Delete section/RenderRingsFlush.cpp (no longer called by inlined hook)
- Add comprehensive calling convention documentation to hook header
  explaining why `this` is passed in EBX/EDI instead of standard locations
  and why direct C++ calls are impossible
- Replace bare `#` markers with descriptive comments on loop counter ops
- Clean up `push offset` syntax to plain `push imm32` for consistency

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@M3RT1N99

M3RT1N99 commented May 1, 2026

Copy link
Copy Markdown
Author

Response to 4z0t's Questions

Q1: "Why 30? What's the math behind it?"

30 was an overly conservative choice without strict mathematical justification. The original reasoning was: "7-bit stencil max = 127, so add a large safety margin and use 30."

Your observation is more sound: 128 is mathematically exact — that's the precise 7-bit boundary (0-127 = 128 values). Your rewrite to 128 is the correct approach. The stencil counter can theoretically approach the edge at 128, but in practice (as you've tested), it works fine.

Q2: "007EEDC9 checks against 1000 even if you changed that. is it intentional?"

Good catch! There are indeed additional 1000-checks at 0x7EEDC9 and 0x7EEE01, but they are intentionally NOT patched — and here's why:

These are two completely different systems:

  1. Your Stencil-Batch patches (0x7EF77E, 0x7EF785):

    • Control the GPU stencil counter overflow in Loop 1
    • Patched: 1000 → 128 ✅
  2. Vertex-Buffer limits (0x7EEDC9, 0x7EEE01):

    • Part of sub_7EEDC0 function (separate from stencil logic)
    • Control vertex RAM allocation, not stencil
    • Both Loop 1 and Loop 2 call this function
    • Should NOT be patched (they're independent of stencil batch size)

The 0x3E8 (1000) limit in sub_7EEDC0 is a vertex buffer capacity check, not a stencil limit. These are orthogonal systems — reducing stencil batches doesn't require changing vertex buffer limits.

Q3: "I was testing and 128 was also fine."

Exactly — your testing validates the approach. 128 is mathematically correct and works in practice. This is the right call for the stencil fix.

The fact that it works well means the theoretical edge case (where exactly 128 overlapping rings hit the stencil boundary) either doesn't occur frequently enough to matter, or the GPU stencil wraps gracefully in this scenario.


Summary: Your rewrite to 128 is solid. The 0x7EEDC9/0x7EEE01 checks are intentionally unchanged — they're vertex buffer management, a separate orthogonal system.

@4z0t

4z0t commented May 1, 2026 •

Copy link
Copy Markdown
Member

Uuugh... I don't need AIs insight on it. I was hoping for a living being response on it. The reason why ask is that 30 was very small and in general case flushing (or whatever was happening with buffer) would perhaps be too frequent and cause a performance impact. From my perspective 128 was okay. I was testing 256 too, but that was as same as 1000. I still need more tests in future to see if there is an impact of performance compared to original behavior. Fixing visual bug is nice, but we cannot neglect performance.

@4z0t

4z0t commented May 1, 2026 •

Copy link
Copy Markdown
Member

Plus while testing I noticed the same issue happening with fog too. Perhaps you could look into it as well, but it isn't something that you clearly see, not a priority.

@M3RT1N99

M3RT1N99 commented May 1, 2026

Copy link
Copy Markdown
Author

Plus while testing I noticed the same issue happening with fog too. Perhaps you could look into it as well, but it isn't something that you clearly see, not a priority.

I saw this also an gray ring around the units if you teleport them in one point you can see it clear, i have tried it but did not figure out what it is

@M3RT1N99

M3RT1N99 commented May 1, 2026

Copy link
Copy Markdown
Author

Uuugh... I don't need AIs insight on it. I was hoping for a living being response on it. The reason why ask is that 30 was very small and in general case flushing (or whatever was happening with buffer) would perhaps be too frequent and cause a performance impact. From my perspective 128 was okay. I was testing 256 too, but that was as same as 1000. I still need more tests in future to see if there is an impact of performance compared to original behavior. Fixing visual bug is nice, but we cannot neglect performance.

Sorry, but I can't remember why I was okay with 30. There was a specific reason, but I can't seem to find the conversation right now. I'll need to look into it more deeply later.

@BlackYps

Copy link
Copy Markdown

Sorry it took a while. Is this ready for merging? Now would be a good time, because we have three months until the next patch, so plenty of testing time

M3RT1N99 and others added 2 commits August 18, 2026 19:50
Two fixes to the inlined RenderRingsFlush hook:

1. The 6-byte 'jle' section at 0x7EF76E is padded to 8 bytes by the
   assembler (COFF sections are 4-byte aligned) and the patcher writes
   the full section size, so the two zero padding bytes clobbered
   'mov eax, ebp' at 0x7EF774 -- the back-branch target of Loop 1.
   Reproduce the instruction inside the section so the padded 8 bytes
   are exactly the original layout.

2. Reduce the Loop 1 batch cap from 128 to 127. The Cast technique
   (range.fx) increments with StencilWriteMask 0x7F, so counts
   accumulate mod 128: a pixel covered by exactly 128 rings of one
   batch wraps to 0 and RangeMask (passes on low bits != 0) leaves it
   unmarked. 127 is the largest batch that can never wrap. Repro of
   the old edge case: exactly 128 stacked units lose their entire fill.

Also drop the stale section/RenderRingsFlush.cpp reference from the
changelog entry (file was removed when the patch was inlined).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M3RT1N99 added a commit to M3RT1N99/FA-Binary-Patches that referenced this pull request Aug 18, 2026
func_ren_FogOfWar (0x81C660) counts every vision circle into the full
8-bit stencil with two-sided z-fail INCR/DECR and paints fog where the
count is 0. The count wraps mod 256, so wherever exactly 256 circles
overlap a pixel, a gray fog ring is painted over visible terrain (128
units on water suffice: vision + water vision). Same bug class as the
range-ring fix in PR FAForever#149, one bit wider.

Cap the batch loop at 127 volumes (imm32 patches at 0x81C9DF/0x81C9E6)
and run a fullscreen VisionMask pass between batches that converts the
7-bit counts to the sticky stencil bit-7 mark, mirroring the proven
RenderRingsFlush scheme. Marked pixels are skipped by later batches
(stencil early-out). No flush is needed after the final batch - the
Vision technique reads any nonzero stencil as seen - so with 127 or
fewer circles (single batch) the patched loop does no extra work.

The engine sets the stream-0 instancing frequency once with the TOTAL
circle count; with capped batches every draw would read past the
current batch's instance data. The first batch is clamped via a detour
at 0x81C95F, later batches re-issue SetVertexBuffer per batch in the
flush stub (this also fixes the latent engine bug above 12288 circles,
where multi-batch draws used the total count). The stub restores the
vertex declaration and index buffer clobbered by CRenFrame::Render.

Requires the paired shader changes in the fa repo: vision.fx CastVision
counts in stencil bits 0-6 and skips marked pixels; frame.fx gains the
VisionMask technique. With old shaders the binary patch degrades
gracefully to vanilla behavior (missing technique renders nothing, full
8-bit counting is unaffected by batching).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M3RT1N99 added a commit to M3RT1N99/FA-Binary-Patches that referenced this pull request Aug 19, 2026
The sampling test checked 24 points on the candidate's outer circle
only. Boundary sampling cannot certify area coverage: a raster-simulated
sweep over 23 layout classes found three failure classes - center holes
(a big ring dropped because a necklace of small keepers covers its rim
but not its center, up to 6.4% of the union area lost), interstitial
holes in ordinary hex-packed formations, and rim scallops from the
15-degree spacing (the maintainer's glitch screenshot). Denser sampling
does not fix this: 48+24+13 samples combined still leak in 8 of 23
layouts.

The replacement is the exact redundancy criterion behind the
Huang-Tseng perimeter-coverage theorem (WSNA'03): a hole in the kept
union is bounded by arcs of kept boundary circles, so a candidate band
is covered iff (1) its outer circle, (2) its inner circle, and (3) every
kept boundary arc strictly inside its band are covered by the union of
the (other) kept bands. All three reduce to angular-interval arithmetic
per keeper - acos of two cosine bounds around atan2 of the center
direction.

Implementation notes:
- coverage works by subtracting keeper intervals from the remaining
  target with early exit (no sort/merge), keeper intervals expanded and
  targets shrunk by an angular epsilon so float32 boundary cases
  (identical stacked rings) stay closed
- polynomial atan2/acos (~1e-5 rad, an order below the epsilon) instead
  of libm/x87, since the build is -nostdlib and fpatan dominated the
  profile; sqrtf maps to the engine CRT
- squared-distance pre-rejects before any sqrt/atan per keeper pair
- 8-point rim pre-test as a keep-only shortcut (sound: an uncovered
  sample proves non-coverage) to cheapen the keep path
- single-keeper containment fast path catches stacked units outright
- MAX_RINGS raised to 1024 -> 4096 (engine has no ring limit, verified:
  both loops chunk any N through the 1000-entry dynamic VB)

Validation: a host build of this exact file (mingw32 -m32) was run
against the Python reference over all 23 layout instances: kept sets
match the double-precision exact reference in every instance (4146 of
8185 rings kept, 49.3% culled - the sampling test culled 50.4%), and
raster ground truth shows 0 error pixels everywhere. Cull runtime on
host: 600-ring dense blob 3.5-8 ms (81% culled), necklaces <= 0.11 ms,
sparse 600 rings 29 us, 200-stack 3 us.

The cull remains opt-in and composes with the 127-batch stencil fix
(PR FAForever#149), which stays necessary: kept sets can still exceed 127
overlaps (r8-necklace keeps 193 mutually overlapping rings).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix pull request fixes an issue or a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants