-
Notifications
You must be signed in to change notification settings - Fork 1.6k
RFC: #[export_visibility = ...] attribute
#3834
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
a79823e to
ab37907
Compare
|
|
||
| ## Benefit: Smaller binaries | ||
|
|
||
| One undesirable consequence of unnecessary public exports is binary size bloat. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should only be the case for libraries. For binaries all functions are already made not-exported.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, though on (weird/stunt) occasions there might be reasons to have public symbols in a binary.
(That case is not common and not important, just mentioning it in case someone figured it was categorically impossible and never happened.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think @joshtriplett's comment implies that there actually might exist a use case for #[export_visibility = "public"] (or interposable) on binary crates. This should be noted in the RFC. We can still choose not to support it initially.
text/0000-export-visibility.md
Outdated
| (when the freeing allocator expects that the pointer it got was earlier | ||
| allocated by the same allocator instance). | ||
|
|
||
| This is what happened in https://crbug.com/418073233. In the smaller repro |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would have expected all of Chromium to use a single rust allocator rather than use a different one for each DSO. Why is that not the case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would have expected all of Chromium to use a single rust allocator rather than use a different one for each DSO. Why is that not the case?
Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects? I would expect in such a case that which Rust allocator / standard library / etc is used would be an internal implementation detail of foo.so. IIUC this detail leaks out only because of an unintentional export of a cxx-generated, internal symbol.
But to try to answer the question - the same allocator is statically linked into Chromium binaries. This means that an executable and an .so may end up with a separate copy of the same global data structures of the allocator.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects?
It is not a requirement. I'm just surprised that Chromium copies the entire rust standard library between the dylib and executable rather than using the copy from the dylib in the executable to save space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that really a requirement if foo.so doesn't export any functions that return pointers to Rust-related objects?
It is not a requirement. I'm just surprised that Chromium copies the entire rust standard library between the dylib and executable rather than using the copy from the dylib in the executable to save space.
That is indeed a bit unfortunate. I think this is to some extent based on the following:
- Chromium requirement to use an external linker
- Assumption that only
rlibs /static_libs can be linked by an external linker, and that an external linker wouldn't be able to handledylibs
But thank you for bringing this up - maybe this should indeed be treated as an alternative fix for https://crbug.com/418073233. I am not sure what the next steps should be for this aspect:
- Maybe I should open a bug (either under https://github.com/rust-lang/rust/issues, or under https://crbug.com) to move this discussion elsewhere?
- Maybe before opening a bug I should first learn more about
dylibs...
|
For most use cases rather than specifying the exact symbol visibility (which may not even be supported by the object file format, like interposable on pe/coff or (with the default two-level namespaces) mach-o) I think having just a way to force SymbolExportLevel::Rust rather than the default SymbolExportLevel::C would be a better idea. This causes it to still be exported from rust dylibs (as necessary to avoid linker errors depending on the exactly when rustc decides to codegen functions), but prevents it from being exported from cdylibs. It doesn't work for staticlibs currently, but for those if you want to limit symbol visibility you have to specify your own version script during linking anyway to prevent exporting all rust mangled symbols too. |
The fact that you are distinguishing between
That's not 100% correct - instead of using a version script, one may also use |
The Chromium case is effectively equivalent to using staticlibs, not to using rust dylibs/cdylibs.
That doesn't apply to the standard library unless you go out of your way using unstable features to recompile the standard library. |
Ack / agreed.
Thank you for bringing up this point. This probably should be explicitly addressed by the RFC (*), but I am not sure if I agree with your conclusions so far. This is because:
(*) I am not sure what the right process is here. Should I add commits to the RFC as we keep discussing here? Should I first give people an opportunity to review the first draft? |
|
I think this is a good opportunity to expand the design space (and documentation) of "various levels of exportendess" a bit, even if the resulting proposal for There are multiple attributes that targets this similar space ( What I'd like to see is a table of "levels of exportedness" combined with the kinds of end artifacts, and how we can users can express all those levels with the attributes listed above.
In particular, one of my requirements is that |
This is something only rustc must be allowed to do (other than for symbols defined in inline asm called from within the same inline asm block). Only rustc knows if all callers will end up in the same object file as the definition and it doesn't provide any guarantees around when this happens. So exposing this to the user is a stability hazard.
For regular functions and
For rlib this doesn't make sense. There is no way to make rlibs a symbol export boundary without introducing an expensive link/object file rewrite step for each individual rlib. For staticlib it would be nice to have a symbol export boundary, but unfortunately we don't have one right now even for
This makes sense to me. See the end of my comment.
This has to always be the case if it is visible outside of the object file. The very point of rust dylibs is that rust code in a separate DSO can call any public function, which thanks to cross-crate inlining can call effectively every function that rustc wouldn't make private to the current object file. And again, rustc doesn't provide any guarantees when this happens, so allowing you to not export symbols from a rust dylib is a stability hazard.
Yes.
No
Not really aside from the visibility information we already tell the linker (export from rust dylib, don't export from cdylib). Currently rustc internally works with three different symbol export levels:
It makes sense to me to allow |
I think this probably should be captured somehow as one of the alternatives in the RFC. Is there a specific syntax that you have in mind here? I guess one option would be to have a |
👍
I don't think this is a good name as it is still meant to be usable from C, just not outside of the linked DSO.
This would be an option, although ideally if we manage to stop exporting all symbols from staticlibs, I would like the same attribute to be usable to prevent export from both cdylib and staticlib, so it should probably not mention cdylib in the name. I don't have suggestions for a better name though. |
|
Frankly, if we have |
|
In Rust 1.87 and before on GNU/Linux, this code: #![feature(rustc_attrs)]
#[rustc_std_internal_symbol]
#[no_mangle]
pub extern "C" fn blah(i: u32) -> u32 {
i+1
}produced an object with the What this issue is not applicable to:
Omitting the In C++, mangling (controlled by The analog to what This errors now in 1.88, the only way i'm aware of being injecting an assembly declaration |
|
@bjorn3 - thank you for proposing a narrower fix, focusing on setting
From my perspective both
|
|
I think the current RFC does a good job of creating a conceptual framework for symbol visibility that can abstract over platform differences, and the remaining issues seem like they can be addressed. Perhaps the issue with dylibs can be solved by linting on calls to non-inlined, hidden-linkage functions from inlined or generic functions. My feeling is that having two export levels, "Rust" and "C", isn't quite the right abstraction – or if it is, we should be able to spell it as I love the idea @petrochenkov raised in #3834 (comment) of having a table of all the things you might want to accomplish and how you would spell them. I wouldn't put the onus on this RFC to completely fill out that table, but it would be helpful to know which gaps exist and which are getting filled. It's also important to me that the users who really know what they're doing when it comes to symbol visibility should be able to exercise direct control. While those users are a small percentage of Rust users, I do see this as a significant benefit to them and to unlocking Rust usage in more arcane contexts. That control can be mediated by a table that maps from "how it's spelled in C compilers and linker scripts" to "how to spell it in Rust"; it doesn't require us to adopt the existing models or terminology wholesale. There are places we might reasonably want to place limits on this (as brought up by @bjorn3 above, setting visibility to an individual codegen unit is almost certainly a bad idea). Otherwise I think we should have some path that favors flexibility and transparency, and avoid forcing outside experts who come to Rust to guess at complex logic the compiler might be doing and why. |
|
@rfcbot fcp merge |
|
Team member @tmandry has proposed to merge this. The next step is review by the rest of the tagged team members: Concerns:
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns. |
#[export_visibility = ...] attribute.#[export_visibility = ...] attribute
|
@tmandry, can you please clarify the concern below?
IIUC the RFC does not add any new kinds of visibility to If you think that delegating the responsibility to LLVM is not okay, then can you please help me understand what kind of test results or what kind of data you'd like to see added to the RFC? |
that's incorrect https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=acebbccd2aeb3384a353161e9be5f801 |
|
I would note that generics being Also, the point about generics is that a generic function may call an |
text/0000-export-visibility.md
Outdated
| OTOH, ideally we would somehow check what happens on some representative subset | ||
| of target platforms (maybe: Posix, Windows, Wasm?): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I'm aware, there is no Windows equivalent of the "protected" visibility class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't COFF have protected as normal visibility class, with no equivalent for ELF default visibility? And same for Mach-O. ELF default visibility allows another dylib to overwrite exported functions of the local dylib for calls from within the local dylib.
The Rust language does not assume LLVM, and rustc supports multiple backends. Saying that these map to LLVM definitions is not enough on its own. Those definitions might have meaning or not on various platforms and in various combinations. The way I see it, there are two directions this RFC could go:
1 is my preferred outcome. The goals of cross-platform and intuitive are in tension, however, and not necessarily related to the way that LLVM surfaces its options. Looking over the LLVM and GCC docs, some of the visibility options strike me as leaky abstractions with a list of platform-specific caveats. We can simplify the problem by restricting scope. As you suggest, it's valid to cut out "hidden" and focus on a single "protected" option, with more options as future possibilities. I would like to know we have a plausible path to stabilizing future options, i.e. we'll need to discuss some tradeoffs but there's at least one option that looks usable. With my current understanding I think "hidden" meets this bar. Another future possibility is that we stabilize both "abstract" and "platform-specific" options under the same attribute. To clarify something, am I correct in understanding that your particular use case is solved with either "hidden" or "protected"? |
My usecase (i.e. fixing https://crbug.com/418073233) can be addressed by either:
I think that my usecase would not be addressed by |
Rationale: * This question is no longer applicable after only supporting the `"inherit"` visibility. * Additionally, this future question will be proactively explored for `"hidden"` visibility - results will be shared through a comment on the RFC PR at rust-lang#3834
@tmandry, I have taken the following steps to address the concern above:
Would you be able to please:
REPRO STEPS (Linux host and target):
|
|
We're talking about this in the lang meeting. Question -- and maybe it's in here -- is there ever a case where one wants to use this without |
|
@rfcbot resolve due diligence on tier 1 platforms With only "target_default" option in this RFC now, there's no question of how how that gets expressed on multiple platforms.
Testing is not required for the RFC; the question was whether the RFC defines semantics that make sense on all platforms. Including only "target_default" sidesteps the question. And since "hidden" and "interposable" have been removed from the RFC, these concerns are no longer relevant: @rfcbot resolve specify how inlining interacts with hidden symbols |
Not really - |
| then it will instead use the default visibility of the target platform | ||
| (which can be overriden with the | ||
| [`-Zdefault-visibility=...`](https://doc.rust-lang.org/beta/unstable-book/compiler-flags/default-visibility.html) | ||
| command-line flag). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean that -Zdefault-visibility=hidden + #[export_visibility = "target_default"] on ELF would result in default visibility? If I understand that correctly, that makes this attribute useless for projects that let rustc do the linking and don't use -Zdefault-visibility as in that case there is still no way to restrict the visibility of #[no_mangle] functions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean that -Zdefault-visibility=hidden + #[export_visibility = "target_default"] on ELF will result in hidden visibility. I've added some examples to the RFC - hopefully that helps?
I think the confusion stems from the fact that "the default visibility of the target platform" can be interpreted as either one of:
- The
defaultELF visibility:- https://llvm.org/docs/LangRef.html#visibility-styles says "default"
man elfalso says "default"
- The default visibility used by
rustc-Zdefault-visibilitymeans this kind of "default"rustc_target::spec::TargetOptions::default_visibilityalso means this kind of "default"rustc_session::Session::default_visibilityalso means this kind of "default"
Does the example help here? Or do you think we should change the names proposed by the RFC? e.g. maybe go back to "inherit"? Or come up with some other synonym for "default" like maybe "baseline"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the code above is built into a DSO, then -Zdefault-visiblity=hidden will affect visibility of the 2nd function and prevent it from getting exported from the DSO. See below for an example of how this may be observed on a Linux system:
I would have expected -Zdefault-visiblity=hidden to mark all functions as hidden unless explicitly overwritten. But that is indeed not the current behavior: https://github.com/rust-lang/rust/blob/779e19d8baa3e3625bd4fc5c85cbb2ad47b43155/compiler/rustc_monomorphize/src/partitioning.rs#L931-L933 I guess that explains my confusion about #[export_visibility = "target_default"].
Might it make sense syntactically, then, to be coupled with those? E.g., |
|
I think I would lean toward keeping
I am not sure if
I note that having a separate attribute may be desirable for extra flexibility. For example, it allows for separate, independent control of visibility by FWIW the initial prototype for this RFC only allows PS. I am not sure if there are other similar attributes to take inspiration from. In particular, I am not sure if the |
|
|
The interaction with Setting aside the syntax, would we specify (It'd be good for the RFC to speak to this if it doesn't already.) |
I would expect this to be an error. Rationale: it's easier to relax this restriction in the future (if needed) than doing things the other way round (starting by allowing it everywhere and then risk breaking changes to add restrictions). And the prototype does treat this as an error:
FWIW the current prototype checks
Done - see 68a5cef |
This RFC proposes to add
#[export_visibility = …]attribute, which seems like a reasonable way to address the following issues:#[no_mangle]symbols are exported from acdylibrust#98449This RFC complements the
-Zdefault-visibility=...command-line flag, which is tracked in rust-lang/rust#131090This PR replaces the Major Change Proposal (MCP) at rust-lang/compiler-team#881
(/cc @bjorn3, @ChrisDenton, @chorman0773, @joshtriplett, @mati865, @workingjubilee, and @Urgau who have kindly provided feedback in the Zulip thread associated with that MCP)
/cc @tmandry from rust-lang/rust-project-goals#253, because one area where this RFC seems needed is FFI tooling
Rendered