1//! Manually manage memory through raw pointers.
2//!
3//! *[See also the pointer primitive types](pointer).*
4//!
5//! # Safety
6//!
7//! Many functions in this module take raw pointers as arguments and read from
8//! or write to them. For this to be safe, these pointers must be *valid*.
9//! Whether a pointer is valid depends on the operation it is used for
10//! (read or write), and the extent of the memory that is accessed (i.e.,
11//! how many bytes are read/written). Most functions use `*mut T` and `*const T`
12//! to access only a single value, in which case the documentation omits the size
13//! and implicitly assumes it to be `size_of::<T>()` bytes.
14//!
15//! The precise rules for validity are not determined yet. The guarantees that are
16//! provided at this point are very minimal:
17//!
18//! * A [null] pointer is *never* valid, not even for accesses of [size zero][zst].
19//! * For a pointer to be valid, it is necessary, but not always sufficient, that the pointer
20//! be *dereferenceable*: the memory range of the given size starting at the pointer must all be
21//! within the bounds of a single allocated object. Note that in Rust,
22//! every (stack-allocated) variable is considered a separate allocated object.
23//! * Even for operations of [size zero][zst], the pointer must not be pointing to deallocated
24//! memory, i.e., deallocation makes pointers invalid even for zero-sized operations. However,
25//! casting any non-zero integer *literal* to a pointer is valid for zero-sized accesses, even if
26//! some memory happens to exist at that address and gets deallocated. This corresponds to writing
27//! your own allocator: allocating zero-sized objects is not very hard. The canonical way to
28//! obtain a pointer that is valid for zero-sized accesses is [`NonNull::dangling`].
29//FIXME: mention `ptr::invalid` above, once it is stable.
30//! * All accesses performed by functions in this module are *non-atomic* in the sense
31//! of [atomic operations] used to synchronize between threads. This means it is
32//! undefined behavior to perform two concurrent accesses to the same location from different
33//! threads unless both accesses only read from memory. Notice that this explicitly
34//! includes [`read_volatile`] and [`write_volatile`]: Volatile accesses cannot
35//! be used for inter-thread synchronization.
36//! * The result of casting a reference to a pointer is valid for as long as the
37//! underlying object is live and no reference (just raw pointers) is used to
38//! access the same memory. That is, reference and pointer accesses cannot be
39//! interleaved.
40//!
41//! These axioms, along with careful use of [`offset`] for pointer arithmetic,
42//! are enough to correctly implement many useful things in unsafe code. Stronger guarantees
43//! will be provided eventually, as the [aliasing] rules are being determined. For more
44//! information, see the [book] as well as the section in the reference devoted
45//! to [undefined behavior][ub].
46//!
47//! ## Alignment
48//!
49//! Valid raw pointers as defined above are not necessarily properly aligned (where
50//! "proper" alignment is defined by the pointee type, i.e., `*const T` must be
51//! aligned to `mem::align_of::<T>()`). However, most functions require their
52//! arguments to be properly aligned, and will explicitly state
53//! this requirement in their documentation. Notable exceptions to this are
54//! [`read_unaligned`] and [`write_unaligned`].
55//!
56//! When a function requires proper alignment, it does so even if the access
57//! has size 0, i.e., even if memory is not actually touched. Consider using
58//! [`NonNull::dangling`] in such cases.
59//!
60//! ## Allocated object
61//!
62//! For several operations, such as [`offset`] or field projections (`expr.field`), the notion of an
63//! "allocated object" becomes relevant. An allocated object is a contiguous region of memory.
64//! Common examples of allocated objects include stack-allocated variables (each variable is a
65//! separate allocated object), heap allocations (each allocation created by the global allocator is
66//! a separate allocated object), and `static` variables.
67//!
68//! # Strict Provenance
69//!
70//! **The following text is non-normative, insufficiently formal, and is an extremely strict
71//! interpretation of provenance. It's ok if your code doesn't strictly conform to it.**
72//!
73//! [Strict Provenance][] is an experimental set of APIs that help tools that try
74//! to validate the memory-safety of your program's execution. Notably this includes [Miri][]
75//! and [CHERI][], which can detect when you access out of bounds memory or otherwise violate
76//! Rust's memory model.
77//!
78//! Provenance must exist in some form for any programming
79//! language compiled for modern computer architectures, but specifying a model for provenance
80//! in a way that is useful to both compilers and programmers is an ongoing challenge.
81//! The [Strict Provenance][] experiment seeks to explore the question: *what if we just said you
82//! couldn't do all the nasty operations that make provenance so messy?*
83//!
84//! What APIs would have to be removed? What APIs would have to be added? How much would code
85//! have to change, and is it worse or better now? Would any patterns become truly inexpressible?
86//! Could we carve out special exceptions for those patterns? Should we?
87//!
88//! A secondary goal of this project is to see if we can disambiguate the many functions of
89//! pointer<->integer casts enough for the definition of `usize` to be loosened so that it
90//! isn't *pointer*-sized but address-space/offset/allocation-sized (we'll probably continue
91//! to conflate these notions). This would potentially make it possible to more efficiently
92//! target platforms where pointers are larger than offsets, such as CHERI and maybe some
93//! segmented architectures.
94//!
95//! ## Provenance
96//!
97//! **This section is *non-normative* and is part of the [Strict Provenance][] experiment.**
98//!
99//! Pointers are not *simply* an "integer" or "address". For instance, it's uncontroversial
100//! to say that a Use After Free is clearly Undefined Behaviour, even if you "get lucky"
101//! and the freed memory gets reallocated before your read/write (in fact this is the
102//! worst-case scenario, UAFs would be much less concerning if this didn't happen!).
103//! To rationalize this claim, pointers need to somehow be *more* than just their addresses:
104//! they must have provenance.
105//!
106//! When an allocation is created, that allocation has a unique Original Pointer. For alloc
107//! APIs this is literally the pointer the call returns, and for local variables and statics,
108//! this is the name of the variable/static. This is mildly overloading the term "pointer"
109//! for the sake of brevity/exposition.
110//!
111//! The Original Pointer for an allocation is guaranteed to have unique access to the entire
112//! allocation and *only* that allocation. In this sense, an allocation can be thought of
113//! as a "sandbox" that cannot be broken into or out of. *Provenance* is the permission
114//! to access an allocation's sandbox and has both a *spatial* and *temporal* component:
115//!
116//! * Spatial: A range of bytes that the pointer is allowed to access.
117//! * Temporal: The lifetime (of the allocation) that access to these bytes is tied to.
118//!
119//! Spatial provenance makes sure you don't go beyond your sandbox, while temporal provenance
120//! makes sure that you can't "get lucky" after your permission to access some memory
121//! has been revoked (either through deallocations or borrows expiring).
122//!
123//! Provenance is implicitly shared with all pointers transitively derived from
124//! The Original Pointer through operations like [`offset`], borrowing, and pointer casts.
125//! Some operations may *shrink* the derived provenance, limiting how much memory it can
126//! access or how long it's valid for (i.e. borrowing a subfield and subslicing).
127//!
128//! Shrinking provenance cannot be undone: even if you "know" there is a larger allocation, you
129//! can't derive a pointer with a larger provenance. Similarly, you cannot "recombine"
130//! two contiguous provenances back into one (i.e. with a `fn merge(&[T], &[T]) -> &[T]`).
131//!
132//! A reference to a value always has provenance over exactly the memory that field occupies.
133//! A reference to a slice always has provenance over exactly the range that slice describes.
134//!
135//! If an allocation is deallocated, all pointers with provenance to that allocation become
136//! invalidated, and effectively lose their provenance.
137//!
138//! The strict provenance experiment is mostly only interested in exploring stricter *spatial*
139//! provenance. In this sense it can be thought of as a subset of the more ambitious and
140//! formal [Stacked Borrows][] research project, which is what tools like [Miri][] are based on.
141//! In particular, Stacked Borrows is necessary to properly describe what borrows are allowed
142//! to do and when they become invalidated. This necessarily involves much more complex
143//! *temporal* reasoning than simply identifying allocations. Adjusting APIs and code
144//! for the strict provenance experiment will also greatly help Stacked Borrows.
145//!
146//!
147//! ## Pointer Vs Addresses
148//!
149//! **This section is *non-normative* and is part of the [Strict Provenance][] experiment.**
150//!
151//! One of the largest historical issues with trying to define provenance is that programmers
152//! freely convert between pointers and integers. Once you allow for this, it generally becomes
153//! impossible to accurately track and preserve provenance information, and you need to appeal
154//! to very complex and unreliable heuristics. But of course, converting between pointers and
155//! integers is very useful, so what can we do?
156//!
157//! Also did you know WASM is actually a "Harvard Architecture"? As in function pointers are
158//! handled completely differently from data pointers? And we kind of just shipped Rust on WASM
159//! without really addressing the fact that we let you freely convert between function pointers
160//! and data pointers, because it mostly Just Works? Let's just put that on the "pointer casts
161//! are dubious" pile.
162//!
163//! Strict Provenance attempts to square these circles by decoupling Rust's traditional conflation
164//! of pointers and `usize` (and `isize`), and defining a pointer to semantically contain the
165//! following information:
166//!
167//! * The **address-space** it is part of (e.g. "data" vs "code" in WASM).
168//! * The **address** it points to, which can be represented by a `usize`.
169//! * The **provenance** it has, defining the memory it has permission to access.
170//!
171//! Under Strict Provenance, a usize *cannot* accurately represent a pointer, and converting from
172//! a pointer to a usize is generally an operation which *only* extracts the address. It is
173//! therefore *impossible* to construct a valid pointer from a usize because there is no way
174//! to restore the address-space and provenance. In other words, pointer-integer-pointer
175//! roundtrips are not possible (in the sense that the resulting pointer is not dereferenceable).
176//!
177//! The key insight to making this model *at all* viable is the [`with_addr`][] method:
178//!
179//! ```text
180//! /// Creates a new pointer with the given address.
181//! ///
182//! /// This performs the same operation as an `addr as ptr` cast, but copies
183//! /// the *address-space* and *provenance* of `self` to the new pointer.
184//! /// This allows us to dynamically preserve and propagate this important
185//! /// information in a way that is otherwise impossible with a unary cast.
186//! ///
187//! /// This is equivalent to using `wrapping_offset` to offset `self` to the
188//! /// given address, and therefore has all the same capabilities and restrictions.
189//! pub fn with_addr(self, addr: usize) -> Self;
190//! ```
191//!
192//! So you're still able to drop down to the address representation and do whatever
193//! clever bit tricks you want *as long as* you're able to keep around a pointer
194//! into the allocation you care about that can "reconstitute" the other parts of the pointer.
195//! Usually this is very easy, because you only are taking a pointer, messing with the address,
196//! and then immediately converting back to a pointer. To make this use case more ergonomic,
197//! we provide the [`map_addr`][] method.
198//!
199//! To help make it clear that code is "following" Strict Provenance semantics, we also provide an
200//! [`addr`][] method which promises that the returned address is not part of a
201//! pointer-usize-pointer roundtrip. In the future we may provide a lint for pointer<->integer
202//! casts to help you audit if your code conforms to strict provenance.
203//!
204//!
205//! ## Using Strict Provenance
206//!
207//! Most code needs no changes to conform to strict provenance, as the only really concerning
208//! operation that *wasn't* obviously already Undefined Behaviour is casts from usize to a
209//! pointer. For code which *does* cast a usize to a pointer, the scope of the change depends
210//! on exactly what you're doing.
211//!
212//! In general you just need to make sure that if you want to convert a usize address to a
213//! pointer and then use that pointer to read/write memory, you need to keep around a pointer
214//! that has sufficient provenance to perform that read/write itself. In this way all of your
215//! casts from an address to a pointer are essentially just applying offsets/indexing.
216//!
217//! This is generally trivial to do for simple cases like tagged pointers *as long as you
218//! represent the tagged pointer as an actual pointer and not a usize*. For instance:
219//!
220//! ```
221//! #![feature(strict_provenance)]
222//!
223//! unsafe {
224//! // A flag we want to pack into our pointer
225//! static HAS_DATA: usize = 0x1;
226//! static FLAG_MASK: usize = !HAS_DATA;
227//!
228//! // Our value, which must have enough alignment to have spare least-significant-bits.
229//! let my_precious_data: u32 = 17;
230//! assert!(core::mem::align_of::<u32>() > 1);
231//!
232//! // Create a tagged pointer
233//! let ptr = &my_precious_data as *const u32;
234//! let tagged = ptr.map_addr(|addr| addr | HAS_DATA);
235//!
236//! // Check the flag:
237//! if tagged.addr() & HAS_DATA != 0 {
238//! // Untag and read the pointer
239//! let data = *tagged.map_addr(|addr| addr & FLAG_MASK);
240//! assert_eq!(data, 17);
241//! } else {
242//! unreachable!()
243//! }
244//! }
245//! ```
246//!
247//! (Yes, if you've been using AtomicUsize for pointers in concurrent datastructures, you should
248//! be using AtomicPtr instead. If that messes up the way you atomically manipulate pointers,
249//! we would like to know why, and what needs to be done to fix it.)
250//!
251//! Something more complicated and just generally *evil* like an XOR-List requires more significant
252//! changes like allocating all nodes in a pre-allocated Vec or Arena and using a pointer
253//! to the whole allocation to reconstitute the XORed addresses.
254//!
255//! Situations where a valid pointer *must* be created from just an address, such as baremetal code
256//! accessing a memory-mapped interface at a fixed address, are an open question on how to support.
257//! These situations *will* still be allowed, but we might require some kind of "I know what I'm
258//! doing" annotation to explain the situation to the compiler. It's also possible they need no
259//! special attention at all, because they're generally accessing memory outside the scope of
260//! "the abstract machine", or already using "I know what I'm doing" annotations like "volatile".
261//!
262//! Under [Strict Provenance] it is Undefined Behaviour to:
263//!
264//! * Access memory through a pointer that does not have provenance over that memory.
265//!
266//! * [`offset`] a pointer to or from an address it doesn't have provenance over.
267//! This means it's always UB to offset a pointer derived from something deallocated,
268//! even if the offset is 0. Note that a pointer "one past the end" of its provenance
269//! is not actually outside its provenance, it just has 0 bytes it can load/store.
270//!
271//! But it *is* still sound to:
272//!
273//! * Create an invalid pointer from just an address (see [`ptr::invalid`][]). This can
274//! be used for sentinel values like `null` *or* to represent a tagged pointer that will
275//! never be dereferenceable. In general, it is always sound for an integer to pretend
276//! to be a pointer "for fun" as long as you don't use operations on it which require
277//! it to be valid (offset, read, write, etc).
278//!
279//! * Forge an allocation of size zero at any sufficiently aligned non-null address.
280//! i.e. the usual "ZSTs are fake, do what you want" rules apply *but* this only applies
281//! for actual forgery (integers cast to pointers). If you borrow some struct's field
282//! that *happens* to be zero-sized, the resulting pointer will have provenance tied to
283//! that allocation and it will still get invalidated if the allocation gets deallocated.
284//! In the future we may introduce an API to make such a forged allocation explicit.
285//!
286//! * [`wrapping_offset`][] a pointer outside its provenance. This includes invalid pointers
287//! which have "no" provenance. Unfortunately there may be practical limits on this for a
288//! particular platform, and it's an open question as to how to specify this (if at all).
289//! Notably, [CHERI][] relies on a compression scheme that can't handle a
290//! pointer getting offset "too far" out of bounds. If this happens, the address
291//! returned by `addr` will be the value you expect, but the provenance will get invalidated
292//! and using it to read/write will fault. The details of this are architecture-specific
293//! and based on alignment, but the buffer on either side of the pointer's range is pretty
294//! generous (think kilobytes, not bytes).
295//!
296//! * Compare arbitrary pointers by address. Addresses *are* just integers and so there is
297//! always a coherent answer, even if the pointers are invalid or from different
298//! address-spaces/provenances. Of course, comparing addresses from different address-spaces
299//! is generally going to be *meaningless*, but so is comparing Kilograms to Meters, and Rust
300//! doesn't prevent that either. Similarly, if you get "lucky" and notice that a pointer
301//! one-past-the-end is the "same" address as the start of an unrelated allocation, anything
302//! you do with that fact is *probably* going to be gibberish. The scope of that gibberish
303//! is kept under control by the fact that the two pointers *still* aren't allowed to access
304//! the other's allocation (bytes), because they still have different provenance.
305//!
306//! * Perform pointer tagging tricks. This falls out of [`wrapping_offset`] but is worth
307//! mentioning in more detail because of the limitations of [CHERI][]. Low-bit tagging
308//! is very robust, and often doesn't even go out of bounds because types ensure
309//! size >= align (and over-aligning actually gives CHERI more flexibility). Anything
310//! more complex than this rapidly enters "extremely platform-specific" territory as
311//! certain things may or may not be allowed based on specific supported operations.
312//! For instance, ARM explicitly supports high-bit tagging, and so CHERI on ARM inherits
313//! that and should support it.
314//!
315//! ## Exposed Provenance
316//!
317//! **This section is *non-normative* and is an extension to the [Strict Provenance] experiment.**
318//!
319//! As discussed above, pointer-usize-pointer roundtrips are not possible under [Strict Provenance].
320//! This is by design: the goal of Strict Provenance is to provide a clear specification that we are
321//! confident can be formalized unambiguously and can be subject to precise formal reasoning.
322//!
323//! However, there exist situations where pointer-usize-pointer roundtrips cannot be avoided, or
324//! where avoiding them would require major refactoring. Legacy platform APIs also regularly assume
325//! that `usize` can capture all the information that makes up a pointer. The goal of Strict
326//! Provenance is not to rule out such code; the goal is to put all the *other* pointer-manipulating
327//! code onto a more solid foundation. Strict Provenance is about improving the situation where
328//! possible (all the code that can be written with Strict Provenance) without making things worse
329//! for situations where Strict Provenance is insufficient.
330//!
331//! For these situations, there is a highly experimental extension to Strict Provenance called
332//! *Exposed Provenance*. This extension permits pointer-usize-pointer roundtrips. However, its
333//! semantics are on much less solid footing than Strict Provenance, and at this point it is not yet
334//! clear where a satisfying unambiguous semantics can be defined for Exposed Provenance.
335//! Furthermore, Exposed Provenance will not work (well) with tools like [Miri] and [CHERI].
336//!
337//! Exposed Provenance is provided by the [`expose_addr`] and [`from_exposed_addr`] methods, which
338//! are meant to replace `as` casts between pointers and integers. [`expose_addr`] is a lot like
339//! [`addr`], but additionally adds the provenance of the pointer to a global list of 'exposed'
340//! provenances. (This list is purely conceptual, it exists for the purpose of specifying Rust but
341//! is not materialized in actual executions, except in tools like [Miri].) [`from_exposed_addr`]
342//! can be used to construct a pointer with one of these previously 'exposed' provenances.
343//! [`from_exposed_addr`] takes only `addr: usize` as arguments, so unlike in [`with_addr`] there is
344//! no indication of what the correct provenance for the returned pointer is -- and that is exactly
345//! what makes pointer-usize-pointer roundtrips so tricky to rigorously specify! There is no
346//! algorithm that decides which provenance will be used. You can think of this as "guessing" the
347//! right provenance, and the guess will be "maximally in your favor", in the sense that if there is
348//! any way to avoid undefined behavior, then that is the guess that will be taken. However, if
349//! there is *no* previously 'exposed' provenance that justifies the way the returned pointer will
350//! be used, the program has undefined behavior.
351//!
352//! Using [`expose_addr`] or [`from_exposed_addr`] (or the `as` casts) means that code is
353//! *not* following Strict Provenance rules. The goal of the Strict Provenance experiment is to
354//! determine how far one can get in Rust without the use of [`expose_addr`] and
355//! [`from_exposed_addr`], and to encourage code to be written with Strict Provenance APIs only.
356//! Maximizing the amount of such code is a major win for avoiding specification complexity and to
357//! facilitate adoption of tools like [CHERI] and [Miri] that can be a big help in increasing the
358//! confidence in (unsafe) Rust code.
359//!
360//! [aliasing]: ../../nomicon/aliasing.html
361//! [book]: ../../book/ch19-01-unsafe-rust.html#dereferencing-a-raw-pointer
362//! [ub]: ../../reference/behavior-considered-undefined.html
363//! [zst]: ../../nomicon/exotic-sizes.html#zero-sized-types-zsts
364//! [atomic operations]: crate::sync::atomic
365//! [`offset`]: pointer::offset
366//! [`wrapping_offset`]: pointer::wrapping_offset
367//! [`with_addr`]: pointer::with_addr
368//! [`map_addr`]: pointer::map_addr
369//! [`addr`]: pointer::addr
370//! [`ptr::invalid`]: core::ptr::invalid
371//! [`expose_addr`]: pointer::expose_addr
372//! [`from_exposed_addr`]: from_exposed_addr
373//! [Miri]: https://github.com/rust-lang/miri
374//! [CHERI]: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
375//! [Strict Provenance]: https://github.com/rust-lang/rust/issues/95228
376//! [Stacked Borrows]: https://plv.mpi-sws.org/rustbelt/stacked-borrows/
377
378#![stable(feature = "rust1", since = "1.0.0")]
379
380use crate::cmp::Ordering;
381use crate::fmt;
382use crate::hash;
383use crate::intrinsics::{
384 self, assert_unsafe_precondition, is_aligned_and_not_null, is_nonoverlapping,
385};
386use crate::marker::FnPtr;
387
388use crate::mem::{self, MaybeUninit};
389
390mod alignment;
391#[unstable(feature = "ptr_alignment_type", issue = "102070")]
392pub use alignment::Alignment;
393
394#[stable(feature = "rust1", since = "1.0.0")]
395#[doc(inline)]
396pub use crate::intrinsics::copy_nonoverlapping;
397
398#[stable(feature = "rust1", since = "1.0.0")]
399#[doc(inline)]
400pub use crate::intrinsics::copy;
401
402#[stable(feature = "rust1", since = "1.0.0")]
403#[doc(inline)]
404pub use crate::intrinsics::write_bytes;
405
406mod metadata;
407#[unstable(feature = "ptr_metadata", issue = "81513")]
408pub use metadata::{from_raw_parts, from_raw_parts_mut, metadata, DynMetadata, Pointee, Thin};
409
410mod non_null;
411#[stable(feature = "nonnull", since = "1.25.0")]
412pub use non_null::NonNull;
413
414mod unique;
415#[unstable(feature = "ptr_internals", issue = "none")]
416pub use unique::Unique;
417
418mod const_ptr;
419mod mut_ptr;
420
421/// Executes the destructor (if any) of the pointed-to value.
422///
423/// This is semantically equivalent to calling [`ptr::read`] and discarding
424/// the result, but has the following advantages:
425///
426/// * It is *required* to use `drop_in_place` to drop unsized types like
427/// trait objects, because they can't be read out onto the stack and
428/// dropped normally.
429///
430/// * It is friendlier to the optimizer to do this over [`ptr::read`] when
431/// dropping manually allocated memory (e.g., in the implementations of
432/// `Box`/`Rc`/`Vec`), as the compiler doesn't need to prove that it's
433/// sound to elide the copy.
434///
435/// * It can be used to drop [pinned] data when `T` is not `repr(packed)`
436/// (pinned data must not be moved before it is dropped).
437///
438/// Unaligned values cannot be dropped in place, they must be copied to an aligned
439/// location first using [`ptr::read_unaligned`]. For packed structs, this move is
440/// done automatically by the compiler. This means the fields of packed structs
441/// are not dropped in-place.
442///
443/// [`ptr::read`]: self::read
444/// [`ptr::read_unaligned`]: self::read_unaligned
445/// [pinned]: crate::pin
446///
447/// # Safety
448///
449/// Behavior is undefined if any of the following conditions are violated:
450///
451/// * `to_drop` must be [valid] for both reads and writes.
452///
453/// * `to_drop` must be properly aligned, even if `T` has size 0.
454///
455/// * `to_drop` must be nonnull, even if `T` has size 0.
456///
457/// * The value `to_drop` points to must be valid for dropping, which may mean
458/// it must uphold additional invariants. These invariants depend on the type
459/// of the value being dropped. For instance, when dropping a Box, the box's
460/// pointer to the heap must be valid.
461///
462/// * While `drop_in_place` is executing, the only way to access parts of
463/// `to_drop` is through the `&mut self` references supplied to the
464/// `Drop::drop` methods that `drop_in_place` invokes.
465///
466/// Additionally, if `T` is not [`Copy`], using the pointed-to value after
467/// calling `drop_in_place` can cause undefined behavior. Note that `*to_drop =
468/// foo` counts as a use because it will cause the value to be dropped
469/// again. [`write()`] can be used to overwrite data without causing it to be
470/// dropped.
471///
472/// [valid]: self#safety
473///
474/// # Examples
475///
476/// Manually remove the last item from a vector:
477///
478/// ```
479/// use std::ptr;
480/// use std::rc::Rc;
481///
482/// let last = Rc::new(1);
483/// let weak = Rc::downgrade(&last);
484///
485/// let mut v = vec![Rc::new(0), last];
486///
487/// unsafe {
488/// // Get a raw pointer to the last element in `v`.
489/// let ptr = &mut v[1] as *mut _;
490/// // Shorten `v` to prevent the last item from being dropped. We do that first,
491/// // to prevent issues if the `drop_in_place` below panics.
492/// v.set_len(1);
493/// // Without a call `drop_in_place`, the last item would never be dropped,
494/// // and the memory it manages would be leaked.
495/// ptr::drop_in_place(ptr);
496/// }
497///
498/// assert_eq!(v, &[0.into()]);
499///
500/// // Ensure that the last item was dropped.
501/// assert!(weak.upgrade().is_none());
502/// ```
503#[stable(feature = "drop_in_place", since = "1.8.0")]
504#[lang = "drop_in_place"]
505#[allow(unconditional_recursion)]
506#[rustc_diagnostic_item = "ptr_drop_in_place"]
507pub unsafe fn drop_in_place<T: ?Sized>(to_drop: *mut T) {
508 // Code here does not matter - this is replaced by the
509 // real drop glue by the compiler.
510
511 // SAFETY: see comment above
512 unsafe { drop_in_place(to_drop) }
513}
514
515/// Creates a null raw pointer.
516///
517/// This function is equivalent to zero-initializing the pointer:
518/// `MaybeUninit::<*const T>::zeroed().assume_init()`.
519/// The resulting pointer has the address 0.
520///
521/// # Examples
522///
523/// ```
524/// use std::ptr;
525///
526/// let p: *const i32 = ptr::null();
527/// assert!(p.is_null());
528/// assert_eq!(p as usize, 0); // this pointer has the address 0
529/// ```
530#[inline(always)]
531#[must_use]
532#[stable(feature = "rust1", since = "1.0.0")]
533#[rustc_promotable]
534#[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")]
535#[rustc_allow_const_fn_unstable(ptr_metadata)]
536#[rustc_diagnostic_item = "ptr_null"]
537pub const fn null<T: ?Sized + Thin>() -> *const T {
538 from_raw_parts(data_pointer:invalid(0), ())
539}
540
541/// Creates a null mutable raw pointer.
542///
543/// This function is equivalent to zero-initializing the pointer:
544/// `MaybeUninit::<*mut T>::zeroed().assume_init()`.
545/// The resulting pointer has the address 0.
546///
547/// # Examples
548///
549/// ```
550/// use std::ptr;
551///
552/// let p: *mut i32 = ptr::null_mut();
553/// assert!(p.is_null());
554/// assert_eq!(p as usize, 0); // this pointer has the address 0
555/// ```
556#[inline(always)]
557#[must_use]
558#[stable(feature = "rust1", since = "1.0.0")]
559#[rustc_promotable]
560#[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")]
561#[rustc_allow_const_fn_unstable(ptr_metadata)]
562#[rustc_diagnostic_item = "ptr_null_mut"]
563pub const fn null_mut<T: ?Sized + Thin>() -> *mut T {
564 from_raw_parts_mut(data_pointer:invalid_mut(0), ())
565}
566
567/// Creates an invalid pointer with the given address.
568///
569/// This is different from `addr as *const T`, which creates a pointer that picks up a previously
570/// exposed provenance. See [`from_exposed_addr`] for more details on that operation.
571///
572/// The module's top-level documentation discusses the precise meaning of an "invalid"
573/// pointer but essentially this expresses that the pointer is not associated
574/// with any actual allocation and is little more than a usize address in disguise.
575///
576/// This pointer will have no provenance associated with it and is therefore
577/// UB to read/write/offset. This mostly exists to facilitate things
578/// like `ptr::null` and `NonNull::dangling` which make invalid pointers.
579///
580/// (Standard "Zero-Sized-Types get to cheat and lie" caveats apply, although it
581/// may be desirable to give them their own API just to make that 100% clear.)
582///
583/// This API and its claimed semantics are part of the Strict Provenance experiment,
584/// see the [module documentation][crate::ptr] for details.
585#[inline(always)]
586#[must_use]
587#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")]
588#[unstable(feature = "strict_provenance", issue = "95228")]
589pub const fn invalid<T>(addr: usize) -> *const T {
590 // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic.
591 // We use transmute rather than a cast so tools like Miri can tell that this
592 // is *not* the same as from_exposed_addr.
593 // SAFETY: every valid integer is also a valid pointer (as long as you don't dereference that
594 // pointer).
595 unsafe { mem::transmute(src:addr) }
596}
597
598/// Creates an invalid mutable pointer with the given address.
599///
600/// This is different from `addr as *mut T`, which creates a pointer that picks up a previously
601/// exposed provenance. See [`from_exposed_addr_mut`] for more details on that operation.
602///
603/// The module's top-level documentation discusses the precise meaning of an "invalid"
604/// pointer but essentially this expresses that the pointer is not associated
605/// with any actual allocation and is little more than a usize address in disguise.
606///
607/// This pointer will have no provenance associated with it and is therefore
608/// UB to read/write/offset. This mostly exists to facilitate things
609/// like `ptr::null` and `NonNull::dangling` which make invalid pointers.
610///
611/// (Standard "Zero-Sized-Types get to cheat and lie" caveats apply, although it
612/// may be desirable to give them their own API just to make that 100% clear.)
613///
614/// This API and its claimed semantics are part of the Strict Provenance experiment,
615/// see the [module documentation][crate::ptr] for details.
616#[inline(always)]
617#[must_use]
618#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")]
619#[unstable(feature = "strict_provenance", issue = "95228")]
620pub const fn invalid_mut<T>(addr: usize) -> *mut T {
621 // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic.
622 // We use transmute rather than a cast so tools like Miri can tell that this
623 // is *not* the same as from_exposed_addr.
624 // SAFETY: every valid integer is also a valid pointer (as long as you don't dereference that
625 // pointer).
626 unsafe { mem::transmute(src:addr) }
627}
628
629/// Convert an address back to a pointer, picking up a previously 'exposed' provenance.
630///
631/// This is a more rigorously specified alternative to `addr as *const T`. The provenance of the
632/// returned pointer is that of *any* pointer that was previously exposed by passing it to
633/// [`expose_addr`][pointer::expose_addr], or a `ptr as usize` cast. In addition, memory which is
634/// outside the control of the Rust abstract machine (MMIO registers, for example) is always
635/// considered to be exposed, so long as this memory is disjoint from memory that will be used by
636/// the abstract machine such as the stack, heap, and statics.
637///
638/// If there is no 'exposed' provenance that justifies the way this pointer will be used,
639/// the program has undefined behavior. In particular, the aliasing rules still apply: pointers
640/// and references that have been invalidated due to aliasing accesses cannot be used any more,
641/// even if they have been exposed!
642///
643/// Note that there is no algorithm that decides which provenance will be used. You can think of this
644/// as "guessing" the right provenance, and the guess will be "maximally in your favor", in the sense
645/// that if there is any way to avoid undefined behavior (while upholding all aliasing requirements),
646/// then that is the guess that will be taken.
647///
648/// On platforms with multiple address spaces, it is your responsibility to ensure that the
649/// address makes sense in the address space that this pointer will be used with.
650///
651/// Using this function means that code is *not* following [Strict
652/// Provenance][self#strict-provenance] rules. "Guessing" a
653/// suitable provenance complicates specification and reasoning and may not be supported by
654/// tools that help you to stay conformant with the Rust memory model, so it is recommended to
655/// use [`with_addr`][pointer::with_addr] wherever possible.
656///
657/// On most platforms this will produce a value with the same bytes as the address. Platforms
658/// which need to store additional information in a pointer may not support this operation,
659/// since it is generally not possible to actually *compute* which provenance the returned
660/// pointer has to pick up.
661///
662/// It is unclear whether this function can be given a satisfying unambiguous specification. This
663/// API and its claimed semantics are part of [Exposed Provenance][self#exposed-provenance].
664#[must_use]
665#[inline(always)]
666#[unstable(feature = "exposed_provenance", issue = "95228")]
667#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
668#[allow(fuzzy_provenance_casts)] // this *is* the explicit provenance API one should use instead
669pub fn from_exposed_addr<T>(addr: usize) -> *const T
670where
671 T: Sized,
672{
673 // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic.
674 addr as *const T
675}
676
677/// Convert an address back to a mutable pointer, picking up a previously 'exposed' provenance.
678///
679/// This is a more rigorously specified alternative to `addr as *mut T`. The provenance of the
680/// returned pointer is that of *any* pointer that was previously passed to
681/// [`expose_addr`][pointer::expose_addr] or a `ptr as usize` cast. If there is no previously
682/// 'exposed' provenance that justifies the way this pointer will be used, the program has undefined
683/// behavior. Note that there is no algorithm that decides which provenance will be used. You can
684/// think of this as "guessing" the right provenance, and the guess will be "maximally in your
685/// favor", in the sense that if there is any way to avoid undefined behavior, then that is the
686/// guess that will be taken.
687///
688/// On platforms with multiple address spaces, it is your responsibility to ensure that the
689/// address makes sense in the address space that this pointer will be used with.
690///
691/// Using this function means that code is *not* following [Strict
692/// Provenance][self#strict-provenance] rules. "Guessing" a
693/// suitable provenance complicates specification and reasoning and may not be supported by
694/// tools that help you to stay conformant with the Rust memory model, so it is recommended to
695/// use [`with_addr`][pointer::with_addr] wherever possible.
696///
697/// On most platforms this will produce a value with the same bytes as the address. Platforms
698/// which need to store additional information in a pointer may not support this operation,
699/// since it is generally not possible to actually *compute* which provenance the returned
700/// pointer has to pick up.
701///
702/// It is unclear whether this function can be given a satisfying unambiguous specification. This
703/// API and its claimed semantics are part of [Exposed Provenance][self#exposed-provenance].
704#[must_use]
705#[inline(always)]
706#[unstable(feature = "exposed_provenance", issue = "95228")]
707#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
708#[allow(fuzzy_provenance_casts)] // this *is* the explicit provenance API one should use instead
709pub fn from_exposed_addr_mut<T>(addr: usize) -> *mut T
710where
711 T: Sized,
712{
713 // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic.
714 addr as *mut T
715}
716
717/// Convert a reference to a raw pointer.
718///
719/// This is equivalent to `r as *const T`, but is a bit safer since it will never silently change
720/// type or mutability, in particular if the code is refactored.
721#[inline(always)]
722#[must_use]
723#[stable(feature = "ptr_from_ref", since = "1.76.0")]
724#[rustc_const_stable(feature = "ptr_from_ref", since = "1.76.0")]
725#[rustc_never_returns_null_ptr]
726#[rustc_diagnostic_item = "ptr_from_ref"]
727pub const fn from_ref<T: ?Sized>(r: &T) -> *const T {
728 r
729}
730
731/// Convert a mutable reference to a raw pointer.
732///
733/// This is equivalent to `r as *mut T`, but is a bit safer since it will never silently change
734/// type or mutability, in particular if the code is refactored.
735#[inline(always)]
736#[must_use]
737#[stable(feature = "ptr_from_ref", since = "1.76.0")]
738#[rustc_const_stable(feature = "ptr_from_ref", since = "1.76.0")]
739#[rustc_allow_const_fn_unstable(const_mut_refs)]
740#[rustc_never_returns_null_ptr]
741pub const fn from_mut<T: ?Sized>(r: &mut T) -> *mut T {
742 r
743}
744
745/// Forms a raw slice from a pointer and a length.
746///
747/// The `len` argument is the number of **elements**, not the number of bytes.
748///
749/// This function is safe, but actually using the return value is unsafe.
750/// See the documentation of [`slice::from_raw_parts`] for slice safety requirements.
751///
752/// [`slice::from_raw_parts`]: crate::slice::from_raw_parts
753///
754/// # Examples
755///
756/// ```rust
757/// use std::ptr;
758///
759/// // create a slice pointer when starting out with a pointer to the first element
760/// let x = [5, 6, 7];
761/// let raw_pointer = x.as_ptr();
762/// let slice = ptr::slice_from_raw_parts(raw_pointer, 3);
763/// assert_eq!(unsafe { &*slice }[2], 7);
764/// ```
765#[inline]
766#[stable(feature = "slice_from_raw_parts", since = "1.42.0")]
767#[rustc_const_stable(feature = "const_slice_from_raw_parts", since = "1.64.0")]
768#[rustc_allow_const_fn_unstable(ptr_metadata)]
769#[rustc_diagnostic_item = "ptr_slice_from_raw_parts"]
770pub const fn slice_from_raw_parts<T>(data: *const T, len: usize) -> *const [T] {
771 from_raw_parts(data_pointer:data.cast(), metadata:len)
772}
773
774/// Performs the same functionality as [`slice_from_raw_parts`], except that a
775/// raw mutable slice is returned, as opposed to a raw immutable slice.
776///
777/// See the documentation of [`slice_from_raw_parts`] for more details.
778///
779/// This function is safe, but actually using the return value is unsafe.
780/// See the documentation of [`slice::from_raw_parts_mut`] for slice safety requirements.
781///
782/// [`slice::from_raw_parts_mut`]: crate::slice::from_raw_parts_mut
783///
784/// # Examples
785///
786/// ```rust
787/// use std::ptr;
788///
789/// let x = &mut [5, 6, 7];
790/// let raw_pointer = x.as_mut_ptr();
791/// let slice = ptr::slice_from_raw_parts_mut(raw_pointer, 3);
792///
793/// unsafe {
794/// (*slice)[2] = 99; // assign a value at an index in the slice
795/// };
796///
797/// assert_eq!(unsafe { &*slice }[2], 99);
798/// ```
799#[inline]
800#[stable(feature = "slice_from_raw_parts", since = "1.42.0")]
801#[rustc_const_unstable(feature = "const_slice_from_raw_parts_mut", issue = "67456")]
802#[rustc_diagnostic_item = "ptr_slice_from_raw_parts_mut"]
803pub const fn slice_from_raw_parts_mut<T>(data: *mut T, len: usize) -> *mut [T] {
804 from_raw_parts_mut(data_pointer:data.cast(), metadata:len)
805}
806
807/// Swaps the values at two mutable locations of the same type, without
808/// deinitializing either.
809///
810/// But for the following exceptions, this function is semantically
811/// equivalent to [`mem::swap`]:
812///
813/// * It operates on raw pointers instead of references. When references are
814/// available, [`mem::swap`] should be preferred.
815///
816/// * The two pointed-to values may overlap. If the values do overlap, then the
817/// overlapping region of memory from `x` will be used. This is demonstrated
818/// in the second example below.
819///
820/// * The operation is "untyped" in the sense that data may be uninitialized or otherwise violate
821/// the requirements of `T`. The initialization state is preserved exactly.
822///
823/// # Safety
824///
825/// Behavior is undefined if any of the following conditions are violated:
826///
827/// * Both `x` and `y` must be [valid] for both reads and writes. They must remain valid even when the
828/// other pointer is written. (This means if the memory ranges overlap, the two pointers must not
829/// be subject to aliasing restrictions relative to each other.)
830///
831/// * Both `x` and `y` must be properly aligned.
832///
833/// Note that even if `T` has size `0`, the pointers must be non-null and properly aligned.
834///
835/// [valid]: self#safety
836///
837/// # Examples
838///
839/// Swapping two non-overlapping regions:
840///
841/// ```
842/// use std::ptr;
843///
844/// let mut array = [0, 1, 2, 3];
845///
846/// let (x, y) = array.split_at_mut(2);
847/// let x = x.as_mut_ptr().cast::<[u32; 2]>(); // this is `array[0..2]`
848/// let y = y.as_mut_ptr().cast::<[u32; 2]>(); // this is `array[2..4]`
849///
850/// unsafe {
851/// ptr::swap(x, y);
852/// assert_eq!([2, 3, 0, 1], array);
853/// }
854/// ```
855///
856/// Swapping two overlapping regions:
857///
858/// ```
859/// use std::ptr;
860///
861/// let mut array: [i32; 4] = [0, 1, 2, 3];
862///
863/// let array_ptr: *mut i32 = array.as_mut_ptr();
864///
865/// let x = array_ptr as *mut [i32; 3]; // this is `array[0..3]`
866/// let y = unsafe { array_ptr.add(1) } as *mut [i32; 3]; // this is `array[1..4]`
867///
868/// unsafe {
869/// ptr::swap(x, y);
870/// // The indices `1..3` of the slice overlap between `x` and `y`.
871/// // Reasonable results would be for to them be `[2, 3]`, so that indices `0..3` are
872/// // `[1, 2, 3]` (matching `y` before the `swap`); or for them to be `[0, 1]`
873/// // so that indices `1..4` are `[0, 1, 2]` (matching `x` before the `swap`).
874/// // This implementation is defined to make the latter choice.
875/// assert_eq!([1, 0, 1, 2], array);
876/// }
877/// ```
878#[inline]
879#[stable(feature = "rust1", since = "1.0.0")]
880#[rustc_const_unstable(feature = "const_swap", issue = "83163")]
881#[rustc_diagnostic_item = "ptr_swap"]
882pub const unsafe fn swap<T>(x: *mut T, y: *mut T) {
883 // Give ourselves some scratch space to work with.
884 // We do not have to worry about drops: `MaybeUninit` does nothing when dropped.
885 let mut tmp: MaybeUninit = MaybeUninit::<T>::uninit();
886
887 // Perform the swap
888 // SAFETY: the caller must guarantee that `x` and `y` are
889 // valid for writes and properly aligned. `tmp` cannot be
890 // overlapping either `x` or `y` because `tmp` was just allocated
891 // on the stack as a separate allocated object.
892 unsafe {
893 copy_nonoverlapping(src:x, dst:tmp.as_mut_ptr(), count:1);
894 copy(src:y, dst:x, count:1); // `x` and `y` may overlap
895 copy_nonoverlapping(src:tmp.as_ptr(), dst:y, count:1);
896 }
897}
898
899/// Swaps `count * size_of::<T>()` bytes between the two regions of memory
900/// beginning at `x` and `y`. The two regions must *not* overlap.
901///
902/// The operation is "untyped" in the sense that data may be uninitialized or otherwise violate the
903/// requirements of `T`. The initialization state is preserved exactly.
904///
905/// # Safety
906///
907/// Behavior is undefined if any of the following conditions are violated:
908///
909/// * Both `x` and `y` must be [valid] for both reads and writes of `count *
910/// size_of::<T>()` bytes.
911///
912/// * Both `x` and `y` must be properly aligned.
913///
914/// * The region of memory beginning at `x` with a size of `count *
915/// size_of::<T>()` bytes must *not* overlap with the region of memory
916/// beginning at `y` with the same size.
917///
918/// Note that even if the effectively copied size (`count * size_of::<T>()`) is `0`,
919/// the pointers must be non-null and properly aligned.
920///
921/// [valid]: self#safety
922///
923/// # Examples
924///
925/// Basic usage:
926///
927/// ```
928/// use std::ptr;
929///
930/// let mut x = [1, 2, 3, 4];
931/// let mut y = [7, 8, 9];
932///
933/// unsafe {
934/// ptr::swap_nonoverlapping(x.as_mut_ptr(), y.as_mut_ptr(), 2);
935/// }
936///
937/// assert_eq!(x, [7, 8, 3, 4]);
938/// assert_eq!(y, [1, 2, 9]);
939/// ```
940#[inline]
941#[stable(feature = "swap_nonoverlapping", since = "1.27.0")]
942#[rustc_const_unstable(feature = "const_swap", issue = "83163")]
943#[rustc_diagnostic_item = "ptr_swap_nonoverlapping"]
944pub const unsafe fn swap_nonoverlapping<T>(x: *mut T, y: *mut T, count: usize) {
945 #[allow(unused)]
946 macro_rules! attempt_swap_as_chunks {
947 ($ChunkTy:ty) => {
948 if mem::align_of::<T>() >= mem::align_of::<$ChunkTy>()
949 && mem::size_of::<T>() % mem::size_of::<$ChunkTy>() == 0
950 {
951 let x: *mut $ChunkTy = x.cast();
952 let y: *mut $ChunkTy = y.cast();
953 let count = count * (mem::size_of::<T>() / mem::size_of::<$ChunkTy>());
954 // SAFETY: these are the same bytes that the caller promised were
955 // ok, just typed as `MaybeUninit<ChunkTy>`s instead of as `T`s.
956 // The `if` condition above ensures that we're not violating
957 // alignment requirements, and that the division is exact so
958 // that we don't lose any bytes off the end.
959 return unsafe { swap_nonoverlapping_simple_untyped(x, y, count) };
960 }
961 };
962 }
963
964 // SAFETY: the caller must guarantee that `x` and `y` are
965 // valid for writes and properly aligned.
966 unsafe {
967 assert_unsafe_precondition!(
968 "ptr::swap_nonoverlapping requires that both pointer arguments are aligned and non-null \
969 and the specified memory ranges do not overlap",
970 [T](x: *mut T, y: *mut T, count: usize) =>
971 is_aligned_and_not_null(x)
972 && is_aligned_and_not_null(y)
973 && is_nonoverlapping(x, y, count)
974 );
975 }
976
977 // Split up the slice into small power-of-two-sized chunks that LLVM is able
978 // to vectorize (unless it's a special type with more-than-pointer alignment,
979 // because we don't want to pessimize things like slices of SIMD vectors.)
980 if mem::align_of::<T>() <= mem::size_of::<usize>()
981 && (!mem::size_of::<T>().is_power_of_two()
982 || mem::size_of::<T>() > mem::size_of::<usize>() * 2)
983 {
984 attempt_swap_as_chunks!(usize);
985 attempt_swap_as_chunks!(u8);
986 }
987
988 // SAFETY: Same preconditions as this function
989 unsafe { swap_nonoverlapping_simple_untyped(x, y, count) }
990}
991
992/// Same behaviour and safety conditions as [`swap_nonoverlapping`]
993///
994/// LLVM can vectorize this (at least it can for the power-of-two-sized types
995/// `swap_nonoverlapping` tries to use) so no need to manually SIMD it.
996#[inline]
997#[rustc_const_unstable(feature = "const_swap", issue = "83163")]
998const unsafe fn swap_nonoverlapping_simple_untyped<T>(x: *mut T, y: *mut T, count: usize) {
999 let x: *mut MaybeUninit = x.cast::<MaybeUninit<T>>();
1000 let y: *mut MaybeUninit = y.cast::<MaybeUninit<T>>();
1001 let mut i: usize = 0;
1002 while i < count {
1003 // SAFETY: By precondition, `i` is in-bounds because it's below `n`
1004 let x: &mut MaybeUninit = unsafe { &mut *x.add(count:i) };
1005 // SAFETY: By precondition, `i` is in-bounds because it's below `n`
1006 // and it's distinct from `x` since the ranges are non-overlapping
1007 let y: &mut MaybeUninit = unsafe { &mut *y.add(count:i) };
1008 mem::swap_simple::<MaybeUninit<T>>(x, y);
1009
1010 i += 1;
1011 }
1012}
1013
1014/// Moves `src` into the pointed `dst`, returning the previous `dst` value.
1015///
1016/// Neither value is dropped.
1017///
1018/// This function is semantically equivalent to [`mem::replace`] except that it
1019/// operates on raw pointers instead of references. When references are
1020/// available, [`mem::replace`] should be preferred.
1021///
1022/// # Safety
1023///
1024/// Behavior is undefined if any of the following conditions are violated:
1025///
1026/// * `dst` must be [valid] for both reads and writes.
1027///
1028/// * `dst` must be properly aligned.
1029///
1030/// * `dst` must point to a properly initialized value of type `T`.
1031///
1032/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned.
1033///
1034/// [valid]: self#safety
1035///
1036/// # Examples
1037///
1038/// ```
1039/// use std::ptr;
1040///
1041/// let mut rust = vec!['b', 'u', 's', 't'];
1042///
1043/// // `mem::replace` would have the same effect without requiring the unsafe
1044/// // block.
1045/// let b = unsafe {
1046/// ptr::replace(&mut rust[0], 'r')
1047/// };
1048///
1049/// assert_eq!(b, 'b');
1050/// assert_eq!(rust, &['r', 'u', 's', 't']);
1051/// ```
1052#[inline]
1053#[stable(feature = "rust1", since = "1.0.0")]
1054#[rustc_const_unstable(feature = "const_replace", issue = "83164")]
1055#[rustc_diagnostic_item = "ptr_replace"]
1056pub const unsafe fn replace<T>(dst: *mut T, mut src: T) -> T {
1057 // SAFETY: the caller must guarantee that `dst` is valid to be
1058 // cast to a mutable reference (valid for writes, aligned, initialized),
1059 // and cannot overlap `src` since `dst` must point to a distinct
1060 // allocated object.
1061 unsafe {
1062 assert_unsafe_precondition!(
1063 "ptr::replace requires that the pointer argument is aligned and non-null",
1064 [T](dst: *mut T) => is_aligned_and_not_null(dst)
1065 );
1066 mem::swap(&mut *dst, &mut src); // cannot overlap
1067 }
1068 src
1069}
1070
1071/// Reads the value from `src` without moving it. This leaves the
1072/// memory in `src` unchanged.
1073///
1074/// # Safety
1075///
1076/// Behavior is undefined if any of the following conditions are violated:
1077///
1078/// * `src` must be [valid] for reads.
1079///
1080/// * `src` must be properly aligned. Use [`read_unaligned`] if this is not the
1081/// case.
1082///
1083/// * `src` must point to a properly initialized value of type `T`.
1084///
1085/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned.
1086///
1087/// # Examples
1088///
1089/// Basic usage:
1090///
1091/// ```
1092/// let x = 12;
1093/// let y = &x as *const i32;
1094///
1095/// unsafe {
1096/// assert_eq!(std::ptr::read(y), 12);
1097/// }
1098/// ```
1099///
1100/// Manually implement [`mem::swap`]:
1101///
1102/// ```
1103/// use std::ptr;
1104///
1105/// fn swap<T>(a: &mut T, b: &mut T) {
1106/// unsafe {
1107/// // Create a bitwise copy of the value at `a` in `tmp`.
1108/// let tmp = ptr::read(a);
1109///
1110/// // Exiting at this point (either by explicitly returning or by
1111/// // calling a function which panics) would cause the value in `tmp` to
1112/// // be dropped while the same value is still referenced by `a`. This
1113/// // could trigger undefined behavior if `T` is not `Copy`.
1114///
1115/// // Create a bitwise copy of the value at `b` in `a`.
1116/// // This is safe because mutable references cannot alias.
1117/// ptr::copy_nonoverlapping(b, a, 1);
1118///
1119/// // As above, exiting here could trigger undefined behavior because
1120/// // the same value is referenced by `a` and `b`.
1121///
1122/// // Move `tmp` into `b`.
1123/// ptr::write(b, tmp);
1124///
1125/// // `tmp` has been moved (`write` takes ownership of its second argument),
1126/// // so nothing is dropped implicitly here.
1127/// }
1128/// }
1129///
1130/// let mut foo = "foo".to_owned();
1131/// let mut bar = "bar".to_owned();
1132///
1133/// swap(&mut foo, &mut bar);
1134///
1135/// assert_eq!(foo, "bar");
1136/// assert_eq!(bar, "foo");
1137/// ```
1138///
1139/// ## Ownership of the Returned Value
1140///
1141/// `read` creates a bitwise copy of `T`, regardless of whether `T` is [`Copy`].
1142/// If `T` is not [`Copy`], using both the returned value and the value at
1143/// `*src` can violate memory safety. Note that assigning to `*src` counts as a
1144/// use because it will attempt to drop the value at `*src`.
1145///
1146/// [`write()`] can be used to overwrite data without causing it to be dropped.
1147///
1148/// ```
1149/// use std::ptr;
1150///
1151/// let mut s = String::from("foo");
1152/// unsafe {
1153/// // `s2` now points to the same underlying memory as `s`.
1154/// let mut s2: String = ptr::read(&s);
1155///
1156/// assert_eq!(s2, "foo");
1157///
1158/// // Assigning to `s2` causes its original value to be dropped. Beyond
1159/// // this point, `s` must no longer be used, as the underlying memory has
1160/// // been freed.
1161/// s2 = String::default();
1162/// assert_eq!(s2, "");
1163///
1164/// // Assigning to `s` would cause the old value to be dropped again,
1165/// // resulting in undefined behavior.
1166/// // s = String::from("bar"); // ERROR
1167///
1168/// // `ptr::write` can be used to overwrite a value without dropping it.
1169/// ptr::write(&mut s, String::from("bar"));
1170/// }
1171///
1172/// assert_eq!(s, "bar");
1173/// ```
1174///
1175/// [valid]: self#safety
1176#[inline]
1177#[stable(feature = "rust1", since = "1.0.0")]
1178#[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")]
1179#[rustc_allow_const_fn_unstable(const_mut_refs, const_maybe_uninit_as_mut_ptr)]
1180#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1181#[rustc_diagnostic_item = "ptr_read"]
1182pub const unsafe fn read<T>(src: *const T) -> T {
1183 // It would be semantically correct to implement this via `copy_nonoverlapping`
1184 // and `MaybeUninit`, as was done before PR #109035. Calling `assume_init`
1185 // provides enough information to know that this is a typed operation.
1186
1187 // However, as of March 2023 the compiler was not capable of taking advantage
1188 // of that information. Thus the implementation here switched to an intrinsic,
1189 // which lowers to `_0 = *src` in MIR, to address a few issues:
1190 //
1191 // - Using `MaybeUninit::assume_init` after a `copy_nonoverlapping` was not
1192 // turning the untyped copy into a typed load. As such, the generated
1193 // `load` in LLVM didn't get various metadata, such as `!range` (#73258),
1194 // `!nonnull`, and `!noundef`, resulting in poorer optimization.
1195 // - Going through the extra local resulted in multiple extra copies, even
1196 // in optimized MIR. (Ignoring StorageLive/Dead, the intrinsic is one
1197 // MIR statement, while the previous implementation was eight.) LLVM
1198 // could sometimes optimize them away, but because `read` is at the core
1199 // of so many things, not having them in the first place improves what we
1200 // hand off to the backend. For example, `mem::replace::<Big>` previously
1201 // emitted 4 `alloca` and 6 `memcpy`s, but is now 1 `alloc` and 3 `memcpy`s.
1202 // - In general, this approach keeps us from getting any more bugs (like
1203 // #106369) that boil down to "`read(p)` is worse than `*p`", as this
1204 // makes them look identical to the backend (or other MIR consumers).
1205 //
1206 // Future enhancements to MIR optimizations might well allow this to return
1207 // to the previous implementation, rather than using an intrinsic.
1208
1209 // SAFETY: the caller must guarantee that `src` is valid for reads.
1210 unsafe {
1211 assert_unsafe_precondition!(
1212 "ptr::read requires that the pointer argument is aligned and non-null",
1213 [T](src: *const T) => is_aligned_and_not_null(src)
1214 );
1215 crate::intrinsics::read_via_copy(src)
1216 }
1217}
1218
1219/// Reads the value from `src` without moving it. This leaves the
1220/// memory in `src` unchanged.
1221///
1222/// Unlike [`read`], `read_unaligned` works with unaligned pointers.
1223///
1224/// # Safety
1225///
1226/// Behavior is undefined if any of the following conditions are violated:
1227///
1228/// * `src` must be [valid] for reads.
1229///
1230/// * `src` must point to a properly initialized value of type `T`.
1231///
1232/// Like [`read`], `read_unaligned` creates a bitwise copy of `T`, regardless of
1233/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the returned
1234/// value and the value at `*src` can [violate memory safety][read-ownership].
1235///
1236/// Note that even if `T` has size `0`, the pointer must be non-null.
1237///
1238/// [read-ownership]: read#ownership-of-the-returned-value
1239/// [valid]: self#safety
1240///
1241/// ## On `packed` structs
1242///
1243/// Attempting to create a raw pointer to an `unaligned` struct field with
1244/// an expression such as `&packed.unaligned as *const FieldType` creates an
1245/// intermediate unaligned reference before converting that to a raw pointer.
1246/// That this reference is temporary and immediately cast is inconsequential
1247/// as the compiler always expects references to be properly aligned.
1248/// As a result, using `&packed.unaligned as *const FieldType` causes immediate
1249/// *undefined behavior* in your program.
1250///
1251/// Instead you must use the [`ptr::addr_of!`](addr_of) macro to
1252/// create the pointer. You may use that returned pointer together with this
1253/// function.
1254///
1255/// An example of what not to do and how this relates to `read_unaligned` is:
1256///
1257/// ```
1258/// #[repr(packed, C)]
1259/// struct Packed {
1260/// _padding: u8,
1261/// unaligned: u32,
1262/// }
1263///
1264/// let packed = Packed {
1265/// _padding: 0x00,
1266/// unaligned: 0x01020304,
1267/// };
1268///
1269/// // Take the address of a 32-bit integer which is not aligned.
1270/// // In contrast to `&packed.unaligned as *const _`, this has no undefined behavior.
1271/// let unaligned = std::ptr::addr_of!(packed.unaligned);
1272///
1273/// let v = unsafe { std::ptr::read_unaligned(unaligned) };
1274/// assert_eq!(v, 0x01020304);
1275/// ```
1276///
1277/// Accessing unaligned fields directly with e.g. `packed.unaligned` is safe however.
1278///
1279/// # Examples
1280///
1281/// Read a usize value from a byte buffer:
1282///
1283/// ```
1284/// use std::mem;
1285///
1286/// fn read_usize(x: &[u8]) -> usize {
1287/// assert!(x.len() >= mem::size_of::<usize>());
1288///
1289/// let ptr = x.as_ptr() as *const usize;
1290///
1291/// unsafe { ptr.read_unaligned() }
1292/// }
1293/// ```
1294#[inline]
1295#[stable(feature = "ptr_unaligned", since = "1.17.0")]
1296#[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")]
1297#[rustc_allow_const_fn_unstable(const_mut_refs, const_maybe_uninit_as_mut_ptr)]
1298#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1299#[rustc_diagnostic_item = "ptr_read_unaligned"]
1300pub const unsafe fn read_unaligned<T>(src: *const T) -> T {
1301 let mut tmp: MaybeUninit = MaybeUninit::<T>::uninit();
1302 // SAFETY: the caller must guarantee that `src` is valid for reads.
1303 // `src` cannot overlap `tmp` because `tmp` was just allocated on
1304 // the stack as a separate allocated object.
1305 //
1306 // Also, since we just wrote a valid value into `tmp`, it is guaranteed
1307 // to be properly initialized.
1308 unsafe {
1309 copy_nonoverlapping(src as *const u8, dst:tmp.as_mut_ptr() as *mut u8, count:mem::size_of::<T>());
1310 tmp.assume_init()
1311 }
1312}
1313
1314/// Overwrites a memory location with the given value without reading or
1315/// dropping the old value.
1316///
1317/// `write` does not drop the contents of `dst`. This is safe, but it could leak
1318/// allocations or resources, so care should be taken not to overwrite an object
1319/// that should be dropped.
1320///
1321/// Additionally, it does not drop `src`. Semantically, `src` is moved into the
1322/// location pointed to by `dst`.
1323///
1324/// This is appropriate for initializing uninitialized memory, or overwriting
1325/// memory that has previously been [`read`] from.
1326///
1327/// # Safety
1328///
1329/// Behavior is undefined if any of the following conditions are violated:
1330///
1331/// * `dst` must be [valid] for writes.
1332///
1333/// * `dst` must be properly aligned. Use [`write_unaligned`] if this is not the
1334/// case.
1335///
1336/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned.
1337///
1338/// [valid]: self#safety
1339///
1340/// # Examples
1341///
1342/// Basic usage:
1343///
1344/// ```
1345/// let mut x = 0;
1346/// let y = &mut x as *mut i32;
1347/// let z = 12;
1348///
1349/// unsafe {
1350/// std::ptr::write(y, z);
1351/// assert_eq!(std::ptr::read(y), 12);
1352/// }
1353/// ```
1354///
1355/// Manually implement [`mem::swap`]:
1356///
1357/// ```
1358/// use std::ptr;
1359///
1360/// fn swap<T>(a: &mut T, b: &mut T) {
1361/// unsafe {
1362/// // Create a bitwise copy of the value at `a` in `tmp`.
1363/// let tmp = ptr::read(a);
1364///
1365/// // Exiting at this point (either by explicitly returning or by
1366/// // calling a function which panics) would cause the value in `tmp` to
1367/// // be dropped while the same value is still referenced by `a`. This
1368/// // could trigger undefined behavior if `T` is not `Copy`.
1369///
1370/// // Create a bitwise copy of the value at `b` in `a`.
1371/// // This is safe because mutable references cannot alias.
1372/// ptr::copy_nonoverlapping(b, a, 1);
1373///
1374/// // As above, exiting here could trigger undefined behavior because
1375/// // the same value is referenced by `a` and `b`.
1376///
1377/// // Move `tmp` into `b`.
1378/// ptr::write(b, tmp);
1379///
1380/// // `tmp` has been moved (`write` takes ownership of its second argument),
1381/// // so nothing is dropped implicitly here.
1382/// }
1383/// }
1384///
1385/// let mut foo = "foo".to_owned();
1386/// let mut bar = "bar".to_owned();
1387///
1388/// swap(&mut foo, &mut bar);
1389///
1390/// assert_eq!(foo, "bar");
1391/// assert_eq!(bar, "foo");
1392/// ```
1393#[inline]
1394#[stable(feature = "rust1", since = "1.0.0")]
1395#[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")]
1396#[rustc_diagnostic_item = "ptr_write"]
1397#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1398pub const unsafe fn write<T>(dst: *mut T, src: T) {
1399 // Semantically, it would be fine for this to be implemented as a
1400 // `copy_nonoverlapping` and appropriate drop suppression of `src`.
1401
1402 // However, implementing via that currently produces more MIR than is ideal.
1403 // Using an intrinsic keeps it down to just the simple `*dst = move src` in
1404 // MIR (11 statements shorter, at the time of writing), and also allows
1405 // `src` to stay an SSA value in codegen_ssa, rather than a memory one.
1406
1407 // SAFETY: the caller must guarantee that `dst` is valid for writes.
1408 // `dst` cannot overlap `src` because the caller has mutable access
1409 // to `dst` while `src` is owned by this function.
1410 unsafe {
1411 assert_unsafe_precondition!(
1412 "ptr::write requires that the pointer argument is aligned and non-null",
1413 [T](dst: *mut T) => is_aligned_and_not_null(dst)
1414 );
1415 intrinsics::write_via_move(ptr:dst, value:src)
1416 }
1417}
1418
1419/// Overwrites a memory location with the given value without reading or
1420/// dropping the old value.
1421///
1422/// Unlike [`write()`], the pointer may be unaligned.
1423///
1424/// `write_unaligned` does not drop the contents of `dst`. This is safe, but it
1425/// could leak allocations or resources, so care should be taken not to overwrite
1426/// an object that should be dropped.
1427///
1428/// Additionally, it does not drop `src`. Semantically, `src` is moved into the
1429/// location pointed to by `dst`.
1430///
1431/// This is appropriate for initializing uninitialized memory, or overwriting
1432/// memory that has previously been read with [`read_unaligned`].
1433///
1434/// # Safety
1435///
1436/// Behavior is undefined if any of the following conditions are violated:
1437///
1438/// * `dst` must be [valid] for writes.
1439///
1440/// Note that even if `T` has size `0`, the pointer must be non-null.
1441///
1442/// [valid]: self#safety
1443///
1444/// ## On `packed` structs
1445///
1446/// Attempting to create a raw pointer to an `unaligned` struct field with
1447/// an expression such as `&packed.unaligned as *const FieldType` creates an
1448/// intermediate unaligned reference before converting that to a raw pointer.
1449/// That this reference is temporary and immediately cast is inconsequential
1450/// as the compiler always expects references to be properly aligned.
1451/// As a result, using `&packed.unaligned as *const FieldType` causes immediate
1452/// *undefined behavior* in your program.
1453///
1454/// Instead you must use the [`ptr::addr_of_mut!`](addr_of_mut)
1455/// macro to create the pointer. You may use that returned pointer together with
1456/// this function.
1457///
1458/// An example of how to do it and how this relates to `write_unaligned` is:
1459///
1460/// ```
1461/// #[repr(packed, C)]
1462/// struct Packed {
1463/// _padding: u8,
1464/// unaligned: u32,
1465/// }
1466///
1467/// let mut packed: Packed = unsafe { std::mem::zeroed() };
1468///
1469/// // Take the address of a 32-bit integer which is not aligned.
1470/// // In contrast to `&packed.unaligned as *mut _`, this has no undefined behavior.
1471/// let unaligned = std::ptr::addr_of_mut!(packed.unaligned);
1472///
1473/// unsafe { std::ptr::write_unaligned(unaligned, 42) };
1474///
1475/// assert_eq!({packed.unaligned}, 42); // `{...}` forces copying the field instead of creating a reference.
1476/// ```
1477///
1478/// Accessing unaligned fields directly with e.g. `packed.unaligned` is safe however
1479/// (as can be seen in the `assert_eq!` above).
1480///
1481/// # Examples
1482///
1483/// Write a usize value to a byte buffer:
1484///
1485/// ```
1486/// use std::mem;
1487///
1488/// fn write_usize(x: &mut [u8], val: usize) {
1489/// assert!(x.len() >= mem::size_of::<usize>());
1490///
1491/// let ptr = x.as_mut_ptr() as *mut usize;
1492///
1493/// unsafe { ptr.write_unaligned(val) }
1494/// }
1495/// ```
1496#[inline]
1497#[stable(feature = "ptr_unaligned", since = "1.17.0")]
1498#[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")]
1499#[rustc_diagnostic_item = "ptr_write_unaligned"]
1500#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1501pub const unsafe fn write_unaligned<T>(dst: *mut T, src: T) {
1502 // SAFETY: the caller must guarantee that `dst` is valid for writes.
1503 // `dst` cannot overlap `src` because the caller has mutable access
1504 // to `dst` while `src` is owned by this function.
1505 unsafe {
1506 copy_nonoverlapping(&src as *const T as *const u8, dst as *mut u8, count:mem::size_of::<T>());
1507 // We are calling the intrinsic directly to avoid function calls in the generated code.
1508 intrinsics::forget(src);
1509 }
1510}
1511
1512/// Performs a volatile read of the value from `src` without moving it. This
1513/// leaves the memory in `src` unchanged.
1514///
1515/// Volatile operations are intended to act on I/O memory, and are guaranteed
1516/// to not be elided or reordered by the compiler across other volatile
1517/// operations.
1518///
1519/// # Notes
1520///
1521/// Rust does not currently have a rigorously and formally defined memory model,
1522/// so the precise semantics of what "volatile" means here is subject to change
1523/// over time. That being said, the semantics will almost always end up pretty
1524/// similar to [C11's definition of volatile][c11].
1525///
1526/// The compiler shouldn't change the relative order or number of volatile
1527/// memory operations. However, volatile memory operations on zero-sized types
1528/// (e.g., if a zero-sized type is passed to `read_volatile`) are noops
1529/// and may be ignored.
1530///
1531/// [c11]: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
1532///
1533/// # Safety
1534///
1535/// Behavior is undefined if any of the following conditions are violated:
1536///
1537/// * `src` must be [valid] for reads.
1538///
1539/// * `src` must be properly aligned.
1540///
1541/// * `src` must point to a properly initialized value of type `T`.
1542///
1543/// Like [`read`], `read_volatile` creates a bitwise copy of `T`, regardless of
1544/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the returned
1545/// value and the value at `*src` can [violate memory safety][read-ownership].
1546/// However, storing non-[`Copy`] types in volatile memory is almost certainly
1547/// incorrect.
1548///
1549/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned.
1550///
1551/// [valid]: self#safety
1552/// [read-ownership]: read#ownership-of-the-returned-value
1553///
1554/// Just like in C, whether an operation is volatile has no bearing whatsoever
1555/// on questions involving concurrent access from multiple threads. Volatile
1556/// accesses behave exactly like non-atomic accesses in that regard. In particular,
1557/// a race between a `read_volatile` and any write operation to the same location
1558/// is undefined behavior.
1559///
1560/// # Examples
1561///
1562/// Basic usage:
1563///
1564/// ```
1565/// let x = 12;
1566/// let y = &x as *const i32;
1567///
1568/// unsafe {
1569/// assert_eq!(std::ptr::read_volatile(y), 12);
1570/// }
1571/// ```
1572#[inline]
1573#[stable(feature = "volatile", since = "1.9.0")]
1574#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1575#[rustc_diagnostic_item = "ptr_read_volatile"]
1576pub unsafe fn read_volatile<T>(src: *const T) -> T {
1577 // SAFETY: the caller must uphold the safety contract for `volatile_load`.
1578 unsafe {
1579 assert_unsafe_precondition!(
1580 "ptr::read_volatile requires that the pointer argument is aligned and non-null",
1581 [T](src: *const T) => is_aligned_and_not_null(src)
1582 );
1583 intrinsics::volatile_load(src)
1584 }
1585}
1586
1587/// Performs a volatile write of a memory location with the given value without
1588/// reading or dropping the old value.
1589///
1590/// Volatile operations are intended to act on I/O memory, and are guaranteed
1591/// to not be elided or reordered by the compiler across other volatile
1592/// operations.
1593///
1594/// `write_volatile` does not drop the contents of `dst`. This is safe, but it
1595/// could leak allocations or resources, so care should be taken not to overwrite
1596/// an object that should be dropped.
1597///
1598/// Additionally, it does not drop `src`. Semantically, `src` is moved into the
1599/// location pointed to by `dst`.
1600///
1601/// # Notes
1602///
1603/// Rust does not currently have a rigorously and formally defined memory model,
1604/// so the precise semantics of what "volatile" means here is subject to change
1605/// over time. That being said, the semantics will almost always end up pretty
1606/// similar to [C11's definition of volatile][c11].
1607///
1608/// The compiler shouldn't change the relative order or number of volatile
1609/// memory operations. However, volatile memory operations on zero-sized types
1610/// (e.g., if a zero-sized type is passed to `write_volatile`) are noops
1611/// and may be ignored.
1612///
1613/// [c11]: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
1614///
1615/// # Safety
1616///
1617/// Behavior is undefined if any of the following conditions are violated:
1618///
1619/// * `dst` must be [valid] for writes.
1620///
1621/// * `dst` must be properly aligned.
1622///
1623/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned.
1624///
1625/// [valid]: self#safety
1626///
1627/// Just like in C, whether an operation is volatile has no bearing whatsoever
1628/// on questions involving concurrent access from multiple threads. Volatile
1629/// accesses behave exactly like non-atomic accesses in that regard. In particular,
1630/// a race between a `write_volatile` and any other operation (reading or writing)
1631/// on the same location is undefined behavior.
1632///
1633/// # Examples
1634///
1635/// Basic usage:
1636///
1637/// ```
1638/// let mut x = 0;
1639/// let y = &mut x as *mut i32;
1640/// let z = 12;
1641///
1642/// unsafe {
1643/// std::ptr::write_volatile(y, z);
1644/// assert_eq!(std::ptr::read_volatile(y), 12);
1645/// }
1646/// ```
1647#[inline]
1648#[stable(feature = "volatile", since = "1.9.0")]
1649#[rustc_diagnostic_item = "ptr_write_volatile"]
1650#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
1651pub unsafe fn write_volatile<T>(dst: *mut T, src: T) {
1652 // SAFETY: the caller must uphold the safety contract for `volatile_store`.
1653 unsafe {
1654 assert_unsafe_precondition!(
1655 "ptr::write_volatile requires that the pointer argument is aligned and non-null",
1656 [T](dst: *mut T) => is_aligned_and_not_null(dst)
1657 );
1658 intrinsics::volatile_store(dst, val:src);
1659 }
1660}
1661
1662/// Align pointer `p`.
1663///
1664/// Calculate offset (in terms of elements of `size_of::<T>()` stride) that has to be applied
1665/// to pointer `p` so that pointer `p` would get aligned to `a`.
1666///
1667/// # Safety
1668/// `a` must be a power of two.
1669///
1670/// # Notes
1671/// This implementation has been carefully tailored to not panic. It is UB for this to panic.
1672/// The only real change that can be made here is change of `INV_TABLE_MOD_16` and associated
1673/// constants.
1674///
1675/// If we ever decide to make it possible to call the intrinsic with `a` that is not a
1676/// power-of-two, it will probably be more prudent to just change to a naive implementation rather
1677/// than trying to adapt this to accommodate that change.
1678///
1679/// Any questions go to @nagisa.
1680#[lang = "align_offset"]
1681pub(crate) const unsafe fn align_offset<T: Sized>(p: *const T, a: usize) -> usize {
1682 // FIXME(#75598): Direct use of these intrinsics improves codegen significantly at opt-level <=
1683 // 1, where the method versions of these operations are not inlined.
1684 use intrinsics::{
1685 assume, cttz_nonzero, exact_div, mul_with_overflow, unchecked_rem, unchecked_shl,
1686 unchecked_shr, unchecked_sub, wrapping_add, wrapping_mul, wrapping_sub,
1687 };
1688
1689 /// Calculate multiplicative modular inverse of `x` modulo `m`.
1690 ///
1691 /// This implementation is tailored for `align_offset` and has following preconditions:
1692 ///
1693 /// * `m` is a power-of-two;
1694 /// * `x < m`; (if `x ≥ m`, pass in `x % m` instead)
1695 ///
1696 /// Implementation of this function shall not panic. Ever.
1697 #[inline]
1698 const unsafe fn mod_inv(x: usize, m: usize) -> usize {
1699 /// Multiplicative modular inverse table modulo 2⁴ = 16.
1700 ///
1701 /// Note, that this table does not contain values where inverse does not exist (i.e., for
1702 /// `0⁻¹ mod 16`, `2⁻¹ mod 16`, etc.)
1703 const INV_TABLE_MOD_16: [u8; 8] = [1, 11, 13, 7, 9, 3, 5, 15];
1704 /// Modulo for which the `INV_TABLE_MOD_16` is intended.
1705 const INV_TABLE_MOD: usize = 16;
1706
1707 // SAFETY: `m` is required to be a power-of-two, hence non-zero.
1708 let m_minus_one = unsafe { unchecked_sub(m, 1) };
1709 let mut inverse = INV_TABLE_MOD_16[(x & (INV_TABLE_MOD - 1)) >> 1] as usize;
1710 let mut mod_gate = INV_TABLE_MOD;
1711 // We iterate "up" using the following formula:
1712 //
1713 // $$ xy ≡ 1 (mod 2ⁿ) → xy (2 - xy) ≡ 1 (mod 2²ⁿ) $$
1714 //
1715 // This application needs to be applied at least until `2²ⁿ ≥ m`, at which point we can
1716 // finally reduce the computation to our desired `m` by taking `inverse mod m`.
1717 //
1718 // This computation is `O(log log m)`, which is to say, that on 64-bit machines this loop
1719 // will always finish in at most 4 iterations.
1720 loop {
1721 // y = y * (2 - xy) mod n
1722 //
1723 // Note, that we use wrapping operations here intentionally – the original formula
1724 // uses e.g., subtraction `mod n`. It is entirely fine to do them `mod
1725 // usize::MAX` instead, because we take the result `mod n` at the end
1726 // anyway.
1727 if mod_gate >= m {
1728 break;
1729 }
1730 inverse = wrapping_mul(inverse, wrapping_sub(2usize, wrapping_mul(x, inverse)));
1731 let (new_gate, overflow) = mul_with_overflow(mod_gate, mod_gate);
1732 if overflow {
1733 break;
1734 }
1735 mod_gate = new_gate;
1736 }
1737 inverse & m_minus_one
1738 }
1739
1740 let stride = mem::size_of::<T>();
1741
1742 // SAFETY: This is just an inlined `p.addr()` (which is not
1743 // a `const fn` so we cannot call it).
1744 // During const eval, we hook this function to ensure that the pointer never
1745 // has provenance, making this sound.
1746 let addr: usize = unsafe { mem::transmute(p) };
1747
1748 // SAFETY: `a` is a power-of-two, therefore non-zero.
1749 let a_minus_one = unsafe { unchecked_sub(a, 1) };
1750
1751 if stride == 0 {
1752 // SPECIAL_CASE: handle 0-sized types. No matter how many times we step, the address will
1753 // stay the same, so no offset will be able to align the pointer unless it is already
1754 // aligned. This branch _will_ be optimized out as `stride` is known at compile-time.
1755 let p_mod_a = addr & a_minus_one;
1756 return if p_mod_a == 0 { 0 } else { usize::MAX };
1757 }
1758
1759 // SAFETY: `stride == 0` case has been handled by the special case above.
1760 let a_mod_stride = unsafe { unchecked_rem(a, stride) };
1761 if a_mod_stride == 0 {
1762 // SPECIAL_CASE: In cases where the `a` is divisible by `stride`, byte offset to align a
1763 // pointer can be computed more simply through `-p (mod a)`. In the off-chance the byte
1764 // offset is not a multiple of `stride`, the input pointer was misaligned and no pointer
1765 // offset will be able to produce a `p` aligned to the specified `a`.
1766 //
1767 // The naive `-p (mod a)` equation inhibits LLVM's ability to select instructions
1768 // like `lea`. We compute `(round_up_to_next_alignment(p, a) - p)` instead. This
1769 // redistributes operations around the load-bearing, but pessimizing `and` instruction
1770 // sufficiently for LLVM to be able to utilize the various optimizations it knows about.
1771 //
1772 // LLVM handles the branch here particularly nicely. If this branch needs to be evaluated
1773 // at runtime, it will produce a mask `if addr_mod_stride == 0 { 0 } else { usize::MAX }`
1774 // in a branch-free way and then bitwise-OR it with whatever result the `-p mod a`
1775 // computation produces.
1776
1777 let aligned_address = wrapping_add(addr, a_minus_one) & wrapping_sub(0, a);
1778 let byte_offset = wrapping_sub(aligned_address, addr);
1779 // FIXME: Remove the assume after <https://github.com/llvm/llvm-project/issues/62502>
1780 // SAFETY: Masking by `-a` can only affect the low bits, and thus cannot have reduced
1781 // the value by more than `a-1`, so even though the intermediate values might have
1782 // wrapped, the byte_offset is always in `[0, a)`.
1783 unsafe { assume(byte_offset < a) };
1784
1785 // SAFETY: `stride == 0` case has been handled by the special case above.
1786 let addr_mod_stride = unsafe { unchecked_rem(addr, stride) };
1787
1788 return if addr_mod_stride == 0 {
1789 // SAFETY: `stride` is non-zero. This is guaranteed to divide exactly as well, because
1790 // addr has been verified to be aligned to the original type’s alignment requirements.
1791 unsafe { exact_div(byte_offset, stride) }
1792 } else {
1793 usize::MAX
1794 };
1795 }
1796
1797 // GENERAL_CASE: From here on we’re handling the very general case where `addr` may be
1798 // misaligned, there isn’t an obvious relationship between `stride` and `a` that we can take an
1799 // advantage of, etc. This case produces machine code that isn’t particularly high quality,
1800 // compared to the special cases above. The code produced here is still within the realm of
1801 // miracles, given the situations this case has to deal with.
1802
1803 // SAFETY: a is power-of-two hence non-zero. stride == 0 case is handled above.
1804 // FIXME(const-hack) replace with min
1805 let gcdpow = unsafe {
1806 let x = cttz_nonzero(stride);
1807 let y = cttz_nonzero(a);
1808 if x < y { x } else { y }
1809 };
1810 // SAFETY: gcdpow has an upper-bound that’s at most the number of bits in a usize.
1811 let gcd = unsafe { unchecked_shl(1usize, gcdpow) };
1812 // SAFETY: gcd is always greater or equal to 1.
1813 if addr & unsafe { unchecked_sub(gcd, 1) } == 0 {
1814 // This branch solves for the following linear congruence equation:
1815 //
1816 // ` p + so = 0 mod a `
1817 //
1818 // `p` here is the pointer value, `s` - stride of `T`, `o` offset in `T`s, and `a` - the
1819 // requested alignment.
1820 //
1821 // With `g = gcd(a, s)`, and the above condition asserting that `p` is also divisible by
1822 // `g`, we can denote `a' = a/g`, `s' = s/g`, `p' = p/g`, then this becomes equivalent to:
1823 //
1824 // ` p' + s'o = 0 mod a' `
1825 // ` o = (a' - (p' mod a')) * (s'^-1 mod a') `
1826 //
1827 // The first term is "the relative alignment of `p` to `a`" (divided by the `g`), the
1828 // second term is "how does incrementing `p` by `s` bytes change the relative alignment of
1829 // `p`" (again divided by `g`). Division by `g` is necessary to make the inverse well
1830 // formed if `a` and `s` are not co-prime.
1831 //
1832 // Furthermore, the result produced by this solution is not "minimal", so it is necessary
1833 // to take the result `o mod lcm(s, a)`. This `lcm(s, a)` is the same as `a'`.
1834
1835 // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
1836 // `a`.
1837 let a2 = unsafe { unchecked_shr(a, gcdpow) };
1838 // SAFETY: `a2` is non-zero. Shifting `a` by `gcdpow` cannot shift out any of the set bits
1839 // in `a` (of which it has exactly one).
1840 let a2minus1 = unsafe { unchecked_sub(a2, 1) };
1841 // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
1842 // `a`.
1843 let s2 = unsafe { unchecked_shr(stride & a_minus_one, gcdpow) };
1844 // SAFETY: `gcdpow` has an upper-bound not greater than the number of trailing 0-bits in
1845 // `a`. Furthermore, the subtraction cannot overflow, because `a2 = a >> gcdpow` will
1846 // always be strictly greater than `(p % a) >> gcdpow`.
1847 let minusp2 = unsafe { unchecked_sub(a2, unchecked_shr(addr & a_minus_one, gcdpow)) };
1848 // SAFETY: `a2` is a power-of-two, as proven above. `s2` is strictly less than `a2`
1849 // because `(s % a) >> gcdpow` is strictly less than `a >> gcdpow`.
1850 return wrapping_mul(minusp2, unsafe { mod_inv(s2, a2) }) & a2minus1;
1851 }
1852
1853 // Cannot be aligned at all.
1854 usize::MAX
1855}
1856
1857/// Compares raw pointers for equality.
1858///
1859/// This is the same as using the `==` operator, but less generic:
1860/// the arguments have to be `*const T` raw pointers,
1861/// not anything that implements `PartialEq`.
1862///
1863/// This can be used to compare `&T` references (which coerce to `*const T` implicitly)
1864/// by their address rather than comparing the values they point to
1865/// (which is what the `PartialEq for &T` implementation does).
1866///
1867/// When comparing wide pointers, both the address and the metadata are tested for equality.
1868/// However, note that comparing trait object pointers (`*const dyn Trait`) is unreliable: pointers
1869/// to values of the same underlying type can compare inequal (because vtables are duplicated in
1870/// multiple codegen units), and pointers to values of *different* underlying type can compare equal
1871/// (since identical vtables can be deduplicated within a codegen unit).
1872///
1873/// # Examples
1874///
1875/// ```
1876/// use std::ptr;
1877///
1878/// let five = 5;
1879/// let other_five = 5;
1880/// let five_ref = &five;
1881/// let same_five_ref = &five;
1882/// let other_five_ref = &other_five;
1883///
1884/// assert!(five_ref == same_five_ref);
1885/// assert!(ptr::eq(five_ref, same_five_ref));
1886///
1887/// assert!(five_ref == other_five_ref);
1888/// assert!(!ptr::eq(five_ref, other_five_ref));
1889/// ```
1890///
1891/// Slices are also compared by their length (fat pointers):
1892///
1893/// ```
1894/// let a = [1, 2, 3];
1895/// assert!(std::ptr::eq(&a[..3], &a[..3]));
1896/// assert!(!std::ptr::eq(&a[..2], &a[..3]));
1897/// assert!(!std::ptr::eq(&a[0..2], &a[1..3]));
1898/// ```
1899#[stable(feature = "ptr_eq", since = "1.17.0")]
1900#[inline(always)]
1901#[must_use = "pointer comparison produces a value"]
1902#[rustc_diagnostic_item = "ptr_eq"]
1903#[allow(ambiguous_wide_pointer_comparisons)] // it's actually clear here
1904pub fn eq<T: ?Sized>(a: *const T, b: *const T) -> bool {
1905 a == b
1906}
1907
1908/// Compares the *addresses* of the two pointers for equality,
1909/// ignoring any metadata in fat pointers.
1910///
1911/// If the arguments are thin pointers of the same type,
1912/// then this is the same as [`eq`].
1913///
1914/// # Examples
1915///
1916/// ```
1917/// use std::ptr;
1918///
1919/// let whole: &[i32; 3] = &[1, 2, 3];
1920/// let first: &i32 = &whole[0];
1921///
1922/// assert!(ptr::addr_eq(whole, first));
1923/// assert!(!ptr::eq::<dyn std::fmt::Debug>(whole, first));
1924/// ```
1925#[stable(feature = "ptr_addr_eq", since = "1.76.0")]
1926#[inline(always)]
1927#[must_use = "pointer comparison produces a value"]
1928pub fn addr_eq<T: ?Sized, U: ?Sized>(p: *const T, q: *const U) -> bool {
1929 (p as *const ()) == (q as *const ())
1930}
1931
1932/// Hash a raw pointer.
1933///
1934/// This can be used to hash a `&T` reference (which coerces to `*const T` implicitly)
1935/// by its address rather than the value it points to
1936/// (which is what the `Hash for &T` implementation does).
1937///
1938/// # Examples
1939///
1940/// ```
1941/// use std::hash::{DefaultHasher, Hash, Hasher};
1942/// use std::ptr;
1943///
1944/// let five = 5;
1945/// let five_ref = &five;
1946///
1947/// let mut hasher = DefaultHasher::new();
1948/// ptr::hash(five_ref, &mut hasher);
1949/// let actual = hasher.finish();
1950///
1951/// let mut hasher = DefaultHasher::new();
1952/// (five_ref as *const i32).hash(&mut hasher);
1953/// let expected = hasher.finish();
1954///
1955/// assert_eq!(actual, expected);
1956/// ```
1957#[stable(feature = "ptr_hash", since = "1.35.0")]
1958pub fn hash<T: ?Sized, S: hash::Hasher>(hashee: *const T, into: &mut S) {
1959 use crate::hash::Hash;
1960 hashee.hash(state:into);
1961}
1962
1963#[stable(feature = "fnptr_impls", since = "1.4.0")]
1964impl<F: FnPtr> PartialEq for F {
1965 #[inline]
1966 fn eq(&self, other: &Self) -> bool {
1967 self.addr() == other.addr()
1968 }
1969}
1970#[stable(feature = "fnptr_impls", since = "1.4.0")]
1971impl<F: FnPtr> Eq for F {}
1972
1973#[stable(feature = "fnptr_impls", since = "1.4.0")]
1974impl<F: FnPtr> PartialOrd for F {
1975 #[inline]
1976 fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
1977 self.addr().partial_cmp(&other.addr())
1978 }
1979}
1980#[stable(feature = "fnptr_impls", since = "1.4.0")]
1981impl<F: FnPtr> Ord for F {
1982 #[inline]
1983 fn cmp(&self, other: &Self) -> Ordering {
1984 self.addr().cmp(&other.addr())
1985 }
1986}
1987
1988#[stable(feature = "fnptr_impls", since = "1.4.0")]
1989impl<F: FnPtr> hash::Hash for F {
1990 fn hash<HH: hash::Hasher>(&self, state: &mut HH) {
1991 state.write_usize(self.addr() as _)
1992 }
1993}
1994
1995#[stable(feature = "fnptr_impls", since = "1.4.0")]
1996impl<F: FnPtr> fmt::Pointer for F {
1997 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1998 fmt::pointer_fmt_inner(self.addr() as _, f)
1999 }
2000}
2001
2002#[stable(feature = "fnptr_impls", since = "1.4.0")]
2003impl<F: FnPtr> fmt::Debug for F {
2004 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2005 fmt::pointer_fmt_inner(self.addr() as _, f)
2006 }
2007}
2008
2009/// Create a `const` raw pointer to a place, without creating an intermediate reference.
2010///
2011/// Creating a reference with `&`/`&mut` is only allowed if the pointer is properly aligned
2012/// and points to initialized data. For cases where those requirements do not hold,
2013/// raw pointers should be used instead. However, `&expr as *const _` creates a reference
2014/// before casting it to a raw pointer, and that reference is subject to the same rules
2015/// as all other references. This macro can create a raw pointer *without* creating
2016/// a reference first.
2017///
2018/// The `expr` in `addr_of!(expr)` is evaluated as a place expression, but never loads
2019/// from the place or requires the place to be dereferenceable. This means that
2020/// `addr_of!(*ptr)` is defined behavior even if `ptr` is null, dangling, or misaligned.
2021/// Note however that `addr_of!((*ptr).field)` still requires the projection to
2022/// `field` to be in-bounds, using the same rules as [`offset`].
2023///
2024/// Note that `Deref`/`Index` coercions (and their mutable counterparts) are applied inside
2025/// `addr_of!` like everywhere else, in which case a reference is created to call `Deref::deref` or
2026/// `Index::index`, respectively. The statements above only apply when no such coercions are
2027/// applied.
2028///
2029/// [`offset`]: pointer::offset
2030///
2031/// # Example
2032///
2033/// ```
2034/// use std::ptr;
2035///
2036/// #[repr(packed)]
2037/// struct Packed {
2038/// f1: u8,
2039/// f2: u16,
2040/// }
2041///
2042/// let packed = Packed { f1: 1, f2: 2 };
2043/// // `&packed.f2` would create an unaligned reference, and thus be Undefined Behavior!
2044/// let raw_f2 = ptr::addr_of!(packed.f2);
2045/// assert_eq!(unsafe { raw_f2.read_unaligned() }, 2);
2046/// ```
2047///
2048/// See [`addr_of_mut`] for how to create a pointer to uninitialized data.
2049/// Doing that with `addr_of` would not make much sense since one could only
2050/// read the data, and that would be Undefined Behavior.
2051#[stable(feature = "raw_ref_macros", since = "1.51.0")]
2052#[rustc_macro_transparency = "semitransparent"]
2053#[allow_internal_unstable(raw_ref_op)]
2054pub macro addr_of($place:expr) {
2055 &raw const $place
2056}
2057
2058/// Create a `mut` raw pointer to a place, without creating an intermediate reference.
2059///
2060/// Creating a reference with `&`/`&mut` is only allowed if the pointer is properly aligned
2061/// and points to initialized data. For cases where those requirements do not hold,
2062/// raw pointers should be used instead. However, `&mut expr as *mut _` creates a reference
2063/// before casting it to a raw pointer, and that reference is subject to the same rules
2064/// as all other references. This macro can create a raw pointer *without* creating
2065/// a reference first.
2066///
2067/// The `expr` in `addr_of_mut!(expr)` is evaluated as a place expression, but never loads
2068/// from the place or requires the place to be dereferenceable. This means that
2069/// `addr_of_mut!(*ptr)` is defined behavior even if `ptr` is null, dangling, or misaligned.
2070/// Note however that `addr_of_mut!((*ptr).field)` still requires the projection to
2071/// `field` to be in-bounds, using the same rules as [`offset`].
2072///
2073/// Note that `Deref`/`Index` coercions (and their mutable counterparts) are applied inside
2074/// `addr_of_mut!` like everywhere else, in which case a reference is created to call `Deref::deref`
2075/// or `Index::index`, respectively. The statements above only apply when no such coercions are
2076/// applied.
2077///
2078/// [`offset`]: pointer::offset
2079///
2080/// # Examples
2081///
2082/// **Creating a pointer to unaligned data:**
2083///
2084/// ```
2085/// use std::ptr;
2086///
2087/// #[repr(packed)]
2088/// struct Packed {
2089/// f1: u8,
2090/// f2: u16,
2091/// }
2092///
2093/// let mut packed = Packed { f1: 1, f2: 2 };
2094/// // `&mut packed.f2` would create an unaligned reference, and thus be Undefined Behavior!
2095/// let raw_f2 = ptr::addr_of_mut!(packed.f2);
2096/// unsafe { raw_f2.write_unaligned(42); }
2097/// assert_eq!({packed.f2}, 42); // `{...}` forces copying the field instead of creating a reference.
2098/// ```
2099///
2100/// **Creating a pointer to uninitialized data:**
2101///
2102/// ```rust
2103/// use std::{ptr, mem::MaybeUninit};
2104///
2105/// struct Demo {
2106/// field: bool,
2107/// }
2108///
2109/// let mut uninit = MaybeUninit::<Demo>::uninit();
2110/// // `&uninit.as_mut().field` would create a reference to an uninitialized `bool`,
2111/// // and thus be Undefined Behavior!
2112/// let f1_ptr = unsafe { ptr::addr_of_mut!((*uninit.as_mut_ptr()).field) };
2113/// unsafe { f1_ptr.write(true); }
2114/// let init = unsafe { uninit.assume_init() };
2115/// ```
2116#[stable(feature = "raw_ref_macros", since = "1.51.0")]
2117#[rustc_macro_transparency = "semitransparent"]
2118#[allow_internal_unstable(raw_ref_op)]
2119pub macro addr_of_mut($place:expr) {
2120 &raw mut $place
2121}
2122