6

An interesting question arose while I was trying to answer this:

Is mv atomic on my fs?

Is the rename() function required to be atomic by standard?

The 'rationale' section of the POSIX standard for rename states:

This rename() function is equivalent for regular files to that defined by the ISO C standard. Its inclusion here expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists. That specification requires that the action of the function be atomic.

But, the latest publicly-available ISO C Standard section on rename, in its entirety, states:

7.21.4.2 The rename function

Synopsis

#include <stdio.h>
int rename(const char *old, const char *new);

Description

The rename function causes the file whose name is the string pointed to by old to be henceforth known by the name given by the string pointed to by new. The file named old is no longer accessible by that name. If a file named by the string pointed to by new exists prior to the call to the rename function, the behavior is implementation-defined.

Returns

The rename function returns zero if the operation succeeds, nonzero if it fails, in which case if the file existed previously it is still known by its original name.

There's no explicit requirement of any kind for any type of atomicity in the rename() section of the ISO C Standard.

Having written many programs that relied upon the apparently implementation-specific atomicity of rename(), I had assumed that atomicity was a requirement and was surprised by the lack in the C Standard.

But the POSIX standard says that the ISO C standard requires rename() to be atomic.

Explanation(s)?

Community
  • 1
  • 1
Andrew Henle
  • 32,625
  • 3
  • 24
  • 56
  • You might want to include footnote 269. The paragraph titled **"Returns"** could be interpreted as implying atomicity, and that notion is further supported by note 269. – user3386109 Nov 10 '16 at 22:47
  • Does `rename` copy the file between different filesystems? If not, I'd expect it to use hard links where available, making the operation indeed atomic at application level. If it is atomic at FS or media-level is another question. – too honest for this site Nov 10 '16 at 22:47
  • 1
    That's not what the POSIX standard says. "That specification" refers to "expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists", i.e., the POSIX standard's additions. In particular, "If the link named by the *new* argument exists, [...] a link named *new* shall remain visible to other threads throughout the renaming operation and refer either to the file referred to by *new* or *old* before the operation began." – T.C. Nov 10 '16 at 22:48
  • @T.C. The POSIX standard states "This `rename()` function is equivalent for regular files to that defined by the **ISO C standard. ... That specification requires that the action of the function be atomic.**" That's pretty unequivocal. The POSIX standard states that the ISO C standard requires `rename()` to be atomic. But as far as I can tell, there's *nothing* in the ISO C standard that *requires* atomicity. Yet the POSIX standard states the ISO C standard requires that atomicity. Hence the question. – Andrew Henle Nov 11 '16 at 03:37
  • 1
    I can use ellipsis too: "Its inclusion here ... **specifies** behavior when the new parameter names a file that already exists. **That specification** requires that the action of the function be atomic." – T.C. Nov 11 '16 at 03:41
  • @T.C. So find "atomic" or some such with respect to `rename()` in the ISO C standards. Else what is "that specification" referring to? – Andrew Henle Nov 11 '16 at 03:42
  • The additional specification in the POSIX standard? – T.C. Nov 11 '16 at 03:43
  • @T.C. If the POSIX standard is referring to itself, that means that, in the *simpler* case of the target file not existing, `rename()` has no requirement to be atomic. Again, the question: Is `rename()` in general *required* to be atomic? If that's qualified with "only if the target path already exists", than the answer should be no, I'd think. – Andrew Henle Nov 11 '16 at 03:48
  • @user3386109 I don't know how that footnote could do that. I'd think copying a file is hard to require atomicity on. (Footnote 269 reads: "Among the reasons the implementation may cause the `rename` function to fail are that the file is open or that it is necessary to copy its contents to effectuate its renaming.") Unless you're relying on the *negative* implications of that footnote? That seems hard to stretch to a general *requirement* that `rename()` be atomic. Could you follow up? – Andrew Henle Nov 11 '16 at 03:51

1 Answers1

5

Your quote from the POSIX standard for rename() comes from the (non-normative) 'Rationale' section. The main entry — the actual normative material — begins:

For rename(): [CX] [Option Start] The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2008 defers to the ISO C standard. [Option End]

The rename() function shall change the name of a file. The old argument points to the pathname of the file to be renamed. The new argument points to the new pathname of the file. [CX] [Option Start] If the new argument does not resolve to an existing directory entry for a file of type directory and the new argument contains at least one non-<slash> character and ends with one or more trailing <slash> characters after all symbolic links have been processed, rename() shall fail.

… [Option End]

All the rest of the entry is within the [CX] (C Extension) tag and discusses other special behaviours.

The rationale you quote says:

This rename() function is equivalent for regular files to that defined by the ISO C standard. Its inclusion here expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists. That specification requires that the action of the function be atomic.

The 'That specification' referred to in the last sentence is the expanded definition that includes the specification of the actions on directories and 'when the new parameter names a file that already exists', not the specification in the C standard which, as you observe, does not say anything about atomicity (very reasonably; there are systems that can support C and rename() without being able to support POSIX's more stringent atomicity requirement).

And I see that this is exactly the argument made by T.C in their comment — I agree with T.C.

Community
  • 1
  • 1
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
  • 2
    Thank you. So the answer is "No, `rename()` isn't required to be atomic." That makes sense - but I asked the question because I'd always *assumed* `rename` had to be atomic, and was surprised when I couldn't find that. And it's always hard to prove a negative. – Andrew Henle Nov 12 '16 at 22:01
  • @AndrewHenle: Correct, Standard C does not require `rename()` to be atomic. POSIX has more stringent (and detailed and complete) rules because it deals with more file types. Standard C doesn't even know what a directory or symlink or FIFO is, let alone how they interact with `rename()`. – Jonathan Leffler Nov 12 '16 at 22:02
  • 1
    @AndrewHenle Not quite right - Standard C doesn't require `rename` to be atomic, *but POSIX does*. The key sentence is "If the link named by the new argument exists, it shall be removed and old renamed to new. In this case, a link named new shall remain visible to other threads throughout the renaming operation and refer either to the file referred to by new or old before the operation began." – zwol Nov 14 '16 at 15:43
  • That makes it atomic when renaming to an existing filename. But what about renaming to a filename that doesn't already exist? There doesn't seem to be anything in POSIX that requires this to be atomic. – Barmar Jan 03 '19 at 17:03