This is really a general Unix question, but someone here will know I hope!
I'm interested in ways of modifying (not necessarily with an editor) files in a way that is robust against the filesystem filling. In particular I want to be sure the file does not get truncated. I think a safe strategy is to do the following (or its equivalent, it does not have to be done by shell commands, obviously). Each stage would be checked for success, which checks I've mostly omitted, and obviously names etc would need checked &c &c, again these checks are omitted.
# make a link to the file so we do not lose it ln critical-file critical-file.safe
# make the copy we will modify cp -p critical-file critical-file.new ln critical-file.new critical-file.edit
# (now edit critical-file.edit. When complete, write out changes, # and *check the resulting file*. Proceed only once it is verified # OK.)
# This is the critical step. I believe this calls rename(2) and that # this is atomic, so will either fail, or succeed completely. mv critical-file.edit critical-file
# now either critical-file should be identical (and in fact should be the very same file) to critical-file.new, or to critical-file.safe
This relies on mv / rename(2) being atomic so that it will either work or not work at all, without the possibility of the original file being unlinked, inodes running out, then the new file not being created.
Is this (or something with many more checks, but using this strategy) a safe approach? If not, *is* there a safe approach? (Or in fact, even if it is, is there a simpler one?).
Darren Dunham 16 January 2008 21:06:57 [ permanent link ]
Tim Bradshaw <tfb+google@tfeb.org> wrote:
I'm interested in ways of modifying (not necessarily with an editor)
files in a way that is robust against the filesystem filling. In
particular I want to be sure the file does not get truncated. I think
a safe strategy is to do the following (or its equivalent, it does not
have to be done by shell commands, obviously). Each stage would be
checked for success, which checks I've mostly omitted, and obviously
names etc would need checked &c &c, again these checks are omitted.
# make a link to the file so we do not lose it
ln critical-file critical-file.safe
I normally make a full copy, but yes, I see why you're using the link here.
# make the copy we will modify
cp -p critical-file critical-file.new
ln critical-file.new critical-file.edit
I'm not sure what this link is for.
# (now edit critical-file.edit. When complete, write out changes,
# and *check the resulting file*. Proceed only once it is verified
# OK.)
# This is the critical step. I believe this calls rename(2) and that
# this is atomic, so will either fail, or succeed completely.
mv critical-file.edit critical-file
Yes.
# now either critical-file should be identical (and in fact should be
the very same file) to critical-file.new, or to critical-file.safe
Since it's a link, critical-file.new and critical_file are the same. I'm not sure why you think it would be identical to critical-file.safe (which is the pre-edit version)
This relies on mv / rename(2) being atomic so that it will either work
or not work at all, without the possibility of the original file being
unlinked, inodes running out, then the new file not being created.
Is this (or something with many more checks, but using this strategy)
a safe approach? If not, *is* there a safe approach? (Or in fact,
even if it is, is there a simpler one?).
Yes, the idea of using rename to make atomic updates is quite common. You'd want to use lock files (or other techniques in some cases) as well to make sure that 2 processes don't copy the original and make independent edits (which will overwrite one of the edits).
In UFS, I think the rename should always succeed if the filesystem is healthy. In other filesystems (like ZFS), it's possible for the rename to fail due to space requirements.
You'd also want to consider what happens if another process has the critical file open during the rename. Is it okay for that process to proceed with reading a now obsolete version? If not, you may have to use a more complex lock mechanism to avoid that issue.
-- Darren Dunham ddunham@taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
Tim Bradshaw 19 January 2008 19:23:58 [ permanent link ]
On Jan 18, 6:52 pm, ddun...@taos.com (Darren Dunham) wrote:
Actually, what I do is to enter the entire user information into the
'passwd' file (including encoded password). Then after I'm out of the
files I call 'pwconv' to create the shadow information.
I'd worry about what happens if something queries the data while the passwd file is updated but before the shadow file. But either way, it's something that has to be checked.
Locking the password file against system utilities is another challenge
Darren Dunham 21 January 2008 20:29:40 [ permanent link ]
Tim Bradshaw <tfb+google@tfeb.org> wrote:
On Jan 18, 6:52 pm, ddun...@taos.com (Darren Dunham) wrote:
Actually, what I do is to enter the entire user information into the
'passwd' file (including encoded password). Then after I'm out of the
files I call 'pwconv' to create the shadow information.
I'd worry about what happens if something queries the data while the
passwd file is updated but before the shadow file. But either way,
it's something that has to be checked.
It's not a problem. /etc/shadow is an optimization, not a requirement.
/etc/shadow is only examined if the character 'x' is present in the password field. For the time between putting /etc/passwd in place and running 'pwconv', the encoded password string is visible in the file, but the account is enabled.
Obviously editing both files is legal as well. I'm just saying that this method should not produce inconsistent operation due to /etc/shadow being out of sync temporarily.
-- Darren Dunham ddunham@taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
