How do I add an avatar to the public avatar library?
(mirroring &) ZFS: 9=no-ZFS; 10=yes?; 11=open=ZFS (improved)?
Hello Guest
  
  • Login
• Register…
• Start blog
  • Who, Where, When
• What can I do?
• What to Read?
  • Polls
• Avatars
• Interests
  • Cities and Countries
• Random blog
• Users search
  • Search
• Games
• Tests
• RYXI
  • Сообщества
• Talxy Chat
• Horoscope
• Online
 
Зарегистрируйся!

RYXI > Solaris > (mirroring &) ZFS: 9=no-ZFS; 10=yes?; 11=open=ZFS (improved)? 15 April 2008 11:11:07

  Recent blog posts: 
  They have birthday today: 
  Forums:   
  Discuss: 
  Recent forum topics: 
  Recent forum comments:
  Moderators:

(mirroring &) ZFS: 9=no-ZFS; 10=yes?; 11=open=ZFS (improved)?

David Combs 15 April 2008 11:11:07
 My last-night's posted question on mirroring suggested ZFS as easiest way.

So, how to get ZFS.

Right now, I have 9. (Have been negligent not going to 10.)

Also, I hear people here (c.u.s) being so haappy with "open"
solaris "(to be s11) -- probably bug-free enough for me,
for simple programming and browsing (firefox, but SLOW it is)
but no NFS no networking (except plugging into household
net, for fast internet access). Simple stuff indeed.

Questions:

1: should I go to 10, or straight to "11"?

2: the suggested ZFS, suggested not quite ready yet, what about that?



3: with ZFS, is letting someone login to one of its padded-cells
100% (99.99%) safe, in that he can't access anything
you don't want him to?


4: If ZFS isn't fully out yet, when might it be?


THANKS!


David


PS: any decent s10 books yet coming out? (I will
periodically be asking this same question.)

Heck, why doesn't Sun PAY some people, like from
this group, to do it -- or at least one on the
new features in 10, in "11", etc!


David



Add comment
ITguy 5 April 2008 05:03:22 permanent link ]
 On Apr 4, 7:22 pm, dkco...@panix.com (David Combs) wrote:
My last-night's posted question on mirroring suggested ZFS as easiest way.
So, how to get ZFS.
Right now, I have 9. (Have been negligent not going to 10.)

The first release of Solaris to include ZFS was 06/06 (S10u2??). Of
course, it's highly recommended to get the latest release.

Questions:
1: should I go to 10, or straight to "11"?

OpenSolaris has proved quite stable for me and has all the latest
features. Sun offers support for SolarisExpress (falls somewhere in
between OpenSolaris and Solaris10) and of course any Solaris 10
release.

2: the suggested ZFS, suggested not quite ready yet, what about that?

The only caveat to ZFS is that the root file system must still be on
UFS.

3: with ZFS, is letting someone login to one of its padded-cells
100% (99.99%) safe, in that he can't access anything
you don't want him to?

Not sure I understand the question. ZFS is a file system and has user/
group permissions on files like any other file system. Are you
referring to ZFS snapshots?? Those are point-in-time read-only copies
of the file system.

4: If ZFS isn't fully out yet, when might it be?

It's fully out. Get the latest Solaris 10 or OpenSolaris release.

David
PS: any decent s10 books yet coming out? (I will
periodically be asking this same question.)

Go to the source: docs.sun.com


Add comment
Rich Teer 5 April 2008 19:05:27 permanent link ]
 On Fri, 4 Apr 2008, David Combs wrote:

My last-night's posted question on mirroring suggested ZFS as easiest way.
So, how to get ZFS.
Right now, I have 9. (Have been negligent not going to 10.)

Time to mothball that ancient stuff! :-)­

Questions:
1: should I go to 10, or straight to "11"?

Get the latest version of Nevada (build 85 as I type). I've used
nothing but various builds of Nevada on my desktop for about 2 years.
Ditto for my (non-technical) wife.

2: the suggested ZFS, suggested not quite ready yet, what about that?

ZFS is very much ready for production use!

3: with ZFS, is letting someone login to one of its padded-cells
100% (99.99%) safe, in that he can't access anything
you don't want him to?

I think you might be confusing ZFS the file system with Zones the
virtulaisiation/com­partmentalisation technology. Something running
in a zone is indeed 100% safe in the sense you mention.

4: If ZFS isn't fully out yet, when might it be?

It is already, though of course features are still being added (e.g.,
on disk encryption).

HTH,

--
Rich Teer, SCSA, SCNA, SCSECA

CEO,
My Online Home Inventory

URLs: http://www.rite-gro­up.com/rich
http://www.linkedin­.com/in/richteer
http://www.myonline­homeinventory.com
Add comment
Rich Teer 5 April 2008 19:06:56 permanent link ]
 On Fri, 4 Apr 2008, ITguy wrote:

The only caveat to ZFS is that the root file system must still be on
UFS.

True, although that caveat will be going away with b87 or b88 of
Nevada. At least, that's the current plan. Roll on ZFS root!

--
Rich Teer, SCSA, SCNA, SCSECA

CEO,
My Online Home Inventory

URLs: http://www.rite-gro­up.com/rich
http://www.linkedin­.com/in/richteer
http://www.myonline­homeinventory.com
Add comment
Wolfgang 6 April 2008 01:27:34 permanent link ]
 Rich Teer schrieb:
On Fri, 4 Apr 2008, David Combs wrote:
My last-night's posted question on mirroring suggested ZFS as easiest way.
So, how to get ZFS.
Right now, I have 9. (Have been negligent not going to 10.)
Time to mothball that ancient stuff! :-)­
Questions:
1: should I go to 10, or straight to "11"?
Get the latest version of Nevada (build 85 as I type). I've used
nothing but various builds of Nevada on my desktop for about 2 years.
Ditto for my (non-technical) wife.
2: the suggested ZFS, suggested not quite ready yet, what about that?
ZFS is very much ready for production use!
3: with ZFS, is letting someone login to one of its padded-cells
100% (99.99%) safe, in that he can't access anything
you don't want him to?
I think you might be confusing ZFS the file system with Zones the
virtulaisiation/com­partmentalisation technology. Something running
in a zone is indeed 100% safe in the sense you mention.

unfortunaly it seems not 100% save:
http://www.zdnet.de­/security/analysen/0­,39029458,39186609,0­0.htm
Tobias Klein seems to have found a kernel exploit to gain access to the
global zone from non-global zone.


Add comment
Wolfgang 6 April 2008 01:32:48 permanent link ]
 Richard B. Gilbert schrieb:
ITguy wrote:
On Apr 4, 7:22 pm, dkco...@panix.com (David Combs) wrote:
My last-night's posted question on mirroring suggested ZFS as easiest
way.
So, how to get ZFS.
Right now, I have 9. (Have been negligent not going to 10.)
The first release of Solaris to include ZFS was 06/06 (S10u2??). Of
course, it's highly recommended to get the latest release.
Questions:
1: should I go to 10, or straight to "11"?
OpenSolaris has proved quite stable for me and has all the latest
features. Sun offers support for SolarisExpress (falls somewhere in
between OpenSolaris and Solaris10) and of course any Solaris 10
release.
2: the suggested ZFS, suggested not quite ready yet, what about that?
The only caveat to ZFS is that the root file system must still be on
UFS.
Oh? Last I heard, Sun had not yet released a means of backing up a ZFS
file sytem to tape. There is ufsbackup but no zfsbackup! You can make
a "snapshot" to a direct access device (disk) but not, AFAIK to tape.
That seems like a pretty big caveat to me!
What do people using ZFS do in lieu of sending tapes off site for
disaster recovery purposes? Ufsbackup of the snapshot?

zfs send/receive
Add comment
James Carlson 6 April 2008 02:04:58 permanent link ]
 "Richard B. Gilbert" <rgilbert88@comcast­.net> writes:
Oh? Last I heard, Sun had not yet released a means of backing up a
ZFS file sytem to tape. There is ufsbackup but no zfsbackup! You can
make a "snapshot" to a direct access device (disk) but not, AFAIK to
tape.
That seems like a pretty big caveat to me!
What do people using ZFS do in lieu of sending tapes off site for
disaster recovery purposes? Ufsbackup of the snapshot?

There are commercial options (Legato Networker), but there are also
simple solutions as well.

First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.

On my own systems, I use:

- Frequent cron-driven snapshots. They're cheap and easy.

- Mirrored pools, because I don't like losing things.

- A simple script that does zfs send into a file, compresses, then
writes out to tape.

- Occasional backups of whole file systems to DVD.

--
James Carlson, Solaris Networking <james.d.carlson@su­n.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Add comment
Guest 6 April 2008 04:15:57 permanent link ]
 On Apr 5, 3:04 pm, James Carlson <james.d.carl...@su­n.com> wrote:
"Richard B. Gilbert" <rgilber...@comcast­.net> writes:
Oh? Last I heard, Sun had not yet released a means of backing up a
ZFS file sytem to tape. There is ufsbackup but no zfsbackup! You can
make a "snapshot" to a direct access device (disk) but not, AFAIK to
tape.
That seems like a pretty big caveat to me!
Indeed!
What do people using ZFS do in lieu of sending tapes off site for
disaster recovery purposes? Ufsbackup of the snapshot?
There are commercial options (Legato Networker), but there are also
simple solutions as well.

So it costs more to use zfs from the production environment
perspective
on top of the other variables like maintenemce training etc.
I have to disagree with Rich that zfs is production ready : <

First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.

Yes but as I read it you have to have zfs (the program) "on the end"
to receive the send. I sent a snapshot earlier to a remote machine
and captured it with "dd" instead. Trying to pipe that 'image' back
went nowhere.

On my own systems, I use:
- Frequent cron-driven snapshots. They're cheap and easy.
- Mirrored pools, because I don't like losing things.
- A simple script that does zfs send into a file, compresses, then
writes out to tape.

I get part of that but not exactly how you extract off tape the "file"
or how you create the file in the first place.... : >
O Im talking SPARC here if that makes a difference

Also the man pages say:
"The format of the stream is evolving. No backwards compati-
bility is guaranteed. You may not be able to receive your
streams on future versions of ZFS." So if something changes in 6
months your
ZFS backups to tape are useless?? Say it aint so!
Add comment
Andrew Gabriel 6 April 2008 13:11:38 permanent link ]
 In article <47f7f000$0$628$9b4­e6d93@newsspool1.arc­or-online.net>,
Wolfgang <wtrappe@AT.web.de>­ writes:
Richard B. Gilbert schrieb:
What do people using ZFS do in lieu of sending tapes off site for
disaster recovery purposes? Ufsbackup of the snapshot?
zfs send/receive

No, you mustn't use zfs send/receive for long term backups.
There's no guarantee a future ZFS version will be able to read
your archives.

--
Andrew Gabriel
[email address is not usable -- followup in the newsgroup]
Add comment
Rich Teer 6 April 2008 22:38:21 permanent link ]
 On Sat, 5 Apr 2008, Wolfgang wrote:

unfortunaly it seems not 100% save:
Tobias Klein seems to have found a kernel exploit to gain access to the global
zone from non-global zone.

Interesting. Alas, I don't speak Deutch, so I can't read it...

--
Rich Teer, SCSA, SCNA, SCSECA

CEO,
My Online Home Inventory

URLs: http://www.rite-gro­up.com/rich
http://www.linkedin­.com/in/richteer
http://www.myonline­homeinventory.com
Add comment
Chris Ridd 7 April 2008 00:19:47 permanent link ]
 On 2008-04-06 19:38:21 +0100, Rich Teer <rich.teer@rite-gro­up.com> said:

On Sat, 5 Apr 2008, Wolfgang wrote:
unfortunaly it seems not 100% save:
Tobias Klein seems to have found a kernel exploit to gain access to the global
zone from non-global zone.
Interesting. Alas, I don't speak Deutch, so I can't read it...

The (google translated) article doesn't go into any detail. Perhaps you
had to be at the IT Defense (sic) conference...

Cheers,

Chris

Add comment
Agt 7 April 2008 02:41:02 permanent link ]
 

On 6 Apr 2008, Andrew Gabriel wrote:

AG : In article <47f7f000$0$628$9b4­e6d93@newsspool1.arc­or-online.net>,
AG : Wolfgang <wtrappe@AT.web.de>­ writes:
AG : > Richard B. Gilbert schrieb:
AG : >> What do people using ZFS do in lieu of sending tapes off site for
AG : >> disaster recovery purposes? Ufsbackup of the snapshot?
AG : > zfs send/receive
AG : No, you mustn't use zfs send/receive for long term backups.
AG : There's no guarantee a future ZFS version will be able to read
AG : your archives.

The zfs man page does warn about this.
So then it does not seems that send/receive is paticularly useful : /

People in production expect to be able to recover from "disaster"
sometimes going months back - even years. You know those silly
ILM legal things, yet I have yet to see posted here any way of
doing what Richard has asked other than buying a 3rd party
solution (which may be impossible to use due to that the fact
that a lot of people have NO Legato products in use and
dont want to start or cant w/o a total redesign of their entire backup
strategy retrainung etc) and a hint by James that it can be done.
But so far he has not revealed how he does it hint hint : >

Apparently zfs send only sends snapshots and only zfs receive knows what
to do with them. Not good enough.



Add comment
James Carlson 7 April 2008 19:14:31 permanent link ]
 usenetpersongerryt@g­mail.com writes:
First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.
Yes but as I read it you have to have zfs (the program) "on the end"
to receive the send. I sent a snapshot earlier to a remote machine
and captured it with "dd" instead. Trying to pipe that 'image' back
went nowhere.

It's just a byte stream. Write it anywhere that's convenient.

If you've had trouble with it, I suggest contacting Sun's support
group. There's no magic here; the commands are supposed to work. (I
haven't had trouble with it, but that doesn't mean you haven't found a
problem.)

On my own systems, I use:
- Frequent cron-driven snapshots. They're cheap and easy.
- Mirrored pools, because I don't like losing things.
- A simple script that does zfs send into a file, compresses, then
writes out to tape.
I get part of that but not exactly how you extract off tape the "file"
or how you create the file in the first place.... : >

I don't quite understand the question. It's roughly like this:

zfs send -i last-snapshot z/myfs@backup-snaps­hot | bzip2 > \
incremental-myfs.bz­2
tar -c5 .

Pulling it back in is just as easy; "tar r" to pull the file in, and
bunzip2 -c | zfs receive to recover the snapshot.

O Im talking SPARC here if that makes a difference

No ... it has nothing to do with SPARC versus x86. It works the same
on both.

Also the man pages say:
"The format of the stream is evolving. No backwards compati-
bility is guaranteed. You may not be able to receive your
streams on future versions of ZFS." So if something changes in 6
months your
ZFS backups to tape are useless?? Say it aint so!

That's (unfortunately) correct, which is why I said I do this:

- Occasional backups of whole file systems to DVD.

... in the part that you snipped. It's particularly important to do a
full backup before an upgrade.

For what it's worth, I've recovered things from backup, but I haven't
run into any versioning problems.

--
James Carlson, Solaris Networking <james.d.carlson@su­n.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Add comment
Wolfgang 8 April 2008 01:40:20 permanent link ]
 Chris Ridd schrieb:
On 2008-04-06 19:38:21 +0100, Rich Teer <rich.teer@rite-gro­up.com> said:
On Sat, 5 Apr 2008, Wolfgang wrote:
unfortunaly it seems not 100% save:
Tobias Klein seems to have found a kernel exploit to gain access to
the global
zone from non-global zone.
Interesting. Alas, I don't speak Deutch, so I can't read it...
The (google translated) article doesn't go into any detail. Perhaps you
had to be at the IT Defense (sic) conference...
Cheers,
Chris

unfortunaly i have also not found more information about that in the
net, but this link. But in the mentioned conference, he talked about
kernel exploits. And it is mentioned that the exploit is from 2007 and
seems still not be fixed .... no good news.

Maybe SUN has a lot to do with integration of mysql or lustre and other
tools. Also the development of ldoms seems very slow.

Wolfgang
Add comment
Wolfgang 8 April 2008 01:46:22 permanent link ]
 Rich Teer schrieb:
On Sat, 5 Apr 2008, Wolfgang wrote:
unfortunaly it seems not 100% save:
Tobias Klein seems to have found a kernel exploit to gain access to the global
zone from non-global zone.
Interesting. Alas, I don't speak Deutch, so I can't read it...
a little bit more public information, but nothing detailed:
http://www.trapkit.­de/advisories/index.­php
Add comment
Guest 8 April 2008 01:59:29 permanent link ]
 On Apr 7, 8:14 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.
Yes but as I read it you have to have zfs (the program) "on the end"
to receive the send. I sent a snapshot earlier to a remote machine
and captured it with "dd" instead. Trying to pipe that 'image' back
went nowhere.
It's just a byte stream. Write it anywhere that's convenient.

Which I did. Now what to do with it.
Someone says - Hey we need a file from last week could you get it off
tape for us?? send/receive seems to only work on whole snapshots or
increments of whole snapshots...

If you've had trouble with it, I suggest contacting Sun's support
group. There's no magic here; the commands are supposed to work. (I
haven't had trouble with it, but that doesn't mean you haven't found a
problem.)
Maybe the above..
On my own systems, I use:
- Frequent cron-driven snapshots. They're cheap and easy.
- Mirrored pools, because I don't like losing things.
- A simple script that does zfs send into a file, compresses, then
writes out to tape.
I get part of that but not exactly how you extract off tape the "file"
or how you create the file in the first place.... : >
I don't quite understand the question. It's roughly like this:
zfs send -i last-snapshot z/myfs@backup-snaps­hot | bzip2 > \
incremental-myfs.bz­2

Have to skip using bzip2 as it takes hours? to compress a 4 GB file..?
Not terribly backup friendly (although making the snap certainly is)

tar -c5 .
Pulling it back in is just as easy; "tar r" to pull the file in, and
bunzip2 -c | zfs receive to recover the snapshot.

Again what if you dont want the entire thing - just a list of file or
directories
which is a more typical request.. Retreive fu/bar from dweezles data
dir
dated 2 days ago please
is accomplished how from a full or incremental zfs snap direct to pool/
fu?

O Im talking SPARC here if that makes a difference
No ... it has nothing to do with SPARC versus x86. It works the same
on both.

Saves me testing that : >

Also the man pages say:
"The format of the stream is evolving. No backwards compati-
bility is guaranteed. You may not be able to receive your
streams on future versions of ZFS." So if something changes in 6
months your
ZFS backups to tape are useless?? Say it aint so!
That's (unfortunately) correct, which is why I said I do this:
- Occasional backups of whole file systems to DVD.

Even a Blu-Ray wont be big enough. Not by a long shot.
Imagine how long that would take to write : <

... in the part that you snipped. It's particularly important to do a
full backup before an upgrade.

As always : >
Anyway it was snipped because Im not talking about teeny weenie
workstation files here.

For what it's worth, I've recovered things from backup, but I haven't
run into any versioning problems.

And JUST when you thought you were safe... : >

In the meantime I see that one can simply tar off to tape any ZFS
volume
just like you can UFS - gee wouda thunk it.. But the snaps are
not the same. zfs snap should emulate fssnap ideally - and it does not
does it?


--
James Carlson, Solaris Networking <james.d.carl...@su­n.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Add comment
James Carlson 8 April 2008 18:35:06 permanent link ]
 usenetpersongerryt@g­mail.com writes:
On Apr 7, 8:14 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.
Yes but as I read it you have to have zfs (the program) "on the end"
to receive the send. I sent a snapshot earlier to a remote machine
and captured it with "dd" instead. Trying to pipe that 'image' back
went nowhere.
It's just a byte stream. Write it anywhere that's convenient.
Which I did. Now what to do with it.
Someone says - Hey we need a file from last week could you get it off
tape for us?? send/receive seems to only work on whole snapshots or
increments of whole snapshots...

Just pull in the whole snapshot, use "zfs promote" to turn it into a
file system, and then copy out the files you want.

I suspect that there's an underlying problem here, and that's
incorrect (or just incomplete) use of ZFS. The design of ZFS strongly
encourages the use of lots of *small* file systems, rather than a few
huge ones.

If you're having trouble manipulating huge snapshots, it could be
because you haven't factored out the file systems well enough yet.

I don't quite understand the question. It's roughly like this:
zfs send -i last-snapshot z/myfs@backup-snaps­hot | bzip2 > \
incremental-myfs.bz­2
Have to skip using bzip2 as it takes hours? to compress a 4 GB file..?
Not terribly backup friendly (although making the snap certainly is)

At least for me, I don't care how long that bzip2 runs. It's running
off of a snapshot, which is effectively instantaneous. All that it
needs to do is complete before the next backup run.

tar -c5 .
Pulling it back in is just as easy; "tar r" to pull the file in, and
bunzip2 -c | zfs receive to recover the snapshot.
Again what if you dont want the entire thing - just a list of file or
directories
which is a more typical request.. Retreive fu/bar from dweezles data
dir
dated 2 days ago please
is accomplished how from a full or incremental zfs snap direct to pool/
fu?

Pull in the snapshot. Mount it up. Make the files available.

There's no support for getting just a couple of files ... but I
haven't seen that as a serious limitation in any of my use.

In any event, it sounds like we're at least talking about different
administrative environments. Perhaps you'd be more comfortable with
something like Legato.

That's (unfortunately) correct, which is why I said I do this:
- Occasional backups of whole file systems to DVD.
Even a Blu-Ray wont be big enough. Not by a long shot.
Imagine how long that would take to write : <

It's not too hard to break things up. Yes, it does take a long time
to write 100GB or more, and I wish I had fancier backup equipment than
DVD+DDS2. ;-}

... in the part that you snipped. It's particularly important to do a
full backup before an upgrade.
As always : >
Anyway it was snipped because Im not talking about teeny weenie
workstation files here.

If you've got huge *files*, you're in trouble no matter what you do.
ufsdump backs up things by inode -- which means it backs up on a
per-file basis. Even incrementals are going to be huge.

For what it's worth, I've recovered things from backup, but I haven't
run into any versioning problems.
And JUST when you thought you were safe... : >
In the meantime I see that one can simply tar off to tape any ZFS
volume
just like you can UFS - gee wouda thunk it.. But the snaps are
not the same. zfs snap should emulate fssnap ideally - and it does not
does it?

I've used both, and I think it's far easier to understand and use ZFS
snapshots than fssnap. The ZFS ones are more like what you'd find on
an external NFS storage device.

--
James Carlson, Solaris Networking <james.d.carlson@su­n.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Add comment
Guest 8 April 2008 19:57:27 permanent link ]
 On Apr 8, 7:35 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
On Apr 7, 8:14 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
First of all, the "zfs send" and "zfs receive" commands are similar in
functionality to ufsbackup/ufsrestor­e, but without an incremental
mode.
Yes but as I read it you have to have zfs (the program) "on the end"
to receive the send. I sent a snapshot earlier to a remote machine
and captured it with "dd" instead. Trying to pipe that 'image' back
went nowhere.
It's just a byte stream. Write it anywhere that's convenient.
Which I did. Now what to do with it.
Someone says - Hey we need a file from last week could you get it off
tape for us?? send/receive seems to only work on whole snapshots or
increments of whole snapshots...
Just pull in the whole snapshot, use "zfs promote" to turn it into a
file system, and then copy out the files you want.

Pull? I assume you mean "receive". receive does not seem
to care for image streams so far : >

I suspect that there's an underlying problem here, and that's
incorrect (or just incomplete) use of ZFS. The design of ZFS strongly
encourages the use of lots of *small* file systems, rather than a few
huge ones.

But in the "real World" quite the opposite seems to be true. More and
more
big fat files everywhere. And ever increasing file system sizes.
So its a bit of a contradiction: a zeta file file system
that prefers smaller things : >

If you're having trouble manipulating huge snapshots, it could be
because you haven't factored out the file systems well enough yet.

Im not having trouble with size as such.

I don't quite understand the question. It's roughly like this:
zfs send -i last-snapshot z/myfs@backup-snaps­hot | bzip2 > \
incremental-myfs.bz­2
Have to skip using bzip2 as it takes hours? to compress a 4 GB file..?
Not terribly backup friendly (although making the snap certainly is)
At least for me, I don't care how long that bzip2 runs. It's running
off of a snapshot, which is effectively instantaneous. All that it
needs to do is complete before the next backup run.

Well thats the charm of a snapshot for sure! However "backups" have
to happen within a time window every night yes? And yes I have seen
some
backups that would take more than 24 hours if left to run to
completion.

tar -c5 .
Pulling it back in is just as easy; "tar r" to pull the file in, and
bunzip2 -c | zfs receive to recover the snapshot.
Again what if you dont want the entire thing - just a list of file or
directories
which is a more typical request.. Retreive fu/bar from dweezles data
dir
dated 2 days ago please
is accomplished how from a full or incremental zfs snap direct to pool/
fu?
Pull in the snapshot. Mount it up. Make the files available.

I will see about that of course thank you.

There's no support for getting just a couple of files ... but I
haven't seen that as a serious limitation in any of my use.
In any event, it sounds like we're at least talking about different
administrative environments. Perhaps you'd be more comfortable with
something like Legato.

Which cannot be inserted in the environment Im working with - they use
another vendor and
what with budgets etc even IF they had the $$ many would fight any
change tooth and nail
I suspect. Production types : /

That's (unfortunately) correct, which is why I said I do this:
- Occasional backups of whole file systems to DVD.
Even a Blu-Ray wont be big enough. Not by a long shot.
Imagine how long that would take to write : <
It's not too hard to break things up. Yes, it does take a long time
to write 100GB or more, and I wish I had fancier backup equipment than
DVD+DDS2. ;-}

now thats going back : >

... in the part that you snipped. It's particularly important to do a
full backup before an upgrade.
As always : >
Anyway it was snipped because Im not talking about teeny weenie
workstation files here.
If you've got huge *files*, you're in trouble no matter what you do.
ufsdump backs up things by inode -- which means it backs up on a
per-file basis. Even incrementals are going to be huge.

But its surprisingly doable so far.

For what it's worth, I've recovered things from backup, but I haven't
run into any versioning problems.
And JUST when you thought you were safe... : >
In the meantime I see that one can simply tar off to tape any ZFS
volume
just like you can UFS - gee wouda thunk it.. But the snaps are
not the same. zfs snap should emulate fssnap ideally - and it does not
does it?
I've used both, and I think it's far easier to understand and use ZFS
snapshots than fssnap. The ZFS ones are more like what you'd find on
an external NFS storage device.

Well the worst case hack around for me at this point is to tar up a
ZFS volume
to a UFS volume before you fssnap the UFS volume. I guess one could
touch a
file to make it incremental.. Argh! Then again the tape agent may
simply "just work".
Ill find out today.

O and why this obsession with ZFS? I get about 3-4x the throughput
compared to
UFS. I'd like to have that throughput.

Add comment
Steve van der Burg 9 April 2008 16:32:01 permanent link ]
 James Carlson <james.d.carlson@su­n.com> wrote in
news:xoavabk4bejp.f­sf@sun.com:

usenetpersongerryt@­gmail.com writes:
On Apr 7, 8:14 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
First of all, the "zfs send" and "zfs receive" commands are
similar in functionality to ufsbackup/ufsrestor­e, but
without an incremental mode.
Yes but as I read it you have to have zfs (the program) "on
the end" to receive the send. I sent a snapshot earlier to a
remote machine and captured it with "dd" instead. Trying to
pipe that 'image' back went nowhere.
It's just a byte stream. Write it anywhere that's convenient.
Which I did. Now what to do with it.
Someone says - Hey we need a file from last week could you get it
off tape for us?? send/receive seems to only work on whole
snapshots or increments of whole snapshots...
Just pull in the whole snapshot, use "zfs promote" to turn it into
a file system, and then copy out the files you want.

Who has enough spare disk space laying around to do that? When my
Oracle dba wants a single 10MB file from a 1.5TB filesystem, you can
see how silly that strategy is.

It's like Sun just doesn't get how their customers use their
systems. All they need to do is give me something with the same
functionality that ufsdump/ufsrestore have (table of contents at the
start of a dump, ability to restore only parts of a dump, etc) and I
can use zfs in production systems.

...Steve

--
Steve van der Burg
Technical Analyst, Information Services
London Health Sciences Centre
London, Ontario, Canada
Email: steve.vanderburg@lh­sc.on.ca
Add comment
Jdh13 9 April 2008 17:58:57 permanent link ]
 Wolfgang a crit :
Chris Ridd schrieb:
On 2008-04-06 19:38:21 +0100, Rich Teer <rich.teer@rite-gro­up.com> said:
On Sat, 5 Apr 2008, Wolfgang wrote:
unfortunaly it seems not 100% save:
Tobias Klein seems to have found a kernel exploit to gain access to
the global
zone from non-global zone.
Interesting. Alas, I don't speak Deutch, so I can't read it...
The (google translated) article doesn't go into any detail. Perhaps
you had to be at the IT Defense (sic) conference...
Cheers,
Chris
unfortunaly i have also not found more information about that in the
net, but this link. But in the mentioned conference, he talked about
kernel exploits. And it is mentioned that the exploit is from 2007 and
seems still not be fixed .... no good news.
or no news at all, why this kind of information appears only in deutsch
website? english zdnet has nothing about it
Add comment
Guest 9 April 2008 18:30:42 permanent link ]
 On Apr 9, 5:32 am, Steve van der Burg <steve.vanderb...@l­hsc.on.ca>
wrote:
James Carlson <james.d.carl...@su­n.com> wrote innews:xoavabk4bejp­.fsf@sun.com:
usenetpersonger...@­gmail.com writes:
On Apr 7, 8:14 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
First of all, the "zfs send" and "zfs receive" commands are
similar in functionality to ufsbackup/ufsrestor­e, but
without an incremental mode.
Yes but as I read it you have to have zfs (the program) "on
the end" to receive the send. I sent a snapshot earlier to a
remote machine and captured it with "dd" instead. Trying to
pipe that 'image' back went nowhere.
It's just a byte stream. Write it anywhere that's convenient.
Which I did. Now what to do with it.
Someone says - Hey we need a file from last week could you get it
off tape for us?? send/receive seems to only work on whole
snapshots or increments of whole snapshots...
Just pull in the whole snapshot, use "zfs promote" to turn it into
a file system, and then copy out the files you want.
Who has enough spare disk space laying around to do that? When my
Oracle dba wants a single 10MB file from a 1.5TB filesystem, you can
see how silly that strategy is.

I think James has conceded that Enterprise environments have
different requirements than his workstation : >

It's like Sun just doesn't get how their customers use their
systems. All they need to do is give me something with the same
functionality that ufsdump/ufsrestore have (table of contents at the
start of a dump, ability to restore only parts of a dump, etc) and I
can use zfs in production systems.

Given tar etc still work on ZFS thats what you have to use at this
point.
A step backwards, not very fine grained, but whats tarred to UFS can
be snapped : <

Yes it seems insane to me as well that a zeta filesystem prefers
things small..
And that there is no zfsbackup/zfsretore­ with snap capabilities.
Or a read-only mode during backup or something - kinda like Oracles
RMAN..

File an RFE? But really its an obvious need.
How could something so basic be missed?
Add comment


James Carlson 14 April 2008 16:27:29 permanent link ]
 usenetpersongerryt@g­mail.com writes:
On Apr 9, 5:32 am, Steve van der Burg <steve.vanderb...@l­hsc.on.ca>
wrote:
Who has enough spare disk space laying around to do that? When my
Oracle dba wants a single 10MB file from a 1.5TB filesystem, you can
see how silly that strategy is.
I think James has conceded that Enterprise environments have
different requirements than his workstation : >

I'm not using Oracle, but several of the machines I manage are much
bigger than just a workstation. Not that you asked ...

Yes it seems insane to me as well that a zeta filesystem prefers
things small..
And that there is no zfsbackup/zfsretore­ with snap capabilities.
Or a read-only mode during backup or something - kinda like Oracles
RMAN..

I still don't understand how UFS has any real advantage in the
particular situation you've described.

If you have huge files -- which you've already said is the issue at
hand -- then ufsdump is going to create *HUGE* incrementals. Just
touching the big file will cause it to appear in your incremental
backups and will force you to extract the whole thing.

There's no "win" that I can see there. You'll need just as much
storage to recover an incremental backup of a huge file as you will
to do "zfs receive" on a snapshot with a huge file in it.

Actually, it's slightly worse with UFS, because you cannot pool your
storage, and need separate slices for each mount point. ZFS does pool
storage, and doesn't need (or want) separate slices.

If the issue is that you've got a directory with a mix of huge files
that never change (?!) and tiny ones that do, then perhaps the
application is intelligent enough to relocate these to separate
directories. If so, then you can use separate ZFS file systems to
optimize the backup/restore requirements.

In any event, this all sounds like an overconstrained problem to me.

File an RFE? But really its an obvious need.
How could something so basic be missed?

Talk to your local support folks. Really. If you discover that
there's something missing, then you *NEED* to talk to them.

If you don't want to do that, then perhaps you could talk to the ZFS
developers directly -- zfs-discuss@opensol­aris.org. I'm sure they'd
be keenly interested in learning what they missed in development.

--
James Carlson, Solaris Networking <james.d.carlson@su­n.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Add comment
Guest 15 April 2008 03:05:11 permanent link ]
 On Apr 14, 5:27 am, James Carlson <james.d.carl...@su­n.com> wrote:
usenetpersonger...@­gmail.com writes:
On Apr 9, 5:32 am, Steve van der Burg <steve.vanderb...@l­hsc.on.ca>
wrote:
Who has enough spare disk space laying around to do that? When my
Oracle dba wants a single 10MB file from a 1.5TB filesystem, you can
see how silly that strategy is.
I think James has conceded that Enterprise environments have
different requirements than his workstation : >
I'm not using Oracle, but several of the machines I manage are much
bigger than just a workstation. Not that you asked ...

Me or Steve? I didnt because Im sure you have worked on bigger
stuff :>

Yes it seems insane to me as well that a zeta filesystem prefers
things small..
And that there is no zfsbackup/zfsretore­ with snap capabilities.
Or a read-only mode during backup or something - kinda like Oracles
RMAN..
I still don't understand how UFS has any real advantage in the
particular situation you've described.

I had the tape tech do a backup off ZFS direct to tape and then a
restore.
No issues. So the main difference here is how snapshots "behave".
Yes I prefer snapshots for the usual reasons.

With ufsdump/restore I feel its easier to deal with single files
directories etc.
No need to promote. What I used to do was have a repository of
ufs snapshots from various machines and 7 days worth. No need to go to
tape
unless the file was older than a week. The daylies were backed to
tape.
The tape access was slow. Now it isnt so they go direct to tape.

If you have huge files -- which you've already said is the issue at
hand -- then ufsdump is going to create *HUGE* incrementals. Just
touching the big file will cause it to appear in your incremental
backups and will force you to extract the whole thing.

Well you kinda brought up that ZFS likes things small and countered
thats the opposite
of whats happening. No biggie!

There's no "win" that I can see there. You'll need just as much
storage to recover an incremental backup of a huge file as you will
to do "zfs receive" on a snapshot with a huge file in it.
Actually, it's slightly worse with UFS, because you cannot pool your
storage, and need separate slices for each mount point. ZFS does pool
storage, and doesn't need (or want) separate slices.
If the issue is that you've got a directory with a mix of huge files
that never change (?!) and tiny ones that do, then perhaps the
application is intelligent enough to relocate these to separate
directories. If so, then you can use separate ZFS file systems to
optimize the backup/restore requirements.

Id prefer to use ZFS because its just plain faster than UFS in any
test Ive tried.
My concern was with backups and how snaps behave with ZFS.

In any event, this all sounds like an overconstrained problem to me.

Then I didnt describe it well : >

File an RFE? But really its an obvious need.
How could something so basic be missed?
Talk to your local support folks. Really. If you discover that
there's something missing, then you *NEED* to talk to them.

Ill pass that on. Thanks.

If you don't want to do that, then perhaps you could talk to the ZFS
developers directly -- zfs-disc...@opensol­aris.org. I'm sure they'd
be keenly interested in learning what they missed in development.

I might!
Add comment


Michael Schmarck 15 April 2008 09:28:08 permanent link ]
 usenetpersongerryt@g­mail.com <usenetpersongerryt­@gmail.com> wrote:

With ufsdump/restore I feel its easier to deal with single files
directories etc.
No need to promote.

I don't get that. With ZFS, you'd backup to tape what's on your
"backup snapshot", wouldn't you? So you would *NOT* backup, let's
say, /data, but /data/.zfs/snapshot­s/backup (if /data is a ZFS).
Isn't that so?

In the case of a restore, you'd have to restore from tape whatever
was under /data/.zfs/snapshot­s/backup and restore it to the correct
place in /data.

I don't see where you'd need to promote a snapshot there.

With UFS, how would you do the backup? Would you store the huge
file that ufsdump creates on tape? How would you do single file
restores in that case?

My concern was with backups and how snaps behave with ZFS.

I don't quite understand your concerns (but that doesn't mean
much *G*).

Michael
Add comment
Matt McLeod 15 April 2008 10:40:35 permanent link ]
 In <6401302.iMkJUoPHNa­@schmarck.cn>, Michael Schmarck wrote:
With UFS, how would you do the backup? Would you store the huge
file that ufsdump creates on tape? How would you do single file
restores in that case?

With 'ufsrestore -i', tagging the files or directories you need.
Only what is tagged is restored. ufsrestore still has to read through
a lot of tape to get what you want, but you don't need enough spare
disk to do a complete restore the way you do with zfs send/receive.

If you're using ufsdump/ufsrestore for backups then in that respect
ZFS is going to be a problem. If you're using "enterprise" tools
like Networker, NetBackup, TSM, whatever, then it isn't an issue.

We're using NetBackup (glorified GNU tar wrapper that it is) and it's
fine with ZFS. Previous gig used TSM, same deal. I used to be very
keen on ufsdump as a backup tool, not so much any more, not unless it's
been wrapped up in something like Amanda to do indexing and tape
management.

Matt

--
* Matt McLeod | mail: matt@boggle.org | blog: http://abortrephras­e.com/ *
--- People can do the work, so machines have time to think ---
Add comment