Over the week, I’ve been slowly moving from mdadm raid to ZFS. My process was:
- create ZFS pool on secondary server
- rsync all files over to zfs server
- Nuke mdadm array on primary and set up zpool
- ssh dataset from secondary server to primary server.
This is 15tb of data and even over gigabit, it took a day and a half to transfer. It finally finished tonight, and somehow I’m the owner and group of every single file. In addition to this generally being weird, it also broke some docker volume binds, and I generally don’t want it.
It looks like the same is the case for the files on the secondary server too, so it must have happened during the initial rsync.
Fortunately, I also rsynced to some offline drives which kept ownership fine.
Anyway, I’m trying to figure out how the hell this happened. The rsync command I used was:
sudo rsync -ahu --delete --info=progress2 -e ssh /mnt/MONSTERDRIVE/ ch00f@192.168.1.65:/bluepool/monsterdrive/
At least I’m pretty sure this is what I used. I had to reverse-i-search to find it.
This is similar to the command I use when backing up to cold storage which has worked fine in the past. My understanding is that -a is shorthand for -rlptgoD where -o is “preserve owner.”
So how could this have happened?
Does it matter that the secondary server doesn’t have the same users as the primary server?
[SOLUTION]
From what I read online, using rsync over ssh as I did does not establish root permissions on the receiving end. So while I have the rights to modify the owners on the local side, I can only set the owners to the user I ssh’d as on the receiving side. Thus, I was the owner of every file.
The solution is two fold. First, I need to specify --rsync-path “sudo rsync” This tells the receiving side to use rsync as a super user.
Secondly, because there is no way to enter a super user password on the receiving side, I added a file to /etc/sudoers.d/ with
ch00f ALL=NOPASSWD:/usr/bin/rsync
This makes it so that the ch00f user doesn’t need to enter a password when running rsync as a super user.
I don’t think this is a security hole, and it got it to work.
Just a wild guess (and also thinking out loud): Does
ch00f@192.168.1.65have the permissions to write files as another user? Is there something funky with zfs where you need to explicitly grant a user permission to basically perform a “write as” or modify file permissions separate from root?You’re on the right path. Check my edit. It was rsync who didn’t have the permissions on the receiving end. It didn’t matter that it was a zfs directory.
You’re setting the owners when you rsync files. If the uid doesn’t match between machines, then the receiving machine won’t let you do anything with the newly existing files.
That wouldn’t be a problem because the plan was to move them back to the initial machine where they’d have matching owners. The issue was that because the ssh user didn’t have root permissions, it could only set the file owner to itself.
I found a workaround. Check my edit. Thanks for responding.
