Sun 7000series storage (7110) NFS user problem

[WiPPaH]

Hi Guys,

 

I've got a 7110 here and I have exported a NFS share to a CentOS box, however I am having problems with non root users being able to create files on the share.

 

It works as root with an NFS exception defined for root access.

 

Normally this is down to the user uid/guid not being existing on the NFS server, but I am not sure how I would go about creating the user on the 7110 as I can only see options for access to the web GUI rather than a standard Linux user.

 

Any thoughts?

[Guest Guest]

iirc you dont do user permissions for NFS on the sun storage boxes, its all done by network ACLs.

[Ric_]

@WiPPaH : Click Configuration -> Users to create more users on the box... plus there may be a way to use identity mapping.

[WiPPaH]

@WiPPaH : Click Configuration -> Users to create more users on the box... plus there may be a way to use identity mapping.

 

Hmm user 'mysql' already exists on the 7110 so might try forcing MySQL to use a new 'mysql_7110' user instead. Will post back results.

[WiPPaH]

No Joy

 

I cant even find a whitepaper on this which seems very odd?!?

[Ric_]

What version opf NFS are you using? Is it worth forcing use of v3 if you're using v4 and vice versa?

[WiPPaH]

Currently using NFS v3 for compatibility.

 

Maybe some more information would be helpful.

 

=On the 7110=

* Created a new default NFS project and share: RDBMS/MySQL

* Setup the 'Root Access' exception for the VM.

 

==On the VM==

The Virtual Machine is running CentOS 5.4.

* mounted NFS share as root to /mnt/RDBMS

* Installed mysqld which creates a new user 'mysql' for the service.

* Edited /etc/my.cm to change the datadir to /mnt/RDBMS

* Ran /usr/bin/mysql_install_db as root which creates a 'mysql' and 'test' folder on /mnt/RDBMS and chown's them to the mysql user.

 

The only thing left to do is start the mysqld via 'service start mysqd' however I receive this error in the /var/log/mysqld error log.

 

100225 12:30:48  mysqld started
100225 12:30:48 [Warning] Can't create test file /mnt/RDBMS/RDBMS.lower-test
100225 12:30:48 [Warning] Can't create test file /mnt/RDBMS/RDBMS.lower-test
/usr/libexec/mysqld: Can't change dir to '/mnt/RDBMS/' (Errcode: 13) 100225 12:30:48 [ERROR] Aborting

100225 12:30:48 [Note] /usr/libexec/mysqld: Shutdown complete

100225 12:30:48  mysqld ended

 

My conclusion to this is that root has access to the share (through the exception I added) which was able to chown the folders. However when the 'mysql' cant access it as it needs to be defined somewhere on the 7110 (normally the uids & guids need to match on the NFS server as the NFS client).

[Duke]

@WiPPaH : Click Configuration -> Users to create more users on the box... plus there may be a way to use identity mapping.

 

That's only for accounts to administer the box, it isn't used for authenticating access to shares:

 

This section describes users who may administer the appliance, roles to manage authorizations granted to users, and how to add them to the system using the BUI or CLI.

 

I thought Identity Mapping might be the answer too, but that's just for mapping Windows user accounts to a Unix UID and handling both types of authentication for where you have a share running CIFS and NFS.

 

Hmm user 'mysql' already exists on the 7110

 

I think you're looking at the actual system accounts on the box itself, not ones that would be used for remote authentication. I think you need to be using the ACLs page on the BUI to configure this. I don't know Unix permissions as well as I should, but what happens if you specify mysql@remotehostname? CIFS permissions have to be done in a similar way (username@domian).

 

Have you looked at the Help wiki on the box itself? It's not entirely clear but does have some information.

 

Is the virtual machine's VHD on the SAN? If so, then why do you need to create another NFS share to mount on the VM for mysql to use? I'm think I've probably missed something there and there's a good reason.

 

Cheers,

Chris

[teejay]

Currently using NFS v3 for compatibility.

 

Maybe some more information would be helpful.

 

=On the 7110=

* Created a new default NFS project and share: RDBMS/MySQL

* Setup the 'Root Access' exception for the VM.

 

==On the VM==

The Virtual Machine is running CentOS 5.4.

* mounted NFS share as root to /mnt/RDBMS

* Installed mysqld which creates a new user 'mysql' for the service.

* Edited /etc/my.cm to change the datadir to /mnt/RDBMS

* Ran /usr/bin/mysql_install_db as root which creates a 'mysql' and 'test' folder on /mnt/RDBMS and chown's them to the mysql user.

 

.

 

You would be better in the Virtualisation software (you haven't said which one you are using) mounting the NFS partition in the storage pool and then creating a virtual hard drive on there which you then mount in your CentOS machine.

Edited February 25, 2010 by teejay

[WiPPaH]

Is the virtual machine's VHD on the SAN? If so, then why do you need to create another NFS share to mount on the VM for mysql to use? I'm think I've probably missed something there and there's a good reason.

 

Cheers,

Chris

 

Hi Chris

 

Thanks for the reply - The VM's VHD is mounted on the 7110 also, however after reading the Sun Big Admin guides and a blueprint document, they have been raving about how well these boxes can run databases when nfs mounted. The speeds they are getting are pretty awesome, infact ill see if I can find a link...

 

MySQL Performance on Sun Storage 7000 : Performance Profiles

 

I have also been following this sun blueprint guide: http://mapping.sun.com/profile/offer.jsp?id=202 (requires SUN account to access)

 

It actually looks pretty impressive, and a job well suited for these high performance boxes.

[WiPPaH]

You would be better in the Virtualisation software (you haven't said which one you are using) mounting the NFS partition in the storage pool and then creating a virtual hard drive on there which you then mount in your CentOS machine.

 

Not a bad idea Teejay - but storing the contents of the database as 1 vmd file might defeat the object of offloading the database files to the 7110 for speed. Check out the links I posted above, if its anything like what Sun are boasting it would be a massive advantage to mount it as a nfs share.

Edited February 25, 2010 by WiPPaH

[WiPPaH]

I think you're looking at the actual system accounts on the box itself, not ones that would be used for remote authentication. I think you need to be using the ACLs page on the BUI to configure this. I don't know Unix permissions as well as I should, but what happens if you specify mysql@remotehostname? CIFS permissions have to be done in a similar way (username@domian).

 

I have set the user of the share to '[email protected]' but that fails with error "User: Unknown or invalid user" on the 7110.

 

Have you looked at the Help wiki on the box itself? It's not entirely clear but does have some information.

 

I have had a look at the wiki but its not _that_ helpful when it comes to stuff like this, I've got a feeling im doing something majorly wrong here though if I cant find anyone else on google with the same problem. I am pretty sure I am not the first person to want to set exact permissions on an NFS share with these boxes.

[Duke]

Hmm, not sure if I'm confused here or not:

 

• The VM which runs MySQL is running from the 7110 (i.e. it's VDHs are on there)
  
• There is an additional share on the 7110 which is mapped to the VM for MySQL to use
  

 

I really, really might be wrong here, but I can't see how this would be of benefit. If the VM is already on the 7110, then if you tell MySQL to use the normal, local storage location it will already be on the 7110 (inside the VHD)? Granted, it wouldn't have it's own dedicated share, but unless there is a performance difference in the shares this wouldn't matter. The 7110 is still doing the same amount of read/writes, just in a slightly different location.

 

I admit I didn't look closely at the Sun doc you linked to, but my guess would be they might be talking about a physical Linux server running from local disk drives, then you map NFS storage on the 7110 to it and tell MySQL to use that remote storage.

 

Again, I may be completely wrong, haven't had a chance to look closely!

 

Chris

[Guest Guest]

Not a bad idea Teejay - but storing the contents of the database as 1 vmd file might defeat the object of offloading the database files to the 7110 for speed.

 

Im not sure thats true. From what ive read (and so far experianced) SUN, NFS, VHDs, and Xen (s it xen your using?) all play very nicely together.

 

ATM (i wont go into exactly why) we actually have our full windows file server as a VHD! Performance seems fine

[Duke]

ATM (i wont go into exactly why) we actually have our full windows file server as a VHD! Performance seems fine

 

I think you might see a slight performance difference for something like a file server (not saying it's a bad idea and I'm sure you had your reasons for doing so!) because data's having to be processed by Windows then get stored on the NAS/SAN, rather than going direct to the NAS/SAN.

 

However, in the case of MySQL, data will always have to be processed by the Linux server anyway before it interacts with the disks, so I can't see it making much difference whether the path is to the Linux VHD on the NAS/SAN, or via the Linux server to another share on the NAS/SAN.

 

j17sparky - For what it's worth, our users talk directly to our Sun 7410 for userdata and resources, but Windows still has the edge for management of userdata in a Windows environment. If you're dependant on using FSRM for quotas or NTFRS for replication then I see no problem (or alternative) to using a Windows server that's connected to the remote storage.

 

Chris

[WiPPaH]

Hmm, not sure if I'm confused here or not:

 

• The VM which runs MySQL is running from the 7110 (i.e. it's VDHs are on there)
  
• There is an additional share on the 7110 which is mapped to the VM for MySQL to use
  

 

I really, really might be wrong here, but I can't see how this would be of benefit. If the VM is already on the 7110, then if you tell MySQL to use the normal, local storage location it will already be on the 7110 (inside the VHD)? Granted, it wouldn't have it's own dedicated share, but unless there is a performance difference in the shares this wouldn't matter. The 7110 is still doing the same amount of read/writes, just in a slightly different location.

 

I admit I didn't look closely at the Sun doc you linked to, but my guess would be they might be talking about a physical Linux server running from local disk drives, then you map NFS storage on the 7110 to it and tell MySQL to use that remote storage.

 

Again, I may be completely wrong, haven't had a chance to look closely!

 

Chris

 

Correct - the VM hd is stored on the 7110, and the mysql data share is also running as a seperate nfs share.

 

The problem is the VHD is one big (big) file that is constantly being modified.

 

The Mysql Database is several small files, for example each table has its own set of files. This allows the 7110 to cache and do its own whizzy bits with the files (tables) that are used the most. This gives you a higher level of optimisation. At least that is my understanding of it.

 

Even if I was running a physical DB server I would still be having the same user problem with the NFS share.

[Guest Guest]

I think you might see a slight performance difference for something like a file server (not saying it's a bad idea and I'm sure you had your reasons for doing so!) because data's having to be processed by Windows then get stored on the NAS/SAN, rather than going direct to the NAS/SAN.

 

However, in the case of MySQL, data will always have to be processed by the Linux server anyway before it interacts with the disks, so I can't see it making much difference whether the path is to the Linux VHD on the NAS/SAN, or via the Linux server to another share on the NAS/SAN.

 

j17sparky - For what it's worth, our users talk directly to our Sun 7410 for userdata and resources, but Windows still has the edge for management of userdata in a Windows environment. If you're dependant on using FSRM for quotas or NTFRS for replication then I see no problem (or alternative) to using a Windows server that's connected to the remote storage.

 

Chris

 

Thats my understanding too.

 

Im currently investigating the idea of having users talk directly to the 7410, whether the permissions are suitable for our needs. We dont actually enforce quotas, or have enough servers to use NTFRS so no problems there. We just have PITA acls where kids can only read in the shared area, but some kids can write new data to a subfolder but cant delete...

Edited February 25, 2010 by Guest

[Duke]

Correct - the VM hd is stored on the 7110, and the mysql data share is also running as a seperate nfs share.

 

The problem is the VHD is one big (big) file that is constantly being modified.

 

The Mysql Database is several small files, for example each table has its own set of files. This allows the 7110 to cache and do its own whizzy bits with the files (tables) that are used the most. This gives you a higher level of optimisation. At least that is my understanding of it.

 

I see where you're coming from and I thought that might be the reasoning. What you're saying makes sense and I honestly don't know enough about it to provide a 100% clear explanation. If caching is done at the file-level rather than the block-level, then the 7110 will attempt to cache the the whole VHD, rather than just the MySQL files.

 

However, can you disable caching per-share? If not, the VHD and MySQL files would both get cached anyway, it's just a case of where the data would be stored in RAM. In the real world, I'm not sure you're going to see any noticeable difference between the two. You should get most of the benefits from the fact that the VHD and the MySQL files inside it are on the 7110 anyway and getting cached.

 

if the caching is done at the block-level, I think the whole point is moot anyway as only the 'MySQL blocks' and other regularly accessed data would be cached. However, file-level caching would make a lot more sense.

 

Even if I was running a physical DB server I would still be having the same user problem with the NFS share.

 

No argument there and I'm not sure on the solution although I think ACLs are the answer, I just don't know enough to say what's needed.

 

The good news? I've got Phil Lawrence from Sun Micro coming down tomorrow and I'll ask him for you!

 

Chris

[WiPPaH]

The good news? I've got Phil Lawrence from Sun Micro coming down tomorrow and I'll ask him for you!

 

Chris

 

Thanks Chris - ACL have to be the answer like you said, I suppose I just need a good guide for it.

 

If I get this working I will test the differences between a seperate NFS mysql db and a "part of the vdk" db and see if it was worth it!

[Duke]

Im currently investigating the idea of having users talk directly to the 7410, whether the permissions are suitable for our needs. We dont actually enforce quotas, or have enough servers to use NTFRS so no problems there. We just have PITA acls where kids can only read in the shared area, but some kids can write new data to a subfolder but cant delete...

 

The ACL management is pretty good, although in reality you're only going to be using the 7410 to manage the top-level permissions. I think Sun actually now recommend you use the Computer Management within Windows (check that before doing it) to configure your permissions if you're going to be using it in a Windows environment.

 

As far as your awkward ACLs go, if you can do it under Windows you should be able to do it on the 7410. Once you get down to the sub-folder level you'll be using the usual Windows Security window to configure permissions anyway.

 

If either of you have anything specific you want me to ask Phil tomorrow, drop me a PM.

 

Chris

[Duke]

If I get this working I will test the differences between a seperate NFS mysql db and a "part of the vdk" db and see if it was worth it!

 

Definitely a good idea, I'd be interested to see the results!

[james_yale]

I think there might be some benefits for caching in terms of intelligent read ahead and cache utilisation. Having the DB on a remote system also has some advantages if you ever need to move the mysqld.

 

On the ID mapping, remember that NFS matches users based on their numeric ID, not the name.

[Duke]

I have a Sun guy looking into this for you. Can't promise anything as I know how busy they are, but you might hopefully get a response.

[WiPPaH]

Any update on this Duke?

 

I am having a similar problem with our webservers that have a shared nfs based webroot folder.

 

Its chown'd by root to user apache, runs when I do "service httpd start" as root (runs as root), but if I restart and it runs as user 'apache' on system startup I get access denied.

[james_yale]

How about adding the users to /etc/passwd on your NFS server for a test?