Home > Command Line, General > CIFS data migrations

CIFS data migrations

March 3rd, 2011

Almost seamless! Sort of…

As with most of my thoughts, it started with an innocent customer query. EMC have some very cool inbuilt tools for doing seamless CIFS data migration, but NetApp don’t. It’s something that often causes a fair amount of problems and some careful planning with NetApp as we don’t have this. But I was thinking today, we kinda do, I just don’t think we leverage the tools available properly.

Enter widelinks. Here is an excerpt from a NetApp KB article on the topic (KB 3011420)…

A symbolic link is a special file created by NFS clients that points to another file or directory. Widelink entries are a way to redirect absolute symbolic links on the filer. They allow the symbolic link destination to be a share on the same filer or on another filer. The following examples illustrate how to create a symlink from volume to qtree on the same filer, and from volume to volume on different filers.

What does this mean and why will my life be easier after reading the rest of this article?

So if I have a nice shiny new NetApp filer (or an old one I haven’t got round to migrating my CIFS data onto yet), and I have my old CIFS file server that is rapidly approaching failure or out of support. I can create my new file and share structure on my NetApp, and then use widelinks to redirect the user to the CIFS file server while I worry about all the data copy out of hours without having the ball-ache of copying all my data all at once.

First of all let me highly recommend that if you aren’t using DFS, start using it. You’re going to have to repoint your users to a new share name anyway, so you might as well do it properly. Setup a basic DFS root and repoint all your users to this. Why? Because next time you come round to reconfiguring or upgrading your CIFS share infrastructure you won’t have to touch your users, just repoint DFS one night.

So widelinks expands on symlinks. Don’t get too excited because at the moment “mklink” from Windows 7/2008 isn’t the same thing, and it won’t work on NetApp shares. Symbolic links in this article are a *NIX thing, but the concept is similar to Windows symbolic links. If you don’t know symbolic links, think of them as shortcuts, they simply redirect you to another location without actually redirecting you. What widelinks will do is translate this symbolic link and bridge the gap to the remote location.

Okay, enough theory, onto the practice…

My setup is quite simple. I have a NetApp simulator running as a cluster, I have a NAS box that has some home media, I have a MBA to do some of the UNIX stuff I need (sorry, I tried to get round this, but a Linux/UNIX box is needed for now). Filer1 is my new system that I want all my users to use. Filer2 (with a folder called test2) and my NAS box (with movies) are legacy systems I’m looking to replace (bit of role-play).

First of all my CIFS share on the new system is created. We need to make sure this has “widelinks” enabled, so the command line is as follows…

cifs shares -add /vol/test1 test1 -widelink

If you already have your shares setup, no problem…

cifs shares -change <share_name> -widelink

If you query the cifs shares, you should get output similar to below…

node1*> cifs shares

Name Mount Point Description

- – – – – – – – – – – – – – –

test1 /vol/test1

. . . widelinks supported

everyone / Full Control

 

I can see that widelinks is enabled for my share. I need to export this as NFS to my MBA so I can create the symbolic link. Before you ask, I tried both Adam Fox‘s “ntap_symlink” and Oliver Krause’s “ln”, but these didn’t do what I wanted (ln just crashes on Win7). I’m not sure if you can mount the target for the symbolic links as anything other than NFS, but it can certainly map to file systems that aren’t just NFS, I have done this also with an external CIFS share.

exportfs -io root=<unix_hostname> /vol/test1

From my MBA I can then mount this up and create some symbolic links. (I already have my NAS mounted)

mount filer1:/vol/test1 /mnt/test1

mount filer2:/vol/test2 /mnt/test2

cd /mnt/test1

ln -s /mnt/test2/test2 test2

ln -s /Volumes/movies movies

 

If you “ls –lah” the folder you’ll see the symlinks created and it should show the mappings. Job done for my MBA, back to Windows and the NetApp (phew you sigh).

We need to create a translations file for the symlinks. Basically this reads the symbolic link that we created on our UNIX host and converts this to a DFS style link to redirect the data path. So make sure you noted the paths you used on your UNIX host for the mappings! The “*” in the paths are useful as you can include many different symbolic links here and they will all be matched and followed.

wrfile /etc/symlink.translations

widelink /mnt/test2/* \\filer2\test2\*

widelink /Volumes/movies/* \\NAS\movies\*

CTRL+C

 

This file is re-read every 30 seconds, so patience, go get a glass of water. Go back to the test1 CIFS share and you should see some magic! (test1 already exists as new data).

 

More magic is shown when you right click one of these folders and go to the DFS tab. (you’ll also see a DFS tab on folders that aren’t widelinks, but you’ll notice the referral list is just the normal share).

 

Additionally you’ll notice that if you go to the “test2″ folder you’ll see all its contents without you being actually redirected to this other host.

 

Now we have the challenge of data migration. The advantage (and major problem) of widelinks is that they appear and feel to a Windows user as data on the share they are looking at. So users can’t edit the symbolic link, or over-write the data (which is probably a good thing). If they copy over the folder, it’ll attempt to update the target (in our case, the legacy systems). But as administrator you have the same constraints. So what do I do???

Unfortunately this is where the cool stuff things break apart a little. What I need to do is remove the symbolic link, remove the entry in “/etc/symlink.translations” and then copy my data across to fill the space. When doing this make sure to make the old data unavailable (change the share permissions is safest and simplest) as the symbolic links can be a little sticky and you don’t want users writing to 2 locations. I had some odd results with this on my Windows 7 desktop as it seemed to cache the widelink, but mapping the drive from a different machine worked fine. I guess you’ll want all clients offline when you do data copies.

Now wouldn’t it be cool if widelinks could be some way integrated into the OSSV mechanism? In case you don’t know, OSSV can do file copies from a Windows host onto the NetApp into qtrees. This would make a very nice migration tool!

Please NetApp consider the following improvements to widelinks:

  • Allow me to create symbolic links from Windows (mklink) – this is a must have! All users out there, log a support call regarding this, the more people request it, the quicker they’ll do it!
  • Give me a mechanism to integrate this with OSSV (I know I’m asking a lot)
  • Once you’ve integrated OSSV into this, give me a mechanism to transparently remove the widelinks once my data has copied across. This surely isn’t asking the impossible as ndmp copies dump the inodes and ACLs last and that’s what needs to be achieved here.
  • A GUI to manage my widelinks and symbolic links. I toyed with the idea of creating a PowerShell to manage this, but the issues of creating symbolic links aren’t easily achieved on Windows.

 

Special thanks to Adam Fox as a few of his topics on the communities pointed me in the right direction. Useful references included below:

http://communities.netapp.com/thread/3752

http://communities.netapp.com/message/24227

http://communities.netapp.com/message/31156

https://kb.netapp.com/support/index?page=content&actp=LIST&id=1011758

https://kb.netapp.com/support/index?page=content&actp=LIST&id=1011091

https://kb.netapp.com/support/index?page=content&id=3011420

Command Line, General , , , ,

  1. Roger Weeks
    | #1

    Have you considered using MultiStore (vfilers) to host your CIFS data on Data ONTAP? vFilers let you migrate a whole vfiler at a time, or set up a DR vFiler at a DR site, and fail over all the CIFS configuration including shares, quotas, etc.

    We can’t migrate or DR failover CIFS shares with no reconnection, because of the nature of the protocol, but if you’re using DFS in front of that, the clients wouldn’t notice at all since the IP address of the vfiler remains the same after migration or DR.

  2. | #2

    Thanks Roger. Yes it’s something we actually regularly do as it does make life a lot simpler. MultiStore brings a lot of very cool features and functionality and I think a lot of people get fixated on it just being used for secure multi-tennancy, but actually it’s uses are a lot more!

    Unfortunately this still doesn’t address getting the CIFS data onto the NetApp in the first place, but MultiStore definitely compliments the CIFS protocol, especially when used with DFS. This is an area where CIFS on NetApp is much stronger than EMC due to the flexibility and power of MultiStore.

  3. Dave
    | #3

    Perhaps not the most elegant solution, but we created our CIFS volumes and shares and then ran robocopy over the space of a few evenings approaching a weekend to get all the CIFS data across. The last thinga we did was to “unshare” the stuff on the Windows server, run robocopy for a final time, change the A record for the Windows box to a CNAME to the NetApp’s actual name then add the name of the old windows box to cifs alias on the NetApp.

    Guess it doesn’t work so well if you only want to move some stuff or if you want the Windows box to continue running..

  4. | #4

    Totally agree with this method Dave, it’s tried and tested and works well. The challenge is that if I have terrabytes of file share data, robocopy isn’t the most elegant solution and as it’s single streamed, it can take some time. Widelinks means that potentially you don’t have to copy all your data at once. It takes the pressure off.

    It’d be nice for NetApp to put together an elegant solution that did a robocopy style transfer in the background, I guess that’s really where I’m going with this :)

  5. John DeBella
    | #5

    Chris, any issues if filer1 and filer2 were different Netapp filers? I didn’t see any restriction to this in your write up but thought I’d ask.

  6. | #6

    No issues at all, the official documentation I believe actually shows this config.

  7. Jon Whitwham
    | #7

    Hi Chris,

    Is there an easy way to import Shares into a vFiler? We are in the process of moving Windows Home Directories and have over 1000 shares. We will use Robocopy to move the data but are hoping we do not have to manually create all these shares.

    Cheers

    Jon

  8. | #8

    Have you considered using the homedir feature? This would avoid the need to create a share for every single user and continual have to manage and update this. Have a look at http://www.wafl.co.uk/cifs_homedir/ to get a rough idea. It’s fairly easy and simple to setup to be honest and gives you the best flexibility.

  9. Alex
    | #9

    @Dave
    what switches did you use robocopy with? I also want to migrate data from one CIFS-share to another.

    thx

  10. steve
    | #10

    You can use robocopy /copyall /e /w:0 /r:0 /MT. Do a robocopy /??? for the mt; multi-thread is a new feature, and I believe allows for 128 or so threads; 8 is default, but check the read me.

  11. | #11

    Thanks Steve!

  12. Erling
    | #12

    Hi and thanks for an informative article.
    Just wanted to add a couple of points that I struggled with getting widelinks to work:
    1.
    I did NOT get the widlinks to work as I edited the symlink.translations file with note(word)pad (after mapping etc$ to my win7 desktop).
    When writing the exact same widelink statement with wrfile /etc/symlink.translations direct on the filer it worked!
    2.
    I got problems with accessing a widlink “coming from” a hidden share (someshare$/widelinkpath…problem!). Sharing out without $ worked OK
    (someshare/widelinkpath…no problem!)

    Erling

  13. | #13

    Cheers for the feedback Erling, really useful thanks!

  14. Jacques Cronje
    | #14

    Might be too late, but there is a small app:

    http://www.scriptlogic.com/products/securecopy/

    Does the same as robocopy but what I appreciated most is that it creates the shares on the source on the target.

  15. | #15

    Thank you Jacques, and yes securecopy is very good. I know a few people who have used this on occasion. Cheers for the feedback!

  16. | #16

    @Chris Kranz
    Or use just one share (i.e. Users) with Access based Enumeration on Data ONTAP 7.3.3 and above.

  17. | #17

    Anther useful tool to sync data between NetApp Storage Systems and Windows DFS shares is Sync Toy (Version 2.1 fixes issues with acls and data loss now making it the best choice). It can do bi-directional replication which OSSV won’t do.

  18. | #18

    Sorry my ABE reply was to Jon Whitwham to replace his 1000 shares.

  19. Cadey
    | #19

    @Chris Kranz I was with a customer yesterday and suggested using OSSV on there file server, that way it allows them to do a block base copy of the data to the filer meaning it would be quicker than robocopy, create shares of the qtree’s created and then run a manual robocopy to sync the files when ready to move over?

    Is this something you have seen or have implemented? what floors are there in doing so. It would only mean using the OSSV software and snapvault schedule the once but should do the trick right?

  20. | #20

    No, this works well, and infact you could continue to use SnapVault to do incremental copies and then just delete the snapshots once you convert it to a normal volume.

    SnapVault doesn’t work well with very large volumes or volumes with millions of files, but then very few solutions work well with that TBH!

  1. No trackbacks yet.



This site is not affiliated or sponsored in anyway by NetApp or any other company mentioned within.
%d bloggers like this: