A look at Netapp’s NDMP Cloud Extension Software (NCE)

Hey, you got a small Netapp? Hey, ya wanna do some backups to Amazon S3 or a compatible object storage system (aka Netapp StorageGrid Webscale)? NCE is for you!

The official blurb on it:

NDMP Cloud Extension Software (NCE) is a host-based utility that transfers data between Data ONTAP® storage systems and the Amazon® Simple Storage Service (Amazon S3) cloud. You can transfer data to Amazon S3 by using the ndmpcopy command. The ndmpcopy command is executed from a Linux host, which communicates with the storage system to transfer data to Amazon S3. Data transfer is supported at the volume, qtree, directory, or file level.

This isn’t a perfect DR solution, but it does have some cool use cases.

( If you want a REALLY awesome hosted SAN-Replication DR, I recommend checking out the services offering from my employer’s sister company – H.A. Services. We can ingest SnapMirror, SnapVaults, spin up VMs, etc. )

If you just have a small local dataset, or are using Netapp Cloud or OnTap Edge in the Cloud, and can go through the slower restores over the internet, an NDMP-based backup to Amazon S3 may be just for you!

So what is it?

Basically it is a linux server (re: VM typically), which runs a Java command line application which can function as “ndmpcopy” to S3 proxy.

If you ever have ran ndmpcopy on a Netapp either copying a volume locally or remotely, the java based command in linux feels much the same. Much of the syntax is identical.

You get it here:

• Download: http://mysupport.netapp.com/NOW/download/software/ndmp_cloud/1.0/
• Install and Admin Guide

You need:

• 64-bit linux system. (I tested on Centos 7. You need to install net-utils, for ifconfig)
• Java 1.7_61 or higher (I tested on the latest 1.8)
• NDMP4 enabled on Ontap
• Ontap 8.1.4 or higher 7-mode or Clustered Mode

Features

• Baseline and incremental data transfer
• Level-based data transfer (levels 0 through 9)
• Incremental forever data transfer
• Checksum-based data verification
• Data transfer using specific Snapshot® copies
• Backup set deletion
• Backup set summary

Note: On Clustered Ontap, it uses Cluster Aware Backup (CAB) by default. This means that you do the host:/vserver_name/volume_name when telling it what to backup (Thanks Joe Walker of Netapp, for reminding me that this was changeable on the filer, if you need node based)

Example of enabling NDMP on a cluster

Now, your Netapp is ready.

Go ahead, log into Amazon AWS and create a bucket for your backups.

Installing is easy. Install java on your linux box, Download & Extract the tar/gz on your linux host. Edit the ndmpcopy file in vi/pico/emacs and point it to where java is (if its not on your path)

I filled up a directory with a lot of files (project thumbnails and public domain books).

You have a few options of how to use it:

• Backup – level 0 ndmp baseline -l0
• Backup – level [1-9] ndmp incremental -l[1-9]
• Backup – Incremental always (inowalk incremental dump)
• Restore – Incremental level or backup set #
• Delete Backups

Let’s do a backup! Here’s some options I used

• -d debug
• -df debug file
• -l 0 level
• -sa source auth
• -cloudconf my config file of AWS info
• -nodeleteconf dont delete the cloud conf file (Yes, by default it does wipe out your clod config file… we don’t want another CodeSpaces AWS hack!)

Wonderful!! Let’s verify. Yep, I see them in Amazon:

Sweet! Delete some files in the CIFS share locally. And… restore.

Now, let’s clean up!

What else should you know about this?

• FILES STORED ON AMAZON S3 UPLOADED ARE NOT ENCRYPTED!!!!!!
  • You can manually update your buckets or subfolders to be server side encrypted, but anyone with access to your AWS key would be able to download your dumps in non-encrypted form.
• Your stream from the Netapp to the Linux proxy is not encrypted. (Make sure it’s on a trusted segment of your network)
• The linux proxy can only have 1 NIC and a Default GW.
  • I originally tried this with a VM with 2 nics. 1 was a default route out to the internet. The other was homed to the same subnet as my Netapp Intercluster lifs. I got all sorts of connection issues I couldn’t figure out for a good long while.
• Remember, you are going to pay for your transfers in, transfers out, puts, gets, etc.
• Don’t expect a throughput faster than 24mbps from an Amazon S3 upload or download.

Be Sociable, Share!

•