...
The NHR systems in Berlin ("Lise") and Göttingen ("Emmy") are equipped with various file systems each. Their properties and their intended utilization are described here.
Disk quotas based on group ownerships are implemented on each site's global (non-local) file systems.
...
External connections to the NHR systems in Berlin or Göttingen require that an SSH key pair is used for authentication. More details can be found here. The location of the private key file can be specified when calling scp or rsync on the user's local machine. Some examples including both data transfer directions are shown below.
...
Data transfer in the context of a batch job is restricted due to limited network access of the compute nodes. Please send a message to the support mailing list in case you need further help.
Internal Data Transfer
Internal data transfer between a Berlin and a Göttingen login node using scp or rsync works right out of the box - that is, without specifying any keys or passwords. This is enabled through host-based authentication which is active by default.
| Hinweis |
|---|
For internal data transfer, please always use the host name alone, omitting the domain suffix ".hlrn.de". You can use the generic names blogin, glogin or specific names like blogin5, glogin2, etc.. This way NHR internal links are used which are faster than external routes. The latter are used when specifying fully qualified host names (blogin.hlrn.de or glogin.hlrn.de) which is not recommended here. |
Data Transfer Between Emmy (Göttingen) And The GWDG SCC
If you have previously been working on the SCC in Göttingen at the GWDG, you can follow these steps if you need to transfer data to/from the Emmy system:
...
Copy a single file FOO from SCC $HOME into your current directory on Emmy
| Codeblock | ||
|---|---|---|
| ||
rsync -e 'ssh -i <fullpath_of_privatekeyfile>' GWDGUSERNAME@transfer-scc.gwdg.de:/usr/users/GWDGUSERNAME/FOO . |
Copy a single file FOO in your current directory on Emmy to $HOME on the SCC
| Codeblock | ||
|---|---|---|
| ||
rsync -e 'ssh -i <fullpath_of_privatekeyfile>' FOO GWDGUSERNAME@transfer-scc.gwdg.de:/usr/users/GWDGUSERNAME/ |
Copy a directory in your SCC /scratch to your current directory on Emmy
| Codeblock | ||
|---|---|---|
| ||
rsync -e 'ssh -i <fullpath_of_privatekeyfile>' -r GWDGUSERNAME@transfer-scc.gwdg.de:/scratch/projects/workshops/forest/synthetic_trees . |
Copy a directory in your current directory on Emmy to /scratch on the SCC
| Codeblock | ||
|---|---|---|
| ||
rsync -e 'ssh -i <fullpath_of_privatekeyfile>' -r synthetic_trees GWDGUSERNAME@transfer-scc.gwdg.de:/scratch/projects/workshops/forest/ |
If you have terrabytes of data that need to be transferred, please contact us so that we can provide a custom solution for this.