|
HELIOS Base addendum |
1.1 Command descriptions1.3 Using "dt" for backup/restore
This chapter describes the "dt tools", which mimic the functionality of most major UNIX commands for handling files, while maintaining the integrity of the desktop database. "dt tools" also provide access to Mac specific file information and streams from a UNIX prompt.
Important: The "dt" program does not consider any file locking and can therefore also copy files in use. In order to avoid damage, consider this and make sure nobody is working concurrently on these particular files before manipulating them with "dt".
If you need to access files which are stored in a HELIOS volume via host scripts or commands, there are a lot of reasons for using the "dt tools".
Note: Host commands means any access under UNIX, e.g. any script or program (like a backup program), which accesses files stored in a HELIOS volume.
Any manipulation of a file or folder inside a HELIOS volume using ordinary UNIX commands like "cp", "mv", "rm" or other programs, will cause inconsistencies between volume information and the related desktop database, and you may lose addtional data, e.g. file streams. Especially restoring files from a backup will cause such an inconsistency.
In the following sections, you will find more information on this problem, and on how the "dt tools" help avoid it.
Files and directories in the Windows/NTFS environment may have a certain number of file streams. File streams contain meta data such as document title, subject, author, etc., similar to the resource fork of a Mac file.
The "dt tools" can handle Windows NTFS file streams. While accessing a file on the network that has been created in the Windows/NTFS environment, e.g. renaming or moving it on a HELIOS server, all pertinent file streams are considered as well.
Also, file attributes such as Archive, Hidden, System, Creation date, label settings, etc. are also preserved when using the "dt tools". In addition, a fast file search can be performed via the desktop database, with file ID support.
Use the utilities whenever you would (usually) use any of the following UNIX commands (if either source and/or destination files/folders are located in a HELIOS volume):
You may use the "dt tools" even if the source or destination is not located in a HELIOS volume, or if you are not sure about this. The "dt tools" automatically select the proper mode of operation, e.g. if source and destination are no volumes "dt" acts exactly like the corresponding UNIX program.
It is required to use the "dt tools" together with EtherShare, PCShare and/or ImageServer. These HELIOS products have improved verification mechanisms to recognize potentially harmful configurations, and will disable access to volumes for which consistency between volume and desktop information cannot be assured. In addition, warning or error messages are logged to the system error log file, which may help you locate the cause of the problem.
Just to help you understand the possible error messages and the specific behavior of the "dt tools", here are the main differences to standard UNIX commands:
- Each Mac file consists of two parts, the so-called "data fork" which is stored as a standard UNIX file, and a corresponding "resource fork" which is stored in an ".rsrc" subdirectory. Additional information for each file is stored in a corresponding entry in a volume based "desktop" database. Therefore, when using UNIX commands, each operation like move, copy, or delete, must consider both the "data fork" and "resource fork" as well as the database entry instead of just one single file.
- Windows files and directories may have additional streams as well.
- When looking at a volume as a UNIX directory you may not notice the differences, because the resource fork subdirectory and the database files are stored as hidden UNIX files.
- The "dt tools" simulate the behavior of standard UNIX commands as close as possible, but there are slight differences in the number and functionality of the available options, and due to the extended functionality you may notice a different behavior and different error messages. For example, you may see error messages about the resource fork of a file, which you would not see when using standard UNIX commands (the UNIX commands - in fact - ignore this part of the file).
- You may see different errors on "dt chmod", "dt chown", and "dt chgrp" because in volumes, you need permissions to change not only the specified file but also the associated resource directory, the resource file and all streams.
- For more information about resource and stream handling errors, you may use the -v (verbose) option.
Before you try and use any of the "dt" commands, you should be well familiar with the corresponding UNIX commands. If you need more information about how the "dt tools" work and how problems arise without the utilities, please read the "Technical notes" following each command description, and 1.2 "Additional information".
We do not recommend to sym-link regular "cp", "mv", "rm" commands to the "dt tools". Although this is possible, you should verify your current and future system environment very carefully, to assure proper operation of your programs, scripts and established workflow.
Especially, you should check whether your site uses special HSM (Hierarchical Storage Management), tape robot, or RAID (Redundant Array of Independent Disks) software which may also use their own versions of these UNIX commands. Take into account that different users and script based programs may make use of different shells and environment settings.
We suggest to check for each HELIOS volume the consistency between the volume information and the volume's desktop database. HELIOS Base offers a rebuild option, namely -s (scan only), which can be used for this purpose (see chapter 7.10 "rebuild" in the Base manual).
Simply issue rebuild -s <volume mount point> for every defined public or private HELIOS volume. Any output of this "scan" indicates a potentially harmful inconsistency between volume and desktop information. You should only continue after having re-synchronized the desktop database by means of an ordinary rebuild.
Take some time to make yourself familiar with the utilities and the way they behave inside a HELIOS volume, between different HELIOS volumes, and between non-HELIOS and HELIOS volumes.
The "dt tools" use standard error messages starting with the program name (including the command argument), followed by the file or directory currently processed, and the error message, e.g.:
The error messages used are similar to the errors issued by the standard UNIX commands. Please note that you may encounter additional error messages regarding resource fork, streams, or desktop database.
In some cases the "dt tools" may issue error messages displaying the full path of a file name instead of the passed relative one. This is due to the fact that the desktop database always needs absolute file names instead of relative ones. In this case, you may see the completely resolved (including symbolic links) absolute path name in the error message.
For the following command descriptions, the knowledge of the functionality of the corresponding UNIX commands is assumed. Please refer to your UNIX manual if you are not familiar with the commands.
If you do not need or want to know, how the "dt tools" operate, you only need to read the main command description for each UNIX command which is emulated by the "dt tools". The technical description following each command is meant for advanced users or system administrators, and explains some details or the special behavior of the respective command.
With very few exceptions, "dt" behaves identically on all platforms supported by HELIOS. Please note that, for this reason, some platform-specific options could not be implemented.
For all commands, the standard UNIX permissions apply when accessing, removing, or overwriting a file.
These files are handled implicitly by "dt" and there is no need to specify them directly. You should never move, copy, or remove these files with other commands, as e.g. the standard UNIX tools. Do not manipulate them at all.
Note: In the following, the term "volume" is used for a directory that is defined as a HELIOS volume which contains a desktop database (in contrast to a simple UNIX "directory"). Likewise, the term "volume root" referes to the top level folder of a HELIOS volume. The "HELIOS Applications" volume, for example, is by default defined to reside in "/usr/local/helios/public", which is its "volume root" directory under UNIX.
"dt" will recognize access to HELIOS volumes from another server only by means of available ".DeskServer" files.
"dt" will process data, resource and file stream parts of files/folders in HELIOS volumes. If no resource information exists, "dt" will be able to create at least minimum information. Especially copy or move operations may require more free space on destination file systems.
Usage: dt { rm | rmdir | mv | cp | set | ls | mkdir | touch | upd | chmod | chown | chgrp | idinfo | iddump }
Calling "dt" with the command argument but no further arguments, prints the usage for the specific command.
(-X option) Usually, the desktop of a volume is kept open for about 30 sec. when being used by "dt" in order to avoid overhead (desktop database completely open/closed). This is because further access is to be expected. However, for certain applications it is not advisable to block the desktop for a volume that long. Hence, with the -X option set, the desktop is closed immediately.
(-y option) Force scanning for streams that are not specified in the resource information of a file. If additional streams exist for a file but the file information is not marked for streams, this option checks the entire ".rsrc" directory if streams are detected for the current file. To avoid unnecessary search operations, the standard resource is marked if additional streams are available.
(-E option) If this option is specified, file events for all registered services are notified to the notification server.
-i Prompt for confirmation for each file. If the
answer begins with y (yes), the file is deleted,
otherwise the file remains on the server.
Note: Missing resource forks are not reported unless the -v verbose option is set.
The volume root directory is automatically touched to announce the changes to any Mac that has mounted the volume.
When removing a directory using the -r option, "dt" automatically tries to remove any orphan resource forks or file streams. The volume root directory may only be removed if this volume is not in use by any other client, either EtherShare, PCShare, WebShare or the "dt tools", otherwise a "directory not empty" error is issued.The "dt rmdir" command will remove the directory entry specified by each dirname operand, provided that this operand refers to an empty directory.
-p Allow users to remove the directory dirname and
its parent directories which become empty. A
message is printed on the standard error about
whether the whole path is removed or part of the
path remains for some reason.
- Rename a file "f1" to the destination file "f2"
- Move the passed files "f1- fn" into the destination directory "d1"
- Move the passed directory into the destination directory "d2". If "d2" does not exist, rename "d1" to "d2".
-f Move the file(s) without prompting even if it is
overwriting an existing target. Note that this is
the default if the standard input is not a terminal.
-i Prompt for confirmation for each file. If the
answer begins with y (yes), the file is moved,
otherwise it remains where it is.
-k Keep ID, try to allocate the source ID and use it
for the destination as well (see
1.3 "Using "dt" for backup/restore"). In case
this is impossible, you will only get a warning
if -v is set. You can use this parameter to
preserve proper working of Mac aliases.
-c Move a file from one HELIOS volume to another
HELIOS volume with a different character set
without converting to the target volume
character set or reporting error messages.
Important: If you do not use the -c option with the "dt mv" and "dt cp" commands, an error message only occurs if you copy/move files from a HELIOS volume to another. Copying/moving from a UNIX directory will not provoke any warning at all.
The behavior of the different UNIX implementations differs if both the -f option and the -i option are set. "dt mv" uses the following rule: if both the -f option and the -i option are set, this is not considered an error; here, the -f option will override the -i option.
Important: If you move a directory between HELIOS volumes, "dt" cannot simply rename just the directory entry. Here, the Mac Finder would have to perform two steps for this operation, namely copying a folder from one volume to the other and then deleting the folder on the source volume. "dt" has a similar task to accomplish. For every file and folder within the directory to move, first the object must be copied to the destination volume and registered in the destination desktop. Then, the moved objects must be deleted from the source volume and unregistered from its desktop database.
This is similar to moving a UNIX directory to a different device, where also all affected files must be copied and then deleted, with the difference that here all resources and streams must be handled, too.Please note that any Mac file/folder to which aliases are pointing will have lost its connection after successfully been moved between volumes. This would be the same if you used the Mac Finder for this operation. If source and target directory are on different file systems, "dt mv" copies the file and deletes the original; any hard links to other files are lost.
Volume root for both source and destination, is automatically touched to announce the changes to any Mac that has mounted the volume.
Usually, if a file is copied to a directory without a desktop (pure UNIX directory), the file ID is set to zero. When using the -k option, the file ID is maintained (if possible).
dt cp [-fikpnvzcyRXE] f1 f2
dt cp [-fikpnvzcyRXE] f1 ... fn d1
dt cp -r [-fikpnvzcyRXE] d1 ... dn-1 dn
- Copy a file "f1" to the destination file "f2"
- Copy the passed files "f1- fn" into the destination directory "d1"
- Copy the passed directories into the destination directory "dn"
-f Copy the file without prompting even if it is
overwriting an existing target. Note that this is
the default if the standard input is not a terminal.
-i Prompt for confirmation for each file. If the
answer begins with y (yes), the file is copied,
otherwise it is not.
-c Copy a file from one HELIOS volume to another
HELIOS volume with a different character set
without converting to the target volume
character set or reporting error messages.
The volume root directory for both source and destination, is automatically touched to announce the changes to any Mac that has mounted the volume.
If your source does not have a resource fork, and your destination is a volume, a default resource fork is created. Please do also take into account that every default resource will require 64 bytes, thus using the minimum disk block size on the destination file system. If there is only little free disk space on the destination volume, first check whether the destination file system can store all of the files. You can roughly calculate 1K (fragment size) per file/folder. Consult your UNIX system administrator or the appropriate pages in your UNIX manual for information on how to retrieve more exact information about your file system settings.
"dt set" modifies the additional file information stored in the file's resource fork. For more information about the flags and fields used here refer to your Mac documentation.
You can use HELIOS Admin to extract the creator/type information an application sets for certain icons. For a description see "Extension Mappings" in the EtherShare manual and the content of "HELIOSDIR/var/conf/suffixes".
-L Set the file to number of allowed concurrent
program launches, where "integer" is the
number of launches.
-C Clear the file ID numbers in the resource
fork (set to zero) and force the "afpsrv" to
allocate new ID numbers.
Please note that you may list these information using
"dt ls". If you need to set type or creator info with non-printable values, you may as well enter these values as octal values with a preceding backslash ("\"), and use the following escape sequences:
Please note that for both type and creator, all characters including 0 are valid. Each passed creator or type must be exactly 4 bytes long.
When you call "dt set" for a file without a resource fork, a default resource fork is created that is updated with the passed parameters, even if the destination file is not located in a HELIOS volume.
Please note that "dt set" does not create a missing ".rsrc" directory; use "dt touch" first on the file name in order to force creation of a related ".rsrc" subdirectory where the file's resource information can be stored.
For each file that is a directory, the directory content is listed; for each file that is an ordinary file, the name and (optional) resource related information are listed. When no argument is given, the current directory is listed.
-a List all entries, including those beginning with a
dot character ("."), which are normally not listed.
Note: Mac aliases cannot be created with the "dt tools". The internal structure of aliases is not documented by Apple.