File Synchronization Operations

Azure Data Expert provides advanced file synchronization operations allowing one to sync files from a local disk or directory to a Microsoft Azure cloud storage container. The user is provided with the ability to display a file synchronization preview dialog, use a number of different file synchronization modes and various performance optimization options.

Azure Data Expert File Synchronization

In order to perform a simple file synchronization operation, press the 'Sync' button located on the main toolbar, select a source directory, select a destination storage account, select a destination storage container and press the 'Start' button. In addition, advanced users are provided with the ability to control the file synchronization mode and display the file synchronization preview dialog.

Azure Data Expert File Synchronization Options

For more advanced file synchronization options, press the 'Advanced Options' button located on the file synchronization operation dialog. The 'Advanced' tab allows one to control the file synchronization performance mode, the number of parallel file synchronization streams and a number of file synchronization fault-tolerance configuration options. The 'Exclude' tab provides the ability to exclude one or more subdirectories from the file synchronization process. The 'Rules' tab allows one to specify which types of files should be synchronized using one or more flexible file matching rules.

Select File Synchronization Mode

Azure Data Expert provides a number of different one-way and two-way file sync modes allowing one to configure file synchronization operations according to user-specific needs and requirements. In order to select an appropriate file synchronization mode, open the file synchronization command dialog and click on the 'Synchronization Mode' combo box.

Azure Data Expert File Synchronization Modes
  • Synchronize Destination Container (one-way) - All changes made in the source directory will be propagated to the destination storage container. Files deleted from the source directory will be deleted from the destination storage container. All files changed or deleted in the destination storage container will be restored using files from the source directory. No changes will be made in the source directory. After the file synchronization process is complete both locations will be identical.
  • Update Destination Container (one-way) - Newly created and modified source files will be copied to the destination storage container. Files deleted from the source directory will be deleted from the destination storage container. Files deleted from the destination storage container will be restored using files from the source directory. Newly created and modified destination files will be kept in place. No changes will be made in the source directory.
  • Accumulate and Update Files in Destination (one-way) - Newly created and modified source files will be copied to the destination storage container. Files deleted from the destination storage container will be restored using files from the source directory. Newly created and modified destination files will be kept in place. No files will be deleted from the destination storage container. No changes will be made in the source directory.
  • Two-Way File Synchronization - Newly created and modified files will be synchronized in both directions. Files deleted from the source directory will be deleted from the destination storage container. Files deleted from the destination storage container will be restored from the source directory. Changes may be made in both locations. After the file synchronization process is complete both locations will be identical.
  • Two-Way Accumulation and Update - Newly created and modified files will be updated in both directions. Files deleted from one location will be restored from the second location. Changes may be made in both locations. After the file synchronization process is complete both locations will be identical.
Synchronize Files With Preview

Azure Data Expert provides an optional file synchronization preview dialog capable of showing all file synchronization actions that should be performed and allowing one to explicitly select which file sync actions to perform and which file sync actions to skip during the file synchronization process. By default, in order to prevent accidental errors, file synchronization operations are performed with the file sync preview dialog enabled.

Azure Data Expert File Synchronization Preview

When the file synchronization preview dialog is enabled, Azure Data Expert will scan the source directory and the destination storage container, compare files stored in the source directory with data blobs stored in the destination storage container and display a list of file synchronization actions that should be performed according to the currently selected file synchronization mode.

Azure Data Expert File Sync Preview Actions

The user is provided with the ability to select/unselect specific file synchronization actions and export the list of sync actions to the HTML, text and Excel CSV report formats. Once finished reviewing the list of file synchronization actions, press the 'Sync' button located on the file synchronization preview dialog and the selected file sync actions will be performed according to the currently selected file synchronization mode.

Control File Synchronization Performance

By default, Azure Data Expert performs file synchronization operations at the maximum possible speed, but sometimes, when the Internet bandwidth is required for a high priority task, it may be required to intentionally slow-down a file synchronization operation. In order to dynamically slow down a running file synchronization operation, open the file synchronization process dialog, click on the 'Performance Mode' combo box and select an appropriate file synchronization performance mode.

Azure Data Expert File Sync Process

A running file synchronization operation may be switched between different performance modes without any limitations allowing one to dynamically free additional Internet bandwidth for high-priority tasks. In addition, the user is provided with the ability to permanently configure a file synchronization operation to use less Internet bandwidth, which may be very useful for low-priority backup tasks.

Azure Data Expert File Sync Performance

In order to permanently slow down a file synchronization operation, open the file synchronization command dialog, press the 'Advanced Options' button, select the 'Advanced' tab, click on the 'Performance Mode' combo box and select an appropriate file synchronization performance mode. During the file synchronization process, Azure Data Expert will execute the file synchronization operation according to the selected file synchronization performance mode.

Synchronizing Specific Types of Files

Azure DEX Pro provides the ability to synchronize specific types of files or groups of files by the file type, category, size, name, extension, last modification date, etc. The user is provided with the ability to configure a number of file matching rules for each file synchronization operation allowing one to synchronize the required files and precisely focus on the required data. For example, the user can synchronize all types of images with the file size more that 10 MB that were modified during the last month.

Azure Data Expert File Synchronization Rules

In order to configure one or more file matching rules for a file synchronization operation, open the file synchronization command dialog, press the 'Options' button, select the 'Rules' tab, press the 'Add' button and select the required rule type. During the file synchronization process, Azure Data Expert will evaluate the existing files and data blobs using the specified file matching rules. Files and data blobs matching the specified rules will be synchronized and files not matching the specified rules will be excluded from the file synchronization process.

Azure Data Expert Negative File Matching Rules

In addition to positive file matching rules allowing one to include specific files in the file synchronization process, Azure Data Expert provides negative file matching rules allowing one to exclude specific files from the file synchronization process. For example, in order to exclude all types of images from the file synchronization process, add a file category rule, select the 'Images, Pictures and Graphic Files' file category and select the 'Not Categorized' rule operator. During the file synchronization process, Azure Data Expert will evaluate the processed files and skip all types of image files.

Excluding Directories From the File Synchronization Process

Sometimes, it may be required to exclude one or more subdirectories from the file synchronization process. For example, if you need to sync files from a disk to a storage container excluding one or two special directories, you may specify the entire disk as the source directory and add the directories that should be skipped to the exclude list.

Azure Data Expert File Sync Exclude Directories

In order to add one or more directories to the exclude list, open the file synchronization command dialog, press the 'Options' button, select the 'Exclude' tab and press the 'Add' button. All files and subdirectories located in the specified exclude directory will be excluded from the file synchronization process. In addition, advanced users are provided with a number of exclude directories macro commands allowing one to exclude multiple directories using a single macro command.

Azure Data Expert provides the following exclude directories macro commands:

  • $BEGINS <Text String> - this macro command excludes all directories beginning with the specified text string.
  • $CONTAINS <Text String> - this macro command excludes all directories containing the specified text string.
  • $ENDS <Text String> - this macro command excludes directories ending with the specified text string.
  • $REGEX <Regular Expression> - this macro command excludes all directories matching the specified regular expression.

For example, the exclude macro command '$CONTAINS Temporary Files' will exclude all directories with 'Temporary Files' at any place in the full directory path and the exclude macro command '$REGEX \.(TMP|TEMP)$' will exclude all directories ending with '.TMP' or '.TEMP'.

Save File Synchronization Logs

Azure Data Expert allows one to save various types of file synchronization logs. In order to enable file synchronization logs, open the file synchronization command dialog, press the 'Advanced Options' button and select the 'Misc' tab. The user is provided with the ability to save file synchronization errors log files, file synchronization tasks log files and complete file synchronization actions log files, which may be very large when synchronizing many thousands of files.

Azure Data Expert File Sync Log Files

For all types of log files, the user needs to specify an existing directory where the selected log files will be saved with automatically generated file names. By default, Azure Data Expert will save a new log file for each sync operation, add the file sync operation date and time to the log file name and keep a history of the last 10 log files in the specified directory. In order to customize a log file, press an appropriate 'Advanced Log Options' button.

Azure Data Expert File Sync Log Files Options

The 'Advanced Log Options' dialog allows one to control the log files generation mode, the log files format and the number of history log files to keep in the specified log directory. In addition to separated file synchronization log files, which are saved for each file sync operation, the user can save consolidated hourly, daily and monthly log files. Also, the user is provided with the ability to save log files in the Excel CSV format, which can be used to analyze log files using the Microsoft Excel application.

Send File Sync Error Notifications

For mission-critical systems, it may be required to send E-Mail notifications about failed file synchronization operation and/or status notifications about successfully completed file synchronization operations. Azure DEX Ultimate provides the ability to enable error and/or status E-Mail notifications for any file synchronization operation. In order to enable E-Mail notifications for a file synchronization command, open the file sync command dialog, press the 'Advanced Options' button, select the 'Misc' tab, select an appropriate notification type and specify an E-Mail address to send notifications to.

Azure Data Expert File Sync Send Notifications

In addition, open the main 'Options' dialog, select the 'E-Mail' tab and specify an E-Mail server, user name and password to use to send E-Mail notifications. Error E-Mail notifications are sent when a file synchronization command is failed or completed with one or more file synchronization errors. A status E-Mail notification always sent after each file synchronization operation disregarding of the status of the file sync operation even when no files where synchronized. Each E-mail notification includes a summary table showing the total number of synchronized files, the number of file synchronization errors, additional statistical information about the file synchronization operation and a list of file sync errors.

Azure Data Expert File Sync Notifications Options

In order to customize an E-Mail notification, press an appropriate 'Notification Options' button. The 'Notification Options' dialog provides the ability to change the format of E-Mail notifications (HTML or text) and set a user-custom subject for E-Mail notifications. If a custom E-Mail subject is not set, Azure Data Expert will generate a default E-Mail subject, which will include the name and the status of the file synchronization command. If a custom E-Mail subject is set, the custom subject will be used 'as is' without any changes.