Introduction#
Rclone is a command-line tool that supports synchronization, uploading, and downloading of data between different object storage and cloud storage services. It also offers practical features such as offline downloading and server backup through various settings. This article will explain some commonly used command parameters in Rclone.
Rclone Configuration#
rclone config - Enter interactive configuration options to add, delete, and manage cloud storage services. For detailed operations, please refer to: "Rclone Installation and Configuration Tutorial"
rclone config file - Display the path of the configuration file. The configuration file is usually located at ~/.config/rclone/rclone.conf
rclone config show - Display the information of the configuration file
Command Syntax#
# Local to cloud storage
rclone [options] <local path> <cloud storage name:path> [parameters] [parameters] ...
# Cloud storage to local
rclone [options] <cloud storage name:path> <local path> [parameters] [parameters] ...
# Cloud storage to cloud storage
rclone [options] <cloud storage name:path> <cloud storage name:path> [parameters] [parameters] ...
Usage Examples#
rclone move -v /Download Onedrive:/Download --transfers=1
Common Options#
rclone copy- Copy filesrclone move- Move files. If you want to delete empty source directories after moving, add the--delete-empty-src-dirsparameter.rclone sync- Synchronize: synchronize the source directory to the target directory, only changing the target directory.rclone size- View the size of files in the cloud storage.rclone delete- Delete the contents of a path.rclone purge- Delete the path and all its contents.rclone mkdir- Create a directory.rclone rmdir- Delete a directory.rclone rmdirs- Delete empty directories in the specified path. If the--leave-rootparameter is added, the root directory will not be deleted.rclone check- Check if the data in the source and destination addresses match.rclone ls- List all files, file sizes, and paths in the specified path.rclone lsl- Similar to the above, but also displays the upload time.rclone lsd- List directories in the specified path.rclone lsf- List directories and files in the specified path.
Common Parameters#
-n=--dry-run- Test run to see which operations rclone will perform.-P=--progress- Display real-time transfer progress, refresh every 500ms, or every 1 minute by default.--cache-chunk-size SizeSuffix- Size of each chunk, default is 5M. In theory, a larger chunk size will result in faster upload speed but also higher memory usage. Setting it too large may cause the process to be interrupted.--cache-chunk-total-size SizeSuffix- Total size of chunks that can be used on the local disk, default is 10G.--transfers=N- Number of parallel file transfers, default is 4. It is recommended to reduce this parameter on VPS with limited memory, such as a 128M VPS, where it is recommended to set it to 1.--config string- Specify the path of the configuration file, wherestringis the path of the configuration file.--ignore-errors- Skip errors. For example, when OneDrive encounters certain special files and prompts "Failed to copy: failed to open source object: malwareDetected: Malware detected", it will terminate subsequent transfer tasks. In this case, you can add this parameter to skip errors. However, please note that the exit status code of RCLONE will not be0.
Logging#
rclone has 4 levels of logging: ERROR, NOTICE, INFO, and DEBUG. By default, rclone generates ERROR and NOTICE level messages.
-q- rclone only generatesERRORmessages.-v- rclone generatesERROR,NOTICE, andINFOmessages. Recommended.-vv- rclone generatesERROR,NOTICE,INFO, andDEBUGmessages.--log-level LEVEL- Control the log level.
Output Log to File#
Use the --log-file=FILE option, rclone will redirect Error, Info, and Debug messages as well as standard errors to FILE, where FILE is the path of the log file you specify.
Another method is to use system redirection commands, for example:
rclone sync -v Onedrive:/DRIVEX Gdrive:/DRIVEX > "~/DRIVEX.log" 2>&1
File Filtering#
--exclude - Exclude files or directories.
--include - Include files or directories.
--filter - File filtering rules, alternative usage of the above two options. Include rules start with +, exclude rules start with -.
File Type Filtering#
For example, --exclude "*.bak", --filter "- *.bak", exclude all bak files. It can also be written as:
For example, --include "*.{png,jpg}", --filter "+ *.{png,jpg}", include all png and jpg files, exclude other files.
--delete-excluded - Delete excluded files. This option needs to be used with filtering parameters, otherwise it will be invalid.
Directory Filtering#
To filter directories, add / after the directory name, otherwise it will be treated as a file. Starting with / will only match the root directory (under the specified directory), otherwise it will match all directories. This also applies to files.
--exclude ".git/" - Exclude the .git directory in all directories.
--exclude "/.git/" - Only exclude the .git directory in the root directory.
--exclude "{Video,Software}/" - Exclude the Video and Software directories in all directories.
--exclude "/{Video,Software}/" - Only exclude the Video and Software directories in the root directory.
--include "/{Video,Software}/**" - Only include all contents of the Video and Software directories in the root directory.
File Size Filtering#
The default size unit is kBytes, but you can use the k, M, or G suffix.
--min-size - Filter files smaller than the specified size. For example, --min-size 50 means files smaller than 50k will not be transferred.
--max-size - Filter files larger than the specified size. For example, --max-size 1G means files larger than 1G will not be transferred.
TIPS: The author found that the two size filtering options cannot be used simultaneously in practical use.
Filtering Rule File#
--filter-from <rule file> - Add inclusion/exclusion rules from a file. For example, --filter-from filter-file.txt.
Example of a filtering rule file:
- secret*.jpg
+ *.jpg
+ *.png
+ file2.avi
- /dir/Trash/**
+ /dir/**
- *
These are just some commonly used and simple filtering methods. For more complex and advanced usage, please refer to the official documentation.
Environment Variables#
Each option in rclone can be set through environment variables. The name of the environment variable can be converted from the long option name by removing the -- prefix, changing - to _, capitalizing it, and adding the prefix RCLONE_. The priority of environment variables is lower than that of command-line options. That is, when the corresponding option is appended on the command line, it will override the value set by the environment variable.
For example, to set the minimum upload size to --min-size 50, the environment variable is RCLONE_MIN_SIZE=50. When the environment variable is set and --min-size 100 is used on the command line, the value of the environment variable will be overridden.
Common Environment Variables#
RCLONE_CONFIG- Custom configuration file pathRCLONE_CONFIG_PASS- If rclone has encryption settings, set this environment variable to the password to automatically decrypt the configuration file.RCLONE_RETRIES- Number of retries for failed uploads, default is 3 times.RCLONE_RETRIES_SLEEP- Waiting time between failed upload retries, disabled by default. Unitss,m,hrepresent seconds, minutes, and hours respectively.CLONE_TRANSFERS- Number of parallel file uploads.RCLONE_CACHE_CHUNK_SIZE- Size of each chunk, default is 5M. In theory, a larger chunk size will result in faster upload speed but also higher memory usage. Setting it too large may cause the process to be interrupted.RCLONE_CACHE_CHUNK_TOTAL_SIZE- Total size of chunks that can be used on the local disk, default is 10G.RCLONE_IGNORE_ERRORS=true- Skip errors.