Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)
Posted: Tue Jun 04, 2024 8:20 am
Before we start i expect you to have the basic knowledge of rclone and what it is.
1) First we need to create a rclone config file, the below example can be copied as a template:
In the first part of the template you have to change the following settings:
For this guide i will simply use rcloneblomp.conf
I’m a linux user, so some flags might not work on windows / osx, please refer to the rclone documentation to see what works and what does not work depending on your OS --> https://rclone.org/flags/
Now for simplicity i have my folder and configuration file in the same directy to keep it simple
config: rcloneblomp.conf
foldername: blompmount
mount command:
Explaining all the flags:
The rcloune mount is called:
1) First we need to create a rclone config file, the below example can be copied as a template:
Code: Select all
[blomp]
type = swift
user = changeme@email.com
key = changemePassword
auth = https://authenticate.ain.net
tenant = storage
auth_version = 2
endpoint_type = public
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false
[blomp_encrypted]
type = crypt
remote = blomp:changeme@email.com
password = changemePassword
[blomp_chunker]
type = chunker
remote = blomp_encrypted:
hash_type = none
name_format = *.#####
transactions = norename
chunk_size = 4800Mi
The second part we have to change is the following:[blomp]
user = changeme@email.com --> change this to your e-mail used with blomp
key = changemePassword --> change this to your password used with blomp
The third part is a chunker, chunker is only needed if you upload files bigger than 5GB as the SWIFT protocol does not allow file size above 5GB.[blomp_encrypted]
remote = blomp:changeme@email.com --> Change this again but leave the “blomp:” part.
password = changemePassword → Change this to a encrypted password (dont use special characters)
Now save the changes to a file with the extension .conf[blomp_chunker]
name_format = *.##### --> The # (hashtag) is the amount of numers behind a file (example filename.zip.00001 filename.zip.00002 etc.)
chunk_size = 4800Mi --> this can be changed to any file size below 5GB, for safety use 4800Mi (MB)
For this guide i will simply use rcloneblomp.conf
I’m a linux user, so some flags might not work on windows / osx, please refer to the rclone documentation to see what works and what does not work depending on your OS --> https://rclone.org/flags/
Now for simplicity i have my folder and configuration file in the same directy to keep it simple
config: rcloneblomp.conf
foldername: blompmount
mount command:
Code: Select all
rclone mount blomp_chunker: blompmount/ --config rcloneblomp.conf --allow-other --vfs-cache-mode=writes --poll-interval 0 --dir-cache-time 1h --use-server-modtime --no-checksum --vfs-fast-fingerprint --vfs-cache-max-size=5G --vfs-read-chunk-size=16M --vfs-read-chunk-size-limit=1G --buffer-size=16M --transfers 1 --vfs-disk-space-total-size 10T -v
The rcloune mount is called:
This refers to where the config file is we just created.blomp_chunker:
Allowing other users / applications to use the mount.--config xxx
I picked writes, as off, minimal and full kept crashing the mount, also Swift does not support off, minimal at all.--allow-other
Polling is not supported on SWIFT protocol --> 0 = disabled.--vfs-cache-mode writes
How long folder structure should be cached before refreshing, you can set it up to 8700h (1year ish.)--poll-interval 0
Needed as SWIFT protocol does not support support reading local modtime--dir-cache-time 1h
Needed as SWIFT protocol does not support checksum data.--use-server-modtime
Just show/read the file, should be quicker on SWIFT protocol according to rclone (optional)--no-checksum
The amount of data cached on your hard drive for quick access. If files are bigger than 5GB it will use more than 5GB, after closing a file unused cache will be deleted.--vfs-fast-fingerprint
Read any file in parts of 16MB increasing each read (16/32/64 etc)--vfs-cache-max-size 5G
Max read chunk size is 1GB, so starting at 16/32/64 etc all the way up to 1024 and then keeps grabbing 1GB)--vfs-read-chunk-size=16M
Try putting read-chunk-size into RAM memory instead of HDD memory.--vfs-read-chunk-size-limit=1G
This affects uploads, not downloads, SWIFT only supports 1 for upload. (if you change this and you use the -vv flag you will get many errors, it will keep retrying but will still only upload 1 file and then the next).--buffer-size=16MB
This changed the 1P space to the 10TB, change this number to your subscription, this is just a visual update (optional)--transfers 1
--vfs-disk-space-total-size 10T
We have a green light from the tech team that the issue is fixed.