Summary

Btrfs is a feature-rich copy-on-write file system that has been around since 2007, it is the default file system for some popular Linux distributions such as Fedora and is used by many large companies such as Meta in their datacenters. Over the years it has proven itself to be stable and more flexible than other popular file systems. Most significant features are the following:

• Birot protection (when used in Btrfs-RAID mode)
• Subvolumes
• Snapshots
• Transfer

Examples

Two disk Btrfs-RAID (RAID1)

In case you want to create a reliable data storage solution with just two disks (for example, NAS), then you can follow this these steps to successfully create a RAID1 array with Btrfs:

1. Find the names of the disks you want to use for setting up a Btrfs-RAID:
   

   /disk print

   In this example, the disks used are going to be called <disk-name-1>  and <disk-name-2> , make sure you replace these placeholders with your actual disk names!

2. Format one of your disks to Btrfs, in this case <disk-name-1> :
   

   /disk format <disk-name-1> file-system=btrfs

3. You can check the current status of Btrfs disks using the following command:
   

   /disk/btrfs/filesystem print

   
   
   In case there was an existing RAID on your disks and you want to remove any obsolete configuration for simplicity, then you can wipe the disks with /disk format-drive <disk-name-x> file-system=wipe-quick . This is useful when you have unwanted entries under /disk/btrfs/filesystem print 
4. Add a label for the Btrfs drive for simplicity:
   

   /disk/btrfs/filesystem set [find where present-devs=<disk-name-1>] label=BtrfsRAID

5. Add a disk to an existing Btrfs disk:
   

   /disk/btrfs/filesystem/add-device [find where present-devs=<disk-name-1>] device=<disk-name-2>

6. Start Btrfs balance process to create a Btrfs-RAID with RAID1 configuration:
   

   /disk/btrfs/filesystem balance-start [find where label=BtrfsRAID] data-profile=raid1 metadata-profile=raid1 system-profile=raid1

7. IMPORTANT: Double check that all Btrfs profiles for the disks match, in this case you need to make sure that the data , meta  and system profile is raid1
   

   /disk/btrfs/filesystem/print

   If you notice, for example, in the output data,single:1 , then you will need to execute the /disk/btfs/filesystem balance-start command again.

   The desired state is when the output is similar to the one below with Btrfs profile for data , system and meta set to raid1 :

   > /disk/btrfs/filesystem/print
   ...
   data,raid1:1.07GB disk1:1.07GB disk2:1.07GB, used:0%                                                                                                       
   system,raid1:33.6MB disk1:33.6MB disk2:33.6MB, used:0%                                                                                                     
   meta,raid1:1.07GB disk1:1.07GB disk2:1.07GB, used:0%    

   
   

   Use the BtrfsRAIDCheck script to warn you about Btrfs profile inconsistencies!

8. Set mount-filesystem=no for second disk to prevent the files showing up twice:
   

   /disk set <disk-name-2> mount-filesystem=no

   
   

   Btrfs has feature that allows you to mount any of the Btrfs-RAID members and you will still be able to access the whole Btrfs-RAID array. A downside of this feature is that if you have mounted the same array twice, then your files will appear twice as well. To prevent this, you can simply disable mounting one of the Btrfs-RAID members automatically.

9. You can also change the mount point's name for simplicity:
   

   /disk set <disk-name-1> mount-point-template=BtrfsRAID

10. Your newly created Btrfs-RAID array is now accessible through /BtrsRAID/ folder.

With a reliable storage solution such as Btrfs-RAID consider adding useful features to your RouterOS device by following the suggested guides below:

• running your own Containers
• DLNA Media Server
• Samba server
• NFS server

Creating Btrfs-RAID check

It is extremely important for you to monitor the Btrfs-RAID array's health. One way you can do this is by using the script below as a working example:

/system scheduler
add interval=1w30s name=BraidBalanceStartCall on-event=BraidBalanceStart policy=ftp,read,write,policy,test,sniff start-date=1970-01-01 start-time=01:00:00
add interval=1w30s name=BraidScrubStartCall on-event=BraidScrubStart policy=ftp,read,write,policy,test,sniff start-date=1970-01-01 start-time=02:00:00
add interval=2m name=BraidBalanceStatusCheckCall on-event=BraidBalanceStatus policy=ftp,read,write,policy,test,sniff start-time=startup
add interval=2m name=BraidScrubStatusCheckCall on-event=BraidScrubStatus policy=ftp,read,write,policy,test,sniff start-time=startup
add interval=30s name=BraidHealthCheckCall on-event=BraidHealthCheck policy=ftp,read,write,policy,test,sniff start-time=startup
add interval=2m name=BraidReplaceStatusCheckCall on-event=BraidReplaceStatus policy=ftp,read,write,policy,test,sniff start-time=startup

/system script
add dont-require-permissions=no name=BraidScrubStart owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfsscrubstatuscheck;\
    \nif (\$btrfsscrubstatuscheck != \"started\") do={\
    \n  :set \$btrfsscrubstatuscheck (\$btrfsscrubstatuscheck \"started\");\
    \n  foreach i in=[/disk/btrfs/filesystem/find] do={ /disk/btrfs/filesystem/scrub-start \$i;\
    \n    :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n    /log info message=\"INFO: Btrfs scrub process started on  \$temp\";\
    \n    :delay 3; \
    \n  }\
    \n}\
    \n"
add dont-require-permissions=no name=BraidBalanceStart owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfsbalancestatuscheck;\
    \nif (\$btrfsbalancestatuscheck != \"started\") do={\
    \n:set \$btrfsbalancestatuscheck (\$btrfsbalancestatuscheck \"started\");\
    \n:local percentage;\
    \n:set \$percentage (\$percentage 50);\
    \n  foreach i in=[/disk/btrfs/filesystem/find] do={ /disk/btrfs/filesystem/balance-start data-usage=\$percentage \$i;\
    \n    :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n    /log info message=\"INFO: Btrfs balance process started on  \$temp\";\
    \n    :delay 3; \
    \n  }\
    \n}"
add dont-require-permissions=no name=BraidBalanceStatus owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfsbalancestatuscheck;\
    \nif (\$btrfsbalancestatuscheck = \"started\") do={\
    \n:local arraycnt [/disk/btrfs/filesystem/print count-only as-value];\
    \n:local counter [:set \$counter (\$counter 0)];\
    \n:local counterdiff [:set \$counterdiff (\$counterdiff 0)];\
    \n  foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n    :local barray [ /disk btrfs filesystem/get value-name=balance-status  \$i;]\
    \n    :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n    if ( \$barray != \"done\" and \$btrfsbalancestatuscheck = \"started\") do={\
    \n        /log info message=\"INFO: Btrfs current balance status on  \$temp is \$barray\";\
    \n    }\
    \n    if ( \$barray = \"done\" and \$btrfsbalancestatuscheck = \"started\") do={\
    \n      :set \$counter (\$counter +1);\
    \n      :set \$counterdiff (\$arraycnt - \$counter);\
    \n      if (\$counterdiff =1) do={\
    \n        /log info message=\"INFO: Btrfs balancing already done on \$counter arrays\";\
    \n      }\
    \n    }\
    \n    if ( \$counter = \$arraycnt) do={\
    \n      /log info message=\"INFO: Btrfs array balance status on  \$temp is \$barray \";\
    \n      :set \$btrfsbalancestatuscheck (\$btrfsbalancestatuscheck \"done\");\
    \n    }\
    \n  } \
    \n}"
add dont-require-permissions=no name=BraidScrubStatus owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfsscrubstatuscheck;\
    \nif (\$btrfsscrubstatuscheck = \"started\") do={\
    \n:local arraycnt [/disk/btrfs/filesystem/print count-only as-value];\
    \n:local counter [:set \$counter (\$counter 0)];\
    \n:local counterdiff [:set \$counterdiff (\$counterdiff 0)];\
    \n  foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n    :local barray [ /disk btrfs filesystem/get value-name=scrub-status  \$i;]\
    \n    :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n    if ( \$barray != \"done\" and \$btrfsscrubstatuscheck = \"started\") do={\
    \n      /log info message=\"INFO: Btrfs current scrub status on  \$temp is \$barray\";\
    \n    }\
    \n    if ( \$barray = \"done\" and \$btrfsscrubstatuscheck = \"started\") do={\
    \n      :set \$counter (\$counter +1);\
    \n      :set \$counterdiff (\$arraycnt - \$counter);\
    \n      if (\$counterdiff =1) do={\
    \n        /log info message=\"INFO: Btrfs scrubbing already done on \$counter arrays\";\
    \n      }\
    \n    }\
    \n    if ( \$counter = \$arraycnt ) do={\
    \n      /log info message=\"INFO: Btrfs array scrub status on  \$temp is \$barray \";\
    \n      :set \$btrfsscrubstatuscheck (\$btrfsscrubstatuscheck \"done\");\
    \n    }\
    \n  } \
    \n}"
add dont-require-permissions=no name=BraidReplaceStatus owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfsreplacestatuscheck;\
    \nif (\$btrfsreplacestatuscheck = \"started\") do={\
    \n:local arraycnt [/disk/btrfs/filesystem/print count-only as-value];\
    \n:local counter [:set \$counter (\$counter 0)];\
    \n:local counterdiff [:set \$counterdiff (\$counterdiff 0)];\
    \n  foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n    :local barray [ /disk btrfs filesystem/get value-name=replace-status  \$i;]\
    \n    :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n    :local multipleprofiles [ /disk btrfs filesystem/get value-name=spaces \$i; ]\
    \n    if ( \$barray ~ \"working\" and \$btrfsreplacestatuscheck = \"started\") do={\
    \n      /log info message=\"INFO: Btrfs current replace status on  \$temp is \$barray\";\
    \n    }\
    \n    if ( \$barray ~ \"done\" and \$btrfsreplacestatuscheck = \"started\" and \$multipleprofiles~\"single\" ) do={\
    \n      /log info message=\"INFO: Braid balance after replace-device  started on  \$temp\";\
    \n         if (\$btrfsbalancestatuscheck!=\"started\") do={\
    \n             /disk/btrfs/filesystem/balance-start \$temp;\
    \n             :set \$btrfsbalancestatuscheck (\$btrfsbalancestatuscheck \"started\");\
    \n         }\
    \n    }\
    \n    if ( \$barray = \"done\" and \$btrfsreplacestatuscheck = \"started\") do={\
    \n      :set \$counter (\$counter +1);\
    \n      :set \$counterdiff (\$arraycnt - \$counter);\
    \n      if (\$counterdiff =1) do={\
    \n        /log info message=\"INFO: Btrfs replace already done on \$counter arrays\";\
    \n      }\
    \n    }\
    \n    if ( \$counter = \$arraycnt ) do={\
    \n      /log info message=\"INFO: Btrfs array replace status on  \$temp is \$barray \";\
    \n      :set \$btrfsreplacestatuscheck (\$btrfsreplacestatuscheck \"done\");\
    \n    }\
    \n  } \
    \n}\
    \n:set \$btrfsreplacestatuscheck (\$btrfsreplacestatuscheck \"started\");"
add dont-require-permissions=no name=BraidHealthCheck owner=admin policy=ftp,read,write,policy,test,sniff source="foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n:local sysadmin; \
    \n\
    \n:set  \$sysadmin \"<servername@domain.tld>\";\
    \n\
    \n:local temp [ /disk/btrfs/filesystem/get value-name=label \$i;]\
    \n:local haserror [/disk/btrfs/filesystem/get value-name=about \$i; ]\
    \n:local hasmissing [ /disk/btrfs/filesystem/get value-name=devs \$i; ]\
    \n:local hasmultiprofile [ /disk/btrfs/filesystem/get value-name=spaces \$i; ]\
    \n:local replacestatus [ /disk/btrfs/filesystem/get value-name=replace-status \$i; ]\
    \n:local multiplediskarray [:len [/disk btrfs filesystem find where label=\$temp and dev-ids~\"2\"];]\
    \n\
    \n  if ( \$hasmissing~\"missing\" and ([:len \$replacestatus]=0)) do= {\
    \n    /log info message=\"ERROR: BtrfsHealthCheck found missing array member on \$temp\";\
    \n    /tool e-mail send to= \$sysadmin  subject=([/system identity get name].\" BtrfsHealthCheck found missing array member\") body=(\"Btrfs array where found missing array member on \" .\$temp . \"   \");\
    \n   :delay 19; \
    \n  }\
    \n\
    \n  if ( \$multiplediskarray > 0 and \$hasmultiprofile~\"single\") do= {\
    \n    /log info message=\"ERROR: BtrfsHealthCheck found multiprofile on \$temp array. To start balance process, run  /disk btrfs filesystem balance-start \$temp command \";\
    \n    /tool e-mail send to= \$sysadmin  subject=([/system identity get name].\" BtrfsHealthCheck found multiprofile on \" .\$temp. \" array.\") body=(\"Btrfs array where found with multiprofile status on \" .\$temp . \"To start balance process, run once\
    \_ /disk btrfs filesystem balance-start \" .\$temp. \" command \");\
    \n   :delay 19; \
    \n  }  \
    \n\
    \n  if (([:len \$haserror]) > 0 ) do= {\
    \n    /log info message=\"ERROR: BtrfsHealthCheck found errors on \$temp\";\
    \n    /tool e-mail send to= \$sysadmin  subject=([/system identity get name].\" BtrfsHealthCheck found errors\") body=(\"Btrfs array where found errors on \" .\$temp . \"   \");\
    \n    :delay 20; \
    \n  }\
    \n}\
    \n"

You also need to adjust e-mail server settings on your RouterOS device:

/tool e-mail
set from=<raidcheck@domain.tld> port=587 server=smtp.domain.com tls=starttls

Creating subvolumes

The main benefit of creating subvolumes is to organize data on your Btrfs main (root) subvolume. Consider subvolumes as folders with features of a partition, while still sharing the total disk space between all subvolumes. You can later use these subvolumes for much more advanced tasks and it is recommended to create subvolumes when you have large amounts of different types of data, especially that requires frequent backups. Follow the guide to setup a few example subvolumes:

1. Find the disk name of your disk that you want to use as your Btrfs disk:
   

   /disk print

   
   

   In this example, the disk used is going to be called <disk-name-1> , make sure you replace the placeholder with your actual disk name!

2. Format the disk to Btrfs, in this case <disk-name-1> :
   

   /disk format <disk-name-1> file-system=btrfs

3. Add a label for the Btrfs disk for simplicity:
   

   /disk/btrfs/filesystem set [find where present-devs=<disk-name-1>] label=BtrfsDisk

4. You can also change the Btrfs disk mount point for simplicity:
   

   /disk set <disk-name-1> mount-point-template=BtrfsDisk

5. Create a new subvolume called Documents to BtrfsDisk :
   

   /disk/btrfs/subvolume/add name=Documents fs=BtrfsDisk

   Subvolumes are also snapshots. You might encounter both of these names in various menus. In simple terms, snapshots are subvolumes created at a specific time and contains data from that time point.

6. Create another subvolume called Photos to BtrfsDisk :
   

   /disk/btrfs/subvolume/add name=Photos fs=BtrfsDisk

7. You can view the currently available subvolumes:
   

   /disk/btrfs/subvolume/print

8. You can now access these subvolumes as /BtrfsDisk/Documents and /BtrfsDisk/Photos 

Creating snapshots

Snapshots are space efficient way to create backups for your data. By creating a snapshot you save the current state of your data which you can later access.

1. Create subvolumes (or use the root subvolume, see above) and put data inside these subvolumes, for example:
   

   [admin@MikroTik] > /file print 
    # NAME   
    0 BtrfsDisk
    1 BtrfsDisk/Documents
    2 BtrfsDisk/Photos 
    3 BtrfsDisk/Documents/document1.txt
    4 BtrfsDisk/Photos/photo1.jpg

2. In this example, we have created /Btrfs/Documents and /Btrfs/Photos/ subvolumes and you can view them with:
   

   /disk/btrfs/subvolume/print

3. In order to make snapshots more organized, create a new subvolume called Snapshots :
   

   /disk/btrfs/subvolume/add name=Snapshots fs=BtrfsDisk

4. Create a snapshot for Documents and Photos :
   

   /disk/btrfs/subvolume/add read-only=yes parent=Documents fs=BtrfsDisk  name=Snapshots/Documents-22012025
   /disk/btrfs/subvolume/add read-only=yes parent=Photos fs=BtrfsDisk  name=Snapshots/Photos-22012025

5. You should now have new subvolumes that are read-only and contain your files:
   

   [admin@MikroTik] > /file print 
    # NAME      
    0 BtrfsDisk
    1 BtrfsDisk/Documents
    2 BtrfsDisk/Photos
    3 BtrfsDisk/Snapshots
    4 BtrfsDisk/Documents/document1.txt
    5 BtrfsDisk/Photos/photo1.jpg
    6 BtrfsDisk/Snapshots/Documents-22012025
    7 BtrfsDisk/Snapshots/Photos-22012025
    8 BtrfsDisk/Snapshots/Documents-22012025/document1.txt
    9 BtrfsDisk/Snapshots/Photos-22012025/photo1.jpg

6. For testing purposes, you can add more data to your subvolumes and you should notice that newly added files do not appear in the snapshots, but only in the subvolumes:
   

   [admin@infra1.mikrotikls.lv] > /file print 
    # NAME   
    0 BtrfsDisk
    1 BtrfsDisk/Documents
    2 BtrfsDisk/Photos
    3 BtrfsDisk/Snapshots
    4 BtrfsDisk/Documents/document1.txt
    5 BtrfsDisk/Documents/document2.txt
    6 BtrfsDisk/Photos/photo1.jpg
    7 BtrfsDisk/Photos/photo2.jpg 
    8 BtrfsDisk/Snapshots/Photos-22012025
    9 BtrfsDisk/Snapshots/Documents-22012025
   10 BtrfsDisk/Snapshots/Documents-22012025/document1.txt
   11 BtrfsDisk/Snapshots/Photos-22012025/photo1.jpg

7. You can now create a new snapshot:
   

   /disk/btrfs/subvolume/add read-only=yes parent=Documents fs=BtrfsDisk  name=Snapshots/Documents-23012025 
   /disk/btrfs/subvolume/add read-only=yes parent=Photos fs=BtrfsDisk  name=Snapshots/Photos-23012025 

8. After a new snapshot, you will have 2 snapshots for each subvolume. One contains older files and another one contains older and newer files:
   

   [admin@MikroTik] > /file print 
    # NAME   
    0 BtrfsDisk
    1 BtrfsDisk/Documents
    2 BtrfsDisk/Photos
    3 BtrfsDisk/Snapshots
    4 BtrfsDisk/Documents/document1.txt
    5 BtrfsDisk/Documents/document2.txt
    6 BtrfsDisk/Photos/photo1.jpg
    7 BtrfsDisk/Photos/photo2.jpg
    8 BtrfsDisk/Snapshots/Photos-22012025
    9 BtrfsDisk/Snapshots/Documents-22012025
   10 BtrfsDisk/Snapshots/Documents-23012025
   11 BtrfsDisk/Snapshots/Photos-23012025
   12 BtrfsDisk/Snapshots/Documents-22012025/document1.txt 
   13 BtrfsDisk/Snapshots/Documents-23012025/document1.txt
   14 BtrfsDisk/Snapshots/Documents-23012025/document2.txt
   15 BtrfsDisk/Snapshots/Photos-22012025/photo1.jpg
   16 BtrfsDisk/Snapshots/Photos-23012025/photo1.jpg
   17 BtrfsDisk/Snapshots/Photos-23012025/photo2.jpg

   Multiple snapshots do not create multiple copies of each file, but if a file has been deleted and it still exists in a snapshot, then the deleted file will take up space. If a file exists in multiple snapshots, then it will take up space for only 1 file.

9. In case you don't need an older snapshot, you can delete it:
   

   /disk/btrfs/subvolume/remove [find where name=Documents-22012025]
   /disk/btrfs/subvolume/remove [find where name=Photos-22012025]

Transfer your snapshots

Btrfs allows to easily send a snapshot between two devices that is using Btrfs. In this example we will use two RouterOS devices running ROSE-storage package. The RouterOS device containing snapshots that need to be backed up is going to be called RouterA and the RouterOS device that is going to receive backups is going to be called RouterB .

openssl genpkey -outform PEM -out btrfstransfer_key.pem -algorithm ED25519
openssl pkey -in btrfstransfer_key.pem -pubout -out btrfstransfer_key_pub.pem

1. OPTIONAL: Increase the security level of your SSH server, run the following commands on RouterA and RouterB :
   

   /ip ssh/set host-key-type=ed25519 strong-crypto=yes
   /ip/ssh/regenerate-host-key

   Regenerating the host key will create a error message next time you try to connect to the RouterOS device using SSH. You will need to adjust your SSH client's configuration (usually in ~/.ssh/known_hosts ) to trust the new host key.

2. On RouterA exports the SSH private and public key:
   

   /ip/ssh/export-host-key key-file-prefix=admin

3. On RouterA import the SSH private key for user admin :
   

   /user/ssh-keys/private/import private-key-file=admin_ed25519.pem user=admin

4. Upload the SSH public key from RouterA to RouterB :
   

   /file/sync/add local-path=admin_ed25519_pub.pem remote-address=RouterB user=admin mode=upload
   /file/sync/remove [find where local-path=admin_ed25519_pub.pem]

5. On RouterB create a new user, for example btrsfstransfer  and set a secure password for it:
   

   /user/add name=btrfstransfer group=write

   While the password is not going to be used for the Btrfs transfer feature, you still need to use a secure password to prevent unauthorized access to your device.

6. On RouterB import the uploaded SSH public key and set it for user btrfstransfer :
   

   /user/ssh-keys/import public-key-file=admin_ed25519_pub.pem user=btrfstransfer

7. On RouterA setup type send :
   

   /disk/btrfs/transfer add type=send fs=BtrfsDisk ssh-address=RouterB send-subvolumes=Documents-23012025 ssh-user=btrfstransfer ssh-receive-mount=BackupBtrfsDisk/Snapshots

   Where:
   - BtrfsDisk is the Btrfs disk label found under /disk/btrfs/filesystem print on RouterA 
   - Documents-23012025 is the snapshots name (not the path)
   - btrsfstransfer is the SSH user on RouterB 
   - BackupBtrfsDisk is the Btrfs disk label found under /disk/btrfs/filesystem print on RouterB 
   - Snapshots is the subvolumes under for your snapshots under BackupBtrfsDisk 
8. On RouterB  setup type receive :
   

   /disk/btrfs/transfer/add fs=BackupBtrfsDisk type=receive file=BackupBtrfsDisk/Snapshots

   Where:
   - BackupBtrfsDisk is the Btrfs disk label found under /disk/btrfs/filesystem print on RouterB 
   - BackupBtrfsDisk/Snapshots is the mount path for Snapshots subvolume

Four disk Btrfs-RAID (RAID10)

In case you want to create redundant and high capacity RAID10 array using Btrfs, follow the commands below.

1. Find the names of the disks you want to use for setting up a Btrfs-RAID:
   

   /disk print

   In this example, the disks used are going to be called <disk-name-1>, <disk-name-2>, <disk-name-3> and <disk-name-4> , make sure you replace these placeholders with your actual disk names!

2. Format one of your disks to Btrfs, in this case <disk-name-1> :
   

   /disk format <disk-name-1> file-system=btrfs

3. You can check the current status of Btrfs disks using the following command:
   

   /disk/btrfs/filesystem print

   
   
   In case there was an existing RAID on your disks and you want to remove any obsolete configuration for simplicity, then you can wipe the disks with /disk format-drive <disk-name-x> file-system=wipe-quick . This is useful when you have unwanted entries under /disk/btrfs/filesystem print 
4. Add a label for the Btrfs drive for simplicity:
   

   /disk/btrfs/filesystem set [find where present-devs=<disk-name-1>] label=BtrfsRAID

5. Add other disks to an existing Btrfs disk:
   

   /disk/btrfs/filesystem/add-device [find where present-devs=<disk-name-1>] device=<disk-name-2>
   /disk/btrfs/filesystem/add-device [find where present-devs=<disk-name-1>] device=<disk-name-3>
   /disk/btrfs/filesystem/add-device [find where present-devs=<disk-name-1>] device=<disk-name-4>

6. Start Btrfs balance process to create a Btrfs-RAID with RAID1 configuration:
   

   /disk/btrfs/filesystem balance-start [find where label=BtrfsRAID] data-profile=raid10 metadata-profile=raid1c4 system-profile=raid1c4

   There are many possible configurations of data-profile , metadata-profile and system-profile . For a RAID10 array it is recommended to use either raid1c3 or raid1c4 for both system-profile and metadata-profile. The raid1c4 profile is going to store 4 copies of the data on different disks, this makes the data more redundant, but uses more disk space.

   Btrfs RAID5 and RAID6 is not supported. Use regular RAID if you need such RAID configurations. Be aware that when using regular RAID with Btrfs you will not have bit-rot protection.

   For most use cases set metadata-profile to the same value as system-profile . Avoid using different values for both of these profiles.

7. Set mount-filesystem=no for other disks to prevent the files showing up twice:
   

   /disk set <disk-name-2> mount-filesystem=no
   /disk set <disk-name-3> mount-filesystem=no
   /disk set <disk-name-4> mount-filesystem=no

   
   
   
8. You can also change the mount point's name for simplicity:
   

   /disk set <disk-name-1> mount-point-template=BtrfsRAID

9. Your newly created Btrfs-RAID array is now accessible through /BtrsRAID/ folder.

Replacing a disk in Btrfs RAID array

In case of a disk failure or you require to replace a disk in your existing Btrfs RAID array, then follow the procedure below:

1. Make sure you determine the correct disk that needs replacing. Use /disk print detail and /disk/blink to determine the correct disk. We will assume that the faulty disk is disk2 .
2. Eject the faulty disk:
   

   /disk/eject disk2

3. Physically remove the faulty disk
4. Print out the current status of your Btrfs RAID array:
   

   /disk/btrfs/filesystem/print

5. Look for the DEV-ID in the commands output for the missing disk:
   

   [admin@MikroTik] /disk> /disk/btrfs/filesystem/print
   Flags: I - MISSING-DEVS
   Columns: LABEL, DEV-IDS, DEVS, DEFAULT-SUBVOLUME, SPACES, BALANCE-STATUS, UUID, WRITE-ERRORS, READ-ERRORS, FLUSH-ERRORS, CORRUPTION-ERRORS, GENERATION-ERRORS
   #   LABEL      DEV-IDS  DEVS     DEFAULT-SUBVOLUME  SPACES                                     BALANCE-STATUS  UUID                                  W  R  F  C  G
   0 I BtrfsRAID        1  disk1    <FS_ROOT>          disk1:480GB, used:0%                       done            9246dfaa-be9f-4e08-a560-53cb8e82023b  0  0  0  0  0
                        2  missing                     data,raid1:1.07GB disk1:1.07GB, used:0%                                                          0  0  0  0  0
                                                       data,single:1.07GB, used:0%                                                                                   
                                                       system,raid1:33.6MB disk1:33.6MB, used:0%                                                                     
                                                       meta,raid1:1.07GB disk1:1.07GB, used:0%                                                                       
                                                       global-reserve:3.41MB, used:0%                                                                                
   
   

   In this case the missing disk's DEV-ID is "2".

6. Insert a new disk
7. Determine the new disks name:
   

   /disk/print

   In this case the new disk's name is disk3 

8. Run the following command to replace the disk in the Btrfs RAID array:
   

   /disk btrfs filesystem replace-device device-to-remove-id=2 device-to-add=disk3 BtrfsRAID

   Make sure you set the correct device-to-remove-id to the DEV-ID that you determined previously!

9. Check the replace status and make sure that REPLACE-STATUS is marked as done .
   

   > /disk btrfs filesystem/print
   Columns: LABEL, DEV-IDS, DEVS, DEFAULT-SUBVOLUME, SPACES, BALANCE-STATUS, REPLACE-STATUS, UUID, WRITE-ERRORS, READ-ERRORS, FLUSH-ERRORS, CORRUPTION-ERRORS, GENERATION-ERRORS
   # LABEL   DEV-IDS  DEVS   DEFAULT-SUBVOLUME  SPACES                                                 BALANCE-STATUS  REPLACE-STATUS  UUID                                  WRITE-ERRORS  READ-ERRORS  F  C  G
   0 BtrfsRAID     1  disk1  <FS_ROOT>          disk1:480GB, used:9%                                   done            done            9246dfaa-be9f-4e08-a560-53cb8e82023b             0            0  0  0  0
                   2  disk3                     disk3:480GB, used:9%                                                                                                                    0            0  0  0  0
                                                data,raid1:40.8GB disk1:40.8GB disk3:40.8GB, used:72%                                                                                                          
                                                system,raid1:101MB disk1:101MB disk3:101MB, used:0%                                                                                                            
                                                meta,raid1:3.22GB disk1:3.22GB disk3:3.22GB, used:1%                                                                                                           
                                                global-reserve:30.7MB, used:0%                                                 

   
   

   Ensure that your Btrfs RAID array does not have inconsistent Btrfs profiles for data , system  or meta . The desired profiles will depend on your specific setup. For RAID1 setups you will most likely want these profiles to match, but in RAID10 setup these values can be different. In any RAID setup there should not be a single profile for either data , system  nor meta .

   
   
   

Maintenance

Similar to other feature-rich filesystems, Btrfs needs periodic maintenance. This section will guide you through various procedures that are relevant to Btrfs.

Periodic scrubbing

In a RAID array, you data is stored on multiple disks or multiple disks contain information how to reassemble the data. Without RAID your disks in rare events might corrupt a few bytes of your data and you might not even notice that the data has been corrupted. With RAID arrays your data is compared with other copies of data (or checked on assembly) when you read a file and will alert you that data is corrupted. With regular RAID and, for example, in RAID1 configuration the RAID array is not able to tell which copy of the file is correct, it will only inform you that the data corruption has been detected. With Btrfs RAID you are not only able to detect the data corruption, but you are also able to distinguish which copy of the file is the correct one by using checksums and restore it automatically.

Scrubbing is a process that re-reads the whole RAID array and, in case of Btrfs RAID, corrects any data corruption. While Btrfs RAID will correct the data on file read operations, for example, you want to download a file from your RouterOS device, avoiding scrubbing is highly NOT recommended. In rare events it is possible that, for example, in RAID1 configuration both disks have corrupted data and Btrfs RAID might not be to restore the data. To avoid such situations and protect valuable data, consider running scrubbing on a regular basis.

The interval of how often to run scrubbing is going to depend on each use case. Scrubbing is an intensive task on your storage devices, it re-reads the whole RAID array and performs additional checks. During scrubbing you can experience noticeable performance drop since the disks until the scrubbing has finished. If you are worried about the performance during scrubbing, consider running scrubbing less often. If you are more worried about data integrity, consider running scrubbing more often.

• Summary: Used to detect and correct data corruption
• Recommended interval: 1 week
• Working example

Example command:

/disk/btrfs/filesystem/print
/disk/btrfs/filesystem/scrub-start 0

You can also cancel scrubbing using the following command:

/disk/btrfs/filesystem/scrub-cancel

Periodic balance

In Btrfs data is stored in allocated chunks, which then allows storing your data in blocks. Over time due to various data activities, the chunks can become partially full and distributed between many chunks in a sub optimal way. Balancing a Btrfs file system means re-arranging the data in these chunks and restoring the unallocated space. As a result you can restore lost usable free space and performance of the Btrfs file system. This is somewhat similar to a defragmentation operation on other file systems.

An important parameter for balancing is the data-usage parameter. This is a filter that prevents the balancing function to process chunks that are above a certain usage percentage. For example, data-usage=50 will only process chunks that are 50% full or less. You can run the balancing command multiple times with different values and therefore reduce the amount of time each balancing operation requires. Balancing can be an intensive task depending on your free space available and how data has been written since last balancing action therefore you might benefit of running the balance command with different data-usage values on separately to reduce the time window when balancing causes a performance drop due to intensive disk reads and writes.

In case you want to run balancing commands separately, you should use data-usage values of 25, 50, 75 and 90. It is not recommended to go above 90%. For most users running balance command separately is not required and running it once per interval with data-usage of 50% is sufficient.

• Summary: Used to restore free space and improve performance
• Recommended interval: twice a month
• Recommended data-usage : 50
• Working example

Example command:

/disk/btrfs/filesystem/print
/disk/btrfs/filesystem/balance-start data-usage=50 0

You can also cancel balance using the following command:

/disk/btrfs/filesystem/balance-cancel

Periodic snapshots

Snapshots can be used to save changes of your files in a set interval. Snapshots are most useful when you have a reliable interval at which data is copied so you can always revert your data to a previous state. Below you can find a ready-to-use script that creates periodic snapshots:

/system/scheduler
add interval=1d name=BraidSnapshotStartCall on-event=BraidSnapshotStart policy=ftp,read,write,policy,test,sniff start-date=1970-01-01 start-time=23:15:00
add interval=1d name=BraidSnapshotCleanUpStartCall on-event=BraidSnapshotCleanUpStart policy=ftp,read,write,policy,test,sniff start-date=1970-01-01 start-time=23:00:00
add interval=3m name=SystemBackupStartCall on-event=SystemBackupStart policy=ftp,read,write,policy,test,sniff start-time=startup

/system/script
add dont-require-permissions=no name=SystemBackupStart owner=admin policy=ftp,read,write,policy,test,sniff source=":global systembackupstatuscheck;\
    \n:global systembackupdirectoryname; \
    \n:local sysadmin;\
    \n\
    \n:set \$sysadmin ( \$sysadmin \"servername@domain.tld\" );\
    \n:set \$systembackupdirectoryname (\$systembackupdirectoryname \"Braid17-20/@system-backup/\");\
    \n\
    \n if (\$systembackupstatuscheck != \"started\") do={\
    \n         :set \$systembackupstatuscheck (\$systembackupstatuscheck \"started\");\
    \n               :local datentime ([/system/clock/get date].\"-\".[/system/clock/get time]);\
    \n               :local servername ([/system identity get name]);\
    \n               /system backup save name=\"\$systembackupdirectoryname\$servername-\$datentime\";\
    \n\
    \n        :set \$systembackupstatuscheck (\$systembackupstatuscheck \"done\");\
    \n} else={\
    \n         /log info message=\"ERROR: Cannot create  \$systembackupdirectoryname\$servername-\$datentime. Set manually :set systembackupstatuscheck (systembackupstatuscheck \\\"done\\\");\";\
    \n}\
    \n"
add dont-require-permissions=no name=BraidSnapshotStart owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfssnapshotstatuscheck;\
    \n:global snapshotdirectoryname; \
    \n:local maxusedspace;\
    \n:local sysadmin;\
    \n\
    \n:set \$maxusedspace ( \$maxusedspace 80 );\
    \n:set \$sysadmin ( \$sysadmin \"<servername@domain.tld>\" );\
    \n:set \$snapshotdirectoryname (\$snapshotdirectoryname \"@snapshots\");\
    \n\
    \n if (\$btrfssnapshotstatuscheck != \"started\") do={\
    \n         :set \$btrfssnapshotstatuscheck (\$btrfssnapshotstatuscheck \"started\");\
    \n         foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n           :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n           :local valueofusedspace [/disk print count-only where use>=\$maxusedspace and fs-label=\$temp];\
    \n           if ( \$valueofusedspace=0) do={\
    \n            foreach j in=[/disk btrfs subvolume/find  where fs=\$temp and top-level!=\$snapshotdirectoryname and fullname!=\$snapshotdirectoryname ] do={\
    \n               :local parentvar [ /disk/btrfs/subvolume/get value-name=name \$j; ];\
    \n               :local datentime ([/system/clock/get date].\"-\".[/system/clock/get time]);\
    \n                  /disk btrfs subvolume add read-only=yes fs=\"\$temp\" parent=\"\$parentvar\" name=\"\$snapshotdirectoryname/\$temp-\$parentvar-\$datentime\";\
    \n                   /log info message=\"INFO: Braid snapshot created for  \$temp subvolume \$parentvar snapshotname  \$snapshotdirectoryname/\$temp-\$parentvar-\$datentime\";\
    \n                :delay 1;\
    \n            }\
    \n          } else={\
    \n                   /log info message=\"ERROR: Snapshot was not created for safety reason.  Braid array \$temp used space exceeded \$maxusedspace %. Add more disks or cleanup storage.\"; \
    \n                   /tool e-mail send to= \$sysadmin  subject=([/system identity get name].\" ERROR: Braid snapshot was not created \") body=(\"Snapshot was not created for safety reason.  Braid array \" .\$temp. \" used space exceeded \" .\$max\
    usedspace. \" % Add more disks or cleanup storage. \" );\
    \n          }\
    \n\
    \n          :delay 3; \
    \n         }\
    \n        :set \$btrfssnapshotstatuscheck (\$btrfssnapshotstatuscheck \"done\");\
    \n}\
    \n"
add dont-require-permissions=no name=BraidSnapshotCleanUpStart owner=admin policy=ftp,read,write,policy,test,sniff source=":global btrfssnapshotcleanupstatuscheck;\
    \n:global snapshotdirectoryname; \
    \n:local maxsnapshotstokeep;\
    \n:local maxdaysoldsnapshotstokeep;\
    \n:local sysadmin;\
    \n\
    \n:set \$maxsnapshotstokeep ( \$maxsnapshotstokeep 10);\
    \n:set \$maxdaysoldsnapshotstokeep (\$maxdaysoldsnapshotstokeep \"10d\");\
    \n:set \$sysadmin ( \$sysadmin \"<servername@domain.tld>\" );\
    \n:set \$snapshotdirectoryname (\$snapshotdirectoryname \"@snapshots\");\
    \n\
    \n if (\$btrfssnapshotcleanupstatuscheck != \"started\") do={\
    \n         :set \$btrfssnapshotcleanupstatuscheck (\$btrfssnapshotcleanupstatuscheck \"started\");\
    \n         foreach i in=[/disk/btrfs/filesystem/find] do={ \
    \n           :local temp [ /disk btrfs filesystem/get value-name=label \$i;]\
    \n           :local currenttimestamp; :set  \$currenttimestamp ( \$currenttimestamp [/system/clock/get date ] );\
    \n          :set  \$currenttimestamp ( \$currenttimestamp  -\$maxdaysoldsnapshotstokeep);\
    \n            foreach j in=[/disk btrfs subvolume/find  where fs=\$temp and top-level=\$snapshotdirectoryname ] do={\
    \n               :local parentname [ /disk/btrfs/subvolume/get value-name=name \$j;];\
    \n               :local parentsubvol [ /disk/btrfs/subvolume/get value-name=parent \$j; ];\
    \n               :local creationtimeofsnapshot; :set \$creationtimeofsnapshot (\$creationtimeofsnapshot [/disk/btrfs/subvolume/get value-name=creation-time \$j; ]);\
    \n               :local countparentsnapshots;  :set \$countparentsnapshots (\$countparentsnapshots [/disk btrfs subvolume/print count-only  where fs=\$temp and top-level=\$snapshotdirectoryname and parent=\$parentsubvol]);\
    \n               if ([:len \$parentsubvol]=0) do={\
    \n                   :local parentfullname [ /disk/btrfs/subvolume/get value-name=fullname \$j;];\
    \n                    /log info message=\"INFO: SnapshotCleanup found snapshot of completely deleted subvolume. Location of snapshot \$temp/\$snapshotdirectoryname/\$parentname. This can only be removed manually.\";\
    \n               } else={\
    \n                if (\$currenttimestamp>=\$creationtimeofsnapshot or  \$countparentsnapshots>\$maxsnapshotstokeep ) do={\
    \n                   /log info message=\"INFO: Braid snapshot  \$snapshotdirectoryname/\$parentname deleted. SnapshotCleanUp keeps  \$maxsnapshotstokeep snapshots or snapshots not older than \$maxdaysoldsnapshotstokeep days.\";\
    \n                  /disk btrfs subvolume remove \$parentname;\
    \n                :delay 1;\
    \n               }\
    \n             }\
    \n            }\
    \n          :delay 3; \
    \n         }\
    \n        :set \$btrfssnapshotcleanupstatuscheck (\$btrfssnapshotcleanupstatuscheck \"done\");\
    \n}\
    \n"

Free space

In Btrfs due to how data is stored in chunks, it is important to make sure there is enough free space in order for not just regular maintenance tasks to run properly, but also for the performance of the file system to be optimal. To make sure your Btrfs file system is working properly, consider checking that there is enough free space on your Btrfs file system. An optimal value of free space is 10%, but should not allow your Btrfs file system to fill up to 5% free space.

• Summary: Do not allow Btrfs file system to be full
• Recommended free space: 5-10%

Example command:

/disk/btrfs/filesystem/print