Versions Compared

Old Version 19

changes.mady.by.user Unknown User (rudolfs)

Saved on

compared with

New Version Current

changes.mady.by.user Oļegs Š.

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Cloud Hosted Router (CHR) is a RouterOS version intended for running as a virtual machine. It supports the x86 64-bit architecture and can be used on most of the popular hypervisors such as VMWare, Hyper-V, VirtualBox, KVM, and others. CHR has full RouterOS features enabled by default but has a different licensing model than other RouterOS versions.

System Requirements

  • Package version: RouterOS v6.34 or newer
  • Host CPU: 64-bit with virtualization support
  • RAM: 128MB 256MB or more (Max: 128GB)
  • Disk: 128MB disk space for the CHR virtual hard drive (Max: 16GB)

...

  • RouterOS v6 - RAM = 128 + [ 8 × (CPU_COUNT) × (INTERFACE_COUNT - 1) ]
  • RouterOS v7 - RAM = 256 512 + [ 8 × (CPU_COUNT) × (INTERFACE_COUNT - 1) ]

Note: We recommend allocating at least 1024MiB of RAM for CHR instances.

CHR has been tested on the following platforms:

  • VirtualBox 6 on Linux and OS X
  • VMWare Fusion 7 and 8 on OS X
  • VMWare ESXi 6.5
  • Qemu 2.4.0.1 on Linux and OS X
  • Hyper-V on Windows Server 2008r2, 2012 and Windows 10 (Only Generation 1 Hyper-V virtual machine is supported at the moment)
  • Xen Server 7.1

Warning: Hypervisors that provide paravirtualization are not supported.

Usable Network and Disk interfaces on various hypervisors:

  • ESX:
    • Network: vmxnet3, E1000
    • Disk: IDE, VMware paravirtual SCSI, LSI Logic SAS, LSI Logic Parallel

...

Note: SCSI controller Hyper-V and ESX is are usable just for secondary disks, system image must be used with IDE controller!

Warning: We do not recommend using the E1000 network interface if better synthetic interface options are available on a specific Hypervisor!

How to Install a virtual RouterOS system with CHR images

We provide 4 different virtual disk images to choose from. Note that they are only disk images, and you can't simply run them.

...

Steps to install CHR

  1. Download the virtual disk image for your hypervisor
  2. Create a guest virtual machine
  3. Use the previously downloaded image file as a virtual disk drive
  4. Start the guest CHR virtual machine
  5. Log in to your new CHR. The default user is 'admin', without a password

Please note that running CHR systems can be cloned and copied, but the copy will be aware of the previous trial period, so you cannot extend your trial time by making a copy of your CHR. However, you are allowed to license both systems individually. To make a new trail trial system, you need to make a fresh installation and reconfigure RouterOS.

...

CHR Licensing

The CHR has 4 license levels:

...

LicenseSpeed limitPrice
Free1MbitFREE
P11Gbit$45
P1010Gbit$95
P-UnlimitedUnlimited$250

Paid licenses

p1

p1 (perpetual-1) license level allows CHR to run indefinitely. It is limited to 1Gbps upload per interface. All the rest of the features provided by CHR are available without restrictions. It is possible to upgrade p1 to p10 or p-unlimited After (new license level can be purchased by standard price). After the upgrade is purchased the former license will become available for later use on your account.

...

The p-unlimited (perpetual-unlimited) license level allows CHR to run indefinitely. It is the highest-tier license and it has no enforced limitations.

Free licenses

There are several options to use and try CHR free of charge.

...

Info
If you plan to use multiple virtual systems of the same kind, it may be possible that the next machine has the same system ID as the original one. This can happen on certain cloud providers, such as Linode. To avoid this, after your first boot, run the command "/system license generate-new-id" before you request a trial license. Note that this feature must be used only while CHR is running on a free type of RouterOS license. If you have already obtained a paid or trial license, do not use the regenerate feature since you will not be able to update your current key any moreanymore

Getting the License

After the initial setup, a CHR instance will have a free license assigned. From there, it is possible to upgrade the license to a higher tier. Once you have a trial license all the work with the license is done on the account server where it is possible to upgrade the license to a higher tier unless it is p-unlimited already.

Upgrade from

...

Free to p1 license level or higher

Initial upgrade from the free tier to anything higher than that incurs CHR instance registration on the account server. To do that you have to enter your MikroTik.com username and password and the desired license level you want to acquire. As a result, a CHR ID number will be assigned to your account on the account server and a 60-day trial created for that ID. There are 2 ways to obtain a license - using WinBox or RouterOS command-line interface:Using

Upgrade license level using WinBox

(Sytem System -> License menu):

Image RemovedImage Added

Image ModifiedUsing the

Upgrade license level using the command-line interface:

[admin@MikroTik] > /system license print 
  system-id: 6lR1ZP/utuJ
      level: free

[admin@MikroTik] > /system license renew 
account: mymikrotikcomaccount
password: *********************
level: p1 
  status: done
  
[admin@MikroTik] > /system license print 
        system-id: 6lR1ZP/utuJ
            level: p1
  next-renewal-at: jan/10/2016 21:59:59
      deadline-at: feb/09/2016 21:59:59

Payment:

To acquire a higher-level trial, set up a new CHR instance, renew the license, and select the desired level.

To upgrade from a Trial license to a Paid one go to to MikroTik.com account server and choose 'all keys' in Cloud Hosted Router (CHR) section:

Image Modified

You will be presented with a list of your CHR machines and licenses:

Image RemovedImage Added

To upgrade from a Trial to a Paid license click 'Upgrade', choose the desired license level (it can be different than the level of the trial license), and click 'Upgrade key':

Image Added

If there are prepaid keys available, it is possible to use it for CHR - press "Pay using Prepaid key". If there are no prepaid keys or you do not want to use them, press "Proceed to checkout".

Image AddedImage Removed

Choose the payment method:

Image Removed

It is possible to pay using account balance (deposit), credit card (CC), PayPal, or using Balance (prepaid) key (if you have any).a credit card (CC) or PayPal.

Image Added

License Update

Image Modified

In '/system license' menu router will indicate the time next-renewal-at when it will attempt to contact the server located on licence.mikrotik.com. Communication attempts will be performed once an hour after the date on next-renewal-at and will not cease until the server responds with an error. If the deadline-at date is reached without successfully contacting the account server, the router will consider that the license has expired and will disallow further software updates. However, the router will continue to work with the same license tier as before.

Note

If you want to upgrade perpetual license to a higher level please transfer the previous perpetual license to another CHR, to exclude the situation where the previous perpetual license is lost on upgrade.


Virtual Network Adapters

Note

Fast Path is supported since in RouterOS v7 for "vmxnet3" and "virtio-net" adapters.

RouterOS v6 does not support Fast Path.

Troubleshooting

Running on VMware ESXi

Changing MTU

VMware ESXi supports MTU of up to 9000 bytes. To get the benefit of that, you have to adjust your ESXi installation to allow a higher MTU. Virtual Ethernet interface added after the MTU change will be properly allowed by the ESXi server to pass jumbo frames. Interfaces added prior to MTU change on the ESXi server will be barred by the ESXi server (it will still report the old MTU as the maximum possible size). If you have this, you have to re-add interfaces to the virtual guests.

...

[admin@chr-vm] > interface ethernet print 
Flags: X - disabled, R - running, S - slave 
 #    NAME           MTU MAC-ADDRESS       ARP       
 0 R  ether1        9000 00:0C:29:35:37:5C enabled   
 1 R  ether2        1500 00:0C:29:35:37:66 enabled

Using bridge on Linux

If Linux bridge supports IGMP snooping, and there are problems with IPv6 traffic it is required to disable that feature as it interacts with MLD packets (multicast) and is not passing them through.

echo -n 0 > /sys/class/net/vmbr0/bridge/multicast_snooping

Packets not passing from guests

The problem: after configuring a software interface (VLAN, EoIP, bridge, etc.) on the guest CHR it stops passing data to the outside world beyond the router.

The solution: check your VMS (Virtualization Management System) security settings, if other MAC addresses are allowed to pass and if packets with VLAN tags are allowed to pass through. Adjust the security settings according to your needs like allowing MAC spoofing or a certain MAC address range. For VLAN interfaces, it is usually possible to define allowed VLAN tags or VLAN tag range.

Using VLANs on CHR in various Hypervisors

In some hypervisors, before VLAN can be used on VMs, they need to first be configured on the hypervisor itself.

ESXI

Enable Promiscuous mode in a port group or virtual switch that you will use for a specific VM.

ESX documentation:

Hyper-V

Hyper-V documentation:

bhyve hypervisor

It won't be possible to run CHR on this hypervisor. CHR cannot be run as a para-virtualized platform.

Linode

When creating multiple Linodes with the same disk size, new Linodes will have the same systemID. This will cause issues to get a Trial/Paid license. To avoid this, run the command /system license generate-new-id after the first boot and before you request a trial or paid license. This will make sure the ID is unique.

...

Guest tools

VMWare

Time synchronization

Must be enabled from GUI ('Synchronize guest time with host'). Backward synchronization is disabled by default - if the guest is ahead of the host by more than ~5 seconds, synchronization is not performed

Power operations

  • poweron and resume scripts are executed (if present and enabled) after power on and resume operations respectively.
  • poweroff and suspend scripts are executed before power off and suspend operations respectively.
  • If scripts take longer than 30 seconds or contain errors, the operation fails
  • In case of failure, retrying the same operation will ignore any errors and complete it successfully
  • Failed script output is saved to a file (e. g. 'poweroff-script.log', 'resume-script.log' etc)
  • Scripts can be enabled/disabled from hypervisor GUI ('run VMware Tools Scripts') or by enbalingenabling/disabling scripts from the console

Quiescing/backup

Guest filesystem quiescing is performed only if requested.

  • freeze script is executed before freezing the filesystem
  • freeze-fail script is executed if the hypervisor failed to prepare for a snapshot or if if freeze script failed
  • thaw script is executed after the snapshot has been taken
  • Script run time is limited to 60 seconds
  • freeze script timeouts and errors result in the backup operation being aborted
  • FAT32 disks are not quiesced
  • Failed script output is saved to a file (e. g. 'freeze-script.log', 'freeze-fail-script.log', 'thaw-script.log')

Guest info

Networking, disk, and OS info are reported to the hypervisor every 30 seconds (GuestStats (memory) are disabled by default, and can be enabled by setting 'guestinfo.disable-perfmon = "FALSE"' in VM config).

  • The order, in which network interfaces are reported, can be controlled by setting 'guestinfo.exclude-nics', 'guestinfo.primary-nics' and 'guestinfo.low-priority-nics' options. Standard wildcard patterns can be used.

Provisioning

Can  You can use the ProcessManager from vim Vim API to execute scripts. Python bindings are available

...

Script progress can be tracked by using the ListProcessesInGuest command. ListProcessesInGuest accepts an array of job id's; passing an empty array will report on all jobs started from the API

  • ListProcessesInGuest returns an array of GuestProcessInfo instances:
    • pid field is set to JobID
    • endTime is only set after completion
    • exitCode is set to 0 on success and -1 on error
    • name is set to 'inline' or 'import' (same as programPath in GuestProgramSpec)

...

  • The vmrun command runScriptInGuest can also be used
  • The PowerCLI cmdlet Invoke-VMScript is not supported
  • Host/guest file transfer is not supported
Python example
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys,time
from pyVim import connect
from pyVmomi import vmodl,vim


def runInline(content,vm,creds,source):
    ''' Execute script source on vm '''
    if isinstance(source, list):
        source = '\n'.join(source)
    ps = vim.vm.guest.ProcessManager.ProgramSpec(
                programPath = 'console',
                arguments = source
        )
    return content.guestOperationsManager.processManager.StartProgramInGuest(vm,creds,ps)

def runFromFile(content,vm,creds,fileName):
    ''' Execute script file located on CHR '''
    ps = vim.vm.guest.ProcessManager.ProgramSpec(
                programPath = 'import',
                arguments = fileName
    )
    return content.guestOperationsManager.processManager.StartProgramInGuest(vm,creds,ps)


def findDatastore(content,name):
    sessionManager = content.sessionManager

    dcenterObjView = content.viewManager.CreateContainerView(content.rootFolder, [vim.Datacenter], True)

    datacenter = None
    datastore = None
    for dc in dcenterObjView.view:
        dstoreObjView = content.viewManager.CreateContainerView(dc, [vim.Datastore], True)
        for ds in dstoreObjView:
            if ds.info.name == name:
                datacenter = dc
                datastore = ds
                break
        dstoreObjView.Destroy()

    dcenterObjView.Destroy()

    return datacenter,datastore

def _FAILURE(s,*a):
    print(s.format(*a))
    sys.exit(-1)

#------------------------------------------------------------------------------#

if __name__ == '__main__':
    host = sys.argv[1] # ip or something
    user = 'root'
    pwd = 'MikroTik'
    vmName = 'chr-test'
    dataStoreName = 'datastore1'



    service = connect.SmartConnectNoSSL(host=host,user=user,pwd=pwd)
    if not service:
        _FAILURE("Could not connect to the specified host using specified username and password")

    content = service.RetrieveContent()


    #---------------------------------------------------------------------------
    # Find datacenter and datastore


    datacenter,datastore = findDatastore(content,dataStoreName)

    if not datacenter or not datastore:
        connect.Disconnect(service)
        _FAILURE('Could not find datastore \'{}\'',dataStorename)


    #---------------------------------------------------------------------------
    # Locate vm


    vmxPath = '[{0}] {1}/{1}.vmx'.format(dataStoreName, vmName)
    vm = content.searchIndex.FindByDatastorePath(datacenter, vmxPath)

    if not vm:
        connect.Disconnect(service)
        _FAILURE("Could not locate vm")


    #---------------------------------------------------------------------------
    # Setup credentials from user name and pasword

    creds = vim.vm.guest.NamePasswordAuthentication(username = 'admin', password = '')


    #---------------------------------------------------------------------------
    # Run script

    pm = content.guestOperationsManager.processManager

    try:
        # Run script
        src = [':ip address add address=192.168.0.1/24 interface=ether1;']
        jobID = runInline(content, vm, creds, src)

        # Or run file (from FTP root)
        # jobID = runFromFile(content,vm,creds, 'scripts/provision.rsc')


        #---------------------------------------------------------------------------
        # Wait for job to finish

        pm = content.guestOperationsManager.processManager
        jobInfo = pm.ListProcessesInGuest(vm, creds, [jobID])[0]
        while jobInfo.endTime is None:
            time.sleep(1.0)
            jobInfo = pm.ListProcessesInGuest(vm, creds, [jobID])[0]

        if jobInfo.exitCode != 0:
            _FAILURE('Script failed!')
    except:
        raise
    else:
        connect.Disconnect(service)

KVM

QEMU guest agent is available. Supported agent commands can be retrieved by using the guest-info command. Host-guest file transfer can be performed by using guest-file-* commands. Guest networking information can be retrieved by using the guest-network-get-interfaces command.

...

  • Script job progress can be monitored with guest-exec-status command. The GuestExecStatus data structure is populated as follows:
    • On success, exitcode member is set to 0
    • If the script timed out exitcode is set to 1
    • If the script contained errors exitcode is set to -1
    • signal member is not set
    • The err-data member is not used
    • If capture-output was true, Base64 encoded script output is stored in out-data

...