apt get life

Life around technology

You are here: Home / Archives for Technology / Linux

KVM converting virtual disks from raw img files to qcow2

by Leave a Comment

If you’re running qemu KVM on Ubuntu and want to take advantage of the qcow2 file format’s snapshotting capabilities and sparse disk population you can easily convert using the command line tool

1
qemu-img convert

First, make sure your virtual machine is turned off! Then you can navigate to the directory your virtual disks are stored in (usually /var/lib/libvirt). It’s probably a good idea to be a root user or otherwise sudo the following command

1
qemu-img convert -f raw -O qcow2 vm_hdd.img vm_hdd.qcow2

The

1
-f

flag tells convert what format it’s reading. If you don’t provide it then it’ll guess based on the file extension.

the

1
-O

flag tells convert what file format to output to, again if not provided it’ll guess based on the file extension.

Now you’ve got a qcow2 file, you’ll need to edit the VM configuration

1
virsh edit vm_name

this will open up an editor for your VM configuration. It’s an XML file, so it’s reasonably easy to follow. What you’re looking for is a disk section so you can change the file extension and disk type

1
2
3
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/rhel62-2.img'/>

1
2
3
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/var/lib/libvirt/images/rhel62-2.qcow2'/>

Note both “raw” and “img” have been changed to “qcow2” for this disk. Make sure you’ve picked the right disk to edit in the XML. It may be a good idea to take a backup first so you can fall back to the img file if needed!

That should be it, your VM should now boot with the new disk file. Once you’re sure it’s working you can delete the original (or keep it safe somewhere).

 

More information about KVM can be found on the RedHat website: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/index or the Ubuntu wiki https://help.ubuntu.com/community/KVM/Installation

Setup Fail2ban for NextCloud

by 2 Comments

Running NextCloud or OwnCloud online comes with some risk, as with any online service. It’s important that your installation remains secure against hackers (or at least as secure as it can be). I’ve opted to implement fail2ban in order to help secure it using some custom rules. It’s worth noting that NextCloud does block unwanted login attempts itself through the application, but you’re having to trust application level security. I feel far safer having fail2ban implement firewall rules to prevent access to anyone probing the server.

First thing to do is create the NextCloud filter configuration file. This file will contain the regex that’s used to scan the logs for anything we don’t like the look of in order to block attacking hosts. My understanding is that this file can remain the same for OwnCloud, although I do not currently have a running instance of it to check.

1
sudo nano /etc/fail2ban/filter.d/nextcloud.conf

Add the following to the file:

1
2
3
4
[Definition]
failregex=^{"reqId":".<em>","remoteAddr":".</em>","app":"core","message":"Login failed: '.<em>' &#40;Remote IP: '<HOST>'&#41;","level":2,"time":".</em>"}$
        ^{"reqId":".<em>","level":2,"time":".</em>","remoteAddr":".<em>","app":"core".</em>","message":"Login failed: '.<em>' &#40;Remote IP: '<HOST>'&#41;".</em>}$
        ^.<em>\"remoteAddr\":\"<HOST>\".</em>Trusted domain error.*$

There are three regular expressions included here. The first and second checks for login failures, and flags the source IP. The third checks for trusted domain errors – which are usually a result of bots accessing your installation via it’s IP, not via it’s domain (thus, suspicious and I wanted to block them).

Once the file is saved, you can test what the filter would report by running the following command. This is entirely optional (although would help identify issues) and isn’t required for the rest of the steps to work.

1
sudo fail2ban-regex /var/nextcloud/data/nextcloud.log /etc/fail2ban/filter.d/nextcloud.conf -v

Next, the configuration file needs setup to activate the configurations we’ve just created. Never edit the Fail2ban jail.conf file, it’s likely to be overridden on upgrades. Always create a “.local” file, ideally a separate one for each application or rule you’re setting up (why? because it makes things more organised and easier to manage one rule over another!) inside the jail.d directory. With this in mind, create a nextcloud (or owncloud) file:

1
sudo nano /etc/fail2ban/jail.d/nextcloud.local

And add the following to it:

1
2
3
4
5
6
7
8
9
10
11
[nextcloud]
ignoreip = 192.168.1.0/24
backend = auto
enabled = true
port = 80,443
protocol = tcp
filter = nextcloud
maxretry = 3
bantime = 36000
findtime = 36000
logpath = /var/nextcloud/data/nextcloud.log

Make sure your ignoreip is your local subnet or IP address. I opted to allow my whole LAN to access it without being auto-blocked. I’ve enabled the rule, set the ports to 80 (HTTP) and 443 (HTTPS) and configured ban times, etc. The most important things are the filter which should match the name of the file that was created inside the filter.d directory (excluding extension), and the log path, which may vary by installation. This path is the default for Ubuntu.

Once done, run the following command to restart nextcloud:

1
sudo service fail2ban restart

You can check the status of the jail by running:

1
sudo fail2ban-client status nextcloud

You’ll see something similar to this:

1
2
3
4
5
6
7
8
9
Status for the jail: nextcloud
|- Filter
|  |- Currently failed: 13
|  |- Total failed: 82
|  `- File list:    /var/nextcloud/data/nextcloud.log
`- Actions
   |- Currently banned: 0
   |- Total banned: 5
   `- Banned IP list:

Setting up a KVM host on Ubuntu 18.04 with bridged networking

by Leave a Comment

Since the launch of Ubuntu server 18.04, I’ve had a few people ask me how to setup bridged networking since the changeover to netplan for networking. The setup’s actually quite straight forwards once you know how. Start out by installing KVM and associated packages (I’m assuming you’re on a fresh Ubuntu 18.04 box):

1
sudo apt install bridge-utils qemu-kvm libvirt-bin

You should now be able to access virsh, to test it just run the command to list the virtual machines on the host (this will of course output an empty list. We just want to make sure the command hasn’t failed before going further):

1
virsh list --all

1
<h2> Id    Name                           State</h2>

Now edit the netplan file for the networking configuration of the host system:

1
sudo nano /etc/netplan/50-cloud-init.yaml

The configuration should be edited to look similar to this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
network:
    ethernets:
        eno1:
            addresses: []
            dhcp4: true
            optional: true
    version: 2
    bridges:
        br0:
            interfaces: [eno1]
            dhcp4: true
            parameters:
                stp: false
                forward-delay: 0

We’re basically defining a single ethernet port (eno1) and assigning it to a bride. Note that you can check your ethernet port name using the ‘ip address’ command on the command line and it will return your network adapters and IP addresses. It should be resonably obvious which one is your ethernet adapter from the list!

It’s worth noting that in this setup, we’re allowing DHCP to allocate addresses to the machine. If this isn’t what you want, take a look at the netplan examples here: https://netplan.io/examples

You can apply these change using:

1
sudo netplan apply

And check it’s working:

1
ip address

or

1
networkctl list

If you’ve gotten this far without a network issue, you should be able to create machines using your favorite tool. I use Virtual Machine Manager (https://virt-manager.org/) which provides a reasonable user interface as well as remote management via SSH.

Setting up a Bond and Bridge in Netplan on Ubuntu 18.04

by 2 Comments

For some of my Ubuntu 18.04 servers, I need to run KVM virtual machines which require a bridge to the network so the machines get public LAN IP addresses and aren’t hidden behind NAT. With the server configuration for both my co-location and servers at work the network interfaces are all bonded for fail-over. This means I need a bond to have a bridge ontop of it for the virtual machines to get public IP addresses, while still allowing for failover of the network connection in the event of a network failure.

Research

There are some good examples of setting up netplan here: https://netplan.io/examples

They have a bridge example:

Netplan bridge example
1
2
3
4
5
6
7
8
network:
  version: 2
  renderer: networkd
  bridges:
    br0:
      dhcp4: yes
      interfaces:
        - enp3s0

And a bond example:

Netplan bond example
1
2
3
4
5
6
7
8
9
10
11
12
network:
  version: 2
  renderer: networkd
  bonds:
    bond0:
      dhcp4: yes
      interfaces:
        - enp3s0
        - enp4s0
      parameters:
        mode: active-backup
        primary: enp3s0

But there’s not a clear indication of how to amalgamate the two.

A bond and a bridge

Here’s what I’ve ended up with in ‘/etc/netplan/50-cloud-init.yaml’:

Netplan bridge and bond
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
network:
    bridges:
        br0:
            addresses:
            - 192.168.10.30
            dhcp4: false
            gateway4: 192.168.10.1
            nameservers:
                addresses:
                - 192.168.10.1
                - 192.168.10.2
                search: []
            interfaces:
                - bond0
    bonds:
        bond0:
            interfaces:
            - eno1
            - eno2
            parameters:
                mode: active-backup
    ethernets:
        eno1:
            addresses: []
            dhcp4: false
            dhcp6: false
        eno2:
            addresses: []
            dhcp4: false
            dhcp6: false

Note that I’ve obviously defined static IP addresses, but this isn’t a requirement. Just set ‘dhcp4: true’ and remove the ‘address’, ‘gateway’ and ‘nameserver’ sections if you’re using DHCP.

Once the file’s got that setup in it, it’s possible to run:

1
sudo netplan apply

and you should be able to run ‘networkctl list’ to check the bridge and bond are setup.

Fail2ban does not enable some rules

by Leave a Comment

While setting up a webserver, I noticed that

1
fail2ban-client status

 did not list all of the rules that I’d configured in

1
jail.d

. Althought configuration files were added to the directory, the status wasn’t showing them listed, so it hadn’t activated the jails. Running

1
service fail2ban restart

 didn’t add them either.

After spending some time researching, some people suggest changing the backend used in fail2ban from auto to polling in jail.conf. Since you shouldn’t edit jail.conf on a debian based system, I tried creating a new file 

1
nano /etc/fail2ban/jail.local

 and adding the line

1
backend = polling

. This actually prevented the fail2ban service from starting for me. The errors in

1
systemctl status fail2ban.service

didn’t show anything useful as to the cause of the error. In the end I reverted this change.

running

1
fail2ban-client reload

 did present error messages detailing which rule couldn’t be activated:

1
2
ERROR No file(s) found for glob /var/log/apache2/*.log
ERROR Failed during configuration: Have not found any log file for apache-xmlrpc jail

This then meant I was able to investigate the

1
jail.d

 config file with the rule indicated as causing the problem and correct the log file path.

1
fail2ban-client status

 then shows the correct jails running.