apt get life

Life around technology

  • Technology
    • Guides
    • Linux
    • Development
      • Laravel
    • Misc
    • Raspberry Pi
  • Writing
  • Crafts
    • Crochet
    • Model Making
    • Painting
  • Privacy Policy
You are here: Home / Archives for Technology / Linux

Fixing zsys daemon: timed out waiting for server handshake on Ubuntu 20.04 ZFS

2021/08/21 by sudo Leave a Comment

When running the apt package manager from the command line, if it starts coming up with the following error when you’re running ZFS it’s likely that your disk is too full to take more snapshots. This is a process that happens automatically on package installation or update.

<!-- wp:paragraph -->
<p>When running the apt package manager from the command line, if it starts coming up with the following error when you're running ZFS it's likely that your disk is too full to take more snapshots. This is a process that happens automatically on package installation or update.</p>
<!-- /wp:paragraph -->

In order to list your ZFS snapshots you can run

zfs list -t snapshot

To delete snapshots, as root, run either zfs destroy along with the specified snapshot name or delete a whole bunch at once:

zfs list -H -o name -t snapshot | xargs -n1 zfs destroy

For more information about zsys check out https://didrocks.fr/2020/06/04/zfs-focus-on-ubuntu-20.04-lts-zsys-state-collection/

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Tumblr
  • Pinterest

Filed Under: Linux, Technology Tagged With: ubuntu 20.04, zfs

Solving Error on statfs() system call with cAdvisor and Docker

2021/01/14 by sudo Leave a Comment

With a docker setup running with Prometheus node exporter and a docker container running cAdvisor, I’ve been seeing error messages similar to the following repeatedly appearing in syslog:

Jan 14 15:15:25 dockerserver-live prometheus-node-exporter[603]: time="2021-01-14T15:15:25Z" level=error msg="Error on statfs() system call for \"/var/lib/docker/containers/719fe4c20d2d274bb034e914006ecfe6760d8aec98efdc8010c85a01cf4059aa/mounts/shm\": permission denied" source="filesystem_linux.go:57"
Jan 14 16:23:25 dockerserver-live prometheus-node-exporter[28623]: time="2021-01-14T16:23:25Z" level=error msg="Error on statfs() system call for \"/var/lib/docker/overlay2/8c25cb3049b4cfc9bebfd4df0ea6104560155bed2c18a9bd75d21323931570f4/merged\": permission denied" source="filesystem_linux.go:57"

These errors are being generated by a Prometheus node exporter process running with the following args:

      --collector.diskstats.ignored-devices=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\\d+n\\d+p)\\d+$ \
      --collector.filesystem.ignored-mount-points=^/(sys|proc|dev|run)($|/) \
      --collector.netdev.ignored-devices=^lo$ \
      --collector.textfile.directory=/var/lib/prometheus/node-exporter

I’m not sure where this block originally came from, but in our case it’s in /etc/defaults/prometheus-node-exporter and easily edited to fix the regular expressions. Specifcally, because there’s problems with permissions on the shm and overlay they can be added to the ignored-mount-point regular expression as below:

      --collector.diskstats.ignored-devices=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\\d+n\\d+p)\\d+$ \
      --collector.filesystem.ignored-mount-points=^/(sys|proc|dev|run|var\/lib\/docker\/containers\/.*\/mounts\/shm|var\/lib\/docker\/overlay2\/.*\/merged)($|/) \
      --collector.netdev.ignored-devices=^lo$ \
      --collector.textfile.directory=/var/lib/prometheus/node-exporter

following this change and restarting the prometheus-node-exporter process stopped new entries appearing in our syslog as it should be ignoring both troublesome directories.

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Tumblr
  • Pinterest

Filed Under: Docker, Linux, Technology, Uncategorized

MySQL in Docker for Local Development

2020/08/04 by sudo Leave a Comment

Outside of my standard docker-compose setup for Laravel, I find myself requiring one-off MySQL instances for short pieces of development, debugging or testing. Instead of building a whole container ecosystem to do this, if I can get away with it I simply run MySQL as a docker container, binding it to the local machine’s ports.

The MySQL docker container is available on docker hub https://hub.docker.com/_/mysql which provides some basic usage instructions. I’m going to outline how I use MySQL in docker for this one off work.

Almost everything I do is based on the older MySQL 5.7. That means I need to use that image specifically when running docker. In order to make sure my local container is up to date and ready and waiting for use I tend to run a docker pull command on my dev machine as soon as it’s setup.

docker pull mysql:5.7

Version information is available on the MySQL docker hub page if you need a different version of MySQL. Now that the docker image is held locally it’s much easier to just start up a container whenever you need it and it will not spend ages re-downloading.

Understanding the Basics of MySQL’s Docker Container

There’s a few things I do with my containers. Firstly I want to expose the container to the host machine. This allows database administration tools like MySQL Workbench or Datagrip to connect to the MySQL docker instance. It also allows code to talk to it, and often this is what I want to do. It’s important not to overlap these ports, but generally I don’t run a temporary MySQL container along side any development stacks or local installs so I bind to the default port (3306). To do this I add the -p 3306:3306 flag to the command. If you want to change the external port (the one you’re using to connect to MySQL inside docker), then change the port number before the colon (:) like so -p 1234:3306. This maps port 1234 on your machine to the docker containers port 3306 inside the container.

Next, a root password and default database should be created. You could skip database creation and do it later with the management tool of your choice, but I find this easier. There’s two environment variables to set and I usually pick a short, insecure password for MySQL’s root account as this is only a test locally and firewalled on my dev machine. -e MYSQL_ROOT_PASSWORD=toor sets the root password to “toor” (root backwards. This was a default on a few Linux distros for a while). Setting the default database is just as easy -e MYSQL_DATABASE=dev. In this case it’s creating a database called “dev”.

Finally, I tend to name the docker container so I can run it again easily if required. I do this long hand with --name mysql57 where “mysql57” is the name of the container I’m creating. You can name this per project if it makes more sense for you, but I do regularly delete and recreate this container as it’s outside my usual dev workflow and usually just for debugging/fixing something once.

Creating a Named MySQL Docker Container Exposed to the Host

Rolling it all together you can run this command to create a named MySQL 5.7 instance that is running in the background (-d).

docker run --name mysql57 -e MYSQL_ROOT_PASSWORD=toor -e MYSQL_DATABASE=dev -p 3306:3306 -d mysql:5.7

Restore Backups to a MySQL Docker Container

If you have a database backup you need to restore, then it’s reasonably easy to pass it into MySQL, although if it’s a big database then it can take some time to do. This can be done by using cat to read the backup file and feeding it into the MySQL docker container. If you’re a user who doesn’t have native docker permissions (like on Ubuntu, which requires sudo docker) then it may be best to change to a user that does have permissions (sudo -i to switch to root, then run the backup restore command).

cat database_backup.sql | docker exec -i mysql57 /usr/bin/mysql -u root --password=toor dev

Backing up a Database Inside MySQL’s Docker Container

If you need to backup your MySQL docker database from the container, you can do so by running the mysqldump command that the container has installed by default, passing it container name, username, password and database you’ve defined when creating the container and defining the output file to save the database dump to.

docker exec mysql57 /usr/bin/mysqldump -u root --password=toor dev > dev_backup.sql

Cleaning up

Once you’re done with your MySQL container, you can stop and delete it by running the following commands, making sure to replace the container name (“mysql57”) with the name of your container if you happened to change it:

docker stop mysql57
docker rm mysql57

That’s it! You’ve created a named docker container running MySQL 5.7. You’ve exposed it to the host machine using port binding and learned how to restore a database backup to it. It’s not as useful as a full docker-compose stack for development. If you’re interested in a docker-compose dev environment check out this article. It does, however give you quick and easy MySQL access when you just need to poke around a database.

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Tumblr
  • Pinterest

Filed Under: Development, Docker, Linux, Technology, Uncategorized Tagged With: docker, mysql, mysql-server

Injecting inbound request headers with Traefik v2

2020/06/30 by sudo Leave a Comment

I’m using Traefik v2.2 as a reverse proxy for my docker containers. Basically all HTTP or HTTPS traffic is handled by Traefik as an ingress container and then routing according to rules defined in my docker-compose file to the appropriate internal container.

Something that I’ve needed to do for a project is add a header to an inbound request in order to identify that the request has been processed by traefik. I tried to follow the documentation but found it… lacking. For anyone interested in the official documentation for adding headers to Traefik you can find it here: https://docs.traefik.io/middlewares/headers/

What I’ve ended up with is a service container (nginx in my case) that looks like this:

nginx:
        image: nginx:stable-alpine
        volumes:
            - ./src:/var/www
            - ./.docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.router1.rule=Host(`localhost`)"
            - "traefik.http.middlewares.testHeader.headers.customrequestheaders.test-header=new-header"
            - "traefik.http.routers.router1.middlewares=testHeader"
    php:
        image: php:7.4-fpm
        volumes:
            - ./src:/var/www

Now something that threw me was the header “test_header” doesn’t exist in requests handled by nginx. A linked PHP container simply running print_r($_SERVER) dumps all the variables to the page. It’s only at this point I discovered that Traefik adds HTTP_ to the header. So instead of :

test-header = new-header

you get:

HTTP_TEST_HEADER = new-header

I think this is one of those things that you can waste alot of time on if you don’t know that Traefik is re-writing the header values in this way.

Looking back at the Traefik configuration, to add a header you basically have two steps:

First, create a new header. This is done in the following format:

traefik.http.middlewares.{your header group name}.headers.customrequestheaders.{header key}={header value}

As an example:

- "traefik.http.middlewares.testHeader.headers.customrequestheaders.test-header=new-header"

This creates a header `HTTP_TEST_HEADER` and assigns it to the `testHeader` middleware group.

Secondly, assign this middleware group to the router you’re using for your service. In my case that’s an nginx container on a router `router1`. This takes the following format:

traefik.http.routers.{router name}.middlewares={middleware name}

As an example:

- "traefik.http.routers.router1.middlewares=testHeader"

If you’re not sure what this looks like in the context of the other containers check the first, more complete example or see the entire docker-compose file below (NOTE: this file exposes the Traefik API for debugging purposes, so don’t blindly deploy it to production):

version: '3'

services:
    proxy:
        image: traefik:v2.2
        command:
            - "--providers.docker=true"
            - "--providers.docker.exposedbydefault=false"
            - "--entrypoints.web.address=:80"
            - "--api.insecure=true"
        ports:
            - "80:80"
            - "8080:8080"
        volumes:
            - "/var/run/docker.sock:/var/run/docker.sock:ro"

    nginx:
        image: nginx:stable-alpine
        volumes:
            - ./src:/var/www
            - ./.docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.router1.rule=Host(`localhost`)"
            - "traefik.http.middlewares.testHeader.headers.customrequestheaders.test-header=new-header"
            - "traefik.http.routers.router1.middlewares=testHeader"
            - "traefik.http.middlewares.test-ratelimit.ratelimit.average=1"
            - "traefik.http.middlewares.test-ratelimit.ratelimit.period=1m"

    php:
        image: php:7.4-fpm
        volumes:
            - ./src:/var/www

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Tumblr
  • Pinterest

Filed Under: Docker, Linux, Technology Tagged With: docker, traefik, ubuntu server

Setting up Display Link drivers on Ubuntu 20.04

2020/06/23 by sudo Leave a Comment

I’ve purchased a Dell “universal” USB 3/USB C docking station featuring Display Link, essentially allowing you to use it to drive external displays with the intention of using it for my laptop when sat at a desk with external monitor, mouse and keyboard. Since I run Ubuntu or Linux Mint as my primary operating system, some extra steps were required to install the Display Link drivers.

The first thing you need to do is download the Display Link drivers from the official website: https://www.displaylink.com/downloads/ubuntu

Once downloaded, it’s a reasonably straight forward process to install the drivers. First thing that’s required is to install pre-requisites

sudo apt-get install dkms libdrm-dev

Next run the downloaded file replacing the version number with whatever the one you downloaded is. (Note, this command assumes you’re in the appropriate directory already):

sudo ./displaylink-driver-5.3.1.34.run

Once finished you should have a message asking if you are running xorg and if you want to reboot. It’s actually best to reboot regardless of your display driver, so select Y for yes and let your machine reboot. Then you should be able to plug in your USB dock and your external devices with them all working.

For more information, the Display Link website actually has some good resources. Checkout their Ubuntu setup guide here: https://support.displaylink.com/knowledgebase/articles/684649

Share this:

  • Twitter
  • Facebook
  • Reddit
  • Tumblr
  • Pinterest

Filed Under: Guides, Linux, Technology Tagged With: DisplayLink, ubuntu, ubuntu 20.04

  • 1
  • 2
  • 3
  • …
  • 5
  • Next Page »

Recent Posts

  • Alternative GPIO pins for Wemos D1 Mini BME280 sensor
  • Fixing zsys daemon: timed out waiting for server handshake on Ubuntu 20.04 ZFS
  • Solving Error on statfs() system call with cAdvisor and Docker
  • MySQL in Docker for Local Development
  • Laravel Docker Development Environment With PHP7

Tags

7zip API auditing Courier MTA crochet data recovery debian debudding development Dingo API docker email Getting started with Laravel 5 & Dingo API hard drive health KVM Laravel larvel 5 lenovo Linux Mail Quota Minion mint netgear nas networking network shares php PHP development Postfix samba security SMART smartctl smartmontools smb smbfs testing traefik ubuntu ubuntu 18.04 ubuntu 20.04 ubuntu server vagrant Virtual machines xdebug xubuntu

© Copyright 2015 apt get life