apt get life

Life around technology

You are here: Home

Add a policy to ImageMagick in Debian

by

As Arstechnica have recently reported, there is a critical security vulnerability in ImageMagick, an image processing library used by many websites. While official patches are being worked on, this is a reccomended workaround to secure ImageMagick on Debian.

1. Check loaded Imagick policies

From a terminal, check to see if any policies are loaded. There will almost certainly be none returned if you’ve not configured any previously.

Check ImageMagick Policies
1
convert -list policy

2. Open Imagick’s policy.xml file

In Debian, you can find the ImageMagick policy file in /etc/ImageMagick:

Edit Policy.xml
1
nano /etc/ImageMagick/policy.xml

In other operating systems your best bet is to run a find:

Find Policy.xml for Non-Debian based systems
1
find /usr | grep "policy.xml"

3. Edit policy.xml

Now we need to edit policy.xml. The current reccomended settings related to the vulnerability are here: https://gist.github.com/rawdigits/d73312d21c8584590783a5e07e124723 It’s best to check this URL for the latest version. Alternatively here’s the code at time of writing:

policy.xml
1
2
3
4
5
6
7
<policymap>
  <policy domain="coder" rights="none" pattern="EPHEMERAL" />
  <policy domain="coder" rights="none" pattern="URL" />
  <policy domain="coder" rights="none" pattern="HTTPS" />
  <policy domain="coder" rights="none" pattern="MVG" />
  <policy domain="coder" rights="none" pattern="MSL" />
</policymap>

Remember it’s Ctrl + x to exit nano and you do want to save the changes. If you’re told you don’t have write permissions try closing the file and opening it again with sudo.

4. Check the Imagick policy is loaded

Now re-run the first step to make sure the policy has been loaded properly.

Check ImageMagick Policies
1
convert -list policy

You should get the following output:

policy output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#
Path: [built-in]
  Policy: Undefined
    rights: None
 
Path: /etc/ImageMagick/policy.xml
  Policy: Coder
    rights: None
    pattern: EPHEMERAL
  Policy: Coder
    rights: None
    pattern: URL
  Policy: Coder
    rights: None
    pattern: HTTPS
  Policy: Coder
    rights: None
    pattern: MVG
  Policy: Coder
    rights: None
    pattern: MSL

For more details on the problem, check out the ArsTechnica post here, and the ImageMagick forum announcement on the subject here.

Official patches are due to be distributed over the weekend, but may take longer to enter your distributions package manager.

Laravel 5.2 API Token Authentication

by

At work I’ve been tasked with improving an API recently, and I decided it would be a good opportunity to take Laravel out for a spin. I’ve been keen on learning more about laravel and it’s API capabilities which are supposedly very strong, although I have noted that there’s not much documentation around them. The existing API is flat PHP and uses token based authentication. This allows users to authenticate with a string “api_key” in the request URL, in the header or in the body of the JSON request. I decided that instead of trying to get existing users to upgrade to something like oAuth (for which there are some interesting plugins https://packagist.org/packages/lucadegasperi/oauth2-server-laravel), I’d just implement the same token based authentication model for the revised API in Laravel. There are already advantages to using Laravel for APIs – it highly encourages a restful approach, as for Laravel 5.2 it includes rate limiting out of the box and allows for route prefixing, so it is possible to have multiple endpoints in one Laravel application.

Setting up token based authenticaton in Laravel is so poorly documented that it took me a while to work out how it is achieved.

1. User API Tokens

Users need to have an API token to be associated with them in order to allow the authentication model to work. This is easy enough to add by editing the user migration in your laravel installation.

edit CreateUsersTable
1
2
// Store an API key for this user.
$table->string('api_token', 60)->unique();

This allows you to store a 60 character unique API Token for each user.

2. Setting up API Authentication

There are several ways you can now call API Token authentication for your application. Probably the best is to use middleware in your routes file:

Adding API authentication to your middleware
1
2
3
4
5
6
Route::group([
    'prefix' => 'api',
    'middleware' => 'auth::api'
    ], function() {
    Route::resource('fruit', FruitController);
});

Now any time requests are made to the route group, the API authentication method will be called. This includes token based authentication (now defined in the users table) as well as the API rate limiting.

3. Making API Requests

You can now submit your API requests to see if the Laravel token authentication is working. To do this you can submit “api_token” as either a GET or POST paramiter. There’s also hidden away the option to have it set as a header, however this requires you to use an Authorization header:

Key: ‘Authorization’

Value: ‘Bearer [token]’

Check out the code here:

https://github.com/laravel/framework/blob/c04159dee4a47b0d7cd508ab720932121927b1b3/src/Illuminate/Http/Request.php#L815-L822

and here:

https://github.com/laravel/framework/blob/master/src/Illuminate/Auth/TokenGuard.php#L81-L94

 

Linux servers – using ClamAV to find malware

by

ClamAV is an open source anti-virus program that can be run from the command line, making it incredibly useful for locating any viruses and malware on Linux based servers. Recently someone I’ve previously worked with reported that they’d had reports of abuse originating form one of their servers. Given the quantity of sites, it was difficult to locate any potential vulnerabilities.

1
grep -RPl --include=*.{php,txt} "(passthru|shell_exec|system|phpinfo|base64_decode|chmod|mkdir|fopen|fclose|readfile) *\(" /var/www/

 

Blindly grepping for potentially malicious strings such as “base64_decode” and “exec” was getting tired fast, as these can be legitimately used for some applications. I stumbled across reports that ClamAV works well for locating potential threats

Shell
1
nice -n 19 clamscan ./ -r -i | grep " FOUND" >> possible_exploits.txt

You can then review these files as you see fit, editing the file to remove ones that are false positives. I then run a command to delete the infected files:

1
while read f; do rm $f ; done<$possible_exploits.txt

 

HP Microserver Mods

by

HP Microservers are a low cost, small form factor cube-sh PC that are great for using as media centres or NAS boxes. I’ve already got a few microservers running FreeNas or Ubuntu server providing shared network drives or running as mass-backup boxes. Since this microserver was broken and relatively low cost, my first action was to investigate what’d gone wrong with it.

Amber power light: Failed PSU

When connected to the mains, the server’s power button was constant amber. It didn’t respond to button presses and wouldn’t power up. I did some research and it suggested that the most likely reason for the amber light was a stuck power button. I pulled the power switch connector from the mother board and used a multimetre on continuity test mode to check each pin pair until I found one that connected when the button was pressed. This indicated that the power switch was working fine, as it would complete a circuit when held down, and disconnect when released. I shorted the matching pins on the mother board, effectively becoming the power switch and the server still didn’t boot, which confirmed that this was not the problem. Delving deeper, I started dismantling the microserver to see if I could work out what had gone wrong – sure enough a quick visual inspection found the CD drive SATA power connector was a melted mess, and scorch marks were on the metal frame. The CD drive was dead, pins on the molex to SATA adapter having shorted inside the plastic housing. I pulled the PSU and found it to  be dead, it wouldn’t turn on when shorting the pins on the ATX header so a new power supply was in order.

Since HP microserver power supplies are quite expensive (~70 GBP), I wanted to make sure the motherboard still worked before paying for a replacement PSU. I used an old ATX power supply which I had floating around and connected it to the microserver. It’d never fit inside the case, so I just left it hanging around next to the microserver. It booted off a memory stick fine, and that was proof enough for me that I needed a new power supply.

I ended up finding this blog: http://chrisstark.co/2013/08/hp-microserver-n36-replacement-power-supply-psu/ which was someone in a similar position to me. They had a microserver which needed a new power supply, but found that LinITX had a similar replacement unit which was significantly cheaper than a genuine spare. LinITX is a fantastic site dedicated to embedded hardware and I’ve often browsed wishing I had enough cash to buy goodies from them. In this case, since the PSU was only £30 (GBP) I bought one straight away. This is a link to the PSU https://linitx.com/product/fsp-150w-1u-compact-power-supply-fsp15050gub/13062 and the quickfind code is FSP150-50GUB.

All back-plane cables are grouped by colour into a chocbloc, then a female molex cable attached to the other side.

All back-plane cables are grouped by colour into a chocbloc, then a female molex cable attached to the other side.

With the PSU connected, the only problem I now had was the SATA backplane requires 4 molex power connectors and the PSU only has one. Since this is likely going to be a “toy” microserver due to the cost and not-properly-tested nature of it, instead of investing in molex splitters, I decided to hack off the existnig backplane molex cables, group the connectors into colours, strip the ends and solder them together. Once this was done I used standard electrical choc blocks to connect one end into a standard molex female that I’d cut off previously. It looks messy but is reasonably electrically sound.

Case Mods

Now that’s done, I decided to have some fun. I’ve had an air brush for years and never properly gotten into it – mastering a skill takes time which I’ve never seemed to have. Plus it’s messy so you’ve got to have dedicated time to flush through cleaner when you’re done, and set up a dirty work area. While tempted to companion cube my micro server I decided in the end to just do black and yellow hazard lines. First off, I opened the door and lifted it slightly to release it from the hinge. I was then able to use a pair of pliers to remove the locking mechanism’s nut on the inside of the door and put this to one side for safety. Once the lock is off (this can take some slight wiggling of the lock cylendar) I was able to unclip the plastic from the back metal frame. I only wanted to paint the plastic, and was worried there would be ghosting effects if I left the metal frame inside the door. I masked off the area using masking tape, following the lines of the air holes on the door. I mixed by yellow paint 50/50 with water and tried to evenly coat the door. That didn’t go so well. Painted yellow Microserver doorAfter 2 coats I gave up and instead removed the masking tape. I then got out some aluminium paint and dry-brushed the edges of the door to give it a weathered look. My method for dry-brushing, since I’d never done it before, was to get a little bit of paint onto a dry paint brush, dab it on my cardboard floor covering to thin it out, and then lightly brush the edges of the door. It actually came out really well – yes I am surprised by that! Finally I reassembled the server with the new door, power supply and hacked molex connector. I still have plans for what to do with the CD drive bay – possibly an LCD screen indicating IP address and drive status, I might replace the lock with a handle of some kind and I may paint the rest of the server. I also need to decide what to install on it!

Almost there - Microserver with new PSU, hacked back-plane power connector and painted front door.

Almost there – Microserver with new PSU, hacked back-plane power connector and painted front door.

 

Getting started with Laravel 5 & Dingo API: 4. Transformers

by

Okay, so the last few lessons have got us up to the point where we’re able to send and receive data to the API, but there are some problems that need to be thought about:

  1. We’re exposing our database architecture – people can see orders have fields “order_ref”, “recipient_id”, etc.
  2. Our index functions are using “all()”, so they get all results from the database.
  3. We’re not validating our data before adding it to the database.
  4. We’re not authenticating users.

Lets star addressing these.

Transformers

Transformers are often used in APIs to obscure and abstract the database later from the responses provided to users. What this means is we “transform” what our database record field names are, and turn them into something else. Say in our database we were storing a field “recipient_name”. Instead of the API returning this to the user on a get request, we could use a transformer to return a different field name “name” for example. This obscures our database architecture so we’re not giving away our field names. Additionally, the abstraction here means that if we change our database architecture we’re not relying on API users to change their tools or utilities as well. We can change the database field names without worrying about what users are doing.

Variants Transformer

Once again, I’m going to start with the Variants as this is the smallest part of the API. All we do here is get variants, we don’t allow them to be added, updated or deleted. I’m going to start by looking at my project structure. At the moment we should have Http/Controllers/api and all of the controllers should be within this. It doesn’t really make sense to put transformers here, as they’re not controllers. Instead, I think we should make a new folder in the app directory, and lets version it too, incase different versions of the API use different transformers

1
2
mkdir app/Transformers
mkdir app/Transformers/V1

Now lets make a new VariantsTransformer.php file in that directory:

1
touch app/Transformers/V1/VariantsTransformer.php

Open that file in Atom and lets make our transformer

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<?php
 
namespace App\Transformers\V1;
 
// We need to reference the Variants Model
use App\Variants;
 
// Dingo includes Fractal to help with transformations
use League\Fractal\TransformerAbstract;
 
class VariantsTransformer extends TransformerAbstract
{
    public function transform(Variants $variant)
    {
            // Specify what elements are going to be visible to the API
            return [
        'id' => (int) $variant->id,
                'size' => $variant->size,
                'brand' => $variant->brand,
                'type' => $variant->type,
                'color' => $variant->colour,
                'design' => $variant->design,
        ];
    }
}

All we’re doing here is transforming our Database collection into an array and returning it. The left hand side of the array define the keys that will be used for the JSON response. The right hand side gets the variant fields from the database. What this empowers you to do is hide database fields – like created at – so there’s no risk they’ll be visible to the API. Only items in this array will be returned in API requests. The next key advantage is that the variant database field name doesn’t have to match the API field name. This means that if there’s a major database update that needs to take place, you can update the transformer and not have to ask customers to re-map everything in their APIs.

In the VariantsController, we need to change our functions to adopt the new transformer

PHP
1
use App\Transformers\V1\VariantsTransformer;

PHP
1
2
3
4
5
public function index()
{
    // Return variants via the Variants Transformer.
    return $this->collection(Variants::all(), new VariantsTransformer);
}

PHP
1
2
3
4
public function show($id)
{
    return $this->item(Variants::find($id), new VariantsTransformer);
}

I’ve added the show function here to return an individual variant instead of a collection. You’ll also need to add the route for it to work:

PHP
1
$api->get('/variants/{id}', 'App\Http\Controllers\api\VariantsController@show');

Now when you call the variants controller in postman, you should see array values coming through as keys in JSON, not database field names.

Following this theme we need to update the Items controller, as well as creating an ItemsTransformer, and the orders controller along with an OrdersTransformer.

ItemsTransformer:

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<?php
 
namespace App\Transformers\V1;
 
// We need to reference the Items Model
use App\Items;
 
// Dingo includes Fractal to help with transformations
use League\Fractal\TransformerAbstract;
 
class ItemsTransformer extends TransformerAbstract
{
    public function transform(Items $item)
    {
        // specify what elements are going to be visible to the API
        return [
            'id' => (int) $item->id,
            'item_ref' => $item->item_ref,
            'quantity' => (int) $item->quantity,
            'variant_id' => (int) $item->variant_id,
        ];
    }
 
    public function deform(Items $item)
    {
        // specify what elements are going to be visible to the API
        return [
            'id' => (int) $item->id,
            'item_ref' => $item->item_ref,
            'quantity' => (int) $item->quantity,
            'variant_id' => (int) $item->variant_id,
        ];
    }
}

ItemsController:

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// At the top of the file, include the items tranformer
use App\Transformers\V1\ItemsTransformer;
 
 
// Update the index function to use the ItemsTransformer
public function index()
    {
        return $this->collection(Items::all(), new ItemsTransformer);
    }  
 
// Update the show function to use the items transformer
public function show($id)
    {
        return $this->item(Items::find($id), new ItemsTransformer);
    }

OrdersTransformer:

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<?php
 
namespace App\Transformers\V1;
 
// We need to reference the Orders Model
use App\Orders;
 
// Dingo includes Fractal to help with transformations
use League\Fractal\TransformerAbstract;
 
class OrdersTransformer extends TransformerAbstract
{
    public function transform(Orders $order)
    {
        // specify what elements are going to be visible to the API
        return [
            'id' => (int) $order->id,
            'order_ref' => $order->order_ref,
            'recipient_id' => $order->recipient_id,
            'shipping_method' => $order->shipping_method,
        ];
    }
 
    public function deform(Orders $order)
    {
        // specify what elements are going to be visible to the API
        return [
            'id' => (int) $order->id,
            'order_ref' => $order->order_ref,
            'recipient_id' => $order->recipient_id,
            'shipping_method' => $order->shipping_method,
        ];
    }
}

OrdersController:

PHP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Include the orders transformer
use App\Transformers\V1\OrdersTransformer;
 
 
// change the index function to use the transformer
public function index()
    {
        return $this->collection(Orders::all(), new OrdersTransformer);
    }
 
// Change the show function to use the transformer
public function show($id)
    {
        return $this->item(Orders::find($id), new OrdersTransformer);
    }

 

Save all of your work, git commit it if you’re being safe (I won’t show you how to do that now, you know how by now!) and run  php artisan serve in order to start the local webserver. Use Postman to send requests to see if your updates are working.

 

One thing to note is that this only transforms the output of our database. Your incoming requests are not handled via the transformer. This is due to the design of fractal, and there is a discussion about it here if you’re interested. Instead of “transforming” input in this way, it’s suggested that models and validation are used instead. Next time we’ll look at running some validation on the input we’re sending to create new orders and order items, as well as working out how to apply the API architecture to the models.