After a bit of juggling with the handle and eventually managing to open the door, it became apparent that there was a problem with the locking system. (See pictures).

Most new PVC replacement doors operate using a five point locking system, unlike a normal wooden door, where the door handle just operates one latch, and the key operates either the same latch or an additional bolt.