These datasets are provided in a binary format. There are four files—two for the training set and two for the test set—in gzipped IDX and IDX3 formats. Although it is a simple format, it is non-standard and requires custom code to parse.

For a professional project, I wrote the following two functions to load these files into NumPy arrays.

import struct
import gzip
import numpy as np

def mnist_images_idx_to_array(images_filename):
    images_f = gzip.open(images_filename, mode="rb")
    images_f.seek(0)
    magic = struct.unpack('>I', images_f.read(4))[0]
    if magic != 0x00000803:
        raise Exception(f"Format error: Need an IDX3 file: {images_filename}")
    n_images = struct.unpack('>I', images_f.read(4))[0]
    n_row = struct.unpack('>I', images_f.read(4))[0]
    n_col = struct.unpack('>I', images_f.read(4))[0]

    n_bytes = n_images * n_row * n_col  # each pixel is 1 byte

    images_data = struct.unpack(
        '>' + str(n_bytes) + 'B', images_f.read(n_bytes))

    images_array = np.asarray(images_data, dtype='uint8')
    images_array.shape = (n_images, n_row, n_col)

    return images_array


def mnist_labels_idx_to_array(labels_filename):
    labels_f = gzip.open(labels_filename, mode="rb")
    labels_f.seek(0)
    magic = struct.unpack('>I', labels_f.read(4))[0]
    if magic != 0x00000801:
        raise Exception(f"Format error: Need an IDX file: {labels_filename}")
    n_labels = struct.unpack('>I', labels_f.read(4))[0]
    labels_data = struct.unpack(
        '>' + str(n_labels) + 'B', labels_f.read(n_labels))
    labels_array = np.asarray(labels_data, dtype='uint8')
    return labels_array

You can use these functions by passing the appropriate filenames, as shown below:

training_images = mnist_images_idx_to_array(
    os.path.join(input_dir, "train-images-idx3-ubyte.gz"))
training_labels = mnist_labels_idx_to_array(
    os.path.join(input_dir, "train-labels-idx1-ubyte.gz"))
testing_images = mnist_images_idx_to_array(
    os.path.join(input_dir, "t10k-images-idx3-ubyte.gz"))
testing_labels = mnist_labels_idx_to_array(
    os.path.join(input_dir, "t10k-labels-idx1-ubyte.gz"))

There are basically two problems to address: NumPy arrays often have data types with more than 8 bits, and OpenCV reads images in BGR format rather than the more common RGB.

The following Python (3.5+) code demonstrates the solution:

import PySide2.QtGui as qtg
import numpy as np

def get_qimage(image: np.ndarray):
    assert (np.max(image) <= 255)
    image8 = image.astype(np.uint8, order='C', casting='unsafe')
    height, width, colors = image8.shape
    bytesPerLine = 3 * width

    image = qtg.QImage(image8.data, width, height, bytesPerLine,
                       qtg.QImage.Format_RGB888)

    image = image.rgbSwapped()
    return image

I’m building a face recognizer with dlib’s frontal face detector, and today, I noticed that some of the results return negative coordinates in face detection. This means the detected face is partial, although it’s a bit of a stretch to use negative coordinates for this.

My code wasn’t checking for negative coordinates and was building the NumPy array incorrectly. However, after reading the issue, although I’m still not convinced that it’s a good approach, I added a few if statements and the problem was solved.