🐇 It should only be needed to update the dependency versions in Cargo.toml, right?

🐢 Let’s start with that.

🐇 We have interface changes regarding aliases; let’s start using uv for building.

🦊 Added requirements to pyproject.toml by running:

uv add -r requirements.txt

These datasets are provided in a binary format. There are four files—two for the training set and two for the test set—in gzipped IDX and IDX3 formats. Although it is a simple format, it is non-standard and requires custom code to parse.

For a professional project, I wrote the following two functions to load these files into NumPy arrays.

import struct
import gzip
import numpy as np

def mnist_images_idx_to_array(images_filename):
    images_f = gzip.open(images_filename, mode="rb")
    images_f.seek(0)
    magic = struct.unpack('>I', images_f.read(4))[0]
    if magic != 0x00000803:
        raise Exception(f"Format error: Need an IDX3 file: {images_filename}")
    n_images = struct.unpack('>I', images_f.read(4))[0]
    n_row = struct.unpack('>I', images_f.read(4))[0]
    n_col = struct.unpack('>I', images_f.read(4))[0]

    n_bytes = n_images * n_row * n_col  # each pixel is 1 byte

    images_data = struct.unpack(
        '>' + str(n_bytes) + 'B', images_f.read(n_bytes))

    images_array = np.asarray(images_data, dtype='uint8')
    images_array.shape = (n_images, n_row, n_col)

    return images_array


def mnist_labels_idx_to_array(labels_filename):
    labels_f = gzip.open(labels_filename, mode="rb")
    labels_f.seek(0)
    magic = struct.unpack('>I', labels_f.read(4))[0]
    if magic != 0x00000801:
        raise Exception(f"Format error: Need an IDX file: {labels_filename}")
    n_labels = struct.unpack('>I', labels_f.read(4))[0]
    labels_data = struct.unpack(
        '>' + str(n_labels) + 'B', labels_f.read(n_labels))
    labels_array = np.asarray(labels_data, dtype='uint8')
    return labels_array

You can use these functions by passing the appropriate filenames, as shown below:

training_images = mnist_images_idx_to_array(
    os.path.join(input_dir, "train-images-idx3-ubyte.gz"))
training_labels = mnist_labels_idx_to_array(
    os.path.join(input_dir, "train-labels-idx1-ubyte.gz"))
testing_images = mnist_images_idx_to_array(
    os.path.join(input_dir, "t10k-images-idx3-ubyte.gz"))
testing_labels = mnist_labels_idx_to_array(
    os.path.join(input_dir, "t10k-labels-idx1-ubyte.gz"))

pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

I noticed that pip has a --default-timeout parameter that can be configured to avoid this issue:

pip --default-timeout=1000 install package-name

There are basically two problems to address: NumPy arrays often have data types with more than 8 bits, and OpenCV reads images in BGR format rather than the more common RGB.

The following Python (3.5+) code demonstrates the solution:

import PySide2.QtGui as qtg
import numpy as np

def get_qimage(image: np.ndarray):
    assert (np.max(image) <= 255)
    image8 = image.astype(np.uint8, order='C', casting='unsafe')
    height, width, colors = image8.shape
    bytesPerLine = 3 * width

    image = qtg.QImage(image8.data, width, height, bytesPerLine,
                       qtg.QImage.Format_RGB888)

    image = image.rgbSwapped()
    return image

Instead, we use parameter passing to the database engine. For example:

SELECT * FROM people WHERE name = ?

We send this query and the parameters separately to the database. Most modern database systems support this approach.

In SQLite 3 with Python, we use it like this:

query = "SELECT * FROM people WHERE name = ?"
params = (name,)
db_result = cursor.execute(query, params)

However, when debugging, we may need to see the actual queries sent to the database—for instance, when data types are important or when we suspect a column is receiving a string instead of an integer.

In these cases, rather than manually reconstructing the query, we can use the set_trace_callback feature available in Python 3.3 and later:

connection.set_trace_callback(print)

The argument can be any function (such as print or a logger function) or None to disable tracing. This makes it easy to integrate with Python’s standard logging module.

I’m building a face recognizer with dlib’s frontal face detector, and today, I noticed that some of the results return negative coordinates in face detection. This means the detected face is partial, although it’s a bit of a stretch to use negative coordinates for this.

My code wasn’t checking for negative coordinates and was building the NumPy array incorrectly. However, after reading the issue, although I’m still not convinced that it’s a good approach, I added a few if statements and the problem was solved.

Python doesn’t have C-style static variables natively. (Although it supports class variables, which can be used for a similar purpose in OOP.) However, since functions are also objects in Python, it’s possible to embed variables inside the function. An elegant solution on Stack Overflow creates a decorator for static variables.

def static_vars(**kwargs):
    def decorate(func):
        for k in kwargs:
            setattr(func, k, kwargs[k])
        return func
    return decorate

@static_vars(counter=0)
def foo():
    foo.counter += 1
    print(f"Counter is {foo.counter}")