The technology for Ottoman OCR was mostly ready before my family obligations interrupted the project. I need to re-evaluate what is currently available, but a more pressing problem for me is the speed of translation; currently, it is so slow that it’s barely usable.

I have written the dictionary as a C library without any database dependencies and integrated it into the Android version. I’m currently updating the search functionality in the mobile app to use this library instead of a web service. It will be orders of magnitude faster than the current version because it’s offline and uses a trie to store the words—making it both small and fast.

In my experience, having a single data structure with dedicated functions for transforming and indexing is much simpler than managing multiple data structures. In my case, the core structure is:

typedef struct _dervaze_lexical_item {
  int index;
  bstr latin_search_key;
  bstr latin;
  bstr visenc_search_key;
  bstr visenc_dotless_search_key;
  bstr visenc;
  bstr annotation;
  bstr meaning;
  bstr abjad;
  lexical_role role;
  int last_vowel;
  int props;
} dervaze_lexical_item;

visenc is our abbreviation for visual encoding, which is used to represent Arabic/Ottoman/Farsi words using basic ASCII letters. It is documented on its own page.

index is a unique identifier assigned to each word. Search keys for Latin, Visenc, and Dotless Visenc (e.g., searching for ﺥ using letters like ح, چ, or ج) are used to locate these lexical items via the tries mentioned earlier.

I’m intentionally avoiding UTF-8 or other Unicode encodings because terminal output is often not well-suited for displaying Arabic text correctly.

When I began writing this software, one of my goals was to enable searching for words by their traditional numeric values (Abjad). These are often used in classical Ottoman poetry to encode a date within a verse. For example, the letter Alif corresponds to 1, Ba (ب) to 2, and so on. To search for words by these numerals—for instance, typing 246 to find words with that total value—we store the value here as well.

The lexical_role is used during translation; currently, we use two roles to distinguish between noun and verb suffixes in Turkish.

last_vowel, as the name implies, represents the last vowel of the word. Since vowel harmony in Turkish is not always reflected in Ottoman spelling, we check the last vowel when converting an Ottoman word to its Latin Turkish equivalent to ensure the correct suffix is added (e.g., gemiler instead of gemilar).

props is a bit field used to represent various properties of the lexical item, such as:

#define HAS_FINAL_VOWEL 0x01
#define HAS_SINGLE_VOWEL 0x02
#define IS_LAST_VOWEL_HARD 0x04
#define IS_FINAL_CONSONANT_HARD 0x08
#define HAS_CONSONANT_SOFTENING 0x10

These properties are particularly important when converting Ottoman text to modern Turkish.

The current version of the translation engine is written in Python, but I plan to rewrite it in C this week. As this is the third rewrite, I don’t expect it to pose a major algorithmic challenge, although debugging the C version may be more demanding.

For example, in Ottoman, a word is spelled as mnwr (mim-nun-vav-ra), following the Arabic orthography, but in Modern Turkish, the spelling reflects the pronunciation as münevver. Since a one-to-one mapping is not possible between these two writing systems, a set of possible Ottoman spellings must be produced using a regular expression.

When the parser sees münevver, it should convert this to a pattern like mv?nh?(a1)?ww?h?r. This pattern produces a set of strings, where mvnha1vvhr might be the longest and mnwr the shortest. By performing a dictionary search, the system can verify that there is a word in Ottoman spelled as mnwr, thus selecting it as the correct spelling.

The dictionary in our study consists of word labels. The system will look up a set of handwritten word images after this label lookup and search for these images in the document. It can also create a set of candidate images from the regular expression by rendering each candidate spelling.