Rust has fearless concurrency for memory access, but for the file system, there seem to be no built-in locked access primitives.

I decided to write one. Parallel execution of file system operations is important for Xvc. The error messages are annoying; having identical files in a repository is common, and in those cases, these messages look like there is a problem. In theory, when files are identical, having only one of them written to the cache is not a problem, but there may be other issues preventing cache access. We could just swallow the error and get away with it, but that’s not ideal.

Two options came to mind. One is modifying the list of cache files before creating threads so that no two threads access the same cache file at the same time. This is hard to implement and brings extra complexity to thread creation. A one-in-a-thousand concern becomes an architectural burden.

The other solution is to lock paths while accessing them so two threads working on the same cache path wait for each other. This is easier and requires just dependency injection into the thread functions. It has the downside of making file system operations slightly slower, as each path operation will now require checking a mutex, but that seems of little concern for file system access, which is already much slower than memory operations.

First, I tried to implement this with a HashMap<PathBuf, Mutex<()>>, but this must also be passed to the function wrapped in Arc<Mutex<HashMap>>, which made the “ceremony” of acquiring the lock for a single file much longer.

The issue is that you don’t want the HashMap itself to be a bottleneck. Multiple threads shouldn’t wait for the HashMap to become available, as it’s not the HashMap we want to lock, but the values inside it.

The solution is to return the lock value from a method of a struct. Something like:

pub struct PathSync {
    locks: Arc<RwLock<HashMap<PathBuf, Arc<Mutex<()>>>>>,
}

Now the ceremony can be performed within the method, and threads working in different directories won’t need to wait for the hash map to become available.

However, after implementing this, I realized I’d probably forget to lock a path at some point. This is a general-purpose solution, and I should apply it to all path operations when multiple threads are working. In most cases, there are multiple paths to lock (cache_path, cache_dir, repository path), and if I forget to lock one of them, a future user, some time, somewhere, will probably see an error message.

So, I decided on a different approach: creating wrappers to run passed closures. This makes it much more obvious that paths Xvc works on must be locked before operations.

    pub fn with_sync_path(
        &self,
        path: &Path,
        mut f: impl FnMut(&Path) -> Result<()>,
    )

This works by passing the path and a closure that operates on that path. It first locks the path and then runs the closure. The locking mechanism allows threads with different paths to run in parallel, but if they try to operate on the same path, they will wait for each other.

The implementation is here.

• The Disruptor is a concurrency framework created by LMAX to achieve very low latency and high throughput on their Java trading platform.
• LMAX found that using queues between system stages introduced latency, leading them to focus on optimizing this area.
• The Disruptor uses a lock-free “ring buffer” approach to avoid cache misses and locks at the CPU level, which are very costly.
• The Disruptor is intended as a general-purpose solution for concurrent programming, not just for financial applications.
• Applying the Disruptor pattern is not as simple as replacing all queues with the ring buffer; the user guide provides further guidance.
• Various blogs, articles, technical papers, and performance tests explain the Disruptor’s inner workings.
• A presentation and discussion group are also available for learning about the Disruptor from its creators.
• Martin Fowler has reviewed the Disruptor’s use at LMAX.
• The Disruptor is significantly faster than the ArrayBlockingQueue, as shown in the latency histogram.
• More performance results are available comparing the Disruptor to other approaches.

Note that there may be thousands of dependencies (e.g., files for a step), and creating these channels beforehand is not feasible.

The Crossbeam library has Select which allows waiting for an arbitrary number of channels. Now, I need to figure out how to structure the threads and update their states. A minor task! 😆

{
  var1
  var2
  {
    var3
    var4
  }
}

The inner scope variables can’t be accessed by the outer scope, but outer scope variables can be accessed by the inner scope. When we are in the inner scope, the stack contains both scopes’ variables. Stack-based memory management can be called table d’hôte. It’s faster and simpler.

However, in order to allocate them at once, variable sizes must be known beforehand. If the compiler doesn’t know the size of an array, it can’t allocate the array on the stack. We need another method to use these indefinite variables.

The other way of obtaining memory is using the heap. The heap is independent of the scope. You can allocate and fill it when you need with a size determined at runtime, share it with other scopes, and free it à la carte. Rust provides several different ways to manage memory on the heap.

In my experience, making the program flow as scoped as possible without cross-cutting memory management is a good sign. Ideally, a program should have a tree-like flow of execution, and the number of heap allocations is minimized in this case. However, most modern software systems are not designed this way, and memory management is not considered during the requirements phase. When you leave this to a garbage collector, even if it does its best, memory management often remains haphazard.

Ideally, when you run the program, it should call a series of functions that branch into further sub-functions:

flowchart LR

  main--->fn1
  main--->fn2
  main--->fn3
  main--->fn4
  fn1--->fn1.1
  fn1--->fn1.2
  fn1--->fn1.3
  fn1--->fn1.4
  fn2--->fn2.1
  fn2--->fn2.2
  fn2--->fn2.3
  fn2--->fn2.4

  fn1.1-->fn1.1.1
  fn1.1-->fn1.1.2
  fn1.1-->fn1.1.3
  fn1.1-->fn1.1.4
  fn1.1-->fn1.1.5

If we adhere to this principle, building concurrent software becomes a lot easier, too. If the functionality and memory of fn1.1.1 and fn2 are completely separate, these can be run in parallel. Hence, our aim in software architecture must be looking for ways to represent the problem at hand as a tree structure, both physically (for memory) and temporally (for execution).

My current project on word spotting in historical documents is fairly complete in functionality, but I decided that searching for word images on page images concurrently would be better for speeding it up. I’m already using Boost for much of the functionality, and instead of creating a dependency on the not-yet-mature C++11 support in various compilers, I decided to use boost::thread.

Suppose we have a functor such as:

class Search_t
{
   public:
       Search_t(Document d) { ... };
       SearchResult operator()(SearchItem i) { ... };
};

And we want to use this functor in multiple threads. We can’t simply do the following:

std::vector<SearchResult> results;
Search_t search(document);
// search_items is a vector<SearchItem> and si is an iterator over it.
for (si = search_items.begin(); si != search_items.end(); ++si)
{
   boost::thread task(boost::bind(search, *si));
   results.push_back(task); // ERROR!
}

This is because task does not return a SearchResult.

Instead, we need to store results within the object and retrieve them after they are generated.

I didn’t want to change the interface of Search_t because multithreading should be optional, and other parts of the program may depend on this interface. Instead, a wrapper class that runs these threads with a similar interface seemed like a better solution.

class SearchMT_t
{
   boost::shared_ptr<std::vector<boost::unique_future<SearchResult> > > futures_;

   public:
   SearchMT_t(Document d) :
   /* The most important assumption here is that Search_t does not alter
      the Document object's state in any way. Otherwise, we need to ensure that 
      document_ is accessed by only a single thread at a time using mutexes. */
   document_(d),
   futures_(new std::vector<boost::unique_future<SearchResult> >)
   {};

   void operator()(SearchItem si)
   {
       /* If you are sure that there won't be any race conditions
       between Search_t threads during search, you can move the following line
       to the constructor and use a single object for all searches. */
       Search_t search(document_);

       boost::packaged_task<SearchResult> search_task(std::bind(search, si));
       futures_->push_back(search_task.get_future());
       boost::thread task(boost::move(search_task));
   };

   std::vector<SearchResult> results()
   {
      std::vector<SearchResult> results;

      /* Wait for all threads to complete their work. */
      boost::wait_for_all(futures_->begin(), futures_->end());

      for(int i = 0; i < futures_->size(); ++i)
      {
          results.push_back((*futures_)[i].get());
      }

      return results;
   }
};

This way, it becomes much more straightforward to use multithreading in a loop:

std::vector<SearchResult> results;
SearchMT_t search(document);
// search_items is a vector<SearchItem> and si is an iterator over it.
for (si = search_items.begin(); si != search_items.end(); ++si)
{
    search(*si);
}

results = search.results();

We kept the Search_t class intact and used a much simpler approach in the loop.