UnThread: Features
UnThread provides a variety of features to help developers achieve multicore parallelism in their code through mid-level multithreading, but without ever directly creating a thread. UnThread has features to solve a number of problems common to writing multicore code.
Problem: Thread creation and management is difficult. You do not want to underutilize available resources, nor do you want to overtax the system. Thinking in terms of threads and hardware is a distraction to your primary objective, which is to write code that executes faster. Furthermore, you need your code to run optimally whether it is on a single processor, a dual core, a quad core, or a server with 32 cores (plus hyperthreading).
Solution: UnThread TaskPools. UnThread TaskPools enable developers to replace thread creation and management with a pool of "task workers." The number of workers can be explicitly set, or automatically derived from the underlying hardware, i.e., the number of cores available to work. Instead of dealing with thread creation and management, you allocate a TaskPool, divide your single monolithic task into subtasks, and the fill the pool with tasks to be completed. The TaskPool handles thread creation and reuse, and automatically assigns workers to tasks as available, reporting back on progress, checking for abort signals, and performing task cleanup duties when complete.
Example: Suppose your application needs to process a large image file, in such a fashion that the processing of each output line requires several adjacent lines of the input file. You provide two routines - one that does the actual processing (a "task work routine") and one that dispatches tasks to the work routine (a "task executive routine"). This latter routine only needs to keep track of which line is to be processed next, and pass appropriate parameters to the work routine.
Problem: Shared memory among tasks requires synchronization that is difficult to program correctly, is prone to deadlocks if not done right, and is hard to code for efficient operation.
Solution: UnThread Shared Virtual Arrays. UnThread's patent-pending Shared Virtual Arrays (SVA) offer a simple, yet effective API to share data between tasks. Shared Virtual Arrays provide double-buffered shared memory with multiple read/write strategies optimized for how your tasks need to share information. UnThread's SVA's also provide automatic and efficient synchronization tailored to the read/write strategy specified.
Example: Return to the image-processing example above, and suppose that each line of the input file requires significant processing to produce (e.g., decoding a compressed image). Then, you might use a SVA to provide the file lines to the work routine - in this case, you provide a routine that produces a "master" array entry (line of the image file) and one that produces a "slave" entry from that master (a straight copy, in this case). Then, whenever a work routine needs a line from the input file, it requests a "slave" array entry - if the "master" has already been computed, only a copy is needed (possibly only a copy of a pointer)
Problem: Some tasks are resource competitive, while others are collaborative. How those tasks are assigned to cores can impact performance, but controlling core affinity is difficult and error prone.
Solution: UnThread Affinity. UnThread's patent-pending Affinity enables developers to identify tasks that share more than the usual amount of code and memory, and will automatically assign tasks to workers in the most optimal configuration available. UnThread Affinity takes into consideration cache levels, shared caches among cores as well as possible resource contention due to hyperthreading.
Example: Return to the image-processing example above, and assume that the size of the image file is such that the number of lines needed by the work routine will all fit into a processor's memory cache. In that case, two successive tasks will have substantial overlap in their input data, implying that, if these tasks are run on two processors that share a cache, the tasks may run slightly faster than otherwise. UnThread has the ability to attempt this processor scheduling, if possible.
Problem: Multiple tasks operating simultaneously can generate tremendous traffic in the form of output, logging, progress, etc. Efficiently managing the increased deluge of activity due to multicore execution is a challenge in and of itself.
Solution: UnThread Server. The UnThread Server is a traffic cop for task activity. A general purpose administrator, the Server waits for requests, responds, and then waits for the next request. By providing automatic queuing, request priority and sequencing, you can leverage the benefits of UnThread TaskPools, and leave the handling of the increased activity to an UnThread Server.
Example: You have a logging feature in your single-threaded application. After converting to mid-level multithreading utilizing UnThread TaskPools, your logging is now generated from multiple tasks. Your previous logging code, which had no need for synchronization, is now generating garbled output. An UnThread Server is a simple solution that can manage your existing logging code, queuing, and prioritizing logging requests, if needed, yielding sensible log output again, with minimal code changes to your logging routines.
Problem: There are many bookkeeping operations that are common to the needs of many executive dispatching routines.
Solution: UnThread Executive Bookkeeping. UnThread's Executive Bookkeeping features maintain a collection of pre-allocated thread resources for use by the work routine, as well as a task-counter, while ensuring that the leading and trailing tasks don't get too far apart.
Example 1: Suppose that a block of scratch storage is needed by each task. You would like to not allocate this block fresh for each task, but instead allocate blocks of storage once, at initialization and parcel these blocks out to each task as needed. UnThread's TRA (Thread Resource Array) can be used for this purpose - the executive requests an index at task-creation time, and the task worker (or task cleanup routine) releases the index when the task is finished. This index is used as a subscript into any collection of per-thread resources that are needed.
Example 2: Suppose that you would like to schedule your tasks in such a way that the leading task and trailing task never get "too far apart." This may be easily accomplished by UnThread's TSW (Task Status Window). Again, the executive requests a task number from the TSW which is released by the task worker (or task cleanup routine) when the task is finished. The TSW handles the bookkeeping of ensuring that no task number is given out that causes the leading and trailing tasks to spread too far apart (one or more threads in the pool idle until the trailing task catches up).
