RECAST3D is based on TomoPackets (ZeroMQ) for communication and OpenGL for visualization. It runs a single server, which communicates with a reconstruction server such as SliceRecon.
First, a brief summary of the RECAST3D design. The slicer environment is implemented in
ReconstructionComponent, but there is much more in RECAST3D. It has a flexible and transparent server that distributes TomoPackets to modules that want to know about them. Using graphics components and modules, it is easy to extend, improve, or change behaviour. The rendering is based on OpenGL, making it flexible but not trivial to extend. I provide texture, shader, meshes and primitive support, which makes it easier to make a new component. The immediate mode user interface is straightforward to use.
Graphics and scenes
The software can have multiple scenes. Each scene corresponds to the visualization of a single scan.
The important classes that realize the graphics are organized as follows:
The graphics are rendered directly using OpenGL, but there are helper classes available (for primitives, shader programs, textures, and so on).
Important shared interfaces for many classes in the graphics implementations are:
RenderTargetis for any component that wants to be rendered, such as a scene or the user interface. A scene eventually calls
drawon its components which takes into account the (3D) camera.
Tickeris implemented by any object that wants to update itself over regular time intervals.
Windowis implemented by any object that wants to add controls to the user interface
PacketListenerallows to easily send packets to upstream components of the reconstruction stack from any point in the graphics (for example, as a side effect due to user input).
We use ImGui for the user interface. This allows parts of the graphics stack to easily draw check boxes, sliders, and text input. This is done through the
There is also built in support for parameters of upstream components, using the
ParameterX packets of TomoPackets.
The server has a number of ‘modules’. Each module corresponds to a part of RECAST3D that is interested in handling certain packages. A selection of the implemented modules:
For example, the manage scene protocol listens only to incoming packets of type
MakeScenePacket, and whenever one comes in it adds a scene to the scene list. Something to note is that
read_packet happens on the server thread, while
process happens on the graphics thead. These are kept separate so that all OpenGL calls happen from within a single thread.
There is a server running the ZeroMQ
REP/REQ protocol that handles incoming packets from upstream (by default on port 5555), and a thread running a
PUB/SUB protocol that publishes outgoing packets to (multiple) listeners (by default on port 5556).
Example: Reconstruction of a re-oriented slice
Let us explore in detail what happens when the user changes a slice. This happens when the user translates along the normal of a slice (using the left mouse button), or rotates around the furthest edge from where they click (using the right button).
ReconstructionComponent optionally has a
ReconDragMachine which is initiated as soon as the user clicks on the mouse. There are two kinds: a
SliceTranslator and a
SliceRotator, which both have an
These machines actually deactivate a slice upon creation, and have a reference to a ‘ghost’ slice that is will be created when the mouse is released.
When this new slice is created, we have to let the reconstruction server know that this has occured. A
SetSlicePacket is created using the orientation of the newly created slice, and published using the
PUB/SUB servers to upstream clients that are subscribed to this packets. For example, SliceRecon registers to this packet.
The reconstruction server then computes a slice reconstruction for the slice, and sends this to the incoming port of RECAST3D (the
REQ/REP server). This is eventually handled by the
ReconstructionProtocol that listens to
SliceDataPackets. When this protocol handles the request, it updates the data of the corresponding scene, and this is ultimately reflected in the ReconstructionComponent which then shows the updated slice.
Note that this entire sequence happens within milliseconds, and is entirely distributed: RECAST3D and the reconstruction server do not have to run on the same computer, and in fact any agent can act as the reconstruction server.
Example: Extend RECAST3D to add control parameters
As an example of a feature that was added to RECAST3D, let’s discuss the steps it took to add control parameters to RECAST3D. The idea of control parameters is that upstream components can register a ‘parameter’ with RECAST3D, which then shows a UI element to change this parameter. When the user changes one of these parameters using the UI, a packet has to be sent to the upstream component that it belongs to.
- Control packets were added to TomoPackets (e.g.
ControlModulewere added to RECAST3D. Note that the control component does not draw anything in the 3D window, but only wants to add additional UI elements.
ControlModuleregisters itself as the handler for e.g. the
ParameterBoolPacketand others (actual implementation also includes benchmarking, and tracking).
- When a
ParameterBoolPacketis received by the module it gets sent to the appropriate
- The implementation of the
describe(), for drawing UI elements: one checkbox for each known parameter.
- If checkbox changed by user, we send a
After these changes, we can use this for real-time alignment parameter adjustment, changing filters, and so on, all without touching existing code in RECAST3D.