Chapter2: Blender “DNA” and Serialization #
The blend file also provides a version’s “DNA” struct definitions called SDNA and information on pointersize and big- vs. little-endian byte order on the host machine that originally saved the file.
(page: 26)
SDNA is short for Structure DNA.
To serialize a blend file, Blender writes to a file a variable called DNAStr. DNAStr is a char array, which is the “DNA1” file-block of a blend file. DNAStr is defined in a file named dna.c and located in the makesdna mirrored directory of the source repository. In other words, dna.c is deposited within the build directory of the same name and relative path. The source file dna.c is not part of the repository source. It is generated from the executable makesdna, compiled and ran during the Blender build process from the file ource/blender/makesdna/intern/makesdna.c.
(page: 32)
The header dna_rename_defs.h is interesting, in that the redefinition macros written there are expanded for the macro definition from the DNA_alias_maps() implementation, defined in dna_utils.c (see Listing 2-3). Conversions are made in the macro expansions for older fields and “DNA” struct names. These are added to hash maps for lookup during blend file loading of prior versions of the Blender executable.
(page: 32)
This is how blender is compatible to old blender file.
The file dna_genfile.c provides functions for creating and accessing a SDNA struct, along with writing the “DNA1” file-block to a blend file. The dna_utils.c file provides low-level API functions.
(page: 34)
Before we can appreciate this process, there are a couple of core Blender concepts to understand. First, the top-level data structure for maintaining Blender’s state is a struct named struct bContext. It is defined, appropriately, in source/blender/blenkernel/intern/context.c.
(page: 35)
This struct contains two additional struct variables (wm and data), each with their own pointers to data structures describing UI (userinterface) elements and scene contents, respectively.
(page: 36)
The data struct in struct bContext holds two other sub-structs: one of the struct Main type and another of struct Scene. The struct Main is defined in source/blender/blenkernel/BKE_main.h, whereas the struct Scene is defined in source/blender/makesdna/DNA_scene_types.h. While there is a vast amount of state stored in both of these data types, it is perhaps more instructive to consider only parts of these records with the most obvious information first. Broadly, scene objects are contained in struct Main, but information related to the scene view is stored in struct Scene (which further uses struct Camera and struct World to encapsulate data, i.e., both of these structs contain physics, audio, and background color settings).
(page: 36)
In source/creator/creator.c’s main(), there are many calls to initialize various parts of the application. The first call directly related to the Blender’s DNA types is DNA_sdna_current_init(), defined in source/blender/makesdna/intern/dna_genfile.c. This function assigns an SDNA struct object to a global static variable called g_sdna. The g_sdna variable is accessible via the extern accessor functions of dna_genfile.c.
(page: 38)
SDNA (see Listing 2-6) is a meta-record, providing information about all of the potential serializable data for the version of Blender that is running. This struct is defined in source/blender/makesdna/DNA_sdna_types.h. It is the same SDNA that is written into a blend file.
(page: 39)
After having created an instance of the singleton SDNA object, we return back to main() and subsequently call BKE_blender_globals_init(). Another global struct, of type Global, is allocated a block of memory, the size of struct Main, and assigned to a pointer called main in Global. The allocation is performed by Blender’s memory allocation module source/intern/memutil, wrapping C library functions for memory management. This will be the struct Main to be filled out by reading either the char array of bytes representing the in-memory startup.blend for a “factory startup” or from a serialized binary data file (arbitrary blend file).
(page: 40)
There is another executable binary called datatoc, not to be confused with the module datatoc, that outputs the source file startup.blend.c. The startup.blend.c file is generated at build time and therefore deposited in the build directory. Within startup.blend.c is a char array with the “factory” startup.blend file contents. The char array is called datatoc_startup_blend.
(page: 41)
Upon entering WM_init(), there are initializations, many of which set callback functions for the windowmanager module.
(page: 41)
Functions that are prefaced with capital letters that abbreviate their respective modules form part of the module’s API (i.e., functional interface) for other Blender modules. For example, we have seen this with WM_init(), from the windowmanager API, and BLO_read_from_file(), etc., from the blenderloader module. Functions that maintain the same abbreviations (e.g., wm_homefile_read() or blo_filedata_from_memory()), but in lowercase, are not meant to be called from outside of the module itself.
(page: 44)
CTX_* are struct bContext accessor functions defined in source/blender/blendkernel/intern/context.c. context.c is located in the blendkernel module, and its interface is defined by BKE_context.h. However, because of the importance of the struct bContext type, Blender uses the CTX_* prefix instead of the BKE_* prefix on the API to the struct bContext type.
(page: 53)
Chapter 3: ghost: Soul of the windowmanager Module #
Generic Handy Operating System Toolkit (GHOST)
We will see that ghost directly interfaces with Xlib (X11), Mac OS, or MicroSoft Windows in order to create an application window. Native OS windows are the receivers of events from an operating system. Thus, ghost captures these events, and an application built upon ghost must then implement an appropriate handler routine.
(page: 57)
Both GLUT and GLFW2 take over an application’s event loop, known as “inversion of control.” We have already seen in Chapter 1 that Blender’s event loop is in the windowmanager module (source/blender/windowmanager/intern/wm.c). As a comparison, GLFW is considered a “framework,” as the library itself runs the event loop. A client application using GLFW must register callback functions with it. In contrast, the windowmanager module maintains the event loop for Blender and is written on top of ghost.
(page: 57)
Like GLUT and GLFW, ghost deals directly with the platform’s API. An application using ghost only interacts with ghost, not the platform API. Thus, ghost makes Blender portable and simplifies the duties of the “core” codebase.
(page: 59)
ghost began as an internal library, whose only dependencies were operating system APIs external to Blender’s repository.6 ghost’s internal dependencies are shown in Figure 3-2. Only “drag-and-drop” references source/blender/imbuf. The dependency arises from ghost/intern/GHOST_EventDragDrop.h, where both IMB_imbuf.h and IMB_imbuf_types.h
are included.
(page: 61)
GLEW (Graphics Library Extension Wrangler) is an external open source library, used for OpenGL extension functions. The library gathers extensions for an application, as its name suggests. Blender’s glew-mx extends GLEW, supporting extensions for multiple rendering contexts. GLEW itself has limited support for this.
(page: 62)
As the name implies, GHOST_ISystem is an abstract base class for an object-oriented representation of the operating system. We see its class diagram in Figure 3-5. GHOST_ISystem is instantiated as a singleton. It has a static factory member function called GHOST_ISystem::createSystem(), and a static accessor member function called GHOST_ISystem::getSystem(). Additionally, its constructor is protected, enforcing that only GHOST_ISystem::createSystem() be allowed to create an instance, when called from one of its child class (see Figure 3-5).
(page: 64)
GHOST_System inherits from GHOST_ISystem, an abstract class. GHOST_System therefore implements members found in concrete platform classes. Two examples are GHOST_System::getMilliSeconds() and GHOST_System::installTimer(). Figure 3-6 shows diagrams for two of these “system” classes: GHOST_SystemWin32 and GHOST_SystemX11. These represent Windows and X11 systems, respectively.
(page: 65)
Blender is written in C. However, ghost is a C++ library. Thus, Blender needs to interface ghost using a function-only interface. This is where intern/ghost/intern/GHOST_C-api.cpp and its associated header intern/ghost/GHOST_C-api.h are involved. They work as an adapter for the twoway C to C++ communication.
(page: 67)
In C programs using ghost, handles are C struct pointers. In the C++ API implementation, these are cast to C++ object pointers. Parameters passed to the C API functions are transferred to the member functions of the corresponding C++ class.
(page: 68)
Chapter 4: The blenlib and blenkernel Modules #
blendlib handles generic utilities in Blender, while blenkernel is more specific to the Blender application. Therefore, blenlib’s functionality, at least in principle, could be used by other similar programs. However, blenlib is tightly coupled to the Blender codebase.
blenkernel, on the other hand, offers a number of API functions for making needed manipulations on Blender “DNA,” and other statekeeping data structures.
(page: 109)
Chapter 5: Blender’s Embedded Python #
CPython is a C-based implementation of Python. It even contains an API. This allows CPython to be used as an external library and linked with a separate application written in C. Functions defined by an “embedding” program may be called via Python script, running on the embedded
interpreter.
(page: 110)
Because Python is object-oriented, Blender creates “built-in” Python modules and classes, some of whose methods map to Blender’s internal C structs (as in the case of struct bContext) and internal module APIs, for example, the bmesh module. mathutils’s mathematical objects, such as vectors and matrices, are assigned a Python class. Math utility functions from blenlib become methods for these same Python classes.
(page: 111)
PyModule_New() and PyModule_Create() both are used to add an extension module. For instance, Blender’s Python mathutils module is added by PyModule_Create(), while its bpy module is via PyModule_New(). PyModule_New() was introduced in version 3.3 of the CPython, after PyModule_Create(). You will see both functions used for creating new
modules in Blender.
(page: 115)
PyModule_AddObject() adds a class to a module. For example, the mathutils.Vector is added as a built-in Python class using PyModule_AddObject(). PyImport_ExtendInittab() registers modules to be included in Python’s initialization table (i.e., Inittab from the function name). This table contains extension modules to be registered when Python is initialized.
(page: 116)
In C, there is no explicit language mechanism for inheritance. By adding common fields to the beginning of a user-defined struct, we can treat these structs as having inherited those fields.
(page: 116)
PyObject_HEAD is expanded by the preprocessor into two fields. One field points to an object of PyTypeObject, the other a count of the object’s references.
(page: 117)
Chapter 6: Blender “RNA” and the Data API #
An additional aspect to the makesrna module is that multiple source files are generated during the build process. They are then fed back into the source code compiled as the runtime Blender executable. This means that makesrna is distinct from other Blender modules. It is implemented in this way, so that objects of “RNA” properties, that is, structs definitions with property data values such as initial settings, value ranges, and accessor function pointers, are not hand-coded.
(page: 137)
The makesrna module has both a runtime and non-runtime portion. The non-runtime portion is dedicated to source code generation. An individual file in the repository’s makesrna module may include both runtime and non-runtime code. Runtime code becomes part of the Blender executable. Non-runtime code is compiled as part of the tools that generate runtime code.
(page: 138)
As such, many of the source files in the repository are compiled twice, first for the non-runtime and then for the runtime after RNA_RUNTIME is defined by the generated source.
(page: 139)
rna_*_api.c functions implement callbacks triggered when an object is updated by the Data API, usually in response to values changed in the user interface or Python API call via an operator.
(page: 142)
Chapter 7: The editors Module, Operators, and Event System #
We will cover the connections between editors and the windowmanager module—as it is responsible for initializing operators and editors—as well as being the hub for events.
(page: 173)
The editors module, put succinctly, contains the code that defines the different views and operations on Blender data. Blender’s graphical user interface (GUI) has a number of editor categories, for example, “General,” “Animation,” “Scripting,” and “Data.” A few editors are “3D Viewport,” “Image Editor,” “Text Editor,” “Outliner,” etc. Each editor provides a separate view, where each visualizes different parts of the Blender data or “DNA.” Editors also usually allow for changing (e.g., editing, deleting, or adding to) “DNA,” where upon their view is updated accordingly.
(page: 173)
The space_outliner.c file contains the registration function, called from the windowmanager module. The callbacks (not the creation and initialization functions) are defined in the other files. This is the usual pattern for the other editors, unless the editor’s implementation is relatively short and entirely within a single space_*.c file, for example, the contents of space_topbar.
(page: 176)
The SpaceLink struct is the “base” type for “concrete” editor types. Each SpaceLink allows “linking” into the linkedlist of all other editors in Main, using its next and prev fields. This is a convention in the Blender codebase.
(page: 177)
Looking at the SpaceLink variables, you can see a field named spacetype. The spacetype field stores a numerical code from 0 to 255, representing the type of editor. These values are defined as an enum, called eSpace_Type, defined in ource/blender/makesdna/DNA_space_types.h.
(page: 178)
There is a tight coupling between the windowmanager and editors modules. All editors must live in a “space” (sometimes called a “Screen Area,” or more simply “area”) within a Blender application window.
At the highest level, all Blender windows are managed by the windowmanager module, as its title implies. We saw that the windowmanager module is a wrapper for GHOST, which is itself a wrapper for the underlying operating system’s windowing API.
(page: 182)
wmWindow represents a Blender window object. It has a pointer to a GHOST window (its void *ghostwin field), the underlying application window created by GHOST from the host operating system.
(page: 184)
wmOperator is implemented in the makesdna module, thus making it Blender “DNA” and serializable.
(page: 192)
Note that the wmOperator struct type is defined in makesdna, making it persistent. This allows for the operator stack to be saved to the blend file as well.
(page: 193)
An event (e.g., a specific set of pressed keys) may be context dependent. The context is determined by the location of the mouse pointer in the application window and the mode that the application is in when an event occurs.
(page: 194)
ghost_event_proc() handles events that are solely at the GHOST window-level. What is an event at the GHOST window-level? These events are anything not semantically Blender-specific, and therefore more general to an application’s behavior. Examples are window resizing and application closing, among other application behaviors.
(page: 198)
Let us look at how ghost_event_proc() treats the GHOST_kEventOpenMainFile event. Refer to Listing 7-18 once again. WM_operatortype_find() is used to look up a registered operator (an wmOperatorType object). In this example, the operator-type struct is accessed via the hashed char array “WM_OT_open_mainfile,” from global_ops_hash—a GHash hash table.
GHash is implemented in source/blender/blenlib/intern/BLI_ghash.c. You will recall we covered the blenlib module in Chapter Four. BLI_*API functions are used to access and manipulate a GHash object. This particular data structure is used, as it is O(1) in search time on average.
This is necessary, because of the timing requirements of handling events. Obviously, there is a trade-off for speed over storage.
Once the proper wmOperatorType object is found, Data API calls are made to prepare the relevant “RNA” properties data (with “display_file_selector”). Then these properties are passed to WM_operator_name_call_ptr().
(page: 198)
We can see in ghost_event_proc() that events which eventually go on to be handled at lower levels (i.e., in “areas” or “regions”) need to go on the active window’s event queue. For the shown events “GHOST_KEventButtonDown” and “GHOST_KEventButtonUp,” wm_event_add_ghostevent() is called.
(page: 200)
Window level events are handled by ghost_event_proc() directly. Events lower levels are sent to queue by function wm_event_add_ghostevent()
The purpose of wm_event_add_ghostevent() is to process GHOST events, by further sorting and then adding information (mouse position, Blender keypress code, etc.) to a wmEvent object.
(page: 201)
After wm_window_process_events() returns in WM_main(), wm_event_do_handlers() is called.See Listing 7-20. In essence, the wm_event_do_handlers() does the following:
• Loops over all wmWindow objects (i.e., the Blender windows)
• Retrieves the active Scene and ViewLayer objects
• For each wmWindow object
• Traverses its respective event queue
• For each such event (a wmEvent object)
• Finds the Editor region the event occured
• And then, calls wm_handlers_do() passing the corresponding AreaRegion object
(page: 202)
wm_handlers_do(), with further processing for multi-click events (not shown), calls wm_handlers_do_internal(). In wm_handlers_do_internal() calls the appropriate wm_handler_*_call(), which is eventually responsible for calling the appropriate callback function in the wmOperatorType. Also, during editor registration, the ARegion struct is provided a list of wmEventHandler structs (implemented in source/blender/windowmanager/wm_event_system.h), defining which event types a region should process.
(page: 204)
Chapter 8: Editor Creation #
The last function call in this loop is wm_draw_update(). It is in this call, after Blender’s data has been updated in the given loop, that we draw the ARegion “data-blocks” for each of our ditors.
(page: 207)
In Listing 8-2, we show wm_draw_window(). Only regions tagged for redraw are copied from an offscreen buffer (i.e., “blitted” to video memory) to an eventual front-buffer, to be shown on the computer monitor.
(page: 209)
In practice, it is better to implement such UI in the Python scripts that run at startup. This is more flexible and nearly as efficient, once registered and run from memory. However, using C in this chapter was intended as an introduction to the UI_*.
(page: 215)
Each editor is registered from the windowmanager module during WM_init(). From WM_init(), ED_spacetypes_init() is called, which itself then calls each of the editors’ separate registration functions.
(page: 216)
The UI_* API is rather extensive. Blender “RNA” wraps it, for the Blender Python API, as much of the UI is scripted in Python. However, for illustration purposes, we use the UI_* API directly in our editor.
(page: 227)
You will recall that we used the source/blender/editors/interface/interface_intern.h in our tutorial editor. This was to gain access to the struct uiBut, allowing us to manually assign its optype field with the wmOperatorType. Normally, you will be adding buttons via the Blender Python API, so including interface_intern.h in your editor’s source file is neither necessary or advised—in order to maintain the preferred practice of encapsulation. The Blender codebase
enforces this by not placing the implementation of the struct uiBut in the UI_*.h files. When code tries to access the struct uiBut outside of editors/interface/, while not using interface_intern.h, a “dereferencing pointer to incomplete type” compile error results.
(page: 232)
If you are familiar with the Blender Python API, you will know that UI elements can be added via Python scripts. You will also be aware that operators can be registered via Python, and the python module will use the callbacks defined in Python scripts for the operator’s functionality. In this chapter, we did everything in C—the language of the “core” codebase.
(page: 236)