Tutorial:
Writing A Simple Shader Extension
For Mental Ray & XSI

Creating A Constant Shader
For Minimal Complexity

Daniel Joseph Pezely
www.play.org
22 January 2003


Be sure to download the source code
(12k gzip'd tar image) so all the links below work

Contents

  1. Introduction
  2. Overview
  3. Custom Shader Basics
  4. Source Code
  5. Building From Source
  6. Using The Custom Shader
  7. Verifying The Shader Really Works
  8. Uninstall Before Reinstalling
  9. Summary
  10. Further Reading


Special Thanks

Special thanks to Mesmer Animation Labs in Seattle for making this tutorial possible.


1. Introduction

You will learn the least you need to know to author a custom shader for Mental Ray 3.1 and have it function within Softimage XSI 3.0. This should work with older versions also.

It's helpful being familiar with rendering either as an artist using a 3D modeling package or as a programmer with a basic working knowledge of ray tracing.

Hopefully you understand programming principles such as what an object or data structure is. You should also know what a Property Page within Softimage XSI is. It's helpful to have used some of the standard shaders bundled with XSI and Mental Ray.

(But if you're a software developer unfamiliar with XSI, I'll supply the least you need to know about XSI and its interface to see that this thing really works.)

2. Overview

Let's create the simplest of shaders for the Mental Ray rendering engine: the constant material. There's no distinction between ambient or diffuse colors. The entire geometry object appears as a single, solid color blob without definition beyond its silhouette.

An associated Property Page within Softimage XSI enables the artist to adjust only one parameter: color value.

This is intended to be the simplest yet complete case.

From an engineer's perspective, it does almost nothing but does it correctly.

From the artist's perspective, however, once this custom shader has been installed, there should be no difference between how this option is used versus any standard feature built into the software as supplied by the vendor.

3. Custom Shader Basics

Shaders tap into the rendering pipeline via data structures:

  1. one structure for receiving parameters supplied by the artist such as color selection,
  2. one structure for receiving vectors describing a ray of light and its associated color value and
  3. one structure for resulting color values that may be passed to another shader in the pipeline.

Shaders do their work by examining yet another data structure for the associated geometry object. So if the object selected is a polygon mesh torus, you have access to its internals as if Mental Ray was just a strange database and XSI was its data entry front-end. (This fourth structure is accessible via library calls and referencing the structure containing input parameters.)

Parameters from the XSI Property Page are passed through a shader-specific data structure. By convention, this gets a name matching your shader.

Since our shader is called "constant_simple", the data structure is called "constant_simple_params".

Mental Ray has its own data types for just about everything. Color values are contained within a type, "miColor". Other values are held within "miScalar".

When extracting values from the primary data structures, it's important to use macros defined in "shader.h". (Find this include file within the XSI SDK; see makefile.)

Apart from C/C++ code and resulting shared libraries, XSI needs to be informed about the add-on. There is an interface file that looks like a Unix shell script. Mental Ray has its ".mi" file (meaning Mental Images, the vendor of Mental Ray), and Softimage has its own file format. SPDL is Softimage Plug-in Definition Language, and the associated file extension is, oddly enough, ".spdl".

4. Source Code

The coding style builds upon examples provided by Softimage. (Specifically, see the "show_edges" example provided with the XSI SDK documentation.)

See the C++ source and header files for the "constant_simple" example. Both files are simple enough that it's easier for a programmer to read the code than to explain it here.

Perhaps more important to a tutorial is the SPDL file.

The Shader Wizard supplied by Softimage creates templates of a C++ file and the various associated files like the SPDL. The templates should compile under both Unix and Windows.

Optionally, you can take the files provided with this tutorial and expand upon them. Keep this in mind! There's nothing special about what the Wizard creates. It merely spares you from having to perform search and replace. Converting from existing files is handy especially if using Mozilla. (For Unix folks, the Shader Wizard seems to only like the Java VM plug-in that ships with RedHat and Netscape's branded browser.)

The tutorial files were created without directly using the Wizard, including the SPDL file.

Softimage Plug-in Definition Language (SPDL)

Within the SPDL file, there is a GUID parameter with a value of "1AC592BF-DEA4-11d5-92BD-009027219F81". There are other parameters with similar looking values.

These values are not random and must be preserved!

Each Parameter that needs to appear within the XSI Property Page (GUI) should have the same key as above. Likewise, all outputs have their own.

(Why this level of detail has been imposed on the programmer rather than being optional reflects that Softimage was momentarily owned by Microsoft while XSI was being developed.)

The PropertySet defines the inputs and outputs familiar to anyone who has used the XSI render nodes. Most material shaders would have "diffuse" and "ambient" inputs and a single "out" or output.

The first one from the PropertySet belongs to the output parameter. With most shaders, the type will be a color value.

The second parameter is "color". I could have used another name such as "insideColorValue" but instead kept things simple. This also demonstrates that there are separate name spaces within the render mechanism.

The "title" associated with each input or output parameter provides the ToolTip pop-up text when using the render view (network diagram of shader nodes). And there's another one mentioned below regarding a ToolTip in the PropertyPage (GUI).

The values for our particular input are a white-space delimited list of numerical Floats. Legal values for colors are between 0.0 and 1.0, inclusively.

The following means: Use Red at 75%, Green at 12%, Blue at 80% and set the Alpha channel mask to opaque.

value = 0.75 0.12 0.80 1.0;

Even if the XSI artist has his or her workstation configured to use Hue/Saturation/Value (HSV) instead of RGB, the application deals with the conversion. Programmers must use RGB.

Other member values within the input and output parameters should be straight-forward to anyone familiar with using XSI. (Shader programmers should have a day-long overview of XSI at minimum before changing these additional values.)

Further down the SPDL file, the "Defaults" section seems to repeat some of the parameters. The "description" associated with each input or output parameter provides the ToolTip pop-up text when using the XSI Property Page (GUI). Although this will typically be identical to the previous one for the render node, sometimes it will be appropriate to have different wording from a usability stand-point.

The filename is specified in the final section. For everyone's sanity, just keep the names consistent through all files and structures.

5. Building From Source

You must have an installed and configured version of Softimage XSI with the SDK. While the demo version should be able to use the add-on, you need the full version for compiling.

Unix: Linux and Irix

% tar zxf constant_simple.tgz
% cd constant_simple

There are separate versions of the "makefile" for Linux versus Irix. (On Linux, the test build used Mandrake 9.0 so RedHat should work fine; for Slackware, you'll require tweaking.)

Compile the source and install it. A make script is supplied to ensure the proper environment variables are defined.

% ./make.sh -f makefile.linux all install

Or

% ./make.sh -f makefile.irix all install

During installation, XSI will run in batch mode without any associated windows or panels. (Depending upon the software license available to you, it may use the actual Batch mode or possibly Enterprise mode.) This may take a few minutes to load-- be patient.

Assuming it installed without errors, start XSI normally.

The add-on should appear in the "User" path from the Render->Texture->More... pop-up. (More about use and testing below.)

Windows

For the non-Unix crowd, there is a .dsp file in the tar file. It was adapted from a custom shader example written by Softimage. Good luck.

Windows With Cygwin

If you have XSI with the SDK on Windows and have installed Cygwin, follow these commands.

NOTE: Understand that the final step is untested. I have been unable to find a Windows XSI installation that has the SDK, so final compiling and linking known to work under Unix can only be theorized to work here. Most of the legwork, however, has been done for you. You should only need to copy and paste the commands below, building with the scripts provided.

The only compiler tested was GNU g++/gcc 3.2 via Cygwin.

See http://sources.redhat.com/cygwin/

If you have the Cygwin utilities (Bash, Perl, gnu compilers, etc), try the following.

First, copy the shader source files to the cygwin directory:

% cd cygwin
% ln -s ../constant_simple* .

Convert the Windows command/batch file, Setenv.bat, to a CSH script file. "C:/Softimage/XSI_*/Application/bin/Setenv.bat" becomes "Setenv.csh".

Fortunately, much of the syntax used here converts directly and scripts are provided. Use either bat2sh.pl or bat2csh.pl, depending upon whether you use Bash/ash/sh or tcsh/csh, respectively.

From the directory including the makefile, run:

% ENVBAT=/cygdrive/c/Softimage/XSI_3.0/Application/bin/
% ./bat2sh.pl $ENVBAT/Setenv.bat > XSI_Setenv.sh

(Depending upon your workflow, you might also want to convert your workgroup environment variables, ~/Softimage/XSI_*/Addons/workgroup_shader_plugins.bat, by saving an equivalent .csh file in that same directory-- not our source directory.)

Perform the compilation. Install and test as you would with the Unix versions.

% ./make.sh all install

The only change is that there is an XSI.sh file in our "cygwin" directory for Bash/sh and another for tcsh/csh. To launch XSI from the command-line, be sure to use one of these scripts.

You should, however, be able to just launch XSI from the Windows Start menu or desktop icon as normal.

6. Using The Custom Shader

Assuming the custom shader was compiled and installed without errors from the previous steps, start XSI as you normally would.

Create an object such as a polygon mesh torus or cube, then apply the magic texture. (Press F1 to bring up the Model menu on the left side, then select: Get->Primitive->Polygon Mesh->Torus)

From the Render menu (F3), select Get->Texture->More.... A pop-up window appears. Go to the "Paths" pull-down in the upper left corner of the pop-up. Change the path to "User".

From the list of files, you should see "constant_simple". Select this, then click "OK".

If you are prompted for whether to create a local material, click Yes.

7. Verifying The Shader Really Works

The default way in which XSI loads a custom texture is to chain it through the scene default material such as Phong.

Because of this, be skeptical as to whether the custom add-on is doing the work or if built-in functionality is providing the effect.

Bring up the Render view. (Assuming you have the usual four view panels with your object appearing from a different angle in each-- move the mouse to one of the bottom panels and press 7. Or from that viewport's title bar pull-down menu, change to "Render Tree".)

You should see the network diagram with "constant_simple" on the left, "Scene Material" on the right and a few nodes in between.

Remap the connections from our shader to the Scene Material, bypassing everything in between.

Connect the "constant_simple" node output to three of the "scene material" inputs: "Surface", "Shadow" and "Photon". (Drag from the red dot in the upper right corner of "constant_simple" node to inside "scene material". From the pop-up, select "shadow". Repeat for the other two parameters. These parameters and only these should have a dot next to their names in the pop-up, making it easier to find them.)

Some of the the original connections are still there, so next, we delete them.

Leave only the "constant_simple" and "scene material" nodes and of course the connections we just established. (Hold the space bar while selecting with the mouse: draw a box around or through the unwanted nodes. Make sure only unwanted items become white, then press Delete.)

Let's see it work.

Create a render region around the object in one of the viewports if you haven't done so already. (Move the mouse to a panel with the wireframe object. While holding the "q" key, diagonally drag a box around the object. Release the q key.)

Fiddle with the color value for the custom shader. (While holding the space bar, click on the "constant_simple" node in the Render window. Release the space bar. Press Alt+Enter and the Property Page should appear if it wasn't up already. Using the color slider should be obvious, but to change from HSV to RGB or vice-versa, click on the button just to the left of the color slider.)

Notice that the parameter is called "color" and is identical to the parameter named in the SPDL and C++ files.

Move the color slider in the Property Page, and watch the color change in the render region!

8. Uninstall Before Reinstalling

Before re-installing a custom shader, you must first uninstall it.

XSI kindly reminds you of this, should you happen to forget.

% ./make.sh -f makefile.linux uninstall

Or

% ./make.sh -f makefile.irix uninstall

Or for Cygwin

% cd cygwin
% ./make.sh uninstall

It's recommended that you use XSI via batch mode to remove the add-on rather than just walking the directory tree and deleting files.

9. Summary

You have a complete set of C++ and the supporting SPDL files to create a simple shader. It does almost nothing but does it properly.

Regarding the possibilities, this example barely makes a dent.

If what you observed here appeals to you at all, jump into shader development.

10. Further Reading

This example shader started life as a baby Rubik's cube example supplied by Mental Images in their manual on the Softimage web site.

http://www.softimage.com/support/xsi/private/userguides/3.0/files/mental_ray/manual/ray-3_29Example_Scene_with_Cust.html

(TIP: If any softimage.com links require a password, replace "3.0" in the URL with "2.0.2" or "2.0" and supply "index.html" or "ray-Content.html" filename instead of directly browsing a directory.)

Or look for "Example Scene" under "Using and Writing Shaders" here: http://www.softimage.com/support/xsi/private/userguides/2.0/files/mental_ray/manual/ray-Contents.html

A little reverse-engineering and simplifying the code created our example.

Want to go further, making your own custom shader extension for XSI and Mental Ray?

First understand the basics:

http://www.softimage.com/support/xsi/private/userguides/2.0.2/files/docs/html/shade/shaderbasics.html

Play with some custom shaders and presets from other people's development efforts:

http://www.highend3d.com/xsi/shaders/

http://www.softimage.com/xsinet/Addon_display_abstract.asp?family=XNET&software=XSI&catid=612&parent=51&child=Shaders

Browse through the Mental Ray manual:

http://www.softimage.com/support/xsi/private/userguides/2.0.2/files/mental_ray/manual/ray-Contents.html

Look at examples.

There's one bundled with the XSI SDK. The shader reveals all triangles on an object's surface. It provides minimal parameters from within an XSI Property Page:

http://www.softimage.com/support/xsi/private/userguides/2.0.2/files/docs/html/sdk/casestudies/cs_showedge.html

Finally, get THE BOOK:

http://www.springer.at/main/book.jsp?bookID=3-211-83851-1&categoryID=3

ISBN 3-211-83851-1

http://www.Powells.com/search/DTSearch/search?author=Driemeyer

Special Thanks

Special thanks to Mesmer Animation Labs in Seattle for making this tutorial possible.


Copyright 2003 Daniel Joseph Pezely