Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft : Implementing structures. #1684

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 34 additions & 5 deletions documents/Specification/MaterialX.Specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ While color<em>N</em> and vector<em>N</em> types both describe vectors of floati

## Custom Data Types

In addition to the standard data types, MaterialX supports the specification of custom data types for the inputs and outputs of shaders and custom nodes. This allows documents to describe data streams of any complex type an application may require; examples might include spectral color samples or compound geometric data.
In addition to the standard data types, MaterialX supports the specification of custom data types for the inputs and outputs of shaders and custom nodes. This allows documents to describe data streams of any complex type an application may require; examples might include spectral color samples or compound geometric data. The structure of a custom type's contents is described using a number of &lt;member> elements.

Types can be declared to have a specific semantic, which can be used to determine how values of that type should be interpreted, and how nodes outputting that type can be connected. Currently, MaterialX defines three semantics:

Expand All @@ -268,8 +268,12 @@ Types not defined with a specific semantic are assumed to have semantic="default
Custom types are defined using the &lt;typedef> element:

```xml
<typedef name="spectrum" semantic="color"/>
<typedef name="manifold"/>
<typedef name="manifold">
<member name="P" type="vector3"/>
<member name="N" type="vector3"/>
<member name="du" type="vector3"/>
<member name="dv" type="vector3"/>
</typedef>
```

Attributes for &lt;typedef> elements:
Expand All @@ -282,10 +286,35 @@ Attributes for &lt;typedef> elements:
* "halfprecision": the values within this type are half-precision
* "doubleprecision: the values within this type are double-precision

Once a custom type is defined by a &lt;typedef>, it can then be used in any MaterialX element that allows "any MaterialX type"; the list of MaterialX types is effectively expanded to include the new custom type. It should be noted however that the &lt;typedef> is only declaring the existence of the type and perhaps some hints about its intended definition, but it is up to each application and code generator to provide its own precise definition for any type.
Attributes for &lt;member> elements:

The standard MaterialX distribution includes definitions for four "shader"-semantic data types: **surfaceshader**, **displacementshader**, **volumeshader**, and **lightshader**. These types are discussed in more detail in the [Shader Nodes](#shader-nodes) section below.
* `name` (string, required): the name of the member variable.
* `type` (string, required): the type of the member variable; can be any built-in MaterialX type; using custom types for &lt;member> types is not supported.
* `defaultvalue` (string, optional): the value the member variable will be initialized to if not explicitly set on a node.

**(Discussion point : We might be able to support custom types as members as well? Given the initializers are now wrapped in braces, its possible to accurately define nested initializers, and potentially also allow for some members to not be initialized)**

If a number of &lt;member> elements are provided, then a MaterialX file can specify a value for that type any place it is used, using an aggregated initializer, a semicolon-separated list of numbers and strings surrounded by braced. The expectation is that the numbers and strings between semicolons exactly line up with the expected &lt;member> types in order. For example, if the following &lt;typedef> was declared:

```xml
<typedef name="exampletype">
<member name="id" type="integer"/>
<member name="compclr" type="color3"/>
<member name="objects" type="stringarray"/>
<member name="minvec" type="vector2"/>
<member name="maxvec" type="vector2"/>
</typedef>
```

Then a permissible input declaration in a custom node using that type could be:

```xml
<input name="in2" type="exampletype" value="{3; 0.18,0.2,0.11; foo,bar; 0.0,1.0; 3.4,5.1}"/>
```

Once a custom type is defined by a &lt;typedef>, it can then be used in any MaterialX element that allows "any MaterialX type"; the list of MaterialX types is effectively expanded to include the new custom type.

The standard MaterialX distribution includes definitions for four "shader"-semantic data types: **surfaceshader**, **displacementshader**, **volumeshader**, and **lightshader**. These types are discussed in more detail in the [Shader Nodes](#shader-nodes) section below.


## MTLX File Format Definition
Expand Down
20 changes: 20 additions & 0 deletions libraries/experimental/experimental_defs.mtlx
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<?xml version="1.0"?>
<materialx version="1.38">
<!--
Copyright Contributors to the MaterialX Project
SPDX-License-Identifier: Apache-2.0

Declarations of experimental data types and nodes
-->

<typedef name="texcoord">
<member name="ss" type="float"/>
<member name="tt" type="float"/>
</typedef>

<nodedef name="ND_extract_s_texcoord" node="extracts" nodegroup="shader" >
<input name="in" type="struct:texcoord" value="0.0;0.0" />
<output name="out" type="float" value="0.0" />
</nodedef>

</materialx>
10 changes: 10 additions & 0 deletions libraries/experimental/experimental_ng.mtlx
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<?xml version="1.0"?>
<materialx version="1.38">
<!--
Copyright Contributors to the MaterialX Project
SPDX-License-Identifier: Apache-2.0

Nodegraph Implementations of experimental nodes
-->

</materialx>
12 changes: 12 additions & 0 deletions libraries/experimental/genglsl/experimental_genglsl_impl.mtlx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<?xml version="1.0"?>
<materialx version="1.38">
<!--
Copyright Contributors to the MaterialX Project
SPDX-License-Identifier: Apache-2.0

GLSL Implementations of experimental nodes
-->

<implementation name="IM_extract_s_texcoord" nodedef="ND_extract_s_texcoord" target="genglsl" sourcecode="{{in__I__ss}}" />

</materialx>
12 changes: 12 additions & 0 deletions libraries/experimental/genmsl/experimental_genmsl_impl.mtlx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<?xml version="1.0"?>
<materialx version="1.38">
<!--
Copyright Contributors to the MaterialX Project
SPDX-License-Identifier: Apache-2.0

MSL Implementations of experimental nodes
-->

<implementation name="IM_extract_s_texcoord" nodedef="ND_extract_s_texcoord" target="genmsl" sourcecode="{{in__I__ss}}" />

</materialx>
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<?xml version="1.0"?>
<materialx version="1.38">
<nodegraph name="test_struct_texcoord">

<extracts name="extracts" type="float">
<input name="in" type="struct:texcoord" value="0.3;0.2"/>
</extracts>

<convert name="float_to_color3" type="color3">
<input name="in" type="float" nodename="extracts"/>
</convert>
<oren_nayar_diffuse_bsdf name="diffuse_brdf1" type="BSDF">
<input name="color" type="color3" nodename="float_to_color3" />
</oren_nayar_diffuse_bsdf>
<surface name="surface1" type="surfaceshader">
<input name="bsdf" type="BSDF" nodename="diffuse_brdf1" />
<input name="opacity" type="float" value="1.0" />
</surface>
<output name="out" type="surfaceshader" nodename="surface1" />
</nodegraph>
</materialx>
5 changes: 5 additions & 0 deletions source/MaterialXGenShader/ShaderGraph.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -831,6 +831,11 @@ ShaderNode* ShaderGraph::createNode(const Node& node, GenContext& context)
ShaderInput* input = newNode->getInput(nodeDefInput->getName());
InputPtr nodeInput = node.getInput(nodeDefInput->getName());

// 'input' is null_ptr if the input is a struct - a fully formed struct implementation likely needs to handle some of these
// additional cases more gracefully and completely.
if (!input)
continue;

const string& connection = nodeInput ? nodeInput->getNodeName() : EMPTY_STRING;
if (connection.empty() && !input->getConnection())
{
Expand Down
139 changes: 116 additions & 23 deletions source/MaterialXGenShader/ShaderNode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ ShaderNodePtr ShaderNode::create(const ShaderGraph* parent, const string& name,
// Create interface from nodedef
for (const ValueElementPtr& port : nodeDef.getActiveValueElements())
{
const TypeDesc* portType = TypeDesc::get(port->getType());
const std::string& portTypeStr = port->getType();
const TypeDesc* portType = TypeDesc::get(portTypeStr);
if (port->isA<Output>())
{
newNode->addOutput(port->getName(), portType);
Expand All @@ -197,10 +198,53 @@ ShaderNodePtr ShaderNode::create(const ShaderGraph* parent, const string& name,
}
else
{
input = newNode->addInput(port->getName(), portType);
if (!portValue.empty())
if (portTypeStr.substr(0,7) == "struct:") {
// if this port is a struct - then instead of setting the value directly, we're just going to set
// the value of each member in the struct

// TODO this is hacky - need a better way to get to the document to find the typedef
auto docPtr = nodeDef.getParent()->asA<Document>();

std::string structName = portTypeStr.substr(7);
auto typeDef = docPtr->getTypeDef(structName);

StringVec subMemberValues = {};
if (!portValue.empty()) {
// slice up the port value for the struct in to individual pieces that can be applied sequentially to each member of the struct
subMemberValues = splitString(port->getResolvedValueString(), ";");
}

unsigned int memberIndex = 0;
for (const auto& member : typeDef->getMembers())
{
const std::string& subPortTypeStr = member->getType();
const TypeDesc* subPortType = TypeDesc::get(subPortTypeStr);

// create the struct member name by concatenating the original port name with a "unique" separator, and the struct member name
// here i'm just choosing "__I__" as a token that is unlikely to used in a parameter name, but we would want something more robust
// and validatable later.
auto subPortName = port->getName() + "__I__" + member->getName();

input = newNode->addInput(subPortName, subPortType);

auto valPtr = ValuePtr();
if (memberIndex < subMemberValues.size()) {
valPtr = Value::createValueFromStrings(subMemberValues[memberIndex], subPortTypeStr);
}
// unclear if we should set an "empty" ValuePtr() here, or if can guard the setValue() call inside the 'if' above
input->setValue(valPtr);

memberIndex++;
}

}
else
{
input->setValue(port->getResolvedValue());
input = newNode->addInput(port->getName(), portType);
if (!portValue.empty())
{
input->setValue(port->getResolvedValue());
}
}
}
if (port->getIsUniform())
Expand Down Expand Up @@ -338,33 +382,82 @@ void ShaderNode::initialize(const Node& node, const NodeDef& nodeDef, GenContext
// Copy input values from the given node
for (InputPtr nodeInput : node.getActiveInputs())
{
ShaderInput* input = getInput(nodeInput->getName());
ValueElementPtr nodeDefInput = nodeDef.getActiveValueElement(nodeInput->getName());
if (input && nodeDefInput)
if (nodeDefInput)
{
ValuePtr portValue = nodeInput->getResolvedValue();
if (!portValue)
std::string portTypeStr = nodeDefInput->getType();
if (portTypeStr.substr(0, 7) == "struct:")
{
InputPtr interfaceInput = nodeInput->getInterfaceInput();
if (interfaceInput)
std::string portValueStr = nodeInput->getResolvedValueString();
if (portValueStr.empty())
{
portValue = interfaceInput->getValue();
InputPtr interfaceInput = nodeInput->getInterfaceInput();
if (interfaceInput)
{
portValueStr = interfaceInput->getValueString();
}
}

// TODO this is hacky.
auto docPtr = nodeDef.getParent()->asA<Document>();

std::string structName = portTypeStr.substr(7);
auto typeDef = docPtr->getTypeDef(structName);

StringVec subMemberValues = {};
if (!portValueStr.empty()) {
// slice up the port value for the struct in to individual pieces that can be applied sequentially to each member of the struct
subMemberValues = splitString(portValueStr, ";");
}

unsigned int memberIndex = 0;
for (const auto& member : typeDef->getMembers())
{
const std::string& subPortTypeStr = member->getType();

auto subPortName = nodeInput->getName() + "__I__" + member->getName();
ShaderInput* input = getInput(subPortName);
if (input)
{
if (memberIndex < subMemberValues.size())
{
auto valPtr = Value::createValueFromStrings(subMemberValues[memberIndex], subPortTypeStr);
input->setValue(valPtr);
}
}
memberIndex++;
}
}
const string& valueString = portValue ? portValue->getValueString() : EMPTY_STRING;
std::pair<const TypeDesc*, ValuePtr> enumResult;
const string& enumNames = nodeDefInput->getAttribute(ValueElement::ENUM_ATTRIBUTE);
const TypeDesc* type = TypeDesc::get(nodeDefInput->getType());
if (context.getShaderGenerator().getSyntax().remapEnumeration(valueString, type, enumNames, enumResult))
{
input->setValue(enumResult.second);
}
else if (!valueString.empty())
else
{
input->setValue(portValue);
}
ShaderInput* input = getInput(nodeInput->getName());
if (input)
{
ValuePtr portValue = nodeInput->getResolvedValue();
if (!portValue)
{
InputPtr interfaceInput = nodeInput->getInterfaceInput();
if (interfaceInput)
{
portValue = interfaceInput->getValue();
}
}
const string& valueString = portValue ? portValue->getValueString() : EMPTY_STRING;
std::pair<const TypeDesc*, ValuePtr> enumResult;
const string& enumNames = nodeDefInput->getAttribute(ValueElement::ENUM_ATTRIBUTE);
const TypeDesc* type = TypeDesc::get(nodeDefInput->getType());
if (context.getShaderGenerator().getSyntax().remapEnumeration(valueString, type, enumNames, enumResult))
{
input->setValue(enumResult.second);
}
else if (!valueString.empty())
{
input->setValue(portValue);
}

input->setChannels(nodeInput->getChannels());
input->setChannels(nodeInput->getChannels());
}
}
}
}

Expand Down
7 changes: 6 additions & 1 deletion source/MaterialXGenShader/TypeDesc.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,11 @@ int TypeDesc::getChannelIndex(char channel) const
const TypeDesc* TypeDesc::get(const string& name)
{
const TypeDescMap& map = typeMap();
auto it = map.find(name);
std::string localName = name;
if (localName.substr(0, 7) == "struct:") {
localName = "struct";
}
auto it = map.find(localName);
return it != map.end() ? it->second.get() : nullptr;
}

Expand All @@ -97,6 +101,7 @@ const TypeDesc* COLOR4 = TypeDesc::registerType("color4", TypeDesc:
const TypeDesc* MATRIX33 = TypeDesc::registerType("matrix33", TypeDesc::BASETYPE_FLOAT, TypeDesc::SEMANTIC_MATRIX, 9);
const TypeDesc* MATRIX44 = TypeDesc::registerType("matrix44", TypeDesc::BASETYPE_FLOAT, TypeDesc::SEMANTIC_MATRIX, 16);
const TypeDesc* STRING = TypeDesc::registerType("string", TypeDesc::BASETYPE_STRING);
const TypeDesc* STRUCT = TypeDesc::registerType("struct", TypeDesc::BASETYPE_STRUCT);
const TypeDesc* FILENAME = TypeDesc::registerType("filename", TypeDesc::BASETYPE_STRING, TypeDesc::SEMANTIC_FILENAME);
const TypeDesc* BSDF = TypeDesc::registerType("BSDF", TypeDesc::BASETYPE_NONE, TypeDesc::SEMANTIC_CLOSURE, 1, false);
const TypeDesc* EDF = TypeDesc::registerType("EDF", TypeDesc::BASETYPE_NONE, TypeDesc::SEMANTIC_CLOSURE, 1, false);
Expand Down
1 change: 1 addition & 0 deletions source/MaterialXGenShader/TypeDesc.h
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ extern MX_GENSHADER_API const TypeDesc* COLOR4;
extern MX_GENSHADER_API const TypeDesc* MATRIX33;
extern MX_GENSHADER_API const TypeDesc* MATRIX44;
extern MX_GENSHADER_API const TypeDesc* STRING;
extern MX_GENSHADER_API const TypeDesc* STRUCT;
extern MX_GENSHADER_API const TypeDesc* FILENAME;
extern MX_GENSHADER_API const TypeDesc* BSDF;
extern MX_GENSHADER_API const TypeDesc* EDF;
Expand Down