Persistent Editing of mayaUsdProxyShapes?

[pierre-ed]

Hello everyone!

I am having a hard time providing users with a comfortable workflow with the proxyShapes implemented by the 'adsk' plugin (mayaUsdProxyShape nodes).

In a nutshell, I would like users to be able to

• easily populate their Maya scenes with several proxyShapes,
• individually edit any of them (variant selection, xforms)
• and retrieve their work after saving and reopening the scene.

Alas, contrary to the proxyShapes from the 'al' plugin (AL_usdmaya_ProxyShape nodes), I fail to attach a separate session layer as the edit target of each individual proxyShape's stage. Because of this, Maya does not differentiate between sets of edits that would target different proxyShape instances sharing a common root layer. Moreover, all edits go to waste when I reopen the scene after saving.

Is there a way of setting a separate, serializable session layer to any mayaUsdProxyShape instance?
Could the mayaUsdLayerEditor and mayaUsdEditTarget commands be of any help for this task?

Please note that I am running maya_usd-0.5.0 in Maya-2020.1 on a Linux workstation, I thus cannot rely on the neat features based on the UFE-2 framework.

Thanks for your help.
Cheers!
-- Pierre

[msercheli]

Hi Pierre,

I am having exactly the same issue. I was about to write a question here but then I saw your question. So I am just adding my experience here as well.

I also need to be able to author USD opinions in sessionLayers for different proxyShapes that are reading from the same rootLayer.
Like Pierre mentioned, the edits made to a prim within a proxyShape is not persistent after re-opening the Maya scene.
Also, using the mayaUsdLayerEditorWindow, after changing the editTarget to the sessionLayer, this change is also not persistent after re-opening the Maya scene.

Versions for me:
maya-usd 0.6.0
Maya 2020.4
OS: Windows 10

My employer is working towards getting access to the beta program of Autodesk so we can evaluate the latest mayaUsd with UFE 2.0.
But for now, we are still limited to UFE 1.0

Any tips or workaround is much appreaciate.
Thanks,
Marcelo Sercheli

[kxl-adsk]

Apologies for coming back to you late. We had a few extra days added to our holiday break.

Assuming I read your questions correctly, I will break my answer into two parts. First, let's talk about the session layer. We have made a change to target by default root layer, but you can:

1. Switch to session layer with layer editor (it's now available in the repository, starting with 0.6.0, and available for Maya 2020) or python. With layer editor:
   • Open Windows->General Editors->USD Layer Editor
   • Make sure to change Option to show the session layer
   • Press on the circle next to the session layer to make it a target
2. Persistency of changes you made. There are different combinations here since you might have edited the session layer or any other layer in the layer stack. Layers can be in-memory or file-backed. I believe you are only interested in the editing session layer, but how do you want these changes to be consumed by the rest of the pipeline?

EDIT to point 1: We also have an optionVar available to switch to legacy behavior when creating new proxy shapes. By using the mayaUsd_ProxyTargetsSessionLayerOnOpen option var, you can enable the session layer to be used as a target by default.

[msercheli]

Thanks Krystian for the answer.

I am aware that I can change to session layer using the Layer Editor. I was not aware of the option var available. Thanks for that.
I understand that, if the user would like to save the edits to disk, we would export the layer (session or root) to disk.

My question in relation to that is concerning saving the Maya file and re-opening it. If artists are working in the Maya scene, want to save it (the Maya scene) and re-open it next day, before performing any export to USD. By re-opening the Maya scene, the edits made were lost.

I know that the Pixar plugin, because it uses the Maya assembly node, Maya stores the edits, as part of the assembly node implementation. I also know that the Animal Logic plugin serializes the USD stage (root and session-layer) to the Maya scene before saving, and unserialize it upon opening the scene.

During my tests, using the MayaUsd plugin (from Autodesk), edits made to USD session-layers were not being restored upon opening the Maya scene (without exporting USD data to disk).

I was wondering if this limitation is because I am using Maya 2020.4 and therefore, not UFE 2.0?
Thanks.

[kxl-adsk]

UFE 2.0 adds more native USD editing capabilities, but it has nothing to do with persistency of in-memory edits.

I see you have a good overview of what different plugins had to offer. We explored taking the Animal Logic plugin approach and this work is available in draft pull-request #871. We found that this approach may cause trouble when working with bigger layers, causing maya file size to increase beyond expectation and file IO performance to be very poor.

This is why I'm curious about your workflow with session layers and which approach for serialization would work for you?

[msercheli]

What a coincidence. I saw that PR a few minutes ago.
While evaluating the AL plugin back in 2016, I was wondering the same thing (if the edits are too large, this would be an IO issue).
I never used the AL plugin in production. Instead, I used the PixarAssembly node, which was sufficient for the needs back then.

The intended workflow with session-layers is that artists would have 1 mayaUsdProxyNode that will hold all the necessary prims, references, payloads, etc for a particular set and/or shot. Set-dressing and Layout artists would then start transforming Xform prims around and selecting different variants. All these opinions would be authored within the session-layer of the USD stage (been used by the mayaUsdProxy node).

Once the artist feels comfortable with their work (this could take a few days), they would export the session-layer to disk, which would hold all the "over" prims with all the authored opinions.
While working in that Maya scene, the artists may need to save and re-open the same scene (for various reasons).

Although the root-layer may be huge (specially for shots), the session-layers may not be necessarily large. A Layout artist will dress the sets for a particular shot by transforming several prims and changing variants but those opinions would not be too many. Maybe a few dozens (and a few hundreds in rare-cases).

Any mesh caching (geometry points) would be cached in their own layers and payloaded inside the root-layer. Geometry caches would come from Maya referenced rigs and therefore, not participating in the session-layer edits (at least not in the beginning). This is done so we can version up those caches independently from the other opinions.
But I understand that other users/studios may have a different approach and design, which could lead to larger session-layers.

One option would be exporting the session-layer to disk and store the file path somewhere so upon opening the Maya scene, I can provide that as session-layer while opening the USD stage. Having a side-car file that, could (by accident) be lost, is not ideal. Also, side-car file or not, if the session-layer is large enough, the IO issue would still be there.

Honestly, I don't have an answer/suggestion for that. But I can imagine that, for my use-case, the session-layers would not be large enough that would represent an IO issue. So, serializing inside the Maya scene could work fine.

[wtelford]

I see three main points being brought up in this thread

1. Managing workflows to target edits to session layers.
2. Questions on the correct methodology for allowing session layer edits to persist between Maya sessions.
3. The need for a unique session layer for each proxyShape, even when they share a root layer

I feel that Krystian did a good job above addressing the first point, but let us know if we can provide any more clarification on that.

For point number 2, we are currently working on a method to serialize session layer edits to the Maya file that will be restored as session layer edits when the file is re-opened. Our current prototype of this does have a 2 gigabyte limit on the size of the layer edits that we serialize. We still need to determine how to gracefully fail this state when the 2 gigabyte limit is exceeded.

Looking into point number 3, we feel that each proxyShape having a unique session layer, even when the root layer is shared, is a good idea. I believe the AL plugin supported this behavior, though I have yet to confirm that through testing.
I can confirm that you are correct that the ADSK plugin does not yet support this. I will log this as a feature request, as it makes sense.
In the meantime one workaround would be to give each proxyShape a unique root and then sublayer in the common layer.

• This would mean that stages are not shared in memory between proxyShapes that have the same root
• Each proxyShape would also have a unique session layer as the stages are not shared
• However, due to the way USD works, common layers WOULD be shared in memory. This means to stages with the same root layer would also share edits directly targeting that root layer.

One additional topic I see touched upon here is the desire to leverage a multi-proxyShape workflow. While we do support this, I would like to better understand your intended use cases. With the native editing capabilities that UFE 2.0 adds,
it should be possible to manage USD data with a single proxyShape. Prior to UFE there were more compelling reasons to break data up between multiple proxyShapes.

[msercheli]

Will, thanks so much for your reply.

I am glad to hear that topic #2 is being addressed as a serialization approach. I understand the edits size limit. I would only suggest that it would serialize edits made to root layers as well so those are also persistent between Maya sessions.

As for topic #3, you are completely right when you suggest that, in Maya 2021 and UFE 2.0, the need for a multi-proxyShape workflow will diminish or be gone for most of the use-cases. Since the studio I am working for is using Maya 2020.4, we still don't have access to UFE 2.0, and that is where my previous comments came from. So, if possible, to allow unique session layers per proxyShape (as needed) would be great and helpful for anyone that is not yet in UFE 2.0.

I believe (not confirmed) that the AL plugin does that only if an edit happens. So, it shares the same stage cache (for performance sake) until the user "request" an unique edit.

I also would like to ask, if possible, to expose as much as possible the features and capabilities in Python. I am a software developer and so much of the artist's workflows need to be automated through tooling provided by myself and my colleagues. But that is another discussion.

Once again, thank you all for the answers and open discussion as well as for the amazing job.

Ps: Quick question: the serialization development is merging the PR #871? Or is it an internal development? I am asking because we are planning to build from source and, if so, we can go ahead and merge in our local repo that PR. Otherwise, we could wait for ADSK code if it would be released "soon".

[kxl-adsk]

We will continue work on #871, so definitely give it a try and report problems you find.

[williamkrick]

Hi @msercheli,

The work in #1142 provides some help with managing proxy shapes that share a root layer. With this PR when you manually create a proxy shape which shares a root layer with an existing proxy shape then it will share a session layer. If instead you duplicate an existing proxy shape then the duplicated proxy will share a root layer but have a unique session layer.

Testing in Maya 2022 it is working correctly and I don't see a reason for it to behave differently in 2020.4.

[wtelford]

@msercheli,
With regard to your request to also serialize the root layer. We are currently exploring the workflows for serialization of non-session layers. This includes the root layer.
This also includes consideration of :

• Anonymous layers with in-memory edits
• File-backed layers with in-memory edits.

We are currently investigating presenting a choice of three options when saving the Maya file (optionally saved as a persistent setting).
Regardless of choice, all three options would serialize the session layer to the Maya file.

Option 1:

• Only save the Maya file, USD edits would not be preserved between Maya sessions.

Option 2:

• Save the Maya file
• Save all file-backed layers with in-memory edits back to their original USD files
• Batch convert all anonymous layers to file-backed layers on disk.

Option 3:

• Serialize all in-memory edits of file-backed or anonymous layers in to the Maya file

Upon re-opening your Maya file, all layers that were serialized in to it would be restored to their respective layers as in-memory USD edits in your Maya session.

Option 3 is the most risky as non-session layers are much more likely to exceed the 2 gig limit, bloat your Maya file, and slow down save and open operations.

[msercheli]

Thanks Will.

I agree that option 3 is risk for non-session layers.
I am exciting to see how it will be resolved.

We merge the PR #871 and I will let you know of any issues that I may find.
Thanks once again.

Marcelo

[msercheli]

Will,

A developer on our studio is about to start merging the PR #871 into the master branch.
I was wondering, before he starts this work, if you and your team have an estimate on when the ASDK internal development for the persistent session layers will be available publicly? Even if it's another branch.

Thanks,
Marcelo