Custom Deadline submission code for caching

[Martijn van Herk]

I'm looking for ways to change how cache jobs are submitted to Deadline. For example attaching environment variables and extra info to the job, or overriding the plugin ini. What's the best way to do this?

Thanks!

[tyFlow]

Currently the deadline submissions button just uses a very basic submitter, based on the Max submission script originally provided by Thinkbox (the one that comes with Deadline)....doesn't offer any customization beyond the options for priority and suspension....

Can you describe your use case a bit more?

[Martijn van Herk]

Currently the deadline submissions button just uses a very basic submitter, based on the Max submission script originally provided by Thinkbox (the one that comes with Deadline)....doesn't offer any customization beyond the options for priority and suspension....

Can you describe your use case a bit more?

Thanks for the quick response!

We add environment variables and extra info key/value pairs to jobs that workers use during max startup to set/adjust certain parameters. This can be shot/asset information, which code repository to pull code from, etc. etc.

I can inspect a job submitted by tyflow to see what submission parameters I need to set and write my own submitter based on that, but that won't prevent anyone from clicking the submit button in the Export Particles node and having their jobs fail, so I was hoping that I could somehow modify that submission code directly.

[tyFlow]

Ok, well basically what happens when you press the Deadline button is I get the deadline repository root and load in

Code:

submitMaxToDeadline_Functions.ms

Then call

Code:

SMTDFunctions.loadSettings()

which fills a global variable called

Code:

SMTDSettings

which I modify with my job name, priority, chunksize, etc. And then I call the rest of the functions necessary to submit the job with those settings.

Since SMTDSettings is a global variable and it contains the struct SMTDSettingsStructure which is fully exposed in "submitMaxToDeadline_Functions.ms", it's conceivable that I could add an optional MAXScript section of the Deadline submission rollout, where you can add your own custom settings that would be applied after tyFlow fills the struct with it's own stuff. So you'd enter something like:

Code:

SMTDSettings.DraftSubmitJob = true
SMTDSettings.ExtraInfo0 = "test"
SMTDSettings.WarnAboutMissingExternalFiles = false
--etc

And then I'd just directly execute that script prior to submission, thereby applying all of your settings as the job is sent along.

Would that work?

[Martijn van Herk]

Yeah that should work! In its simplest form it could be just a callback maybe? So after TyFlow has done whatever it needs to do to submit the job, it would call a "preSubmit" mxs function (potentially with a few arguments) that allows for custom code to be executed before the job is actually submitted.

[tyFlow]

Sure, that could also work.