+- tyFlow Forum (https://forum.tyflow.com)
+-- Forum: tyFlow Discussion (https://forum.tyflow.com/forum-1.html)
+--- Forum: General Discussion (https://forum.tyflow.com/forum-2.html)
+--- Thread: Tyflow too slow to simulate (/thread-1386.html)
Tyflow too slow to simulate - VickyVey - 01-16-2020
Hello there, I am new to Tyflow in 3dsMax.
I am trying to simulate a train with its first carriage pushing some snow out of the track. I am starting with 500.000 particles.
It's just impossible to scrub back and forward as it keeps loading the cache for each frame, therefore freezing my viewport.
So how is the workflow ? Do I export particles as PRT from the outset and then start experimenting?
Many thanks.
RE: Tyflow too slow to simulate - tyFlow - 01-16-2020
Sounds like something is broken in your scene. 500k particles should scrub without issue. I don't know what you mean by "loading the cache for each frame". The live cache stays in memory and nothing needs to be loaded. Maybe share your scene so we can see what's up.
RE: Tyflow too slow to simulate - VickyVey - 01-17-2020
Thanks for the reply.
Yes, you were right. After creating another scene and replicating the same simulation with more simple objects, found out the "collision operator" was the culprit in the previous scene. Even deactivated/dimmed it was still breaking the scene somehow. So I decided to ditch that scene and start all over.
It's now possible to scrub the timeline more easily.
When I said the cache is "loaded", I actually meant "updated". So what can I do to calculate all event's simulation and leave it in the memory for easy access ? In Phoenix we can press "start" button to calculate and we can manipulate the scene in the meantime. Then, after calculating, it's very easy to scrub the timeline. With Tyflow I am not sure what is going on... the more events and operators I insert, the more calculation I need, the less timeline scrubbing I can do, and more time I spend just staring at the monitor...
I thought maybe if I exported the Tycache or PRT and then importing it back somehow I could make the process easy, but my scene crashes midway while exporting 400 frames with 500.000 particles.
So what I am doing wrong ?
I have attached a screengrab below. Sometimes I disable the flow, move the time slider to the frame I want to see and enable the flow...
RE: Tyflow too slow to simulate - shadowkiller - 01-17-2020
Did you disable the caching option in the tyflow object? If you disable this it will have to update all the calculations per frame and is very slow. If this is enabled it will save all those calculations in the ram and will enable real time scrubbing and playback of your scene. It can fill up a lot of ram if you have a ton of particles though, i can easily fill 64gb ram with a few million.
If you have a sequence that you want to save with the PRT/tyflow cache operator, but it has more particles than your ram can handle, you can disable the cache in the main tyflow object settings. It will have to recalculate every frame (before it saves the cache on HDD) however it will allow you to scale things up higher than you normally could. This is very useful when you are partitioning caches for a large simulation such as a krakatoa render. (tens or hundreds of millions of particles)
RE: Tyflow too slow to simulate - VickyVey - 01-20-2020
Thanks for the reply Shadowkiller,
I normally leave the caching option in the tyflow object ON. The scrubbing is okay in the beginning of the time range. But if I scrub around the end, say around 300-400, I have to wait a lot for the simulation update.
That's why I thought maybe exporting PRT cache would be the best. I did Friday night before left my studio to export PRT 1-100 frames only, and again, my 3dsMax crashed at frame 81... This is the third time my 3dsMax crashes while exporting PRT files.
Does it have to do with my RAM ?
So do you mean that if my supposedly RAM limitation is preventing me from exporting the PRT cache, I will have to disable the "Enable Caching" in TyFlow and only then export /Generate the PRT files ? But don't I need the "Enable Caching" ON in TyFlow to export the same cache ?
I am not sure about what "partitioning caches" you mean, I think I have to research more about this topic. So Tyflow alone is good for small simulations and if I need large simulations like the one I am doing (a train pushing a bunch of snow out of the track), we will have to buy Krakatoa render ?
Thanks for the reply guys.
RE: Tyflow too slow to simulate - tyFlow - 01-20-2020
We can't tell what the problem is unless you provide a scene file, crash minidump, etc.
tyFlow can handle huge numbers of particles but you have to be mindful of your RAM when working with large numbers of them.
RE: Tyflow too slow to simulate - VickyVey - 01-22-2020
Hello there,
I was busy working on other projects. Did another test, which I have attached.
We updated the Open CL drives today but it's still slow when I activate the following operators: cluster, particle physics and Particle Bind.
Hope you find out what is going on... I am working on 3dsMax 2018.
Thanks again.
Another thing: once it starts updating all frames, it's very difficult to either pause it or cancel it... it simply ignores me pressing "escape" or "shift+esc"
RE: Tyflow too slow to simulate - shadowkiller - 01-23-2020
(01-20-2020, 10:44 AM)VickyVey Wrote: Thanks for the reply Shadowkiller,The more frames you need to save into memory the longer it can take, but once it is cached the timeline scrubbing should be smooth. Is it trying to recalculate those later frames even after it has done it the first time?
I normally leave the caching option in the tyflow object ON. The scrubbing is okay in the beginning of the time range. But if I scrub around the end, say around 300-400, I have to wait a lot for the simulation update.
That's why I thought maybe exporting PRT cache would be the best. I did Friday night before left my studio to export PRT 1-100 frames only, and again, my 3dsMax crashed at frame 81... This is the third time my 3dsMax crashes while exporting PRT files.
Does it have to do with my RAM ?
So do you mean that if my supposedly RAM limitation is preventing me from exporting the PRT cache, I will have to disable the "Enable Caching" in TyFlow and only then export /Generate the PRT files ? But don't I need the "Enable Caching" ON in TyFlow to export the same cache ?
I am not sure about what "partitioning caches" you mean, I think I have to research more about this topic. So Tyflow alone is good for small simulations and if I need large simulations like the one I am doing (a train pushing a bunch of snow out of the track), we will have to buy Krakatoa render ?
Thanks for the reply guys.
If you run out of ram while its saving caches to disk it can crash max. If you disable the playback caching (in the tyflow obj) it uses FAR less ram for the simulation while you're saving to disk because its not accumulating all the params of the particles into your ram for every frame. The downside is if you move the timeline at all it has to recalculate that frame, but that is irrelevant if you're saving a scene to disk. The playback caching in the tyflow object is only really needed if you want to be able to scrub the timeline back and forth without having to wait for it to update.
As for partitioning, it is a term used from the krakatoa renderer. It refers to caching a particle scene on the HDD, then automatically changing the position seed of the particles and then saving again. This way you can multiply your particles as much as you want and allow for final renders that contain more particles than you would normally be able to do. My largest sim had 500 million particles (5 million x 100 partitions) and that was done on an old 4930k with only 32gb ram. It took 2.5TB on my HDD though. Tyflow can also do this if you save as a PRT cache.
Using lots of bindings might get pretty expensive. Rendering a ton of verts can also be quite expensive. My gpu is struggling when im drawing 5 million tetra geom shapes in my view port.
RE: Tyflow too slow to simulate - VickyVey - 01-24-2020
Oh Thanks for the detailed explanation.
I am doing a side test with regular Pflow using mParticle Flow which seems slightly similar to Tyflow. So far it seems quicker...
I am not sure how to handle TyFlow RAM use so probably I have to cut down to a simpler simulation and use it in a small portion of our production, in a shot lasting maybe 4 seconds... Gotta do more tests to see how far I can stretch to use TyFlow within our resources. Our computers are the Boxx workstations customised for visualisation works. It works alright when we use Phoenix FD sims...
By the way, I am using the latest TyFlow version.
RE: Tyflow too slow to simulate - d4rk3lf - 01-24-2020
I've looked your file (i7 - 32GB Ram), and indeed it's very slow.
But I wouldn't blame Tyflow here, but rather the workflow. For example, I saw you placed 100 bind connections per particle.
I hardly know the case where you need more then 10.
Your voxels are very very small, and I understand why you used custers (you probably wanted organic sticky snow), but when I visualize cluster, it doesn't make much sense to me (they are grouped at the middle).
What I'd suggest you to do, is to create a "small res" Tyflow sim of snow with voxels, and then use another flow (and birth flow) to increase particle numbers.
Here is the sample:
https://vimeo.com/335055310
And the tutorial:
https://youtu.be/BpslvHU4PFQ
There is also other granural tutorials on youtube, and I suggest you to take a look at it.
Also, I think you can't compare Tyflow with Phoenix FD, because Tyflow is not a fluid solver, and a completely different beast.
Hope that helps.
RE: Tyflow too slow to simulate - VickyVey - 01-27-2020
Thanks for checking my file.
I actually followed a few grain tutorial on YouTube and they were all using small voxels. I didn't know this was the actual culprit for the slow down of my machine. I thank you for pointing out that. Also, I have decided not to use the Cluster operator for now...
I will do further tests now that I can manage my scene.
I appreciate the links you've given me. The guy is very kind to post lots of file links there for many types of simulations.
You see, sometimes I get confused when to use Tyflow and when to use PhoenixFD or both. Maybe the more I experiment/study, the more I will understand the limitations and integration of both ?
Okay, let me work on this snow simulation again. Will leave the Pflow on the side for now...
RE: Tyflow too slow to simulate - shadowkiller - 01-28-2020
Yeah when working with voxels you need to be careful. You can easily obliterate any system if you scale them down too small. It is a lot of data your working with.
As for phoenixFD, it is a fluid/gas solver for calculating fluid dynamics. You only need to use it if your are doing some sort of liquid or smoke/fire simulation and you want your particle to swirl the same way. Tyflow doesnt have its own fluid solver implementation, it only has a basic physics collision system. You can use phoenix to push your particles around (fluid force operator) to make effects such as an underwater ink, super wispy smoke or a fluid splash. Phoenix does have the ability to emit its own particles for said effects, however you dont have nearly as much control as you do with the tyflow particles. You can also partition the tyflow particles (prt cache operator) to get a much denser particle cloud and smoother rendering results, at the cost of HDD space.
RE: Tyflow too slow to simulate - VickyVey - 02-06-2020
Hello Shadowkiller, thanks for explaining the difference between Tyflow and PhoenixFD's particle.
You said:"You can also partition the tyflow particles (prt cache operator) to get a much denser particle cloud and smoother rendering results, at the cost of HDD space."
I think I have to do that because when I apply the Tymesher to render out the particles, the number of particles "seem" to diminish considerably loosing that "grainy" detail... See the preview attached.
I reckon if I export a denser particle cloud using PRT cache operator, I can increase the rendered snow using TyMesher, am I correct ?
Thanks guys.
RE: Tyflow too slow to simulate - d4rk3lf - 02-07-2020
Just stumbled by accident on this:
https://www.youtube.com/watch?v=jiGGYMMj-D8
Looks pretty good to me, and the guy provided max file.
I haven't checked Max file, but maybe you should, and maybe it will give you some answers.
Hope that helps.
RE: Tyflow too slow to simulate - VickyVey - 02-24-2020
This example was the one I came across in the beginning and it's the only one close to what I want. But for some reason, that link didn't have a 3dsMax file. So thank you as this link provides with a 3dsmax file.
I was able to achieve the dynamics of the grain correctly after reading loads of different grain tutorials. The problem was how to cache it out and render... The grainy snow turned out to look like a cream when using TyCache and TyMesh... Maybe need to cache far more particles ? Maybe tinker with TyMesh further on ? Maybe buy Krakatoa ?
Unfortunately, we have decided to use only PhoenixFD Fire with motion blur for our project without Tyflow as it's far quicker to experiment with the amount of time allocated for my R&D.
Pity... but I will still continue this snow trial in my downtime... who knows for the next projects...