But that aside, I found myself working with huge files that swapped like crazy on my 32GB of RAM, or were completely infeasible to work with. Janky local experiments I wouldn’t want to run on a cloud instance billed by the minute.
128GB has been a godsend, but I could completely fill 192GB and still swap some if I had it.
But I found there are happy side effects to having so much disk cache, too. For instance, game modding/tinkering has sped up immensely over 32GB. So has hashing for transfers, or other scenarios where it’s useful to already have a file cached in RAM.
Machine learning/data science stuff.
Local LLMs is half of it.
But that aside, I found myself working with huge files that swapped like crazy on my 32GB of RAM, or were completely infeasible to work with. Janky local experiments I wouldn’t want to run on a cloud instance billed by the minute.
128GB has been a godsend, but I could completely fill 192GB and still swap some if I had it.
But I found there are happy side effects to having so much disk cache, too. For instance, game modding/tinkering has sped up immensely over 32GB. So has hashing for transfers, or other scenarios where it’s useful to already have a file cached in RAM.
Fair, I guess I’d also want some ram if I did not have access to a compute cluster.
But to be fair, for what I do I don’t really think you can fit that much ram on a consumer motherboard.
In machine learning I don’t generally find memory to be the bottleneck, as long as the GPU can handle it.
But sure, opening huge files can be big trouble.