Latest post Thu, Oct 27 2016 6:25 PM by mtahir. 5 replies.
Page 1 of 1 (6 items)
Sort Posts: Previous Next
  • Mon, Oct 24 2016 2:16 PM

    • mtahir
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • UK
    • Posts 480
    • Points 5,865

    Running out of memory with a 30s sequence

    I have two 15s stills (logo) on one track, side-by-side, that I was experimenting with. One has a colour effect and the other BCC FX Browser effect. Then there was another track above it with the same BCC effect on it across both clips. There is no audio. The effects aren'rt rendered. It's a 1080i/50 project. MC8.6.3.

    I played around with it for a while and the left it overnight and memory usage was already very high: when I left it, total memory usage on my machine was ~35GB/48GB. When I came back the next morning, there was a Windows message telling me that I should shutdown Avid as I was running of memory. Closing Avid sent it (eventually) down to ~10GB after disabling the Avid Background Service Manager. When I re-started Avid and then shut it down straightaway, the total reported memory usuage on my machine was ~11.5GB, in other words, about 1.5GB went missing.

    I get the impression Avid uses a hefty cache (I'm using the default cache settings) and there seems to be a memory leak when closing the application. It was only last year that I was using MC5.5. on a 16GB machine with many tracks, long sequences, and complex BCC effects with no memory issues.

    MC2018.1 (Win7) | Dell T7500 with 2 x 6-Core Intel Xeon X5690 3.46GHz CPUs | Triple-Channel 64GB (32GB per CPU) 1333MHz DDR3 EEC FBDIMM | 3GB nVidia Quadro... [view my complete system specs]
  • Tue, Oct 25 2016 10:21 AM In reply to

    • Bruno M
    • Top 200 Contributor
    • Joined on Thu, Feb 11 2010
    • London, UK
    • Posts 604
    • Points 7,620

    Re: Running out of memory with a 30s sequence

    It's one of the problems of modern software, that the memory requirements have grown to what many people believe are ridiculous proportions. The main installer for MC 8.6 is around three times the size of that for MC 5.5, and extras such as Avid's Application Manager and Background Services only add to the amount of memory required. Modern plugins can also be memory hogs, and having Boris FX on your timelines can make your bin file sizes stupedly large.

    I used to run MC 5.5 on an old HP workstation with just 8MB of RAM and it never ran out of memory!

    As to your particular issues, I think memory leakage is quite common, and one reason why you should re-start your workstations (say) once a day. I would personally remove App manager and Background services from your startup, unless you really need them running constantly. You might also want to check to see if you have a Windows paging file set up. I've seen workstations with larger amounts of memory installed, where the prevailing wisdom was that you didn't need a paging file, as disk memory is slow. Although it's a good idea to have enough memory so that you never need to 'page' memory out to disk, having one set up may avoid software crashes.

    Bruno

    HP Z800, HP G3 Studio, SonnetFusion RAID, Mojo DX, Symphony 2018.9, JVC DTV1910 HD tube monitor, HP Dreamcolor, Avid Artist Color [view my complete system specs]
  • Tue, Oct 25 2016 11:38 AM In reply to

    • mtahir
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • UK
    • Posts 480
    • Points 5,865

    Re: Running out of memory with a 30s sequence

    Thanks for the reply, Bruno. I've got a pagefile setup but it's 50MB - I  think that's needed to handle any memory dumps during crashes etc. Odd as it my sound, perhaps I need to increase that as Windows will use it/needs it irrespective of the amount of memory installed? I was surprised that Windows is saying I'm running out of memory when there was 13GB left. I did some searching and found this:

    "No matter how much RAM you have, you want the system to be able to use it efficiently. Not having a page file at all forces the operating system to use RAM inefficiently for two reasons:

    First, it cannot make pages discardable, even if they have not been accessed or modified in a very long time, which forces the disk cache to be smaller. Second, it has to reserve physical RAM to back allocations that are very unlikely to ever require it (for example, a private, modifiable file mapping), leading to a case where you can have plenty of free physical RAM and yet allocations are refused to avoid over-committing.

    You need to have a page file if you want to get the most out of your RAM, even if it is never used. It acts as an insurance policy that allows the operating system to actually use the RAM it has, rather than having to reserve it for possibilities that are extraordinarily unlikely."

    [http://www.howtogeek.com/199990/should-i-disable-the-page-file-if-my-computer-has-a-lot-of-ram/]

    I'll increase it to something sensible and see how things workout.

    MC2018.1 (Win7) | Dell T7500 with 2 x 6-Core Intel Xeon X5690 3.46GHz CPUs | Triple-Channel 64GB (32GB per CPU) 1333MHz DDR3 EEC FBDIMM | 3GB nVidia Quadro... [view my complete system specs]
    Filed under: , ,
  • Tue, Oct 25 2016 1:17 PM In reply to

    • Bruno M
    • Top 200 Contributor
    • Joined on Thu, Feb 11 2010
    • London, UK
    • Posts 604
    • Points 7,620

    Re: Running out of memory with a 30s sequence

    mtahir:
    You need to have a page file if you want to get the most out of your RAM, even if it is never used.

    Must say, I've never heard this reason for having a page file. I remember reading somewhere that your page file size should be somewhere between two and three times the size of your physical memory, so the 50MB you have is very small.

    There are also various discussions as to the use of page files with SSD drives. Some say it's not a good idea, because of SSD's limited read/write cycles, you will be shortening the life of your drive. Others say it's good to put your page file on a SSD because it's much faster than conventional drives.

    I have a SSD in my setup, and I never used to have a page file until one day, when I was experimenting with the Fusion AVX connection plugin. Whilst I was rendering out my Fusion comp back to Avid, I ran out of memory and Fusion simply crashed. Once I'd added a page file all was good. I was mindful of the advice about wearing out the SSD, so I made a page file with the same minimum and maximum size. This should result in less read/write cycles, as normally the page file is being dynamically re-sized when being managed by the OS.

    HP Z800, HP G3 Studio, SonnetFusion RAID, Mojo DX, Symphony 2018.9, JVC DTV1910 HD tube monitor, HP Dreamcolor, Avid Artist Color [view my complete system specs]
  • Tue, Oct 25 2016 2:25 PM In reply to

    • mtahir
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • UK
    • Posts 480
    • Points 5,865

    Re: Running out of memory with a 30s sequence

    I don't use SSDs but I've always set the min/max sizes the same for the pagefile. Yes, there is rule about it being some factor of the RAM size but with a large 48GB of RAM, the pagefile will be the size of one of my partitions! There's a lot of information about this out there. This article explains how you work out what your application needs by using the Performance Monitor [https://blogs.technet.microsoft.com/motiba/2015/10/15/page-file-the-definitive-guide/].

     

    MC2018.1 (Win7) | Dell T7500 with 2 x 6-Core Intel Xeon X5690 3.46GHz CPUs | Triple-Channel 64GB (32GB per CPU) 1333MHz DDR3 EEC FBDIMM | 3GB nVidia Quadro... [view my complete system specs]
  • Thu, Oct 27 2016 6:25 PM In reply to

    • mtahir
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • UK
    • Posts 480
    • Points 5,865

    Re: Running out of memory with a 30s sequence

    This is the most detailed explanation for how the pagefile is used. It's quite a read but I'm begining to understand why Windows was reporting low memory - it's the value of the 'Commit (GB)' that matters. Right now, Windows Task Manager is showing 24.2/48 GB for physical memory usage but the Commit (GB) is 35/48 GB - more than 10GB higher. More about this in the article (see the 'Committed Memory' section). 

    MC2018.1 (Win7) | Dell T7500 with 2 x 6-Core Intel Xeon X5690 3.46GHz CPUs | Triple-Channel 64GB (32GB per CPU) 1333MHz DDR3 EEC FBDIMM | 3GB nVidia Quadro... [view my complete system specs]
Page 1 of 1 (6 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller