Latest post Thu, May 30 2019 10:28 PM by cable. 7 replies.
Page 1 of 1 (8 items)
Sort Posts: Previous Next
  • Tue, May 28 2019 10:38 PM

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 395
    • Points 5,070

    Nexis Pro import filesizes vs. local import filesizes

    RE:  Nexis Pro 40TB v 19.2.0.7,  MC. 8.6.5,   Mac OSX Trashcan 10.13.6. 32GB RAM, 1Gb ethernet connection via Netgear switch. 

     

    I noticed today that when I import 1920x1080 jpegs (DNxHD 115, set for 30 seconds, no alpha, no size changes, scale color level) they are 12.6 MB each on the Nexis workspace.  If I import them to the local OS drive they are 1 Mb each. An uncompressed 8-bit 1920x1080 file is generally around 6.3 MB including header info. 

     

    Why would the file size on the nexis workspace be so large?  My current solution is to import them locally and then copy them to the workspaces so this is a cumbersome workflow.

     

    I tried looking up which version of MC are supported with the Nexis Pro 19.2.x  but I can find no reference to this matrix anywhere online. I've found the MC matrix for MC 8.6.5 which is much older than Nexis 19.2.x.  

     

    Has anyone else seen this file size behavior?  Thanks.

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Wed, May 29 2019 12:03 AM In reply to

    • BobRusso
    • Top 50 Contributor
    • Joined on Thu, Oct 13 2005
    • New Jersey
    • Posts 2,261
    • Points 27,935
    • Avid Employee
      Moderator: Avid ISIS
      Moderator: Avid NEXIS PRO
      Moderator: Avid Storage Products

    Re: Nexis Pro import filesizes vs. local import filesizes

    There are a couple things taking place;
    The smallest block in local storage is likely 4KB. With a Scale Out storage group the smallest block is 512KB and 1024KB with High Performance Storage Group. 
    With a scale out storage group if the file is 512KB or less it will use 512KB on the disk. 
    Then the protection is added to this.

    As a rule, 2 Disk Protection on disk is best case is 20% for parity on all large files. For files, less than 512KB size it can be as much as 66%.

    You can stop importing locally; you're just making more work for yourself.

    https://www.youtube.com/avidevangelistbob [view my complete system specs]

    -- Bob Russo Post-Production Workflow Manager - NFL Films

    Bob Russo on Expert Connections

  • Wed, May 29 2019 5:04 AM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 395
    • Points 5,070

    Re: Nexis Pro import filesizes vs. local import filesizes

    Interesting. While it seems I am making more work for myself I am saving 11Mb of space per file and I bring in hundreds a day for about six months. When I import them to my local drive and then copy them to the Nexis workspace they remain 1Mb, they don't become 12Mb once moved. 

     

    Today I brought in 400 frames directly to the Nexis and it used about 5GB, those sames files imported to my local drive and then moved to the Nexis workspace used 400MB total on the workspace. That's a big difference.

    I think there must be something wrong going on. I did this exact workflow on a previous project using a Nexis Enterprise E4 and didn't see anything like this. Tomorrow I'll try using a more current version of MC to see if there is any difference.

    Thanks.

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Wed, May 29 2019 2:31 PM In reply to

    • BobRusso
    • Top 50 Contributor
    • Joined on Thu, Oct 13 2005
    • New Jersey
    • Posts 2,261
    • Points 27,935
    • Avid Employee
      Moderator: Avid ISIS
      Moderator: Avid NEXIS PRO
      Moderator: Avid Storage Products

    Re: Nexis Pro import filesizes vs. local import filesizes

    How are you determining the file size when it's stored on a NEXIS Workspace vs local storage? Are you certain the import settings are identical? 
    I can see no situation where importing locally and copying to a workspace would have any impact on file size. My only conjecture is that the Finder isn't reporting the correct file size when you Get Info.

    https://www.youtube.com/avidevangelistbob [view my complete system specs]

    -- Bob Russo Post-Production Workflow Manager - NFL Films

    Bob Russo on Expert Connections

  • Wed, May 29 2019 11:38 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 395
    • Points 5,070

    Re: Nexis Pro import filesizes vs. local import filesizes

    Hi,

    I have done several more tests today. Our assistant editor also brought in several panels (1920x1080 jpegs, tiffs, pngs). Every file we bring in becomes 12.6MB on the Nexis but only 1MB on local storage. There is no difference between DNxHD36 and DNxHD115. They are still both 12.6MB.  

     

    I upgraded our MC version to 2018.12.2, no difference. 

    I am using the stat command in Terminal to view the file info. I have also created a new Nexis workspace with zero disk protection. No file size difference. 

     

    Here are two stat outputs.  They are both of one of the color bars found in the Applications/Avid Media Composer/SupportingFiles/Test_Patterns/HD_1080. They were both done with identical import settings (other than the drive).

     

    Local disk

    16777224 130 -rw-rw-rw- 1 trevor admin 0 655969 "May 29 15:55:00 2019" "May 29 15:54:48 2019" "May 29 15:54:48 2019" "May 29 15:54:48 2019" 4194304 1288 0 /Volumes/local_media/Avid MediaFiles/MXF/1/SMPTE_BaV01.05522599.586320.mxf

     

    Nexis Workspace with no disk protection

    838860805 2144670444551856 -rwx------ 1 trevor staff 0 12583521 "May 29 15:53:33 2019" "May 29 15:49:50 2019" "May 29 15:49:50 2019" "May 29 15:49:49 2019" 33554432 25600 0 /Volumes/zeroProtection/Avid MediaFiles/MXF/Edit04.1/SMPTE_BaV01.0456019D.586320.mxf

     

    We can see that the local ingested file is 655K while the Nexis ingested file 12.6 MB.  

     

    Tomorrow I'll ask a friend with a large Nexis installation to perform a test for me to see if he sees the same results.

     

    Thanks.

     

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Thu, May 30 2019 1:12 PM In reply to

    • BobRusso
    • Top 50 Contributor
    • Joined on Thu, Oct 13 2005
    • New Jersey
    • Posts 2,261
    • Points 27,935
    • Avid Employee
      Moderator: Avid ISIS
      Moderator: Avid NEXIS PRO
      Moderator: Avid Storage Products

    Re: Nexis Pro import filesizes vs. local import filesizes

    Is the NEXIS using High Performance Storage Group or Scale-out? If the file is copied from NEXIS to local storage, does the size decrease?

     

    https://www.youtube.com/avidevangelistbob [view my complete system specs]

    -- Bob Russo Post-Production Workflow Manager - NFL Films

    Bob Russo on Expert Connections

  • Thu, May 30 2019 4:50 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 395
    • Points 5,070

    Re: Nexis Pro import filesizes vs. local import filesizes

    We are using scale-out.  When we copy files from the Nexis to the local OSX desktop the size remains 12.6MB. 

     

    This system is two Nexis Pro 40TB units setup as one scale-out storage group. 

     

    I've looked at the image data inside the mxf files using a binary editor and found the following:

    The 1Mb file has all of the image data together as one stream.

    The 12.6 MB file has about 11 MB of zeros splitting the image data into two sections.  So several hundred k of image data, 11MB of zeroes and then the remaining image data.

     

    I used mxf2raw to determine the two files metadata are basically identical.  

    If I use mxf2raw to export the raw essence in either of the two files the size is correct. So mxf2raw does only pull out the actual image data, not a bunch of zeroes.  

     

    Weird.

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Thu, May 30 2019 10:28 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 395
    • Points 5,070

    Re: Nexis Pro import filesizes vs. local import filesizes

    OK. I'm hearing now that it's other Nexis installations as well so not just me. And different versions of Nexis and MC.

    Guess I'll call support and complain about the wasted disk space for projects like ours. 

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
Page 1 of 1 (8 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller