Latest post Mon, Jan 14 2019 3:30 AM by AvowedMediaComposer. 18 replies.
Page 2 of 2 (19 items) < Previous 1 2
Sort Posts: Previous Next
  • Fri, Jan 11 2019 12:21 PM In reply to

    • chuckkahn
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 330
    • Points 3,690

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Is there a way to use a Mac formatted drive with symlinks and SDBs on a Windows Avid?

    Avid Media Composer 2018.11 / Mac OS Sierra / MacPro5,1 / 6-Core Intel Xeon 2.66 GHz x2 / 64 GB 1333 MHz DDR3 ECC [view my complete system specs]
  • Fri, Jan 11 2019 4:47 PM In reply to

    • chuckkahn
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 330
    • Points 3,690

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Comparing ways to create Sparse Bundles....

     

    1) Disk Utility -- I set the SBD size to 8 TB and the size of the .sparsebundle file is 1.51 GB.

     

    2) CreateDiskImage app -- also set the SBD size to 8 TB and the size of the .sparsebundle file is 59.6 MB

     

    Why the difference in sizes?

     

    Also, how do I create a proper SBD in the OS X Terminal?   I looked at the options in the hdiutil command and they seemed quite numerous:

    https://pastebin.com/mEWL3i82

     

    So I tried this:

     

    hdiutil create -size 8tb -type SPARSEBUNDLE -volname "bkp_WM_S2_Masters" -fs "Case-sensitive Journaled HFS+" /Volumes/WM3_Edit_Bkp3/bkp_WM_S2_Masters.sparsebundle

    and the size was 1.53 GB.

     

    So why does the app make the .sparsebundles smaller?

     

    update:  I think I see the reason:  the Create Disk Image app defaults to 10 GB and so its .sparsebundles are smaller than the 8 TB ones I created.

     

    update: adding a -nospotlight option:

     

    hdiutil create -nospotlight -size 8tb -type SPARSEBUNDLE -volname "bkp_WM_S3_Imports" -fs "Case-sensitive Journaled HFS+" /Volumes/WM3_Edit_Bkp3/bkp_WM_S3_Imports.sparsebundle

    Avid Media Composer 2018.11 / Mac OS Sierra / MacPro5,1 / 6-Core Intel Xeon 2.66 GHz x2 / 64 GB 1333 MHz DDR3 ECC [view my complete system specs]
    Filed under:
  • Fri, Jan 11 2019 5:01 PM In reply to

    • chuckkahn
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 330
    • Points 3,690

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    My use case is to back up project ISIS workspaces to an external drive to subfolders and then create sparsebundles with symlinks so that an editor can access the sub-foldered backup in Media Composer.   That way I don't have to predict what final partitions sizes I need and I can hide the MXF backups from Avid's scanning for new mediafiles -- the SBD/symlink is the gateway to making the subfoldered MXF backups mountable/unmountable.

    Avid Media Composer 2018.11 / Mac OS Sierra / MacPro5,1 / 6-Core Intel Xeon 2.66 GHz x2 / 64 GB 1333 MHz DDR3 ECC [view my complete system specs]
  • Mon, Jan 14 2019 3:30 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    chuckkahn, in direct response to your first question, to my knowledge - no. But I read your second post about your use-case and unless I've misunderstood what you're trying to do then I think I can advise you what steps to take.

    For the first problem, that of mac drives on windows machines, Windows doesn't support HFS drives natively, however, if you haven't already chosen what drive to use and haven't already begun your back up process before reading this, then you can use the option of formatting the external drive to the exFAT filesystem (rather than HFS) which will be accessible to both windows and MacOS. exFAT has drawbacks in reliability, and is reputedly more likely to have corrupted data (for example if something happens to interrupt a file transfer in progress) than NTFS or HFS+, I mention this because it would be bad not to disclose the risks however I'll also note that, you're calling this a 'backup' drive and since you're handing it off to editors there's presumably several backups of that backup that are left inactive specifically just to store the data and not be messed with. So then, as long as you take this corruption risk in to account, AND HAVE OTHER, SAFER BACKUPS IN PLACE THAT AREN'T BEING REGULARLY ACCESSED BY EDITORS, my anectdotal experience is that exFAT has worked perfectly well for editing with no increased incidence of issues compared to any other filesystem. Otherwise if there is a reason it must be Mac formatted, there are also paid software options to provied support for HFS drives in Windows, so one way or the other you can access the material on the mac drive on your windows machine.

    Symlinks created in MacOS do not work in windows, however windows is capable of creating symbolic links of its own, which likewise work only on windows, and also only on NTFS. Unfortunately, the mixing of platforms makes things complicated. Theoretically, since the filesystem itself allows for symlinks, I wonder if the paid solutions for HFS support on windows also provide such support, but I have no idea, and somehow I doubt it.

    If you are wanting to backup the contents of one Avid MediaFiles directory to an arbitrary folder (other than \volume root\Avid MediaFiles) in order to hide those backed up files from Avid's scanning, but you also want a quick way to make those arbitrary folders behave like a traditional (\volume root\Avid MediaFiles) that Avid can use it as a media directory, your means and ability to do so depends on the platform the editor will be using. I hope I'm not misunderstanding your question but I'm guessing you'd be giving the editor this drive either as a contingency in case an emergency has happened and the ISIS is down and work must continue using the backups, or because that particular editor is for some reason not given access to the ISIS for their particular work and must use the external drive. I also have to assume that there's a reason you can't use the simple method for achieving this and that's why you want to use symlinks and sparsebundles. I'll list what the simple method (which uses neither symlinks nor sparsebundles or any kind of virtual volumes) is below for reference but will also list what you need to do if indeed you can't use the Simple Method

    The Simple method

    So you are making backups from ISIS storage to an external drive and you want the destination folder for those backups to be something other than \volume root\Avid MediaFiles in order to avoid Avid scans. Presumably then, when a backup is complete, and you've handed the drive to the editor and you're on their machine, you can just cut and paste the mxf folder you backed up from the arbitrary folder you chose, to the drive root in a folder called Avid MediaFiles.

    Drawbacks:

    This method allows only one project accessible at a time on the editor's machine but I'm guessing on average each of the editors you'd hand the drive to would only need the one project at a time and if they need to switch you simply close Avid, move the folder back to it's original backup location and do the same trick with a different backup folder you may have on that drive.

    Using arbitrary folders as real working Avid media folders (windows method)

     

    1. 1) Although windows is capable of symlinks, if your goal is just as the title for this method describes, you don't need them. That's a good thing too because windows doesn't as far as I know have a sparse bundle equivelant meaning disk images would need to be huge otherwise. Actually achieving this goal is easier in windows and requires less rigmarole than MacOS.
    2. 2) To make a given folder act like an Avid media folder, you need to place an 'Avid MediaFiles' directory and MXF subdirectory inside it, as though the folder were itself a drive and then you need to right click on the folder containing the media, access it's properties and make it a shared folder.
    3. 3) Now access your own computer (the editor's computer) as a network location by typing \\COMPUTERNAME in to an explorer window address bar or through the run command, and you'll see all the sharepoint locations available on the computer, one of which will be the folder you flagged as shared earlier.
    4. 4) Right click the sharepoint and click 'map as network drive' and assign it a drive letter. You'll now have a new volume listed in the windows 'computer' menu with it's own drive letter, within this 'drive' there will be an 'avid mediafiles' folder meaning as far as Avid is concerned, you've plugged in a drive with an Avid MediaFiles folder on the root, which is all the preconditions necessary for Avid to the read the media from that location.

     

    Drawbacks

    Although you are accessing material on locally attached storage and not over a network of remote machines, it is still a 'network drive' as far as windows is concerned meaning the maximum available speed of the drive is limited to the speed of the machine's NIC which for most machines is 1Gb. This is mostly fine, and if you have 10Gb ethernet card or something even better then it's totally fine, but sometimes with certain material or in certain rare situations, this is problematic. You should test first for a given project. It also means that if you use the 'consolidate' function to move a lot of material around then the speed is compartively slower than it otherwise might have been on that locally attached drive (depending on the type of physical storage and data connection used). However if you're talking about a mechanical hard drive connected by USB 3.1, the read speeds of spinning platters isn't often that much higher than 1Gbps anyway. RAID0 drives can probably do better so you would be hampering the speed of your external RAID in such a case but again, with testing you'll probably find that it handles most requirements with ease.

     

    Using arbitrary folders as real working Avid media folders (MacOS method)

    The full method is described earlier in the thread, so this is just here for me to point out that, if the windows machine in the mix is just an intermediary being used for making the backups to the external drive, while the editors' machines are macs, there is no way you can prepare the backup drive ahead of time for editors to simply plugin and mount a sparsebundle and be ready to go, windows gets in the way here because it A: supports only its own special form of symlinks that aren't compatible with MacOS B: only supports symlinks on NTFS formatted drives C: has no equivelant of a sparsebundle.

    However, there's nothing stopping you from using the windows machine to perform the backup on to an external drive in an arbitrary folder that Avid can't scan, and then hooking up the drive to the MacOS computer where the editing is performed and creating both a sparebundle to act as a mountable volume and then a symlink to act as a conduit/funnel to redirect the media written to/read from that volume to the folder of your choice (in your case the existing folder where the media was backed up to).

    Drawbacks:

    This method will require that you give the drive to the editor using a mac and then perform a bunch of steps on that mac itself to make it work, meaning you can't just give them the drive and say "doubleclick on the .sparsebundle file and it'll work". Either you will have to take the steps on their machine prior to the editor starting, or they'll have to read the steps themselves and perform them, or if you have a spare mac, you can then take an extra step of pluggin the drive in there and preparing everything beforehand.

     

    I hope I've not completely misinterpreted what you're trying to do, but it sounds like you can skip all the complexity and just use The Simple Method.

     

Page 2 of 2 (19 items) < Previous 1 2

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller