Onsite Data Wrangling with YoYotta

At a Glance:

YoYottaID Ideal Customer

• onsite data wrangler
• any file-based media producer from student to professional

What it does

• create backups of file based media and checksum data
• write directly to LTO tape
• dual destination write
• custom folder naming
• add options for transcoding/conversion, add LUTs, etc
• conforming of data to XML and EDLs

Digistor Solutions with YoYotta

We can supply a turnkey solution with YoYotta ID and options required, a suitable Mac system to meet the needs, LTFS- capable LTO6 tape drive, installation/commissioning services, training.

YoYotta: File transfer and Conversion Utility

Digistor is always on the lookout for products to make your life easier when it comes to storage and workflow. I would like to discuss a new software and hardware product called YoYotta.

"Data wrangling" has rapidly become one of the most important tasks in the modern content-creation process. Since the move to file-based media for all origination in our industry, it is the one task that we all must do to ensure safety of our data.

It doesn't matter whether you're simply copying files from a DSLR for use in Photoshop or Lightroom, getting that GoPro footage into your edit system so you can re-use the Micro-SD cards, or you are the Data Wrangler/Digital Imaging Technician (DIT) on a major feature production. At every stage you are going to need to need to create file copies and backups as you go, or else you run the real risk of losing your valuable data.

Importing files to your storage seems like a simple enough process, however the job of handling files is getting more complex each year. The number of new cameras, file formats and wrappers, and file-based media disk types released is increasing rapidly. Combined with the need to supply different file versions for different needs along the way, it can be complex. We also have all experienced some loss of data at some stage to know that without some organised and repeatable procedure, it is quiet easy to make a mess of your storage - and ultimately your projects. The time spent storing, finding, and transcoding files is increasing exponentially as we produce more data and want to store it away for safekeeping.

So, what is YoYotta? To start with, it's a weird name for sure. It's also a sophisticated yet inexpensive Mac-based software designed to streamline many day-to-day tasks for any content creation professional.

YoYotta offers a single family of software products that can organise your entire media workflow. There are several other products that do some of the tasks required in a similar way, but not many that combine all facets and the many operations that YoYotta provides in one affordable package.

YoYotta benefits include:

• fast, secure backup of camera media
• easy full or selective import of media from shuttle or drives
• a built-in project and metadata store allowing better organisation and fast searching
• archive capability to LTO tapes using the flexible and compatible LTFS format.
• the (rare) ability to span multiple LTO tapes for large projects
• supports two standalone LTO5 or LTO6 drives (HP, IBM, Quantum)
• optionally supports Tape Libraries for automated and multi-tape archive and restore
• can transcode files to common working/editing formats
• Ability to convert ARRIRAW, DPX, RED R3D, MOV, MP4, MXF clips to H.264 MP4, ProRes Proxy, LT, 422, HQ, 4444 MOV and DPX
• Ability to apply ASC CDL and LUT while transcoding
• 16 Bit GPU accelerated decode and high quality multi-pass encoding
• Select media for archive, restore, trim, or transcode using an ALE or EDL
• Trim ARRIRAW, DPX, RED R3D, and XAVC clips
• Create multi-tape archives efficiently and effortlessly
• Manage up to 48 tapes online and unlimited number of offline tapes

This is all bundled into a very cost effective package. If you don't need all features, you can start with the base package and add options as your needs grow.

Below are the software applications and the options available:

YoYottaID

The entry level package, YoYottaID has all of the basics you need. The capabilities include:

• Select Media Sources, select backup destinations, and start the backups. Simultaneous backups can be created, such a creating a copy on LTO as well as a working copy for the SAN or NAS. The time remaining is displayed at all times
• Organising the job is the priority. YoYotta allows you to structure your jobs using a built in Project and Metadata store.
• YoYottaID can make consistent folders based on Project, Day, Date, Camera and Media names. This will make life easy throughout your project. Presets are automatically recalled for different camera types, making life even easier.
• File access permissions can be painful when working across multiple users and systems. YoYotta includes the ability to define file permissions on all files during each process stage.
• Don't make any mistakes: once your cameras cards or drives are connected, they are mounted by YoYotta as "read-only". The system calculates a MD5 checksum whilst copying and verifies this throughout the process. Each backup is verified independently. And you can output a CSV or PDF file report so you know exactly what was copied.

There are several optional packages available to add to YoYottaID. These are:

YoYotta LTFS and Automation

Linear Tape File System (LTFS) is a standard format for writing files to tape. It's ideal for storing production and post media, as the LTO tapes can be mounted and read on Linux, Mac OS X and Windows. You also aren't locked into a proprietary archive. LTO6 tape holds 2.5TB and copy and restore files a around 60MB/s. Unlike hard drives, these tapes are designed with a 30 year shelf life

YoYottaID LTFS includes all the features of YoYottaID and add the ability to create archives to two standalone LTO drives using LTFS. LTFS does allow access from the desktop, but it's not the fastest way to read and write. LTFS is a tape that cannot be accessed in parallel, however YoYottaID has a smart queue to get around this issue. Multiple backups to tape can be started and they will be processed in order through the YoYotta software.

YoYotta Automation adds the ability to handle Tape Library units.

YoYotta LTFS and Automation benefits:

• Quickly create industry standard archives on LTO5 or LTO6 tapes.
• YoYottaID names, formats, mounts and repairs tape archives.
• YoYottaID remembers the contents of every tape created so later on you can search to find the location of shots.
• If the backup or archive won't fit on a single volume, YoYotta intelligently fills each volume without splitting shots. If you are using a LTFS drive, you use this feature top span multiple LTO tapes.
• YoYotta supports several brands of standalone LTO tape drives including the popular Thunderbolt attached mTape unit available from Digistor.

YoYotta Transcode

This option includes high quality transcoding creating DPX, MOV or MP4 copies of material
along with 3D LUT processing and image resizing. Perfect for the onset "Data Wrangler", you can connect a drive, or drop files into a folder and transcode all the material. You can also select which folders or source types need to be transcoded.

Camera formats currently supported:

• ARRI Alexa ARRIRAW, ProRes including ProRes 4444 XQ and DNxHD
• ARRI Amira ProRes
• Sony F700, F5, F55 and F65, MPEG2, SStP, XAVC, RAW
• RED 1, EPIC with Dragon, R3D
• DPX

Output codecs currently supported:

• H.264 MOV
• H.264 MP4 with AAC audio
• ProRes Proxy, LT, 422, 422 HQ and 4444
• DPX 10 bit or 16 bit

A the time of transcode, CDL, 3D LUT, REC709 Matrix processing is also possible. You can:

• add a CDL file to apply slope, offset, power and saturation
• add a 16/17/32/33/64/65 point 3D LUT to convert Log material to REC709
• add a CDL file to apply power, offset and saturation

An finally, you can also create DPX sequences, outputting these to 10 or 16 bit DPX, or ProRes 444 for online workflows. Nice!

YoYotta Conform

Conform allows the restore of clips from archive using project EDLs (soon AAF and ALE as well).
Copy or Trim selected material to match conform. YoYottaID searches the database for material with matching reel and timecode. There is a scrollable timeline showing the conform. To add the conform to YoYottaID click the Add Conform as Source button.

Other features include:

Raw trimming: Instead of copying the entire clip YoYottaID can trim the files, creating new smaller clips without transcoding. This way you can bring raw into grading sessions without the large file size overhead. Here YoYottaID will create two identical trimmed conforms, one on the RAID and one on an external drive. Again saving huge amounts of time.

YoYottaID Conform will trim ARRIRAW, DPX, RED R3D and Sony SStP, RAW or XAVC files.

Advanced conforming: In the conform panel you can adjust the handles. YoYottaID will extend each edit by this number of frames. Filter the clips by codec or resolution.

Conform and Transcode: Use the conform as a source for the YoYottaID Transcode option. Easily create over-length selects from the source material for online.

Digistor-supported YoYotta Turn-key Systems:

Digistor offers a customised solution for different uses. Solutions include:

• Mac Pro system to meet the need, whether in-house or a more portable solution for location use
• Thunderbolt LTO Tape Unit or tape library unit
• YoYotta ID with LTFS option
• Other options as required for your needs: Transcode, Conform, Automation, etc.
• Installtion, training and support services, as required