Have you been recording sound effects for months, or even years? Perhaps your library began with a handful of tracks, then grew with sputtering leaps and collapsed into a jumbled heap of tangled metadata.
In the last article I explored simple ways to touch up metadata. I featured a particular metadata app, Soundminer, and highlighted 10 tools it uses to add, edit, and save metadata.
Today I’ll continue exploring how Soundminer can enhance your collection with metadata. In this post I’ll share a more complex but powerful method: updating your metadata with external files.
It’s more involved than what we learned last post. However, it has great power to fix, transform, and polish vast amounts of metadata simply. Are you a sound librarian? Do you work with a massive collection of sound effects or music? Today’s article is for you.
What Are External Files?
What are external files? They are text, Excel, or PDF files that contain information about your sound library. They’re external because this data isn’t bonded with the audio itself. It resides outside of your metadata app. They could include a PDF shooting list or notes in a Word document.
This suggests a problem. What if these files are lost? They’ll be separated from your sound library, and you’ll lose this information. The safest way to preserve this information is to embed it into the audio itself as metadata, forever marrying it to the sound file.
Why Use External Files?
The last article featured 10 increasingly powerful ways to add and modify sound file metadata. Isn’t this good enough?
In most cases, yes. However, applying and modifying data with external files has compelling incentives:
- Modify metadata with more powerful, specialized apps. Soundminer’s find and replace and metadata admin functions are strong. They are ideal for touching up large portions of data. Detailed work isn’t as easy. Even Soundminer’s tools feel cumbersome and constraining when working with complex sound libraries. Work is often fragmented and finicky. Apps like Excel, FileMaker, and Word are better suited to making large changes. FileMaker and Excel in particular have powerful tricks that apply complex text changes quickly and in bulk.
- Add external data. Not every collection has metadata already. Often a sound library’s information is stored in PDFs or text documents. This information isn’t attached to your sounds. Merging data from these “external” files to Soundminer data binds the text with the audio. There is no other way to import large amounts of foreign data into metadata other than manually transferring and typing entries one by one.
- Correct your data. Did you use “fighter jet” or “jet fighter” in your descriptions? Maybe you’ve mixed American and British spelling. Some words may be spelled incorrectly. Perhaps you’ve since decided to use a better synonym. Use external files to fix problems, then apply changes to all your existing metadata in one broad sweep. Most metadata apps have a “find and replace” function that works similarly, but only on a single field (column) of data. External files do the heavy lifting far more powerfully.
- Merge data. Perhaps your metadata is 90% complete. Maybe the “microphone” field is empty, but you have this info in an Excel spread sheet. There’s no need to sift through each of your sounds and add the microphone model manually. An external file can merge the new data smoothly with the old.
Overview of Using External Files
How is this done? Here’s a quick overview of what we’ll accomplish:
- Export the existing data from Soundminer to a text file.
- Modify the text file with better metadata.
- Reimport this text file back into your metadata app.
- Embed the changes into the metadata.
So, in other words, it’s taking our existing info, changing it, then replacing updated info on top of it. Then we lock it in by embedding it into the sound file.
How to Use an External File to Add Metadata
Creating the External File
Select the files you want to change. There are a few ways to do this:
- Perform a search for the files you need.
- Select a portion of files from a list.
- Create a new database (Database/Create a new database menu selection). Add just the files you want changed into that database by draging and dropping them onto the app, or with the Database/Scan Sounds into Database menu item. This option is best since it isolates your changes. There’s no chance of damaging other, unrelated files.
Sort the selected records. Display all headers/columns/fields. Soundminer will only export whichever fields are displayed. So, use the “ColumnView” pull down to display all the fields you need. Click the headers to sort them as you prefer. I sort alphabetically via Filename.
Export the file list. Next, we need to create a text list that we can modify in other apps. Choose the Database/Dump Query to Text File menu item. This will output a plain text file of all the sounds in your list with the data from each of the columns, sorted as you decided in step 2.
Soundminer is rare in that it can both export and import data via text files. Your milage may vary with other metadata apps. Also note that only advanced versions of Soundminer allow this.
Don’t have any metadata in your existing sound files? Export the list anyway. It will contain only the file name, which we will need in a moment.
Modify the text file encoding In versions prior to 4.5, Soundminer spits out this list as a particular text encoding format known as Unicode UTF-16. To allow it to be used more simply in other apps, we’ll need to change this encoding.
Open the text file using the free TextWrangler app, choose the File/Save As… menu item, and change the encoding to Unicode UTF-8. Save the file with a slightly different name. This allows the file to be opened in Excel neatly sorted in columns. This makes sorting through the data far easier.
Change the data. This step varies quite a bit depending on the app you’re using to modify the text file. I’ll show an example using Excel, since it displays things a bit more clearly.
Here’s an image of my Soundminer text as viewed in Excel. I want to populate the “Location” column. I also want to add two other columns which don’t appear in that list: “Category” and “Subcategory.” Thankfully, I have all this info saved in another Excel file, as you can see in the next image.
I simply select the new data, copy it, and paste it into the Soundminer file. The result looks like this:
This creates our new “import file.” Of course, this is a very basic way of changing data. Scripts and formulas can alter text in intricate ways, beyond the abilities of metadata apps. Other ideas include:
- Spell and grammar check in Word, across dozens of metadata fields at once.
- Using Excel’s Concatonate function to build complex strings of text. Other elaborate Excel equations can replace and subsitute text.
- Use FileMaker scripts to find, replace, and populate dozens of fields with the click of a single button.
- Quickly find and replace across a number of fields (Soundminer limits replacements to a single column or selection).
- Extract and apply data. Legacy sound libraries may have information locked in PDFs. External files allow you to use OCR technology to extract text data, then dump it into columns.
- Copy and paste text from the Web, other text files, and so on, and apply it directly to metadata.
Check your work. It’s easy to accidently add errors to your new import file.
- Mismatched fields. Rows or columns can become offset, meaning new data will applied in the wrong row. A sound file may mistakenly receive the data from a record before, or after. Review and ensure everything lines up properly.
- Never modify Soundminer’s “Filename” or “FilePath” fields. Soundminer uses these to link your changes with the files on your drive. Changing them means files won’t be found, and the new data won’t be applied.
- Always keep the column names at the top of the list. Soundminer uses these titles to learn where to insert the new data. The names must match the spelling of the Soundminer originals precisely.
- Ensure there’s a “tab character” between the data in each column. This keeps the data in one column from becoming confused with its brothers on either side.
Create the import file. At this point the hard work is over. Save your revised data as a new text file.
Applying the Changes
- Create a new, temporary database in Soundminer. This ensures that any changes you’re about to do won’t mix with other files. It also allows you to scrap any attempts without affecting other Soundminer databases.
Import the new, updated import file into Soundminer. Select the Database/Import Text Into Database menu item. Soundminer will ask you for a text file. You’ll notice the warning text explaining the importance of the “FilePath,” and headings. Select your new import file.
Immediately afterwards, Soundminer will ask you to locate the sound files from your text file. Navigate to the parent folder of the files, then press the return key. Soundminer will search for the files you listed in your new import file. If everything goes well, and has linked properly, and you’ll see your new data unfurl beautifully in Soundminer’s browsing window.
You may receive a few error messages along the way, though. The left image below shows a message that displays if Soundminer finds a header name it doesn’t recognize. Another explains that your preferences have chosen to omit duplicate files.
If Soundminer has problems finding a sound file, it lists each error in a log window.
This commonly happens if:
- A file has been moved since you exported your file list from Soundminer.
- A file has been renamed since the export.
To fix this, re-open your import file. Refer to the log information to find the troublesome record in your import file. Correct the error and try importing the file again. Often this just is a matter of updating an incorrect file name in your import file. In the example above I had changed “20s” to “twenties” in the file name, but this wasn’t reflected in the import file. This destroyed the link between the record and the file. I simply touched up the text in the import file to read “twenties,” reimported it, and everything was fine.
Commit metadata. Select all records, right-click the selection, and choose Embed Selected Records from the pop up menu. This will write your new metadata to the file.
Refresh your metadata. You may have the same files in other Soundminer databases. You’ll notice that the new metadata hasn’t been updated there. That’s no problem though. Simply find those files, select them, right-click the selection, and choose Rescan Selected from the pop-up menu. This will look up the file metadata again, and automatically fill the empty fields.
The Appeal of External Files
Why bother using an external file to touch up your metadata? Isn’t it simpler to just use your metadata app’s native options?
It’s true that that is an easy way to modify your text. It’s just limited. Often changes are only possible across one field, or column. It may involve repeating finicky work, and monotony.
Using an external file is incredibly powerful. There’s no other way to add vast amounts of foreign data quickly. It allows you to use scripts and functions to populate empty fields with a single formula, or a couple of clicks.
You’ll save time. Your sound library will become attractive, and reliable. It will become enriched. And, your clips will find themselves in more projects the better your metadata becomes.