When Olminator complains about an invalid .olm archive – First Aid to the rescue

The most likely cause for Olminator to complain about an invalid or corrupt .olm archive is that indeed the .olm archive is invalid or corrupt.

Given that some .olm files are many GB in size and taken that exporting the .olm archive to a USB memory stick or transferring it over a wide-area network like the Internet to some remote Cloud storage as intermediate step can damage the archive during transition, this unfortunate situation can occur. A few users ran into it and reached out to the support guy to help analyse and resolve.

As in all such situations, there's no need to panic: Most likely there's no lives at risk, and for everything else there's a good chance it can be fixed before anyone gets harmed. If both of us are lucky you have a chance to simply re-export a fresh archive from 'Outlook for Mac' containing your items and with the necessary diligence you can ensure that this time the file doesn't get corrupted in the process. To increase the chances of this being the case, you can decide to store the file directly on a device capable of reliably handling large data files. This could be your Mac's built-in harddrive or – if free disk space is an issue there – an external disk drive offering sufficent space. USB sticks for some reason tend to be less reliable. Nevertheless, if you decide to use one of them, make sure it is big enough, still properly working and please don't just unplug it manually from the USB slot when done, but properly eject it via Finder first so that MacOS and its file system can ensure that all data still residing in some write caches is actually written and persisted properly before ejecting. And as a rule of thumb as this has turned out to be a cause for headache: avoid transferring such large archive files from one Mac to another via your favorite Cloud storage (like Dropbox, iCloud, Google Drive or all the others). If we are lucky, you are already successful with this approach and Olminator succeeds loading the new archive.

But sometimes one ain't lucky. Maybe you can't just re-export a fresh archive. Or for some reason Olminator keeps complaining no matter how you try. Then it may be time to switch on First Aid functionality in Olminator itself by selecting 'First Aid' from Olminator's 'Help' menu. Hold on to your seat, because this is not for the faint at heart: in First Aid we will try to extract those items from the corrupt .olm archive that can be rescued. Please read the on-screen instructions carefully and follow the steps as advertised. God speed.

In either case, it would be great to hear from you should you run into this issue. Please contact the author of Olminator via email at mailto:'Björn Goerke' for further support or just to get some emotional backing.

What does First Aid actually do?

First Aid is your self-aid area within Olminator that supports you in the attempt to fix a broken .olm archive. It guides you through several steps or measures to do so. Detailed instructions are displayed within the First Aid section of Olminator to explain the steps you need to take and – depending on each measures outcome – what further options may apply.

Caveat: Let's mention though that First Aid may not successfully fix any broken archive – this really depends on the level of corruption to the file.

In general, to understand what the First Aid section is doing, one needs to recall that .olm archives are essentially .zip files with a different file extension. If an .olm file is broken (i.e. error 21 gets reported by Olminator when you try to select your archive as the input file) and cannot be properly loaded into Olminator, in all cases that have been reported and analyzed so far, this was due to the archive file been corrupted "physically", i.e. it was cut off (so only partially stored) or had parts of the file's data and content rendered unusable due to copy errors apparently. This may not only affect individual items in the file, but also structural information that is used to keep the .zip file consistent in itself.

Depending on the damage that has happened to the archive (= the .olm file = the .zip file) we can try to salvage its still valid content, unpack it and re-pack it into a fresh, consistent archive file containing the salvaged items. Since any .olm archive is a .zip file, we can use, for example, the zip command-line tools to do so – this of course requires you to be familiar with using the command-line tools on your Mac and/or get yourself familiar with the options of zip or unzip command tools. Alternatively, Olminator can do this for you 'under the hood'. And this is actually what First Aid does. Not more. Not less

First Aid offers 4 measures to apply to a corrupt .olm archive:

1. Validate the archive: The first step should always be to check the validity of the archive itself, its structure and its content; this step will give us more details about this and clarity on why the loading into Olminator fails
2. Unpack the archive into a folder structure: A typical next step is to try to unpack the archive's content into a folder and file structure on a sufficiently large storage device like your Mac's harddrive. If this succeeds, there's a good chance not all is lost. If this step fails miserably, you can try to apply the fourth measure: salvage the archive's content by getting around structural damage of the archive using some heuristical methods.
3. Pack a fresh archive from a folder structure: If you have been able to unpack a broken archive – or the salvaged copy of one – into a folder structure as described before, you can then reconstruct a valid archive from it. If this step succeeds, it's highly likely that the resulting archive file will digestable by Olminator – even though there cannot be any guarantee as to how much of the original content has been rescued. The log files written during each of these steps gives you detailed insights though whether items were dropped, couldn't be removed or were properly unpacked or re-packed.
4. Salvage an archive's content: You may be able to directly unpack even a broken archive, but if the damage is more severe, you may not be successful with this head-on approach. In such cases try first 'salvaging the archive' which apply some smarter approach to reconstruct structural information in the archive or work around missing pieces of information. You can then try to unpack the resulting archive – most likely with more success. Salvaging of the archive comes in two modes that you can select: the normal salvaging and the 'try harder' salvaging. Please try 'normal' first and see whether this already salvages most of the archive. Only if this is not successful, enable the 'try harder' mode, as this will on the one hand use a more sophisticated approach to rescue items from the archive even if there are fundamental structural issues with the archive, but on the other hand comes at the risk of losing more items in the process. If you try the other way around, there's the chance to may unnecessarily let go of items that could otherwise have been salvaged without problems.

When applying the various measures to the archive, Olminator needs to be provided with a few hints about to where to write or from where to read depending on the measure. Here's the relevant spots and their purpose:

• Working folder: This is the folder into which Olminator will write unpacked archive's folder structure, or a salvaged copy of the respective archive. If undefinied initially, Olminator will use Olminator's regular output folder (if it has been set earlier already). Please make sure that this folder resides on a device, e.g. your Mac's built-in harddrive, or a reliable external diskdrive (or USB stick if nothing else is available) that offers sufficient space to store all the items. If you run through the entire set of measures, you'll need at least 2 times the size of the archive as free space (once for unpacking all items, twice for then repacking the unpacked content into a new archive). If you run into free disk space issues between steps, the working folder selector allows you to potentially switch between several devices, e.g. unpack to an external drive, then pack to your Mac's harddrive or another USB stick. In most cases this won't be necessary, I hope, but I've seen some users juggling with 150 GB archives where it became an issue. Olminator, by the way, will display the free space available on the respective device for your reference.
• Working [archive] file: This is the file to which you want to apply the respective measure. Again, by default, if not defined initially, this will point to your input olm file selected in previous steps, so you can start validating or unpacking right away. But during the process, you may have to switch to a salvaged copy of the archive for it to be unpacked in the next steps – this is where you switch those archive files to handle. Olminator, btw, will allow you to do the switching to the output of a measure as the new input for the next logical measure by just clicking the respective button that will show up once successfully processed a measure.
• Folder to pack: If you want to pack a folder containing archive content back into an .olm file, the 'Folder to Pack' is where you specify the folder to pack. Also this will be prefilled based on prior steps by default, but you may override this manually at any time.

Other than that – and as mentioned before – you'll be guided with detailed on-screen instructions during the process. And it is always advisable to reach out to Olminator's support guy for help if you get stuck.

Log information is written to the file Olminator.log in the working folder for all measures applied.

Checking or changing default conversion settings

Need to deviate from the defaults? Olminator comes with reasonable default settings for the conversion process "out of the box". Those settings ensure that an import into your favorite alternative applications for emails, calendars and contacts on Mac finish successfully. But you might run into unforeseen issues (like an email app crashing suddenly when importing a massive (50GB) .mbox file, or a Calendar application complaining about calendar events carrying too many or too big attachment files with them).

In such cases, try changing default settings for size or number limits in the Settings section (see Configuring Olminator for details).

Also, when you're feeling lucky, you may try to remove some of the limits and check whether your target application can still successfully import the conversion results. There's no reason to strip off attachments or meeting participants from Calendar events if there's not a hard limitation in place to do so.

Missing something in the conversion output? The opposite case may also be the case: You've once changed the default settings (e.g. removed Calendar items from the conversion process by unchecking the respective option, or limited the number of attachments to zero to strip all attachments from Calendar items) – and potentially forgot you've done so – and now when you run another conversion, some unexpected effects seem to happen (like there's no Calendars showing up in the output).

Hence – when something seems to be missing or broke – re-check whether there's any changes to the default settings that might be the reason. You can press the Reset Settings button at the bottom of the Settings section to get back to the default ("factory") configuration settings.

When things get dire during the conversion – enabling debug mode for Olminator

I knew it would happen one day: someone would run into a sudden and unexplicible crash of Olminator during a conversion run or one or more severe errors get reported because Olminator stumbled over some unforeseen content. Apologies for the inconvenience. Let's see what we can do about this unfortunate situation:

We may try to locate the content that is causing the issue and then try manually creating a workaround by removing or fixing this particular content (see above). You may also contact the author of Olminator (that's me) at mailto:'Björn Goerke' to share the issue and see if there's a bug fix available or possible. This may require some of your kind help by providing information about what exactly caused the crash or error.

To make all of this easier, Olminator has a built in feature called "debug mode", in which it will log more detailed information about the content processed.

With debug mode enabled, Olminator will ...

• Log information about the .olm files structure and content in a file called Olminator_debug_olm.log in the Olminator output folder.
• Log the current item's content as seen by Olminator in a file called Olminator_debug_source.log in the Olminator output folder.
• Log the current item's logical structure as parsed by Olminator in a file called Olminator_debug_xml.log in the Olminator output folder.
• Log any internal error messages into the Olminator_errors.log file in the Olminator output folder (it always does this independent of debug mode)
• With debug mode enabled Olminator will stop the conversion run as soon as the first error (which will be logged in the Olminator_errors.log file) occurs. Normally, without debug mode, Olminator will continue converting the remaining items in the .olm file and simply drop the item causing an error – this is to make Olminator as robust as possible against erroneous or non-convertible content and convert as much of your .olm archive's content as possible.

Hence, with debug mode switched on, there's an increased chance that you will find the exact content causing an issue in the log files in the Olminator output folder for additional analysis or manual fixing. You may also decide to share these log files with the author of Olminator for deeper analysis and to fix potential bugs in Olminator – only after you've removed or "redacted" any personal/confidential information in there, of course.

How to switch on debug mode?

• You can enable debug mode in Olminator, by selecting "Help/Debug Mode" from Olminator's menu or by holding the Shift-key pressed when opening the Settings section.

• At the bottom of the Settings section, you will then find a little red “bug” icon. Check the “Enable debug mode” checkbox if it is not already selected. Furthermore, the following options are available:
• Collate debug dump files: If you can't narrow down an issue to a single item, you may want to collect simply all debug dump files for analysis (or sharing with the author of Olminator after redacting the content for further analysis). Simply switch on the "Collate debug dump files" option to do so. Be aware though that this may create very large dump files; you may consider limiting the set of items processed by e.g. excluding certain folders from the .olm file or certain types (lile Calendar items if debugging contact items) to keep log files limited in size.

• Redacting log content: If you want the real content (email addresses, subject lines, email or calendar event content, etc.) be redacted from the log file output, check the “Redact conversion content” checkbox. Olminator will automatically redact all relevant information from the log file, so that it can be safely shared for debugging / analysis purposes without revealing any personal data (Olminator replaces all characters with “A” or “a” and digits with “0”, so that the technical structure of the item remains in place, but the actual content becomes meaningless).
• Stop processing immediately after item with sequence number [number]: When processing items in an .olm archive during the conversion, Olminator assigns a unique sequence number to each of the items. It will also write this sequence number into the error log. You can have Olminator stop immediately after processing the item with the sequence number you specify here by checking this option and providing the respective sequence number as "break point". Please see below sections on typical "debugging" scenarios on how to make use of this feature.
• Stop processing immediately after item containing character sequence [text]: By checking this option and providing a search text, you can have Olminator stop converting immediately after processing an item that contains the given character sequence. Please see below sections on typical "debugging" scenarios on how to make use of this feature.

Caveat: Since debug mode comes at the performance overhead of creating the beforementioned log files for each item processed, make sure to switch debug mode off again when you're done with your analysis.

Hint: As long as the "Enable debug mode" checkbox is checked, Olminator will show the "Debug Mode" section in the Settings section, allowing you to easily switch debug mode off again. Once switched off again and once you restart Olminator, the "Debug Mode" section will be hidden again from the Settings section. To show the section again, simply restart Olminator or open the Settings section with the Shift-key pressed as before.

Typical "debugging" scenarios

A. Olminator simply crashes or stops with a “fatal” error message:

1. Make sure you haven’t changed any other configuration settings that might influence what content gets processed by Olminator and re-run the conversion that led to the prior crash or fatal error message.
2. When Olminator crashes (again) or stops (again) with the fatal error message, the Olminator output folder will contain log files with the last item’s raw data (from the .olm file) and the parsed version of it that Olminator was going to process internally.
3. Chances are very high that the logged item’s data was instrumental in crashing Olminator or throwing the fatal error. The data needs further analysis to reproduce the error and hopefully find a fix.

B. The olminator_errors.log contains error or warning messages for an item and lists a “sequence no.” for this item:

1. After you’ve enabled debug mode as described above, re-run the conversation with the same settings! This is important to get to unique, repeatable sequence numbers for all the items during the conversion. Check the Olminator_errors.log file again and take note of the sequence number that is now (!) attached to the respective item that is causing you the headache.
2. Now enable (check) the “Stop processing immediately after item with sequence number” and enter the exact same sequence number.
3. Re-run the conversion once more with the same input data and configuration settings, so that Olminator processes the items in the same sequence as before.
4. When Olminator has processed the .olm file item with the sequence number you specified, it will stop the conversion process.
5. The log files in the Olminator output directory will now contain the data for this exact item. It can now be analysed further.

C. Olminator doesn’t show any weird behaviour, but the output does not properly reflect what you’d expect

1. Most of the time, there will hopefully be no crash to debug, or a warning or error message in the log file that needs in-depth debugging analysis (at least Olminator has detected something bogus apparently and hopefully provided some content information already). Nevertheless, sometimes things may just not work as designed.
2. If you need to isolate information about a single item that is causing some headache, but there is no sequence number available to refer to, you may isolate the item either by using the email or calendar filtering capability of Olminator to isolate the conversation for this one item.
3. To do so, in the Settings section, enable “Convert Emails matching these Filter Criteria” or “Convert Calendars matching these Filter Criteria” and provide filter criteria that are precise and narrow enough so that only the one item you want to analyse passes the filter: It may be the Subject line of an email in combination with the email address of the sender, or it may be a calendar event with a certain title (and/or a specific date/time). You may have to tweak your filter criteria a bit to make sure you hit your one item exactly.
4. Alternatively, you can specify a character sequence as search pattern in the Debugging section of the Settings section and trigger a break-point during the conversion as soon as the search pattern is detected anywhere in an items .olm source data. This allows to quickly narrow down on a specific item. Beware though, that the search pattern is applied to the entire source data, not just specific fields.
5. Once you’ve re-run the conversion and the one item has been processed, the Olminator output folder will again contain the items information for debugging analysis.

Log files for analysis

The following two log files are of particular interest for the analysis of a single .olm file item. Both are found in the Olminator output folder after an conversion run with “Enable Debug Mode” selected:

1. Olminator_debug_source.log: contains the (redacted, if this option was enabled) raw data as read by Olminator from your input .olm file
2. Olminator_debug_xml.log: contains the (redacted, if this option was enabled) parsed/processed representation of the items data as used by Olminator for the conversion

In some cases, when you can't narrow an issue down to a single conversion item, it may be necessary to collect simply all xml and source dumps from all conversion items. Depending on the number of items, though, this may add up to a lot of data. So use this with care. You can switch on collation of the Olminator_debug_source.log file and the Olminator_debug_xml.log file in the debug preferences section (option 'Collate debug dump files'). You may exlude certain types of items (e.g. emails) if you're debugging contact items, of course, and this may help keeping log file sizes in bounds.

Hint: Before sharing those log files, you can take a look at their content: simply double-click the files in Finder to open them in log file viewer or any other application that can display plain text files. If redacting was enabled in Olminator, all sensitive information has already been rendered useless by Olminator automatically. You may also edit the content of the file by yourself before sharing with someone for analysis to render e.g. individual email addresses unrecognizable, but please take caution not to remove any “structural” information that may create other issues with the source!

Crashes

In the hopefully extremely unlikely case – but we've all experienced those before unfortunately – of a crash of Olminator, please get in touch with the author (see below). Thanks a lot and apologies for the inconvenience caused!

As of version 1.72, Olminator will automatically detect if a previous conversion run has terminated unexpectedly (i.e. crashed Olminator) once you re-start Olminator again: It will then present you with some help on how to quickly re-run the conversion with 'debug mode' and 'redacting' of log files switched on, so that hopefully some additional information relating to the crash can be logged and shared with me for support purposes afterwards. Simply follow the on-screen guidance or reach out to me in case of concerns/questions, once you experience a crash and the red screen appears when re-starting Olminator.

Preparing/fixing the .olm input file manually

For those amongst you not faint at heart and assuming they know what they're doing, there's the option to manually edit an exported .olm file and to try manually fixing any issues experienced while converting or importing the resulting output. You can e.g. remove complete accounts, individual messages or calendars, or you can edit message content or calendar events, to give you some examples.

In order to do so, follow those general steps with the .olm file you've exported:

1. Make a copy of your .olm export file: Only edit the copy, so you can easily revert any changes you made by mistake or "improvements" that turned out to backfire.
2. Unzip the .olm file copy: Right-click your .olm file copy in Finder and select the "Open with..."/"Archive Utility" menu item to have it "unzipped" into a folder structure of the same name in the same directory as the .olm file. Luckily, .olm files are "just" zipped files/folders.

   Hint: You may also use Olminator's built-in First Aid functionality to unpack archives. Please see above for more details.

3. Try locating the trouble-some content: This may be the difficult part in the exercise. When running an unsuccessful conversion in Olminator, look at the error log file (created as "olminator_errors.log" in the output folder you specify) for possible hints indicating what may have caused the trouble. You may also look at the detailed conversion progress sheet during the conversion to see in which folder or file the conversion ran into an issue (e.g. like a crash of Olminator). If you're desperate enough to try fixing an issue manually, you may also want to enable Olminator's debug mode (see below for more details), which may help you narrowing down things significantly.
4. Try removing the trouble-some content: Once you have a decent guess at what might cause the issue, remove troublesome content (e.g. remove the single message.xml file or remove a complete sub-structure of folders and files). By re-running the conversion, see if you've been successful with your removal. If so, you may try to narrow in on the issue further. Repeat those steps until you're happy – or until you give up.
5. Try editing the trouble-some content: This is for the pros! Once you know which file is causing the issue, you may open it in an editor (.olm content is stored in XML files) and manually "iron the issue out". Again, don't do this light-heartedly – you may easily cause more trouble than help if you break the structure of the XML file or its content.
6. Zip up your edited content: When you're done with your changes, right-click the root folder created by the unzipping of your copied .olm file in Finder and select "Compress" in the context menu that pops up. This will "zip" the extracted folder structure back into a file which is a valid .olm file. You may rename this .zip file back into a .olm extended filename, but Olminator will also accept the .zip file as input file.

   Hint: You may also use Olminator's built-in First Aid functionality to re-pack archives from folders. Please see above for more details.

7. Convert the edited .olm file again: and keep your fingers crossed that you've fixed things properly. With any of this, you're on your own. Godspeed.

Caveat: Fiddling around with the content of the .olm file or the XML files contained in there should be done with great caution. Make a copy of your .olm export file and only modify the copy ideally, so that you can quickly revert any changes you applied.

If all else fails...

If you've tried all of the above options without success, there's four possible options left:

1. Contact the author of Olminator via email at mailto:'Björn Goerke' and provide an understandable description of the problem you're experiencing. Perhaps there's a bug in Olminator that can be fixed or a workaround that may help. No guarantee. No promises. But we'll try.
2. Start a rant in Olminator's review section on the Apple App Store: Well, you can of course do this, but the author (me again) usually prefers to be contacted via email first to have a chance to take a look at your issue. If you then don't get the support you've been looking for, you can still go for a rant on the App Store.
3. Try a different conversion tool that may work in your situation. Sorry for disappointing with Olminator.
4. All of the above three options ... in this or any other order.

Please be assured, that it wasn't the authors intention to annoy you with any Olminator issues, steal your time trying to figure out why things don't live up to your expectations or keep you from spending money for a "professional" conversion tool. Just trying to be helpful instead.