Business Information Archiving: The Past, The Present, and What Could Be Next

Archiving – the origins & the future

Today, enterprise information archiving is the norm and a best practice for organizations of all sizes, public and private, in and outside regulated industries. But how did it all begin? As an archiving company, we were eager to explore how archiving has evolved from cuneiform tablets and papyrus to AI-assisted cloud archiving software. To understand the present and prepare for the future, we first need to take a journey into the past, all the way to the first archives.

First archives: how it all began…

Did you know that the first archives are 5000+ years old? The origin of the word archiving can be traced back to ancient Greek ἄρχω, which means to rule or govern. First archived clay tablets and papyrus texts date back to 3000 BC and have been fundamental in preserving knowledge of ancient alphabets, languages, cultures and politics. In the 7th century BC in modern-day Iraq, the world’s first known library housed 30K clay tablets archived and organized based on topic.

From monasteries to the court of Queen Elizabeth

Let’s fast forward to the Middle Ages and the Renaissance – Ancient Greece gave Western Europe a bunch of amazing things: philosophy, democracy and the Olympic Games. But it gave us archiving too. In medieval times, monks acted as archivists, but the first real archive in Europe was established by Queen Elizabeth I in 1578.

Shelves, files and cardboard boxes

Okay, let’s fast forward a few centuries… to the 1800s and 1900s. If your first thought at the mention of “archiving” is a dusty basement packed with museum artefacts and cardboard boxes carefully organized on floor to ceiling shelves, we feel you! Even today, the biggest archivists are museums, libraries and universities. Cambridge University archives now occupy more than 3000 meters of shelving. The largest library in the world is the Library of Congress with 170 million books.

The birth of business archives

And now, let’s take a look at business archiving BE (as in Before Enron). As Wall Street emerged as the second US capital in the 1950s and 1960s, first corporate archives were born. The pioneers were companies such as Ford, Proctor and Gamble, Bank of America and Coca-Cola. The purpose of the archives was to help companies avoid past mistakes and differentiate them from competitors.

Record management

In the 1960s and 70s, historical business archiving and record management started to be viewed as separate concepts. While historical archiving was focused on the preservation of company history, brand management and marketing, record management was about handling corporate records throughout the lifecycle and treating them as evidence of a company’s activities. Did you know there are 14,000+ laws, standards and regulations that dictate how long businesses must keep records?

One small message for a man…

In 1971, Ray Tomlinson sent the world’s first email – a simple test message that read “QWERTYUIOP or something similar”. Computers were introduced to business and the office world would never be the same again. As the rate of document production increased, so did the legislation forcing companies to retain records. Electronic storage first appeared. Today, a typical corporate user sends and receives around 120 email messages every day.

The end of an era

In the 1990s and 2000s, business email grew exponentially. As the world became more digital, electronic records largely replaced paper and analog records. These were the times when many of us last rewound a favorite tape by putting a pen through a reel hole. And you, Walkman, we miss you dearly.

The scandal that changed everything

In 2001, during the downfall of Enron, a top accounting firm Arthur Andersen illegally destroyed nearly 30,000 emails and files detailing its activities as an accessory to fraud. This event prompted the passing of Sarbanes-Oxley Act – the legislation that forced public companies to diligently maintain records, including digital communication for 5 years. But how do you keep your entire email history? Somebody had to work that out.

The first email archiving solution

Early noughts saw the emergence of the first software email archiving solutions made by large providers such as IBM, EMC and Fortiva. These email archiving pioneers relied on log shipping to extract data from mail platforms. The process was slow and often required extensive (and boring) configuration of multiple components. Something had to change.

The birth of the box

As compliance laws got stricter and kept multiplying, companies became more serious about email archiving. Before Enron, only 1% of businesses archived email. Suddenly, the figure jumped to 50%.

In 2004, Jatheon (that’s us!) made the world’s first network email archiving appliance. Invented by IBM, journaling replaced log shipping and stepped up as the process that was simple and that made archiving much easier to configure. Microsoft released a journaling feature in service packs for Exchange 2000 and 2003.

The age of the archiving appliance

As earlier software solutions proved too complex and difficult to install and maintain, newer, simpler and secure on-premise archiving solutions became the norm. Email archiving became an established practice for public companies of all sizes, and vendors like Jatheon, Barracuda, Commvault, ArcMail and Datacove led the way. 85% of companies archived email using an on-premise setup. But things were about to change again…

Something new on the horizon

Despite their numerous advantages, on-premise archiving solutions had one major drawback – the limited amount of storage. In 2008 and 2009, Amazon AWS cloud services that offer bottomless storage suddenly seemed like an exciting alternative. IT managers, however, were still wary of cloud technology.

The times, they are a-changing

In 2010, there were some major leaps in search technologies. Email search and retrieval became much faster thanks to search engines such as Elasticsearch. Jatheon developed new PnC appliances that had revolutionary search capabilities, never before seen in the archiving industry.

While appliances still ruled the email archiving world, the cloud was gaining momentum. Email deduplication was introduced, which meant more available storage for your data.

The shift

First virtual and hybrid email archiving solutions appeared on the market. Archiving companies started focusing on smaller market segments and verticals. Slowly but surely, cloud technology revolutionized email archiving.

In spite of that, many IT managers still had their doubts regarding cloud security, and 73% of all email archiving users still relied on on-premise solutions. In 2011, Microsoft launched Office 365, and companies were drawn to the subscription-based pricing model.

The data requested in legal cases, investigations and FOIA requests started to include electronically stored information, primarily email. This meant an enhanced feature set for the archiving solutions to support the integrity, easy management and production of digital evidence and metadata.

The cloud revolution

With the rise of the cloud (2013-2016), many predicted the death of the archiving appliance. Simultaneously, new ways of communicating threatened email itself. Social media and instant messaging platforms took the world by storm, and business messaging apps followed suit. Apps like Slack and messaging platforms integrated in social media became an integral part of the modern workplace. Even federal government and national agencies embraced social media to inform or alert the public of any threats and increase engagement and transparency.

Regulations changed as well, so social media and mobile communications needed to be archived too, at least in the regulated industries. Companies like Netbox Blue, ArchiveSocial, PageFreezer and TeleMessage stepped on the market and integrated with ever-developing email archiving solutions.

Did you know that 90% of all the data in the world has been generated over the last two years?

Archive everything

Around 2017, all major vendors started offering archiving support for multiple types of structured and unstructured data, both on-premise and using a hosted solution. Fewer and fewer companies were hesitant about storing enterprise data on the cloud, as security improved considerably. Bad news? Not everyone excelled at supporting cloud solutions, and thanks to archives volumes increasing exponentially, it was often impossible to predict costs.

Things get messy

Large enterprises still felt the safest with their on-premise setups, but smaller companies could not support nor afford hardware solutions. Time took its toll on legacy archivers, with many former industry leaders cancelling their maintenance and support services or merging with the big players. Many SMEs wanted cost-effective cloud alternatives that are easy to use and maintain. There were many vendors to choose from, but most solutions failed to meet expectations…

The pandemic and WFH

In early 2020, as Covid-19 spread around the globe, strict lockdowns were introduced. This included working from home, wherever possible. This has led to one of the most profound changes in workplace communication, creating even more complexity in all areas of compliance and records management.

As the number of emails sent daily reached 306 billion, new platforms like Zoom and Teams emerged or became more prominent for video conferencing and collaboration. Where does that leave enterprise information archiving?

Where present and future collide

It’s late 2021 and the archiving landscape seems more complex than ever.

On the one hand, key data privacy laws like the GDPR or CCPA were passed, giving individuals more control over how their personal data is processed, handled and used by governments and companies.

On the other hand, data retention laws are getting stricter and include a multitude of data types. From being the only (or at least main) business communication channel, email is now one of many, together with social media, text messages, instant messaging apps, collaboration platforms, audio and video. Critical business data also resides in CRM platforms and across company-issued and BYOD devices.

As data types and volumes continue to multiply, organizations need reliable, fast solutions with powerful search and ediscovery capabilities and attractive additional features like automatic AI-powered redaction of PII and sensitive data, various API integrations, multi-layer security and advanced analytics.

So can you imagine a cloud archiving solution with the best user interface you’ve ever seen? A system that’s so easy to use that your employees won’t need a single day of training? Search and export that will never let you down?

We could. So we made it.

About the Author

Bojana Krstic

Bojana Krstic is the Head of Content and SEO at Jatheon and an experienced writer on topics like data archiving, ediscovery, and compliance. When AFK, you’ll find her hiking, discovering new music, or road-tripping.