Written by Deborah Richman, Consultant, Zions Bank
Deborah Richman, Consultant, Zions Bank
“We just had our 10 petabyte party,” declared Brewster Kahle, to Information Industry Summit attendees this week. Universal access to all knowledge may sound like a pipe dream, yet Kahle and his Internet Archive team have been doggedly pursuing this goal and using up petabytes to collect, digitize and share content.
The Internet Archive is best known for creating the de-facto web historical repository. Since 1996, Kahle’s team has visited “every page on every web site, every two weeks.” There are more than 240 billion URLs in the archive today. For better or worse, anyone may access them at the WayBack Machine.
Fortunately web tools and sources have improved, and Kahle also relies on others to help. At this point, there are some 1,700 curated collections from 200 places included in the archives. “Personal digital archives are next,” says Kahle. “But our stuff is all over the place. And things are gone.”
It’s more than websites
The archive.org team, comprised of 150 staffers, has been making books, audio, video and TV news available at a dizzying rate. Kahle reported on archival progress for SIIA members:
- Books: 3mm e-books, 500k for blind and 300k modern e-books.
- Audio: 1mm items in 100 collections, including 100k concerts from 5k bands.
- Video: 2 – 3k movies, plus industrial, educational, other specialty films.
- TV news: 20 channels collected since 2000, and TV news for three years.
The Internet Archive sidesteps copyright issues by behaving like a library consortium. Libraries and individuals are free to make their multimedia collections available online. Then patrons, aka site visitors, are able to view unrestricted materials or check out others from the holdings.
No modern-day industry titans, like Andrew Carnegie, have come forth and made this digital access dream come true. Instead, a non-profit organization filled with dreamers and technologists have been knocking down access barriers to digitized content for two decades. It’s pretty sweet.
____________________________________________________________________________
Debby Richman spent her formative years at D&B, leading the reference business from print to online and web offerings. She has since held digital leadership roles at Overstock, About.com, Looksmart, Starz, Collarity and Zions Bank.
