Who doesn't wish to keep a record of a beautiful sunset that particularly impressed us in childhood, our first kiss or, for that matter, an important conversation with the boss that took place a few months back? One of our shortcomings is a constant st ruggle to remember. How difficult it can be sometimes to recall the name of the person you need to meet in an hour, the important phone number your secretary just read you on the phone, or that very important item your wife told you not to forget to bring home this evening. But what if you had a magical device that would allow you to rewind reality and see exactly what happened? A few years back I began thinking about what could bring this dream closer to reality.

The features required to make such a device possible can be divided into three elements: first there is the hardware, which, as we shall soon see, actually presents the smaller set of problems. The software is saddled with a far more complicated series of hurdles, and finally there are social and perhaps legal issues that would inevi tably accompany the large-scale implementation of such a technology.

In terms of hardware, we would require one or two miniature high definition wide-angle cameras, which could take high definition (HD) video, that could be attached to our glasses or sunglasses. (They might look geeky at first but in a few years it might be possible to actually integrate them into the glasses similarly to MP3 and Spy eyewear glasses.) Microphones with the ability to locate the direction from which the sound originates would also be required. Other hardware add-ons could be eye-tracking equipment, which would register what you were looking at in any given moment (using equipment similar to the Stanford Poynter Project, for example), and a GPS receiver, which would inform you of where you where at any given time. Since most of our day is often spent indoors and current generation GPS receivers don't work well indoors, new solutions would have to be found for this problem (such as inc reased sensitivity GPS receivers). 

u-blox SuperSense Indoor GPS chip

But hardware issues are slight in comparison to the problems on the software end. Let's say that you have recorded a full month of your life and would like to find what you said to a friend in a meeting though you don't recall when or even where it took place. You could of course start running the 480 hour video and look, but even fast forwarding, it probably would not be a very good use of your time. So how can you search a 1.7 Terabyte video for a few specific seconds? Well, currently there is no simple solution to this problem but there are a few technologies that if integrated might help. Voice recognition technology has been evolving for many years and recently has attained a pretty acceptable level. However it usually requires training and relatively ideal conditions (a microphone in front of your face and a fairly noiseless environment). For our purpose, the voice recognition software would need to be able to recognize what different people are saying without training and in less than ideal conditions (outdoors, in noisy crowded places, etc.). The software would also have to perform another trick, that is, recognize voices without long training sessions. Facial recognition, could be implemented to enhance this. By combining the information from the voice recognition and facial recognition softwares, it would be possible to analyze exactly who we were talking to and what was said. The eye tracking movement hardware could also provide important information in this respect. GPS could add another level by stating where the conversation took place and even what we were looking at at the time.

Even if all these hardware and software hurdles could be overcome to create a device capable of real time HD video recording and data analysis, socio-psychological and legal problems would inevitably crop up in our society in which privacy is a sacred value. Not to mention what what would happen if some unauthorized person somehow gained access to your life's database!

Actually implementing the idea described above is still at least 10-20 years away, but, as described below, partial implementation is already underway and might bring actual benefits to all of us in the near future.

The headline of Vannevar Bush's article

Nearly half a century later, Bush's conception of a device that can store many aspects of our lives is starting to become a genuine possibility due to several recent advancements in computer technology. Gordon Bell, a senior researcher at Microsoft's Bay Area Research Center, began thinking about the memex in the late 1990's. In 1999, Microsoft began MyLifeBits, an experimental project whose aim was to fulfill Bush's concept by storing various aspects of Bell's life on a searchable database. By scanning his books and old pictures, recording his phone conversations, meetings, and favorite radio stations, collecting his emails, viewed web pages, and IM conversations, and practically almost every other recordable aspect of Bell's life, Bush's memex dream started to come into fruition.  

Gordon Bell with Deja View cams

MyLifeBits collection dialog

Interview with Gordon Bell

 To learn more about the MyLifeBits project, TFOT interviewed Microsoft's Gordon Bell:

Q: When did you begin the MyLifeBits project and what was the motivation to start it?

A: The project started around the end of the last decade. The initial motivation was to scan my content and to get rid of all the paper in my life.

10/98: Bell to Raj Reddy, “It’s fine to scan and put my books on line. Don’t worry about copyright stuff. Microsoft has lots of lawyers.”
1/99: 1'st scans with “Digital Sender”, 4/99 “We” start to scan.
1/01: CACM “A Personal Digital Store” (16 GB; +2/yr). Began search for search engines, especially for email.
10/01: Jim & Roger join the project to explore the issue of using SQL as the platform.
11/02: New Scientist Article.
2/03: Techfest MyLifeBits demo.
10/03: SenseCam addition.                          
12/04: CARPE workshop founded by Gemmell.
 2/05: Epiphany! Memex is a database & personal TP system.
 2/05: NSF PIM Workshop@ UW.

Q: What were the different stages of the project and where does it stand now?

A: Here is a quick list of the main stages of the project:
1. Scan, search for a good search engine.
2. Start a serious project to build Memex using SQL.
3. Add all the desktop content we could: web pages, key strokes, mouse clicks, and phone calls.
4. Add real time video through the SenseCam.
5. Start of a Memex Research program with universities.
We are currently working on adding real time health and wellness data.  

A: We use various components from various tools. The initial was the text search in SQL. Today we use some of the search engines as part of the MSN Desktop search.

A: We are working with WinFS to get the functionality for a file/database provides and requires.

Q: Finally, as for your plans for the future, how do you see the project in 5 or even 10 years time?

A: Something beyond our current imagination, but that we treat as a wide range of use from repository for everything to a memory surrogate to something that becomes an assistant to be pro-active.