Thinking while you see and later seeing all that you thought, are the twin ideals for those who travel. Well, that’s the ideal way, but you know what really happens. You use your smartphone to click the image but too many of your thought bursts do just that–they burst and fly.

All those quirky jokes, those observations, those data bits, could be rolled into a perfect travel story that never or barely gets told. You had your bunch of photos and videos ready to go, but memory dims what you thought during that special time.

A PhD student in Singapore can attest to that. Hyeongcheol Kim went off to a computer conference in Montreal and was excited to make his own side journey to Quebec City, to take in its sights and vibe. His mind was on the conference after that, and when he tried to write about everything he wanted in his travel blog he realized his Quebec recall wasn’t good.

He wondered if there was an easy way to capture his thoughts and feelings in words with images in real-time, rather than struggling with half-baked memories after returning home. Enter a challenge in “experience writing,” where, with the right solution, all those clever little thoughts burst don’t go away because they are captured in the moment, and without always whipping out a notepad and weighing down the moment. He talked about the challenge with Shengdong Zhao, head of the NUS Human-Computer Interaction lab; joining in were Can Liu and Kotaro Hara. Result: LiveSnippets, a voice-based multimedia journaling app designed to run on Android.

They discussed their work in a paper prepared as a conference presentation.

“LiveSnippets: Voice-based Live Authoring of Multimedia Articles about Experiences” was prepared for MobileHCI ’20: the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services.

Cognitive demands

They reminded readers why documenting experiences can be difficult.

“Documenting experiences digitally in blogs and journals is a common activity that allows people to socially connect with others by sharing their experiences (e.g., travelogue). However, documenting such experiences can be time-consuming and cognitively demanding as it is typically done OUT-OF-CONTEXT (after the actual experience).”

How LiveSnippets works: You dictate thoughts about relevant photos and videos. You save them as a stack of editable snippets. The stack can be reordered. In other words, you would have a way to put the travel experience back into context. Another plus is that reluctant writers may have confidence, once they see their thoughts organized, to show the world and publish.

Recipes and product reviews

Kim said in the NUS news story that a user can use the app to generate an HTML file of the travel blog or social media post complete with pictures and captions, along with other details. You just publish with a click.

The NUS site said, “A user — say someone seeing the Eiffel Tower for the first time — pulls out his smartphone and starts to take pictures or videos. At the same time, he begins to describe what he sees, how he feels, and any other thoughts that comes to mind. The app captures all of this, along with contextual information such as time, date, and location, with a single press of a button. It also transcribes the user’s narration into text.”

Apart from travel, LiveSnippets can be used for other types of experiential writing. They tried out LiveSnippets using three scenarios: travel writing, creating recipes, and reviewing products.

How they tested: A small group with different writing backgrounds (expert and novice) were test participants. Each had to generate three snippets and use them to write an article based on one of those three scenarios. Results: Kim said the team was satisfied with results as they showed the feasibility of using LiveSnippets as an alternative way of writing articles.

What’s next?

The team is posing a new question: What if they can integrate LiveSnippets into smart glasses? While the app goes beyond just scribbling down words into a spiral notebook, they think that as a smart glasses tool, it could even beat having to use a mobile phone in the special moment.