This is fantastic! Thanks for sharing the details so honestly and openly. This page is a great resource and I've passed it along to others.

(I realize this reads like one of those giddy spam comments but I won't link to any services I'm selling here - this is a real comment :)

I'm subscribed to your feed now...

Hi Jan,

What a great project! We have been doing some similar work in this vein by actually setting up indvidual blogs for study participants, and having them post to their own blogs - sometimes directly from their phones via SMS. I second much of what you point out as strengths and weaknesses of this approach. We did a project earlier this summer for the Consumer Electronics Association where we had 33 teens blogging a week in their lives with their phones. As some of the participants had camera phones - we captured some amazing images, and we got a LOT of data. I would be very interested in learning more about the volume of data you generated for this project and how you handled analyzing it.

Best Regards,

Steve August

Hei Steve,

I'll try and cover the volume thing in a later post - whilst it?s not particularly complicated its will be time consuming. If you have SMS and camera phone photos LifeBlog is a good off the shelf product for sorting these into a timeline but alas does not yet have a print function.

I tend to produce very visual reports simply because it?s much easier to grab people's attention to communicate serious, informed, valid research results. These do not equate to boring. I always assume that my report has to compete with 500+ other emails for the reader's time, so what will interest the reader? Apart from everything else diary photos are real and the photographic results can be quite edgy - very much communicating a sense of 'being there'. There is also an element of voyeurism from the report reader's perspective in that ultimately everyone is interested in sneak peeks inside the lives of ordinary people. There are also shots and spaces you'd never get to see as a researcher - for example the seeing the knees of someone sitting on a toilet reaching to grab a magazine - for a study on where people consume media.

When it comes to capturing/reporting/posting data via mobile phones I've seen a number of sophisticated systems including an excellent one by Nine Yards Design of Helsinki (http://www.nyd.fi/). Many of these systems are certainly 'neat' (in the overly innocent Am-er-i-can tourist with big camera looking at Buckingham Palace sense of the word) but when it comes down to it I have yet to have a compelling reason to require participants to adopt the limitations of the phone user interface to record data.

I can see it being useful provide corrective feedback 'please try using the flash' or updates on themes to record 'take photos of products you want to buy' but writing frankly I've never done studies where time was such a critical factor. Another aspect of it is that the design research team simply is too busy to monitor data as it arrives in dribs and drabs.

As for providing participants with a mobile phone to record data I find this too much of a jolt for participants - except for longer term, 3 month+ studies.

Do you have any publicly accessible material from your studies available online?

Hi Jan,

Thanks for your thoughtful response. Regarding capturing data for our teen cell phone study, we built our own tool that makes it easy for us to set up blogs for each study participant. These blogs can accept posts from multiple email adresses - so a person can post via their computer or their phone. During the study, most of the kids posted to the blogs from their computers, but a few enterprising kids did post directly from their phones. We gave them shortcut codes for recording usage information via text messages. Once the info is posted to the blogs, our system can then spit out a clean HTML transcript. I'd be happy to tell you more about our tool if you are interested.

This study was good in that the kids already had their phones and many had good abilities with them. I wouldn't send out a cell phone per se for a project, but I am thinking more about sending Sidekicks or other devices that have more managable keyboards, the camera and everything out to pariticpants and see how they fare. I'm hoping to do a test soon.

I agree with you on making sure reports are visual and impactful to fight through the clutter. Pictures always make experiences come alive. You spoke of how camera phone pictures give you a sense of being there. My favorite shot from the study was that taken by a participant in the middle of her morning math class (we had asked kdis with camera phones to take pictures of the location of a given use). It shows the teacher up at the blackboard with his back turned and the backs of the other kids. She had been using her phone to play games because she was bored. Given that is 10:30am in the morning on a school day on the 3 day of a 7 day study, it would have very hard/expensive to capture that moment with a researcher in the field.

My issues around data are more about analysis than presentation. The big issue with data analysis on this project was that we captured both structured usage data and unstructured responses to exercises such as "No Phone Day." We got almost 700 distinct usage records with 10 fields and we also got essays and poems. Trying to mesh all that information was a bit of a challenge given the timeframe we had to work with for analysis and report writing. So I am currently on the hunt for techniques to master large and mixed data sets.

The report we did for the project is available through CEA. As Nokia is a member company, you actually have access to it for free. Here's the link: http://www.ebrain.org/crs/crs_arch.asp?crscode=CRS244

Unfortunately, because the report is for sale, some of the best pictures (including the classroom one described above) were not included due to not being able to get waivers from other folks in the pictures - i.e. the teacher at the blackboard.

I also have a short case study PDF that I can send you.

Best Regards,

Steve

> Trying to mesh all that information was a bit of a challenge given the timeframe we had to work with for analysis and report writing. So I am currently on the hunt for techniques to master large and mixed data sets.

I'm just about to go through the data from the last study, so very timely question.

Short glib answer:
- Lots of blank paper
- No interruptions, find that mental space
- Sufficient caffine buzz, off-set by physical exhaustion. Swimming does it for me

Short, slightly less glib answer:
- Data is like milk, it goes off - consume fresh
- Spend time at the beginning of the study figuring out how the data will be used in the reporting. How can it can be archived to make it easier for later retrieval by people within and outside the team?

A particular challenge we face is that we are nearly always working from different locations/time zones, so we need to get as much analysis time whilst we are in the field. We do devote some time to this on site, but the time pressures mean that breakfasts, lunches, dinners and late night sessions are times for the initial reflection.

It's an interesting topic - I'll try and put something more comprehensive together.

Hi Jan,

My current take on the data overload issue is that not only does a researcher need to consider how the data will be parsed and used for reporting, but also how to start integrating initial analysis into the collection process. In a perfect world, you'd be able to separate the two to some degree and let the data guide you, but business timelines don't allow for such luxury. One of the really good things about collecting via electronic means is that it's much easier to "code as you go" and start organizing the data as it comes in - thus hopefully minimizing "the big pile" at the end of field work. But to be able to code as you go means you need to have a least an initial analytic framework at the outset. The framework needs to flexible enough to go with anything that the data reveals, so you don't box yourself in and can still go where the data takes you.

I think some interesting tools for this are starting to emerge - XML especially seems like it should be able to be harnessed for this purpose.

Steve

PS Haven't heard from CEA yet - I know they are at a big event in Vegas, so all other things are probably on backburner.