Post theme 6

I selected my paper for the qualitative method part of theme 6 based on the fact that it covered a similar topic as the one I used for quantitative methods. To me it was interesting because the point of the whole paper was to challenge the current state of research into the subject that is mainly done quantitively, and I share the researchers point of view myself.

During the seminar we didn't really delve that much deeper into qualitative methods since we have been discussing it during multiple seminars already, but rather just talking about the papers we all selected.

We did however put some more time into talking about case studies, and I have to say that I'm really happy with the explanation I gave in my pre-reflection. The only thing I would like to add is a small addition to better explain theoretical saturation. In the beginning of a study it's relatively easy to find connection and information unknown to you, but while you keep iterating the cost of gaining new information keeps getting higher in what I believe to be an exponential rate. Sooner or later you will reach a point where the cost of learning something new is not worth the investment in time or money. I would also like to clarify that a case study is not a method itself but rather a strategy which I was a bit unclear about in my pre-reflection.

This is not to say that everything interesting is known about the case, but it is probably better to cover it in a future specialized study instead. Unfortunately we didn't have any lecture on this theme and I honestly didn't get any big eye openers between making my pre-reflection and now. Ilias even told me “I have nothing more to add” after sharing my view on what a case study is, which of course felt good, but I would have loved to have more things to discuss during the seminar from listening to more examples given during a lecture like the other weeks.

Overall I think case studies are interesting with it's cyclic nature of iterations rather than just having one predefined research question, even if those could be altered a bit as well.

Comments theme 4

1. http://reb2572.blogspot.se/2015/10/post-theme-4.html?showComment=1444661875641#c1048481461354002745

2. http://mediafluttery.blogspot.se/2015/10/theme-4-reflection.html?showComment=1444662871354#c6327037596416644812

3. http://mashasthoughts123.blogspot.se/2015/10/seminar-4-refection.html?showComment=1444663337892#c272769935324487996

4. http://u1vfukfo.blogspot.se/2015/10/theme-4-post-posting-quantitative.html?showComment=1444664105128#c6642115301102584770

Post Theme 5

Post Haibo Li

For me Haibo's lecture was kind of refreshing. While he didn't really talk much about design research itself, I was very interested in his point of view about research from an industrial rather than purely theoretical perspective.

What I believe to be his most important point was what I call “the 90/10 principle” (can't remember what he actually called it), which means that you should spend 90% of the time finding and defining the actual problem you want to solve and just 10% of the time actually solving it. The old joke with a new spin, while cheesy explained it rather well. A teacher tries to figure out if it's possible to outrun a bear and decide it's not possible. His student on the other hand puts on his running shoes and say “I don't have to outrun the bear, I just have to outrun you”. The student clearly spent more time thinking about the problem itself rather than trying to find a solution to a bad question.

He also talked about something I mentioned in my pre-theme post, namely the importance of securing funding for your project and how prototypes can help you do this with a proof of concept. He did however expand on it even more while talking about the importance of knowing both the technical and market aspects when pitching the idea, or pitching it as an entrepeneur. You have to question if your project actually adress a real problem, if there is a market, account for timing and if you are actually qualified to do it.

Lastly he talked about the importance of not always trusting yourself. When you are working with a project you always run the risk of seeing what you want you want to see, also known as tunnel vision. I agree that this is very important to keep in mind, and isn't this one of the reasons to why the HCI-field and its focus on usability is so important in the first place?

Maybe I liked this lecture a lot because I myself believe in the importance of having a business mind as an engineer and see what I want to see in a lecture that agrees with it. Never the less. It gave me some more food for thought.

Post Anders Lundström

While Haibo was very business oriented, Anders took us back to the theoretical aspects of research. He argued that a prototype in research doesn't necessarily have to try and solve a problem at all. The aim of a prototype in design research should be to provoke discussions about a specific topic with the intention to gain some kind of new knowledge. While he agree with Haibo when it comes to prototypes in a business setting, I took it as his view being vastly different when it comes to research where he doesn't have to take money into account. I think both views are very valid depending on your personal goals with your work. I think that I have already stated where I stand enough however...

Anders did delve deeper into design research as a concept, explaining that the actual process can be seen as the empirical data of the study, which differs from quantitative and qualitative studies we covered earlier. The knowledge contribution could be seen as the lessons learned and the problems identified while doing it rather than the result or solution.

Something I never did before this lecture was actually trying to explain the word design, which is kind of weird knowing how much I use the word in every day life. It's not like I don't “understand” the word but I liked the definition he gave: “Design is an intention to change reality into something more preferable”. It might just be an anecdote but I'm surely going to start using it myself.

Pre Theme 6

Public Displays of Play: Studying Online Games in Physical Settings

Nicholas Taylor, Jennifer Jenson, Suzanne de Castell, Barry Dilouya

http://onlinelibrary.wiley.com/enhanced/doi/10.1111/jcc4.12054

This paper studies gamers in a physical setting rather than online surveys that can be seen as the norm. The authors themselve state that they want to challenge these quantitative methods that are normally used because context play a big role in how gamers will identify with the identity as a gamer and games,

1) Which qualitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?

The participants in the study were interviewed at locations associated with gaming in a social setting like LAN-parties, internet cafes and events. What was discovered was that many of the participants that answered that they don't take part in a virtual world was in fact MMOG players that might just be on a hiatus from or in between games, but still had a strong connection to virtual world avatars. This is something that wouldn't be seen if the study was conducted purely quantitatively.

The downside of this method can be seen when the researchers tried to recruit participants at internet cafes, where the setting was uninviting for intrusion, and on top of this I imagine it being harder for a participant to express him or herself as freely as one can in an anonimous online survey.

Some participants also agreed to make a travelogue where they answered questions regarding online/offline activities to see if their adventures in MMOGs affect their behavior in real life.

2) What did you learn about qualitative methods from reading the paper?

What I take away from this paper is not knowledge about a qualitative method I didn't know existed before, but rather once again emphasize the importance of using qualitative methods to understand your sample, especially if you want to conduct data gathering through quantitative means. Other papers I have read on the subject ignores this, and one could question the validity of those results and conclusions when considering how the research was performed. Conclusions can surely be drawn from the data, but do we truly understand why?

Building Theories from Case Study Research

Eisenhardt, K. M.

http://www.jstor.org.focus.lib.kth.se/stable/258557

Methods for Game User Research: Studying Player Behavior to Enhance Game Design

Desurvire, H., El-Nasr, M.S.

http://ieeexplore.ieee.org.focus.lib.kth.se/stamp/stamp.jsp?tp=&arnumber=6562711

1) Briefly explain to a first year university student what a case study is.

A case study is research performed on a person, organisation, service etc. and its context. The methods used could be of both quantitative and qualitative nature, and using knowledge gained during the research to improve methods or research questions is encouraged, in fact you are not supposed to formulate your hypotheses until after you have some form of data to analyze. This will go on in an iterative process until a theoretical saturation is met or you run out of resources in the form of time or funds. The goal is to answer the research question and make some sort of framework answer that could be generalized to similar problems.

2) Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.

The main problem I had with my paper was the fact that a lot of details about the process was left out, presumably due to an NDA enforced by the game developing company the study was performed on. They do however set out with a goal in mind to look at the game design process and apply different usability testing methods to improve user experience.

With the help of qualitative methods like think-aloud and quantitative data from questionnaires surveying user satisfaction the game designers were able to solve many of their problems by developing a tutorial in a way that learned the user how to play the game properly without making them lose interest in the game before it started for real. Unfortunately the specifics were not mentioned but it seems like it was successful and the paper seems to focus on giving other developers a framework on how to perform user testing themselves.

What I felt was lacking was the exploration or at least description of potential problems that could occur when following the recipe suggested in the paper. Surely there are advantages and disadvantages with all methods and not giving these enough consideration at least in writing is a minus in my opinion.

Comments theme 3

1. http://butlikewhyisit.blogspot.se/2015/09/theme-3-reflection.html?showComment=1443996571210#c2197836872768580133

2. http://securepathofscience.blogspot.se/2015/09/theme-3-research-and-theory-reflection.html?showComment=1443997564191#c8653941997687557210

3. http://dm2572byen.blogspot.se/2015/09/theme-3-research-and-theory_27.html?showComment=1443998746345

4. http://bjornsblogggg.blogspot.se/2015/09/theme-3-post-reflection.html?showComment=1443999285365#c404097275335936178

5. http://rickardsdm2572.blogspot.se/2015/09/theme-3-post-seminar-reflections.html?showComment=1443999979540

Post theme 4

Quantitative methods as a concept should be easier to understand than any of the other themes so far. Who haven't done some kind of data collection through questionnaires and similar some time in school. While the concept itself wasn't anything new, I still found it interesting to see how it is applied in research studies. I will not pretend that I understand all the math used while calculating the connections nor have I looked at it closer, but it made me think about the importance of identifying as many parameters as possible before you perform your test that can affect the result both directly and indirectly.

Something I found interesting however during the lecture was how much work is put into refining the questions through qualitative methods to look for any weaknesses in your test. All the iterations of pilot tests where you try to avoid bias, confusion etc. with evaluations before you believe you have a good test that hopefully will give you usable data.

With this in mind, I don't think any study related to human behavior can be done effectively with quantitative methods alone, even though the data gathering is. I got some flashbacks to Kant and his quote “perception without conception is blind” thinking you can't draw any conclusions and understand your data unless you know what you want to achieve and that your test actually answer that question in a good way.

In the seminar we talked about some strengths of quantitative methods, and what came up is the fact that you can reach a high statistical significance without having to spend as much time and resources compared to if the data collection was qualitative, much like I discussed in my pre theme post. The most interesting thing, that I didn't think about before the seminar that seems obvious after hearing it is the fact that you might not be able to get any interesting information from a quantitative study. Ilias used the example of a study on homeless people where doing a qualitative study where the researcher lived with them for a time was way better to understand the environment they live in and take part of all what I like to call silent knowledge.

I also see a weakness in only using quantitative methods when attempting innovation in e.g. design work. I can't see any way you could use quantitative methods to make e.g. the study on tangible user interface in connection to learning children programming like in the paper for theme 5. Observation of the behaviour of the subjects in the study gave more information than any questionnaire could, and similarly any attempt to measure input data would miss all the important contextual information that might be of even more importance.

Pre Theme 5

Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration

Réhman, S., Sun, J., Liu, L., & Li, H. (2008)

http://ieeexplore.ieee.org.focus.lib.kth.se/stamp/stamp.jsp?tp=&arnumber=4637879

1) How can media technologies be evaluated?

First and foremost technology you develop should be evaluated based on it's usability. Like the authors mention in the paper they use the ISO recommendations regarding effectiveness, efficiency and satisfaction. It's important that your service or product is intuitive or easy to learn or the user will lose interest. In the paper this is evaluated by looking at empirical data from tests e.g. error rates before and after training but also through a questionnaire.

When looking at evaluation of media technologies in general however, I believe it's important to look at economical factors as well. Instead of just asking “can we do it?”, you have to include “should we do it?”. This might not be the the most important aspect during research like in this paper since they are investigating if the concept works at all. However the authors still include that aspect somewhat when asking if the test subjects would be willing to pay for the service. Money is almost always of importance (even though there are examples like e.g. the development of Linux) and I would dare say it's an aspect you can't ignore. Will the taste justify the price? Did we develop with the user that is willing to pay in mind, or a user that will not?

2) Why could it be necessary to develop a proof of concept prototype?

One explanation could be linked to what I previously mentioned about money. Research and development requires funding and even if a proof of concept is not required by e.g. an investor, it could still help in lowering the perceived risk in granting the funding.

It is also a way to test theory when putting it into practice. Does it translate well into an actual product or service or are there new parameters we have to keep in mind and account for to reach a satisfying result? Prototypes serve this purpose well since you in an early stage can test specific functionalities and reiterate design choices. Worst case scenario might be that you choose to scrap a project, but better do it in an early stage than turning it into a big sunk cost fallacy.

Finding Design Qualities in a Tangible Programming Space

Ylva Fernaeus & Jakob Tholander

https://www.kth.se/social/files/560a438df276542fe1e371e5/p447-fernaeus.pdf

Differentiated Driving Range: Exploring a Solution to the Problems with the “Guess-O-Meter” in Electric Cars

Anders Lundström

https://www.kth.se/social/files/560a43adf2765424a93619ad/32.Lundstrom.pdf

3) Are there any important differences with design driven research compared to other research practices?

When comparing these two papers to the ones I chose myself for the previous themes, I would say the main difference is that the design driven research is attempting innovation. A simpler way to explain my point would be that it tries to explain how? rather than why?.

4) Can practical design work in itself be considered a 'knowledge contribution'?

Choices made during design work should always be motivated by other theories and/or user tests in some form. While doing this we are entering the field of “design and action theory” mentioned in the article by Gregor in theme 3. Findings during the design process and can be evalutated by others as can the method itself. I agree with this since you do not only benefit from learning from your own misstakes, but also those of others. It is of course also very benefitial to find inspiration in what have worked for others.

5) Are there any differences in design intentions within a research project, compared to design in general?

Once again I want to bring up economical aspects to back my statement. Within research the aim should be how to best achieve high usability in a theoretical sense. Design in general however still have usability as a priority, but monetary gain is likely to be a driving force compared to knowledge itself.

6) Is research in tech domains such as these ever replicable? How may we account for aspects such as time/historical setting, skills of the designers, available tools, etc?

I think it's hard to replicate the research fully since design is very context and user base specific. However this is likely not an issue since I believe the important thing to take with you from reading these kinds of research papers is inspiration for your own methods and early prototypes since design work is iterative.