Ingredients to cook a delicious CSCW research

December 25, 2015

Doing research simply means the systematic use of some set of theoretical and empirical tools to try to increase our understanding of some set of phenomena or events.

Joseph E.Mcgrath, the author of article “Methodology matters:doing research in the behavioral and social sciences” says.

In fact both of the two first articles are addressing the fact that what you are going to get out of a study totally depends on the method you are using and your philosophical stance. However the first article “Selecting Empirical Methods for Software Engineering Research” concentrates on software engineering research which makes it more sensible for me, while the second reading talks about research in social sciences.

The first reading mentions that we can not certainly say which research method is suitable for which research problems and various local factors should be considered when selecting a method, including available resources, access to subjects, opportunity to control the variables of interest and skills of the researcher. Furthermore, it precisely mentions that because each method has its own flaws, comprehensive research strategies which benefits from multiple research methods are more viable, so that weaknesses of each used method could be addressed and compensated by use of other methods.

First we must see what kind of research question we are asking. Potential questions include:

one important factor is your philosophical stance, which dramatically affects how evidences and responses to your research question(s) satisfies you. Here are four important philosophical stances:

To classify possible research methods, we could introduce 5 major classes for software engineering research, however not all researchers in this area will necessarily have consensus on the names and domains of these classes:

Furthermore, the second reading says that a research process always involve with a content of interest, some ideas that give meaning to it and some techniques that enable the researcher(s) to study them. The author depicts three domains of research in behavioral and social science including:


Then the author mentions 8 research strategies and groups them into 4 quadrants as following:

  1. Field strategies including field study and field experiment. Both emphasize that the behavior system under study is natural, in the sense that it would occur whether or not the researcher were there and whether or not it were being observed as part of a study.
  2. Experimental strategies including laboratory experiment and experimental simulation. In contrast to those of Quadrant 1, they involve concocted rather than natural settings. The laboratory experiment and the experimental simulation are strategies that involve “actor-behavior-context” systems that would not exist at all were it not for the researcher’s interest in doing the study.
  3. Respondent strategies including sample and judgement study. They concentrate on the systematic gathering of responses of the participants to questions or stimuli formulated by the experimenter, in contrast to the observation of behaviors of the participants within an ongoing behavior system. Studies are usually done under neutral conditions of room temperature, lighting, chair comfort to nullify any effects of the behavior setting or context on the judgments that are the topic of study.
  4. Theoretical strategies including formal theory and computer simulation. The inclusion of these two strategies reminds us of the importance of the theoretical side of the research process. One of the more powerful general strategies for research is the simultaneous use of one of the theoretical strategies and one of the empirical strategies.

The author of this paper try to highlight the fact that results of our experiment always depend on methods, and like first reading, it verifies that a combination of multiple methods should be used. It also mentions the important fact that the results of a study should not be interpreted in isolation. Researcher should always consider other evidences and studies on the same research question.

3-2In the 3rd reading, the author mentions some conceptual models of CSCW which are largely descriptive and then talks about her framework, which is currently descriptive as well, but supposed to be more developed by further investigation. She defends her model of “coordinated action” by claiming that it frees us from having to decide on only one common field of work and one “clear-cut goal”. She also mentions that they chose the word “action” to emphasize the importance of goal-directedness implied by the word “work”. MoCA has 7 dimensions as following:

  1. Synchronicity: According to Johansen’s matrix, it concerns a continuum of coordinated action ranging from being conducted synchronously, or at the same time, to asynchronously, or at different times.
  2. Physical Distribution: Again similar to Johansen’s matrix, this continuum concerns that if all actions are occurring in the same geographic location or at different places. It emphasizes that working from different locations for a long time will definitely be a big problem.
  3. Scale: Addresses number of participants involved in collaboration. Because a lot of articulation work needs to be done to organize members of a teamwork.
  4. Number of Communities of Practice: It focuses specifically on the notion of different cultural communities, so that when people from different disciplines come together and want to collaborate, how could we manage and resolve these differences.
  5. Nascence: Discusses Unstablished v.s. established coordinated actions.
  6. Planned Permanence: It refers to planned or intended permanence which is less addressed in similar works and it’s important because it’s usually impossible to say until when a coordinated action will continue and also hard to say when things are at a stable situation.
  7. Turnover: Refers to the rapidity with which participants enter and leave. This dimension cover collaborations that range from closed, private collaborations where participants leave slowly, if at all, to collaborations that are fully open and public so that might have many participants.

You can watch her presentation about this work in Stanford seminar:

As a new researcher in software engineering areas, I found the first reading surprisingly useful. It drew a comprehensive picture of the art of doing empirical research in software engineering, however many key points of this reading is not just limited to this area. It elegantly shows us how to propose a valid and valuable research question as the key starting point of doing research. Then describes what to consider when looking for an answer to this question and emphasizes that the way we think and our philosophical expectations and stances will have a considerable impact on potential answers. Then highlights the importance of having a theory which is like a lens through which the world is observed. Then provide nice information about various empirical methods in software engineering and how they could be combined together to compensate each other’s shortfalls. And finally provides information about how to collect data and how to validate results based on the stance and method we have used. The second reading also does same effort and in some cases provide more details on methods which the first reading didn’t discuss in-depth. It also reminisces me how closely interconnected social sciences and collaborative software engineering topics are. Finally, the final reading represents a framework of coordinated action which tries to add more value to related works and models around CSCW studies by discussing some aspects (dimensions in the author’s words) that were less focused by other works while addressing many useful works that have been done in the past.

Leave a Reply

Your email address will not be published. Required fields are marked *