Wednesday, July 29, 2009

July 29 (late night) - PhD thesis of Suzanne Lynn Stevens

One last one before I go to bed....
Reading up just one more PhD dissertation.

Suzanne Lynn Stevens did her PhD dissertation on DEVELOPING GUIDELINES FOR DESIGNING CHILD SAFETY PRINTED EDUCATIONAL MATERIALS: A USER-CENTERED APPROACH.

“This research increased our understanding of information design and well as generating general design guidelines for pamphlets. … produced a pamphlet for credible sources to use as an education tool for parents….”

3 Phases, 5 Studies. Variables (readability criteria) – name and details.

Pictorial illustrations, schematic diagrams, flowchart,

Study 1. Questionnaire with 5-point Likert scale; descriptive statistics; Chi-square test; Kendalls-Tau correlation of rankings and ratings.

Study 2. Questionnaire with 10 MCQ and 1 open-ended question.

Study 3. Questionnaire with 10 key points, 9-point Likert scale. Cloze test; ANOVA; Tukey’s HSD; Fisher’s Exact test.

Study 4. SMOG formula; BIDS-3 scale; RAIN instrument.

Study 5. Questionnaire with “two questions, the first assessing intent and the second assessing perceived control. Each of the questions used a bipolar scale using likely/unlikely and easy/difficult,” 14-day follow-up questionnaire (over the telephone). 3 hypothesis. Descriptive statistics; Tukey’s HSD; bar charts showing mean and standard deviation; data tables.

Guidelines for the design of an effective pamphlet was produced.

Stevens had done her studies in orderly and structured manner. Her data analysis were simplistic and easy to understand. I think this dissertation is worth revisiting...observing closely how she varied her research from Study 1 to Study 5....despite the fact that what she did is nothing close to what I am planning to do. Noted that her research did involve Usability to certain extent.

Tuesday, July 28, 2009

July 29 - PhD thesis of Countney V Martin, N Delia Grenville and Young Seok Lee

Coutney V Martin did her PhD dissertation on “Usability of Pictorial Toy Assembly Instructions for Young Children.”

Martin had one overarching question and 4 objectives. She used mixed-method approach, quantitative and qualitative. Tables, schematic diagrams, data tables, bar charts, photos, line graphs, and flowcharts were used

Usability assessment was for the Objectives 1-3. There were 4 hypothesis for Objective 1, none for Objective 2, and 3 hypothesis for Objective 3.

Rating used was Fun / No; Easy / Hard; Not Frustrating / Frustrating.
ANOVA was used for quantitative analysis.

For study of key usability factors, PCA (principal component analysis) method was employed to do the factor analysis. Variables used were:
1. Percentage of Incorrect Parts
2. Correct Parts Assembled Per Minute
3. Look Frequency
4. Look Duration
5. Look Time per Correct Part
6. Ranking Ease
7. Rating Ease
8. Ranking Fun
9. Rating Fun
10. Ranking Instructions
11. Again-Again
12. Verbal Protocol - Utterances of Hard per Minute of Assembly Time
13. Verbal Protocol - Problems Verbalized per Minute of Assembly Time
14. Extent of Building Experience
15. Previous Building Frequency.


N. Delia Grenville did his PhD dissertation on “DEVELOPING HEURISTICS TO OPTIMIZE THE CONFIGURATION OF THE VIDEO-MEDIATED ENVIRONMENT.”

RQ1, RH1a, RH1b, RQ2, RH2a, RH2b, RQ3, RH3, RQ4, RH4.
OR (Operation Research) Model, tables (to summarise info/literature review), schematic diagrams, data tables, bar charts, high-low bar charts,
His research had involved Human Factors, Facilities Planning/Design and Operation Research. My guess is that he did Industrial Engineering at Bachelor degree level.

7-point likert scale was used for the questionnaire survey. Compared data for naïve and experienced users. Used pairwise comparison. Mean and Standard Deviation.

This thesis is one which I should benchmark in the aspect of statistical methods because data collection was from questionnaire survey (with Likert scale) and 2 sets of evaluators were used (naïve and experience). This is what I plan to do….questionnaire survey using 2 groups of evaluators.

“A set of design heuristics was created from the design themes articulated by the
participants. These ten heuristics are focused on the design process repeatedly mentioned
by naïve and expert users. An eleventh heuristic was added as an instructional guideline.”
The design guidelines/heuristics used are not aligned to Nielsen’s or anybody’s.

He summarized the key differences between the naïve users and experienced users.
Expert evaluation was also used; 6 experts.
He had a revised OR Model.

In conclusion, he had done 2 studies. First study involved 25 naïve users and 25 experienced users. Second study involved experts. A set of eleven heuristics was developed.

Young Seok Lee did his PhD dissertation on “OLDER ADULTS’ USER EXPERIENCES WITH MOBILE PHONES: IDENTIFICATION OF USER CLUSTERS AND USER REQUIREMENTS.”

“Three user clusters (explorers, basicians, and minimalists) were identified based on mobile phone usage behavior, and their characteristics were described. User satisfaction was effected by three attributes of mobile phones: usefulness, ease of use, and pleasure of use.”
“Barriers (perceptual, cognitive, attitudinal, knowledge, and information barriers) were found to hinder older adults’ utilization of mobile phone technology over the four dimensions of the domestication process (appropriation, objectification, incorporation, and conversion).”

Mixed method research; Study 1 was quantitative - Self-report questionnaire; Study 2 was qualitative - Phenomenology interview.

Tables to summarise literature review.

For Study 1, 5-point likert scale was used for questionnaire survey. ANOVA and t-test were used. 3 Reseach Questions were: “1. How do older adults use the mobile phones? Age and gender differences? 2. How many user groups can be formed? Their characteristics? 3. How will usefulness, ease of use, pleasure of use will affect user satisfaction with mobile phones? Age, gender, cluster differences?” Data analysis were: “1. Descriptive statistics and univariate analysis 2.Cluster analysis and validation 3. Factor analysis and multiple regression analysis.” Bar charts showing mean and standard deviation; line graphs; factor analysis; multiple regression analysis.

For Study 2, he used 4 participants from each of the 3 user clusters. Qualitative study.

What can I learn from this? Probably, I could benchmark on how he did his data analysis. It is noted that his Study 1 would be quite different from mine because he seemed to be exploratory for Study 1. Despite the Exploratory approach, he had used descriptive statistics.

July 29 - Dr Tonya Smith-Jackson...search Usability dissertations

After reading PhD dissertations of Ryu, Hu and Capra, I somehow feel that those at Virginia State University doing Industrial & Systems Engineering....
1) had done their research in a planned, orderly and systematic manner;
2) knew where they were heading and were quite focused;
3) had clearly defined & synergistic research objectives, phases, research questions, hypothesis, and approach/research method.
They have a common Advisor who is Dr Tonya Smith-Jackson.

Hence, this morning, I search and download Usability dissertations whereby Smith-Jackson is one of the Committee.

Although my intention is to get PhD dissertations, I have also downloaded MSc dissertations too.....since all these would similarly provide some inputs/reference. So far I have 5 PhD dissertations, i.e. Mooney, Greenville, Stevens, Lee and Martin.

When I am free, I will be browsing/reading through these dissertation. Now, I think by reading other people's PhD dissertation, this will help me in doing mine.

July 28 - PhD thesis of Capra; ym with Tan Wee Hoe

Chatted with Tan Wee Hoe on ym this evening. He is pursuing PhD in Warwick University. His one and only Research Question is “How subject matter experts and game experts could collaborate to design and develop games for use in formal education contexts?” He estimates remaining tenure would be 1 month 7 months.

Miranda G. Capra did her PhD dissertation on “Usability Problem Description and the Evaluator Effect in Usability Testing.” Her Advisory Committee Chair is also Dr Tonya Smith-Jackson, just as Ryu and Hu.

She had 3 goals and 3 research questions; RQ1 with hypothesis 1a-1d, RQ2 with 2a-2b, RQ3 with 3a-3b.

Capra summarized the 3 Problems, 3 Goals and the respective Approach into a table. She also tabulated the Chapter/Phase, Study and Output. All these show clearly the structure of her dissertation and study/research. She had numerous tables, charts, formulae and screenshots and love to use bullets.

She analysed result of survey using Mean, Standard Deviation and Median.

She used factor analysis Jaccard-styled to reduce the many to 10 UPD guidelines. Participants completed a questionnaire to rate each of the 10 guidelines from the card sort for difficulty and importance. Difficulty was rated on a scale of difficult/easy. Importance was rated on three scales: required/optional, relevant/irrelevant and helpful/harmful. She studied the correlations using ANOVA.

MANOVA was also used for study of evaluator effect in usability testing. Flowchart was used to show the Usability Report Coding Process.

Tables were used to summarise the results to each Hypothesis for all the Research Questions.

One last comment: lots of statistics.

Monday, July 27, 2009

July 27 - PhD thesis of Hu, MSc thesis of Delikostidis

Ying Hu did a PhD dissertation on “MOTIVATION, USABILITY AND THEIR INTERRELATIONSHIPS IN A SELF-PACED ONLINE LEARNING ENVIRONMENT.” Hu’s Advisory Committee Chair is similar to Ryu’s, i.e. Dr Tonya Smith-Jackson.

Hu had 2 research objectives and 4 research questions. She had one or a few hypotheses for each research question. Phases were clearly described. Research method framework was clearly shown in a diagram.

She use 3 online learning application, i.e. existing, improved motivation, and improved usability. She attached her Usability Checklist/Questionnaire in the Appendix; I could see what usability criteria she had used.

Although her area of research is not much like mine, it is good to her orderly and systematic flow of research.

Ioannis Delikostidis did MSc thesis on “Methods and techniques for field-based usability testing of mobile geo-applications.”

Delikostidis wrote his Research Objectives and Research Questions. RQ were a bit lengthy. Research Stages and the resulting output (i.e. Chapters) were clearly stated in the thesis structure. Research method framework was in diagram.

He included numerous mobile screenshots, photos and schematic diagrams. His Chapter 3 was on Usability Testing Methods and Techniques. He used tables to present info on comparisons.

His conclusion was done by answering his Research Questions.

Sunday, July 26, 2009

July 24 - TQ to all my MeL Lecturers; PhD proposal draft 1

Wrote emails to my Lecturers who taught me the various subjects of MeL program.

MeL = Master of Multimedia (e-Learning Technologies).

Prepared the first piece of PhD proposal, i.e. Schematic Diagram....showing my proposed research area....Usability Inspection Tool for Mobile Learning Applications.

This was Schematic Diagram rev.1 after comment from Dr Koo suggesting an "instrument." She is also now into HCI, in addition to e-Learning and Mobile Learning.

Wow!! This is great. This is strategic. After all, my intended research is actually related to HCI, Mobile Learning and e-Learning.

Browse and read through lots of papers on Mobile Learning. Quite a number from JCAL and mlearn.com.za particularly. Did copy and paste into PhD Proposal Draft 1, and listing the References.

July 26 - PhD thesis of Ryu and Trifonova

I found Ryu's PhD theses to be in stategic usefulness to my future PhD research.
Young Sam Ryu did "Development of Usability Questionnaires for Electronic Mobile Products and Decision Making Methods."

Usability Questionnaire is one type of UET (usability evaluation tool). Electronic mobile products in his thesis are wireless mobile devices. Mobile learning is done using wireless mobile devices. So, there is stategic usefulness and alliance to my propose research.

I think I could also refer to lots of references on Usability that he quoted. He tends to concentrate on a few authors namely Keinonen, Ketola, Kirakowski, Nielsen, Saaty, Shackle, and ISO standards. He loved to focus on usability dimensions (what I would call usability categories in my MeL project dissertation). He picked his usability criteria (for his study/research) from referred sources, namely SUMI, QUEST, PUTQ, QUIS, PSSUQ, Keinonen, Kwakh, Lindholm, Jordan, Szuc, Kolckar, and researcher (himself).

I find that Ryu had done his research systematically, phase by phase, very much me who has planned out the research too. He had also documented his PhD in a good flow (according to his phases). I find this this to be a good benchmark.

Subsequently, I opened up Trifonova's PhD thesis. I have Trifonova name rather frequently, so that's why her thesis appealed to me.

Anna Trifonova did PhD thesis of "MOBILE LEARNING: WIRELESS AND MOBILE ECHNOLOGIES IN EDUCATION TOWARDS HOARDING CONTENT IN M-LEARNING CONTEXT ."

Hoarding content is important as the wireless mobile device would often go offline. I think this should be a Usability Criteria. Good lead!!

Trifonova had numerous research objectives which seems to be multi-directional. As I browsed through, I did not see her Research Questions; may be she did not have. This PhD thesis seemed to be done using Case Study method....writing about her experiences on hoarding and Mobile ELDIT, and incorporating some other Mobile Learning-related research/studies.
Not much statistical analysis...apart from demographics analysis...only simple stat like percentages.

She 's not a visual thinker....only a few graphs/charts, diagram.....did i see any table??? Ryu, on the other hand, loves to make use of tables, diagrams, graphs/charts, formulae...to express himself. Ryu did mathematical modeling and pairwise comparison.

I would like to be like Trifonova using simple stats for easy understanding. I would like to be like Ryu for his systematic flow of phases, and lots of tables, diagrams, graphs/charts.

July 25 - PhD Proposal - Research Planning & Research Questions

Today, at a hotel in Genting Highlands, at 6 am, I managed to draft out on pieces of A4 paper...my Research Questions, my Research Mind Map, and my Research Planning which in the form of flowchart containing processes (in boxes) with description of activities for specific processes.
So, it was a fruitful 2 hours....putting on my thinking cap....while my "brothers" Tung Mun Tat and Khoo Koon Wah were sleeping.

When I reached home at about 2pm, I quickly wrote down everything formally using PowerPoint, Word and Excel.
The output was:
1) PhD Proposal: Research Questions, Contribution to Knowledge, Objectives & Deliverables. [doc, pdf]
2) PdD Proposal: Research Planning (containing Flowchart with processes, timeline, activities). [ppt, pdf]
3) PhD Proposal: Gantt Chart (showing proposed Research Schedule). [xls, pdf]

At night, I emailed to Dr Koo Ah Choo (hopefully, she will be my Supervisor) the 3 files.

Later in the night, I added more stuff to my draft PhD Propsal...which is now Draft 2. Had just included the output of today into the PhD Proposal Draft 2 [doc].

Starting this blog on my PhD journey

Today, i am starting this blog to document down my journey for PhD.

On 22/7/2009 (Wed), I had a renewed interest to do a PhD program after (1) encourage from beloved wife, (2) status of "graduated" from Master of Multimedia (e-Learning Technologies). I spent an entire evening searching for info on PhD program in MMU and other Malaysian universities.

Finally, the result revealed that the universities of choice are MMU and UKM. MMU is of course my first choice since I did my Masters there, I already know several Faculty members and MMU's good service being a private univ. UKM offers specialisation of Multimedia Usability Evaluation which is what I want to research.

So I would apply for MMU. If I can't get MMU, then I will apply for UKM.