MARS

The Rating scale assesses app quality on four dimensions. All items are rated on a

5-point scale from “1.Inadequate” to “5.Excellent”. Circle the number that most

accurately represents the quality of the app component you are rating. Please use

the descriptors provided for each response category.

SECTION A

Engagement – fun, interesting, customisable, interactive (e.g. sends alerts, messages, reminders, feedback, enables sharing), well-targeted to audience

Entertainment: Is the app fun/entertaining to use? Does it use any strategies to increase engagement through entertainment (e.g. through gamification)?

Select one answer

1 Dull, not fun or entertaining at all

2 Mostly boring

3 OK, fun enough to entertain user for a brief time (< 5 minutes)

4 Moderately fun and entertaining, would entertain user for some time (5-10 minutes total)

5 Highly entertaining and fun, would stimulate repeat use

Interest: Is the app interesting to use? Does it use any strategies to increase engagement by presenting its content in an interesting way?

Select one answer

1 Not interesting at all

2 Mostly uninteresting

3 OK, neither interesting nor uninteresting; would engage user for a brief time (< 5 minutes)

4 Moderately interesting; would engage user for some time (5-10 minutes total)

5 Very interesting, would engage user in repeat use

Customisation: Does it provide/retain all necessary settings/preferences for apps features (e.g. sound, content, notifications, etc.)?

Select one answer

1 Does not allow any customisation or requires setting to be input every time

2 Allows insufficient customisation limiting functions

3 Allows basic customisation to function adequately

4 Allows numerous options for customisation

5 Allows complete tailoring to the individual’s characteristics/preferences, retains all settings

Interactivity: Does it allow user input, provide feedback, contain prompts (reminders, sharing options, notifications, etc.)? Note: these functions need to be customisable and not overwhelming in order to be perfect

Select one answer

1 No interactive features and/or no response to user interaction

2 Insufficient interactivity, or feedback, or user input options, limiting functions

3 Basic interactive features to function adequately

4 Offers a variety of interactive features/feedback/user input options

5 Very high level of responsiveness through interactive features/feedback/user input options

Target group: Is the app content (visual information, language, design) appropriate for your target audience?

Select one answer

1 Completely inappropriate/unclear/confusing

2 Mostly inappropriate/unclear/confusing

3 Acceptable but not targeted. May be inappropriate/unclear/confusing

4 Well-targeted, with negligible issues

5 Perfectly targeted, no issues found

SECTION B

Functionality – app functioning, easy to learn, navigation, flow logic, and gestural design of app

Performance: How accurately/fast do the app features (functions) and components (buttons/menus) work?

Select one answer

1 App is broken; no/insufficient/inaccurate response (e.g. crashes/bugs/broken features, etc.)

2 Some functions work, but lagging or contains major technical problems

3 App works overall. Some technical problems need fixing/Slow at times

4 Mostly functional with minor/negligible problems

5 Perfect/timely response; no technical bugs found/contains a ‘loading time left’ indicator

Ease of use: How easy is it to learn how to use the app; how clear are the menu labels/icons and instructions?

Select one answer

1 No/limited instructions; menu labels/icons are confusing; complicated

2 Useable after a lot of time/effort

3 Useable after some time/effort

4 Easy to learn how to use the app (or has clear instructions)

5 Able to use app immediately; intuitive; simple

Navigation: Is moving between screens logical/accurate/appropriate/ uninterrupted; are all necessary screen links present?

Select one answer

1 Different sections within the app seem logically disconnected and random/confusing/navigation is difficult

2 Useable after a lot of time/effort

3 Useable after some time/effort

4 Easy to use or missing a negligible link

5 Perfectly logical, easy, clear and intuitive screen flow throughout, or offers shortcuts

Gestural design: Are interactions (taps/swipes/pinches/scrolls) consistent and intuitive across all components/screens?

Select one answer

1 Completely inconsistent/confusing

2 Often inconsistent/confusing

3 OK with some inconsistencies/confusing elements

4 Mostly consistent/intuitive with negligible problems

5 Perfectly consistent and intuitive

SECTION C

Aesthetics – graphic design, overall visual appeal, colour scheme, and stylistic consistency

Layout: Is arrangement and size of buttons/icons/menus/content on the screen appropriate or zoomable if needed?

Select one answer

1 Very bad design, cluttered, some options impossible to select/locate/see/read device display not optimised

2 Bad design, random, unclear, some options difficult to select/locate/see/read

3 Satisfactory, few problems with selecting/locating/seeing/reading items or with minor screensize problems

4 Mostly clear, able to select/locate/see/read items

5 Professional, simple, clear, orderly, logically organised, device display optimised. Every design component has a purpose

Graphics: How high is the quality/resolution of graphics used for buttons/icons/menus/content?

Select one answer

1 Graphics appear amateur, very poor visual design - disproportionate, completely stylistically inconsistent

2 Low quality/low resolution graphics; low quality visual design – disproportionate, stylistically inconsistent

3 Moderate quality graphics and visual design (generally consistent in style)

4 High quality/resolution graphics and visual design – mostly proportionate, stylistically consistent

5 Very high quality/resolution graphics and visual design - proportionate, stylistically consistent throughout

Visual appeal: How good does the app look?

Select one answer

1 No visual appeal, unpleasant to look at, poorly designed, clashing/mismatched colours

2 Little visual appeal – poorly designed, bad use of colour, visually boring

3 Some visual appeal – average, neither pleasant, nor unpleasant

4 High level of visual appeal – seamless graphics – consistent and professionally designed

5 As above + very attractive, memorable, stands out; use of colour enhances app features/menus

SECTION D

Information – Contains high quality information (e.g. text, feedback, measures, references)

from a credible source. Select N/A if the app component is irrelevant.

Accuracy of app description (in app store): Does app contain what is described?

Select one answer

1 Misleading. App does not contain the described components/functions. Or has no description

2 Inaccurate. App contains very few of the described components/functions

3 OK. App contains some of the described components/functions

4 Accurate. App contains most of the described components/functions

5 Highly accurate description of the app components/functions

Goals: Does app have specific, measurable and achievable goals (specified in app store description or within the app itself)?

Select one answer

N/A Description does not list goals, or app goals are irrelevant to research goal (e.g. using a game for educational purposes)

1 App has no chance of achieving its stated goals

2 Description lists some goals, but app has very little chance of achieving them

3 OK. App has clear goals, which may be achievable.

4 App has clearly specified goals, which are measurable and achievable

5 App has specific and measurable goals, which are highly likely to be achieved

Quality of information: Is app content correct, well written, and relevant to the goal/topic of the app?

Select one answer

N/A There is no information within the app

1 Irrelevant/inappropriate/incoherent/incorrect

2 Poor. Barely relevant/appropriate/coherent/may be incorrect

3 Moderately relevant/appropriate/coherent/and appears correct

4 Relevant/appropriate/coherent/correct

5 Highly relevant, appropriate, coherent, and correct

Quantity of information: Is the extent coverage within the scope of the app; and comprehensive but concise?

Select one answer

N/A There is no information within the app

1 Minimal or overwhelming

2 Insufficient or possibly overwhelming

3 OK but not comprehensive or concise

4 Offers a broad range of information, has some gaps or unnecessary detail; or has no links to more information and resources

5 Comprehensive and concise; contains links to more information and resources

Visual information: Is visual explanation of concepts – through charts/graphs/images/videos, etc. – clear, logical, correct?

Select one answer

N/A There is no visual information within the app (e.g. it only contains audio, or text)

1 Completely unclear/confusing/wrong or necessary but missing

2 Mostly unclear/confusing/wrong

3 OK but often unclear/confusing/wrong

4 Mostly clear/logical/correct with negligible issues

5 Perfectly clear/logical/correct

Credibility: Does the app come from a legitimate source (specified in app store description or within the app itself)?

Select one answer

1 Source identified but legitimacy/trustworthiness of source is questionable (e.g. commercial business with vested interest)

2 Appears to come from a legitimate source, but it cannot be verified (e.g. has no webpage)

3 Developed by small NGO/institution (hospital/centre, etc.) /specialised commercial business, funding body

3 OK but often unclear/confusing/wrong

4 Developed by government, university or as above but larger in scale

5 Developed using nationally competitive government or research funding (e.g. Australian Research Council, NHMRC)

Evidence base: Has the app been trialled/tested; must be verified by evidence (in published scientific literature)?

Select one answer

N/A The app has not been trialled/tested

1 The evidence suggests the app does not work

2 App has been trialled (e.g., acceptability, usability, satisfaction ratings) and has partially positive outcomes in studies that are not randomised controlled trials (RCTs), or there is little or no contradictory evidence.

3 App has been trialled (e.g., acceptability, usability, satisfaction ratings) and has positive outcomes in studies that are not RCTs, and there is no contradictory evidence.

4 App has been trialled and outcome tested in 1-2 RCTs indicating positive results

5 App has been trialled and outcome tested in > 3 high quality RCTs indicating positive results

Focus: what the app targets

Theoretical background/Strategies

Affiliations:

Age group

Technical aspects of app

Entertainment: Is the app fun/entertaining to use? Does it use any strategies to increase engagement through entertainment (e.g. through gamification)?

Interest: Is the app interesting to use? Does it use any strategies to increase engagement by presenting its content in an interesting way?

Customisation: Does it provide/retain all necessary settings/preferences for apps features (e.g. sound, content, notifications, etc.)?

Interactivity: Does it allow user input, provide feedback, contain prompts (reminders, sharing options, notifications, etc.)? Note: these functions need to be customisable and not overwhelming in order to be perfect

Target group: Is the app content (visual information, language, design) appropriate for your target audience?

Performance: How accurately/fast do the app features (functions) and components (buttons/menus) work?

Ease of use: How easy is it to learn how to use the app; how clear are the menu labels/icons and instructions?

Navigation: Is moving between screens logical/accurate/appropriate/ uninterrupted; are all necessary screen links present?

Gestural design: Are interactions (taps/swipes/pinches/scrolls) consistent and intuitive across all components/screens?

Layout: Is arrangement and size of buttons/icons/menus/content on the screen appropriate or zoomable if needed?

Graphics: How high is the quality/resolution of graphics used for buttons/icons/menus/content?

Visual appeal: How good does the app look?

Accuracy of app description (in app store): Does app contain what is described?

Goals: Does app have specific, measurable and achievable goals (specified in app store description or within the app itself)?

Quality of information: Is app content correct, well written, and relevant to the goal/topic of the app?

Quantity of information: Is the extent coverage within the scope of the app; and comprehensive but concise?

Visual information: Is visual explanation of concepts – through charts/graphs/images/videos, etc. – clear, logical, correct?

Credibility: Does the app come from a legitimate source (specified in app store description or within the app itself)?

Evidence base: Has the app been trialled/tested; must be verified by evidence (in published scientific literature)?

Would you recommend this app to people who might benefit from it?

How many times do you think you would use this app in the next 12 months if it was relevant to you?

Would you pay for this app?

What is your overall star rating of the app?

Awareness: This app is likely to increase awareness of the importance of addressing cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)

Knowledge: This app is likely to increase knowledge/understanding of cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)

Attitudes: This app is likely to change attitudes toward improving cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)

Intention to change: This app is likely to increase intentions/motivation to address cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)

Help seeking: Use of this app is likely to encourage further help seeking for cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)

Behaviour change: Use of this app is likely increase/decrease cancer risk factors (smoking, alcohol use, physical activity, diet, sun exposure)