Roadmap of the RIOT test and Riot IQ Platform.

Backlog

Next up

In Progress

Done

RIOT Roadmap

Hello - the user experience on the test was good but I was a little disappointed that I was charged $50 for this test for two reasons:(1) the reaction time subtest will be determined by the speed of internet connection.(2) the VSI test seems inflated.For these two reasons I’m not sure the test is directionally accurate. I was expecting it not to be prefect given it’s in beta phase but it’s a very long way from being accurate in my view.

In Review

[Question] Does Internet Affect Reaction Time?

An Anonymous User

Gwyneth Rolph requested that we build an easy way to link third party questionnaires or create an in-built questionnaire system within the app so that when administering a test you can add questions before/after an assessment.

[Feature] Pre-test questionnaire

Robert Neir

I think it would be interesting to see a score for the general ability index (GAI), particularly when the test taker has a discrepant profile. - Ethan Woods

[Feature] GAI

Dougal Phillips (Senior Clinical Neuropsychologist):“I want to put the iPad in front of my patient, press go, and have it go through everything so I can do something else. Current iteration of RIOT platform not intuitive enough yet.As a neuropsych, if my patient does poorly on a test, I want to infer that it’s because of some particular issue, not simply because they didn’t quite understand the instructions or the user interface or they couldn’t see the stimuli clearly or it simply timed out before they knew what they were meant to be doing. I’m not a fan of the videos. Instead I think you should incorporate an interactive element where the patient has to respond in a certain way to progress through the instructions. That way the patient gets confident they’re responding correctly and you as the examiner are reassured they’ve understood. I use the CANTAB and it does this well. Worth checking out (but maybe discretely as they’re your competition).Many of my patients struggle with their vision. Some of the stimuli seem unnecessarily small or have poor contrast or are a bit blurry. While I like the aesthetic of having the test heading up the top left and the timer up the top right in a banner, by doing this, you’re leaving less room for the stimuli. The same goes for the (blank) banner down the bottom. I would recommend doing everything you can to reconfigure the layout to make the stimuli as large as possible. I’d try piloting it on a tech-naive older adult. If you can make it easy for them to use, then it’ll be easy for the rest of us to use as well.”

[Feedback] Improve Ease of Use

Dougal Phillips

I am 68 years old, am high-functioning autistic, with ADHD.  I scored 131 on the Free Test; would my diagnoses affect my performance/score on your test?

As soon as is feasible, let’s expand the subtests that have their item pool created via algorithms. SToVeS, Matrix Reasoning, and Figure Weights will probably be the easiest subtests to create new items for using automatic item generation.

[Feature] More automatic item generation

russwarne

Currently, we have a flag system that flags suspicious behavior. This is a good start, but we have received requests to enhance the anti-cheat systems.

[Feature] Beef up the anti-cheat systems

The Riot IQ Team

Hi, does your RIOT Basic Test disaggregate fluid intelligence -related test sections and score them relative to bands of testee ages versus a single scoring algorithm for everyone "age 18 and over"?

For the spatial orientation subtest, the map image is too small to read. The click to zoom button increases the image size in until it covers the entire screen, blocking out the question and answer. I found myself scrambling to click back and forth between map and q &amp; a, even trying to memorize the q &amp; a, rather than focusing on figuring out the answer.

Spatial Orientation Map Zoom

Hello, I was wondering if you have an estimate for the g-loading of the (full) Riot IQ test?

g-loading of Riot IQ test

We have gotten numerous requests for translations of the test. Let’s do this.

Planned

[Feature] Add localization support (translations)

The Basic and Enhanced assessments show too much information in the results. We do not have enough data for some of these pieces of information to be shared with the examinee. We will be covering sections within the test results that we are not confident in showing. Some results just need more data for us to be confident in revealing them.

[Feature] Partial Results

If someone takes an Enhanced or Custom test and later takes a Full test, it would be beneficial if their scores from the shorter test automatically populate in the Full test, so they don’t have to retake any subtests.

[Feature] Ability to upgrade Basic -> Full assessment

People have asked for a simple way to share results. Add a link that can be shared as well as cards that give a preview of an examinee’s results.

[Featue] Share link

The devices that examinees will use to take the RIOT will vary greatly. On this note, users have requested a calibration step to level the playing field and increase the accuracy of the reaction time subtests.

[Feature] Add calibration step to reaction time

For an examinee to take a test they much purchase it before looking through all the subtests. We have gotten feedback that examinees want to see the “test overview” page before they decide to purchase. So, we are adjusting the onboarding flow to allow for this. No payment will be required until the user decides to actually commence the assessment.

[Improvement] Improve Onboarding Efficiency

First, the user experience. Windows 10/Chrome/14” laptop screen. The test was very easy to navigate. I liked the explanatory videos and practice examples, and the checklist was a nice addition to ensure that the taker understood what to do. The subtests were very quick to complete.Admittedly, I was making notes to provide feedback and may not have been 100% engaged on every question, I appreciate that this is a beta version, and also that regression to the mean is a thing, but I wasn’t expecting the final score and wonder if some kind of ceiling is in place. Given that I will be including highly gifted samples as part of my research, I do not want them scoring too far below their “official” scores (as I kept running into with the SB5).

Completed

Feedback on the freebie version

Would be great to see some validity studies upcoming. Will allow me to feel even more prepared to use this in a clinical setting. LOVE the direction/vision for this platform. Will be quite helpful for my work. Thank you Riot team. Keep up the good work. Cheers.

Validity Studies with WAIS, SB-5, etc.

Amy Summers

A couple of things,Without giving too many specifics, I scored very low on analogies. I noticed that this subtest was noticeable shorter than the other verbal sections. Does it cut off if you get a certain number wrong? Or is was it supposed to be shorter and I did much worse on that section? I ask because I scored a 13.8 percentile on this subtest compared to a 95.3 information and 87.6 vocabulary, so it seems odd when also considering how short the subtest seemed to be. The other issue, which was a definite issue, was the section where you use maps (I think spatial orientation?). I don’t know how I scored high on it, because the answers that you choose from where superimposed over the question, so I could not see a large chunk of the town map questions (this was not as big of an issue for the elevation map questions just because most of the question was below where the answers were superimposed over the question). Thank you,Anthony

High Priority

[Feedback] Definite issue on one subtest, possible issue on another

I have seen a few people on here say that the test needs to be ne-normed to improve accuracy. Approximately when will we have renormed tests by?

When will we have properly normed results?

in my results it seems the reaction test didn’t work correctly. scored abnormally low 0/34. i don’t recall the test glitching while doing it. and i believe i hit the wrong color one time only, but it all functioned seemingly normal. can this be reset so i can try it on my pc instead of my phone? otherwise, how much of a difference might that make to the result score, if the reaction were an average result at least? here is the link to my result. <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://www.riotiq.com/app/share/556f4d88-c262-4648-b2c7-26c73fc4f836">https://www.riotiq.com/app/share/556f4d88-c262-4648-b2c7-26c73fc4f836</a>

reaction test

James V

I took the full test when it was publicly released. Today I checked the site, and my score was ten points lower. Are these scores likely to change a lot as the test is calibrated, and how should we interpret such large changes in the results?

Score dropped 10 points

The question asked “could weight” when it should have said “could weigh”. I can’t remember which test but it was about halfway through. I wish I wrote down which one it was. Sorry.

[Bug] Misspelling

Dear Riot IQ Team,<pre><code>The Riot IQ website/application leaks user email addresses. Currently, this concerns around 200 addresses.

The addresses are shown (included) in the HTML source of several (all?) www.riotiq.com (and, previously, app.riotiq.com) pages.

Demonstration: search for the string "whitelist" in the HTML source code of, for example, https://www.riotiq.com/ or https://www.riotiq.com/app/auth.

Please fix this issue as soon as possible.

Thank you,

An anonymous riotiq.com website visitor (NOT being one of the current private beta users)</code></pre>

Riot IQ website/application possible security vulnerability

When completing the Basic Test (and probably the longer tests as well?), there is a required demographics survey afterwards. I apologize that I did not take screenshots, but I want to point out that the way the survey UX was laid out (possibly given my particular zoom settings on my mobile browser), it was not clear that there was more to the survey than the first couple questions. That is, after completing the first couple questions, I would then try to press the submit button (which was always present)—however, there was no indication that the survey had submitted or failed… unless I (which I had only later realized) scrolled down to see the unanswered, required questions. For some individuals, I would think this could be a confusing, if not frustrating, experience. I hope this makes sense, and if not, I can try explain further and/or reproduce the issuse with screenshots. I would suggest either removing the always available “submit” button (such as moving it the end of the survey) or at least providing more obvious feedback (such as through automatic scrolling to the unanswered question, etc.) when pressed.Thank you for your awesome work!

UX Feedback re: Demographics Survey on Mobile Device

Anrey Wang

Tim Bates said: “Hi Rioters :-) One thing to ponder: We nearly always use Prolific+ Qualtrics. So integrating with that workflow would be useful verging on essential.”