One of the most interesting discussion panels at the conference concentrated on the movement of internet activity to cell phones and other mobile devices. As we noted in a previous article, I-O Psychologists were playing catch-up with the internet in the design and utilization of psychometric tests. Things are moving fast in the IT age and more and more applications are becoming available on mobile devices. With this comes an expectation from clients of psychometric tests along with their candidates that tests should be made available on the devices.
There are some obvious advantages in using these mobile devices to administer online psychometric tests to candidates. Advantages include making the tests more inclusive. Research data shows that in many parts of the world, even where people may lack a home computer with internet access, they do have a cell phone with such connectivity. A number of large organizations provide downloadable apps for cell phones that assist in job searches. At the current time, whilst candidates can find out about job openings and even start to apply for jobs, most organizations do not allow psychometric testing within the same app and thus the candidate needs to be able to relocate themselves to a connected desktop computer at some point. Providing follow through for all stages of the initial recruitment process in the app will likely make the candidate experience more positive, result in more candidates actually following through and enhance an organization's "connected" image.
However, the discussion addressed a number of potential negative issues that clients and I-O Psychologists themselves need to be aware of in moving psychometric tests to mobile devices. One of the biggest issues in ensuring psychometric test validity is standardization. PsyAsia constantly reinforces during our psychometric training courses that to mess up in terms of standardization is also to mess up in terms of reliability which in turn impacts upon test validity. Mobile device platforms differ much more than desktop platforms. One needs to consider operating system, screen resolution, reliability of the internet connection and even where the user is at the time of testing. There was little chance of the candidate taking their desktop with them to the local bar and to complete the test there. As laptops became more accessible price-wise, there was more chance. Now, with cell phones there is a high chance. So, if the candidate gets stuck on a question, perhaps they just ask a friend or even a stranger in the bar! The environment has become much more difficult to control now and hence standardization is at threat. There will need to be tests of differential item functioning for each type of mobile device and given that in the USA 33% white, 46% black and 51% Hispanic have Smartphones according to recent research, further research needs to address possible differential performance by subgroup.
Given that there are so many different smartphones or mobile devices and a multitude of operating systems, the support required by candidates must also be forecast to increase. Candidates need to have a positive experience of the testing process and providing poor support will detract from this and may mean that test candidates don't make deadlines for testing.
In conclusion, it seems that we are going to need to accept that testing will take place on mobile devices. We may however not be able to offer all types of tests - items may be dictated by what mobile applications allow us to do and by the performance of various types of items in pilot tests. It's likely that given these issues, the testing industry will probably go with the lowest common acceptable and workable denominator.
As always, PsyAsia will ensure that we are at the cutting edge of these developments and be ready to offer advice to candidates and clients. In the meantime however, there is perhaps a need for updating and rethinking of guidelines on psychological test use to encompass testing on mobile devices. This goes to show just how fast things are moving given that at last year's SIOP conference in Atlanta the test committee reported they were nearly ready to publish revised testing guidelines (which did not include thought on mobile testing!). Oh dear, they need revising again!!
