We will help you to create and conduct your online experiment in SoPHIE. Using our platform will help you to gather the data you need for your research.
We already came up with the answers to your questions
Interactive parts are crucial for many experiments. When looking at online experiments there are multiple situations to keep in mind. When running experiments in the lab you can use predefined groupings and you know, if enough participants show up prior to starting the experiment. You also know that participants will most likely stay after starting the experiment.
In an online experiment you should always keep in mind that participants might not show up or leave early and you need to think about potential problems of your experimental design ahead of time. For that reason you need a more dynamic approach for participant groupings, as you will group them based on the time they will join the experiment at. In addition its preferable to have some individual screens without interaction first.
As participant tend to drop out early from an online experiment, if they drop out at all, consent pages, instructions or comprehension questions should be shown prior to actually grouping the participants. That is not always in line with the experimental design, but as long as it is you should consider grouping the subjects as late as possible.
When running experiments in the lab, participants will use the devices you provide and you also tested your experiment with. However, subjects in online experiments are using their own devices and you need to specify any limitations that you have. If you are using large tables or images you would want your participants to use Laptops or Desktop Computers instead of mobile devices eventually. Apart from screen size, navigation through a touch screen might also have an impact on the outcome for your study, as participants will make slower inputs when typing on their tablet.
Furthermore, if you have time-sensitive parts within your study that require participants to have a decent internet connection. For that you can test the participants connection and ask for improvement or screen them out, if they do not match your requirements. The same situation also applies for specific browsers. If you have a complex study, which is not only using default web functionality you can also limit the access to the experiment to browsers you tested the experiment with.
Another limitation might occur, if you are planning to use a video chat in your online experiment. Prior to matching the participants you need to check, if camera and microphone are available on the participants devices. You should also provide an option to test those devices, to limit potential problems during the video chat.
We created a specific requirements step for these situations. That way you can simply define your requirements and make sure that you will get the results you are looking for.
When running your experiment online you need to make sure that every participant can access your study. System limitations stated before might be one thing, where you need to make sure that all parts of your study are working in older browsers for example. However, there is another situation you need to keep in mind. People participating in your study might have handicaps that you are not aware of. If one of the participants is color-blind you have to make sure that you are using high contrast colors. The same might apply for a participant being deaf. It comes down to understanding the limitations of your experiment design.
We will be happy to talk about your study design and provide suggestions for improvement:
If you are already running experiments in the lab you are most likely using a pool of students to run your experiments. You can still use that pool and send out individual participant links to have your students participate in the experiment. That way the change to your typical experiments recruiting process is minimal.
When you are not planning to use your own subject pool, you need to recruit study participants elsewhere. Common platforms for that are Prolific, Amazon MTurk or Qualtrics. The focus and setup differs for all of them and they do have a different focus. However, studies created with SoPHIE can be integrated with all of them and we are happy to discuss the best platform for your experiment.
Having attention checks is a highly debated topic. The main focus of every researcher is to gather high quality data and most of the time that includes various types of attention checks. However, having too many attention checks solely focussed on a higher data quality can be counterproductive. Participants will get annoyed after having the fifth attention checks that asks to select a specific option. If applicable, it is more reasonable to use comprehension questions to filter in terms of participants' attention. That way the participants will not lose focus of the experiment by getting annoyed.
Although the most common CAPTCHAs can be tricked by bots they are fairly good at giving suggestions, whether a participant is making automated inputs or not. In SoPHIELabs we included a CAPTCHA in the background to give a suggestion on how human-like the subjects' behavior was. You will receive a score between 0.1 and 0.9 giving you an idea of the data quality in addition to any comprehension questions or attention checks.
When actually running an online experiment the main focus for you should be the actual time. As most recruiting platforms have more workers from specific countries, you need to keep their timezones in mind. Also, you need to communicate everything to the subjects on screen. Implicit information like a low noise level are obvious in the lab but not for participants working from home. In addition you need to keep in mind that its harder to communicate with subjects during the study. Simplify the interface and prevent potential problems.
The payment of participants depends on the platform you are recruiting participants from. However, as long as you are thinking of the participants payment when designing the study, the output for payment will be automatically generated for you. You will either simply "Approve" the calculated bonus payment or upload a CSV file, but any of this is really simple process. Especially in comparison to handing out payments one by one in a lab experiment.