Issues in Internet research☆☆☆

Human subject protection 

Recently, legal and ethical issues related to Internet research have begun to be discussed.23, 24, 25, 26 Ethical considerations such as informed consent, privacy, copyright, ethics, policy statements, unanticipated consequences of research, and impact of Internet information availability on future research have been the major ethical issues discussed. Also, international, national, and institutional efforts have been made to develop standardized guidelines for human subject protection in electronic transmission, including the Health Insurance Portability and Accountability Act.22, 27 Despite these efforts, standardized guidelines for human subject protection to deal with the ethical issues in Internet research currently are not available.22 Furthermore, since the use of the Internet in research is relatively new to academia in general, the institution where the researchers in this study were affiliated did not have guidelines for human subject protection in Internet research.

Because no guidelines were available, the Institutional Review Board (IRB) guidelines for traditional research on the basis of the Belmont Report were used to prepare the IRB protocol for the study presented in this article. On the basis of communication with the IRB director, the Internet survey was regarded as “exempt” because of the assumption that the survey was no more of a risk than traditional survey procedures. Yet, e-mail group discussions could not be classified as exempt according to the IRB guidelines. Rather, they were regarded as a minimal risk project because the risk of harm anticipated in the research was not greater than that ordinarily encountered in daily life or during the performance of routine physical or psychologic tests.

During the preparation process of the IRB protocol, the IRB office and the researchers needed to negotiate several aspects because there were no specific IRB guidelines for Internet research. The first negotiation with the IRB was related to the overall structure of the project Web site. Usually for traditional research projects, instruments are required to be provided as appendices. Thus, the Internet survey questions and the discussion topics (see Table 1) were included as appendices when the IRB protocol was first submitted. The IRB reviewers, however, wanted to see the overall structure of the project Web site and the introductory page (the first page of the Web site), thus additional appendices needed to be prepared for the revised IRB protocol.

Another requirement by the IRB review process was technical information about the project server and Web site. In the original protocol, it was just mentioned that an independent project server would be prepared and a project Web site would be established for the study. Without information on technical aspects of the project server and Web site, the reviewers could not determine the risks to the participants (eg, hacking, loss of privacy, intrusion). Therefore, more technical information needed to be provided in the revised protocol. Consequently, the review process was delayed, and the IRB office and the researchers sought consultation with experts in Internet research. After approval of the IRB protocol for the study presented in this article, the IRB offices of the institution where the researchers were affiliated developed the IRB guidelines for research that uses the Internet as a data collection method(available at: http://www.uwm.edu/Dept/RSA/index.html).

The major issue discussed during the IRB approval process was how to ensure anonymity and confidentiality of the data collected through the Internet. Anonymity and confidentiality could be relatively easily ensured for the Internet survey because the participants needed to enter their answers through the Internet, the data were directly saved on the project server, and their identities were never asked. However, ensuring anonymity and/or confidentiality of e-mail group discussions was disputable. For example, even though the participants' e-mail addresses were registered as a group and participants' names and other information related to identities were not disclosed, all the participants could see senders' e-mail addresses. The use of an e-mail list would not ensure anonymity because senders' e-mail addresses would appear in e-mail messages. Thus rather than ensuring anonymity, confidentiality was emphasized in the study. To ensure confidentiality, only research staff had access to the server containing data, data were saved in personal files, and electronic copies of data were stored in a locked cabinet.

Knowledge and skill in computers 

Bush et al27 explored the development of a customized Web site to assist breast and cervical health program outreach staff in a community screening program and evaluated the Internet knowledge, access issues, and barriers of outreach staff for 2 years with use of the Web site. They found that most of the staff believed that the Internet was becoming increasingly important to their work but also indicated that they needed more training and familiarization with the new technology. Duffy28 also found problems related to underdeveloped skills and lack of support for researchers who use the Internet as a data collection method. Indeed, Internet research should performed on the basis of researchers' understanding of Internet technologies.

Purchasing and preparing a project server was the first technical step in the study presented in this article. The next steps included linking the server to the main server of the institution and getting a server address. The project server was an independent computer under a subnet of the institution where the researchers were affiliated. Access to the server was always through the gateway server of the institution. The server was installed with Debian 2.0 Linux. This version of Linux (more information is available at: http://www.debian.org) has been more stable and more difficult to hack into than others. The security of the stored data in the server was enforced by Secure Shell (SSH, Palo Alto, CA),which restricted Telecommunications Network Protocol (telnet) and File Transfer Protocol (ftp)access with 128-bit encryption. An Apache HTTP server (Apache Software Foundation, Forest Hill, MD) was used. Knowledge and skill in computer technology were essential throughout this process.

The next step was the design and development of the project Web site. First, the overall structure of the Web site was designed by one of the researchers (a nurse researcher) to reflect the data collection procedures and data analysis process. The nurse researcher structured the Web site with the Microsoft Front Page that was the most frequently used software for Web site design and the No. 1 Web design software for novice users. Then, the Web site was programmed by one of the researchers who was a faculty member of the Engineering School and has substantial expertise in computer Web site design and development. Since the nurse researcher did not have expertise in computer programming, some of the overall structure of the Web site that the nurse researcher designed did not reflect the current technology in computer programming. Thus the engineering researcher needed to communicate with the nurse researcher continuously during the programming process. Furthermore, since the server supported by the institution where the researchers were affiliated did not support some of the Microsoft Front Page programming features, the programming needed to be redone with a different software program, Dreamweaver (Macromedia, Inc, San Jose, CA),which was supported by the institution server.

The project Web site was developed and maintained to conform to the guidelines and policies of the institution where the researchers were affiliated. The Web site was located on an independent server in the research office and linked to the researchers' Web site on the institution's server (http://845-linux00.ceas.uwm.edu/ONS/). Since the Internet survey data on sociodemographic characteristics of the participants were obtained through the Web site and discussion topics were posted on the Web site to provide information on e-mail group discussions, maintenance of the Web site was critical and computer expertise was essential during the data collection process. Therefore, communication between the nurse researcher and the engineering researcher was critical throughout the research process, and the nurse researcher needed to study about computer technologies whenever new knowledge and skill were needed.

Recruitment and response rate 

As previously described, research participants were recruited through contacting the Web sites of schools of nursing in 10 countries. Recruitment through the Internet was expected to be easier than traditional recruitment methods (eg, recruitment through mail/telephone announcement) because of the nature of the Internet communication (eg, high speed, flexible, easy access to potential participants). However, the recruitment process was not as easy as expected. The Web sites of schools of nursing in some countries (eg, France, Germany, and Italy) did not have English versions. Therefore, researchers did not understand the information provided on the Web sites. Also, in some other countries (eg, People's Republic of China, Egypt) it was difficult to find nursing schools offering a 4-year Bachelor of Science in Nursing program.In addition, many Web sites did not provide faculty lists, which are usually provided by Web sites of US universities.

Previous studies indicated that people responded more promptly to e-mails and that differences in time zones and geographic distance were not important issues in Internet use.17 Research participants also could more easily ask questions and get answers through e-mail compared with traditional research methods (eg, paper-and-pencil questionnaires). Compared with traditional data collection methods, data collection through the Internet and e-mail may provide increased opportunities for research participants to get information about the research and determine their willingness to participate.18 In this study, most of the potential participants promptly responded to the invitation to the study and asked questions related to the study (eg, payment for participation, unclear inclusion criteria, risk and benefits from the study, and safety issues) within 48 hours.

Usually, in Internet survey studies, when participants miss some questions on the Internet survey, error messages appear and ask the participants to enter their answers for the missing items. Since the study was a pilot study to determine the feasibility of Internet survey and e-mail group discussions, the Internet survey questionnaire tended to be simple and did not make technical arrangements (eg, error messages) to force the participants enter their answers for all the survey questions. The researchers also allowed the participants to skip questions that they did not want to answer. Even so, very few missing data were noted; only one participant did not answer the question on family income.

As the e-mail group discussions preceded, the response rate became a major issue of the data collection. Across the 11 topics, we received an average of 8.32 e-mails from 19 participants (range, 4–11 [SD = 2.89]). Therefore, a total of 158 e-mails were received during the 11 weeks. Per topic, each participant sent an average of 0.76 (SD = 0.26) messages, and 44 percent of the participants sent responses related to all 11 topics. Per country of the participants, 6 to 11 e-mails were received across the 11 topics. During the first week of discussions, about 90% of the participants responded to the topic sent by the researcher. At the final week of discussions, however, only 10% of the participants responded. Major holidays (eg, Christmas, New Year's Eve, and winter vacation) were identified by the participants as a major reason for the dramatic decrease in the response rate. In addition, institutionsin some Asian countries observed a 3-month winter vacation, and some participants from Asian and African countries did not have access to the Internet and/or e-mail during the vacation. Also, the participants' busy schedules were another issue; for example, some participants were away at conferences or during family vacations in other countries and therefore were unable to particpate. Another reason for the decreased response rate during the last several weeks of e-mail group discussions might have been lack of motivation to participate. No payment or benefits were given for participation.

Unique features of Internet interactions 

Unique features of Internet interactions include benefits from asynchronous interaction.2, 17 Asynchronous interaction allows people on very different schedules or in distant time zones to exchange messages and sustain discussions. In addition, e-mail lists offer their owners significant control over who can contribute to their group.2, 20 In other words, researchers have control over the data collection process. For example, when a participant sends a commercial announcement, the researcher has the option to eliminate the announcement. In addition, the Internet has been described as an “equalizer” of interactions without the visual distractions (age, gender, dress, social status) present in traditional face-to-face interactions.21 However, because Internet interactions are not face-to-face, their authenticity cannot be ensured. Internet interactions may, in the near future, resemble face-to-face interactions with the use of cameras and microphones. Researchers assume that authentic interactions take place between them and research participants; however, the authenticity of interactions and phenomenon reported and/or explored through the Internet is questionable.22 As previously described, researchers contacted each school of nursing identified through several Internet search engines, including Yahoo.com and AltaVista.com, and obtained lists of faculty members through the institutions' home pages. Faculty members were then contacted through their e-mail addresses identified on the Internet. Even though the researchers could ensure that all the potential participants were faculty members of the schools of nursing, they could not ensure that all the potential participants were experts in oncology because some of the institutions did not disclose the expertise of their faculty members and participants were recruited on the basis of their self-reports about their expertise.

Another unique feature of Internet interactions (research activities) is financial cost-saving because they do not require long-distance travel and the expenses of paper, pencils, photocopying, and mailing fees.2, 17, 20 Therefore, researchers who plan to use the Internet as a data collection method usually expect lower costs compared with traditional data collection methods. The study presented in this article, however, required a computer server, a printer, computer software, and consumable computer supplies (eg, floppy diskettes, and printer cartridges). Thus, the cost was much more expensive when compared with traditional research methods.

Universal language 

Another issue raised during data collection was that some participants used foreign language fonts that could not be read by participants in other countries. Even when the participants used plain text or rich-format text, the MS Window programs sometimes work with different language fonts of English in different countries. Thus the researchers needed to contact the senders and ask them to reformat and resend their messages in universally readable fonts (eg, Times New Roman, Arial) and in universally readable files (eg, plain text, rich-format text).When conducting e-mail group discussions in English, participants from the countries where English was the dominant language tended to actively participate in and lead the discussions. Participants from other countries where English was not the dominant language tended to be very passive in their participation and rarely raised issues related to the discussions. Yet, during data collection with the Internet survey, no problems related to language were indicated, even though some of the participants needed to spend more time to answer the questions because their mother languages were not English.