Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS20070282664 A1
Publication typeApplication
Application numberUS 11/614,962
Publication dateDec 6, 2007
Filing dateDec 21, 2006
Priority dateNov 10, 1999
Also published asUS7548846, US20070174108, WO2001035215A2, WO2001035215A3, WO2001035252A2, WO2001035252A8, WO2001035295A2, WO2001035295A8, WO2001044981A2, WO2001044981A8, WO2001046850A2, WO2001046850A3, WO2001046850A8
Publication number11614962, 614962, US 2007/0282664 A1, US 2007/282664 A1, US 20070282664 A1, US 20070282664A1, US 2007282664 A1, US 2007282664A1, US-A1-20070282664, US-A1-2007282664, US2007/0282664A1, US2007/282664A1, US20070282664 A1, US20070282664A1, US2007282664 A1, US2007282664A1
InventorsRobert Monster
Original AssigneeMonster Robert W
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
Multi-region market research study processing
US 20070282664 A1
Abstract
A market research service is provided with a study creation service that includes a number of functions to support the creation of multi-region market research studies. In one embodiment, the functions include a function in support of implicit country and/or city selection within a region. In another embodiment, the functions include a function in support of providing a real time cost estimate for the study being created. In yet another embodiment, the functions include a function in support of auto notification to translators providing translation services for translating study elements (such as questions, messages, pick lists and concepts) into supported target languages. In yet another embodiment, the functions include a function in support of on-line check in of the translated study elements. In yet another embodiment, the functions include a function in support of on-line monitoring of translation status.
Images(48)
Previous page
Next page
Claims(21)
1-153. (canceled)
154. A computer-implemented method for creating a multi-region market research study, the method comprising:
displaying information to facilitate on-line selection of one or more geographical regions in which to conduct a multi-region market study;
receiving, from a market research client, an on-line selection of one or more geographical regions in which to conduct a multi-region market study;
in response to receiving the on-line selection from the market research client, providing a real time cost estimate for conducting the multi-region market research study, wherein the real time cost estimate is based at least in part on the geographical regions selected by the market research client; and
automatically notifying one or more translation services to translate elements of the multi-region market research study from an original language into one or more target languages, wherein the one or more target languages are associated with the one or more geographical regions selected by the market research client.
155. The computer-implemented method of claim 154 wherein, after the elements of the multi-region market research study have been translated, the method further comprises:
conducting the multi-region market research study for the market research client; and
analyzing the results.
156. The method of claim 154, wherein automatically notifying the one or more translation services includes automatically notifying the one or more translation services in response to on-line definition of the multi-region market research study.
157. The computer-implemented method of claim 154, further comprising:
providing on-line check in for the translated market research study elements; and
providing on-line status monitoring for the translated market research study elements.
158. The method of claim 157, wherein providing on-line check in for the translated study elements includes providing on-line check in for at least two of study questions, study messages, study pick lists, and study concepts.
159. The method of claim 157, where providing on-line check in for translated study elements includes separately storing study questions, study messages, study pick lists, and study concepts.
160. The method of claim 157, wherein providing on-line status monitoring for the translations comprises displaying a summary report of the translation status of the study elements by target languages.
161. The method of claim 154, wherein automatically notifying one or more translation services to translate study elements includes automatically notifying one or more translation services via electronic mail.
162. A method for creating a multi-region market research study, the method comprising:
displaying information to facilitate on-line selection of one or more geographical regions in which to conduct a multi-region market study;
receiving, from a market research client, an on-line selection of one or more geographical regions in which to conduct a multi-region market study;
automatically notifying one or more translation services to translate elements of the multi-region market research study from an original language into one or more target languages, wherein the one or more target languages are associated with the one or more geographical regions selected by the market research client; and
providing on-line check in for translated study elements of the multi-region market research study translated from an original language into one or more target languages.
163. The method of claim 162 wherein providing on-line check in for translated study elements includes providing on-line check in for at least two of study questions, study messages, study pick lists, and study concepts.
164. The method of claim 162 wherein providing on-line check in for translated study elements includes separately storing study questions, study messages, study pick lists, and study concepts.
165. The method of claim 61, further comprising providing on-line status monitoring for the translated study elements.
166. A multi-region market research study creation system, the system comprising:
a computer-readable medium containing a plurality of programming instructions designed to automatically implement a plurality of functions to support on-line creation of a multi-region market research study, the programming instructions including:
a first function to display information to facilitate on-line selection of one or more geographical regions in which to conduct a multi-region market study;
a second function to receive, from a market research client, an on-line selection of one or more geographical regions in which to conduct a multi-region market study; and
a third function to notify one or more translation services to translate elements of the multi-region market research study from an original language into one or more target languages, wherein the one or more target languages are associated with the one or more geographical regions selected by the market research client; and
one or more processors operably coupled to the computer-readable medium to execute the programming instructions.
167. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to facilitate on-line check in for the translated study elements.
168. The system of claim 167 wherein the fourth function is designed to provide on-line check in for at least one of study questions and study concepts.
169. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to facilitate on-line selection of panelists from the one or more selected regions to participate in an on-line survey relating to the market research.
170. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to select one or more countries/cities within a region in response to a selection of the geographical region via the second function.
171. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to provide a real time cost estimate for the multi-region market research study.
172. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to provide a real time cost estimate for the multi-region market research study based at least in part on the number of geographical regions selected for the study.
173. The system of claim 166 wherein the programming instructions are further designed to automatically implement a fourth function to display a summary report on the translation status of the study elements by target languages.
Description
    RELATED APPLICATION
  • [0001]
    This application claims priority to U.S. Provisional Application No. 60/164,585, entitled “System and Method for Obtaining and Collating Survey Information In Real Time For Multiple Languages and Multiple Character Encodings,” filed on Nov. 10, 1999, which is hereby fully incorporated by reference.
  • BACKGROUND OF THE INVENTION
  • [0002]
    1. Field of the Invention
  • [0003]
    The present invention relates to the field of data processing. More specifically, the present invention relates to the conducting of multi-region market research studies. The present invention also relates to the creation of market research studies for multiple regions, in particular, when non-English speaking regions are involved. The present invention further relates to reporting and analyzing data from a multi-region research survey.
  • [0004]
    2. Background Information
  • [0005]
    With increased globalization of commerce, market researchers increasingly have to conduct market research that involves multiple geographic regions of the globe. Creating, conducting, collecting data, analyzing data and reporting on such a market research study present a number of challenges that otherwise do not exist in the case of a single region market research study. Multi-region studies necessarily involve panelists (also referred to as participants, respondents, interviewees, study subjects, and so forth) that are dispersed in various geographic locations, with language differences, cultural differences, price point sensitive differences and so forth shared between them.
  • [0006]
    Historically, market researchers would develop a study for a first region, e.g., an English-speaking region, and then work with local counterparts in the various other regions to have the study translated, and conducted. Under the prior art, it was often difficult to coordinate the conducting, or processing of such surveys including the tasks of survey generation and delivery, as well as ensuring the data collected are consistent, and compatible for consolidation, analysis and reporting.
  • [0007]
    Thus, an improved approach to the generation and delivery of surveys, as well as data collection, analysis and reporting is desired.
  • [0008]
    With increased globalization of commerce, increasingly market researchers have to conduct market research that involves multiple regions of the globe. Creating, conducting, collecting data, analyzing data and reporting on such a market research study present a number of challenges that otherwise do not exist in the case of a single region market research study. Multi-region studies necessarily involve panelists (also referred to as interviewees, study subjects, and so forth) that are dispersed in locations. Often, it involves language differences, cultural differences, price point sensitive differences and so forth.
  • [0009]
    Historically, market researchers would develop a study for a first region, e.g., an English speaking region, and then work with local counterparts in the various regions to have the study translated and conducted. Under the prior art, it was often difficult to coordinate the creation of the study, ensuring the data collected are consistent, and compatible for consolidation, analysis and reporting. It was also difficult to manage the translations, and most importantly, the overall cost of the studies.
  • [0010]
    Thus, an improved approach to the creation, conducting, data collection and analysis, as well as reporting is desired.
  • [0011]
    With increased globalization of commerce, market researchers increasingly have to conduct market research that involves multiple regions of the globe. Creating and conducting such a survey, and collecting, analyzing, and reporting data from such a survey present a number of challenges that otherwise do not exist in the case of single region market research studies. That is, multi-region surveys necessarily involve diverse panelists who may speak different languages, have cultural differences, different price point sensitivity, and so forth.
  • [0012]
    Historically, market researchers would develop and implement a survey in one region, like the United States, and then work with local counterparts in the various regions to have the survey translated and conducted. Under the prior art, it was often difficult to coordinate the creation of the survey so as to ensure consistency in the collected data for consolidation, analysis, and reporting.
  • [0013]
    Thus, an improved approach to the creation and administration of a multi-region survey, as well as data collection, analysis, and reporting from such a survey is desired.
  • SUMMARY OF THE INVENTION
  • [0014]
    An electronic survey engine is provided with a number of support functions to support electronic survey processing and content delivery in a preferred language selected from a large number of supported languages. In one embodiment, the support functions include a function for identifying an electronic survey involving a product concept, a function for generating a price for the product concept based at least in part upon pricing constraints, and a function for storing the generated price for the product concept in association with the identified electronic survey and in association with the respondent participating in the electronic survey.
  • [0015]
    In another embodiment, the support functions include a function for retrieving from a data store, a plurality of electronic survey questions composed according to a first language, a function for displaying in a preferred language identified by a respondent, a select one of the plurality of electronic survey questions, a function for receiving from the respondent, a response to the select one of the plurality of electronic survey questions, and a function for validating the response to the select one of the plurality of electronic survey questions based at least in part upon one or more response criteria associated with the first of the plurality of electronic survey questions.
  • [0016]
    A market research service is provided with a study creation service that includes a number of functions to support the creation of multi-region market research studies. In one embodiment, the functions include a function in support of implicit country and/or city selection within a region. In another embodiment, the functions include a function in support of real time costing of a multi-region study, taking into consideration cost factors such as the number of panelists, their distribution in the study regions/countries/cities, and the amount of translations required. In yet another embodiment, the functions include a function in support of auto notification to translators providing translation services for translating study elements (such as questions, messages, pick lists and concepts) into supported target languages. In yet another embodiment, the functions include a function in support of on-line check in of the translated study elements. In yet another embodiment, the functions include a function in support of on-line monitoring of translation status.
  • [0017]
    A research service includes a number of functions for reporting and analyzing data from a multi-region research survey. The service provides an analysis interface to a research client to receive client input using a regional communication mode corresponding to the research client. Based on the client input, the service analyzes data from an on-line survey of a panelist. The on-line survey uses a second, distinct regional communication mode corresponding to the panelist. Analysis results are provided to the client in the analysis interface, again using the regional communication mode corresponding to the research client. In various embodiments, the analysis service similarly provides for analysis of survey data from multiple panelists using multiple regional communication modes.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0018]
    The present invention will be described by way of exemplary embodiments, but not limitations, illustrated in the accompanying drawings in which like references denote similar elements, and in which:
  • [0019]
    FIG. 1 illustrates an overview of the present invention, including the multi-region market research processing service of the present invention, in accordance with one embodiment;
  • [0020]
    FIG. 2 illustrates one embodiment of a multi-region market research service provider including an electronic survey engine of the present invention;
  • [0021]
    FIG. 3 illustrates the operational flow of the relevant aspects of the electronic survey engine of FIG. 2 including main module 220 for generating and delivering electronic surveys in accordance with one embodiment of the present invention;
  • [0022]
    FIG. 4 is a graphical illustration of a semantic differential question in accordance with one embodiment of the invention;
  • [0023]
    FIG. 5 illustrates the operational flow for displaying concept-specific marketing information in accordance with one embodiment of the invention;
  • [0024]
    FIG. 6 is a graphic illustration of a marketing concept including differently formatted sections;
  • [0025]
    FIGS. 7(A-C) illustrate various operational flows for determining a price point in accordance with one embodiment of the invention;
  • [0026]
    FIG. 8 illustrates an operational flow for administering an electronic on-line CDA;
  • [0027]
    FIGS. 9A and 9B illustrate an exemplary operational flow of a result capturing process in accordance with one embodiment of the invention;
  • [0028]
    FIG. 10 illustrates an example computer system suitable for use to practice the present invention, in accordance with one embodiment;
  • [0029]
    FIG. 11 illustrates an example of preferred language based content dynamically constructed in accordance with one embodiment of the present invention;
  • [0030]
    FIG. 12 illustrates the operational flow of the relevant aspects of a content delivery engine including the main module of FIG. 2 for delivering content in a preferred language basis in accordance with one embodiment of the invention;
  • [0031]
    FIG. 13 is a graphical illustration of a screen for providing search criteria for use in selecting panelists in accordance with one embodiment of the invention;
  • [0032]
    FIG. 14 is a graphical illustration of a summary page displaying a list of panelists meeting specified search criteria in accordance with one embodiment of the invention;
  • [0033]
    FIG. 15 is a graphical illustration of a panelist details page in accordance with one embodiment of the invention;
  • [0034]
    FIG. 16 illustrates an overview of the present invention, including the multi-region market research service of the present invention, in accordance with one embodiment;
  • [0035]
    FIG. 17 illustrates a method view of the present invention, in accordance with one embodiment;
  • [0036]
    FIG. 18 illustrates the operational flow of the study creation service of the multi-region market research service of FIG. 1, in accordance with one embodiment;
  • [0037]
    FIGS. 19A-19B illustrate an example user interface and the operational flow of the relevant aspects of the study creation service for selecting target regions, in accordance with one embodiment;
  • [0038]
    FIGS. 20A-20B illustrate an example user interface and the operational flow of the relevant aspects of the study creation service for supporting real time costing of a multi-region study, in accordance with one embodiment;
  • [0039]
    FIG. 21 illustrates an example user interface of the study creation service for checking in translated study elements, in accordance with one embodiment;
  • [0040]
    FIG. 22 illustrates the operational flow of the study creation service for supporting checking in of translated study elements, in accordance with one embodiment;
  • [0041]
    FIG. 23 illustrates an example user interface of the study creation service for monitoring the status of the translations, in accordance with one embodiment;
  • [0042]
    FIG. 24 illustrates the operational flow of the study creation service for supporting translation status monitoring, in accordance with one embodiment;
  • [0043]
    FIGS. 25A-25B illustrate various example data organizations suitable for use to store the message and pick list elements for practicing the present invention, in accordance with one embodiment;
  • [0044]
    FIGS. 26A-26B illustrate various example data organizations suitable for use to store the questions and concepts for practicing the present invention, in accordance with one embodiment;
  • [0045]
    FIG. 26C illustrates an example data organization suitable for use to store the various cost factors for practicing the present invention, in accordance with one embodiment;
  • [0046]
    FIG. 27 illustrates an example computer system suitable for use to practice the present invention, in accordance with one embodiment;
  • [0047]
    FIG. 28 illustrates one embodiment of the present invention;
  • [0048]
    FIG. 29 demonstrates one embodiment of the present invention;
  • [0049]
    FIG. 30 illustrates one embodiment of an identification window;
  • [0050]
    FIG. 31 illustrates one embodiment of a survey list;
  • [0051]
    FIG. 32 illustrates one embodiment of a survey summary;
  • [0052]
    FIG. 33 illustrates one embodiment of an analysis report for a particular response variable;
  • [0053]
    FIG. 34 illustrates one embodiment of an analysis report for demographic information;
  • [0054]
    FIG. 35 illustrates one embodiment of an analysis report for semantic differential questions;
  • [0055]
    FIG. 36 illustrates one embodiment of a three dimensional analysis report;
  • [0056]
    FIG. 37 illustrates one embodiment of a price point analysis report;
  • [0057]
    FIG. 38 illustrates one embodiment of a computer system to perform various operations of the present invention;
  • [0058]
    FIG. 39 illustrates one embodiment of a machine readable medium to store instructions to implement various operations of the present invention;
  • [0059]
    FIG. 40 illustrates one embodiment of a text coding analysis report; and
  • [0060]
    FIG. 41 illustrates one embodiment of a configuration window for a text coding analysis report.
  • DETAILED DESCRIPTION OF THE INVENTION
  • [0061]
    In the following description, various aspects of the present invention will be described. However, it will be apparent to those skilled in the art that the present invention may be practiced with only some or all aspects of the present invention. For purposes of explanation, specific numbers, materials and configurations are set forth in order to provide a thorough understanding of the present invention. However, it will also be apparent to one skilled in the art that the present invention may be practiced without the specific details. In other instances, well-known features are omitted or simplified in order not to obscure the present invention.
  • [0062]
    Parts of the description will be presented in terms of operations performed by a processor based device, using terms such as data, tables, requesting, determining, retrieving, displaying, and the like, consistent with the manner commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. As well understood by those skilled in the art, the quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through mechanical and electrical components of the processor based device; and the term processor includes microprocessors, micro-controllers, digital signal processors, and the like, that are standalone, adjunct or embedded.
  • [0063]
    Various operations will be described as multiple discrete steps in turn, in a manner that is most helpful in understanding the present invention; however, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations need not be performed in the order of presentation. Further, the description repeatedly uses the phrase “in one embodiment,” which ordinarily does not refer to the same embodiment, although it may.
  • Overview
  • [0064]
    Referring first to FIG. 1, a block diagram illustrating an overview of the present invention, in accordance with one embodiment, is shown. As illustrated, multi-region market research service provider 102 (hereinafter “service provider”) is advantageously provided with the multi-region market research processing service of the present invention. As will be described in further detail below, the multi-region market research processing service (hereinafter simply “study processing service”) includes a number of novel services including, electronic survey generation, delivery and result capturing services. In particular, the study processing service provides dynamic generation of electronic surveys based on a preferred language of the participant, determination and inclusion of geography sensitive price points, concept specific text and graphic formatting, and response recordation services as described further herein.
  • [0065]
    As shown, the computing equipment of multi-region market research respondents 104 and 106 (hereinafter simply respondents 104 and 106) are coupled to each other as well as the computing equipment of service provider 102 via networking fabric 108. In accordance with one embodiment of the invention, service provider 102 generates and distributes one or more language specific electronic surveys to one or more of respondents 104 and 106 via networking fabric 108. In turn, respondents 104 and 106 complete the electronic survey(s) by entering the data and returning (i.e., submitting) the electronic survey(s) back to service provider 102 also via networking fabric 108. In one embodiment, the electronic surveys of the present invention are distributed to respondents 104 in the form of a single markup language encoded page. In an alternative embodiment, service provider 102 may facilitate administration of the electronic survey(s) whereby service provider 102 enters the data remotely for a respondent. For example, service provider 102 may act as an intermediary or translator who enters a respondent's answers to an electronic survey in lieu of the respondent directly entering the data. It should be noted that service provider 102 may be a “single” entity or an “alliance” of affiliated entities. In other embodiments, third parties other than service provider 102 may also act as intermediaries for the respondent.
  • [0066]
    Except for the novel services incorporated within the study processing service of service provider 102, the various equipment used by service provider 102, and respondents 104 and 106, as well as networking fabric 108 are intended to represent a broad range of these elements known in art. Examples of computing and networking equipment suitable for practicing the present invention include but are not limited to various palm-sized, notebook size, or desktop computers, as well as set-top boxes, servers, routers/switches, personal digital assistants, wireless telephones utilizing i-Mode and/or wireless access protocols (WAP) and so forth. The functions and constitutions of these elements are known and accordingly will not be further described.
  • [0067]
    In various embodiments, networking environment 100 including service provider 102 and respondents 104 and 106 are employed to create, conduct, collect, analyze and report on consumer/marketing research/surveys in a large number of languages. In particular, in one embodiment, network environment 100 is employed to create surveys as described in co-pending U.S. Patent Application, PCT US00/41972 (Ser. No. 09/744,774), entitled Multi-Region Market Research Study Creation, filed contemporaneously, and having common inventorship with the present invention. In yet another embodiment, network environment 100 is employed to analyze and report on survey results as described in co-pending U.S. Patent Application, PCT US00/30652 (Ser. No. 09/744,775), entitled Reporting and Analyzing Data From a Multi-Region Research Survey, filed contemporaneously, and having common inventorship with the present invention. These co-pending applications are all hereby fully incorporated by reference.
  • Generating an Electronic Survey
  • [0068]
    Referring now to FIG. 2, a diagram illustrating one embodiment of service provider 102 of the present invention is shown. As illustrated, service provider 102 is advantageously equipped with one or more databases 212 having stored therein a variety of support data, and electronic survey engine 210 to generate and deliver the electronic surveys to client devices, e.g., respondents 104 and 106, as part of a larger research study.
  • [0069]
    Electronic survey engine 210 dynamically generates electronic surveys utilizing language specific content elements such as questions, response options, and display text and graphics, and delivers such electronic surveys to one or more client devices in a preferred language indicated by each respondent. In accordance with one embodiment of the invention, electronic survey engine 210 dynamically generates a geographically adjusted price point for the product concept(s) of each study based upon established pricing criteria such as maximum and minimum price constraints. In one embodiment of the invention, prices for the product concepts of the various studies are randomly generated specifically for each respondent upon that particular respondent being invited to participate in the survey. Accordingly, upon each successive access (including, e.g., a web page refresh) by the respondent to a particular electronic survey, the same previously generated price will appear every time the same product concept being surveyed is displayed to the respondent. In accordance with the present invention, support data stored in databases 212 are expressed in a variety of supported languages and include survey questions, valid response options for each survey question, client identifiers, price constraints and conversion rates, as well as textual messages and graphics/images specifically designed for the various supported languages. Electronic survey engine 210 further includes a main module 220 incorporating the teachings of the present invention and supported by a number of support functions 222 to be discussed in further detail below.
  • [0070]
    FIG. 2 a is a diagram illustrating another embodiment of service provider 102 of the present invention. In FIG. 2 a, service provider 102 is further equipped with content delivery engine 210 a to deliver contents to respondents 104 and 106, based on each respondent's preferred language. In one embodiment, content delivery engine 210 a is equipped to deliver electronic surveys of the present invention to one or more respondents. The content delivery engine 210 a dynamically constructs the preferred language based contents to be delivered using the language specific content elements stored in databases 212 a for example. More specifically, content elements stored in databases 212 a include graphics/images specifically designed for the various supported languages, textual message and pick list elements as well as static content pages expressed in the supported languages; and the content delivery engine also includes a main module (i.e., 220 a) incorporated with the teachings of the present invention, and supported by a number of support functions 222 a including in particular, a Create_Header function, a Get_Message function, and a Get_PickList Function to dynamically construct the required contents. As a result, content delivery engine 210 a may deliver content based on a respondent's preferred language for a large number of languages. In one embodiment, service provider 102 delivers dynamically constructed contents to respondents, based on a respondent's preferred language, supporting as many as 33 languages at the same time, a significantly higher number than any known prior art systems using prior art techniques. From the description to follow, those skilled in the art would appreciate that the present invention is highly scalable, and may easily be extended to support even more languages at the same time. In fact, there is no theoretical limit to the number of languages that can be currently supported by the present invention, as long as the necessary storage, computing resources, and communication bandwidth are available.
  • Preferred Language Based Content
  • [0071]
    Referring ahead to FIG. 11, an example of preferred language based content dynamically constructed in accordance with one embodiment of the present invention is illustrated. As illustrated, if included, graphics/images 1112 of preferred language based content 1110 are dynamically constructed from graphics/images specifically designed for the various languages. That is, different graphics/images sensitive to the culture and community standards of the various communities speaking the supported languages are provided for the different languages. Further, if included, textual contents 1114 of preferred language based content 1110 are dynamically constructed from message elements expressed in the supported languages. That is, the contents are constructed from individual translated messages expressed in the supported languages.
  • [0072]
    Moreover, if included, links 1116 are generated in the preferred language. That is, secondary contents are accessed with resource location names expressed in the supported languages. Similarly, if included, pick lists 1118 are ordered in view of the preferred language, and static pages 1124 of pop-ups 1122 are expressed in the preferred language. That is, pick list elements are ordered depending on whether the preferred language is an alphabet based or a character based language, and the options are ordered in accordance with the manner the language customarily orders items, e.g., alphabetically for English, by character strokes for Kanji (Japanese), and so forth.
  • [0073]
    Further, display space efficient drop-down list 1120 (as opposed to the prior art display space inefficient button/icon approach) is advantageously employed to facilitate a respondent in changing the preferred language.
  • [0074]
    Finally, if included, pop-up static pages 1124 are also generated in the preferred language. That is, different pop-up static pages expressed in the corresponding supported languages are employed for different preferred languages.
  • Operational Flow of Content Delivery Engine
  • [0075]
    FIG. 12 illustrates the operational flow of the relevant aspects of the content delivery engine including the main module of FIG. 2 for delivering content in a preferred language basis. As illustrated, at block 1202 main 220 (using, e.g., in-line instructions) sets one of the supported languages as the default language. In one embodiment, the default language is English. In other embodiments, the default language is the last language selected by the respondent as the preferred language. Various approaches known in the art may be employed to allow main 220 to remember the preferred language of a respondent across sessions.
  • [0076]
    At block 1204, main 220 sets the character code setting to the required character code set based on the preferred language. For the illustrated embodiment, main 220 sets the character code setting using the Create_Header support function. More specifically, in an embodiment, where the contents are encoded using the Hypertext Markup Language (HTML), main 220, using Create_Header function, sets the character code setting by generating a meta statement for the HTML header, specifying the required character code set.
  • [0077]
    Thereafter, at block 1206, if applicable, main 220 generates the graphics/images of the preferred language content, by retrieving appropriate ones of the stored graphics/images, which as described earlier, are specifically designed for the supported languages. For the illustrated embodiment, the different graphics/images are differentiated employing a graphic/image identification convention that includes the target language. Main 220 constructs the preferred language dependent graphic/image identifier to retrieve the appropriate ones of the graphics/images.
  • [0078]
    At block 1208, if applicable, main 220 generates the textual contents of the preferred language content, by retrieving appropriate ones of the stored message elements, which as described earlier, are expressed in the supported languages. For the illustrated embodiment, main 220 retrieves the appropriate ones of the message elements, using the Get_Message function.
  • [0079]
    At block 1210, if applicable, main 220 generates the links in the preferred language, by retrieving the appropriate language specific portion of the resource location names from databases 212, and combining them with the non-language specific portions to form the preferred language tailored links. For the illustrated embodiment, main 220 also retrieves the appropriate ones of the language specific portions of the resource location names, using the Get_Message function.
  • [0080]
    At block 1212, if applicable, main 220 generates the pick lists, by retrieving the pick list elements from databases 212, and ordering the retrieved pick list elements in accordance with the preferred language's ordering convention. For the illustrated embodiment, main 220 retrieves the appropriate ones of the pick list elements, and orders them, using the Get_List function.
  • [0081]
    At block 1214, main 220 generates the drop down language selection list to facilitate a respondent in changing the preferred language.
  • [0082]
    Thereafter, main 220 awaits for respondent interactions with the content displayed, block 1216. Upon receipt of respondent inputs, main 220 determines if the inputs are associated with the respondent selecting one of the languages listed on the drop down language selection list to change the preferred language, block 1218. If it is determined that the respondent is changing the preferred language, main 220 returns to block 1204 and re-performs the earlier described operations illustrated by blocks 1204-1218. If it is determined that the respondent is not changing the preferred language, the inputs are processed and handled in an application dependent manner, as in the prior art.
  • [0083]
    Accordingly, main 220, augmented by support functions 222, is able to dynamically generate contents based on a respondent's preferred language, supporting a large number of languages at the same time. These operations, and other related topics, are further discussed in co-pending U.S. patent application Ser. No. 09/744,773, entitled Content Delivery in a Preferred Language for a Large Number of Languages, filed contemporaneously, and having common inventorship with the present invention.
  • Operational Flow of Electronic Survey Engine
  • [0084]
    FIG. 3 illustrates the operational flow of the relevant aspects of the electronic survey engine of FIG. 2 including main module 220 for generating and delivering electronic surveys in accordance with one embodiment of the present invention. In one embodiment, electronic survey engine 210 generates and delivers electronic surveys utilizing markup languages such as hypertext markup language (HTML), extended markup language (XML), standard generalized markup language (SGML), and so forth, and/or scripting languages such as JavaScript and Pearl, for example.
  • [0085]
    In FIG. 3, it is assumed that the respondent has indicated a particular electronic survey by selecting the electronic survey from, e.g., a pick list or by entering the electronic study number directly into a web page, for example.
  • [0086]
    Other similar methods for indicating an electronic survey may also be utilized. As illustrated, at block 302, relevant characteristics for each question of the indicated survey are retrieved from, e.g., databases 212. In one embodiment, such characteristics include an identification number (i.e., question ID) for each question, a sequence number indicating each question's relative order in the electronic survey, and a response required characteristic indicating which (if any) individual questions require a response. In one embodiment, this and other data are stored in and retrieved from one or more tables or equivalent data structures within databases 212. At block 304, each question including its title, text and valid response options are retrieved from databases 212. The retrieval may be accomplished using the following query instruction:
    “SELECT question_title, question_text, question_type, question_text,
       valid_response, etc.
      FROM question_base
        WHERE question_id=$question_id[$j] AND
        language =‘$lang’”
  • [0087]
    At block 306 a determination is made as to whether a translation is available for the question. If the query (shown above) retrieves only one or no results, it is an indication that the question is only available in a default language (i.e., English) and that question is therefore selected, block 310. If, however, the query results indicate that one or more translations of the question are available, the routine attempts to retrieve the appropriate question translation corresponding to the respondent's preferred language as indicated by, e.g., the $lang variable, block 308.
  • [0088]
    Next, at block 312, the selected question and its valid response options are displayed according to the type of question it is. For example, if the question is a free-form text (FFT) question type, the question is displayed in a first manner (e.g., print qq(<center><TEXTAREA name=“f$question_id[$j]” cols=“60” rows=“5”></TEXTAREA></center></td></tr>); print “\n”;), whereas if the question is a multiple choice question type (MCM or MCS) or a semantic differential (SDI) question type, the respective questions may be formatted differently.
  • [0089]
    As described herein, an MCM question type generally allows a respondent to select multiple or no answers, whereas an MCS question type generally allows a respondent to select only a single response. In one embodiment, MCM question types are implemented via checkbox form elements, whereas MCS question types are implemented via radio button form elements. Valid response options for MCM and MCS type questions may be retrieved via the following series of instructions:
  • [0090]
    “SELECT response_id, response_text
      • FROM resp_short
        • WHERE question _id=$question_id[$j] AND language=‘$this_lang’
        • ORDER BY response_id”
  • [0094]
    if ($question_type eq ‘MCM’){$response_id=‘f’. $question_id[$j]. ‘,’. $response_id; $value=‘Y’;}
  • [0095]
    else {$value=$response_id; $response_id=‘f’. $question_id[$j];}
  • [0096]
    print qq(<input type=“$form_input” value=“$value” name=“$response_id”></td>);
  • [0097]
    SDI type questions enable a respondent to respond to a question based upon a weighted scale. In one embodiment, SDI type questions are each presented with one or more statements to which the respondent is given the opportunity to enter their level of agreement. FIG. 4 is a graphical illustration of a semantic differential (SDI) question in accordance with one embodiment of the invention. Referring briefly to FIG. 4, question 402 is displayed in window 400 along with associated statements 1-N. Displayed adjacent to each of statements 1-N are corresponding valid response options indicated by radio buttons 404. Each of the respective response options (e.g., radio buttons 404) are weighted on a numeric scale, corresponding to labels ranging from strongly agree to strongly disagree. Valid response options for SDI question types may be retrieved via the following series of instructions:
  • [0098]
    $response_id=‘f’. $question_id[$j]. ‘,’. $response_id;
  • [0099]
    for ($y=1; $y<=5; $y++)
  • [0100]
    print qq(<input type=“radio” value=“$the_val” name=“$response_id”>);
  • [0101]
    print “\n”;
  • [0102]
    Additionally, dialog button 406 is dynamically generated to display a context appropriate message. For example, if the currently displayed question is the next to last question, “next question” is displayed, whereas if the currently displayed question is the last question, “Finish” is displayed.
  • [0103]
    Referring back to FIG. 3, once the questions and responses are displayed (block 312), the routine detects whether the respondent wishes to proceed to the next question, e.g., by detecting the respondent's clicking on dialog button 406 (block 313), and upon so detecting, performs a check function which checks whether a respondent has in fact responded to the question, block 314. If the respondent has responded to the question, the system determines whether the response is valid given the valid response options, block 318. If the response is not valid, the current question and valid response options are redisplayed, whereas if the response is valid, the next question and corresponding valid response options are displayed. Similarly, if the respondent has not responded to the question (block 316) and it is determined that a response is required (e.g., via the response_required field), the current question and valid response options are redisplayed. If, however, it is determined that a response is not required (block 316), the next question and corresponding response options are displayed, block 320.
  • Displaying a Product and/or Marketing Concept
  • [0104]
    In displaying a concept to a respondent, it is often important to convey sufficient information to enable the respondent to understand the idea that is being presented as well as to assess whether this idea would be appealing to them. In one embodiment of the invention, this is accomplished by including within the concept display information that is comprised of some combination of text, images, and pricing data. Additionally, a variety of fonts, styles, and font sizes are utilized in order to emphasize and enhance certain information and text. Moreover, concepts may include images that are either still or animated and may incorporate sound, video, or other multimedia information to enhance the virtual experience intended to be conveyed by the concept. As will be discussed in further detail below in accordance with one embodiment of the invention, various price points may be generated and displayed in association with a concept in order to further enhance the research value of the concept. In one embodiment, the price points are adjusted for the respondent's geographic location (i.e., country) and are displayed in the respondent's national currency.
  • [0105]
    In one embodiment, the concept is divided into multiple sections with each section of the concept capable of displaying a unique font, font size, and/or font style. Briefly jumping ahead, a graphic illustration of a marketing concept including differently formatted sections is shown in FIG. 6. Marketing concept 600 is shown including sections 1-4 and concept price 602. In accordance with one embodiment of the invention, concept price 602 is randomly generated based at least in part upon provided pricing constraints.
  • [0106]
    Referring now back to FIG. 5, an operational flow of for displaying concept-specific marketing information in accordance with one embodiment of the invention is illustrated. At block 502, section-specific information for an identified concept is retrieved from a data store such as databases 212. The retrieval may be accomplished using the following query instruction:
    “SELECT section_type, section_text, section_font, section_fontsize,
    section_style, section_alignment, section_action, image_id
      FROM concept_sections
        WHERE concept_id=$concept_id AND
        concept_lang=‘$lang’ ORDER BY section_sequence”
  • [0107]
    In accordance with one embodiment of the invention, and as depicted in the query language shown above, the section-specific information is retrieved in accordance with the preferred language of the respondent. Once the section-specific information has been retrieved, the text associated with each section (i.e., $section_text) is appropriately formatted and displayed based upon the retrieved formatting information, block 504. At block 506, a price point is generated for the product concept illustrated and described by the various retrieved text and image sections. Price point generation will be discussed below with respect to FIG. 7. Once a price point has been generated, a financial incentive to be awarded to a respondent in exchange for the respondent's participation in the study (i.e., by responding to the electronic survey) is determined and displayed in conjunction with the generated price point, block 508. In one embodiment, the financial incentive is determined as a function of the generated price point. Once the price point and financial incentive are determined, either one or both are adjusted to take into account the respondent's geographic location such as the respondent's country of residence, blocks 510 & 512.
  • [0108]
    Because different economies have developed at different rates, it is sometimes desirable to adjust the financial incentive offered to respondents to more accurately reflect local norms such as cost of living, competitive conditions, etc. of the country within which the respondent resides. In one embodiment of the invention, the price point generated at block 506 is converted from the default currency, such as U.S. dollars, to a currency that corresponds to the respondent's geographic location as indicated by $contact_country. In one embodiment, the respondent's currency, associated symbol, and conversion rates are retrieved from a database, such as databases 212, in order to perform the appropriate conversion. The retrieval may be accomplished using the following query instruction:
    “SELECT currency, symbol, conversion
      FROM currencies
        WHERE country =‘$contact_country’”
  • [0109]
    In one embodiment of the invention, the base level of incentive offered to a respondent is adjusted by assigning a coefficient to the incentive to either increase or decrease the incentive according to, e.g., the respondent's geographic location. For example, such functionality may be implemented through the following exemplary instructions:
    if ($the_country eq ‘RU’ $the_country=‘URE’)
     {$factor=.6;}
    elsif ($the_country eq ‘JP’)
     {$factor=1.3;}
    else
     $factor=1.0
  • Price Point Determination
  • [0110]
    When conducting consumer research, it can be desirable to set a plurality of price points for a product concept being marketed. In order to preserve the efficacy of the data when testing with multiple price points, however, precaution should be taken against randomly assigned prices to a product every time the respondent takes or participates in the survey. This is because some respondents will learn that the price is reset upon each restart of the survey. Accordingly, without further precautions, respondents would be able to refresh the web page displaying the electronic survey multiple times in order to continually regenerate the price point until an acceptable one is displayed. To guard against this occurrence, in accordance with one embodiment of the invention, pricing information is set once per respondent. In one embodiment of the invention, each price point is generated based upon pre-set pricing constraints specific to the product concept being surveyed. In one embodiment, the generated price points are balanced along discrete intervals so that response information can be correlated versus pricing information that the respondent viewed in forming a decision about the concept that was displayed to them.
  • [0111]
    In FIG. 5, an operational flow for displaying concept-specific marketing information including a price point was shown. FIGS. 7(A-C) illustrate various operational flows for determining such a price point (such as that of FIG. 5) in accordance with one embodiment of the invention. Referring now to FIG. 7A, pricing constraints associated with the product concept being surveyed are first retrieved using the &lookup_study subroutine (shown in FIG. 7B). Basically, the &lookup_study subroutine retrieves pricing constraints such as a maximum price, a minimum price, and various pricing intervals defined to exist there between. Once the pricing constraints have been retrieved, a price is generated for the product concept based at least in part upon the retrieved constraints (discussed further in FIG. 7C). Once the price has been generated, it is stored in association with a respondent identifier and the corresponding survey identifier, block 706. Exemplary instructions to implement such a store procedure are:
  • [0112]
    “REPLACE into invite (study_number, user_id, invite_date, price_point,started) VALUES ($study_number,$contact_id[$i], ‘$mdate’, $the_price,”)”
  • [0113]
    FIG. 7C illustrates an operational flow for generating a price for a product concept as mentioned above with respect to FIG. 7A, in accordance with one embodiment of the invention. In one embodiment, minimum and maximum price constraints (e.g., as determined from the &lookup_study function of FIG. 7B) are passed to the &price_point function when it is called. At block 708, the routine determines whether the pricing constraints include any inadvertent errors. If so, the price is assigned to be equal to $0, block 710, which value is then returned to the calling function, block 712. If, however, the conditions of block 708 are met (namely, ($the_price_points>0) && ($the_min_price>0) && ($the_max_price>=$the_min_price)), then $the_range is calculated to be the difference between the minimum and maximum price constraints, and $interval_value is obtained from the division of $the_range by the number of predetermined price points, block 714. If the minimum price constraint is greater than ‘100’, the price is rounded to the nearest integer (block 722), whereas if the minimum price constraint is less than ‘100’, the price is not rounded, block 718. In either case, the generated price is then returned to the routine that called the function block 720, 724.
  • Electronic Confidentiality Disclosure Agreement
  • [0114]
    In some instances, where a survey is being conducted covering a topic that is deemed confidential, it is desirable to use a confidential disclosure agreement (CDA). In one embodiment of the invention, a CDA is administered online providing a traceable record of when and who agreed to the terms described in a CDA. FIG. 8 illustrates an operational flow for administering an electronic on-line CDA in accordance with one embodiment of the invention. To begin, the CDA is displayed in the preferred language of the respondent (blocks 802, 804). Once he respondent has read and agreed to the various terms of the CDA, they are instructed, for example, to enter a unique password and select an “accept” button in order to indicate their acceptance to the terms of the CDA. Once such an acceptance is detected, block 806, the system verifies that the user has provided a proper password, block 808. Although not required, the password feature provides a mechanism by which to authenticate that the respondent agreeing to the terms of the CDA is in fact who they purport to be. If the identity of the respondent cannot be authenticated, block 810, the respondent is asked to again enter their password. If, however, the respondent is authenticated, a record of the respondent's acceptance is stored in a table, block 812. In one embodiment, the respondent's name, study identifier, as well as the date and time when the respondent accepted the displayed terms is recorded.
  • Participant Selection
  • [0115]
    In accordance with the teachings of the present invention, once an electronic survey has been generated using, for example, the language specific content elements described above, the electronic survey is delivered to one or more identified survey participants. In one embodiment a service provider, such as multi-regional service provider 102 of FIG. 1, includes electronic survey delivery services for facilitating the selection of one or more survey participants from a multi-regional pool of potential panelists to whom the electronic survey shall be distributed, and for facilitating the distribution of the electronic survey to the selected one or more survey participants. In one embodiment, the service provider includes a search engine for searching through the multi-regional pool of potential panelists to identify one or more panelists to participate in the electronic survey based upon provided search criteria, such as participant country, language, age, interests, economic status, and so forth. In one embodiment, panelists matching at least a portion of the provided search criteria are identified and displayed to a research client, for example, to be selected for participation in the electronic survey. In an alternative embodiment, panelists matching at least a portion of the provided search criteria are automatically selected by the service provider, for example, for participation in the electronic survey.
  • [0116]
    In one embodiment of the invention, several graphical user interface screens are provided to facilitate entry of the search criteria and display of search results. FIG. 13 is a graphical illustration of a screen for providing search criteria for use in selecting panelists in accordance with one embodiment of the invention. As illustrated in FIG. 13, a number of search criteria may be entered as part of a query for selecting survey participants. For example, such queries may be tailored based upon panelist identity information 1302, panelist country, language and interest data 1304, or by other preferences such as those shown in FIG. 13 (1306). For example, based upon the search criteria entry screen of FIG. 13, a query may be constructed that will search for all panelists over a particular age from Angola who are interested in advertising. In one embodiment, if checkbox 1308 is selected, the panelists resulting from the constructed query (i.e., matching the query criteria) are automatically selected as participants in the electronic survey, whereas if checkbox 1308 is not selected, the resulting panelists are displayed in a list from which the participants can be manually selected.
  • [0117]
    FIG. 14 is a graphical illustration of a summary page displaying a list of panelists meeting search specified criteria in accordance with one embodiment of the invention. According to one embodiment of the invention, checkboxes 1402 are provided to enable manual selection of one or more participants from the list of panelists resulting from the previously described query. In one embodiment, checkboxes 1402 are checked by default, whereas in an alternative embodiment checkboxes 1402 are unchecked by default. It should be noted that checkboxes 1402 could similarly be replaced by other graphical selection mechanisms known in the art, such as radio buttons, for example.
  • [0118]
    In addition to selecting/deselecting a participant from the resulting list of panelists as described above, a research client may view further details associated with one or more panelists and/or provide additional criteria to further refine the list. For example, a research client can highlight an entry in the list and press a “continue” button to view more details about the associated panelist. Any number of details can be provided as discussed below with respect to FIG. 15. Likewise, any number of alternate approaches can be used to view details such as “double clicking” on an entry of interest.
  • [0119]
    Panelists can also be assigned grades or ranks in the search result summary. For example, based on any number of criteria, such as information derived from responses made to previous research studies, each panelist may be assigned a grade or rank to categorize the panelist according to a subjective quality grade. In FIG. 14, for example, column ‘Q’ (1404) is provided to facilitate such categorization. In various embodiments, a research client can automatically select groups of listed panelists by rank. For example, once at least some of the panelists have been ranked, depending upon the number of participants desired, a research client may opt to select all panelists with the rank of “A.” If the research client determines that the resulting group of panelists is not large enough, the research client may go back and select all panelists having an “A” or “B” grade, for instance.
  • [0120]
    Once one or more panelists are selected from the resulting list of panelists, the selected panelists are invited to participate in an electronic survey. The research client may potentially have a number of electronic messages or invitations from which to select. For example, while creating an electronic survey, a research client may have generated a number of different invitations offering to a participant various levels of incentives such as compensation for participation in the electronic survey. One panelist who has received a high rank or grade, for example, may receive a larger incentive than a panelist that was ranked or graded lower. In one embodiment, invitation messages are independently selectable based upon a number of criteria. In various embodiments, invitation messages may be automatically selected based upon a rank or grade of a panelist, whereas in other embodiments invitation messages may be automatically selected based upon some other criteria such as age, economic status, location, ethnicity, religion, political affiliations, and so forth.
  • [0121]
    A research client may have created or have available a number of different electronic surveys or variations of electronic surveys from which to choose to distribute to one or more selected panelists. In one embodiment, the research client may assign a panelist to a particular survey by entering an electronic survey code. Any number of criteria can be used to match surveys to panelists. For example, panelists offered a bigger incentive to participate may be assigned a longer, more detailed survey. Alternatively, a panelist may be offered multiple surveys having different levels of incentive for each.
  • [0122]
    FIG. 15 is a graphical illustration of a panelist details page in accordance with one embodiment of the invention. A research client may use such details page to view details about a particular panelist and/or change certain panelist information. For example, a details page may include ranks or grades assigned to the panelist by previous research clients based upon subjective quality of data provided in previous surveys. A research client may be able to assign a new grade or change a previously assigned grade. A research client, depending upon the access authority, may also be able to change or update other information about a panelist, such as their address, age, and other demographic information, for example. In one embodiment, hypertext links to one or more of the electronic surveys previously completed by the panelist are provided.
  • Capturing Results
  • [0123]
    Once the electronic survey questions have been generated and delivered to the participant, and the participant (now turned respondent) has had an opportunity to answer the questions as described with respect to FIG. 3, it is then necessary to store the respondent's responses in a database. In one embodiment, a list of questions that are included in the survey is read in as, an operating system environment, parameters submitted via a browser URL, an HTML field, or another stored source, for example. In one embodiment, this information is utilized to read in the corresponding responses for each of the questions using, for example, a CGI query. In one embodiment, the results are stored in a single database having one or more tables to collect the responses, depending upon the type of question asked.
  • [0124]
    FIGS. 9A and 9B illustrate an exemplary operational flow of a result capturing process in accordance with one embodiment of the invention. The total number of questions involved in the electronic survey is determined at block 902, and the type of each question is determined at block 904. In one embodiment, each question is characterized as being an MCM type question, an MCS type question, a FFT type question, or an SDI type question. If the question is determined to be an MCS type question (block 906), the respondent's response is queried (block 914) and then stored (i.e., replaced) into a table named “tab_results” along with the respondent ID, study number, and response date, for example, block 916. If the question is determined to be an FFT type question (block 908), the respondent's response is likewise queried (block 918) and then stored along with the corresponding respondent ID, study number, and response date data items. For the FFT question type, however, the response data is stored into a second table named “text_results”. If the question is determined to be an MCM type question (block 910), each response ID and corresponding response text for the question are first retrieved in the respondent's preferred language from another table, block 922. For example, the following instructions may be used to perform such a retrieval from a table named resp_short:
  • [0125]
    “SELECT response_id,response_text FROM resp_short WHERE question_id=$quest_id AND language=‘$lang’ ORDER BY response id”);
  • [0126]
    Because multiple results are possible with the MCM type question, the result query varies slightly from the previous response queries, block 924. If the MCM query results in a value of “Y” indicating that at least one response has been selected, the selected responses and supporting data are stored (i.e., replaced) into the tab_results table (block 926) as were the MCS question type response data. Lastly, if the question is determined to be an SDI type question (block 912), the same query illustrated above with respect to the MCM question type is performed with respect to the SDI question type, block 928. Likewise, the result query of an SDI type question (block 930) is formed substantially similar to that of the MCM type question (block 924), and the data is stored in a tab_results table much like MCM question types, block 932.
  • Exemplary Computer System
  • [0127]
    FIG. 10 illustrates an example computer system suitable for use to practice the present invention, in accordance with one embodiment. Computer system 1000 may represent any of a wide variety of devices such as but not limited to conventional desktop computers, laptop computers, handheld computers, personal digital assistants (PDAs), wireless telephones utilizing i-Mode and/or wireless access protocols (WAPs), and so forth. As shown, computer system 1000 includes one or more processors 1002 and system memory 1004. Additionally, computer system 1000 includes mass storage devices 1006 (such as diskette, hard drive, CDROM and so forth), input/output devices 1008 (such as keyboard, cursor control and so forth) and communication interfaces 1010 (such as network interface cards, modems and so forth). The elements are coupled to each other via system bus 1012, which represents one or more buses. In the case of multiple buses, they are bridged by one or more bus bridges (not shown). Each of these elements performs its conventional functions known in the art. In particular, system memory 1004 and mass storage 1006 are employed to store a working copy and a permanent copy of the programming instructions implementing the multi-region market research processing service of the present invention. The permanent copy of the programming instructions may be loaded into mass storage 1006 in the factory, or in the field, as described earlier, through a distribution medium (not shown) or through communication interface 1010 (from a distribution server (not shown)). The constitution of these elements 1002-1012 are known, and accordingly will not be further described.
  • [0128]
    While the present invention has been described in terms of the above illustrated embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described. The present invention can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of restrictive on the present invention.
  • Multi-Region Market Research Study Creation DETAILED DESCRIPTION OF THE INVENTION
  • [0129]
    In the following description, various aspects of the present invention will be described. However, it will be apparent to those skilled in the art that the present invention may be practiced with only some or all aspects of the present invention. For purposes of explanation, specific numbers, materials and configurations are set forth in order to provide a thorough understanding of the present invention. However, it will also be apparent to one skilled in the art that the present invention may be practiced without the specific details. In other instances, well known features are omitted or simplified in order not to obscure the present invention.
  • [0130]
    Parts of the description will be presented in terms of operations performed by a processor based device, using terms such as data, tables, requesting, determining, retrieving, displaying, and the like, consistent with the manner commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. As well understood by those skilled in the art, the quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through mechanical and electrical components of the processor based device; and the term processor includes microprocessors, micro-controllers, digital signal processors, and the like, that are standalone, adjunct or embedded.
  • [0131]
    Various operations will be described as multiple discrete steps in turn, in a manner that is most helpful in understanding the present invention, however, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations need not be performed in the order of presentation. Further, the description repeatedly uses the phrase “in one embodiment,” which ordinarily does not refer to the same embodiment, although it may.
  • Overview
  • [0132]
    Referring now first to FIG. 16, wherein a block diagram illustrating an overview of the present invention, in accordance with one embodiment, is shown. As illustrated, multi-region market research service provider 1602 is advantageously provided with the multi-region market research study creation service of the present invention. As will be described in more detail below, the multi-region market research study creation service (hereinafter simply study creation service) includes a number of novel services, in particular, region wide target geography selection, real time costing, automatic notification to translation service providers, and services for managing translation, as multi-region study often involves translation of study elements.
  • [0133]
    As shown, the computing equipment of multi-region market research clients, specifically, creators of the multi-region studies, 1604, are coupled to the computing equipment of multi-region market research service provider 1602 via networking fabric 1608. Similarly, the computing equipment of translation service providers 1606 are also coupled to the computing equipment of multi-region market research service provider 1602 via networking fabric 1608.
  • [0134]
    Through their respective equipment, study creators 1604 and translation service providers 1606 use the study creation services provided by market research service provider 1602 to create their multi-region market research studies, and provide translation services respectively, thereby allowing multi-region market research studies, in particular, those involving multiple target languages and translations, to be created much more efficiently.
  • [0135]
    Except for the novel services incorporated with the study creation service of multi-region market research service provider 1602, the various equipment used by multi-region market research service provider 1602, market research clients 1604, and translation service providers 1606, as well as networking fabric 1608 are intended to represent a broad range of these elements known in art. Examples of computing and networking equipment suitable for practicing the present invention include but are not limited to various palm-sized, notebook size, or desktop computers available from e.g., IBM of Armonk, N.Y., servers available from Sun Microsystems of Mountain View, Calif., and routers/switches available from CISCO Systems of San Jose, Calif. The functions and constitutions of these elements are known, accordingly they will not be further described.
  • [0136]
    The study creation method of the present invention, including the study creation service of service provider 1602, and other related subjects will be described in turn below. [Note that multi-region market research provider 1602 may be a “single” entity or an “alliance” of affiliated entities.]
  • Method
  • [0137]
    Referring now to FIG. 17, wherein a block diagram illustrating a method view of the study creation method of the present invention, in accordance with one embodiment, is shown. As illustrated, at block 1702, market research client 1604 uses the study creation service of multi-region market research service provider 1602 to create a multi-region market research study. One embodiment of the process for creating such study is illustrated in FIG. 18, to be described more fully below. Upon creation of the multi-region market research study, in accordance with the present invention, at block 1704, the study creation service advantageously provides real time cost estimates for the study being created for market research client 1604. In one embodiment, the cost factors considered in computing the real time cost estimates include at least the number of panelists selected, including their location distributions, i.e., the regions/countries/cities the study is to be conducted in as well as the amount of translations required, which includes the number of questions, messages, concepts, and so forth, as well as the number of languages to be translated into.
  • [0138]
    Thereafter, at block 1706, the study creation service facilitates market research client 1604 making adjustments to the study being created, e.g., reducing/increasing the number of panelists or the number of study regions/countries/cities, or reducing/increasing the amount of translations (by reducing/increasing the number of questions, messages, concepts, and so forth, or the number of languages to be translated into). As before, in response to the adjustments, the study creation service provides real time feedback to market research client 1604 on what the new cost is
  • [0139]
    Eventually, when the user finishes making adjustments (if any) to the study being created, at block 1708, the study creation service determines if any of the study elements require translation. In one embodiment where the “standard” language is English, translation is required whenever a non-English speaking region, or a country or a city within such a region, is selected as one of the target geography of the study, and the study elements have not been previously translated (by either service 1602 or creator 1604). In one embodiment, the study elements may include study questions, study messages, study pick lists and/or study concepts.
  • [0140]
    If it is determined that translation is required for at least some of the study elements, at block 1710, the study creation service automatically notifies translation service providers 1606 of the various target languages. The identities, communication addresses, and communication medium of translation service providers 1606 for the various languages may be predetermined and/or pre-provisioned in any one of a number of known ways, e.g., through pre-registrations. In a preferred embodiment, a large number of languages, as many as upward of 30, are supported, with one or more translation service providers “registered” for each of the supported languages. In one embodiment, the translation service providers are notified via emails. The emails are advantageously constituted in a language dependent manner. Such emails are the subject of co-pending U.S. Patent Applications, PCT US00/41973 (Ser. No. 10/031,263), entitled Language Sensitive Electronic Mail Generation And Associated Applications, filed contemporaneously, and having common inventorship with the present invention. Except for any circular incorporation, the application is hereby fully incorporated by reference.
  • [0141]
    Thereafter, the study creation service offers various utilities for the study creator to monitor the status of the translations, block 1712. In one embodiment, the study creation service offers a single summary view for the translation status of all study elements requiring translation for all languages. Eventually, when all translations are completed (or if it was earlier determined that no translation is required), the process continues at block 1714, where the created study is conducted, data collected, analyzed and reported.
  • [0142]
    In various embodiments, conducting the study, and analyzing/reporting on the study are performed as described in co-pending U.S. Patent Applications, PCT US00/30583 (Ser. No. 09/744,776), entitled Multi-Region Market Research Study Processing, and PCT US00/30652 (Ser. No. 09/744,775), entitled Reporting And Analyzing Data From A Multi-Region Research Survey, both filed contemporaneously, and have common inventorship with the present invention. Except for any circular incorporation, these co-pending applications are also hereby fully incorporated by reference.
  • Functions and Operations for Creating a Market Research Study
  • [0143]
    FIG. 18 illustrates the operation flow of the study creation service of the multi-region market research service 1602 of FIG. 16, in accordance with one embodiment. As illustrated, study creation service includes a first function that facilitates the study creator in first defining the research method of the study, block 1802. Examples of study methods include but are not limited to complaint monitor, electronic concept test, electronic copy test, habits and practice test, paired comparison test, satisfaction test, single product test, and so forth. Study creation service also includes a second function that facilitates the study creator in selecting the target geography of the study, block 1804. As will be described in more detail below, in one embodiment, the study creation service is advantageously equipped to support implicit target geography selection for a country or a city within a region of the globe.
  • [0144]
    Further, study creation service includes a third function that facilitates the study creator in providing various details on the involved products of the study, block 1806. Examples of these product details include but are not limited to the form of each of the involved products, the average weight of each of the involved products, the duration of usage for each of the involved products, and so forth. A fourth function that facilitates the study creator in specifying the panelists (also known as interviewees or study subjects), block 1808, is also provided. Examples of panelist specification include but are not limited to the size of the panel, their statistical distribution in terms of demographics, and so forth.
  • [0145]
    In one embodiment, a fifth function that facilitates the study creator in segmenting the panel, block 1810, is also provided. A panel may be segmented in terms of one or more demographic factors, such as, age, education, household income, and so forth. Lastly, for the illustrated embodiment, a sixth function that facilitates the study creator in defining which survey/concept to use, and whether translation of the survey/concept elements are necessary, block 1812, is also provided.
  • Target Geography Selection
  • [0146]
    As described earlier, one of the novel services offered by the study creation service of multi-region market research service provider 1602 is the service that supports implicit selection of target geography by regions. FIG. 19A illustrates an example user interface suitable for use to practice the implicit country/city selection within a region of the present invention, and FIG. 19B illustrates the operation flow of the relevant aspects of the associated functions in support of such advantageous implicit selection.
  • [0147]
    As illustrated, interface 1900 advantageously summarizes and presents the study creator with a small number of global study regions 1904. For the illustrated embodiments, 10 regions, North America, European Union, G-7 countries, and so forth are presented. Additionally, a scrollable list of countries supported 1908 as well as a scrollable list of cities supported 1910 are also presented. In response to the study creator selecting one of the regions, e.g., Greater China, the supported countries within the selected region, e.g., Taiwan, and the supported cities within the selected region, e.g., Shanghai, China, Shenyan, China, and Taipei, Taiwan, in the corresponding lists 1908-1910 are highlighted for the study creator. Thus, upon selecting a region of interest, a study creator may immediate be appraised of the target countries and/or cities the study creator may conduct the defined study. In a preferred embodiment, the study creator may also deselect any of the highlighted countries/cities that are of no interest to the study creator.
  • [0148]
    As illustrated in FIG. 19B, associated with example user interface 1900 are a number of Region_Checked functions, which as shown, are invoked in response to the study creator selecting one of the enumerated global regions, block 1922. Upon invocation, each of the Region_Checked functions determines if its corresponding region has been selected by the study creator, block 1926. If so, each of the Region_Checked functions select the corresponding countries/cities within the selected region, and highlight the selection for the study creator, block 1928, otherwise, no actions are taken, block 1930.
  • [0149]
    The determination and selection may, e.g., be accomplished using instructions similar to the example instructions below (for checking whether the European region was selected):
    function eu_change( )
    {
     if (document.forms[0].tgt eu.checked) {
     document.forms[0].tgt_country.options[3].selected = true //Austria
     document.forms[0].tgt_country.options[4].selected = true //Belgium
     document.forms[0].tgt_country.options[11].selected = true //Denmark
     document.forms[0].tgt country.options[13].selected = true //Finland
     document.forms[0].tgt_country.options[14].selected = true //France
     document.forms[0].tgt_country.options[22].selected = true //Ireland
      etc.
     document.forms[0].citylist.options[2].selected = true //Barcelona
     document.forms[0].citylist.options[11].selected = true //Essen
     document.forms[0].citylist.options[16].selected = true //London
     document.forms[0].citylist.options[21].selected = true //Madrid
     document.forms[0].citylist.options[22].selected = true //Manchester
     document.forms[0].citylist.options[25].selected = true //Milan
     document.forms[0].citylist.options[30].selected = true //Paris
     }
    }
  • [0150]
    Thus, it can be seen under the present invention, a multi-region market research study creator 1604 can easily discern where in the various regions/countries/cities he/she can conduct his/her market research through the services offered by market research service provider 1602, thereby improving the efficiency in the manner creator 1604 may create multi-region market research studies.
  • Real Time Costing
  • [0151]
    As described earlier, one of the most troublesome problems that confronts a multi-region market research study creator is the problem of controlling the cost of a large multi-region market research study, preferably early in the development cycle of the market research study, and one of the novel services offered by the study creation service of multi-region market research service provider 1602 is the service that supports real time costing of a multi-region market research study being created, thereby enabling a multi-region market research creator 1604 to be in better position in controlling the cost of such study. FIG. 20A illustrates an example user interface suitable for use to practice the real time costing feature of the present invention, and FIG. 20B illustrates the operation flow of the relevant aspects of the associated functions in support of such advantageous costing of a multi-region market research study.
  • [0152]
    As illustrated, interface 2000 advantageously summarizes a number of attributes of the market research being created. In particular, for the illustrated embodiment, these attributes include the type of research design selected (2004), e.g., “paired comparison-acceptance test,” the geographic focus (2006), i.e., the regions, countries and cities the study is to be conducted, the amount of product usage (2010), the methodology (2012), i.e., the amount of panelists and their distribution by regions/countries/cities, and the number of “documents” (2014), i.e., the number of questions, concepts, pick lists, messages and so forth, and for how many languages the “documents” are to be translated into. In accordance with the present invention, based on at least selected ones of these factors, study creation service 1602 computes a cost estimate 2016 in real time for market research study creator 1604. In one embodiment, as described earlier, study creation service 1602 takes into account at least the number of panelists in each of the regions/countries/cities a study is to be conducted, and the amount of translations (i.e., the number of messages, questions, pick lists, and so forth, and the number of languages involved). In one embodiment, study creation service 1602 computes the cost estimates by correspondingly multiplying the “attributes” with their cost units, and summing the various cost components.
  • [0153]
    As illustrated in FIG. 20B, in response to a request to display a summary of a market research study being created, at block 2022, study creation service 1602 displays a summary of the “attributes” of a study being created, as described earlier, including in particular, a cost estimate of the study, taking into consideration at least the earlier described cost factors. Thereafter, at block 2024, study creation service 1602 awaits for user inputs. Upon receipt of a user input, study creation service 1602 determines if the user input is associated with the user making adjustment to any of the cost factors, i.e., number of panelists, number of regions/countries/cities, number of questions, messages, etc., and number of languages involved. If the user input is associated with the user making adjustments to one or more of these cost factors, the adjusted cost factors are updated, and the estimated cost is re-calculated in real time to reflect the adjustment made. If the user input is not associated with the user making adjustment to any of the cost factors, the input is handled in an application dependent manner as in the prior art.
  • [0154]
    Thus, it can be seen under the present invention, a multi-region market research creator 1604 has much better control, right up front at the time a study is created, on the cost of conducting a large multi-region market research study involving many regions/countries/cities, a large number of panelists and numerous languages.
  • Sample Data Organizations
  • [0155]
    Skipping to FIG. 26C, wherein a diagram illustrating an example data organization suitable for use to store the cost factors, for practicing the present invention, is shown. As illustrated, cost factor table 2650 includes columns 2652 for storing the “panelist” cost units for the various regions, countries, and/or cities, e.g., $p1 per 100 panelists in China or $p2 per 100 panelists in Shanghai, and so forth. Cost factor table 2650 also includes columns 2654 for storing the “translation” cost units for the various languages, e.g., $q1 per 200 messages for Chinese-Modern or $q2 per 50 questions for Chinese-Traditional, and so forth, and $r1 per 200 messages for Chinese-Modern or $r2 per 50 questions for Chinese-Modern, and so forth. In alternate embodiments, other cost factors as well as other cost units may be employed instead.
  • On-Line Translation Check-In
  • [0156]
    As described earlier, one of the novel services offered by the study creation service of multi-region market research service provider 1602 is the service that supports on-line check in of translated study elements by translation service providers 1606. As those skilled in the art would appreciate, the novel service is extremely useful, especially when a large number of languages are supported.
  • [0157]
    FIG. 21 illustrates an example user interface suitable for use to practice the on-line translation check in aspect of the present invention, and FIG. 22 illustrates the operation flow of the relevant aspects of a function in support of such advantageous on-line check in. As illustrated, interface 2100 includes display fields 2104-2106 displaying the study element identifier, and the study element text in the original language, which are especially useful for the translation service providers when inputting the translated study elements. Interface 2100 advantageously provides input field 2108 proximately placed adjacent to display fields 2104-2106 for translation service providers 1606 to enter the translated study elements for the target languages, upon completing the translations. As described earlier, study elements may include study questions, study messages, study lists, study concepts, and the like.
  • [0158]
    As illustrated in FIG. 22, associated with example user interface 2100 is a function, which as shown, is invoked in response to the submission of the translated study elements of a target language by a translation service provider 1606. Upon invocation, the associated function sets the current target language to the target language for which the translation service provider is submitting translated study elements, block 2202. For the illustrated embodiment, thereafter, the function advantageously processes the submitted translated study elements by study element types, i.e., study questions, study messages, and so forth. For each of these study element types, the function processes the study elements of the study element type, element by element, e.g., question by question, message by message, and so forth, blocks 2204-2210. For each study element, the function retrieves the study element's properties from the original language version, e.g., English, assigns the retrieved study element properties to the target language version (i.e., letting the target language version inherit the retrieved properties), and then saving the target language version into a corresponding study element table (i.e., the question table, the message table, and so forth).
  • [0159]
    The afore described retrieving, assigning and saving operation may, e.g., be accomplished with instructions similar to the example instructions below (for processing a question):
  • [0160]
    SELECT question_text, question_type, access, owner, industries, res_types, question_title FROM question_base WHERE question_id=$question_id[$i] AND language=‘$original_language’;
  • [0161]
    REPLACE INTO question_base (question_id, language, question_text, question_type, access, owner, question_title) VALUES ($question_id[$i], ‘$target_language’, ‘$result id[$i]’, ‘$old_question_type’, ‘$old_access’, $old_owner, ‘$old title’);
  • [0162]
    where “question_base” is the example name of the question table, and “question_type”, “access”, “owner” etc. are question attributes, and $result_id[$i] is the variable holding the translated question text for the current question being processed.
  • [0163]
    As those skilled in the art would appreciate, the storage organization storing by study element types, independent of target language, i.e., all questions stored in a question table regardless of the language version, all messages stored in a message table regardless of the language version, and so forth, provides for an efficient way of storing the large quantities of study elements (since a large number of languages are supported), as the storage structures (more specifically, the tables) may be normalized.
  • [0164]
    Thus, it can be seen under the present invention, the manner in which the various translation service providers 1606 provide the translation results to a market research study client 1604 is much more efficient.
  • Sample Data Organizations
  • [0165]
    Skipping to FIGS. 25A-25B and 26A-26B, wherein four diagrams illustrating four example data organizations suitable for use to store the message elements, the pick list elements, the question elements, and the concept elements, for practicing the present invention, are shown. As illustrated in FIG. 25A, message table 2500 includes column 2502 and column 2504 for storing the message identifiers and the language identifiers of the stored messages or message elements. Message table 2500 further includes column 2506 for storing the message texts of the messages, and columns 2508 for storing the message attributes, such as fonts, alignments, and so forth associated with the stored messages.
  • [0166]
    Similarly, as illustrated in FIG. 25B, pick list table 2510 includes column 2512 and column 2514 for storing the pick list identifiers and the language identifiers of the stored pick list elements. Pick list table 2510 further includes column 2516 for storing the texts of the pick list elements, column 2518 for storing list sequence values for the pick list elements (in particular, for the pick list elements of non-alphabet languages), and columns 2520 for other related data.
  • [0167]
    Shown in FIG. 26A is example question table 2600, which includes column 2602 and column 2604 for storing the question identifiers and the language identifiers of the stored questions or question elements. Question table 2600 further includes column 2606 for storing the question texts of the questions, and columns 2608 for storing the question attributes, such as fonts, alignments, and so forth associated with the stored questions.
  • [0168]
    Shown in FIG. 26B is concept table 2620, which includes column 2612 and column 2614 for storing the concept identifiers and the language identifiers of the stored concepts. Concept table 2620 further includes column 2626 for storing the texts of the concepts, and columns 2628 for other related data.
  • [0169]
    In one embodiment, tables 2500, 2510, 2600 and 2610 are relational tables of one or more relational databases. In alternate embodiments, other equivalent data structures may be used instead.
  • Translation Management
  • [0170]
    As described earlier, another novel service offered by the study creation service of multi-region market research service provider 1602 is the service that facilitates management of the translations. More specifically, the study creation service advantageously provides a summary overview of the translation status for all study elements of all supported languages (notwithstanding the large number of languages supported). As those skilled in the art would appreciate, the novel service is extremely useful for the study creators to monitor the progress of the translation effort and determines if a created study is ready to be conducted.
  • [0171]
    Returning now to FIGS. 23-24, wherein two diagrams illustrating an example user interface suitable for use to practice the on-line translation monitoring aspect of the present invention, and the operation flow of the relevant aspects of an associated function in support of such advantageous on-line monitoring of the transaction status. As illustrated in FIG. 23, interface 2300 advantageously provides a table summarizing the translation status 2308 of the study elements for the various supported languages. For the illustrated embodiment, a translation complete indicator “Y” is displayed if translation of the study element (i.e., question, message, and so forth) is complete. As a result, by scrolling up the display, a study creator may quickly determine the status of the required translations.
  • [0172]
    As illustrated in FIG. 24, in response to the request for the translation status of the study elements by, e.g., the study creator, at block 2402, the function first retrieves and outputs all the study elements' identifiers and texts in the original language (e.g., English). Thereafter, at block 2404, the function selects one of the supported languages as the current language. Then, at block 2406, for as long as there are study elements remaining to be processed, the function retrieves each of the study elements of the current language, element by element, and determines if the translation of the study element has completed. For the illustrated embodiment, completion of translation may simply be determined by the study element under examination having a non-zero study element identifier. If the translation is completed, the function outputs an appropriate translation complete indicator, e.g., the letter “Y” described earlier; otherwise, the function outputs a blank (denoting translation incomplete).
  • [0173]
    Thereafter, the function determines if additional language remains to be processed, block 2408. Blocks 2404-2406 are repeated for as long as there are study elements of supported languages to process.
  • [0174]
    Thus, it can be seen under the present invention, a multi-region market research creator 1604, in addition to being able to have much better control of the cost of a large multi-region market research study right up front, the multi-region market research creator 1604 is also able to have much better control over the progress of the translation, which oftentimes is the “critical factor” that determines how quickly a study can be launched.
  • Example Computer System
  • [0175]
    FIG. 27 illustrates an example computer system suitable for use to practice the present invention, in accordance with one embodiment. As shown, computer system 2700 includes one or more processors 2702 and system memory 2704. Additionally, computer system 2700 includes mass storage devices 2706 (such as diskette, hard drive, CDROM and so forth), input/output devices 2708 (such as keyboard, cursor control, and so forth), and communication interfaces 2710 (such as network interface cards, modems, and so forth). The elements are coupled to each other via system bus 2712, which represents one or more buses. In the case of multiple buses, they are bridged by one or more bus bridges (not shown). Each of these elements performs its conventional functions known in the art. In particular, system memory 2704 and mass storage 2706 are employed to store a working copy and a permanent copy of the programming instructions implementing the study creation service of the present invention. The permanent copy of the programming instructions may be loaded into mass storage 2706 in the factory, or in the field, as described earlier, through a distribution medium (not shown) or through communication interface 2710 (from a distribution server (not shown)). The constitution of these elements 2702-2712 are known, and accordingly will not be further described.
  • Conclusion and Epilogue
  • [0176]
    Thus, it can be seen from the above descriptions, a novel method and apparatus for creating multi-region market research studies has been described. The novel method/apparatus is advantageously scalable to support a large number of languages.
  • [0177]
    While the present invention has been described in terms of the above illustrated embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described. The present invention can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of restrictive on the present invention.
  • Reporting and Analyzing Data from a Multi-Region Research Survey DETAILED DESCRIPTION OF THE INVENTION
  • [0178]
    In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention may be practiced without these specific details, that the present invention is not limited to the depicted embodiments, and that the present invention may be practiced in a variety of alternate embodiments. In other instances, well known methods, procedures, components, and circuits have not been described in detail.
  • [0179]
    Parts of the description will be presented using terminology commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. Also, parts of the description will be presented in terms of the execution of operations. As well understood by those skilled in the art, these operations often take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through, for instance, electrical components.
  • [0180]
    Various operations will be described as multiple discrete steps performed in turn in a manner that is helpful in understanding the present invention. However, the order of description should not be construed as to imply that these operations are necessarily performed in the order they are presented, or even order dependent. Lastly, repeated usage of the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may.
  • Overview
  • [0181]
    Generally, a survey collects better data when it is administered in the panelist's native mode of communication including, for instance, the panelist's native language, currency, and cultural customs. In which case, to administer a survey across multiple regions, the best data can be obtained by translating the survey into the various modes of communication native to panelists in the various regions. As discussed above however, coordinating multiple versions of a survey presents a number of challenges for a researcher.
  • [0182]
    Creating a multi-region survey is the subject of co-pending U.S. Patent Application PCT US00/41972 (Ser. No. 09/744,774), entitled “Multi-Region Market Research Study Creation” (Docket # P003). The administration of a multi-region survey is the subject of co-pending U.S. Applications PCT US00/30583 (Ser. No. 09/744,776), entitled “Multi-Region Market Research Study Processing” (Docket # P004), PCT US00/30646 (Ser. No. 09/744,773), entitled “Content Delivery In A Preferred Language For A Large Number Of Languages” (Docket # P002), and PCT US00/41973 (Ser. No. 10/031,263), entitled “Language Sensitive Electronic Mail Generation And Associated Applications” (Docket # P006). Each application has been filed contemporaneously, and has common inventorship, with the present invention. Except for any circular incorporation, the applications are hereby fully incorporated by reference.
  • [0183]
    The present invention is drawn to analyzing survey data created and collected by the processes of these related applications. The present invention advantageously allows a researcher to access, analyze, and report on multi-region survey data using his or her native mode of communication without any need whatsoever to understand the diverse modes of communication used by the panelists.
  • [0184]
    FIG. 28 illustrates one embodiment of the present invention. A multi-region research service provider 2802 is connected to multi-region research clients 2804 and multi-region survey panelists 2806 through a networking fabric 2808. Research clients use service provider 2802 to conduct on-line, multi-region surveys of the diverse pool of survey panelists 2806. Surveys can focus on virtually any concept including, for instance, market research for a product or service, marketability of a political candidate or issue, sociological studies, health studies, and the like. Surveys can also take virtually any form including, for instance, text-based, graphics-based, audio-based, or multimedia interactions (called “response variables”) designed to illicit responses.
  • [0185]
    As described in the incorporated applications referred to above, research clients create a survey using one particular mode of communication, such as the United States version of the English language and U.S. dollars, and specify multiple regions in which the survey is to be administered. The service provider 2802 interacts with translation services (not shown) to translate the response variables into the various modes of communication used in the specified survey regions. The multi-region survey is then administered to the diverse set of panelists 2806 on-line through networking fabric 2808.
  • [0186]
    The service provider 2802 includes a multi-region survey reporting and analysis service 2810. Analysis service 2810 includes analysis interface 2814 for interacting with research clients 2804 and has access to survey data 2812. Survey data 2812 may include data representing the surveys themselves as well as data representing any responses that have already been collected.
  • [0187]
    Typically, a researcher wants to analyze survey data 2812 to determine some objective statistical issue. For instance, the researcher may want to know what percentage of a pool of panelists is interested in a particular product or service and the researcher may want to rank their level of interest within a predefined range. To obtain survey data that can be statistically analyzed, surveys are often created using various forms of multiple choice questions rather than free form answers. Since the service provider 2802 uniformly administers surveys, whether a question and set of possible answers is presented to a panelist in German, Japanese, Arabic, or some other language, the service provider 2802 can uniformly record that the panelist selected a particular answer from the set of possible answers. In which case, the analysis service 2810 can perform statistical analysis on the uniformly recorded data without regard to the mode of communication used to administer individual questions.
  • [0188]
    Furthermore, under various circumstances, a researcher may be interested to see the responses from a particular panelist. Again, since the service provider 2802 uniformly administers surveys, whether a question and set of possible answers is presented to a panelist in German, Japanese, Arabic, or some other language, the analysis service 2810 can display the questions in one particular language and match-up the uniformly recorded answers. The same approach can be applied to currency conversions, measurement conversions, and the like.
  • [0189]
    In other words, according to the teachings of the present invention, the analysis service 2810 takes advantage of the uniformly administered survey data provided by the multi-region research service provider 2802 to allow a research client 2804 using a single mode of communication to analyze multi-region survey data. In various embodiments, analysis can be performed in real-time on centralized data collected from multiple regions using multiple modes of communication. By collecting data from multiple time zones around the world, data processing loads for large surveys, or multiple simultaneous surveys, are likely to be distributed or balanced across a 24 hour operating clock. By supporting multiple modes of communication, a researcher can select his or her native mode of communication from a number of supported modes and analyze data from surveys created and/or administered in different modes of communication. Furthermore, any number of data mining techniques and statistical analysis can be performed. Various examples of the analysis service 2810 are described below in more detail.
  • [0190]
    Except for the novel aspects of the research service provider 2802, the various equipment used by the service provider 2802, the research clients 2804, the survey panelists 2806, and the networking fabric 2808 are intended to represent a broad range of these elements known in the art. Examples of computing and networking equipment suitable for practicing the present invention include, but are not limited to, various palm-sized, notebook sized, or desktop computers available from, for instance, IBM of Armonk, N.Y., servers available from Sun Microsystems of Mountain View, Calif., and routers/switches available from CISCO Systems of San Jose, Calif.
  • Method
  • [0191]
    In general, the analysis service provides the analysis interface to a research client, receives client input through the analysis interface in one mode of communication, analyzes data from the multi-region, on-line survey based on the client input, and provides analysis results through the analysis interface. The analysis interface could be, for instance, provided as a web page on-line or as an installed software application on the client's machine.
  • [0192]
    FIG. 29 demonstrates one embodiment of the present invention including a number of implementation specific details. Other embodiments may not include all of the illustrated elements, may include additional elements, and/or may perform various elements in a different order. The illustrated process is basically several iterations of the general process. That is, the illustrated process repeatedly provides some form of analysis interface, receives client input, analyzes survey data based on the input, and provides results through the analysis interface.
  • [0193]
    In the illustrated embodiment, the process begins by prompting the research client in his or her native mode of communication to provide identification and authentication at block 2910. This is the first analysis interface. As discussed in the incorporated patent applications, any number of approaches can be used by the client to indicate his or her native mode of communication. For instance, a one-click pick list could be used to select from a list of supported modes of communication. Identification and authentication could be a login and password, an encryption key, biometric data, or the like. The analysis service then waits to receive client input at block 2915.
  • [0194]
    When input is received, the analysis service analyzes survey data based on the input at block 2920. For instances, the analysis service filters survey data to identify any surveys that the client is authorized to access. In other words, survey data can include more than just the questions and answers. Survey data may also include supporting information such as who created the survey, who has authority to access the survey, what mode of communication was used to create the survey, purpose statements for the survey, and the like.
  • [0195]
    If one or more surveys are found, the analysis service displays summary information for the surveys in a list. Summary information may include a title or number for the survey or some other identifying information, as well as the number of panelists in the survey, regions of administration for the survey, and just about anything else that a client might want to know. Listed surveys may have already completed, may be currently in progress, or may be waiting to start. The type of information to be displayed in the summary may depend on the stage the survey is currently in and may depend on the availability of data. That is, if a survey has not started, no response data will be available for statistical analysis. In certain embodiments, the client can configure the fields in the summary list to display information of particular interest to the client.
  • [0196]
    Of course, the surveys can include data in diverse modes of communication. As discussed above, the analysis service takes advantage of the uniformly administered data provided by the multi-region research survey provider to display information in the client's mode of communication.
  • [0197]
    The summary list also includes at least one mechanism to select and act upon one or more of the surveys. Any number of mechanisms can be used. A number of examples are illustrated with respect to FIGS. 30 through 37 below, and may include, for instance, hyper-links, check boxes and window buttons, pop-up menus, drop-down menus, single-click icons, double-click icons, pick lists, and a window or a series of windows including various configuration options.
  • [0198]
    At block 2925, if no surveys are listed, the process returns to block 2910 to prompt the client again. In alternate embodiments, various security measures could be used to prevent an un-authorized user from repeatedly trying input combinations until he or she gains access. For instance, the analysis interface could lockup after a certain number of failed attempts.
  • [0199]
    If one or more surveys are listed, the process waits for client input at block 2930. When client input is received, if the client input indicates that the client wishes to exit at block 2935, the process returns to block 2910. If the client does not wish to exit, the process performs another iteration of analysis on the survey data at block 2940. The analysis is based on the client input and can include any number of data mining and/or statistical analysis operations including, for instance, data filtering such as identifying a particular group of panelists who gave a particular response to a question or identifying related data such as other surveys in which a particular panelist participated, data calculations such as calculating an average response to a particular question, generating mechanisms to configure further analysis such as links to demographic information about particular panelists identified by filtering the survey data, and displaying an analysis report such as a table, graph, or chart.
  • [0200]
    After analyzing data in block 2940, the process returns to block 2930 to wait for client input. The process will continue to go through iteration after iteration of blocks 2930, 2935, and 2940, until the client exits, repeatedly analyzing data, presenting analysis results in the analysis interface including at least one mechanism for further analysis, and waiting for further client input. Using this iterative process, the client can move up and down through various levels of detailed analysis of the survey data, viewing analysis reports in a wide variety of formats.
  • EXAMPLES
  • [0201]
    FIGS. 30 through 37, 40, and 41 illustrate a number of embodiments of analysis interfaces, input mechanisms, and analysis reports. Any number of alternate embodiments can be used to provide an analysis interface, receive client input, and provide analysis results.
  • [0202]
    FIG. 30 illustrates one embodiment of an identification window 3000. The illustrated window may be the first step to access survey data using the analysis service of the present invention. In the illustrated embodiment, the client is prompted to supply a login and password. The “continue” button submits the supplied information. As discussed above, any number of alternate approaches can be used to identify and authenticate a client.
  • [0203]
    FIG. 31 illustrates one embodiment of a survey list window 3100. Assuming, for instance, that the client successfully gained access to the analysis service through the interface of FIG. 30, the client is presented with a list of summary information for surveys to which the client has access. A client may have access only to surveys that the client created, or the client may have been granted access to surveys, or parts of surveys, created by others. Various forms of security could be used to limit a client's access to certain data. For instance, a client may not be given access to specific names and addresses of panelists, or other personal or proprietary information.
  • [0204]
    In the illustrated embodiment, the summary information includes a survey identification number, a study type, the number of panelists, the geographic regions for administering the survey, and a concept identifier and status for each survey. The underlined entries in each summary illustrate one example of an input mechanism. That is, each underlined entry is a hyper-link to related or supporting data. For instance, if a client wants to know more about the make-up of a particular survey, he or she may click on the survey identification number to, for instance, see a title of the survey, who created it, the questions presented, etc. Clicking on the concept identification number may lead to information on the purpose for the survey and what information the creator hopes to glean from it. Clicking on the countries or cities entries may lead to more detailed information about the target pool of panelists for the survey, including demographic information, modes of communication used for different groups of panelists, and the like. In alternate embodiments, any number of input mechanisms, such as those discussed above, could be used in place of hyperlinks to lead to more detailed information from a summary page. Various browser functions could also be used to exit the analysis service, return to a previous screen, etc.
  • [0205]
    In the illustrated embodiment, the first two surveys are either complete or in progress because results are available for analysis as indicated by the hyper-link “Analyze Results.” The third survey has not yet been initiated or has not yet received any results. For the surveys that have received results, the “Analyze Results” hyperlink can lead to more detailed information about those results, including some form of overall survey summary.
  • [0206]
    FIG. 32 illustrates one embodiment of a survey summary window 3200. The survey summary includes a list of question identifiers, number of responses to each question, average responses, and a check box for each question to select one or more questions for further analysis. An average response can be calculated by assigning a weighted valued to each possible response, adding the weighted values from all the received responses, and dividing the total by the maximum possible weighted value.
  • [0207]
    In one embodiment, the data can be updated in real time. For instance, were the client is analyzing data on-line, the analysis service can update the numbers whenever new responses are received by incrementing the response count and recalculating the average response. Furthermore, the centralized database allows real-time analysis of data from multiple regions collected using multiple modes of communication.
  • [0208]
    The exemplary input mechanism in the illustrated embodiment is the combination of the check boxes and the “Continue Analysis” button. The client can select one or more questions for further analysis by checking the boxes and pressing the button. Any number of alternate embodiments may be used, of course, such as making the question identifiers hyper-links and browser controls to return to previous pages.
  • [0209]
    FIG. 33 illustrates one embodiment of an analysis report window 3300 providing further details about a question presented, for instance, in the overall survey summary of FIG. 32. The window includes the wording of the actual question, the number of responses received, the list of possible answers, and the percentage of responses received for each possible answer. The percentages are also illustrated using bar graphs. In alternate embodiments, a pie chart, histogram, linear or area function, or the like could be used to display the distribution of responses. If the client had selected more than one check box in the summary window of FIG. 32, multiple reports such as the one illustrated in FIG. 33 could be generated in a single window or in multiple windows. The data could also be updated in real time as new responses are received.
  • [0210]
    The input mechanisms in FIG. 33 include hyper-links leading from each of the possible responses. For instance, selecting one of the responses in FIG. 33 could lead to the analysis report window 3400 shown in FIG. 34, which details demographic information about panelists who selected the corresponding response. Demographic information in the illustrated embodiment includes gender, marital status, educational level, age, employment status, and household composition. In alternate embodiments, any number of demographic factors could be used.
  • [0211]
    FIG. 34 also illustrates another example of an input mechanism, menu/tool bar 3410. From the bar 3410, a client can select a variety of analysis reports and report formats including graphics, related questions, related surveys, information on individual panelists, and a survey specification. In one embodiment, a user may select a field from the window by, for instance, highlighting it, and then select a menu option related to that field. In an alternate embodiment, a pop-up window of menu options may pop-up when the client passes a cursor over the field in the window. In yet another alternate embodiment, fields may be hyper-links so that, for instance, a client can select the “male” hyper-link and view an analysis report of responses from all male panelists. Any number of alternate input mechanisms can be used, including browser commands to return to previous windows.
  • [0212]
    FIG. 35 illustrates one embodiment of an analysis report window 3500 for a particular type of response variable, semantic differential questions. In the illustrated embodiment, semantic differential questions are listed with weighted average responses for each. Any number of input mechanisms can be used to delve into more or different details.
  • [0213]
    FIG. 36 illustrates one embodiment of a particular type of analysis report. The table in window 3600 includes three axes, a horizontal axis 3605, a vertical axis 3610, and a Z axis 3620. Using various forms of configuration windows, a user can specify various filter criteria for each axis to identify particular data. In the illustrated embodiment, the horizontal axis filter criteria is education level so that each column corresponds to a different level of education. The vertical axis filter criteria is income so that each row corresponds to a different level of income. The Z axis filter criteria is age so that each table in the Z axis corresponds to a different age category. The data that is being filtered is intent to buy. The each cell in each table is a weighted average response to the intent to buy question for respondents of a particular education, income, and age category.
  • [0214]
    FIG. 37 illustrates one embodiment of another type of analysis report. In window 3700, subcategories of panelists were presented with different price points for a product and asked about their intent to buy. The table includes the number of responses, illustrates the distribution of responses in graphical form, and specifies the average weighted response for each subcategory.
  • [0215]
    The illustrated embodiment presents a good example of analyzing multi-region data. That is, the price points would have been presented to the panelists in their respective native currencies. The table, however, reports the price points using the native currency and language of the researcher.
  • [0216]
    Skipping to FIG. 40, FIG. 40 illustrates one embodiment on an analysis report that can be used to assign codes to open-ended questions. The illustrated embodiment includes two windows. In the window on the right, a client as configured the report to list open-ended responses to a particular question in English. As discussed in the related applications, translation services can be used to present results in any number of supported modes of communication. The question is displayed at the top of the window and the first five responses are listed below.
  • [0217]
    Next to each response is a code field. The client can select from codes listed in the window on the left. That is, after reading a particular response, the client can assign a code to the response for statistical analysis purposes. For instance, the first response indicates that the panelist is interested in wireless networking because the panelist is generally interested in cutting edge technology and wireless networking is new to the panelist. Therefore, the client codes the response with the “New” code, indicating that the panelist was interested in wireless networking because it is new.
  • [0218]
    In the window on the left, each listed code has next to it a number of configuration buttons. The configuration buttons can be used to add new codes if, for instance, the client begins to see a number of similar responses and wants to create a new code for those types of responses, to delete codes if, for instance, no responses receive a particular code number and the client wants to eliminate the code to shorten the list of possibilities, and to configure various aspects of a code.
  • [0219]
    For instance, as shown in FIG. 41, a code figuration window can be used to specify text to identify a particular code number, as well as key words typically found in responses that receive this particular code number. The key words can be used to perform an automatic coding of responses. For instance, if a client selects the “Autocode” button in FIG. 40, the analysis service can scan open-ended responses for key words and automatically assign codes accordingly. Once the responses are coded, any kind of data mining or statistical analysis can be performed based on the codes.
  • Example Computer System
  • [0220]
    FIG. 38 illustrates one embodiment of a hardware system intended to represent a broad category of computer systems such as personal computers, workstations, and/or embedded systems. In the illustrated embodiment, the hardware system includes processor 3810 coupled to high speed bus 3805, which is coupled to input/output (I/O) bus 3815 through bus bridge 3830. Temporary memory 3820 is coupled to bus 3805. Permanent memory 3840 is coupled to bus 3815. I/O device(s) 3850 is also coupled to bus 3815. I/O device(s) 3850 may include a display device, a keyboard, one or more external network interfaces, etc.
  • [0221]
    Certain embodiments may include additional components, may not require all of the above components, or may combine one or more components. For instance, temporary memory 3820 may be on-chip with processor 3810. Alternately, permanent memory 3840 may be eliminated and temporary memory 3820 may be replaced with an electrically erasable programmable read only memory (EEPROM), wherein software routines are executed in place from the EEPROM. Some implementations may employ a single bus, to which all of the components are coupled, or one or more additional buses and bus bridges to which various additional components can be coupled. Those skilled in the art will be familiar with a variety of alternate internal networks including, for instance, an internal network based on a high speed system bus with a memory controller hub and an I/O controller hub. Additional components may include additional processors, a CD ROM drive, additional memories, and other peripheral components known in the art.
  • [0222]
    In one embodiment, the various elements of the present invention described above are each implemented using one or more computers such as the hardware system of FIG. 38. Where more than one computer is used, the systems can be coupled to communicate over an external network, such as a local area network (LAN), an Internet protocol (IP) network, etc. In one embodiment, the present invention is implemented as software routines executed by one or more execution units within the computer(s). For a given computer, the software routines can be stored on a storage device, such as permanent memory 3840.
  • [0223]
    Alternately, as shown in FIG. 39, the software routines can be machine executable instructions 3910 stored using any machine readable storage medium 3920, such as a diskette, CD-ROM, magnetic tape, digital video or versatile disk (DVD), laser disk, ROM, Flash memory, etc. The series of instructions need not be stored locally, and could be received from a remote storage device, such as a server on a network, a CD ROM device, a floppy disk, etc., through, for instance, I/O device(s) 3850 of FIG. 38.
  • [0224]
    From whatever source, the instructions may be copied from the storage device into temporary memory 3820 and then accessed and executed by processor 3810. In one implementation, these software routines are written in the C programming language. It is to be appreciated, however, that these routines may be implemented in any of a wide variety of programming languages.
  • [0225]
    In alternate embodiments, the present invention is implemented in discrete hardware or firmware. For example, one or more application specific integrated circuits (ASICs) could be programmed with one or more of the above described functions of the present invention. In another example, one or more functions of the present invention could be implemented in one or more ASICs on additional circuit boards and the circuit boards could be inserted into the computer(s) described above. In another example, field programmable gate arrays (FPGAs) or static programmable gate arrays (SPGAs) could be used to implement one or more functions of the present invention. In yet another example, a combination of hardware and software could be used to implement one or more functions of the present invention.
  • [0226]
    Thus, reporting and analyzing data from a multi-region survey is described. Whereas many alterations and modifications of the present invention will be comprehended by a person skilled in the art after having read the foregoing description, it is to be understood that the particular embodiments shown and described by way of illustration are in no way intended to be considered limiting. Therefore, references to details of particular embodiments are not intended to limit the scope of the claims.
Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
US4816904 *Apr 10, 1987Mar 28, 1989Control Data CorporationTelevision and market research data collection system and method
US4905080 *Jul 31, 1987Feb 27, 1990Video Research Ltd.Apparatus for collecting television channel data and market research data
US5041972 *Apr 15, 1988Aug 20, 1991Frost W AlanMethod of measuring and evaluating consumer response for the development of consumer products
US5497319 *Sep 26, 1994Mar 5, 1996Trans-Link International Corp.Machine translation and telecommunications system
US5535120 *Jun 7, 1995Jul 9, 1996Trans-Link International Corp.Machine translation and telecommunications system using user ID data to select dictionaries
US5740035 *Jul 23, 1991Apr 14, 1998Control Data CorporationSelf-administered survey systems, methods and devices
US5884246 *Dec 4, 1996Mar 16, 1999Transgate Intellectual Properties Ltd.System and method for transparent translation of electronically transmitted messages
US5999908 *Sep 19, 1997Dec 7, 1999Abelow; Daniel H.Customer-based product design module
US6041303 *Jun 6, 1997Mar 21, 2000Mathews; Edward HenryMethod of assisting the conducting of a research project
US6044357 *May 5, 1998Mar 28, 2000International Business Machines CorporationModeling a multifunctional firm operating in a competitive market with multiple brands
US6064971 *Jun 7, 1995May 16, 2000Hartnett; William J.Adaptive knowledge base
US6138088 *Dec 22, 1997Oct 24, 2000International Business Machines CorporationMethod and apparatus for process control by using natural language processing (NLP) technology
US6161082 *Nov 18, 1997Dec 12, 2000At&T CorpNetwork based language translation system
US6219638 *Nov 3, 1998Apr 17, 2001International Business Machines CorporationTelephone messaging and editing system
US6256663 *Jan 22, 1999Jul 3, 2001Greenfield Online, Inc.System and method for conducting focus groups using remotely loaded participants over a computer network
US6275789 *Dec 18, 1998Aug 14, 2001Leo MoserMethod and apparatus for performing full bidirectional translation between a source language and a linked alternative language
US6311190 *Feb 2, 1999Oct 30, 2001Harris Interactive Inc.System for conducting surveys in different languages over a network with survey voter registration
US6477504 *Mar 2, 1998Nov 5, 2002Ix, Inc.Method and apparatus for automating the conduct of surveys over a network system
US6513014 *Mar 31, 1999Jan 28, 2003Walker Digital, LlcMethod and apparatus for administering a survey via a television transmission network
US6564190 *Dec 30, 1999May 13, 2003General Electric Capital CorporatonMethod and formulating an investment strategy for real estate investment
US6741967 *Mar 26, 1999May 25, 2004Vividence CorporationFull service research bureau and test center method and apparatus
US6754635 *Nov 15, 1999Jun 22, 2004Ix, Inc.Method and apparatus for automating the conduct of surveys over a network system
US6792399 *Sep 8, 1999Sep 14, 2004C4Cast.Com, Inc.Combination forecasting using clusterization
US6859784 *Sep 28, 1999Feb 22, 2005Keynote Systems, Inc.Automated research tool
US6901367 *Jan 28, 1999May 31, 2005International Business Machines CorporationFront end translation mechanism for received communication
US6993495 *Feb 8, 2001Jan 31, 2006Insightexpress, L.L.C.Dynamically assigning a survey to a respondent
US20040064371 *Sep 30, 2003Apr 1, 2004Crapo Jeff D.On-line registration system and method
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US7548846Nov 6, 2000Jun 16, 2009Global Market Insite, Inc.Language sensitive electronic mail generation and associated applications
US7599956 *Dec 12, 2006Oct 6, 2009At&T Intellectual Property I, L.P.Reusable online survey engine
US8086648 *Dec 27, 2011The Rand CorporationComputer assisted data collection for surveys and the like
US8271540 *Sep 18, 2012The Rand CorporationComputer assisted data collection for surveys and the like
US8412537Apr 2, 2013Intuit Inc.System and method for episode service item cost estimation based on historical data
US8595051 *Dec 24, 2003Nov 26, 2013Hewlett-Packard Development Company, L.P.Metrics capability self assessment
US9031849 *Mar 27, 2009May 12, 2015Huawei Technologies Co., Ltd.System, method and multipoint control unit for providing multi-language conference
US9266356 *Mar 15, 2011Feb 23, 2016Seiko Epson CorporationSpeech output device, control method for a speech output device, printing device, and interface board
US20030004786 *Jul 4, 2001Jan 2, 2003Hidemasa TokiedaMethod for performing multilingual market presearch through a two-way computer communication network and a communication system and information recording medium for the same method
US20050144592 *Dec 24, 2003Jun 30, 2005Below Paul A.Metrics capability self assessment
US20060265348 *May 1, 2006Nov 23, 2006The Rand CorporationComputer assisted data collection for surveys and the like
US20070094606 *Dec 12, 2006Apr 26, 2007Bellsouth Intellectual Property CorporationReusable online survey engine
US20080114748 *Nov 13, 2006May 15, 2008Richard VarnerPeer review system and method therefor
US20080140534 *Dec 11, 2007Jun 12, 2008Touchtone Technologies, Inc.Method and system that provides an incentive for a consumer to participate in a marketing program
US20090187400 *Jul 23, 2009Huawei Technologies Co., Ltd.System, method and multipoint control unit for providing multi-language conference
US20090249314 *Jun 5, 2009Oct 1, 2009The Rand CorporationComputer assisted data collection for surveys and the like
US20110238421 *Sep 29, 2011Seiko Epson CorporationSpeech Output Device, Control Method For A Speech Output Device, Printing Device, And Interface Board
Classifications
U.S. Classification704/200
International ClassificationG06Q10/10, G06Q30/02, G06F9/44, G06F17/30
Cooperative ClassificationG06Q30/0203, G06Q10/107, G06Q30/02, G06F9/4448, G06Q30/0205
European ClassificationG06Q30/02, G06Q30/0205, G06F9/44W6, G06Q10/107, G06Q30/0203