|Publication number||US20090203413 A1|
|Application number||US 12/030,684|
|Publication date||Aug 13, 2009|
|Filing date||Feb 13, 2008|
|Priority date||Feb 13, 2008|
|Publication number||030684, 12030684, US 2009/0203413 A1, US 2009/203413 A1, US 20090203413 A1, US 20090203413A1, US 2009203413 A1, US 2009203413A1, US-A1-20090203413, US-A1-2009203413, US2009/0203413A1, US2009/203413A1, US20090203413 A1, US20090203413A1, US2009203413 A1, US2009203413A1|
|Inventors||Anthony Jefts, Gregory Eldridge, Javier Fernandez-Ivern, Michael A. Fogleman, John M. Hughes, David Messinger, George Tsipolitis|
|Original Assignee||Anthony Jefts, Gregory Eldridge, Javier Fernandez-Ivern, Fogleman Michael A, Hughes John M, David Messinger, George Tsipolitis|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (11), Non-Patent Citations (2), Referenced by (5), Classifications (4), Legal Events (1)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This invention relates to computer-based methods and systems for facilitating the development of an asset.
Competitions have been held for the development of an asset, such as essays, music, computer software, and so on.
In some competitions, a larger work is divided up into a number of smaller pieces, and competitions held for the development of those pieces. For example, a software program may be divided up into components, and competitions held for the development of each of the components. In some cases, development of a component may be divided into design tasks and coding tasks, and a contest held for the design of the component, and another contest held for the coding of the component. As another example, a competition may be held for the development of a graphic design, such as a logo.
In such competitions, specifications typically are not made available until the competition begins. In such cases, specifications are developed without the benefit of input from potential competitors. Sometimes substantial interaction with the competitors during the competition is required to clarify parts of the specification that are not clear. Correction to specifications may be necessary in order to fix problems identified by the competitors, potentially requiring increases to the competition time and reducing the likelihood of competition success.
Various asset development competitions have been used to develop assets, such as content, computer software, graphics, user interfaces, and so forth. Such competitions have been held, for example, by TOPCODER, INC. of Glastonbury, Conn., and may be seen at http://www.topcoder.com.
As described in co-pending U.S. patent application Ser. No. 11/035,783, entitled SYSTEMS AND METHODS FOR SOFTWARE DEVELOPMENT by John M. Hughes, filed Jan. 14, 2005, incorporated herein by reference, a software application may be developed by contest, where the task of developing the software application is divided up into discrete tasks, and competitions are held for the performance of each of the discrete tasks. For example, a software program may be divided up into components, and competitions held for the development of each of the components. Development of a component may be divided into design tasks and coding tasks, and a contest held for the design of the component, and another contest held for the coding of the component.
As described in co-pending U.S. patent application Ser. No. 11/655,768, entitled SYSTEM AND METHOD FOR DESIGN DEVELOPMENT by John M. Hughes, filed Jan. 19, 2007, incorporated herein by reference, design contests may be held, for example, for graphics design and user interface development as well as other types of designs and work product. Submissions in such contests may be evaluated for technical merit (i.e., meeting the described requirements) and/or based on customer affinity and/or appeal to a designated group of individuals.
As described in co-pending U.S. patent application Ser. No. 11/755,909, entitled PROJECT MANAGEMENT SYSTEM, by Messinger et al., filed May 31, 2007, incorporated herein by reference, a series of contests may be managed through use of project management system. Portions of a contest, such as a review effort, may begin when submissions are received, deadlines are met, and so on. Such contest systems can provide contest-related information to contest administrators as well as potential competitors.
In various embodiments, techniques may be used to inform potential contestants in advance about upcoming competitions. At the same time, specifications may be made available to potential contestants, for review and comment. Making the specifications available in advance enables potential competitors to make better decisions about whether to participate in a competition, and also allows them to comment on or clarify the specification for the competition in advance of the competition.
For example, in some embodiments, a list of competitions is provided. Some of the competitions in the list may be upcoming. In addition, some of the competitions in the list may be active competitions and/or completed competitions. The list may be sorted and/or filtered by date, type of competition and/or other information. The list may be communicated using any suitable information communication technique, for example by one or more of a web page, email, RSS feed, text message, and so on. Each competition is for the development of one or more assets. Each competition listing may include a reference to one of a number of workspaces, typically a workspace allocated to the respective asset. For example, the workspace may be or may include one or more of a shared document, a wiki page, a web page, a blog page, a forum, a bulletin board, and so on. The workspace includes a place for providing a draft competition specification.
The competition specification may include a technical description of the asset to be developed. The competition specification may include the rules and/or parameters of the competition (e.g., dates, prizes, and so on). The competition specification may include other information, such as other documents and tools that may be used, reference information, submission details, competition logistics, and so on.
Feedback may be received from potential competitors on the draft competition specification in the workspace. Typically, feedback is received for a period of time prior to the competition. The feedback may include (among other things) any one or more of comments on the specification, text changes to the specification, graphical or design changes, other modifications to the specification, additions to the specification, and so on. The draft competition specification (including without limitation information about competition logistics) may be finalized based on the feedback, and a competition for the development of the asset held using the finalized specification as the competition specification.
The asset that is to be developed by competition may be any type of asset. Just as some non-limiting examples, the asset may be or may include software component design, graphic design, and mechanical part design, software component, product design, product specification or plans, creative work, artistic work, video, audio, multimedia presentation, music, etc. The list of competitions may include, for example, a competition name, a planned competition start date, other pertinent meta data describing the competition such as an overview, technologies involved, timelines, and so on and/or a reference to the workspace. The reference to the workspace may include a URL.
In general, in another aspect, a system for developing an asset by competition comprises a competition list including a list of competitions, each competition for the development of an asset, each element of the list including a reference to one of a number of workspaces, the one of the workspaces allocated to a respective asset. The system includes a workspace including a draft competition specification and a facility for receiving feedback from potential competitors on the draft competition specification in the workspace for a period of time prior to the competition. The system includes a competition system for holding a competition for the development of the asset using a specification that is based on the draft competition specification taking into account the feedback from potential competitors.
The systems described can be implemented as software running on computers, mobile devices and telephones, televisions, home entertainment centers, gaming consoles, and other devices, for example, as described in the above-referenced co-pending patent applications.
Other aspects and advantages of the invention will become apparent from the following drawings, detailed description, and claims, all of which illustrate the principles of the invention, by way of example only.
In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
Typically, a competition involves two or more participants competing to develop an asset. A specification describes requirements for an asset and may also include competition rules and logistics. Standards may be specified for the asset and may be specific to the type of asset. Standards may be described for example, in a checklist or scorecard, and may be general or specific as appropriate. In some embodiments, a review of an asset is conducted using the specification and the standards in order to identify a competition winner, and the review criteria include the relevant standards.
In various embodiments, techniques may be used to enable communication between potential contestants, contest administrators, and contest sponsors in advance of upcoming asset development competitions. In some cases, such techniques allow contestants to be informed about the logistics of upcoming competitions, so as to allow them to plan their schedules to allow participation. Such techniques allow contestants to provide feedback about the logistics, for example, to point out potential conflicts with other competitions or events, or to otherwise give an early indication that the planned logistics may negatively or positively affect the success of the competition.
In various embodiments, specifications may be made available to potential contestants in advance of the competitions for review and comment. Making the specifications available in advance enables potential competitors to make better decisions about whether to participate in a competition, and also allows them to comment on or clarify the specification for the competition in advance of the competition. The feedback received may be used to modify the specification or otherwise clarify the specification, to make the competition complete more smoothly. The feedback received also may be used to determine interest in the competition, or otherwise determine the success of the competition.
For example, in some embodiments, a list of competitions is provided 101. At least some of the competitions are upcoming and have not yet started. In some embodiments, some of the competitions may be active competitions and/or completed competitions. The list may be provided by any suitable means. Just as a few non-limiting examples, the list may be communicated by one or more of a web page, email, RSS feed, text message, and so on. The list may include such information as the name of the competition, a planned competition start date, and a reference to the workspace. The reference to the workspace may include a URL and/or other location indicia.
Each competition typically is for the development of one or more assets. Each listing may include a reference to one of a number of workspaces, typically a workspace allocated to that respective asset. The workspace may be any suitable workspace. Just as a few examples, not intended to be limiting, the workspace may be or may include one or more of a shared document, a wiki page, a web page, a blog page, a forum, a bulletin board, and so on. The workspace may be specific to the asset under development. In some cases, the asset to be developed is a part of a larger or combined asset to be developed at least in part by a series of development competitions, and the workspace may include documents and information associated with the larger asset.
The workspace includes a place for providing a draft competition specification 102. The draft specification may in some cases be first provided in a substantially complete form that would be used as the specification if no comments are received or as a rough draft that is expected to be further revised by the administrator, potential competitors and/or contest sponsors. One benefit of providing a draft specification is that potential competitors can review the draft specification and provide substantive feedback on the specification prior to the competition. The potential competitors can provide input that allows for the development of a specification that already addresses potential concerns of competitors. In a drafting process, for example, in which an architect or designer provides a draft, and a reviewer provides feedback to the architect or designer, this process of draft and revisions may be visible to those who have access to the workspace. Access to the specification drafting and revision process may give competitors further insight as to the desired asset to be developed in the competition.
In some embodiments, depending on competition and/or administrative policies, potential competitors may need to perform certain actions or have certain privileges in order to have access to the workspace and/or provide feedback. For example, competitors may be required to have participated in a number of competitions, have a rating in a type of competition, have a rating over a predetermined threshold, etc. in order to have access and/or the ability to provide certain types of feedback. Potential competitors may be required to have submitted contractual documentation, such as a non-disclosure agreement and/or an intellectual property assignment document or other documentation or certifications as a prerequisite to participation.
Feedback may be received on the draft competition specification 103 in the workspace. Feedback may be received from potential competitors, administrators, reviewers, competitions sponsors (e.g., customers) and/or other parties. Different parties may provide the same types or different types of feedback. For example, some participants, such as reviewers, administrators, sponsors, or potential competitors with particular prerequisites may have privileges to edit text, or edit certain sections, while others may only be able to mark text or provide comments. For example, if prize money and/or start date/times are provided in the competition specification, potential competitors may be allowed to comment on the logistics of the competition but not modify the competition specification. Potential competitors may be able to edit technical information in the specification or add technical information or comments to the specification.
Typically, a period of time prior to the competition is specified for feedback. For example, if the draft specification is first provided 2 weeks prior to a competition, a feedback period may be specified that is from two weeks prior to the competition to one week prior to the competition. The feedback may be any type of feedback. Just as a few examples not intended to be limiting, feedback may include any one or more of comments on the specification, suggestions for changes to the specification, text edits to the specification, suggestions for changes to graphics or designs, edits to graphics or designs, additions to the specification, general or specific comments on the specification and/or the competition logistics, and so on.
In some embodiments, the feedback may include an indicia of interest or disinterest in participating in the competition. For example, in some implementations there may be one or more “voting” buttons that allow a potential competitor to indicate interest in participating in that competition. In some implementations, indicia of interest may include a single voting button in which a potential competitor can indicate interest in participating in that competition. In some implementations, indicia of interest may include a two voting buttons in which a potential competitor can indicate interest in participating in that competition, or indicate that the potential competitor is not interested in participating in the competition. In some implementations, indicia of interest includes a degree of interest. In one such implementation, as an example, five radio buttons, each representing a different degree of interest are provided, in which a first button indicates that a potential competitor will participate, a second button indicates that the potential competitor is likely to participate, a third button indicates that the potential competitor may or may not participate, a forth button indicates that the potential competitor likely will not participate, and the fifth button indicates that the potential competitor will not participate. A comment area may also be provided for the potential competitor to give a reason for the indicated interest indicia.
In some implementations, indicia of interest may be recorded without storing information about who has indicated interest. In other implementations, the indicia of interest may be stored with information about the identity and/or qualifications of the potential competitor(s) who have provided a positive (e.g., will compete or are likely to compete) indicia of interest. Such information may be used to determine the likelihood of success of the competition. For example, in some cases if two or more potential competitors with ratings over a predetermined threshold (e.g., a rating threshold of 900, 1200, 1500, or 2200) have provided a positive indicia of interest, it is likely that the competition will be successful. Likewise, if, after a predetermined time period, there have been no indicia of interest by a predetermined number of potential competitors with sufficiently high ratings, it may be beneficial to take steps to increase participation, such as to increase prize money, modify timelines and/or logistics, and so on. Likewise, if there is an amount of interest above the predetermined threshold, it may be possible to start the competition early, or to reduce prize money or points, or to take other steps with some confidence that the competition will complete successfully.
The indicia of interest may in some implementations be publicly available. In such cases, all visitors can gain some understanding about the feedback with respect to interest expressed. The published indicia may include all information and/or may include a summary of the indicia. For example, the indicia may include a table demonstrating interest for range of skill level (e.g., totals of indications from potential competitors in the 900-1199 skill rating range, totals of indications from potential competitors in the 1200-1499 skill rating range, totals of indications from potential competitors in the 1500-2199 skill rating range, totals of indications from potential competitors in the 2200+ skill rating range, etc.)
The draft competition specification may be finalized based on the feedback 104. In some cases, this may involve simply “locking” the specification workspace to further edits. In some cases, this may involve a final review and edit of the specification by a designated administrator, review board member, and/or contest sponsor. In some cases, this may involve changes to competition logistics. In some cases, there may be one or more rounds of edits and comment periods.
In some cases a specification is not considered finalized until a number of potential competitors have provided an indication of interest in participating in the competition. For example, the start of the competition may be postponed to a date/time later than a scheduled time if sufficient indicia of interest are not received. In some cases, a competition may be scheduled to begin at some time period after sufficient indicia of interest are received. Techniques may be used to determine the likelihood of success based on expected participation, such as described in co-pending U.S. Provisional Patent Application Ser. No. 61/020,703 by Fairfax et al, entitled SYSTEM AND METHOD FOR CONDUCTING COMPETITIONS, filed on Jan. 11, 2008, the teachings of which are incorporated herein by reference.
In general, in another aspect, a system 200 for developing an asset by competition may include a competition list including a list of competitions, where each competition is for the development of an asset. Each element of the list may include a reference to one of a number of workspaces, where the workspace is relevant and/or allocated to the asset competition. As shown in the figure, Competition 1 includes a reference to Workspace 1, Competition 2 includes a reference to Workspace 2. Competition n includes a reference to Workspace n. In should be understood that there may be any number or combination of competitions and workspaces, and that the number shown in the figure is intended to be exemplary.
The system includes one or more workspaces 203 a, 203 b, 203 n, generally 203, each including a facility (which may include or communicate with a data store) for a competition specification 204 a, 204 b, 204 n, generally 204. The specification facility 204 may or may not have a provided at the time that a workspace is first created. At some time prior to the asset development competition, a competition specification 204 is provided in the workspace 203. In some embodiments, the workspace 203 includes facility for receiving feedback on the draft competition specification. The feedback may be received from potential competitors, administrators, competition sponsors, and so on. In some cases, the specification is designated as a draft for a period of time prior to the competition. At some time prior to or and the start of the competition, the specification 204, is designated as final. There may be an indication in the competition list of the status of the specification (e.g., not yet provided, available for feedback, feedback period closed, competition underway, etc.)
Competitions 205 a, 205 b, 205 n, generally 205, are held for the development of the specified assets using the specification 204 taking into account the feedback from potential competitors.
In some embodiments, the development process is monitored and managed by a facilitator 1000. The facilitator 1000 can be any individual, group, or entity capable of performing the functions described here. In some cases, the facilitator 1000 can be selected from the distributed community of contestants based on, for example, achieving exemplary scores on previous submissions, or achieving a high ranking in a competition. In other cases, the facilitator 1000 may be appointed or supplied by an entity requesting the development, and thus the entity requesting the competition oversees the competition.
The facilitator 1000 has a specification 1010 for an asset to be developed by competition. In general, a specification 1010 is intended to have sufficient information to allow contestants to generate the desired asset. In some embodiments, the specification has been made available to potential competitors for review and feedback as described with reference to
The facilitator specifies rules for the competition. The rules may include the start and end time of the competition, and the awards(s) to be offered to the winner(s) of the competition, and the criteria for judging the competition. There may be prerequisites for registration for participation in the competition. In some cases, the specification may be assigned a difficulty level, or a similar indication of how difficult the facilitator, entity, or other evaluator of the specification, believes it will be to produce the asset according to the specification. In some cases, in addition to the technical requirements, competition rules and logistics may be included in (or even if separate, considered part of) the competition specification.
The specification is distributed to one or more developers 1004, 1004′, 1004″ (generally, 1004), who may be members, for example, of a distributed community of asset developers. In one non-limiting example, the developers 1004 are unrelated to each other. For example, the developers may have no common employer, may be geographically dispersed throughout the world, and in some cases have not previously interacted with each other. As members of a community, however, the developers 1004 may have participated in one or more competitions, and/or have had previously submitted assets subject to reviews. This approach opens the competition to a large pool of qualified developers.
In some embodiments, the communication of the specification can occur over a communications network using such media as email, instant message, text message, mobile telephone call, a posting on a web page accessible by a web browser, through a news group, facsimile, or any other suitable communication. In some embodiments, the communication of the specification can be accompanied by an indication of the rules including without limitation the prize, payment, or other recognition that is available to the contestants that submit specified assets. In some cases, the amount and/or type of payment may change over time, or as the number of participants increases or decreases, or both. In some cases submitters may be rewarded with different amounts, for example a larger reward for the best submission, and a smaller reward for second place. The number of contestants receiving an award can be based on, for example, the number of contestants participating in the competition and/or other criteria.
The recipients 1004 of the specification can be selected in various ways. In some embodiments, members of the community may have expressed interest in participating in a particular type of asset development competition, whereas in some cases individuals are selected based on previous performances in competitions, prior projects, and/or based on other methods of measuring programming skill of a software developer. For example, the members of the community may have been rated according to their performance in a previous competition and the ratings may be used to determine which programmers are eligible to receive notification of a new specification or respond to a notification. The community members may have taken other steps to qualify for particular competitions, for example, executed a non-disclosure agreement, provided evidence of citizenship, submitted to a background check, and so forth. Recipients may need to register for a competition in order to gain access to a finalized competition specification.
In one embodiment, a facilitator 1000 moderates a collaborative discussion forum among the various participants to answer questions and/or to facilitate development by the contestants. The collaborative forum can include such participants as facilitators, developers, customers, prospective customers, and/or others interested in the development of certain assets. In one embodiment, the collaboration forum is an online forum where participants can post ideas, questions, suggestions, or other information. In some embodiments, only a subset of the members can access and/or post to the forum, for example, participants in a particular competition or on a particular team.
Upon receipt of the specification 1010, one or more of the developers 1004 each develop assets to submit (shown as 1012, 1012′ and 1012″) in accordance with the specification 1010. The development of the assets can be accomplished using any suitable development tools or system, depending, for example, on the contest rules and requirements, the type of asset, and the facilities provided. For example, there may be specified tools and/or formats that should be used.
Once a developer 1004 is satisfied that her asset meets the specified requirements, she submits her submission, for example via a communications server, email, upload, facsimile, mail, or other suitable method.
To determine which asset will be used as the winning asset as a result of the contest, a review process 1014 may be used. A review can take place in any number of ways. In some cases, the facilitator 1000 can engage one or more members of the community and/or the facilitator and/or the entity requesting the asset. In some embodiments, the review process includes one or more developers acting as a review board to review submissions from the developers 1004. A review board preferably has a small number of (e.g., less than ten) members, for example, three members, but can be any number. Generally, the review board is formed for only one or a small number of related contests, for example three contests. Review boards, in some embodiments, could be formed for an extended time, but changes in staffing also can help maintain quality. In some embodiments, where unbiased peer review is useful, the review board members are unrelated (other than their membership in the community), and conduct their reviews independently. In some embodiments, reviewers are allocated such that they only infrequently work on the same contests.
In some embodiments, one member of the review board member is selected as a primary review board member. In some cases, a facilitator 1000 acts as the primary review board member. The primary review board member may be responsible for coordination and management of the activities of the board. In some embodiments, the customer and/or facilitator 1000 serves as the only review board member(s).
In some embodiments, a screener, who may be a primary review board member, a facilitator, or someone else, screens 1016 the submissions before they are reviewed by the (other) members of the review board. In some embodiments, the screening process includes scoring the submissions based on the degree to which they meet formal requirements outlined in the specification (e.g., format and elements submitted). In some embodiments, scores are documented using a scorecard, which may be a document, spreadsheet, online form, database, or other documentation. The screener may, for example, verify that the identities of the developers 1004 cannot be discerned from their submissions, to maintain the anonymity of the developers 1004 during review. A screening review 1016 may determine whether the required elements of the submission are included (e.g., all required files are present, and the proper headings in specified documents). The screening review can also determine that these elements appear complete.
In some embodiments, the screening 1016 includes a preliminary substantive review and selection/elimination by the entity that requested the competition. For example, if the competition is for a wireframe, the entity may select the wireframes that seem to be the best. This smaller group may then go on to the next step.
In some embodiments, the screener indicates that one or more submissions have passed the initial screening process and the reviewers are notified. The reviewers then evaluate the submissions in greater detail. In preferred embodiments, the review board scores the submissions 1018 according to the rules of the competition, documenting the scores using a scorecard. The scorecard can be any form, including a document, spreadsheet, online form, database, or other electronic interface or document. There may be any number of scorecards used by the reviewers, depending on the asset and the manner in which it is to be reviewed.
In some embodiments, the scores and reviews from the review board are aggregated into a final review and score. In some embodiments, the aggregation can include compiling information contained in one or more documents. Such aggregation can be performed by a review board member, or in one exemplary embodiment, the aggregation is performed using a computer-based aggregation system. In some embodiments, the facilitator 1000 or a designated review board member resolves discrepancies or disagreements among the members of the review board.
In one embodiment, the submission with the highest combined score is selected as the winning asset 1020. The winning asset may be used for implementation, production, or for review and input and/or specification for another competition. A prize, payment and/or recognition is given to the winning developer.
In some embodiments, in addition to reviewing the submissions, the review board may identify useful modifications to the submission that should be included in the asset prior to final completion. The review board documents the additional changes, and communicates this information to the developer 1004 who submitted the asset. In one embodiment, the primary review board member aggregates the comments from the review board. The developer 1004 can update the asset and resubmit it for review by the review board. This process can repeat until the primary review board member believes the submission has met all the necessary requirements. In some embodiments, the review board may withhold payment of the prize until all requested changes are complete.
In some embodiments, a portion of the payment to a developer of one asset in a series of assets is withheld until the until after other competitions that make use of the asset are complete. If any problems with the asset are identified in the further competitions, these are provided to the reviewer(s) and the developer, so that appropriate changes can be made by the developer 1004.
There can also be prizes, payments, and/or recognition for the developers of the submissions other than first place submissions. For example, the contestants that submit the second and/or third best submissions may also receive payment, which in some cases may be less than that of the winning contestant. Additional prizes may be awarded for ongoing participation and/or reliability. Payments also may be made for creative use of technology, submitting a unique feature, or other such submissions. In some embodiments, the software developers can appeal the score assigned to their design, program, or other submissions.
It should be understood that the development contest model may be applied to different portions of work that are required for the development of an overall asset. A series of development contests is particularly applicable to assets in which the development may be divided into stages or portions. It can be beneficial in many cases to size the assets developed in a single competition such that work may be completed in several hours or a few days. The less work required to develop a submission, the lower the risk for the contestants that they will not win.
A series of contests may be held to develop the software application. For example, a person, referred to as a customer, may have an idea or concept for a software application. The idea may be, for example, thought-out in detail or only with a high level of description. A series of development competitions (with each such competition, for example, operating as shown with respect to
The requirements documentation that is developed in the requirements contest 1101 may then be used as part of the contest specification 1010 (
When the wireframe contest 1102 is complete, a contest may be held for the development of a static prototype (e.g., an implementation of a web site in HTML or other markup language, typically without data persistence or other server-based functionality) using the wireframes as a starting point. The static prototype shows screen displays as they would look in the application, but does not have implemented functionality. The review of the static prototype may involve one or more peer reviewers (i.e., members of the contestant community), as well as the customer. The selection of a static prototype may be based on the degree to which it implements the requirements, the actual desires of the customer, and technical understanding and feasibility.
When the static prototype contest 1103 is complete, a contest may be held for the development of working prototypes (e.g., working implementations) based on the static prototypes. The working prototype is code that implements the requirements, wireframes, and static prototypes, along with any other comments or instructions provided by the customer. This working prototype may have certain restrictions or requirements that are described in the contest specification 1010 (
By having a customer involved in the requirements for and selection of the deliverables, a contest-based development process results in the efficient creation of software that the customer is happy with. At any stage in the process, if a customer is not happy with the final results, the customer can hold another contest to revise the previous asset with new or changed requirements.
The working prototype may be sufficient for some customers 1105 as a useful application. For others, the working prototype is a first step for confirming the desired requirements for a software application. Once they have used and tested the functionality of the working prototype, the working prototype may be used as the input to another series of competitions for development of an enterprise quality software application.
Shown as “STAGE 2” in the figure, another series of contests, beginning with the development of an application specification 1106 based on the working prototype may be held. In some such embodiments, a contest for development of an application specification 1106 may be held. The contest specification 1010 (
Once the winning application specification is selected, it may be used as the contest specification 1010 (
In some embodiments, the components specified in the winning architecture may be developed by holding a series of component design competitions 1108. The winning component designs are then used as input for component development competitions 1109. As illustrative examples, component design and development competitions have been described, for example, in the above-referenced U.S. patent application Ser. No. 11/035,783. When all components have been completed, they may be assembled in assembly competitions 1110. Finally, test scripts may be developed, tested, and run in testing competitions 1111. At the completion of STAGE 2, an enterprise-ready software application that meets the requirements is completed.
In some embodiments, reusable assets may be provided to increase the speed of development in each of the various stages. For example, templates, graphics, tools, design patterns, and so forth may be used to increase the productivity of the contestants, and to give a common style to make evaluation and integration easier.
It should be understood that one, two, or more of the steps may be performed in a different order, or combined or omitted. For example, in STAGE 1, there may not be a need for a requirements competition 1101. Rather, the contestants in the wireframe may start from the description provided by the customer, and ask questions in a forum or otherwise to generate their wireframes, without use of more formal requirements documentation. Likewise, there may be at any stage multiple competitions and/or multiple levels of competitions. For example, for a complex application, there may be an overall architecture design, and then individual competitions for the architecture design of subassemblies. The architecture of the subassemblies may be designed, needed components built, and the subassemblies assembled. The subassemblies may then be assembled into complete application in an additional competition or competitions. Testing competitions may be held for various portions of an application, for example, for a subassembly, or for a distinct portion of an application, such as a user interface with another competition held for testing of back-end functionality.
It also should be understood that development by a series of competitions is flexible, in that contests can be repeated if the results were not as expected, or if additional changes or new functionality is desired. Likewise, a customer can undertake as much work as it likes through development by other methods, for example, by using internal staff or outside consultants. For example, rather than holding a contest 1104 for a working prototype, a customer can take the static prototype, and develop the working prototype itself.
In some embodiments, where the assets developed outside of the contest environment are being used as the input to another contest, it may be useful to engage reviewers to review the asset. For example, a static prototype review can be conducted on a static prototype developed by a customer before that static prototype is used as input to the working prototype contest 1104. As another example, in STAGE 2, an entity might develop a specification itself, engage reviewers to review the specification, make any desired changes, then use the specification in a contest for some or all of the architecture, hold a contest for development of some or all of the components, and then assemble the application itself. A review can be conducted on the assembly, and a testing competition held to develop test scripts and other functionality.
In some embodiments, a “project plan” competition may be held prior to the competitions described. In the project plan competition, contestants develop a plan for development of the customer's project needs. For example, the plan may specify a series of development contests and timeframes for developing the desired software application. The plan may provide approximate costs, based on historical or other data, for similar competitions. The plan may include development strategies, and assumptions about customer participation. A project plan may be specified as a series of competitions, which may then be monitored through interaction with a competition management server. One asset developed in such a plan may be a configuration of contests as specified in a project management system, such as that described in co-pending U.S. patent application Ser. No. 11/755,909.
In one embodiment, a competition web site provides a registration process for a development project. Following project registration, a customizable dashboard, or “cockpit” is configured with a variety of functional widgets with which a customer can begin the process of having development work done by a member community. The customer can launch competitions and track projects through delivery and member payment. In some embodiments, the web site project-specific public or private ‘group’ pages, generated by the customer from the cockpit, on which the nature of the project can be described, relevant attachments can be viewed, and the customer can openly communicate with members via a bulletin board or forum to answer questions, receive suggestions, and so on. In some embodiments, these pages allow participants to see the evolution of the project, interact, collaborate, provide status reports, and make delivery.
In some embodiments, representatives of a customer, and members in different roles, may have access to different capabilities for the project. For example, some customers may be able to post projects, while others may only be able to review status, and other review and approve payments.
In some embodiments, a dashboard page serves as an entry point for customers to interact with the TopCoder Community. From this dashboard page, a registered customer may launch and track projects, send and receive messages and monitor a variety of information sources. The dashboard may include a profile box with information about the individual, and a link to other information, or to update the information, a button that allows the launching of a project, a button or link for help and “how-to” information, navigation to other pages, such as contests and discussion forums, and in some cases space for broadcast messages or advertising, search field, and so forth. In some implementations, the page is customizable, with a framework allowing for “widgets” to be dragged and dropped into place into a user-configured order.
In some embodiments, the dashboard includes an interface that provides information about the user's projects, which may be updated dynamically, such as the project name, the status of the project, the name of the personnel associated with the project, the status of the project, the status of payments, the number of contest participants, the number of submissions, the project phase, whether there are new forum posts (and in some cases, information or a link to the post(s)).
In some embodiments, the dashboard includes a messaging interface that indicates whether new messages are waiting, and allows a user to compose new messages. In some embodiments, a team information interface shows information about individuals who are working on a current project, or, for example, a past project, or have been designated for another reason to be included. An RSS feed interface provides an indication of whether content has been added to a particular web page. A news interface provides information about new features, or, for example, newly-available components that may be used in a competition.
In some embodiments, the date 501 may include a date (as shown) or a more specific date/time indication of when the competition is scheduled to begin. The indication may be localized for the time zone of an individual. The client 502 listing may be the sponsor of the competition, and/or the ultimate recipient of the asset to be developed. The client listing 502 may be omitted for certain competitions, or for certain viewers who are not authorized to have client information. The project listing 503 provides an indication of which project the competition is associated. There may not be a particular project, or the project may be the name of a larger asset (e.g., a software application, a multi-media presentation) for which multiple or a series of asset development competitions are being held. The project listing 503 may be omitted for certain competitions, or for certain viewers who are not authorized to have project information. For example, in some cases, only administrators have access to the client and project information, while potential and actual competitors do not.
The asset type 504 provides an indication of the type of asset that is to be developed. Some of the types shown in the example include Graphic Design, Component Design, Component Development, and Assembly. Other asset types would be appropriate for other competitions. In some embodiments, the assets may include each of the assets described with reference to
The name of the asset 507 gives a further indication of the technology and/or the larger asset that is being developed. For example, the Manifest Manager is shown as a .NET Generic component, and this competition is for the design of that component. Following design, there may be another competition for the coding of this component and such a competition may already be scheduled in anticipation of successful completion of this competition. Such a competition would have a similar listing (with a different date) and an indication that the type 504 is “Component Development.”
In this example, the name 507 of each asset is a link to a workspace for that asset. The workspace includes further information and details about the asset, as well as a web page allocated for the competition specification. In this example, the competition specification is provided in a “wiki” such that any user with sufficient permissions can make edits to the wiki, and add or delete text. The changes to the page made by a user can be viewed, so that a reviewer or administrator or another competitor can see who made what changes. In addition, there is a portion of the page dedicated to comments, such that a user can post comments, general or specific, about the technical description, competition logistics, or other specification information. Links to a “forum” is also provided, such that a conversation can take place among administrators (e.g., managers, architects), competitions sponsors and/or potential competitors about the competition.
The status 508 shown in this list is the status of the specification drafting, commenting, and review. The status of the specification can change from “Not Received” to “Comments” when a draft is made available for comments. Following the comment period, the status will change to “Final Review” which is when the designated reviewer will review the specification along with the designated architect and/or the designated manager. If review fails, the status becomes “Failed Review—Edits Required” until the specification passes review, and then it is “Complete.” Potential competitors thus can review the list, and review the specifications (if the potential competitor is qualified and has fulfilled any necessary prerequisites).
The names associated with the manager 510, architect 511 and reviewer 512 positions may be last names (as shown with the exemplary names listed), handles/pseudonyms, first names, and/or other identifiers. These names may be omitted for certain competitions, or for certain viewers who are not authorized to have full project information. For example, in some cases, only administrators have access to the names of the manager and architect information, while potential and actual competitors do not. In some cases, the names are provided, but they are “handles” (pseudonyms) used by members of a community of developers. In some cases, the handles displayed are designated for a role (e.g., Contest Administrator 231)
Referring generally to
In this example, the exemplary widgets include a “Competition Details” widget 602, which provides details about a competition. This includes some of the information from the list of
Although described here with reference to software, and useful when implemented with regard to software assets, the cooperatively developed asset can be any sort of tangible or intangible object that embodies intellectual property. As non-limiting examples, the techniques could be used for computer hardware and electronics designs, or other designs such as architecture, construction, or landscape design. Other non-limiting examples for which the techniques could be used include the development of all kinds of written documents and content such as documentation and articles for papers or periodicals (whether on-line or on paper), research papers, scripts, multimedia content (including without limitation video, audio, graphics images, cartoons, sounds, graphical presentations, business presentations, etc.), legal documents, and more.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6532448 *||Nov 19, 1999||Mar 11, 2003||Insightful Corporation||Contest server|
|US7162433 *||Feb 22, 2006||Jan 9, 2007||Opusone Corp.||System and method for interactive contests|
|US7292990 *||Apr 7, 2003||Nov 6, 2007||Topcoder, Inc.||System and method for software development|
|US20010025267 *||Dec 12, 2000||Sep 27, 2001||Stephen Janiszewski||System and method for facilitating bidding transactions and conducting project management utilizing software metric collection|
|US20030192029 *||Apr 7, 2003||Oct 9, 2003||Hughes John M.||System and method for software development|
|US20050240623 *||Oct 20, 2003||Oct 27, 2005||Kobza Kim P||Platform for management of Internet based public communications and public comment|
|US20060229929 *||Jun 7, 2006||Oct 12, 2006||Hughes John M||System and method for software development|
|US20060248504 *||Mar 14, 2006||Nov 2, 2006||Hughes John M||Systems and methods for software development|
|US20070220479 *||Jan 31, 2007||Sep 20, 2007||Hughes John M||Systems and methods for software development|
|US20070244570 *||Apr 17, 2006||Oct 18, 2007||900Seconds, Inc.||Network-based contest creation|
|US20090063177 *||Aug 29, 2007||Mar 5, 2009||East Edward W||Web-based services enabling structured secure communication related to the design and management of projects|
|1||*||Lapp H, Bala S, Balhoff JP, Bouck A, Goto N, Holland R, et al. The 2006 NESCent Phyloinformatics Hackathon: A Field Report. Evolutionary Bioinformatics 2007:3. 2007; 287-296.|
|2||*||TopCoder.com website, selected pages June06-Nov06, downloaded from web.archive.com|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7770143||Jan 19, 2007||Aug 3, 2010||Hughes John M||System and method for design development|
|US8909541 *||May 13, 2009||Dec 9, 2014||Appirio, Inc.||System and method for manipulating success determinates in software development competitions|
|US20100178978 *||Jul 15, 2010||Fairfax Ryan J||System and method for conducting competitions|
|US20100299650 *||May 20, 2009||Nov 25, 2010||International Business Machines Corporation||Team and individual performance in the development and maintenance of software|
|US20120191774 *||Jan 25, 2012||Jul 26, 2012||Vivek Bhaskaran||Virtual dial testing and live polling|
|Jun 16, 2008||AS||Assignment|
Owner name: TOPCODER, INC., CONNECTICUT
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JEFTS, ANTHONY;ELDRIDGE, GREGORY D.;FERNANDEZ-IVERN, JAVIER;AND OTHERS;REEL/FRAME:021099/0878;SIGNING DATES FROM 20080507 TO 20080609