|Publication number||US7836135 B2|
|Application number||US 11/431,225|
|Publication date||Nov 16, 2010|
|Filing date||May 9, 2006|
|Priority date||Jun 14, 2001|
|Also published as||EP1397768A1, EP1397768B1, US7076527, US7856479, US20030009526, US20060206574, US20070106742, WO2002103604A1|
|Publication number||11431225, 431225, US 7836135 B2, US 7836135B2, US-B2-7836135, US7836135 B2, US7836135B2|
|Inventors||Jerome R. Bellegarda, Devang Naik, Kim E. A. Silverman|
|Original Assignee||Apple Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (63), Non-Patent Citations (29), Classifications (9), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation application of U.S. patent application Ser. No. 09/881,986, filed Jun. 14, 2001 now U.S. Pat. No. 7,076,527.
The present invention relates generally to message filtering. More particularly, this invention relates to email filtering using latent semantic analysis.
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to the software and data as described below and in the drawings hereto: Copyright© 2000, Apple Computer, Inc., All Rights Reserved.
As the use of computers and the Internet have proliferated, so too has the use of email. Many businesses and consumers use email as a prominent means of communication. Not surprisingly, the exponential growth of the medium has also attracted the interest of commercial email advertisers. Commercial email advertisers obtain email addresses from a variety of sources, for example, from email vendors, or from commercial web sites, often without the permission of the owners of the email addresses. The email addresses may then be used to promote the products and services of the commercial email advertisers, or of the parties they represent.
The result is a deluge of unsolicited email received by hapless email users. One method to deal with unsolicited email is for a user to manually select and delete the unsolicited email. Other methods provide for recognizing a message sent in bulk to multiple recipients, and to either discard or tag the message as a possible unsolicited message. Still other methods maintain a database of addresses of known senders of unsolicited email and on receipt of the email, automatically discard those received from the known senders of unsolicited email. Still other methods use key-word filters. This method provides for scanning the subject and/or the body of the email message for some pre-determined keywords, and if detected, the message may be either discarded or tagged as suspicious.
Despite the methods described above, commercial email advertisers use ingenious methods to frustrate the efforts of email recipients. For example, to defeat the detection of bulk email, the email messages may be routed through a maze of servers so that ultimately, the message does not appear to be a bulk emailing. To defeat the system that tracks the address of known senders of unsolicited messages, the originating address of the unsolicited email may be changed often. To confuse keyword filter methods, the subject field of the email may be deceitfully titled, for example, “In response to your query”. Moreover, the key-word filtering method suffers from other significant problems, for example, when trying to filter out email messages from pornographic email advertisers using the word “sex”, legitimate anatomical or biological articles that include the word “sex” may also be eliminated.
A method and apparatus for filtering messages, in particular email messages is described herein. According to one aspect of the present invention, the method comprises determining a first semantic anchor corresponding with a first group of messages, for example, legitimate email messages and a second semantic anchor corresponding with a second group of messages, for example, unsolicited email messages. The method further determines a vector corresponding with an incoming message, compares the vector with at least one of the first semantic anchor and the second semantic anchor to obtain at least one comparison value, and filters the incoming message based on the comparison value.
Embodiments of the invention may be represented as a software product stored on a machine-accessible medium (also referred to as a computer-accessible medium or a processor-accessible medium). According to one aspect of the invention, the machine-accessible medium includes instructions that, when executed by a machine causes the machine to perform operations comprising determining a first semantic anchor corresponding with a first group of messages, for example, legitimate email messages. The machine-accessible medium includes further instructions for determining a second semantic anchor corresponding with a second group of messages, for example, unsolicited email messages. The machine-accessible medium includes further instructions for determining a vector corresponding with an incoming message, compares the vector with at least one of the first semantic anchor and the second semantic anchor to obtain at least one comparison value, and filters the incoming message based on the comparison value.
According to one aspect of the invention, the invention may be represented as an apparatus, e.g. computer system. The computer system comprises a bus, a data storage device coupled to the bus and a processor coupled to the data storage device, said processor to perform a method that comprises determining a first semantic anchor corresponding to a first group of messages. The processor also determines a second semantic anchor corresponding to a second group of messages. The processor further determines a vector corresponding to an incoming message, compares the vector corresponding to the incoming message with at least one of the first semantic anchor and the second semantic anchor to obtain a first comparison value and a second comparison value. The processor filters the incoming message based on the first comparison value and the second comparison value.
The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
Described is a method and apparatus for filtering email using latent semantic analysis.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known architectures, steps, and techniques have not been shown to avoid unnecessarily obscuring the present invention.
Parts of the description may be presented using terminology commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. Also, parts of the description may be presented in terms of operations performed through the execution of programming instructions. As well understood by those skilled in the art, these operations often take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through, for instance, electrical components.
The invention may utilize a distributed computing environment. In a distributed computing environment, program modules may be physically located in different local and remote memory storage devices. Execution of the program modules may occur locally in a stand-alone manner or remotely in a client/server manner. Examples of such distributed computing environments include local area networks, enterprise-wide computer networks, and the global Internet.
In addition, it should be understood that the programs, processes, method, etc. described herein are not related or limited to any particular computer or apparatus nor are they related or limited to any particular communication network architecture. Rather, various types of general purpose machines may be used with program modules constructed in accordance with the teachings described herein. Similarly, it may prove advantageous to construct a specialized apparatus to perform the method steps described herein by way of dedicated computer systems in a specific network architecture with hard-wired logic or programs stored in nonvolatile memory such as read only memory.
Various operations will be described as multiple discrete steps performed in turn in a manner that is helpful in understanding the present invention. However, the order of description should not be construed as to imply that these operations are necessarily performed in the order they are presented, or even order dependent. Lastly, repeated usage of the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may.
Latent semantic analysis (LSA) is a method that automatically uncovers the salient semantic relationships between words and documents in a given corpus. Discrete words are mapped onto a continuous semantic vector space, in which clustering techniques may be applied. The method for filtering email messages comprises determining a first semantic anchor corresponding with a first group of email messages, for example, legitimate email messages and a second semantic anchor corresponding with a second group of email messages, for example, unsolicited email messages. Determining a vector corresponding with an incoming email message, comparing the vector with at least one of the first semantic anchor and the second semantic anchor to obtain at least one comparison value, and filtering messages based on the comparison value.
Email filtering system 100 comprises an email training unit 105 that includes an email training corpus T, for example, a database comprising a collection of N1 legitimate email messages and N2 unsolicited email messages. In one embodiment, the legitimate and unsolicited email messages are obtained from the existing email received by a recipient. Alternate embodiments may allow for a user to manually classify each incoming email message until an adequate email training corpus T has been established. The words used in the collection of the legitimate email messages, and in the collection of the unsolicited email messages are from some underlying vocabulary v comprising, for example, the M most frequently used words in the language. In one embodiment, M may be ten thousand, and 1≧N1=N2≧150.
Co-occurrences unit 110 of email filtering system 100, comprises a two dimensional (M×2) matrix W formed using the email training corpus T. Matrix W essentially keeps track of which word is found in what document by keeping a record of the number of times each word appears in each legitimate and each unsolicited email message. In particular, entries ωij of matrix W reflects the extent to which each word ωi ε v appeared in the legitimate email message (j=1), or in an unsolicited email message (i=2). Various methods may be used to keep a record of the number of occurrences of a word in a document, for example, a simple normalized count of the number of occurrences of each word. However, in one embodiment, co-occurrence unit 110 uses function
that normalizes for document length and word entropy to form matrix W. ci,j denotes the number of times each word ωi occurs in the collection of legitimate email messages, and the number of times each word ωi occurs in the collection of unsolicited email messages. In equation (1) Nj, for j=1 and j=2, represents the total number of words in the collection of legitimate email messages, and unsolicited email messages. εi is the normalized entropy of ωi in the training email corpus T. (1−εi) is merely a weighting factor, or a word distribution factor, and is a measure of the distribution of a particular word in the email training corpus T. This is explained further below.
In one embodiment, co-occurrences unit 110 calculates εi using equation:
where N=N1+N2. By definition, 0≦εi≦1, with equality if and only if
Thus, a value of εi close to 1 indicates a word distributed across many email messages throughout the email training corpus T. However, a value of εi close to 0 indicates that the word is present only in a few email messages. Thus, the weighting factor is a measure of the distribution of a word across the training email corpus T. In particular, weighting factor (1−εi) is a measure of the indexing power of the word ωi.
After co-occurrences unit 110 constructs matrix W, Singular Value Decomposition (SVD) unit 115 decomposes matrix W, and subsequently obtains the semantic anchors
where U is the (M×2) left singular matrix with row vectors ui (1≦i≦M), S is the (2×2) diagonal matrix of singular values s1≧s2≧0, V is the (2×2) right singular matrix with row vectors vj (j=1,2), and T denotes matrix transposition. Thus, vector
and the second semantic anchor given by
are obtained after apropriate scaling by the diagonal matrix S. One skilled in the art will appreciate that Vi T 215 and V2 T 220 are unscaled semantic anchors in (2×2) matrix VT 235, and may be easily converted to 2-dimensional vectors
The 2-dimensional vector
One skilled in the art will appreciate that equation (6) is an approximate representation of the message in the existing LSA space. Since the new email message was not part of the original SVD extraction, words in the new email message, not found in training corpus T may cause the SVD expansion to no longer apply. As such, in one embodiment, an optional feed back path 180, as illustrated in
The invention contemplates capturing structural associations between words. Hence, two words whose representations are “close” (in some suitable metric) tend to appear in the same kind of documents, whether or not they actually occur within identical word contexts in the documents. Each semantic anchor
After calculating the semantic anchors
for j=1, 2. Other methods may be employed to calculate the measure of closeness K including, but not limited to, calculating the length of the normals between vectors
After calculating the measure of closeness K, logic unit 165 determines whether the new email is unsolicited 170, legitimate 175, or ambiguous 180. In one embodiment, if
If the angle between
As an example consider the following email messages received by a person in the fishing business: (a) Fishing is excellent in the south bank of the river, and (b) The Merchant bank has high interest rates. Although both email messages have the word ‘bank’ in the text of the message, the method described will properly classify message (a) as a legitimate email message and message (b) as an unsolicited email message.
The email training corpus T is developed using existing email messages of the user in the fishing business. After the email training corpus T is generated by email training unit 105, co-occurrences unit 110 generates matrix W using the email training corpus T. SVD unit 115 decomposes matrix W and obtains semantic anchors
At 320, an incoming email message is received, and at 325, vector
At 335, a determination is made whether the new email message is legitimate. If the angle between
At 350 a determination is made whether the new email message is unsolicited. If the angle between
However, if the angle between
Embodiments of the email filtering system may be employed individually on a machine for a particular user or on a central machine, e.g., an email server, to filter out email messages for a group of email recipients. Alternate embodiments may include employing the email filtering system on a server or other device that communicates with a remote user, for example, a user using a wireless device such as a wireless personal digital assistant (PDA) or wireless palm top computer, so that the limited memory of the wireless device is not unnecessarily filled with unsolicited email messages. Alternate embodiments may employ the email filtering system on the PDA and unsolicited messages may be discarded as soon as they are received.
In general, such computer systems as illustrated by
Display device 405 is coupled to processor 402 through bus 401 and provides graphical output for computer system 400. Input devices 406 such as a keyboard or mouse are coupled to bus 401 for communicating information and command selections to processor 402. Also coupled to processor 402 through bus 401 is an input/output interface 410 which can be used to control and transfer data to electronic devices (printers, other computers, etc.) connected to computer system 400. Computer system 400 includes network devices 408 for connecting computer system 400 to a network 414 through which email messages may be received, e.g., from remote device 412. Network devices 408, may include Ethernet devices, phone jacks and satellite links. It will be apparent to one of ordinary skill in the art that other network devices may also be utilized.
One embodiment of the invention may be stored entirely as a software product on mass storage 407. Another embodiment of the invention may be embedded in a hardware product, for example, in a printed circuit board, in a special purpose processor, or in a specifically programmed logic device communicatively coupled to bus 401. Still other embodiments of the invention may be implemented partially as a software product and partially as a hardware product.
Embodiments of the invention maybe represented as a software product stored on a machine-accessible medium (also referred to as a computer-accessible medium or a processor-accessible medium) as illustrated in
Experiments conducted using one embodiment of the method and apparatus of the present invention revealed that for a database comprising one legitimate email message N1 and one unsolicited email message N2 in the training corpus T the email filtering system performed reasonably well. An exponential increase in the performance of the email filtering system occurred as the values of N1 and N2 approached 50. Subsequent increases in the values of N1 and N2 revealed a relative plateau in the performance of the email filtering system. In one embodiment, more than 95% of a user's incoming email messages were properly classified, with approximately less than 3% of the user's email messages being passed to the user for disambiguation. A significantly lower misclassification rate was observed as compared with the misclassification rate of prior art methods.
While there has been illustrated and described what are presently considered to be example embodiments of the present invention, it will be understood by those skilled in the art that various other modifications may be made, and equivalents may be substituted, without departing from the true scope of the invention. Additionally, many modifications may be made to adapt a particular situation to the teachings of the present invention without departing from the central inventive concept described herein. Therefore, it is intended that the present invention not be limited to the particular embodiments disclosed, but that the invention include all embodiments falling within the scope of the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4885704||Sep 22, 1987||Dec 5, 1989||Kabushiki Kaisha Toshiba||Electronic document filing apparatus with icon selection|
|US4945475||Nov 21, 1989||Jul 31, 1990||Apple Computer, Inc.||Hierarchical file system to provide cataloging and retrieval of data|
|US5021976||Nov 14, 1988||Jun 4, 1991||Microelectronics And Computer Technology Corporation||Method and system for generating dynamic, interactive visual representations of information structures within a computer|
|US5060135||Nov 1, 1988||Oct 22, 1991||Wang Laboratories, Inc.||Apparatus for manipulating documents in a data processing system utilizing reduced images of sheets of information which are movable|
|US5140677||May 11, 1990||Aug 18, 1992||International Business Machines Corporation||Computer user interface with window title bar mini-icons|
|US5202828||May 15, 1991||Apr 13, 1993||Apple Computer, Inc.||User interface system having programmable user interface elements|
|US5241671||Oct 26, 1989||Aug 31, 1993||Encyclopaedia Britannica, Inc.||Multimedia search system using a plurality of entry path means which indicate interrelatedness of information|
|US5247437||Oct 1, 1990||Sep 21, 1993||Xerox Corporation||Method of managing index entries during creation revision and assembly of documents|
|US5287448||Mar 24, 1993||Feb 15, 1994||Apple Computer, Inc.||Method and apparatus for providing help information to users of computers|
|US5297249||Oct 31, 1990||Mar 22, 1994||International Business Machines Corporation||Hypermedia link marker abstract and search services|
|US5305435||May 7, 1993||Apr 19, 1994||Hewlett-Packard Company||Computer windows management system and method for simulating off-screen document storage and retrieval|
|US5341293||Sep 3, 1992||Aug 23, 1994||Apple Computer, Inc.||User interface system having programmable user interface elements|
|US5347628||Jan 18, 1990||Sep 13, 1994||International Business Machines Corporation||Method of graphically accessing electronic data|
|US5349658||Nov 1, 1991||Sep 20, 1994||Rourke Thomas C O||Graphical user interface|
|US5353391||May 6, 1991||Oct 4, 1994||Apple Computer, Inc.||Method apparatus for transitioning between sequences of images|
|US5377354||Jun 8, 1993||Dec 27, 1994||Digital Equipment Corporation||Method and system for sorting and prioritizing electronic mail messages|
|US5442778||Nov 12, 1991||Aug 15, 1995||Xerox Corporation||Scatter-gather: a cluster-based method and apparatus for browsing large document collections|
|US5442795||Nov 19, 1990||Aug 15, 1995||Wang Laboratories, Inc.||System and method for viewing icon contents on a video display|
|US5619709||Nov 21, 1995||Apr 8, 1997||Hnc, Inc.||System and method of context vector generation and retrieval|
|US5734916 *||Jun 1, 1994||Mar 31, 1998||Screenplay Systems, Inc.||Method and apparatus for identifying, predicting, and reporting object relationships|
|US5857179 *||Sep 9, 1996||Jan 5, 1999||Digital Equipment Corporation||Computer method and apparatus for clustering documents and automatic generation of cluster keywords|
|US5987446 *||Nov 12, 1996||Nov 16, 1999||U.S. West, Inc.||Searching large collections of text using multiple search engines concurrently|
|US6029195 *||Dec 5, 1997||Feb 22, 2000||Herz; Frederick S. M.||System for customized electronic identification of desirable objects|
|US6072942 *||Sep 18, 1996||Jun 6, 2000||Secure Computing Corporation||System and method of electronic mail filtering using interconnected nodes|
|US6161130 *||Jun 23, 1998||Dec 12, 2000||Microsoft Corporation||Technique which utilizes a probabilistic classifier to detect "junk" e-mail by automatically updating a training and re-training the classifier based on the updated training set|
|US6192360 *||Jun 23, 1998||Feb 20, 2001||Microsoft Corporation||Methods and apparatus for classifying text and for building a text classifier|
|US6205472 *||Mar 17, 1999||Mar 20, 2001||Tacit Knowledge System, Inc.||Method and apparatus for querying a user knowledge profile|
|US6208971 *||Oct 30, 1998||Mar 27, 2001||Apple Computer, Inc.||Method and apparatus for command recognition using data-driven semantic inference|
|US6243724||Aug 8, 1994||Jun 5, 2001||Apple Computer, Inc.||Method and apparatus for organizing information in a computer system|
|US6393460 *||Aug 28, 1998||May 21, 2002||International Business Machines Corporation||Method and system for informing users of subjects of discussion in on-line chats|
|US6460036 *||Dec 5, 1997||Oct 1, 2002||Pinpoint Incorporated||System and method for providing customized electronic newspapers and target advertisements|
|US6578025 *||Jun 11, 1999||Jun 10, 2003||Abuzz Technologies, Inc.||Method and apparatus for distributing information to users|
|US6613101||Feb 27, 2001||Sep 2, 2003||Apple Computer, Inc.||Method and apparatus for organizing information in a computer system|
|US6643686 *||Dec 16, 1999||Nov 4, 2003||At&T Corp.||System and method for counteracting message filtering|
|US6654787 *||Dec 31, 1998||Nov 25, 2003||Brightmail, Incorporated||Method and apparatus for filtering e-mail|
|US6675162 *||May 7, 2001||Jan 6, 2004||Microsoft Corporation||Method for scanning, analyzing and handling various kinds of digital information content|
|US6701305 *||Oct 20, 2000||Mar 2, 2004||The Boeing Company||Methods, apparatus and computer program products for information retrieval and document classification utilizing a multidimensional subspace|
|US6718368 *||Jun 3, 1999||Apr 6, 2004||General Interactive, Inc.||System and method for content-sensitive automatic reply message generation for text-based asynchronous communications|
|US6807566 *||Aug 16, 2000||Oct 19, 2004||International Business Machines Corporation||Method, article of manufacture and apparatus for processing an electronic message on an electronic message board|
|US6816885 *||Sep 21, 2000||Nov 9, 2004||International Business Machines Corporation||Method and system to handle large volume of E-mail received from a plurality of senders intelligently|
|US7076527 *||Jun 14, 2001||Jul 11, 2006||Apple Computer, Inc.||Method and apparatus for filtering email|
|US20020116463 *||Feb 20, 2001||Aug 22, 2002||Hart Matthew Thomas||Unwanted e-mail filtering|
|US20020116641 *||Feb 22, 2001||Aug 22, 2002||International Business Machines Corporation||Method and apparatus for providing automatic e-mail filtering based on message semantics, sender's e-mail ID, and user's identity|
|US20020129111 *||Jan 15, 2001||Sep 12, 2002||Cooper Gerald M.||Filtering unsolicited email|
|US20020169954 *||Jun 22, 2001||Nov 14, 2002||Bandini Jean-Christophe Denis||Method and system for e-mail message transmission|
|US20030041178 *||Mar 30, 2001||Feb 27, 2003||Lev Brouk||System and method for routing messages between applications|
|US20070106742 *||Dec 20, 2006||May 10, 2007||Bellegarda Jerome R||Method and apparatus for filtering email|
|DE3714172A1||Apr 28, 1987||Nov 19, 1987||Hitachi Ltd||Einrichtung zum durchsuchen von dokumenten in einem dokumentenablagesystem|
|GB2347053A||Title not available|
|JP2504439B2||Title not available|
|JPH022458A||Title not available|
|JPH0449411A||Title not available|
|JPH01276347A||Title not available|
|JPH02158871A||Title not available|
|JPH02297286A||Title not available|
|JPH03172966A||Title not available|
|JPH03191467A||Title not available|
|JPH03232341A||Title not available|
|JPS6129277A||Title not available|
|JPS63214831A||Title not available|
|JPS63228327A||Title not available|
|WO2000/26795A1||Title not available|
|WO2000026795A1||Oct 18, 1999||May 11, 2000||Justsystem Pittsburgh Research Center, Inc.||Method for content-based filtering of messages by analyzing term characteristics within a message|
|1||"Addition of an Expansion Module to System 7-Further Reinforcement of Human Interface," Nikkei Electronics, Jan. 6, 1992, No. 544, pp. 269-274.|
|2||"Addition of an Expansion Module to System 7—Further Reinforcement of Human Interface," Nikkei Electronics, Jan. 6, 1992, No. 544, pp. 269-274.|
|3||*||"Indexing by Latent Semantic Analysis", Deerwester et al. 1990.|
|4||*||"Intelligent email management system", Tsai et al., 1999.|
|5||*||"Personalized Information Delivery: An Analysis of Information Filtering Methods", Foltz et al. 1992.|
|6||*||"ProntoMail Professional Edition", Alwang, Greg, May, 1998.|
|7||*||"Strategies for filtering email messages combining content-based and sociological filtering with user-stereotypes", Shoval et al. 1999.|
|8||"Trend Letter from U.S.A.," Montly ASCII, Aug. 1, 1991, pp. 230-231.|
|9||"What is Windows 3.0? Part 1. What Sort of Environment Will it Be?," Monthly ASCII Magazine, ASCII K.K., Mar. 1, 1991, vol. 15, #3, pp. 229-241.|
|10||Bellegarda, Jerome R., "Exploiting Latent Semantic Information in Statistical Language Modeling", Proceedings of the IEEE, 88(8)1279-1296, Aug. 2000.|
|11||Bellegarda, Jerome R.; Butzberger, John W.; Chow, Yen-Lu; Coccaro, Noah B.; and Naik, Devang. "A Novel Word Clustering Algorithm Based on Latent Semantic Analysis." International Conference on Acoustics, Speech, and Signal Processing, Conference Proceedings, vol. 1 (May 7-10, 1996), pp. 172-175.|
|12||Borland, Quattro Pro User's Guide, 1989, p. 229, 235-236, 251-255, 275-286.|
|13||Cutting, Douglas et al., "Scatter/Gather: A Cluster-Based Approach to Browsing Large Documents Collections," 1992.|
|14||Elisabeth Crawford, Automatic Induction of Rules for E-Mail Classification. Proceedings of the Sixth Australian Document Computing Symposium, Coffs Harbour, Australia, Dec. 7, 2001.|
|15||EP Search Report for EP 0955591, dated Mar. 19, 2004.|
|16||Foltz, Peter W. "Using Latent Semantic Indexing for Information Filtering." Proceedings of the Conference on Office Information Systems, (1990), pp. 40-47.|
|17||Foltz, Peter W. and Dumais, Susan T. "Personalized Information Delivery: An Analysis of Information Filtering Methods." Communications of the ACM. vol. 35, No. 12 (Dec. 1992), pp. 51-60.|
|18||Li, Y.H. and Jain, A.K. "Classification of Text Documents." The Computer Journal, vol. 41, No. 8 (1998), pp. 537-546.|
|19||Malone, T.W., "How do People Organize Their Desks?, Implications for the Design of Office Information Systems," ACM Transactions on Office Information System, vol. 1, No. 1, Jan. 1983, pp. 99-112.|
|20||Mander, Richard et al., "A Pile' Metaphor for Supporting Casual Organization of Information," 1992.|
|21||Mander, Richard et al., specification, claims, abstract, and figures as filed Apr. 30, 1992, assigned U.S. Appl. No. 07/876,921, now abandoned.|
|22||Mehran Sahami, et al, A Bayesian Approach to Filtering Junk E-Mail. In AAAI-98 Workshop on Learning for Text Categorization, 1998.|
|23||Miller, P. et al., "Business an End-User Oriented Application Development Language," SIG-MOD Record. 12, 1, (Oct. 1981), pp. 38-69.|
|24||PCT International Search Report for PCT Application No. PCT/US93/02878, mailed Jul. 14, 1993, 4 pages.|
|25||PCT International Search Report for PCT Int'l Appln. No. PCT/US02/19015, mailed Nov. 26, 2002 (7 pages).|
|26||Register, Michael et al., "A Hybrid Architecture for Text Classification," 1992.|
|27||Richard B. Segal and Jeffrey O. Kephart, MailCat: An Intelligent Assistant for Organizing E-Mail. Proceedings of the Third Internation Conference on Autonomous Agents, pp. 276-282, Seattle, WA, 1999.|
|28||Salton, Gerard et al., "Term Weighting in Automatic Text Retrieval," 1987.|
|29||Salton, Gerard et al., "Term-Weighting in Automatic Text Retrieval," Information Processing & Management. vol. 24, No. 5, pp. 513-523, 1988.|
|International Classification||G06Q10/10, G06F15/16, H04L12/58|
|Cooperative Classification||G06Q10/107, H04L51/12|
|European Classification||H04L12/58F, G06Q10/107, H04L51/12|
|Apr 30, 2007||AS||Assignment|
Owner name: APPLE INC., CALIFORNIA
Free format text: CHANGE OF NAME;ASSIGNOR:APPLE COMPUTER, INC.;REEL/FRAME:019240/0979
Effective date: 20070109
Owner name: APPLE INC.,CALIFORNIA
Free format text: CHANGE OF NAME;ASSIGNOR:APPLE COMPUTER, INC.;REEL/FRAME:019240/0979
Effective date: 20070109
|Nov 22, 2011||CC||Certificate of correction|
|Apr 16, 2014||FPAY||Fee payment|
Year of fee payment: 4