WO2002010982A2 - Computer system for collecting information from web sites - Google Patents
Computer system for collecting information from web sites Download PDFInfo
- Publication number
- WO2002010982A2 WO2002010982A2 PCT/US2001/022426 US0122426W WO0210982A2 WO 2002010982 A2 WO2002010982 A2 WO 2002010982A2 US 0122426 W US0122426 W US 0122426W WO 0210982 A2 WO0210982 A2 WO 0210982A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- web
- site
- web site
- pages
- determining
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/953—Organization of data
- Y10S707/959—Network
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99935—Query augmenting and refining, e.g. inexact access
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99936—Pattern matching access
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99937—Sorting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99943—Generating database or data structure, e.g. via user interface
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
- Y10S707/99945—Object-oriented database structure processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99948—Application of database or data structure, e.g. distributed, multimedia, or image
Definitions
- a global computer network e.g., the internet
- a global computer network is formed of a plurality of computers coupled to a communication line for communicating with each other.
- Each computer is referred to as a network node.
- Some nodes serve as information bearing sites while other nodes provide connectivity between end users and the information bearing sites.
- the explosive growth of the Internet makes it an essential component of every business, organization and institution strategy, and leads to massive amounts of information being placed in the public domain for people to read and explore.
- the type of information available ranges from information about companies and their products, services, activities, people and partners, to information about conferences, seminars, and exhibitions, to news sites, to information about universities, schools, colleges, museums and hospitals, to information about government organizations, their purpose, activities and people.
- the Internet became the venue of choice for every organization for providing pertinent, detailed and timely information about themselves, their cause, services and activities.
- the Internet essentially is nothing more than the network infrastructure that connects geographically dispersed computer systems. Every such computer system may contain publicly available (shareable) data that are available to users connected to this network. However, until the early 1990's there was no uniform way or standard conventions for accessing this data. The users had to use a variety of techniques to connect to remote computers (e.g. telnet, ftp, etc) using passwords that were usually site-specific, and they had to know the exact directory and file name that contained the information they were looking for.
- remote computers e.g. telnet, ftp, etc
- the World Wide Web was created in an effort to simplify and facilitate access to publicly available information from computer systems connected to the Internet.
- a set of conventions and standards were developed that enabled users to access every Web site (computer system connected to the Web) in the same uniform way, without the need to use special passwords or techniques.
- Web browsers became available that let users navigate easily through Web sites by simply clicking hyperlinks (words or sentences connected to some Web resource).
- the sheer size and explosive growth of the Web has created the need for tools and methods that can automatically search, index, access, extract and recombine information and knowledge that is publicly available from Web resources.
- Web domain is an Internet address that provides connection to a Web server (a computer system connected to the Internet that allows remote access to some of its contents).
- URL stands for Uniform Resource Locator.
- URLs have three parts: the first part describes the protocol used to access the content pointed to by the URL, the second contains the directory in which the content is located, and the third contains the file that stores the content: ⁇ protocol> : ⁇ domain> ⁇ directory> ⁇ file>
- ⁇ protocol> For example: http://www.corex.com/bios.html http://www.cardscan.com/index.html http://fn.cnn.com/archives/may99/pr37.html ftp://shiva.lin.com/soft/words.zip
- the ⁇ protocol> part may be missing. In that case, modern Web browsers access the URL as if the http:// prefix was used, h addition, the ⁇ file> part may be missing. In that case, the convention calls for the file "index.html" to be fetched.
- Web Page Web page is the content associated with a URL. hi its simplest form, this content is static text, which is stored into a text file indicated by the URL. However, very often the content contains multi-media elements (e.g. images, audio, video, etc) as well as non-static text or other elements (e.g. news tickers, frames, scripts, streaming graphics, etc). Nery often, more than one files form a Web page, however, there is only one file that is associated with the URL and which initiates or guides the Web page generation.
- multi-media elements e.g. images, audio, video, etc
- non-static text or other elements e.g. news tickers, frames, scripts, streaming graphics, etc.
- Web browser is a software program that allows users to access the content stored in Web sites. Modern Web browsers can also create content "on the fly”, according to instructions received from a Web site. This concept is commonly referred to as “dynamic page generation”. In addition, browsers can commonly send information back to the Web site, thus enabling two-way communication of the user and the Web site.
- Hyperlink Hyperlink is an element in a Web page that links to another part of the same Web page or to an entirely different Web page.
- links on that page can be typically activated by clicking on them, in which case the Web browser opens the page that the link points to.
- the visual component can be text (often colored and underlined) or it can be a graphic (a small image). In the latter case, there is optionally some hidden text associated with the link, which appears on the browser window if the user positions the mouse pointer on the link for more than a few seconds.
- the text associated with a link will be referred to as "link text”
- the target URL associated with a link will be referred to as "link URL”.
- search engines that index millions of Web pages based on keywords have been developed. Some of these search engines have a user-friendly front end that accepts natural languages queries, hi general, these queries are analyzed to extract the keywords the user is possibly looking for, and then a simple keyword-based search is performed through the engine's indexes.
- this essentially corresponds to querying one field only in a database and it lacks the multi-field queries that are typical on any database system.
- Web queries cannot become very specific; therefore they tend to return thousands of results of which only a few may be relevant.
- the "results" returned are not specific data, similar to what database queries typically return; instead, they are lists of Web pages, which may or may not contain the requested answer.
- the information needs to be structured, so that it can be stored in database format. Since the Web contains mostly unstructured information, methods and techniques are needed to extract data and discover patterns in the Web in order to transform the unstructured information into structured data.
- Examples of some well-known search engines today are Yahoo, Excite, Lycos, Northern Light, AltaNista, Google, etc.
- Examples of inventions that attempt to extract structured data from the Web are 5, 6, and 7.
- These two separate groups of applications have different approaches to the problem of Web information retrieval; however, they both share a common need: they need a tool to "feed” them with pages from the Web so that they can either index those pages, or extract data.
- This tool is usually an automated program (or, "software robot") that visits and traverses lists of Web sites and is commonly referred to as "Web crawler". Every search engine or Web data extraction tool uses one or more Web crawlers that are often specialized in finding and returning pages > with specific features or content.
- these software robots are "smart" enough to optimize their traversal of Web sites so that they spend the minimum possible time in a Web site but return the maximum number of relevant Web pages.
- the Web is a vast repository of information and data that grows continuously, information traditionally published in other media (e.g. manuals, brochures, magazines, books, newspapers, etc.) is now increasingly published either exclusively on the Web, or in two versions, one of which is distributed through the Web.
- older information and content from traditional media is now routinely transferred into electronic format to be made available in the Web, e.g. old books from libraries, journals from professional associations, etc.
- the Web becomes gradually the primary source of information in our society, with other sources (e.g. books, journals, etc) assuming a secondary role.
- the Web becomes the world's largest information repository, many types of public information about people become accessible through the Web.
- club and association memberships, employment information, even biographical information can be found in organization Web sites, company Web sites, or news Web sites.
- many individuals create personal Web sites where they publish themselves all kinds of personal information not available from any other source (e.g. resume, hobbies, interests, "personal news", etc).
- people often use public forums to exchange e-mails, participate in discussions, ask questions, or provide answers. E-mail discussions from these forums are routinely stored in archives that are publicly available through the Web; these archives are great sources of information about people's interests, expertise, hobbies, professional affiliations, etc.
- the invention method for searching for people and organization information on Web pages, in a global computer network comprises the steps of: accessing a Web site of potential interest, the Web site having a plurality of Web pages, determining a subset of the plurality of Web pages to process, and for each Web page in the subset, (i) determining types of contents found on the Web page, and (ii) based on the determined content types, enabling extraction of people and organization information from the Web page.
- the step of accessing includes obtaining domain name of the Web site, and the step of determining content types includes collecting external links and other domain names. Further, the step of obtaining domain names includes receiving the collected external links and other domain names from the step of determining content types.
- the step of determining the subset of Web pages to process includes processing a hsting of internal links and selecting from remaining internal links as a function of keywords.
- the step of determining a subset of Web pages to process includes: extracting from a script a quoted phrase ending in ".ASP", ".HTM” or “.HTML”; and treating the extracted phrase as an internal link.
- the step of determining the subset of Web pages to process includes determining if a subject Web page contains a listing of press releases or news articles, and if so, following each internal link in the listing of press releases/news articles.
- the step of accessing includes determining whether the Web site has previously been accessed for searching for people and organization information.
- the invention includes obtaining a unique identifier for the Web site; and comparing the unique identifier to identifiers of past accessed Web sites to determine duplication of accessing a same Web site.
- the step of obtaimng a unique identifier may further include forming a signature as a function of home page of the Web site.
- Another aspect of the present invention provides time limits or similar respective thresholds for processing a Web site and a Web page, respectively.
- the present invention maintains a domain database storing, for each Web site, indications of: Web site domain name; name of content owner; site type of the Web site; frequency at which to access the Web site for processing; date of last accessing and processing; outcome of last processing; number of Web pages processed; and number of data items found in last processing.
- processing means e.g., a crawler
- Fig. 1 is a block diagram illustrating the main components of a system embodying the present invention and the data flow between them.
- Fig. 2 is a flowchart of the crawling process employed by the invention system of Fig. 1.
- Fig. 3 is a flowchart of the function that examines and processes newly found links during crawling.
- the present invention is a software program that systematically and automatically visits Web sites and examines Web pages with the goal of identifying potentially interesting sources of information about people and organizations. This process is often referred to as “crawling” and thus the terms “Crawler” or “software robot” will both be used in the next sections to refer to the invention software program.
- the input to the Crawler 11 is the domain 10 (URL address) of a Web site.
- the main output of Crawler 11 is a set of Web pages 12 that have been tagged according to the type of information they contain (e.g. "Press release”, “Contact info”, “Management team info + Contact info”, etc). This output is then passed to other components of the system (i.e. data extractor) for further processing and information extraction.
- the Crawler 11 also collects/extracts a variety of other data, including the type of the Web site visited, the organization name that the site belongs to, keywords that describe that organization, etc. This extracted data is stored in a Web domain database 14.
- a high level description of the Crawler's 11 functionality and how it is used with a data-extraction system is as follows and illustrated in Fig. 2: a) A database 14 is provided to the system with a list of domains and associated information for each domain (e.g. date of last visit by the Crawler
- each Crawler 11 picks an "available” domain from the database 14 and starts crawling it (a domain is "available” if none of the other Crawlers 11 is processing it at the time). All the domains that have been currently assigned to some Crawler 11 are marked as "unavailable”.
- the Crawler 11 visits pages in the given domain by starting from the root (home) page and follows recursively the links it finds if the links belong to the current domain as illustrated by the loop of steps 29, 30, 27, 28, 21, 19,
- the Crawler 11 first loads the home page (step 22) and determines whether the corresponding Web site is a duplicate of a previously processed site (step 23), detailed later. If the Crawler 11 is unsuccessful at loading the home page or if the site is determined to be a duplicate, then Crawler processing ends 46. If the Web site is determined to be non-duplicative, then Crawler 11 identifies the site type and therefrom the potential or probable structure of the contents at that site (step 24).
- Crawler 11 initializes 26 a working table 16 (Fig. 1) held in Crawler memory and referred to as the "links to visit" table 16 further detailed in Fig. 3.
- Crawler 11 selects and processes internal links (i.e., links belonging to the current domain), one at a time, from this table 16.
- Crawler 11 loads 27 the Web page corresponding to the link, (ii) examines and classifies 28 the Web page, (iii) collects 21 from the Web page and prunes 19 new internal links to process, and (iv) collects 18 new domains/URL addresses of other Web sites to crawl.
- the step of collecting 21 new internal links and updating table 16 therewith is further described below in Fig. 3.
- the Crawler 11 examines each Web page it visits and decides if it contains interesting information or not. For each page that contains interesting information, the Crawler 11 assigns a type to it that denotes the type of information the subject Web page contains, and then it saves (step 42) the page in a storage medium 48 as detailed below.
- the Crawler 11 maintains a table in internal crawler memory and stores in the table (i) the links for all the interesting pages it finds, (ii) the location of the saved pages in the storage medium 48, and (iii) an indication of type of data each interesting page contains.
- Crawler 11 determines the content owner's name for the site (step 40) and saves the determined name in domain database 14. Further the Crawler 11 saves interesting pages found at this site (step 42) in data store 48 (Fig. 1). The Crawler 11 saves (step 44) in the domain database 14 the off-site links it finds as potential future crawling starting points. Accordingly, the invention system must maintain and grow a comprehensive database 14 of domain URLs with additional information about each domain. This information includes: • Domain URL • Name of owner of the URL as identified from the Web site (organization name)
- Size of domain i.e., number of Web pages
- This database 14 is used by the Crawler 11 in selecting the domain to visit next, and it is also updated by the Crawler 11 after every crawl session as described above in steps 40 and 44 of Fig. 2.
- every domain is associated with some "visiting frequency”. This frequency is determined by how often the domain is expected to significantly change its content, e.g. for news sites the visiting frequency may be "daily", for conference sites “weekly”, whereas for companies "monthly” or "quarterly”.
- step 40 of Fig. 2 one important task that the Crawler
- Crawler 11 identifies the site's owner name as "ABC Corporation", then a list of people found in a paragraph headed "Management Team" can be safely assumed to be employees of "ABC Corporation".
- the current invention uses a system based on Bayesian Networks described in Invention 1 as disclosed in the related Provisional Application No. 60/221 ,750 filed on July 31 , 2000 for a "Computer Database Method and Apparatus”.
- a problem that the Crawler 11 faces is to be able to resolve duplicate sites.
- Duplicate sites appear when an organization uses two or more completely different domain URLs that point to the same site content (same Web pages).
- a signature can be as simple as a number or as complex as the whole site structure.
- Another way to address the problem is to completely ignore it and simply recrawl the duplicate site. But this would result in finding and extracting duplicate information which may or may not pose a serious problem.
- the organization name as it is identified by the Crawler could be used as the site's signature.
- the probability of having two different organizations with the same name is not negligible.
- the Crawler has to crawl at least two levels deep into the Web site.
- a signature should be created by only processing the home page of a Web site. After all, a human needs to look only at the home page to decide if two links point to the same site or to different sites. Three techniques that only examine the home page are outlined next.
- Every Web page has some structure at its text level, e.g. paragraphs, empty lines, etc.
- a signature for a page may be formed by taking the first letter of every paragraph and a space for every empty line, and putting them in a row to create a string. This string can be appended then to the page's title, to result in a text
- signature This text signature may finally be transformed into a number by a hash function, or used as it is.
- Another way to create a text signature is to put the names of all pages that are referenced in the home page in a row creating a long string (e.g. if the page has links: news/basket/todayscore.html, contact/address.html, contact/directions/map. html, ... the string would be:
- An alternative way to create a signature is to scan the home page and create a list of the items the page contains (e.g. text, image, frame, image, text, link, text, ). This list can then be encoded in some convenient fashion, and be stored as a text string or number.
- one element of the home page that is likely to provide a unique signature in many cases is its title. Usually the title (if it exists) is a whole sentence which very often contains some part of the organization name, therefore making it unique for organization sites. The uniqueness of this signature can be improved by appending to the title some other simple metric derived from the home page, e.g. the number of paragraphs in the page, or the number of images, or the number of external links, etc.
- Signature comparison can either be performed by directly comparing (i.e., pattern/character matching) signatures looking for a match, or, if the signatures are stored as text strings, then a more flexible approximate string matching can be performed. This is necessary because Web sites often make small modifications to their Web pages that could result in a different signature.
- the signature comparison scheme that is employed should be robust enough to accommodate small Web site changes. Approximate string matching algorithms that result in a matching "score" may be used for this purpose.
- the Crawler 11 As described at steps 18 and 21 in Fig. 2, as the Crawler 11 traverses the Web site, it collects and examines the links it finds on a Web page. If, a link is external (it points to another Web site) then Crawler 11 saves the external domain URL in the domain database 14 as a potential future crawling point. If a link is internal (points to a page in the current Web site) then the Crawler 11 examines the link text and URL for possible inclusion into the table 16 list of "links to visit". Note that when the Crawler 11 starts crawling a Web site, it only has one link, which points to the site's home page. In order to traverse the site though it needs the links to all pages of the site.
- Fig. 3 is a flow chart of this algorithm/(process) 58.
- the process 58 begins 32 with an internal link (i.e., newlink.URL and newlink.text) found on a subject Web page.
- the foregoing first IF statement is asked at decision junction 34 to determine whether newlink.URL for this internal link already exists in table 16. If so, then step 36 finds the corresponding table entry and step 38 subsequently retrieves or otherwise obtains the respective text (tablelmk.text) from the table entry.
- Next decision junction 52 asks the second IF statement in the above algorithm to determine whether the subject newlink.text is contained in the table entry text tablelmk.text. If so, then the process 58 ends 56. Otherwise the process 58 appends (step 54) newlink.text to tablelmk.text and ends 56.
- step 50 adds the subject internal link (i.e., newlink.URL and newlink.text) to table 16. This corresponds to the ELSE statement of the foregoing algorithm for updating table 16, and process 58 ends at 56 in Fig. 3.
- a special case of collecting links from a Web page is when the page contains script code, hi those cases, it is not straightforward to extract the links from the script.
- One approach would be to create and include in the Crawler 11 parsers for every possible script language. However, this would require a substantial development and maintenance effort, since there are many Web scripting languages, some of them quite complex.
- a simpler approach though that this invention implements is to extract from the script anything that looks like a URL, without the need to understand or parse "correctly" the script.
- the steps that are used in this approach are the following: a) Extract from the script all tokens that are enclosed in quotes (single or double quotes) b) Discard tokens that contain any whitespace characters (i.e.
- menu.addltem new MenuItem("Phone Orders", “how_to_buy /phone_orders.asp")); menu.addltem(new MenuItem("Retail Stores", “how_to_buy/retailers. html”));
- step (a) produces the following tokens: " ⁇ center>Orders ⁇ /center>" "Online Orders"
- Step (b) reduces these tokens to the following: " ⁇ center>Orders ⁇ /center>” lilt
- step (c) concludes to the following tokens: "ho w_to_buy/online_orders . asp " "how_to_buy/phone_order s . asp “ “how_to_buy/retailers.html”
- the number of Web pages that a Web site may contain varies dramatically. It can be anywhere from only one home page with some contact information, to hundreds or thousands of pages generated dynamically according to user interaction with the site. For example a larger retailer site may generate pages dynamically from its database of products that it carries. It is not efficient and sometimes not feasible for the Crawler 11 to visit every page of every site it crawls, therefore a "pruning" technique is implemented which prunes out links that are deemed to be useless.
- the term "pruning" is used because the structure of a Web site looks like an inverted tree: the root is the home page, which leads to other pages in the first level (branches), each one leading to more pages (more branches out of each branch), etc. If a branch is considered “useless”, it is "pruned” along with its "children” or branches that emanate from it. h other words the Crawler 11 does not visit the page or the links that exist on that Web page.
- the pruning is preferably implemented as one of the following two opposite strategies: a) the Crawler 11 decides which links to ignore and follows the rest; b) the Crawler 11 selects which links to follow and ignores the rest.
- bookmark links that lead to a section of the current page
- One of the most significant tasks for the Crawler 11 is to identify the type of every interesting page it finds as in step 28 of Fig. 2.
- the Crawler 11 classifies the pages into one of the following categories: Organization Sites Management team pages (info about the management team)
- the Crawler 11 uses several techniques.
- the first technique is to examine the text in the referring link that points to the current page.
- a list of keywords is used to identify a potential page type (e.g. if the referring text contains the word "contact” then the page is probably a contact info page; if it contains the word "jobs” then it is probably a page with job opportunities; etc.)
- the second technique is to examine the title of the page, if there is any. Again, a list of keywords is used to identify a potential page type.
- the third technique is to examine directly the contents of the pages.
- the Crawler 11 maintains several lists of keywords, each list pertaining to one page type.
- the Crawlerl 1 scans the page contents searching for matches from the keyword lists; the list that yields the most matches indicates a potential page type.
- keyword lists is the simplest way to examine the page contents; more sophisticated techniques may also be used, for example, Neural Networks pattern matching, or Bayesian classification (for example, see invention 3 as disclosed in the related Provisional Application No. 60/221,750 filed on July 31, 2000 for a "Computer Database Method and Apparatus").
- the outcome is one or more candidate page types.
- the Crawler 11 has a list of potential content (Web page) types, each one possibly associated with a confidence level score.
- the Crawler 11 at this point may use other "site-level” information to adjust this score; for example, if one of the potential content/page types was identified as "Job opportunities" but the Crawler 11 had already found another "Job opportunities" page in the same site with highest confidence level score, then it may reduce the confidence level for this choice.
- the Crawler 11 selects and assigns to the page the type(s) with the highest confidence level score. Correctly identifying the Web site type is important in achieving efficiency while maintaining a high level of coverage, namely, not missing important pages, and accuracy, identifying correct information about people. Different types of sites require different frequency of crawling. For example, a corporation Web site is unlikely to change daily, therefore it is sufficient to re-crawl it every two of three months without considerable risk of losing information, saving on crawling and computing time. On the other hand, a daily newspaper site completely changes its Web page content every day and thus it is important to crawl that site daily.
- Web site types also require different crawling and extraction strategies. For example a Web site that belongs to a corporation is likely to yield information about people in certain sections, such as: management team, testimonials, press releases, etc. whereas this information is unlikely to appear in other parts, such as: products, services, technical help, etc. This knowledge can dramatically cut down on crawling time by pruning these links, which in many cases are actually the most voluminous portions of the site, containing the major bulk of Web pages and information.
- Certain types of Web sites include information about two very distinct groups of people, those who work for the organization (the news site, the association or the organization) and those who are mentioned in the site, such as people mentioned or quoted in the news produced by the site or a list of members of the association.
- the Crawler 11 has to identify which portion of the site it is looking at so as to properly direct any data extraction tools about what to expect, namely a list of people who work for the organization or an eclectic and "random" sample of people. This knowledge also increases the efficiency of crawling since the news portion of the news site has to be crawled daily while the staff portion of the site can be visited every two or three months.
- the domain itself reveals the site type, i.e. domains ending with ".edu” belong to educational sites (universities, colleges, etc), whereas domains ending with ".mil” belong to military (government) sites.
- the content owner name as identified by the Crawler can be used, e.g. if the name ends with "Hospital” then it's likely a hospital site, if the name ends with "Church” then it's likely a church site, etc.
- This map contains a table of links that are found in the site (at least in the first level), the page type that every link leads to, and some additional information about every page, e.g. how many links it contains, what percentage is the off-site links, etc.
- the system works with a number of components arranged in a "pipeline” fashion. This means that output from one component flows as input to another component.
- the Crawler 11 is one of the first components in this pipeline; part of its output (i.e. the Web pages it identifies as interesting and some associated information for each page) goes directly to the data extraction tools.
- the Crawler 11 crawls completely a site, and when it finishes it passes the results to the Data Extractor which starts extracting data from the cached pages.
- the Crawler 11 may be stuck indefinitely in a site which is composed of dynamically generated pages, but which contain no useful information).
- a site may be experiencing temporary Web server problems, resulting in extremely long delays for the Crawler 11.
- each Crawler 11 there are two independent "time-out" mechanisms built into each Crawler.
- the first is a time-out associated with loading a single page (such as at 22 in Fig. 2). If a page cannot be loaded in, say, 30 seconds, then the Crawler 11 moves to another page and logs a "page time-out" event in its log for the failed page. If too many page time-out events happen for a particular site, then the Crawler 11 quits crawling the site and makes a "Retry later" note in the database 14. hi this way it is avoided crawling sites that are temporarily unavailable or experience Internet connection problems.
- the second time-out mechanism in the Crawler 11 refers to the time that it takes to crawl the whole site. If the Crawler 11 is spending too long crawling a particular site (say, more than one hour) then this is an indication that either the site is unusually large, or that the Crawler 11 is visiting some kind of dynamically created pages which usually do not contain any useful information for our system. If a "site time-out" event occurs (step 25 of Fig. 2), then the Crawler 11 interrupts crawling and it sends its output directly to Data Extractor, which tries to extract useful data. The data extraction tools report statistical results back to Crawler 11 (e.g. the amount of useful information they find) and then the Crawler 11 decides if it's worth to continue crawling the site or not. If not, then it moves to another site. If yes, then it resumes crawling the site (possibly from a different point than the one it had stopped, depending on what pages the data extractor deemed as rich in information content).
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2001278938A AU2001278938A1 (en) | 2000-07-31 | 2001-07-17 | Computer system for collecting information from web sites |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US22175000P | 2000-07-31 | 2000-07-31 | |
US60/221,750 | 2000-07-31 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2002010982A2 true WO2002010982A2 (en) | 2002-02-07 |
WO2002010982A3 WO2002010982A3 (en) | 2003-07-10 |
Family
ID=22829204
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2001/022381 WO2002010955A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining content owner of a website |
PCT/US2001/022385 WO2002010956A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining site type of a web site |
PCT/US2001/022430 WO2002010957A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining content types of web pages |
PCT/US2001/022426 WO2002010982A2 (en) | 2000-07-31 | 2001-07-17 | Computer system for collecting information from web sites |
PCT/US2001/023343 WO2002010960A2 (en) | 2000-07-31 | 2001-07-25 | Computer method and apparatus for extracting data from web pages |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2001/022381 WO2002010955A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining content owner of a website |
PCT/US2001/022385 WO2002010956A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining site type of a web site |
PCT/US2001/022430 WO2002010957A2 (en) | 2000-07-31 | 2001-07-17 | Computer method and apparatus for determining content types of web pages |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2001/023343 WO2002010960A2 (en) | 2000-07-31 | 2001-07-25 | Computer method and apparatus for extracting data from web pages |
Country Status (3)
Country | Link |
---|---|
US (7) | US6618717B1 (en) |
AU (5) | AU2001273522A1 (en) |
WO (5) | WO2002010955A2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008079048A1 (en) * | 2006-12-26 | 2008-07-03 | Pavel Mikhaylovich Malyshev | Computerized method for converting the sequence of conforming computer codes requested by an information user and a system for carrying out said method |
US8954416B2 (en) | 2004-11-22 | 2015-02-10 | Facebook, Inc. | Method and apparatus for an application crawler |
US9053179B2 (en) | 2006-04-05 | 2015-06-09 | Lexisnexis, A Division Of Reed Elsevier Inc. | Citation network viewer and method |
US9405833B2 (en) | 2004-11-22 | 2016-08-02 | Facebook, Inc. | Methods for analyzing dynamic web pages |
EP3214557A4 (en) * | 2014-10-30 | 2017-09-06 | Alibaba Group Holding Limited | Web page deduplication method and apparatus |
RU2683157C2 (en) * | 2016-12-27 | 2019-03-26 | Федеральное Государственное Бюджетное Научное Учреждение "Всероссийский Научно-Исследовательский Институт Картофельного Хозяйства Имени А.Г. Лорха" (Фгбну Вниикх) | Method of searching information |
Families Citing this family (474)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8352400B2 (en) | 1991-12-23 | 2013-01-08 | Hoffberg Steven M | Adaptive pattern recognition based controller apparatus and method and human-factored interface therefore |
US7168084B1 (en) | 1992-12-09 | 2007-01-23 | Sedna Patent Services, Llc | Method and apparatus for targeting virtual objects |
US9286294B2 (en) | 1992-12-09 | 2016-03-15 | Comcast Ip Holdings I, Llc | Video and digital multimedia aggregator content suggestion engine |
JP3841233B2 (en) * | 1996-12-18 | 2006-11-01 | ソニー株式会社 | Information processing apparatus and information processing method |
US7904187B2 (en) | 1999-02-01 | 2011-03-08 | Hoffberg Steven M | Internet appliance system and method |
US9843447B1 (en) | 1999-09-09 | 2017-12-12 | Secure Axcess Llc | Authenticating electronic content |
WO2001018636A1 (en) * | 1999-09-09 | 2001-03-15 | American Express Travel Related Services Company, Inc. | System and method for authenticating a web page |
US7203838B1 (en) | 1999-09-09 | 2007-04-10 | American Express Travel Related Services Company, Inc. | System and method for authenticating a web page |
KR100357098B1 (en) * | 1999-11-12 | 2002-10-19 | 엘지전자 주식회사 | apparatus and method for display of data information in data broadcasting reciever |
KR100856149B1 (en) * | 1999-11-26 | 2008-09-03 | 네테카 인코포레이티드 | Electronic mail server |
US20010049707A1 (en) * | 2000-02-29 | 2001-12-06 | Tran Bao Q. | Systems and methods for generating intellectual property |
JP2004501421A (en) * | 2000-03-27 | 2004-01-15 | ドキュメンタム,インコーポレイティド | Method and apparatus for generating metadata for documents |
US6666377B1 (en) | 2000-07-18 | 2003-12-23 | Scott C. Harris | Bar code data entry device |
US20070027672A1 (en) * | 2000-07-31 | 2007-02-01 | Michel Decary | Computer method and apparatus for extracting data from web pages |
US6618717B1 (en) * | 2000-07-31 | 2003-09-09 | Eliyon Technologies Corporation | Computer method and apparatus for determining content owner of a website |
JP2002171232A (en) * | 2000-08-01 | 2002-06-14 | Matsushita Electric Ind Co Ltd | Transmitting and receiving system and transmitter/ receiver |
US6957224B1 (en) * | 2000-09-11 | 2005-10-18 | International Business Machines Corporation | Efficient retrieval of uniform resource locators |
AU2002243448A1 (en) | 2000-10-24 | 2002-06-24 | Singingfish.Com, Inc. | Method of sizing an embedded media player page |
US8122236B2 (en) | 2001-10-24 | 2012-02-21 | Aol Inc. | Method of disseminating advertisements using an embedded media player page |
FR2816157A1 (en) * | 2000-10-31 | 2002-05-03 | Thomson Multimedia Sa | PROCESS FOR PROCESSING DISTRIBUTED VIDEO DATA TO BE VIEWED ON SCREEN AND DEVICE IMPLEMENTING THE METHOD |
US8060816B1 (en) * | 2000-10-31 | 2011-11-15 | International Business Machines Corporation | Methods and apparatus for intelligent crawling on the world wide web |
EP2261599B1 (en) | 2000-11-08 | 2013-01-02 | Institut Straumann Ag | (Dental) Surface mapping and generation |
US7925967B2 (en) * | 2000-11-21 | 2011-04-12 | Aol Inc. | Metadata quality improvement |
US20040030683A1 (en) * | 2000-11-21 | 2004-02-12 | Evans Philip Clark | System and process for mediated crawling |
US7043473B1 (en) * | 2000-11-22 | 2006-05-09 | Widevine Technologies, Inc. | Media tracking system and method |
US8230323B2 (en) * | 2000-12-06 | 2012-07-24 | Sra International, Inc. | Content distribution system and method |
US20020123996A1 (en) * | 2001-02-06 | 2002-09-05 | O'brien Christopher | Data mining system, method and apparatus for industrial applications |
US6662190B2 (en) * | 2001-03-20 | 2003-12-09 | Ispheres Corporation | Learning automatic data extraction system |
WO2002093334A2 (en) * | 2001-04-06 | 2002-11-21 | Symantec Corporation | Temporal access control for computer virus outbreaks |
US7197506B2 (en) * | 2001-04-06 | 2007-03-27 | Renar Company, Llc | Collection management system |
US8005870B1 (en) * | 2001-06-19 | 2011-08-23 | Microstrategy Incorporated | System and method for syntax abstraction in query language generation |
US20020198859A1 (en) * | 2001-06-22 | 2002-12-26 | International Business Machines Corporation | Method and system for providing web links |
US7793326B2 (en) | 2001-08-03 | 2010-09-07 | Comcast Ip Holdings I, Llc | Video and digital multimedia aggregator |
CN1167027C (en) * | 2001-08-03 | 2004-09-15 | 富士通株式会社 | Format file information extracting device and method |
US8285701B2 (en) * | 2001-08-03 | 2012-10-09 | Comcast Ip Holdings I, Llc | Video and digital multimedia aggregator remote content crawler |
US7908628B2 (en) | 2001-08-03 | 2011-03-15 | Comcast Ip Holdings I, Llc | Video and digital multimedia aggregator content coding and formatting |
US20030028890A1 (en) * | 2001-08-03 | 2003-02-06 | Swart William D. | Video and digital multimedia acquisition and delivery system and method |
US8249885B2 (en) * | 2001-08-08 | 2012-08-21 | Gary Charles Berkowitz | Knowledge-based e-catalog procurement system and method |
US20030061232A1 (en) * | 2001-09-21 | 2003-03-27 | Dun & Bradstreet Inc. | Method and system for processing business data |
US7788111B2 (en) * | 2001-10-22 | 2010-08-31 | Siemens Medical Solutions Usa, Inc. | System for providing healthcare related information |
US20030078807A1 (en) * | 2001-10-22 | 2003-04-24 | Siemens Medical Solutions Health Services Corporation | System for maintaining organization related information for use in supporting organization operation |
US7051012B2 (en) * | 2001-10-22 | 2006-05-23 | Siemens Medical Solutions Health Services Corporation | User interface system for maintaining organization related information for use in supporting organization operation |
US7437302B2 (en) * | 2001-10-22 | 2008-10-14 | Siemens Medical Solutions Usa, Inc. | System for managing healthcare related information supporting operation of a healthcare enterprise |
US20040064500A1 (en) * | 2001-11-20 | 2004-04-01 | Kolar Jennifer Lynn | System and method for unified extraction of media objects |
US7194464B2 (en) | 2001-12-07 | 2007-03-20 | Websense, Inc. | System and method for adapting an internet filter |
US7333966B2 (en) | 2001-12-21 | 2008-02-19 | Thomson Global Resources | Systems, methods, and software for hyperlinking names |
EP3401794A1 (en) | 2002-01-08 | 2018-11-14 | Seven Networks, LLC | Connection architecture for a mobile network |
US7284195B2 (en) * | 2002-01-31 | 2007-10-16 | International Business Machines Corporation | Structure and method for linking within a website |
US7228335B2 (en) * | 2002-02-19 | 2007-06-05 | Goodcontacts Research Ltd. | Method of automatically populating contact information fields for a new contract added to an electronic contact database |
US6856679B2 (en) * | 2002-05-01 | 2005-02-15 | Sbc Services Inc. | System and method to provide automated scripting for customer service representatives |
US20040205484A1 (en) * | 2002-05-01 | 2004-10-14 | Pennington Stanford E. | System and method for dynamically generating customized pages |
US6993534B2 (en) * | 2002-05-08 | 2006-01-31 | International Business Machines Corporation | Data store for knowledge-based data mining system |
US7010526B2 (en) * | 2002-05-08 | 2006-03-07 | International Business Machines Corporation | Knowledge-based data mining system |
US8214391B2 (en) * | 2002-05-08 | 2012-07-03 | International Business Machines Corporation | Knowledge-based data mining system |
US7395329B1 (en) * | 2002-05-13 | 2008-07-01 | At&T Delaware Intellectual Property., Inc. | Real-time notification of presence availability changes |
US7353455B2 (en) * | 2002-05-21 | 2008-04-01 | At&T Delaware Intellectual Property, Inc. | Caller initiated distinctive presence alerting and auto-response messaging |
US7367056B1 (en) | 2002-06-04 | 2008-04-29 | Symantec Corporation | Countering malicious code infections to computer files that have been infected more than once |
US7165068B2 (en) * | 2002-06-12 | 2007-01-16 | Zycus Infotech Pvt Ltd. | System and method for electronic catalog classification using a hybrid of rule based and statistical method |
US20060190561A1 (en) * | 2002-06-19 | 2006-08-24 | Watchfire Corporation | Method and system for obtaining script related information for website crawling |
US7496636B2 (en) * | 2002-06-19 | 2009-02-24 | International Business Machines Corporation | Method and system for resolving Universal Resource Locators (URLs) from script code |
US7228496B2 (en) * | 2002-07-09 | 2007-06-05 | Kabushiki Kaisha Toshiba | Document editing method, document editing system, server apparatus, and document editing program |
US8924484B2 (en) | 2002-07-16 | 2014-12-30 | Sonicwall, Inc. | Active e-mail filter with challenge-response |
US8396926B1 (en) | 2002-07-16 | 2013-03-12 | Sonicwall, Inc. | Message challenge response |
US7539726B1 (en) | 2002-07-16 | 2009-05-26 | Sonicwall, Inc. | Message testing |
US7478121B1 (en) * | 2002-07-31 | 2009-01-13 | Opinionlab, Inc. | Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website |
US7370285B1 (en) * | 2002-07-31 | 2008-05-06 | Opinionlab, Inc. | Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website |
US7370278B2 (en) * | 2002-08-19 | 2008-05-06 | At&T Delaware Intellectual Property, Inc. | Redirection of user-initiated distinctive presence alert messages |
US20040044734A1 (en) * | 2002-08-27 | 2004-03-04 | Mark Beck | Enhanced services electronic mail |
US7365221B2 (en) * | 2002-09-26 | 2008-04-29 | Panacos Pharmaceuticals, Inc. | Monoacylated betulin and dihydrobetulin derivatives, preparation thereof and use thereof |
US7254573B2 (en) * | 2002-10-02 | 2007-08-07 | Burke Thomas R | System and method for identifying alternate contact information in a database related to entity, query by identifying contact information of a different type than was in query which is related to the same entity |
US7337471B2 (en) * | 2002-10-07 | 2008-02-26 | Symantec Corporation | Selective detection of malicious computer code |
US7469419B2 (en) | 2002-10-07 | 2008-12-23 | Symantec Corporation | Detection of malicious computer code |
US7260847B2 (en) * | 2002-10-24 | 2007-08-21 | Symantec Corporation | Antivirus scanning in a hard-linked environment |
US7249187B2 (en) | 2002-11-27 | 2007-07-24 | Symantec Corporation | Enforcement of compliance with network security policies |
US20050010556A1 (en) * | 2002-11-27 | 2005-01-13 | Kathleen Phelan | Method and apparatus for information retrieval |
CA2508791A1 (en) * | 2002-12-06 | 2004-06-24 | Attensity Corporation | Systems and methods for providing a mixed data integration service |
US7373664B2 (en) * | 2002-12-16 | 2008-05-13 | Symantec Corporation | Proactive protection against e-mail worms and spam |
US8468126B2 (en) | 2005-08-01 | 2013-06-18 | Seven Networks, Inc. | Publishing data in an information community |
US7917468B2 (en) | 2005-08-01 | 2011-03-29 | Seven Networks, Inc. | Linking of personal information management data |
US7779247B2 (en) | 2003-01-09 | 2010-08-17 | Jericho Systems Corporation | Method and system for dynamically implementing an enterprise resource policy |
US7293290B2 (en) | 2003-02-06 | 2007-11-06 | Symantec Corporation | Dynamic detection of computer worms |
US20040158546A1 (en) * | 2003-02-06 | 2004-08-12 | Sobel William E. | Integrity checking for software downloaded from untrusted sources |
US7246227B2 (en) * | 2003-02-10 | 2007-07-17 | Symantec Corporation | Efficient scanning of stream based data |
US20040164961A1 (en) * | 2003-02-21 | 2004-08-26 | Debasis Bal | Method, system and computer product for continuously monitoring data sources for an event of interest |
US7203959B2 (en) | 2003-03-14 | 2007-04-10 | Symantec Corporation | Stream scanning through network proxy servers |
US7546638B2 (en) | 2003-03-18 | 2009-06-09 | Symantec Corporation | Automated identification and clean-up of malicious computer code |
US20040237037A1 (en) * | 2003-03-21 | 2004-11-25 | Xerox Corporation | Determination of member pages for a hyperlinked document with recursive page-level link analysis |
US20050188300A1 (en) * | 2003-03-21 | 2005-08-25 | Xerox Corporation | Determination of member pages for a hyperlinked document with link and document analysis |
US7305612B2 (en) * | 2003-03-31 | 2007-12-04 | Siemens Corporate Research, Inc. | Systems and methods for automatic form segmentation for raster-based passive electronic documents |
JP2004303160A (en) * | 2003-04-01 | 2004-10-28 | Oki Electric Ind Co Ltd | Information extracting device |
US7680886B1 (en) | 2003-04-09 | 2010-03-16 | Symantec Corporation | Suppressing spam using a machine learning based spam filter |
US20040215610A1 (en) * | 2003-04-22 | 2004-10-28 | Lawson Software, Inc. | System and method for extracting and applying business organization information |
US7650382B1 (en) | 2003-04-24 | 2010-01-19 | Symantec Corporation | Detecting spam e-mail with backup e-mail server traps |
US7739494B1 (en) | 2003-04-25 | 2010-06-15 | Symantec Corporation | SSL validation and stripping using trustworthiness factors |
US7640590B1 (en) | 2004-12-21 | 2009-12-29 | Symantec Corporation | Presentation of network source and executable characteristics |
US7366919B1 (en) | 2003-04-25 | 2008-04-29 | Symantec Corporation | Use of geo-location data for spam detection |
US7600001B1 (en) * | 2003-05-01 | 2009-10-06 | Vignette Corporation | Method and computer system for unstructured data integration through a graphical interface |
US7558726B2 (en) * | 2003-05-16 | 2009-07-07 | Sap Ag | Multi-language support for data mining models |
WO2004107219A1 (en) * | 2003-05-29 | 2004-12-09 | Locateplus Holdings Corporation | Current mailing address identification and verification |
US7293063B1 (en) | 2003-06-04 | 2007-11-06 | Symantec Corporation | System utilizing updated spam signatures for performing secondary signature-based analysis of a held e-mail to improve spam email detection |
US7827487B1 (en) | 2003-06-16 | 2010-11-02 | Opinionlab, Inc. | Soliciting user feedback regarding one or more web pages of a website without obscuring visual content |
US7792828B2 (en) * | 2003-06-25 | 2010-09-07 | Jericho Systems Corporation | Method and system for selecting content items to be presented to a viewer |
US8707312B1 (en) | 2003-07-03 | 2014-04-22 | Google Inc. | Document reuse in a search engine crawler |
US7725452B1 (en) * | 2003-07-03 | 2010-05-25 | Google Inc. | Scheduler for search engine crawler |
US20050027566A1 (en) * | 2003-07-09 | 2005-02-03 | Haskell Robert Emmons | Terminology management system |
US7543016B2 (en) | 2003-07-31 | 2009-06-02 | International Business Machines Corporation | Method, system and program product for automatically assigning electronic addresses to users |
US20050033633A1 (en) * | 2003-08-04 | 2005-02-10 | Lapasta Douglas G. | System and method for evaluating job candidates |
US7739278B1 (en) | 2003-08-22 | 2010-06-15 | Symantec Corporation | Source independent file attribute tracking |
JP4174392B2 (en) * | 2003-08-28 | 2008-10-29 | 日本電気株式会社 | Network unauthorized connection prevention system and network unauthorized connection prevention device |
US20050060140A1 (en) * | 2003-09-15 | 2005-03-17 | Maddox Paul Christopher | Using semantic feature structures for document comparisons |
US20050076013A1 (en) * | 2003-10-01 | 2005-04-07 | Fuji Xerox Co., Ltd. | Context-based contact information retrieval systems and methods |
US7921159B1 (en) | 2003-10-14 | 2011-04-05 | Symantec Corporation | Countering spam that uses disguised characters |
US8726145B2 (en) * | 2003-11-18 | 2014-05-13 | Gh Llc | Content communication system and methods |
US20050138129A1 (en) * | 2003-12-23 | 2005-06-23 | Maria Adamczyk | Methods and systems of responsive messaging |
US20050149527A1 (en) * | 2003-12-31 | 2005-07-07 | Intellipoint International, Llc | System and method for uniquely identifying persons |
US20070130018A1 (en) * | 2004-01-05 | 2007-06-07 | Yasuo Nishizawa | Integrated intelligent seo transaction platform |
US20050160014A1 (en) * | 2004-01-15 | 2005-07-21 | Cairo Inc. | Techniques for identifying and comparing local retail prices |
US20050166137A1 (en) * | 2004-01-26 | 2005-07-28 | Bao Tran | Systems and methods for analyzing documents |
US9848086B2 (en) * | 2004-02-23 | 2017-12-19 | Nokia Technologies Oy | Methods, apparatus and computer program products for dispatching and prioritizing communication of generic-recipient messages to recipients |
US7761923B2 (en) * | 2004-03-01 | 2010-07-20 | Invensys Systems, Inc. | Process control methods and apparatus for intrusion detection, protection and network hardening |
US20050210008A1 (en) * | 2004-03-18 | 2005-09-22 | Bao Tran | Systems and methods for analyzing documents over a network |
US7418458B2 (en) * | 2004-04-06 | 2008-08-26 | Educational Testing Service | Method for estimating examinee attribute parameters in a cognitive diagnosis model |
US7130981B1 (en) | 2004-04-06 | 2006-10-31 | Symantec Corporation | Signature driven cache extension for stream based scanning |
US7519954B1 (en) * | 2004-04-08 | 2009-04-14 | Mcafee, Inc. | System and method of operating system identification |
CA2504118A1 (en) * | 2004-04-09 | 2005-10-09 | Opinionlab, Inc. | Using software incorporated into a web page to collect page-specific user feedback concerning a document embedded in the web page |
US20080140626A1 (en) * | 2004-04-15 | 2008-06-12 | Jeffery Wilson | Method for enabling dynamic websites to be indexed within search engines |
US7783476B2 (en) * | 2004-05-05 | 2010-08-24 | Microsoft Corporation | Word extraction method and system for use in word-breaking using statistical information |
US7861304B1 (en) | 2004-05-07 | 2010-12-28 | Symantec Corporation | Pattern matching using embedded functions |
US7484094B1 (en) | 2004-05-14 | 2009-01-27 | Symantec Corporation | Opening computer files quickly and safely over a network |
US7373667B1 (en) | 2004-05-14 | 2008-05-13 | Symantec Corporation | Protecting a computer coupled to a network from malicious code infections |
US8181112B2 (en) * | 2004-05-21 | 2012-05-15 | Oracle International Corporation | Independent portlet rendering |
US8719142B1 (en) | 2004-06-16 | 2014-05-06 | Gary Odom | Seller categorization |
JP4583218B2 (en) * | 2004-07-05 | 2010-11-17 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Method, computer program, and system for evaluating target content |
US7409393B2 (en) * | 2004-07-28 | 2008-08-05 | Mybizintel Inc. | Data gathering and distribution system |
US7996462B2 (en) * | 2004-07-30 | 2011-08-09 | Sap Ag | Collaborative agent for a work environment |
JP2006053745A (en) * | 2004-08-11 | 2006-02-23 | Saora Inc | Data processing method, device and program |
JP4350001B2 (en) * | 2004-08-17 | 2009-10-21 | 富士通株式会社 | Page information collection program, page information collection method, and page information collection apparatus |
US7987172B1 (en) | 2004-08-30 | 2011-07-26 | Google Inc. | Minimizing visibility of stale content in web searching including revising web crawl intervals of documents |
US8244726B1 (en) | 2004-08-31 | 2012-08-14 | Bruce Matesso | Computer-aided extraction of semantics from keywords to confirm match of buyer offers to seller bids |
US7991787B2 (en) * | 2004-08-31 | 2011-08-02 | Sap Ag | Applying search engine technology to HCM employee searches |
US20060047691A1 (en) * | 2004-08-31 | 2006-03-02 | Microsoft Corporation | Creating a document index from a flex- and Yacc-generated named entity recognizer |
US20060047690A1 (en) * | 2004-08-31 | 2006-03-02 | Microsoft Corporation | Integration of Flex and Yacc into a linguistic services platform for named entity recognition |
US7596555B2 (en) * | 2004-08-31 | 2009-09-29 | Sap Ag | Fuzzy recipient and contact search for email workflow and groupware applications |
US20060047500A1 (en) * | 2004-08-31 | 2006-03-02 | Microsoft Corporation | Named entity recognition using compiler methods |
US7509680B1 (en) | 2004-09-01 | 2009-03-24 | Symantec Corporation | Detecting computer worms as they arrive at local computers through open network shares |
US7490244B1 (en) | 2004-09-14 | 2009-02-10 | Symantec Corporation | Blocking e-mail propagation of suspected malicious computer code |
US7555524B1 (en) | 2004-09-16 | 2009-06-30 | Symantec Corporation | Bulk electronic message detection by header similarity analysis |
US8090776B2 (en) * | 2004-11-01 | 2012-01-03 | Microsoft Corporation | Dynamic content change notification |
US7546349B1 (en) | 2004-11-01 | 2009-06-09 | Symantec Corporation | Automatic generation of disposable e-mail addresses |
US7620996B2 (en) * | 2004-11-01 | 2009-11-17 | Microsoft Corporation | Dynamic summary module |
US7565686B1 (en) | 2004-11-08 | 2009-07-21 | Symantec Corporation | Preventing unauthorized loading of late binding code into a process |
US8126890B2 (en) * | 2004-12-21 | 2012-02-28 | Make Sence, Inc. | Techniques for knowledge discovery by constructing knowledge correlations using concepts or terms |
US9330175B2 (en) | 2004-11-12 | 2016-05-03 | Make Sence, Inc. | Techniques for knowledge discovery by constructing knowledge correlations using concepts or terms |
WO2006053306A2 (en) | 2004-11-12 | 2006-05-18 | Make Sence, Inc | Knowledge discovery by constructing correlations using concepts or terms |
WO2006055983A2 (en) * | 2004-11-22 | 2006-05-26 | Truveo, Inc. | Method and apparatus for a ranking engine |
US20060123478A1 (en) * | 2004-12-02 | 2006-06-08 | Microsoft Corporation | Phishing detection, prevention, and notification |
US7634810B2 (en) * | 2004-12-02 | 2009-12-15 | Microsoft Corporation | Phishing detection, prevention, and notification |
EP1669896A3 (en) * | 2004-12-03 | 2007-03-28 | Panscient Pty Ltd. | A machine learning system for extracting structured records from web pages and other text sources |
US7428491B2 (en) * | 2004-12-10 | 2008-09-23 | Microsoft Corporation | Method and system for obtaining personal aliases through voice recognition |
US20060168046A1 (en) * | 2005-01-11 | 2006-07-27 | Microsoft Corporaion | Managing periodic electronic messages |
WO2006081307A2 (en) * | 2005-01-25 | 2006-08-03 | Aureon Laboratories, Inc. | Methods and systems for induction and use of probabilistic patterns to support decisions under uncertainty |
US20060212305A1 (en) * | 2005-03-18 | 2006-09-21 | Jobster, Inc. | Method and apparatus for ranking candidates using connection information provided by candidates |
US20060212448A1 (en) * | 2005-03-18 | 2006-09-21 | Bogle Phillip L | Method and apparatus for ranking candidates |
US20060229184A1 (en) * | 2005-04-07 | 2006-10-12 | Hewlett-Packard Development Company, L.P. | Creaser |
DE102005016815A1 (en) * | 2005-04-07 | 2006-10-12 | Deutsche Telekom Ag | Method of operation, in particular for creating a database |
US8438633B1 (en) | 2005-04-21 | 2013-05-07 | Seven Networks, Inc. | Flexible real-time inbox access |
US8666964B1 (en) | 2005-04-25 | 2014-03-04 | Google Inc. | Managing items in crawl schedule |
US8386459B1 (en) | 2005-04-25 | 2013-02-26 | Google Inc. | Scheduling a recrawl |
US8630996B2 (en) * | 2005-05-05 | 2014-01-14 | At&T Intellectual Property I, L.P. | Identifying duplicate entries in a historical database |
US20060265368A1 (en) * | 2005-05-23 | 2006-11-23 | Opinionlab, Inc. | Measuring subjective user reaction concerning a particular document |
JP4772378B2 (en) * | 2005-05-26 | 2011-09-14 | 株式会社東芝 | Method and apparatus for generating time-series data from a web page |
US7801881B1 (en) * | 2005-05-31 | 2010-09-21 | Google Inc. | Sitemap generating client for web crawler |
US7769742B1 (en) | 2005-05-31 | 2010-08-03 | Google Inc. | Web crawler scheduler that utilizes sitemaps from websites |
US7725476B2 (en) | 2005-06-14 | 2010-05-25 | International Business Machines Corporation | System and method for automated data retrieval based on data placed in clipboard memory |
US8805781B2 (en) * | 2005-06-15 | 2014-08-12 | Geronimo Development | Document quotation indexing system and method |
US8768911B2 (en) | 2005-06-15 | 2014-07-01 | Geronimo Development | System and method for indexing and displaying document text that has been subsequently quoted |
US20060287765A1 (en) * | 2005-06-20 | 2006-12-21 | Kraft Harold H | Privacy Information Reporting Systems with Broad Search Scope and Integration |
WO2006136660A1 (en) | 2005-06-21 | 2006-12-28 | Seven Networks International Oy | Maintaining an ip connection in a mobile network |
GB0512744D0 (en) * | 2005-06-22 | 2005-07-27 | Blackspider Technologies | Method and system for filtering electronic messages |
US7509315B1 (en) | 2005-06-24 | 2009-03-24 | Google Inc. | Managing URLs |
US8898134B2 (en) | 2005-06-27 | 2014-11-25 | Make Sence, Inc. | Method for ranking resources using node pool |
US8140559B2 (en) * | 2005-06-27 | 2012-03-20 | Make Sence, Inc. | Knowledge correlation search engine |
US7895654B1 (en) | 2005-06-27 | 2011-02-22 | Symantec Corporation | Efficient file scanning using secure listing of file modification times |
US7975303B1 (en) | 2005-06-27 | 2011-07-05 | Symantec Corporation | Efficient file scanning using input-output hints |
US7652112B2 (en) * | 2005-07-06 | 2010-01-26 | E.I. Du Pont De Nemours And Company | Polymeric extenders for surface effects |
US7669119B1 (en) * | 2005-07-20 | 2010-02-23 | Alexa Internet | Correlation-based information extraction from markup language documents |
US8069166B2 (en) * | 2005-08-01 | 2011-11-29 | Seven Networks, Inc. | Managing user-to-user contact with inferred presence information |
US7565358B2 (en) * | 2005-08-08 | 2009-07-21 | Google Inc. | Agent rank |
US7653617B2 (en) * | 2005-08-29 | 2010-01-26 | Google Inc. | Mobile sitemaps |
WO2007029348A1 (en) | 2005-09-06 | 2007-03-15 | Community Engine Inc. | Data extracting system, terminal apparatus, program of terminal apparatus, server apparatus, and program of server apparatus |
US7672833B2 (en) * | 2005-09-22 | 2010-03-02 | Fair Isaac Corporation | Method and apparatus for automatic entity disambiguation |
WO2007038713A2 (en) * | 2005-09-28 | 2007-04-05 | Epacris Inc. | Search engine determining results based on probabilistic scoring of relevance |
US20070078821A1 (en) * | 2005-09-30 | 2007-04-05 | Kabushiki Kaisha Toshiba | System and method for managing history of plant data |
US7849093B2 (en) * | 2005-10-14 | 2010-12-07 | Microsoft Corporation | Searches over a collection of items through classification and display of media galleries |
US8429148B1 (en) * | 2005-11-01 | 2013-04-23 | At&T Intellectual Property Ii, L.P. | Method and apparatus for automatically generating headlines based on data retrieved from a network and for answering questions related to a headline |
US7792870B2 (en) * | 2005-11-08 | 2010-09-07 | Yahoo! Inc. | Identification and automatic propagation of geo-location associations to un-located documents |
US8024653B2 (en) | 2005-11-14 | 2011-09-20 | Make Sence, Inc. | Techniques for creating computer generated notes |
US20070118607A1 (en) * | 2005-11-22 | 2007-05-24 | Niko Nelissen | Method and System for forensic investigation of internet resources |
US20070143415A1 (en) * | 2005-12-15 | 2007-06-21 | Daigle Brian K | Customizable presence icons for instant messaging |
US7949646B1 (en) | 2005-12-23 | 2011-05-24 | At&T Intellectual Property Ii, L.P. | Method and apparatus for building sales tools by mining data from websites |
US20070156653A1 (en) * | 2005-12-30 | 2007-07-05 | Manish Garg | Automated knowledge management system |
US7831382B2 (en) * | 2006-02-01 | 2010-11-09 | TeleAtlas B.V. | Method for differentiating duplicate or similarly named disjoint localities within a state or other principal geographic unit of interest |
US7769395B2 (en) | 2006-06-20 | 2010-08-03 | Seven Networks, Inc. | Location-based operations and messaging |
US7945533B2 (en) * | 2006-03-01 | 2011-05-17 | Oracle International Corp. | Index replication using crawl modification information |
US7475069B2 (en) * | 2006-03-29 | 2009-01-06 | International Business Machines Corporation | System and method for prioritizing websites during a webcrawling process |
US7860857B2 (en) * | 2006-03-30 | 2010-12-28 | Invensys Systems, Inc. | Digital data processing apparatus and methods for improving plant performance |
US9390422B2 (en) * | 2006-03-30 | 2016-07-12 | Geographic Solutions, Inc. | System, method and computer program products for creating and maintaining a consolidated jobs database |
US11062267B1 (en) | 2006-03-30 | 2021-07-13 | Geographic Solutions, Inc. | Automated reactive talent matching |
US20070255675A1 (en) * | 2006-04-26 | 2007-11-01 | Jacquelyn Fuzell-Casey | Auto-updating, web-accessible database to facilitate networking and resource management |
US7603350B1 (en) | 2006-05-09 | 2009-10-13 | Google Inc. | Search result ranking based on trust |
US9507778B2 (en) | 2006-05-19 | 2016-11-29 | Yahoo! Inc. | Summarization of media object collections |
US20070294646A1 (en) * | 2006-06-14 | 2007-12-20 | Sybase, Inc. | System and Method for Delivering Mobile RSS Content |
US8090658B2 (en) * | 2006-06-23 | 2012-01-03 | International Business Machines Corporation | System and method of member unique names |
US8332947B1 (en) | 2006-06-27 | 2012-12-11 | Symantec Corporation | Security threat reporting in light of local security tools |
US8239915B1 (en) | 2006-06-30 | 2012-08-07 | Symantec Corporation | Endpoint management using trust rating data |
US10223671B1 (en) | 2006-06-30 | 2019-03-05 | Geographic Solutions, Inc. | System, method and computer program products for direct applying to job applications |
US8020206B2 (en) | 2006-07-10 | 2011-09-13 | Websense, Inc. | System and method of analyzing web content |
US8615800B2 (en) | 2006-07-10 | 2013-12-24 | Websense, Inc. | System and method for analyzing web content |
US9633356B2 (en) | 2006-07-20 | 2017-04-25 | Aol Inc. | Targeted advertising for playlists based upon search queries |
US8775237B2 (en) | 2006-08-02 | 2014-07-08 | Opinionlab, Inc. | System and method for measuring and reporting user reactions to advertisements on a web page |
US9547648B2 (en) * | 2006-08-03 | 2017-01-17 | Excalibur Ip, Llc | Electronic document information extraction |
US8533226B1 (en) | 2006-08-04 | 2013-09-10 | Google Inc. | System and method for verifying and revoking ownership rights with respect to a website in a website indexing system |
US7930400B1 (en) | 2006-08-04 | 2011-04-19 | Google Inc. | System and method for managing multiple domain names for a website in a website indexing system |
US8190868B2 (en) | 2006-08-07 | 2012-05-29 | Webroot Inc. | Malware management through kernel detection |
US20080040352A1 (en) * | 2006-08-08 | 2008-02-14 | Kenneth Alexander Ellis | Method for creating a disambiguation database |
US8121915B1 (en) | 2006-08-16 | 2012-02-21 | Resource Consortium Limited | Generating financial plans using a personal information aggregator |
US8930204B1 (en) | 2006-08-16 | 2015-01-06 | Resource Consortium Limited | Determining lifestyle recommendations using aggregated personal information |
US7809602B2 (en) * | 2006-08-31 | 2010-10-05 | Opinionlab, Inc. | Computer-implemented system and method for measuring and reporting business intelligence based on comments collected from web page users using software associated with accessed web pages |
GB2441598A (en) * | 2006-09-07 | 2008-03-12 | Fujin Technology Plc | Categorisation of Data using Structural Analysis |
US8099415B2 (en) * | 2006-09-08 | 2012-01-17 | Simply Hired, Inc. | Method and apparatus for assessing similarity between online job listings |
US7685201B2 (en) * | 2006-09-08 | 2010-03-23 | Microsoft Corporation | Person disambiguation using name entity extraction-based clustering |
US8271429B2 (en) | 2006-09-11 | 2012-09-18 | Wiredset Llc | System and method for collecting and processing data |
DE502006009446D1 (en) * | 2006-09-13 | 2011-06-16 | Ivoclar Vivadent Ag | Multicolored molded body |
US7561041B2 (en) * | 2006-09-13 | 2009-07-14 | At&T Intellectual Property I, L.P. | Monitoring and entry system presence service |
US8234379B2 (en) * | 2006-09-14 | 2012-07-31 | Afilias Limited | System and method for facilitating distribution of limited resources |
US20080077685A1 (en) * | 2006-09-21 | 2008-03-27 | Bellsouth Intellectual Property Corporation | Dynamically configurable presence service |
US8316117B2 (en) | 2006-09-21 | 2012-11-20 | At&T Intellectual Property I, L.P. | Personal presentity presence subsystem |
US8554638B2 (en) | 2006-09-29 | 2013-10-08 | Microsoft Corporation | Comparative shopping tool |
US7599920B1 (en) | 2006-10-12 | 2009-10-06 | Google Inc. | System and method for enabling website owners to manage crawl rate in a website indexing system |
WO2008049219A1 (en) * | 2006-10-24 | 2008-05-02 | Afilias Limited | Supply chain discovery services |
US8594702B2 (en) | 2006-11-06 | 2013-11-26 | Yahoo! Inc. | Context server for associating information based on context |
US9110903B2 (en) | 2006-11-22 | 2015-08-18 | Yahoo! Inc. | Method, system and apparatus for using user profile electronic device data in media delivery |
US8402356B2 (en) | 2006-11-22 | 2013-03-19 | Yahoo! Inc. | Methods, systems and apparatus for delivery of media |
US9654495B2 (en) | 2006-12-01 | 2017-05-16 | Websense, Llc | System and method of analyzing web addresses |
US20080133676A1 (en) * | 2006-12-01 | 2008-06-05 | John Choisser | Method and system for providing email |
US20080141110A1 (en) * | 2006-12-07 | 2008-06-12 | Picscout (Israel) Ltd. | Hot-linked images and methods and an apparatus for adapting existing images for the same |
US20080147631A1 (en) * | 2006-12-14 | 2008-06-19 | Dean Leffingwell | Method and system for collecting and retrieving information from web sites |
US20080147588A1 (en) * | 2006-12-14 | 2008-06-19 | Dean Leffingwell | Method for discovering data artifacts in an on-line data object |
US20080147641A1 (en) * | 2006-12-14 | 2008-06-19 | Dean Leffingwell | Method for prioritizing search results retrieved in response to a computerized search query |
US20080147578A1 (en) * | 2006-12-14 | 2008-06-19 | Dean Leffingwell | System for prioritizing search results retrieved in response to a computerized search query |
US20080147642A1 (en) * | 2006-12-14 | 2008-06-19 | Dean Leffingwell | System for discovering data artifacts in an on-line data object |
DE102006061143A1 (en) * | 2006-12-22 | 2008-07-24 | Aepsilon Rechteverwaltungs Gmbh | Method, computer-readable medium and computer relating to the manufacture of dental prostheses |
DE102006061134A1 (en) * | 2006-12-22 | 2008-06-26 | Aepsilon Rechteverwaltungs Gmbh | Process for the transport of dental prostheses |
US8769099B2 (en) | 2006-12-28 | 2014-07-01 | Yahoo! Inc. | Methods and systems for pre-caching information on a mobile computing device |
US20080071886A1 (en) * | 2006-12-29 | 2008-03-20 | Wesley Scott Ashton | Method and system for internet search |
GB2458094A (en) | 2007-01-09 | 2009-09-09 | Surfcontrol On Demand Ltd | URL interception and categorization in firewalls |
US8595635B2 (en) | 2007-01-25 | 2013-11-26 | Salesforce.Com, Inc. | System, method and apparatus for selecting content from web sources and posting content to web logs |
US7860872B2 (en) * | 2007-01-29 | 2010-12-28 | Nikip Technology Ltd. | Automated media analysis and document management system |
US7693833B2 (en) | 2007-02-01 | 2010-04-06 | John Nagle | System and method for improving integrity of internet search |
US7895515B1 (en) * | 2007-02-28 | 2011-02-22 | Trend Micro Inc | Detecting indicators of misleading content in markup language coded documents using the formatting of the document |
US20080235213A1 (en) * | 2007-03-20 | 2008-09-25 | Picscout (Israel) Ltd. | Utilization of copyright media in second generation web content |
US8068986B1 (en) | 2007-04-27 | 2011-11-29 | Majid Shahbazi | Methods and apparatus related to sensor signal sniffing and/or analysis |
US20080281827A1 (en) * | 2007-05-10 | 2008-11-13 | Microsoft Corporation | Using structured database for webpage information extraction |
GB0709527D0 (en) | 2007-05-18 | 2007-06-27 | Surfcontrol Plc | Electronic messaging system, message processing apparatus and message processing method |
US8805425B2 (en) | 2007-06-01 | 2014-08-12 | Seven Networks, Inc. | Integrated messaging |
US20090037412A1 (en) * | 2007-07-02 | 2009-02-05 | Kristina Butvydas Bard | Qualitative search engine based on factors of consumer trust specification |
US8321359B2 (en) * | 2007-07-24 | 2012-11-27 | Hiconversion, Inc. | Method and apparatus for real-time website optimization |
US8260619B1 (en) | 2008-08-22 | 2012-09-04 | Convergys Cmg Utah, Inc. | Method and system for creating natural language understanding grammars |
WO2009032814A2 (en) * | 2007-09-04 | 2009-03-12 | Nixle, Llc | System and method for collecting and organizing popular near real-time data in a virtual geographic grid |
US20090070419A1 (en) * | 2007-09-11 | 2009-03-12 | International Business Machines Corporation | Administering Feeds Of Presence Information Of One Or More Presentities |
US20090070410A1 (en) * | 2007-09-12 | 2009-03-12 | International Business Machines Corporation | Managing Presence Information Of A Presentity |
CN101855632B (en) * | 2007-11-08 | 2013-10-30 | 上海惠普有限公司 | URL and anchor text analysis for focused crawling |
US8069142B2 (en) | 2007-12-06 | 2011-11-29 | Yahoo! Inc. | System and method for synchronizing data on a network |
US8671154B2 (en) | 2007-12-10 | 2014-03-11 | Yahoo! Inc. | System and method for contextual addressing of communications on a network |
US8364181B2 (en) | 2007-12-10 | 2013-01-29 | Seven Networks, Inc. | Electronic-mail filtering for mobile devices |
US8307029B2 (en) | 2007-12-10 | 2012-11-06 | Yahoo! Inc. | System and method for conditional delivery of messages |
US9002828B2 (en) | 2007-12-13 | 2015-04-07 | Seven Networks, Inc. | Predictive content delivery |
US8166168B2 (en) | 2007-12-17 | 2012-04-24 | Yahoo! Inc. | System and method for disambiguating non-unique identifiers using information obtained from disparate communication channels |
US9706345B2 (en) | 2008-01-04 | 2017-07-11 | Excalibur Ip, Llc | Interest mapping system |
US9626685B2 (en) | 2008-01-04 | 2017-04-18 | Excalibur Ip, Llc | Systems and methods of mapping attention |
US8762285B2 (en) | 2008-01-06 | 2014-06-24 | Yahoo! Inc. | System and method for message clustering |
US20090182618A1 (en) | 2008-01-16 | 2009-07-16 | Yahoo! Inc. | System and Method for Word-of-Mouth Advertising |
US10275524B2 (en) | 2008-01-23 | 2019-04-30 | Sears Holdings Management Corporation | Social network searching with breadcrumbs |
US8862657B2 (en) | 2008-01-25 | 2014-10-14 | Seven Networks, Inc. | Policy based content service |
US20090193338A1 (en) | 2008-01-28 | 2009-07-30 | Trevor Fiatal | Reducing network and battery consumption during content delivery and playback |
US8583639B2 (en) * | 2008-02-19 | 2013-11-12 | International Business Machines Corporation | Method and system using machine learning to automatically discover home pages on the internet |
US8560390B2 (en) | 2008-03-03 | 2013-10-15 | Yahoo! Inc. | Method and apparatus for social network marketing with brand referral |
US8538811B2 (en) | 2008-03-03 | 2013-09-17 | Yahoo! Inc. | Method and apparatus for social network marketing with advocate referral |
US8554623B2 (en) | 2008-03-03 | 2013-10-08 | Yahoo! Inc. | Method and apparatus for social network marketing with consumer referral |
WO2009111869A1 (en) * | 2008-03-10 | 2009-09-17 | Afilias Limited | Platform independent idn e-mail storage translation |
US8756286B2 (en) * | 2008-03-10 | 2014-06-17 | Afilias Limited | Alternate E-mail address configuration |
US7865455B2 (en) * | 2008-03-13 | 2011-01-04 | Opinionlab, Inc. | System and method for providing intelligent support |
US20090240699A1 (en) * | 2008-03-18 | 2009-09-24 | Morgan Christopher B | Integration for intelligence data systems |
US8745133B2 (en) | 2008-03-28 | 2014-06-03 | Yahoo! Inc. | System and method for optimizing the storage of data |
US8589486B2 (en) * | 2008-03-28 | 2013-11-19 | Yahoo! Inc. | System and method for addressing communications |
US10242104B2 (en) * | 2008-03-31 | 2019-03-26 | Peekanalytics, Inc. | Distributed personal information aggregator |
US8271506B2 (en) | 2008-03-31 | 2012-09-18 | Yahoo! Inc. | System and method for modeling relationships between entities |
US20090287641A1 (en) * | 2008-05-13 | 2009-11-19 | Eric Rahm | Method and system for crawling the world wide web |
US8082248B2 (en) * | 2008-05-29 | 2011-12-20 | Rania Abouyounes | Method and system for document classification based on document structure and written style |
US8190594B2 (en) | 2008-06-09 | 2012-05-29 | Brightedge Technologies, Inc. | Collecting and scoring online references |
US8787947B2 (en) | 2008-06-18 | 2014-07-22 | Seven Networks, Inc. | Application discovery on mobile devices |
JP5562328B2 (en) * | 2008-06-23 | 2014-07-30 | ダブル ベリファイ インコーポレイテッド | Automatic monitoring and matching of Internet-based advertisements |
US8065310B2 (en) * | 2008-06-25 | 2011-11-22 | Microsoft Corporation | Topics in relevance ranking model for web search |
US8078158B2 (en) | 2008-06-26 | 2011-12-13 | Seven Networks, Inc. | Provisioning applications for a mobile device |
US8706406B2 (en) | 2008-06-27 | 2014-04-22 | Yahoo! Inc. | System and method for determination and display of personalized distance |
US8452855B2 (en) | 2008-06-27 | 2013-05-28 | Yahoo! Inc. | System and method for presentation of media related to a context |
US8214346B2 (en) * | 2008-06-27 | 2012-07-03 | Cbs Interactive Inc. | Personalization engine for classifying unstructured documents |
US8813107B2 (en) | 2008-06-27 | 2014-08-19 | Yahoo! Inc. | System and method for location based media delivery |
CN102077201A (en) | 2008-06-30 | 2011-05-25 | 网圣公司 | System and method for dynamic and real-time categorization of webpages |
US8170974B2 (en) * | 2008-07-07 | 2012-05-01 | Yahoo! Inc. | Forecasting association rules across user engagement levels |
US8273182B2 (en) * | 2008-07-15 | 2012-09-25 | WLR Enterprises, LLC | Devices and methods for cleaning and drying ice skate blades |
US9047285B1 (en) * | 2008-07-21 | 2015-06-02 | NetBase Solutions, Inc. | Method and apparatus for frame-based search |
US8286171B2 (en) * | 2008-07-21 | 2012-10-09 | Workshare Technology, Inc. | Methods and systems to fingerprint textual information using word runs |
US10230803B2 (en) | 2008-07-30 | 2019-03-12 | Excalibur Ip, Llc | System and method for improved mapping and routing |
US8583668B2 (en) | 2008-07-30 | 2013-11-12 | Yahoo! Inc. | System and method for context enhanced mapping |
US20120166414A1 (en) * | 2008-08-11 | 2012-06-28 | Ultra Unilimited Corporation (dba Publish) | Systems and methods for relevance scoring |
US8386506B2 (en) | 2008-08-21 | 2013-02-26 | Yahoo! Inc. | System and method for context enhanced messaging |
US20100049761A1 (en) * | 2008-08-21 | 2010-02-25 | Bijal Mehta | Search engine method and system utilizing multiple contexts |
US8555080B2 (en) * | 2008-09-11 | 2013-10-08 | Workshare Technology, Inc. | Methods and systems for protect agents using distributed lightweight fingerprints |
US8281027B2 (en) | 2008-09-19 | 2012-10-02 | Yahoo! Inc. | System and method for distributing media related to a location |
US8108778B2 (en) | 2008-09-30 | 2012-01-31 | Yahoo! Inc. | System and method for context enhanced mapping within a user interface |
US9600484B2 (en) | 2008-09-30 | 2017-03-21 | Excalibur Ip, Llc | System and method for reporting and analysis of media consumption data |
US8984165B2 (en) * | 2008-10-08 | 2015-03-17 | Red Hat, Inc. | Data transformation |
US8676782B2 (en) * | 2008-10-08 | 2014-03-18 | International Business Machines Corporation | Information collection apparatus, search engine, information collection method, and program |
US8909759B2 (en) | 2008-10-10 | 2014-12-09 | Seven Networks, Inc. | Bandwidth measurement |
FR2937449B1 (en) * | 2008-10-17 | 2012-11-16 | Philippe Laval | METHOD AND SYSTEM FOR ENRICHING MEL |
US8032930B2 (en) * | 2008-10-17 | 2011-10-04 | Intuit Inc. | Segregating anonymous access to dynamic content on a web server, with cached logons |
US8412709B1 (en) | 2008-10-23 | 2013-04-02 | Google Inc. | Distributed information collection using pre-generated identifier |
WO2010059747A2 (en) * | 2008-11-18 | 2010-05-27 | Workshare Technology, Inc. | Methods and systems for exact data match filtering |
US9805123B2 (en) | 2008-11-18 | 2017-10-31 | Excalibur Ip, Llc | System and method for data privacy in URL based context queries |
US8032508B2 (en) | 2008-11-18 | 2011-10-04 | Yahoo! Inc. | System and method for URL based query for retrieving data related to a context |
US8060492B2 (en) | 2008-11-18 | 2011-11-15 | Yahoo! Inc. | System and method for generation of URL based context queries |
US8024317B2 (en) | 2008-11-18 | 2011-09-20 | Yahoo! Inc. | System and method for deriving income from URL based context queries |
US8406456B2 (en) | 2008-11-20 | 2013-03-26 | Workshare Technology, Inc. | Methods and systems for image fingerprinting |
US9224172B2 (en) | 2008-12-02 | 2015-12-29 | Yahoo! Inc. | Customizable content for distribution in social networks |
US8055675B2 (en) | 2008-12-05 | 2011-11-08 | Yahoo! Inc. | System and method for context based query augmentation |
US8639493B2 (en) * | 2008-12-18 | 2014-01-28 | Intermountain Invention Management, Llc | Probabilistic natural language processing using a likelihood vector |
US8166016B2 (en) | 2008-12-19 | 2012-04-24 | Yahoo! Inc. | System and method for automated service recommendations |
US20100211533A1 (en) * | 2009-02-18 | 2010-08-19 | Microsoft Corporation | Extracting structured data from web forums |
US8150967B2 (en) | 2009-03-24 | 2012-04-03 | Yahoo! Inc. | System and method for verified presence tracking |
US20100250562A1 (en) | 2009-03-24 | 2010-09-30 | Mireo d.o.o. | Recognition of addresses from the body of arbitrary text |
US11489857B2 (en) | 2009-04-21 | 2022-11-01 | Webroot Inc. | System and method for developing a risk profile for an internet resource |
US8793152B2 (en) * | 2009-05-20 | 2014-07-29 | Joseph Ruston Bishop | Mining of distributed scientific data for enriched product/contact valuation |
US9130972B2 (en) | 2009-05-26 | 2015-09-08 | Websense, Inc. | Systems and methods for efficient detection of fingerprinted data and information |
US8495151B2 (en) * | 2009-06-05 | 2013-07-23 | Chandra Bodapati | Methods and systems for determining email addresses |
US8463692B2 (en) * | 2009-06-25 | 2013-06-11 | Tradeking Group, Inc. | Method and system to facilitate on-line trading |
US8463652B2 (en) * | 2009-06-25 | 2013-06-11 | Tradeking Group, Inc. | Method and system to facilitate on-line trading |
US8473847B2 (en) * | 2009-07-27 | 2013-06-25 | Workshare Technology, Inc. | Methods and systems for comparing presentation slide decks |
US9841282B2 (en) | 2009-07-27 | 2017-12-12 | Visa U.S.A. Inc. | Successive offer communications with an offer recipient |
US9258376B2 (en) * | 2009-08-04 | 2016-02-09 | At&T Intellectual Property I, L.P. | Aggregated presence over user federated devices |
US10223701B2 (en) | 2009-08-06 | 2019-03-05 | Excalibur Ip, Llc | System and method for verified monetization of commercial campaigns |
US8914342B2 (en) | 2009-08-12 | 2014-12-16 | Yahoo! Inc. | Personal data platform |
US8364611B2 (en) | 2009-08-13 | 2013-01-29 | Yahoo! Inc. | System and method for precaching information on a mobile device |
US9092424B2 (en) * | 2009-09-30 | 2015-07-28 | Microsoft Technology Licensing, Llc | Webpage entity extraction through joint understanding of page structures and sentences |
US8671089B2 (en) | 2009-10-06 | 2014-03-11 | Brightedge Technologies, Inc. | Correlating web page visits and conversions with external references |
US8595058B2 (en) * | 2009-10-15 | 2013-11-26 | Visa U.S.A. | Systems and methods to match identifiers |
US8332232B2 (en) * | 2009-11-05 | 2012-12-11 | Opinionlab, Inc. | System and method for mobile interaction |
US9576251B2 (en) * | 2009-11-13 | 2017-02-21 | Hewlett Packard Enterprise Development Lp | Method and system for processing web activity data |
US20110125733A1 (en) * | 2009-11-25 | 2011-05-26 | Fish Nathan J | Quick access utility |
US8606792B1 (en) | 2010-02-08 | 2013-12-10 | Google Inc. | Scoring authors of posts |
US8620849B2 (en) * | 2010-03-10 | 2013-12-31 | Lockheed Martin Corporation | Systems and methods for facilitating open source intelligence gathering |
US8819148B2 (en) * | 2010-03-10 | 2014-08-26 | Afilias Limited | Alternate E-mail delivery |
US9183560B2 (en) | 2010-05-28 | 2015-11-10 | Daniel H. Abelow | Reality alternate |
CN102279856B (en) | 2010-06-09 | 2013-10-02 | 阿里巴巴集团控股有限公司 | Method and system for realizing website navigation |
US20110314001A1 (en) * | 2010-06-18 | 2011-12-22 | Microsoft Corporation | Performing query expansion based upon statistical analysis of structured data |
US9043433B2 (en) | 2010-07-26 | 2015-05-26 | Seven Networks, Inc. | Mobile network traffic coordination across multiple applications |
US8838783B2 (en) | 2010-07-26 | 2014-09-16 | Seven Networks, Inc. | Distributed caching for resource and mobile network traffic management |
KR20130065710A (en) | 2010-09-08 | 2013-06-19 | 에버노트 코포레이션 | Site memory processing and clipping control |
US10089404B2 (en) | 2010-09-08 | 2018-10-02 | Evernote Corporation | Site memory processing |
US9195774B2 (en) * | 2010-09-17 | 2015-11-24 | Kontera Technologies, Inc. | Methods and systems for augmenting content displayed on a mobile device |
CN102455997A (en) * | 2010-10-27 | 2012-05-16 | 鸿富锦精密工业(深圳)有限公司 | Component name extraction system and method |
US20120110480A1 (en) * | 2010-10-31 | 2012-05-03 | Sap Portals Israel Ltd | Method and apparatus for rendering a web page |
US8484314B2 (en) | 2010-11-01 | 2013-07-09 | Seven Networks, Inc. | Distributed caching in a wireless network of content delivered for a mobile application over a long-held request |
WO2012060995A2 (en) | 2010-11-01 | 2012-05-10 | Michael Luna | Distributed caching in a wireless network of content delivered for a mobile application over a long-held request |
US8843153B2 (en) | 2010-11-01 | 2014-09-23 | Seven Networks, Inc. | Mobile traffic categorization and policy for network use optimization while preserving user experience |
FR2966949B1 (en) * | 2010-11-02 | 2013-08-16 | Beetween | METHOD FOR AUTOMATING THE CONSTITUTION OF A STRUCTURED DATABASE OF PROFESSIONALS |
US9171089B2 (en) * | 2010-11-16 | 2015-10-27 | John Nicholas and Kristin Gross Trust | Message distribution system and method |
GB2500327B (en) | 2010-11-22 | 2019-11-06 | Seven Networks Llc | Optimization of resource polling intervals to satisfy mobile device requests |
GB2495463B (en) | 2010-11-22 | 2013-10-09 | Seven Networks Inc | Aligning data transfer to optimize connections established for transmission over a wireless network |
US10783326B2 (en) | 2013-03-14 | 2020-09-22 | Workshare, Ltd. | System for tracking changes in a collaborative document editing environment |
US20120133989A1 (en) | 2010-11-29 | 2012-05-31 | Workshare Technology, Inc. | System and method for providing a common framework for reviewing comparisons of electronic documents |
US11030163B2 (en) | 2011-11-29 | 2021-06-08 | Workshare, Ltd. | System for tracking and displaying changes in a set of related electronic documents |
GB2501416B (en) | 2011-01-07 | 2018-03-21 | Seven Networks Llc | System and method for reduction of mobile network traffic used for domain name system (DNS) queries |
US10007915B2 (en) | 2011-01-24 | 2018-06-26 | Visa International Service Association | Systems and methods to facilitate loyalty reward transactions |
US9898533B2 (en) | 2011-02-24 | 2018-02-20 | Microsoft Technology Licensing, Llc | Augmenting search results |
US20120246137A1 (en) * | 2011-03-22 | 2012-09-27 | Satish Sallakonda | Visual profiles |
WO2012145533A2 (en) | 2011-04-19 | 2012-10-26 | Seven Networks, Inc. | Shared resource and virtual resource management in a networked environment |
WO2012149434A2 (en) | 2011-04-27 | 2012-11-01 | Seven Networks, Inc. | Detecting and preserving state for satisfying application requests in a distributed proxy and cache system |
GB2504037B (en) | 2011-04-27 | 2014-12-24 | Seven Networks Inc | Mobile device which offloads requests made by a mobile application to a remote entity for conservation of mobile device and network resources |
US20120284036A1 (en) * | 2011-05-03 | 2012-11-08 | Ecomsystems, Inc. | System and method for linking together an array of business programs |
US8984004B2 (en) * | 2011-05-09 | 2015-03-17 | Smart-Foa | Information collecting system |
US9170990B2 (en) | 2013-03-14 | 2015-10-27 | Workshare Limited | Method and system for document retrieval with selective document comparison |
US9613340B2 (en) | 2011-06-14 | 2017-04-04 | Workshare Ltd. | Method and system for shared document approval |
US9948676B2 (en) | 2013-07-25 | 2018-04-17 | Workshare, Ltd. | System and method for securing documents prior to transmission |
US10574729B2 (en) | 2011-06-08 | 2020-02-25 | Workshare Ltd. | System and method for cross platform document sharing |
US10880359B2 (en) | 2011-12-21 | 2020-12-29 | Workshare, Ltd. | System and method for cross platform document sharing |
US10963584B2 (en) | 2011-06-08 | 2021-03-30 | Workshare Ltd. | Method and system for collaborative editing of a remotely stored document |
US9430583B1 (en) | 2011-06-10 | 2016-08-30 | Salesforce.Com, Inc. | Extracting a portion of a document, such as a web page |
US8706723B2 (en) * | 2011-06-22 | 2014-04-22 | Jostle Corporation | Name-search system and method |
WO2013015994A1 (en) | 2011-07-27 | 2013-01-31 | Seven Networks, Inc. | Monitoring mobile application activities for malicious traffic on a mobile device |
US8650198B2 (en) | 2011-08-15 | 2014-02-11 | Lockheed Martin Corporation | Systems and methods for facilitating the gathering of open source intelligence |
JP5824974B2 (en) * | 2011-08-31 | 2015-12-02 | ブラザー工業株式会社 | Image processing device |
CN103092855B (en) * | 2011-10-31 | 2016-08-24 | 国际商业机器公司 | The method and device that detection address updates |
US9152730B2 (en) * | 2011-11-10 | 2015-10-06 | Evernote Corporation | Extracting principal content from web pages |
CN103150307B (en) * | 2011-12-06 | 2016-02-10 | 株式会社理光 | The method and apparatus of the title relevant to descriptor is searched from network |
US8934414B2 (en) | 2011-12-06 | 2015-01-13 | Seven Networks, Inc. | Cellular or WiFi mobile traffic optimization based on public or private network destination |
US8868753B2 (en) | 2011-12-06 | 2014-10-21 | Seven Networks, Inc. | System of redundantly clustered machines to provide failover mechanisms for mobile traffic management and network resource conservation |
WO2013086447A1 (en) | 2011-12-07 | 2013-06-13 | Seven Networks, Inc. | Radio-awareness of mobile device for sending server-side control signals using a wireless network optimized transport protocol |
US9009250B2 (en) | 2011-12-07 | 2015-04-14 | Seven Networks, Inc. | Flexible and dynamic integration schemas of a traffic management system with various network operators for network traffic alleviation |
EP2792188B1 (en) | 2011-12-14 | 2019-03-20 | Seven Networks, LLC | Mobile network reporting and usage analytics system and method using aggregation of data in a distributed traffic optimization system |
GB2499306B (en) | 2012-01-05 | 2014-10-22 | Seven Networks Inc | Managing user interaction with an application on a mobile device |
CN103218719B (en) | 2012-01-19 | 2016-12-07 | 阿里巴巴集团控股有限公司 | A kind of e-commerce website air navigation aid and system |
WO2013116856A1 (en) | 2012-02-02 | 2013-08-08 | Seven Networks, Inc. | Dynamic categorization of applications for network access in a mobile network |
US9326189B2 (en) | 2012-02-03 | 2016-04-26 | Seven Networks, Llc | User as an end point for profiling and optimizing the delivery of content and data in a wireless network |
US8812695B2 (en) | 2012-04-09 | 2014-08-19 | Seven Networks, Inc. | Method and system for management of a virtual network connection without heartbeat messages |
US10263899B2 (en) | 2012-04-10 | 2019-04-16 | Seven Networks, Llc | Enhanced customer service for mobile carriers using real-time and historical mobile application and traffic or optimization data associated with mobile devices in a mobile network |
US8473293B1 (en) * | 2012-04-17 | 2013-06-25 | Google Inc. | Dictionary filtering using market data |
US9753926B2 (en) * | 2012-04-30 | 2017-09-05 | Salesforce.Com, Inc. | Extracting a portion of a document, such as a web page |
WO2014011216A1 (en) | 2012-07-13 | 2014-01-16 | Seven Networks, Inc. | Dynamic bandwidth adjustment for browsing or streaming activity in a wireless network based on prediction of user behavior when interacting with mobile applications |
US9330093B1 (en) * | 2012-08-02 | 2016-05-03 | Google Inc. | Methods and systems for identifying user input data for matching content to user interests |
GB2506450A (en) * | 2012-10-01 | 2014-04-02 | Wonga Technology Ltd | Web page categorisation |
US9161258B2 (en) | 2012-10-24 | 2015-10-13 | Seven Networks, Llc | Optimized and selective management of policy deployment to mobile clients in a congested network to prevent further aggravation of network congestion |
WO2014093456A2 (en) * | 2012-12-11 | 2014-06-19 | Compete, Inc. | Direct page view measurement tag placement verification |
US9031887B2 (en) | 2012-12-18 | 2015-05-12 | International Business Machines Corporation | Determining a replacement document owner |
US9307493B2 (en) | 2012-12-20 | 2016-04-05 | Seven Networks, Llc | Systems and methods for application management of mobile device radio state promotion and demotion |
FR3000253B1 (en) * | 2012-12-21 | 2016-03-11 | Aleph Networks | METHOD OF COLLECTING THE CONTENT OF PAGES AND CONSTITUTING A RELATIONAL STRUCTURE FROM THE CONTENT |
US9241314B2 (en) | 2013-01-23 | 2016-01-19 | Seven Networks, Llc | Mobile device with application or context aware fast dormancy |
US8874761B2 (en) | 2013-01-25 | 2014-10-28 | Seven Networks, Inc. | Signaling optimization in a wireless network for traffic utilizing proprietary and non-proprietary protocols |
US9002818B2 (en) | 2013-01-31 | 2015-04-07 | Hewlett-Packard Development Company, L.P. | Calculating a content subset |
US8750123B1 (en) | 2013-03-11 | 2014-06-10 | Seven Networks, Inc. | Mobile device equipped with mobile network congestion recognition to make intelligent decisions regarding connecting to an operator network |
US11567907B2 (en) | 2013-03-14 | 2023-01-31 | Workshare, Ltd. | Method and system for comparing document versions encoded in a hierarchical representation |
US9477759B2 (en) | 2013-03-15 | 2016-10-25 | Google Inc. | Question answering using entity references in unstructured data |
US9065765B2 (en) | 2013-07-22 | 2015-06-23 | Seven Networks, Inc. | Proxy server associated with a mobile carrier for enhancing mobile traffic management in a mobile network |
US10911492B2 (en) | 2013-07-25 | 2021-02-02 | Workshare Ltd. | System and method for securing documents prior to transmission |
US9342608B2 (en) | 2013-08-01 | 2016-05-17 | International Business Machines Corporation | Clarification of submitted questions in a question and answer system |
US8831969B1 (en) * | 2013-10-02 | 2014-09-09 | Linkedin Corporation | System and method for determining users working for the same employers in a social network |
US10929858B1 (en) * | 2014-03-14 | 2021-02-23 | Walmart Apollo, Llc | Systems and methods for managing customer data |
US20150347489A1 (en) * | 2014-03-31 | 2015-12-03 | Scott David Sherwin | Information retrieval system and method based on query and record metadata in combination with relevance between disparate items in classification systems |
US11838851B1 (en) | 2014-07-15 | 2023-12-05 | F5, Inc. | Methods for managing L7 traffic classification and devices thereof |
US20160253766A1 (en) * | 2014-10-06 | 2016-09-01 | Shocase, Inc. | System and method for curation of notable work and relating it to involved organizations and individuals |
US9942361B2 (en) * | 2014-10-28 | 2018-04-10 | Cisco Technology, Inc. | Reporting page composition data |
US10133723B2 (en) | 2014-12-29 | 2018-11-20 | Workshare Ltd. | System and method for determining document version geneology |
US11182551B2 (en) | 2014-12-29 | 2021-11-23 | Workshare Ltd. | System and method for determining document version geneology |
US10490306B2 (en) | 2015-02-20 | 2019-11-26 | Cerner Innovation, Inc. | Medical information translation system |
US10834065B1 (en) | 2015-03-31 | 2020-11-10 | F5 Networks, Inc. | Methods for SSL protected NTLM re-authentication and devices thereof |
US10505818B1 (en) | 2015-05-05 | 2019-12-10 | F5 Networks. Inc. | Methods for analyzing and load balancing based on server health and devices thereof |
US20170024375A1 (en) * | 2015-07-26 | 2017-01-26 | Microsoft Technology Licensing, Llc | Personal knowledge graph population from declarative user utterances |
US11763013B2 (en) | 2015-08-07 | 2023-09-19 | Workshare, Ltd. | Transaction document management system and method |
CN106503017A (en) * | 2015-09-08 | 2017-03-15 | 摩贝(上海)生物科技有限公司 | A kind of distributed reptile system task grasping system and method |
US20180276304A1 (en) | 2015-09-21 | 2018-09-27 | Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd | Advanced computer implementation for crawling and/or detecting related electronically catalogued data using improved metadata processing |
US20170091270A1 (en) * | 2015-09-30 | 2017-03-30 | Linkedln Corporation | Organizational url enrichment |
US10430478B1 (en) | 2015-10-28 | 2019-10-01 | Reputation.Com, Inc. | Automatic finding of online profiles of an entity location |
US11570188B2 (en) * | 2015-12-28 | 2023-01-31 | Sixgill Ltd. | Dark web monitoring, analysis and alert system and method |
US10404698B1 (en) * | 2016-01-15 | 2019-09-03 | F5 Networks, Inc. | Methods for adaptive organization of web application access points in webtops and devices thereof |
EP3108849B1 (en) | 2016-04-25 | 2019-04-24 | 3M Innovative Properties Company | Multi-layered zirconia dental mill blank and process of production |
US10606952B2 (en) | 2016-06-24 | 2020-03-31 | Elemental Cognition Llc | Architecture and processes for computer learning and understanding |
US10469394B1 (en) | 2016-08-01 | 2019-11-05 | F5 Networks, Inc. | Methods for configuring adaptive rate limit based on server data and devices thereof |
US10608972B1 (en) | 2016-08-23 | 2020-03-31 | Microsoft Technology Licensing, Llc | Messaging service integration with deduplicator |
US10754914B2 (en) * | 2016-08-24 | 2020-08-25 | Robert Bosch Gmbh | Method and device for unsupervised information extraction |
CN106599297A (en) * | 2016-12-28 | 2017-04-26 | 北京百度网讯科技有限公司 | Method and device for searching question-type search terms on basis of deep questions and answers |
JP2019056954A (en) * | 2017-09-19 | 2019-04-11 | 富士ゼロックス株式会社 | Information processing apparatus and information processing program |
US11409814B2 (en) * | 2017-12-01 | 2022-08-09 | The Regents Of The University Of Colorado | Systems and methods for crawling web pages and parsing relevant information stored in web pages |
US10698937B2 (en) | 2017-12-13 | 2020-06-30 | Microsoft Technology Licensing, Llc | Split mapping for dynamic rendering and maintaining consistency of data processed by applications |
US11605018B2 (en) | 2017-12-27 | 2023-03-14 | Cerner Innovation, Inc. | Ontology-guided reconciliation of electronic records |
US11163840B2 (en) * | 2018-05-24 | 2021-11-02 | Open Text Sa Ulc | Systems and methods for intelligent content filtering and persistence |
US11055365B2 (en) * | 2018-06-29 | 2021-07-06 | Paypal, Inc. | Mechanism for web crawling e-commerce resource pages |
US11361076B2 (en) * | 2018-10-26 | 2022-06-14 | ThreatWatch Inc. | Vulnerability-detection crawler |
CN109857498A (en) * | 2019-01-09 | 2019-06-07 | 明基智能科技(上海)有限公司 | Intelligent content template recommender system and its method |
US11126673B2 (en) * | 2019-01-29 | 2021-09-21 | Salesforce.Com, Inc. | Method and system for automatically enriching collected seeds with information extracted from one or more websites |
US10866996B2 (en) | 2019-01-29 | 2020-12-15 | Saleforce.com, inc. | Automated method and system for clustering enriched company seeds into a cluster and selecting best values for each attribute within the cluster to generate a company profile |
CN110110193B (en) * | 2019-04-24 | 2021-04-30 | 北京百炼智能科技有限公司 | Information processing method and device and computer readable storage medium |
US11134054B2 (en) | 2019-11-05 | 2021-09-28 | International Business Machines Corporation | Classification of a domain name |
US11675805B2 (en) | 2019-12-16 | 2023-06-13 | Cerner Innovation, Inc. | Concept agnostic reconcilation and prioritization based on deterministic and conservative weight methods |
US11467716B1 (en) | 2022-01-28 | 2022-10-11 | Microsoft Technology Licensing, Llc | Flexibly identifying and playing media content from any webpage |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU5303198A (en) * | 1997-02-21 | 1998-08-27 | Dudley John Mills | Network-based classified information systems |
WO1999067728A1 (en) * | 1998-06-23 | 1999-12-29 | Microsoft Corporation | Methods and apparatus for classifying text and for building a text classifier |
Family Cites Families (59)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4270182A (en) | 1974-12-30 | 1981-05-26 | Asija Satya P | Automated information input, storage, and retrieval system |
US5319777A (en) | 1990-10-16 | 1994-06-07 | Sinper Corporation | System and method for storing and retrieving information from a multidimensional array |
GB9220404D0 (en) * | 1992-08-20 | 1992-11-11 | Nat Security Agency | Method of identifying,retrieving and sorting documents |
US5764906A (en) | 1995-11-07 | 1998-06-09 | Netword Llc | Universal electronic resource denotation, request and delivery system |
US5974455A (en) | 1995-12-13 | 1999-10-26 | Digital Equipment Corporation | System for adding new entry to web page table upon receiving web page including link to another web page not having corresponding entry in web page table |
WO1997025798A1 (en) | 1996-01-11 | 1997-07-17 | Mrj, Inc. | System for controlling access and distribution of digital property |
US6076088A (en) | 1996-02-09 | 2000-06-13 | Paik; Woojin | Information extraction system and method using concept relation concept (CRC) triples |
US6418432B1 (en) | 1996-04-10 | 2002-07-09 | At&T Corporation | System and method for finding information in a distributed information system using query learning and meta search |
US5813006A (en) | 1996-05-06 | 1998-09-22 | Banyan Systems, Inc. | On-line directory service with registration system |
US5918236A (en) * | 1996-06-28 | 1999-06-29 | Oracle Corporation | Point of view gists and generic gists in a document browsing system |
US5923850A (en) | 1996-06-28 | 1999-07-13 | Sun Microsystems, Inc. | Historical asset information data storage schema |
US6052693A (en) | 1996-07-02 | 2000-04-18 | Harlequin Group Plc | System for assembling large databases through information extracted from text sources |
US6065016A (en) * | 1996-08-06 | 2000-05-16 | At&T Corporation | Universal directory service |
US5764905A (en) * | 1996-09-09 | 1998-06-09 | Ncr Corporation | Method, system and computer program product for synchronizing the flushing of parallel nodes database segments through shared disk tokens |
US5855011A (en) | 1996-09-13 | 1998-12-29 | Tatsuoka; Curtis M. | Method for classifying test subjects in knowledge and functionality states |
JP2940501B2 (en) | 1996-12-25 | 1999-08-25 | 日本電気株式会社 | Document classification apparatus and method |
AUPO525497A0 (en) * | 1997-02-21 | 1997-03-20 | Mills, Dudley John | Network-based classified information systems |
US5895470A (en) * | 1997-04-09 | 1999-04-20 | Xerox Corporation | System for categorizing documents in a linked collection of documents |
US5835905A (en) * | 1997-04-09 | 1998-11-10 | Xerox Corporation | System for predicting documents relevant to focus documents by spreading activation through network representations of a linked collection of documents |
US5924090A (en) * | 1997-05-01 | 1999-07-13 | Northern Light Technology Llc | Method and apparatus for searching a database of records |
JPH10320315A (en) | 1997-05-14 | 1998-12-04 | Nippon Telegr & Teleph Corp <Ntt> | Electronic mail transmission management device and recording medium for recording program for executing electronic mail transmission management processing |
US6415250B1 (en) * | 1997-06-18 | 2002-07-02 | Novell, Inc. | System and method for identifying language using morphologically-based techniques |
US6128613A (en) * | 1997-06-26 | 2000-10-03 | The Chinese University Of Hong Kong | Method and apparatus for establishing topic word classes based on an entropy cost function to retrieve documents represented by the topic words |
AU9513198A (en) * | 1997-09-30 | 1999-04-23 | Ihc Health Services, Inc. | Aprobabilistic system for natural language processing |
US6266664B1 (en) * | 1997-10-01 | 2001-07-24 | Rulespace, Inc. | Method for scanning, analyzing and rating digital information content |
US6055510A (en) | 1997-10-24 | 2000-04-25 | At&T Corp. | Method for performing targeted marketing over a large computer network |
US6269369B1 (en) * | 1997-11-02 | 2001-07-31 | Amazon.Com Holdings, Inc. | Networked personal contact manager |
US5991756A (en) * | 1997-11-03 | 1999-11-23 | Yahoo, Inc. | Information retrieval from hierarchical compound documents |
US6665841B1 (en) * | 1997-11-14 | 2003-12-16 | Xerox Corporation | Transmission of subsets of layout objects at different resolutions |
US5943670A (en) * | 1997-11-21 | 1999-08-24 | International Business Machines Corporation | System and method for categorizing objects in combined categories |
US6807537B1 (en) | 1997-12-04 | 2004-10-19 | Microsoft Corporation | Mixtures of Bayesian networks |
US6640224B1 (en) * | 1997-12-15 | 2003-10-28 | International Business Machines Corporation | System and method for dynamic index-probe optimizations for high-dimensional similarity search |
US6389436B1 (en) * | 1997-12-15 | 2002-05-14 | International Business Machines Corporation | Enhanced hypertext categorization using hyperlinks |
US6212552B1 (en) | 1998-01-15 | 2001-04-03 | At&T Corp. | Declarative message addressing |
US6112203A (en) * | 1998-04-09 | 2000-08-29 | Altavista Company | Method for ranking documents in a hyperlinked environment using connectivity and selective content analysis |
US6044375A (en) * | 1998-04-30 | 2000-03-28 | Hewlett-Packard Company | Automatic extraction of metadata using a neural network |
US6122647A (en) | 1998-05-19 | 2000-09-19 | Perspecta, Inc. | Dynamic generation of contextual links in hypertext documents |
US6336139B1 (en) | 1998-06-03 | 2002-01-01 | International Business Machines Corporation | System, method and computer program product for event correlation in a distributed computing environment |
US6374259B1 (en) * | 1998-10-01 | 2002-04-16 | Onepin, Llc | Method and apparatus for storing and retreiving business contact information in computer system |
US6397205B1 (en) * | 1998-11-24 | 2002-05-28 | Duquesne University Of The Holy Ghost | Document categorization and evaluation via cross-entrophy |
AU1926300A (en) | 1998-11-30 | 2000-06-19 | Lexeme Corporation | A natural knowledge acquisition method |
FR2790846B1 (en) * | 1999-03-09 | 2001-05-04 | S F C E | DOCUMENT IDENTIFICATION PROCESS |
US6253198B1 (en) * | 1999-05-11 | 2001-06-26 | Search Mechanics, Inc. | Process for maintaining ongoing registration for pages on a given search engine |
US6493703B1 (en) | 1999-05-11 | 2002-12-10 | Prophet Financial Systems | System and method for implementing intelligent online community message board |
US6349309B1 (en) * | 1999-05-24 | 2002-02-19 | International Business Machines Corporation | System and method for detecting clusters of information with application to e-commerce |
US6601026B2 (en) | 1999-09-17 | 2003-07-29 | Discern Communications, Inc. | Information retrieval by natural language querying |
US6442555B1 (en) * | 1999-10-26 | 2002-08-27 | Hewlett-Packard Company | Automatic categorization of documents using document signatures |
US6301614B1 (en) * | 1999-11-02 | 2001-10-09 | Alta Vista Company | System and method for efficient representation of data set addresses in a web crawler |
US6668256B1 (en) * | 2000-01-19 | 2003-12-23 | Autonomy Corporation Ltd | Algorithm for automatic selection of discriminant term combinations for document categorization |
US6519580B1 (en) * | 2000-06-08 | 2003-02-11 | International Business Machines Corporation | Decision-tree-based symbolic rule induction system for text categorization |
US6463430B1 (en) | 2000-07-10 | 2002-10-08 | Mohomine, Inc. | Devices and methods for generating and managing a database |
US6618717B1 (en) * | 2000-07-31 | 2003-09-09 | Eliyon Technologies Corporation | Computer method and apparatus for determining content owner of a website |
US6621930B1 (en) * | 2000-08-09 | 2003-09-16 | Elron Software, Inc. | Automatic categorization of documents based on textual content |
US6647396B2 (en) * | 2000-12-28 | 2003-11-11 | Trilogy Development Group, Inc. | Classification based content management system |
US6697793B2 (en) | 2001-03-02 | 2004-02-24 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | System, method and apparatus for generating phrases from a database |
US6621960B2 (en) * | 2002-01-24 | 2003-09-16 | Oplink Communications, Inc. | Method of fabricating multiple superimposed fiber Bragg gratings |
US20030221163A1 (en) * | 2002-02-22 | 2003-11-27 | Nec Laboratories America, Inc. | Using web structure for classifying and describing web pages |
US20030225763A1 (en) * | 2002-04-15 | 2003-12-04 | Microsoft Corporation | Self-improving system and method for classifying pages on the world wide web |
US20060288015A1 (en) * | 2005-06-15 | 2006-12-21 | Schirripa Steven R | Electronic content classification |
-
2000
- 2000-11-01 US US09/704,080 patent/US6618717B1/en not_active Expired - Lifetime
- 2000-11-01 US US09/703,907 patent/US6778986B1/en not_active Expired - Lifetime
-
2001
- 2001-01-24 US US09/768,869 patent/US7356761B2/en not_active Expired - Lifetime
- 2001-03-30 US US09/821,908 patent/US6983282B2/en not_active Expired - Lifetime
- 2001-07-17 WO PCT/US2001/022381 patent/WO2002010955A2/en active Application Filing
- 2001-07-17 AU AU2001273522A patent/AU2001273522A1/en not_active Abandoned
- 2001-07-17 AU AU2001273513A patent/AU2001273513A1/en not_active Abandoned
- 2001-07-17 WO PCT/US2001/022385 patent/WO2002010956A2/en active Application Filing
- 2001-07-17 AU AU2001276940A patent/AU2001276940A1/en not_active Abandoned
- 2001-07-17 WO PCT/US2001/022430 patent/WO2002010957A2/en active Application Filing
- 2001-07-17 AU AU2001278938A patent/AU2001278938A1/en not_active Abandoned
- 2001-07-17 WO PCT/US2001/022426 patent/WO2002010982A2/en active Application Filing
- 2001-07-20 US US09/910,169 patent/US7065483B2/en active Active
- 2001-07-25 WO PCT/US2001/023343 patent/WO2002010960A2/en active Application Filing
- 2001-07-25 AU AU2001279003A patent/AU2001279003A1/en not_active Abandoned
- 2001-07-27 US US09/917,200 patent/US7054886B2/en not_active Expired - Lifetime
- 2001-07-30 US US09/918,312 patent/US20020032740A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU5303198A (en) * | 1997-02-21 | 1998-08-27 | Dudley John Mills | Network-based classified information systems |
WO1999067728A1 (en) * | 1998-06-23 | 1999-12-29 | Microsoft Corporation | Methods and apparatus for classifying text and for building a text classifier |
Non-Patent Citations (2)
Title |
---|
MILLER R C ET AL: "SPHINX: a framework for creating personal, site-specific Web crawlers" COMPUTER NETWORKS AND ISDN SYSTEMS, NORTH HOLLAND PUBLISHING. AMSTERDAM, NL, vol. 30, no. 1-7, 1 April 1998 (1998-04-01), pages 119-130, XP004121434 ISSN: 0169-7552 * |
POWELL, T.A. ET AL.: "HTML Programmer's Reference" 1998 , OSBORNE/MCGRAW-HILL , BERKELEY, USA XP002228395 page 356, line 14 -page 357, line 34 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8954416B2 (en) | 2004-11-22 | 2015-02-10 | Facebook, Inc. | Method and apparatus for an application crawler |
US9405833B2 (en) | 2004-11-22 | 2016-08-02 | Facebook, Inc. | Methods for analyzing dynamic web pages |
US9053179B2 (en) | 2006-04-05 | 2015-06-09 | Lexisnexis, A Division Of Reed Elsevier Inc. | Citation network viewer and method |
WO2008079048A1 (en) * | 2006-12-26 | 2008-07-03 | Pavel Mikhaylovich Malyshev | Computerized method for converting the sequence of conforming computer codes requested by an information user and a system for carrying out said method |
EP3214557A4 (en) * | 2014-10-30 | 2017-09-06 | Alibaba Group Holding Limited | Web page deduplication method and apparatus |
US10691769B2 (en) | 2014-10-30 | 2020-06-23 | Alibaba Group Holding Limited | Methods and apparatus for removing a duplicated web page |
RU2683157C2 (en) * | 2016-12-27 | 2019-03-26 | Федеральное Государственное Бюджетное Научное Учреждение "Всероссийский Научно-Исследовательский Институт Картофельного Хозяйства Имени А.Г. Лорха" (Фгбну Вниикх) | Method of searching information |
Also Published As
Publication number | Publication date |
---|---|
WO2002010960A2 (en) | 2002-02-07 |
WO2002010955A2 (en) | 2002-02-07 |
WO2002010957A3 (en) | 2003-04-10 |
WO2002010955A3 (en) | 2003-08-07 |
WO2002010960A3 (en) | 2003-07-17 |
WO2002010957A2 (en) | 2002-02-07 |
US20020091688A1 (en) | 2002-07-11 |
US7356761B2 (en) | 2008-04-08 |
US6618717B1 (en) | 2003-09-09 |
US20020052928A1 (en) | 2002-05-02 |
US20020032740A1 (en) | 2002-03-14 |
US6778986B1 (en) | 2004-08-17 |
WO2002010982A3 (en) | 2003-07-10 |
US7065483B2 (en) | 2006-06-20 |
US6983282B2 (en) | 2006-01-03 |
WO2002010956A3 (en) | 2003-08-21 |
AU2001273522A1 (en) | 2002-02-13 |
AU2001278938A1 (en) | 2002-02-13 |
US20020138525A1 (en) | 2002-09-26 |
US7054886B2 (en) | 2006-05-30 |
WO2002010957A8 (en) | 2002-07-25 |
AU2001273513A1 (en) | 2002-02-13 |
AU2001279003A1 (en) | 2002-02-13 |
US20020059251A1 (en) | 2002-05-16 |
AU2001276940A1 (en) | 2002-02-13 |
WO2002010956A2 (en) | 2002-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6983282B2 (en) | Computer method and apparatus for collecting people and organization information from Web sites | |
US7469254B2 (en) | Method and apparatus for notifying a user of new data entered into an electronic system | |
US8577867B2 (en) | Method and system for expanding a website | |
US8510339B1 (en) | Searching content using a dimensional database | |
Beel et al. | The architecture and datasets of Docear's Research paper recommender system | |
Zuccala et al. | Web intelligence analyses of digital libraries: A case study of the National electronic Library for Health (NeLH) | |
Lai et al. | A system architecture of intelligent-guided browsing on the Web | |
WO2002010968A2 (en) | Data mining system | |
Vidmar et al. | Internet Search Tools: History to 2000 | |
Streilein | ISPRS ON THE INTERNET–PRESENCE AND PROSPECTS | |
Fisher et al. | The role for Web search engines | |
O'leary | Guest editor's introduction: AI-Assisted browsing | |
Rackley | Navigating the Web Archives: A Study of Users' Understanding of Context | |
Nirankari | WEB USERS SEARCH BEHAVIOUR ANALYSIS USING MACHINE LEARNING | |
Wu et al. | Information Exploration on the World Wide Web |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: JP |