WO2002017083A3 - Data processing method and apparatus - Google Patents

Data processing method and apparatus Download PDF

Info

Publication number
WO2002017083A3
WO2002017083A3 PCT/US2001/026032 US0126032W WO0217083A3 WO 2002017083 A3 WO2002017083 A3 WO 2002017083A3 US 0126032 W US0126032 W US 0126032W WO 0217083 A3 WO0217083 A3 WO 0217083A3
Authority
WO
WIPO (PCT)
Prior art keywords
data
lookup cache
instances
data set
data processing
Prior art date
Application number
PCT/US2001/026032
Other languages
French (fr)
Other versions
WO2002017083A2 (en
Inventor
Eva Man-Yan Tse
Pinaki Mukhopadhyay
Sumitro Samaddar
Original Assignee
Informatica Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Informatica Corp filed Critical Informatica Corp
Priority to DE60121231T priority Critical patent/DE60121231T2/en
Priority to EP01966022A priority patent/EP1410214B1/en
Priority to CA002420214A priority patent/CA2420214C/en
Priority to AU2001286566A priority patent/AU2001286566A1/en
Publication of WO2002017083A2 publication Critical patent/WO2002017083A2/en
Publication of WO2002017083A3 publication Critical patent/WO2002017083A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99931—Database or file accessing
    • Y10S707/99938—Concurrency, e.g. lock management in shared database
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99951—File or database maintenance
    • Y10S707/99952—Coherency, e.g. same view to multiple users

Abstract

A method and apparatus for processing (transporting) data, such as in a data warehouse system. In one embodiment, the data are received from a source and compared to data in a lookup cache comprising a subset of data from a first data set (e.g., a dimension table). Instances of the data not present in a lookup cache (that is, new data) are identified. Information corresponding to these instances are generated (e.g., a unique identifier is associated with each of these instances), and the first data set is updated accordingly. The lookup cache is then updated with the new data and the unique identifiers. Accordingly, the information (data) in the lookup cache and in the first data set are in synchronization. The lookup cache does not need to be rebuilt (e.g., to update a second data set such as a fact table), and therefore data processing can be more quickly completed using less computational resources.
PCT/US2001/026032 2000-08-22 2001-08-20 Data processing method and apparatus WO2002017083A2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
DE60121231T DE60121231T2 (en) 2000-08-22 2001-08-20 DATA PROCESSING
EP01966022A EP1410214B1 (en) 2000-08-22 2001-08-20 Data processing method
CA002420214A CA2420214C (en) 2000-08-22 2001-08-20 Data processing method and apparatus
AU2001286566A AU2001286566A1 (en) 2000-08-22 2001-08-20 Data processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/644,280 2000-08-22
US09/644,280 US6895471B1 (en) 2000-08-22 2000-08-22 Method and apparatus for synchronizing cache with target tables in a data warehousing system

Publications (2)

Publication Number Publication Date
WO2002017083A2 WO2002017083A2 (en) 2002-02-28
WO2002017083A3 true WO2002017083A3 (en) 2003-12-31

Family

ID=24584221

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/026032 WO2002017083A2 (en) 2000-08-22 2001-08-20 Data processing method and apparatus

Country Status (7)

Country Link
US (1) US6895471B1 (en)
EP (1) EP1410214B1 (en)
AT (1) ATE331988T1 (en)
AU (1) AU2001286566A1 (en)
CA (1) CA2420214C (en)
DE (1) DE60121231T2 (en)
WO (1) WO2002017083A2 (en)

Families Citing this family (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7225219B2 (en) * 2000-11-29 2007-05-29 Broadspider Networks, Inc. Distributed caching architecture for computer networks
US20020103974A1 (en) * 2000-11-29 2002-08-01 Giacomini Peter Joseph Method and apparatus for economical cache population
US20020143901A1 (en) * 2001-04-03 2002-10-03 Gtech Rhode Island Corporation Interactive media response processing system
US7200720B1 (en) * 2001-12-28 2007-04-03 Oracle International Corporation System and method for efficiently performing memory intensive computations including a bidirectional synchronization mechanism for maintaining consistency of data
TW200419413A (en) 2003-01-13 2004-10-01 I2 Technologies Inc Master data management system for centrally managing core reference data associated with an enterprise
US7899784B2 (en) * 2003-05-28 2011-03-01 Oracle International Corporation Method and apparatus for performing multi-table merge operations in a database environment
US9946779B2 (en) * 2003-05-28 2018-04-17 Oracle International Corporation Pipleline merge operations using source data and multiple destination data structures
US7222123B2 (en) * 2003-05-28 2007-05-22 Oracle International Corporation Technique for using a current lookup for performing multiple merge operations using source data that is modified in between the merge operations
US7206784B2 (en) * 2003-05-28 2007-04-17 Oracle International Corporation Method and apparatus for performing multiple merge operations using source data that is modified in between the merge operations
EP1501021A1 (en) * 2003-07-22 2005-01-26 Sap Ag A system and method for extracting data sets from an online relational database into a data warehouse
US20050273721A1 (en) * 2004-06-07 2005-12-08 Yantis David B Data transformation system
US7702686B2 (en) * 2005-07-29 2010-04-20 Microsoft Corporation Retrieving and persisting objects from/to relational databases
US20080140696A1 (en) * 2006-12-07 2008-06-12 Pantheon Systems, Inc. System and method for analyzing data sources to generate metadata
US7644112B2 (en) * 2007-02-01 2010-01-05 International Business Machines Corporation Apparatus, system, and method for initializing a synchronized remote database
US8533661B2 (en) 2007-04-27 2013-09-10 Dell Products, Lp System and method for automated on-demand creation of a customized software application
US7991794B2 (en) * 2007-12-18 2011-08-02 Oracle International Corporation Pipelining operations involving DML and query
US8001332B2 (en) * 2008-04-30 2011-08-16 Siemens Aktiengesellschaft Adaptive caching for high volume extract transform load process
US9692856B2 (en) * 2008-07-25 2017-06-27 Ca, Inc. System and method for filtering and alteration of digital data packets
US8401990B2 (en) * 2008-07-25 2013-03-19 Ca, Inc. System and method for aggregating raw data into a star schema
EP2184688A1 (en) * 2008-11-06 2010-05-12 Amadeus s.a.s Method of integrating in real time large volumes of updates in a database
US20100161552A1 (en) * 2008-12-24 2010-06-24 Dan Murarasu Method and system for referencing measures between tables of analytical report documents
US20110153563A1 (en) * 2009-12-22 2011-06-23 International Business Machines Corporation Enhanced replication of databases
US8868484B2 (en) * 2010-07-08 2014-10-21 Oracle International Corporation Efficiently updating rows in a data warehouse
US9710282B2 (en) 2011-12-21 2017-07-18 Dell Products, Lp System to automate development of system integration application programs and method therefor
US8943076B2 (en) 2012-02-06 2015-01-27 Dell Products, Lp System to automate mapping of variables between business process applications and method therefor
US8805716B2 (en) 2012-03-19 2014-08-12 Dell Products, Lp Dashboard system and method for identifying and monitoring process errors and throughput of integration software
US8782103B2 (en) 2012-04-13 2014-07-15 Dell Products, Lp Monitoring system for optimizing integrated business processes to work flow
US9015106B2 (en) 2012-04-30 2015-04-21 Dell Products, Lp Cloud based master data management system and method therefor
US9158782B2 (en) 2012-04-30 2015-10-13 Dell Products, Lp Cloud based master data management system with configuration advisor and method therefore
US9606995B2 (en) 2012-04-30 2017-03-28 Dell Products, Lp Cloud based master data management system with remote data store and method therefor
US8589207B1 (en) 2012-05-15 2013-11-19 Dell Products, Lp System and method for determining and visually predicting at-risk integrated processes based on age and activity
US9069898B2 (en) 2012-05-31 2015-06-30 Dell Products, Lp System for providing regression testing of an integrated process development system and method therefor
US9092244B2 (en) 2012-06-07 2015-07-28 Dell Products, Lp System for developing custom data transformations for system integration application programs
US20140074659A1 (en) * 2012-09-07 2014-03-13 Oracle International Corporation Ramped ordering for cloud services
US9253113B2 (en) 2012-09-07 2016-02-02 Oracle International Corporation Customizable model for throttling and prioritizing orders in a cloud environment
US9667470B2 (en) 2012-09-07 2017-05-30 Oracle International Corporation Failure handling in the execution flow of provisioning operations in a cloud environment
US10521746B2 (en) 2012-09-07 2019-12-31 Oracle International Corporation Recovery workflow for processing subscription orders in a computing infrastructure system
US10148530B2 (en) 2012-09-07 2018-12-04 Oracle International Corporation Rule based subscription cloning
US9704118B2 (en) * 2013-03-11 2017-07-11 Sap Se Predictive analytics in determining key performance indicators
US9183074B2 (en) 2013-06-21 2015-11-10 Dell Products, Lp Integration process management console with error resolution interface
WO2015134019A1 (en) * 2014-03-05 2015-09-11 Hewlett-Packard Development Company, L.P. Processing primary key modifications
US10164901B2 (en) 2014-08-22 2018-12-25 Oracle International Corporation Intelligent data center selection
US10498858B2 (en) 2016-12-14 2019-12-03 Dell Products, Lp System and method for automated on-demand creation of and execution of a customized data integration software application
CN106997378B (en) * 2017-03-13 2020-05-15 上海摩库数据技术有限公司 Redis-based database data aggregation synchronization method
GB2589193B (en) * 2019-08-23 2022-09-21 Raytheon Tech Corp Augmented drive of compressors via differential and multistage turbine
US11809400B2 (en) 2020-12-18 2023-11-07 Samsung Electronics Co., Ltd. Electronic apparatus and controlling method thereof

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6032158A (en) * 1997-05-02 2000-02-29 Informatica Corporation Apparatus and method for capturing and propagating changes from an operational database to data marts

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5832496A (en) * 1995-10-12 1998-11-03 Ncr Corporation System and method for performing intelligent analysis of a computer database
US5991742A (en) * 1996-05-20 1999-11-23 Tran; Bao Q. Time and expense logging system
US5987454A (en) * 1997-06-09 1999-11-16 Hobbs; Allen Method and apparatus for selectively augmenting retrieved text, numbers, maps, charts, still pictures and/or graphics, moving pictures and/or graphics and audio information from a network resource
US6339775B1 (en) * 1997-11-07 2002-01-15 Informatica Corporation Apparatus and method for performing data transformations in data warehousing
US20020099691A1 (en) * 1998-06-24 2002-07-25 Michael Dean Lore Method and apparatus for aggregation of data in a database management system
US6272598B1 (en) * 1999-03-22 2001-08-07 Hewlett-Packard Company Web cache performance by applying different replacement policies to the web cache
US6493800B1 (en) * 1999-03-31 2002-12-10 International Business Machines Corporation Method and system for dynamically partitioning a shared cache
US6484179B1 (en) * 1999-10-25 2002-11-19 Oracle Corporation Storing multidimensional data in a relational database management system
US6446062B1 (en) * 1999-12-23 2002-09-03 Bull Hn Information Systems Inc. Method and apparatus for improving the performance of a generated code cache search operation through the use of static key values
US6601062B1 (en) * 2000-06-27 2003-07-29 Ncr Corporation Active caching for multi-dimensional data sets in relational database management system
US6629102B1 (en) * 2000-07-28 2003-09-30 International Business Machines Corporation Efficiently updating a key table during outline restructure of a multi-dimensional database

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6032158A (en) * 1997-05-02 2000-02-29 Informatica Corporation Apparatus and method for capturing and propagating changes from an operational database to data marts

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CHAUDHURI S ET AL: "An Overview of Data Warehousing and OLAP Technology", SIGMOD RECORD, SIGMOD, NEW YORK, NY, US, vol. 26, no. 1, March 1997 (1997-03-01), pages 65 - 74, XP002193792, ISSN: 0163-5808 *
MOHANIA M ET AL: "ADVANCES AND RESEARCH DIRECTIONS IN DATA WAREHOUSING TECHNOLOGY", AVIS. AUSTRALIAN JOURNAL OF INFORMATION SYSTEMS, WOLLONGONG, AU, vol. 7, no. 1, September 1999 (1999-09-01), pages 41 - 59, XP000978044, ISSN: 1039-7841 *
PRASAN, SESHADRI, SHENOY, SUDARSHAN: "Don't Trash your Intermediate Results, Cache'em", TECHNICAL REPORT, 2 March 2000 (2000-03-02), IIT Bombay, pages 1 - 22, XP002260088, Retrieved from the Internet <URL:http://arxiv.org/abs/cs.DB/0003005> [retrieved on 20031103] *

Also Published As

Publication number Publication date
AU2001286566A1 (en) 2002-03-04
DE60121231D1 (en) 2006-08-10
CA2420214C (en) 2007-05-15
ATE331988T1 (en) 2006-07-15
US6895471B1 (en) 2005-05-17
CA2420214A1 (en) 2002-02-28
DE60121231T2 (en) 2007-06-06
EP1410214B1 (en) 2006-06-28
EP1410214A2 (en) 2004-04-21
WO2002017083A2 (en) 2002-02-28

Similar Documents

Publication Publication Date Title
WO2002017083A3 (en) Data processing method and apparatus
TWI264639B (en) Method and apparatus for accessing shared-access information stored in memory, and computer program product residing on a computer-readable medium
US8006194B2 (en) Associating an object with a relevant data source
DE60323116D1 (en) NWORTVORVERARBEITUNG
MY146705A (en) Method, device and system for synchronizing of data providing for the handling of an interrupted synchronization process
WO1991000666A3 (en) Method and apparatus for processing digital data
WO2005104569A3 (en) Network presence updating apparatus and method
MY133528A (en) System and method for associating properties with objects
WO2005029356A3 (en) Method, system and program for data synchronization
AU2002337940A1 (en) Method and apparatus for a packet classifier using a two-step hash matching process
MY150115A (en) System and method for managing and communicating software updates
CN104778266A (en) Multi-data source searching method and device
CN103873883B (en) Video playing method and device and terminal equipment
GB2442409A (en) Techniques to improve contrast enhancement using a luminance histogram
TW200620091A (en) System and method for an optimized least recently used lookup cache
WO2002101525A3 (en) Apparatus and methods for a hash processing system using integrated message digest and secure hash architectures
AU2003208511A1 (en) Method of updating an authentication algorithm in a computer system
WO2001069527A3 (en) Trainable, extensible, automated data-to-knowledge translator
HK1046322B (en) Communication architecture for distributed computing environment
CN106709066B (en) Data synchronization method and device
WO2016061925A1 (en) Method, apparatus for maintaining routing table and storage medium
WO2005120034A3 (en) System and method for intelligent dynamic message addressing
US20070226271A1 (en) System and method for reliable symmetric data synchronization
EP0862273A3 (en) Data padding method and apparatus
DE60301899D1 (en) Network management system, managed device, management device and program

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PH PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2420214

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2001966022

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWP Wipo information: published in national office

Ref document number: 2001966022

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: JP

WWG Wipo information: grant in national office

Ref document number: 2001966022

Country of ref document: EP