WO2006110485A3 - Method and system for handling large data sets in a statistical language - Google Patents

Method and system for handling large data sets in a statistical language Download PDF

Info

Publication number
WO2006110485A3
WO2006110485A3 PCT/US2006/012891 US2006012891W WO2006110485A3 WO 2006110485 A3 WO2006110485 A3 WO 2006110485A3 US 2006012891 W US2006012891 W US 2006012891W WO 2006110485 A3 WO2006110485 A3 WO 2006110485A3
Authority
WO
WIPO (PCT)
Prior art keywords
bdol
big data
data sets
abstract
large data
Prior art date
Application number
PCT/US2006/012891
Other languages
French (fr)
Other versions
WO2006110485A2 (en
Inventor
David M Smith
Michael J Sannella
Charles B Roosen
William W Dunlap
Original Assignee
Insightful Corp
David M Smith
Michael J Sannella
Charles B Roosen
William W Dunlap
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Insightful Corp, David M Smith, Michael J Sannella, Charles B Roosen, William W Dunlap filed Critical Insightful Corp
Priority to CA2603515A priority Critical patent/CA2603515C/en
Priority to EP06749443.5A priority patent/EP1872229A4/en
Publication of WO2006110485A2 publication Critical patent/WO2006110485A2/en
Publication of WO2006110485A3 publication Critical patent/WO2006110485A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor

Abstract

Methods and systems for providing support for large data sets are provided. Example embodiments provide a Big Data Object Library 'BDOL,' which defines data structures and routines for handling big data objects using out of memory techniques. In one embodiment, the BDOL defines a bdFrame object which stores the data in binary form in a cache on an external storage medium, such as a file on a disk. The example BDOL provides support for user defined block processing a bdFrames using a pipeline engine. Also, the BDOL provides for Trellis plots, and other charts, of big data objects using hexagonal binning. This abstract is provided to comply with rules requiring an abstract, and it is submitted with the intention that it will not be used to interpret or limit the scope or meaning of the claims.
PCT/US2006/012891 2005-04-07 2006-04-07 Method and system for handling large data sets in a statistical language WO2006110485A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CA2603515A CA2603515C (en) 2005-04-07 2006-04-07 Method and system for handling large data sets in a statistical language
EP06749443.5A EP1872229A4 (en) 2005-04-07 2006-04-07 Method and system for handling large data sets in a statistical language

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US66985405P 2005-04-07 2005-04-07
US60/669,854 2005-04-07

Publications (2)

Publication Number Publication Date
WO2006110485A2 WO2006110485A2 (en) 2006-10-19
WO2006110485A3 true WO2006110485A3 (en) 2007-03-22

Family

ID=37087528

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/012891 WO2006110485A2 (en) 2005-04-07 2006-04-07 Method and system for handling large data sets in a statistical language

Country Status (4)

Country Link
US (1) US7739311B2 (en)
EP (1) EP1872229A4 (en)
CA (1) CA2603515C (en)
WO (1) WO2006110485A2 (en)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070055619A1 (en) * 2005-08-26 2007-03-08 Sas Institute Inc. Systems and methods for analyzing disparate treatment in financial transactions
US20090182899A1 (en) * 2008-01-15 2009-07-16 Microsoft Corporation Methods and apparatus relating to wire formats for sql server environments
US20100175049A1 (en) * 2009-01-07 2010-07-08 Microsoft Corporation Scope: a structured computations optimized for parallel execution script language
US9298789B2 (en) * 2009-01-23 2016-03-29 Hewlett Packard Enterprise Development Lp Placement of cells in bins to provide non-overlapping visualization of data points of a scatter plot
US8643646B2 (en) * 2009-03-16 2014-02-04 Hewlett-Packard Development Company, L.P. Constructing a cell-based cluster of data records of a scatter plot
US8407588B1 (en) * 2009-10-22 2013-03-26 The Boeing Company Large columnar text file editor
US10289636B2 (en) * 2010-02-08 2019-05-14 Here Global B.V. Virtual table generator for analyzing geographic databases
US9679401B2 (en) 2010-03-30 2017-06-13 Hewlett Packard Enterprise Development Lp Generalized scatter plots
US8538934B2 (en) * 2011-10-28 2013-09-17 Microsoft Corporation Contextual gravitation of datasets and data services
US9507807B1 (en) * 2011-11-07 2016-11-29 EMC IP Holding Company, LLC Meta file system for big data
US9275059B1 (en) * 2011-11-07 2016-03-01 Emc Corporation Genome big data indexing
US9280612B2 (en) 2012-12-14 2016-03-08 Hewlett Packard Enterprise Development Lp Visualizing a relationship of attributes using a relevance determination process to select from candidate attribute values
US8977589B2 (en) 2012-12-19 2015-03-10 International Business Machines Corporation On the fly data binning
US9607019B1 (en) * 2013-01-17 2017-03-28 Amazon Technologies, Inc. Splitting database partitions
US8583631B1 (en) 2013-01-31 2013-11-12 Splunk Inc. Metadata tracking for a pipelined search language (data modeling for fields)
US9348855B2 (en) 2013-02-13 2016-05-24 International Business Machines Corporation Supporting big data in enterprise content management systems
US20140358954A1 (en) * 2013-03-15 2014-12-04 Ideal Innovations Incorporated Biometric Social Network
US9720940B2 (en) * 2013-03-15 2017-08-01 Konstantinos (Constantin) F. Aliferis Data analysis computer system and method for parallelized and modularized analysis of big data
US20140282188A1 (en) * 2013-03-15 2014-09-18 Moresteam Development Llc Computer graphical user interface, system, and method
KR20150033453A (en) * 2013-09-24 2015-04-01 주식회사 엘지씨엔에스 Method of big data processing, apparatus performing the same and storage media storing the same
US10860186B2 (en) * 2014-09-26 2020-12-08 Oracle International Corporation User interface component wiring for a web portal
CN105718491A (en) * 2014-12-04 2016-06-29 阿里巴巴集团控股有限公司 Updating method and device between databases
US9934257B2 (en) * 2015-07-14 2018-04-03 American Express Travel Related Services Company, Inc. System and method for recursive metadata layers on big data sets
US10055426B2 (en) 2015-11-18 2018-08-21 American Express Travel Related Services Company, Inc. System and method transforming source data into output data in big data environments
CN108388605A (en) * 2018-02-06 2018-08-10 广东暨通信息发展有限公司 Big data analysis platform based on Internet of Things
CN111708621B (en) * 2020-05-22 2024-03-29 伟恩测试技术(武汉)有限公司 Display method of Pattern file based on multithread parallel processing
CN115598455B (en) * 2022-11-15 2023-04-07 西安弘捷电子技术有限公司 Automatic test system and test method for electronic information equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6262740B1 (en) * 1997-08-01 2001-07-17 Terarecon, Inc. Method for rendering sections of a volume data set
US6442666B1 (en) * 1999-01-28 2002-08-27 Infineon Technologies Ag Techniques for improving memory access in a virtual memory system
US20030144753A1 (en) * 2002-01-10 2003-07-31 Shuji Otani Programmable controller unit and method of processing user program
US20040205697A1 (en) * 2001-02-16 2004-10-14 Dave Hylands Transferring data along with code for program overlays
US20050066147A1 (en) * 2003-04-30 2005-03-24 Miller Steven C. System and method for performing address translation in a computer system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5963879A (en) * 1997-11-26 1999-10-05 Schlumberger Technology Corporation Binning of three dimensional seismic data
EP1125224A4 (en) * 1998-10-02 2006-10-25 Ncr Corp Techniques for deploying analytic models in parallel
CA2625653C (en) * 1999-04-21 2011-08-02 Spss, Inc. Computer method and apparatus for creating visible graphics by using a graph algebra
US6480950B1 (en) * 2000-01-24 2002-11-12 Oracle International Corporation Software paging system
US20020029207A1 (en) * 2000-02-28 2002-03-07 Hyperroll, Inc. Data aggregation server for managing a multi-dimensional database and database management system having data aggregation server integrated therein
US7191183B1 (en) * 2001-04-10 2007-03-13 Rgi Informatics, Llc Analytics and data warehousing infrastructure and services
US7272590B2 (en) * 2002-04-26 2007-09-18 International Business Machines Corporation System and method for determining numerical representations for categorical data fields

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6262740B1 (en) * 1997-08-01 2001-07-17 Terarecon, Inc. Method for rendering sections of a volume data set
US6442666B1 (en) * 1999-01-28 2002-08-27 Infineon Technologies Ag Techniques for improving memory access in a virtual memory system
US20040205697A1 (en) * 2001-02-16 2004-10-14 Dave Hylands Transferring data along with code for program overlays
US20030144753A1 (en) * 2002-01-10 2003-07-31 Shuji Otani Programmable controller unit and method of processing user program
US20050066147A1 (en) * 2003-04-30 2005-03-24 Miller Steven C. System and method for performing address translation in a computer system

Also Published As

Publication number Publication date
EP1872229A2 (en) 2008-01-02
CA2603515C (en) 2015-02-10
EP1872229A4 (en) 2017-08-02
CA2603515A1 (en) 2006-10-19
US20070040094A1 (en) 2007-02-22
US7739311B2 (en) 2010-06-15
WO2006110485A2 (en) 2006-10-19

Similar Documents

Publication Publication Date Title
WO2006110485A3 (en) Method and system for handling large data sets in a statistical language
WO2009126644A3 (en) Methods and systems for improved throughput performance in a distributed data de-duplication environment
TW200634622A (en) Register file regions for a processing system
EP2577470A4 (en) Cache management and acceleration of storage media
WO2007030757A3 (en) Systems and methods for organizing media based on associated metadata
EP2293191A3 (en) Task and data management in a multiprocessor system
WO2019170176A3 (en) System and method for data processing
TW200515270A (en) Promotion and demotion techniques to facilitate file property management between object systems
WO2007138600A3 (en) Method and system for transformation of logical data objects for storage
WO2006102621A3 (en) System and method for tracking changes to files in streaming applications
WO2007143592A3 (en) Content description system
WO2007002282A3 (en) Managing memory pages
WO2005052734A3 (en) Block level data snapshot system and method
EP2026188A3 (en) Storage system
WO2011011120A3 (en) Selective hibernation of activities in an electronic device
WO2006115589A3 (en) Manipulating data in a data storage syste
WO2009094594A3 (en) Distributed indexing of file content
TW200641599A (en) Large file storage management method and system
WO2012023967A3 (en) System and method for efficient data storage
WO2006115769A3 (en) Methods and systems for processing objects in memory
WO2007109707A3 (en) Method and system for rendering harmless a locked pestware executable object
WO2007047062A3 (en) Storage of transformed units of data in a memory system having fixed sized storage blocks
WO2006067435A3 (en) Microprocessor systems
WO2007072051A3 (en) Data tracking system
WO2005069766A3 (en) Self-contained cell culture apparatus and method of use

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase

Ref document number: 2603515

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2006749443

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2006749443

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: RU