WO2002015162A2 - System and method for displaying large images with reduced capacity buffer, file format conversion, user interface with zooming and panning, and broadcast of different images - Google Patents
System and method for displaying large images with reduced capacity buffer, file format conversion, user interface with zooming and panning, and broadcast of different images Download PDFInfo
- Publication number
- WO2002015162A2 WO2002015162A2 PCT/JP2001/005967 JP0105967W WO0215162A2 WO 2002015162 A2 WO2002015162 A2 WO 2002015162A2 JP 0105967 W JP0105967 W JP 0105967W WO 0215162 A2 WO0215162 A2 WO 0215162A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- image data
- image
- displaying
- data items
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/393—Arrangements for updating the contents of the bit-mapped memory
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/04—Changes in size, position or resolution of an image
- G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/02—Networking aspects
- G09G2370/027—Arrangements and methods specific for the display of internet documents
Definitions
- This present invention is related to a system and method for displaying huge images or integrated images of multiple data and, in particular, to a system for directly displaying a huge image without first storing the image data in the main memory in order to quickly display images.
- the present invention is related to a data file browsing device and program for browsing data files which contain text and image, and related to a data file conversion program for generating data files which can be browsed by the data file browsing device and program.
- unattended terminals located in museums, public offices and so forth, designed to provide a huge amount of information, may do so by integrating its information into a visual image so that visitors can easily search desired information.
- Quickly processing a huge amount of image data is becoming essential also in the fields of science and education.
- the human genome project that generates a huge amount of image data it is critical that scientists be able to easily and quickly locate information they need from the accumulated data.
- the ever expanding Internet environment creates new needs for techniques of processing and searching huge image data.
- the Internet E project in progress in the U.S.A., would provide an infrastructure on which an extremely large amount of data could flow, thereby allowing provision of remote services such as telemedicine, distance education and so forth.
- the image display system is composed of a hard disk drive (l), a main memory (3) for temporarily storing data read from the hard disk drive (l), a video buffer (5) for storing the data read from the main memory (3), a screen window to display images with the help of a graphics user interface (GU Graphic User Interface).
- a graphics user interface GUI User Interface
- the main memory is a bottle neck, preventing a prompt display of the image to the screen window.
- the bottle neck becomes worse as the amount of data increases.
- inventive system it is possible to provide information in an intuitive manner to human users.
- GUI based strongly upon human intuition so that the user of the huge image information can easily and quickly make use of the graphical information.
- the individual data items are integrated into a single huge image manifold, which is quickly reproduced by means of the system for displaying integrated image. Furthermore, a new GUI design enables users to easily search the graphical information.
- GUI Graphic User Interface
- One aspect of the present invention relates to a system for displaying huge images, which comprises ⁇ a storing means for storing image data which is divided either in row or in column into at least two blocks; and a first temporary storing means for temporarily storing data read from said storing means and, when filled up to a predetermined level, outputting all the data.
- the system for displaying huge images serves to improve the processing speed of huge images by bypassing the main memory (3) during the processing operation of image data.
- Huge image data stored in the hard disk drive (l) is divided in an appropriate manner into divided image data items, which are transferred directly to the video buffer (5) bypassing the main storage.
- the huge image data is transferred to the video buffer (5) unit-by-unit corresponding to a predetermined unit of division.
- the respective divided image data items are temporarily stored in a line buffer (9).
- the capacity of the line buffer (9) is sufficient as long as it is capable of storing one divided item of the huge image data while it is possible to quickly read and write data without delay like the main memory (3).
- the system for displaying huge images does not use a conventional GUI.
- an interface is provided in a manner strongly based upon the user's intuition so that the user of image data can easily and quickly search images displayed in the screen window (7).
- the system for displaying huge images correlates the location in the huge image as displayed in the screen window (7) with the location of a the mouse cursor in order to provide information at the huge image location without particular manipulation of a menu key.
- another input device such as a joystick can be used in place of a mouse.
- the system for displaying huge images provides a constant resolution regardless of the zoom ratio of the images, i.e., expansion or reduction.
- the huge image data is stored in the hard disk drive (l) in a hierarchical structure.
- Another aspect of the present invention relates to an integrated image data display system of a plurality of individual data items, which comprises ⁇ an image integration means for aggregating said plurality of individual data items into said single integrated image data; a storage device for storing said integrated image data by dividing in either row or column into at least two blocks; a first temporary storing means for temporarily storing data from said storing means device; and outputting all the data from said first temporary storing means when said first temporary storing means is filled with a predetermined amount of data.
- individual data items are integrated into a single huge image data item to generate a single integrated image data.
- the individual data items When the individual data items are integrated, the locations of the individual data items are determined in accordance with the extents to which the individual data items have relevance to each other.
- the individual data items may be image data, or text data such as html (hyper text markup language) data.
- the system for displaying integrated images displays huge image data as a single integrated image data item without delay. For this purpose, in contrast to reproducing images in a conventional imaging system, the images are reproduced bypassing the RAM which is the cause of signal delay.
- the system for displaying integrated images does not use a conventional GUI but provides an interface strongly based upon the human intuition. By this configuration, the user can easily and quickly search information in the integrated image. While the user can change the zoom ratio of the screen image through the GUI in accordance with the present invention, a constant resolution is maintained irrespective of the zoom ratio of the images selected by the user.
- the system for displaying integrated images provides an integrated image display service through a network.
- the server of the integrated image display service serves to generate integrated images and transfer necessary portions to a display device when requested by a client.
- the present invention is related to a data file browsing program for browsing data files that include text data and image data.
- the data file browsing program comprises the steps of : obtaining said data file as a plurality of image data items in a hierarchy with different image sizes according to zoom rat ms; generating virtual layers for the respective zoom ratios; setting the positional coordinates of said respective hierarchical image data items in each virtual layer,' receiving a zoom ratio from a user,' selecting hierarchical image data items located within the display area matching said zoo ratio,' and reading selected hierarchical image data items from a storage device and generating, as display data, image data portions which are actually to be displayed in the display area.
- the data which is browsed by the data file browsing program and system as described above can be generated by a data file conversion program comprising a step of converting the data file as- into a plurality of the image data items (the hierarchical image data items) with different image sizes by expansion or- compression,' and a step of generating virtual layers for the respective zoom ratios, calculating the positional coordinates of the respective hierarchical image data items in each virtual layer and writing the positional coordinates to a virtual coordinate file.
- another aspect of the present invention is related to a data file browsing system for browsing a data file that contains text and image data.
- the system comprises a storing mechanism for storing the data file as a plurality of image data items ( hierarchical image data items) with different image sizes by expansion or compression; a virtual coordinate generation mechanism for generating virtual layers for the respective zoom ratios and for setting the positional coordinates of the respective hierarchical image data items in each virtual layer,' zoom ratio switching mechanism for obtaining the zoom ratio as required by the user in order to determine the layer to be displayed; a display area detection mechanism for selecting all the hierarchical image data items located within the display area as required by the user," and a display data generation mechanism for reading the selected hierarchical image data items from a storage device and generating, as display data, image data portions which are actually displayed in the display area.
- Fig.1 is a block diagram showing an image display system in accordance with a conventional technique.
- Fig.2 is a block diagram showing the configuration of an embodiment of a huge image display system in accordance with the present embodiment.
- Fig.3a shows the process of transferring image data in accordance with the conventional imaging system.
- Fig.3b shows the process of transferring image data in accordance with the conventional imaging system.
- Fig.3c shows the process of transferring image data in accordance with the present embodiment.
- Fig.3d is an explanatory view for showing the method of reading out image data items in accordance with a conventional technique.
- Fig.3e is an explanatory view for showing the method of reading out image data items in accordance with the present invention.
- Fig.4 is a schematic representation showing the image reproducing method using compressed data in the imaging system as illustrated in Fig.3c.
- Fig.5a is a schematic representation showing an example of the GUI (graphic user interface) of the huge image displaying system in accordance with the present embodiment.
- Fig.5b is a schematic representation illustrating a rectangular area in which is not painted the graphical information during the movement of the image in the case of the conventional imaging system.
- Fig.5c is a schematic representation showing the process of the movement of the display data in the video buffer when the virtual screen is introduced in accordance with the present invention.
- Fig.6 is a schematic representation showing the hierarchical structure of the image data in accordance with the present embodiment while maintaining a constant resolution irrespective of the zoom ratio of images, i.e., expansion or reduction.
- Fig.7 is a schematic representation showing the process of providing image data associated with the screen image designated with the pointer by correlating the pointer location and the screen window with each other.
- Fig.8 is a flowchart showing the process for providing integrated images by integrating individual data items in the integrated image display system in accordance with the present embodiment.
- Fig.9 shows an embodiment of the integrated image displayi&g- system in accordance with the present invention.
- Fig.10 is a schematic representation showing an embodiment of the process in which an integrated image data item is generated by integrating individual data items having the same file dimensions.
- Fig.11 is a schematic representation showing an embodiment of the process in which an integrated image data item is generated by integrating individual data items having different file dimensions.
- Fig.12 is a flowchart showing the integrated image supply service through a network.
- Fig.13 is a block diagram showing an embodiment of a broadcast system which uses the integrated image display system in accordance with the present invention.
- Fig.14a is a schematic representation showing an embodiment of digital newspaper information service making use of the broadcast system in accordance with the present invention.
- Fig.15 is a block diagram showing the system for displaying images in accordance with an embodiment of the present embodiment.
- Fig.16 is an explanatory view for schematically showing the file conversion process in accordance with an embodiment of the present invention.
- Fig.17 is an explanatory view for schematically showing the process of displaying data files in accordance with an embodiment of the present invention.
- Fig.18 is a flowchart showing the procedure for converting data files in accordance with an embodiment of the present invention.
- Fig.19 is a flowchart showing the procedure for displaying data files in accordance with an embodiment of the present invention.
- Fig.20 is an explanatory view showing an exemplary modification in accordance with the present invention to explain how to insert data items into the image data.
- Fig.21 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the arrangement of Web pages in the form of an array in a plane.
- Fig.22 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the interface for manipulating a cursor.
- Fig.23 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the process of generating an album file.
- Fig.24 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the method of distributing books through the Internet.
- Fig.25 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the interface of handling motion pictures.
- Fig.26 is an explanatory view showing an exemplary modification in accordance with the present invention in which a map is handled as the image data.
- Fig.27 is an explanatory view showing an exemplary modification in accordance with the present invention for explaining the method of distributing images and motion pictures through the Internet.
- Fig.2 is a block diagram showing the configuration of an embodiment of the huge image display system in accordance with the present invention.
- a line buffer (9) is added. This line buffer is between the hard disk and the video buffer, with the main memory bypassed. Accordingly, delay due to the main memory (3) is eliminated, which helps to quickly reproduce huge images.
- the line buffer (9) is provided with a sufficient capacity to store one division of the image data. Unlike the slower main memory (3), the line buffer (9) receives data one unit at a time in the line buffer, thereby eliminating a delay that would have been caused if data had to be temporarily stored. As a result, it is possible to quickly reproduce images by dividing huge image data in a suitable manner and sequentially transferring the respective divided data, bypassing the main memory (3).
- Fig.3a shows the flow of image data from the hard disk drive (l) to the screen window (7) in the case that the image is stored without being divided.
- a portion (120a) of the huge image data (110) is to displayed in the screen window (7).
- neighboring data (120b and 120c) must be read too because the huge image data (110) is stored in the main memory (3) without divisions. Only the display data is read out from the main memory (3) and is stored in the video buffer (5). The use of this slower main memory causes substantial delays when reproducing images.
- Fig.3b shows the flow of data in the reproduction an image whose data is stored divided in accordance with the present technique.
- the huge image data (110) is composed of a number of subdivided blocks (Fij represents a subdivided block at the i-th row and the j-th column).
- the subdivided blocks (F22, F32, F23 and F33) associated with the display data (120), initially stored in the hard disk drive, are loaded into main memory. Then, only the display data (120) is read out from the main memory (3) and transferred to the video buffer (5).
- the present technique it is possible to reduce the amount of data that are stored in the main memory (3) but are not actually displayed in the screen. However, there is a substantial delay because the main memory (3) having a slower operation speed is still used.
- Fig.3c shows the flow of image data in accordance with the present embodiment.
- the main memory (3) is not used.
- the image data (110) stored in the hard disk drive (l) is read into the line buffer unit-by-unit up to the capacity of the line buffer, before subsequently being transferred to the video buffer.
- the image data (110) is stored in the hard disk drive (l) in the form of divided blocks.
- the image data (110) is divided into subdivided image data (113 and 115) either in row or in column while a block size (113a to 113d and 115a to 115d) corresponds to the capacity of the line buffer (9).
- the size of the subdivisions may be selected according to the user's needs. It is illustrated in Fig.3c that the image data (110) is divided into two
- the image data is generally composed of a number of pixels arranged in an array.
- the pixel values are arranged in the form of a plurality of lines LINE 1, LINE2, LINE3 • • • which are extending in the lateral direction respectively and aligned in the vertical direction in order to represent the image.
- LINE 1, LINE2, LINE3 • • • which are extending in the lateral direction respectively and aligned in the vertical direction in order to represent the image.
- the necessary data 120 is written to the disk as the image data portions 120j to 1201 that are certain portions of the respective LINEs. Accordingly, when the necessary data 120 is to be displayed, it is required to read the image data portions 120j to 1201 one after another which are dispersed in the disk.
- the portion of image data (I20j to 1201) needed are separately located so that it is inevitable to jump intervening sectors in order to read the next image data portion. Two sectors jumps occur in the case as illustrated in Fig.3d. A mechanical operation is necessary for each sector jump. Thus, a larger number of sector changes lead to a longer delay time to complete the read operation.
- the image data 110 is divided in either lateral or vertical direction so that it is possible to reduce the time delay by not having to read all the necessary data 120.
- the image data 110 is divided into a plurality of image data sections 113 and 115 in a direction perpendicular to the direction in which data would be read.
- Each of the image data sections 113 and 115 is obtained by partially reading out the respective LINEs which are arranged in the lateral direction (the reading direction) in the form of a band.
- the data in the form of a band has been written to the disk.
- a desired portion located in the middle of the image is also divided into a left half 113 composed of the data items 120d to 120f and a right half composed of the data items 120g to 120i.
- data items 120d"120f are written continuously onto one track in the disk. The same is true with data item 120g to 120i.
- the desired portion of the image 120 would be retrieved by sequentially reading the data items 120d to 120f, and then jumping to the beginning of data item 120g.
- Data items 120g to 120i are sequentially read. In this way, only a single sector change operation is required. As a result, the reading speed becomes higher compared with the conventional technique. A vertically longer portion of image would be read more quickly than a horizontally longer portion.
- the display data (120) is a portion of the image to be actually displayed in the screen window (7). As illustrated in the figure, the display data (120) partially overlaps the two vertically divided blocks (113 and 115). The first block (113) is read before the second block (115).
- the respective subdivisions (113a to 113d and 115a to 115d), each corresponding to the capacity of the line buffer (9), are sequentially read and transferred to the line buffer (9).
- the display data (120) is composed of eight subdivisions (113a to 113d and 115a to 115d), resulting in the loading of the line buffer (9) with a subdivision and its subsequent transfer to the video buffer (7) fe* eight times. Since, unlike the main memory RAM of the computer, the line buffer (9) has a smaller capacity but a faster operation speed ⁇ there is little delay associated with the process of transferring the subdivisions from the hard disk drive (l) to the video buffer (5).
- the video buffer (5) stores data in units of subdivisions. As a result, extra data other than the display data (120) are inevitably stored in the video buffer (5). Generally speaking, the capacity of the video buffer (5) exceeds that required for displaying a full screen of image. In fact, according to the present embodiment, smooth movement of images is provided by making use of the video buffer (5) which can store more than a full screen of data. An image is finally displayed on the screen window (7) by reading only the display data (120) from subdivisions stored in the video buffer.
- Fig.4 is a schematic representation showing a method for displaying an compressed image data.
- each subunit Since data is transferred in units of sub-image data, each subunit has to be separately generated.
- a raw image data is compressed by separately compressing the subdivisions, using a compressor (410).
- the sub-image data, now compressed, is stored in the hard disk drive (l) again. Since the compressed data itself cannot be used for the quick image reproduction it is first decompressed by means of a decompressor (430) and stored in the line buffer (9). Then, the image can be displayed in the same manner as illustrated in Fig.3c.
- the compression scheme may be JPEG, JPG, MPEG or the like.
- Fig.5a is a schematic representation of the GUI for the huge image displaying system in accordance with the present embodiment.
- GUIs are not suitable for displaying a huge image because of a delay stemming from the use of the computer's main memory.
- image scrolling on the screen is slow, thereby making a search effort a difficult job.
- the inventive imaging system provides a pointer (630) to easily "scroll” the huge image data displayed on the screen window (7) and to display relevant data related to the display image. Also, icons such as (+) and (-) are displayed on the screen to scale the displayed image.
- the image is scrolled in any directions through straightforward manipulation of the pointer (630).
- the location of the pointer (630) on the image is the location of the image data in the hard drive. Therefore, the movement of the pointer (630) is instantly interpreted to read the correct data from the hard disk drive, resulting in the corresponding scroll of the image.
- By dragging the pointer one can see the image panoramically.
- a benefit is that there is no momentary blank screen area as the image is moved, due to a delay in transferring the image data (110) from the hard disk drive.
- Fig.5b shows a blank area (650) at one side of the screen when the image is moved in the other direction according to a conventional imaging system. Specifically, when the image is moved to the right, the blank area appears due to delay in reading image data corresponding to that area from the hard disk drive. However, no such blank area appears according to the present invention and the image moves smoothly.
- Fig.5c shows the process of the movement of the display data (120) in the video buffer in when a virtual screen is introduced. It is possible to provide more natural movement of images on the screen by the use of the video buffer (5) to store more than a full screen of image data.
- the following table 1 compares a system in accordance with the present embodiment and a representative conventional image display system (sold under the trade names of "adobe photoshop"), when a satellite image of the North American Continent is displayed.
- the image display system in accordance with the present embodiment there is little delay because the main memory is not involved.
- the present invention provides images on the screen with a variety of zoom ratios without decreasing the resolution at high zoom ratios-r. For instance, the resolution of an image of the globe will have the same resolution as a zoomed-in image of a country.
- the present invention pre-stores images in a hierarchical structure.
- Fig.6 shows hierarchically stored images.
- the image data (710) on the upper layer is used to display the whole image.
- the image data (730) on the lower layers are used to display zoom-in portions of the image from successively higher layers. For example, when a screen image 710 currently displayed on the screen is to be zoomed in by a factor of 10, an image having data ten times greater than the zoomed-in portion is required for maintaining the same resolution.
- the image data (720) on the lower layer is used in this case.
- a multi-level hierarchical structure could be designed to provide various zoom ratios in the GUI. For instance, if the GUI provides 10 zoom ratios, the image data (110) has to be of a ten-layered hierarchical structure. Respective zoomed images can also be generated for desired sizes of data by interpolation and decimation.
- the present invention enables a smooth transition from one zoom ratio to another since data can be quickly read from the hard disk drive (l) as describe above.
- Fig.7 shows how the pointer is synchronized with the image data.
- the GUI of the system in accordance with the present embodiment serves to provide the user with references related to the locations on the screen image specified by the pointer (630) without particular manipulation of a menu key by synchronizing the pointer (630) and the screen window (7) with each other.
- the location of the image corresponding to the location of the pointer (630) is indicated by (x).
- the pointer (630) moves over the screen window the data of the screen image specified by said pointer is tracked.
- Information about the location of the screen image is stored as information data (140) in the hard disk drive.
- the information data (140) includes a field containing reference data and another field of image co-ordinates for that data indicative of the location of the screen image and a field containing the data.
- the information data is stored in the main memory and is displayed in the screen window (7) in response to a predetermined event such as keeping the pointer (630) in the same position.
- the huge image display system as illustrated in Fig.2 can be extended through a network. Namely, a huge image information service can be provided by connecting a database system of huge images to users at their computers through a network. Since the inventive imaging system makes use ⁇ f- the line buffer (9) without a need for a main memory, it can be practiced even with a low-cost personal computer. In addition, because, only those data to be displayed on the screen (corresponding to the size of the line buffer) is read and transmitted to the user, it is particularly suitable for transmission of huge image data through a network.
- Fig.8 is a flowchart showing the process of providing an integrated image by combining individual data items in accordance with the present embodiment. Specifically, individual data items are first collected at the collecting step (SIOO). The individual data items generally will relate to each other.
- Examples of the individual data items include pages of a newspaper, linked web text, satellite information as received from a satellite and so forth.
- integration is applicable to individual data items having no particular association with each other.
- the individual data items can be any data types including image and text.
- Step (S100) includes three substeps (S101, S103 and S105). Individual data items are input to the integrated image display system in the substep (S101).
- the locations of the individual data items in the integrated image have to be carefully selected in order to provide users with a convenient interface environment to control the image.
- One criterion will be to take advantage of human intuition. For instance, when pages of a newspaper is to be integrated, it seems desirable to arrange the pages according to the page numbers because, in most cases, a reader of newspaper would read from the first page to the last page.
- the linking structure of the web pages could be used because a web user is normally led to a specific page through one or more links.
- the locations of the individual data items in an integrated image are determined depending on how the individual data items are relevant to each other in the step (S103).
- a single image data item can be constructed not only with individual data items having the same file dimensions but also with individual data items having different file dimensions. For example, web pages linked to each other could have different file dimensions.
- Fig.10 and Fig.11 show the integration of individual data items of the same file dimensions into a single file and individual data items of different file dimensions.
- the individual data items collected at step (S100) are converted to data indicative of a single integrated image in the integrated image generation step (S110), which includes two substeps (Sill and S113).
- Step (Sll) the individual data items from a variety of formats such as GIF, TIFF or HTML etc, are converted to a file format suitable for integration. Converted data items may contain text data, image data or other types. These converted data items are used to generate a single integrated image at step (SI 13).
- an integrated image can be quickly reproduced, as described in details below.
- the integrated image display system is composed of a hard disk drive (l) for storing individual image data items and integrated image data, an image data combining device (ll) for reading the individual data items and generating an integrated image data, a RAM (3) for temporarily storing the image data, a video buffer (5) for storing the image data before displaying the image data, and a screen window (7) for displaying the image data through a GUI.
- a hard disk drive (l) for storing individual image data items and integrated image data
- an image data combining device (ll) for reading the individual data items and generating an integrated image data
- a RAM (3) for temporarily storing the image data
- a video buffer (5) for storing the image data before displaying the image data
- a screen window (7) for displaying the image data through a GUI.
- the line buffer (9) provides a path via which the image data is directly transferred to the video buffer (5).
- the integrated image data is divided in a predetermined way.
- the integrated image data (110) is read on a unit-by-unit basis, wherein the unit corresponds to the capacity of the line buffer (9), before being transferred to the video buffer (5) through the line buffer (9).
- an integrated image data (110) is divided into subdivisions either in row or in column and the subdivisions are read sequentially.
- the line buffer(9) stores one image subdivision.
- Fig.10 shows an embodiment in which an integrated image data (110) is generated by integrating individual data items (401, 403, 405 and 407) having the same file dimensions.
- the integrated image data (110) is divided either vertically or horizontally. As illustrated in Fig.10, the integrated image data
- Fig.10 shows an integrated image data with four individual data items combined. The image is then divided into four subdivisions (llOa, 110b, 110c and llOd). The display data (120), extending across three subdivisions (110b, 110c and llOd), is displayed in accordance with the method as illustrated in Fig.3c.
- Fig.11 shows an embodiment in which an integrated image data (110) is generated by integrating individual data items (501, 503, 505 and 507) having different file dimensions.
- the integrated image data (110) can be divided into subdivided blocks at the boundaries between the individual data items (501, 503, 505 and 507). But it dose not have to be that way.
- the Figure shows a case where the single integrated image data (110) is divided into subdivided blocks at the boundaries.
- the individual data items (501, 503, 505 and 507) are divided into subdivisions (501a, 501b, 503a, 503b, 505a, 505b, 507a and 507b).
- the display data (120) is reproduced as a visual image in accordance with the process as illustrated in Fig.3c.
- web pages of different dimensions can be combined.
- the image data in the predetermined location cannot be obtained without parsing the entirety of the HTML page.
- the amount of HTML data to be loaded is reduced when making use of a HTML page that is partially parsed.
- users can easily and conveniently search desired data from an integrated image through the graphic interface in accordance with the present invention.
- the most significant characteristics introduced by the present invention reside in that it is possible to manipulate an integrated image with only a pointer (630) for pointing a location in the screen without particular manipulation of a menu key.
- a user is provided with a constant resolution of integrated images irrespective of the zoom ratio of images selected by the user of graphical information.
- Fig.12 is a schematic representation of the integrated image supply service through a network.
- the integrated image server (800) in response to a request of a client (830) for integrated image data, the integrated image server (800) reads integrated image data from the integrated image database (810) and transmits it to the client (830) through a network (820).
- the network (820) includes the Internet.
- the client computer (830) would make use of the line buffer (9) rather than the computer's main memory (see Fig.9), a low cost personal computer can be used. There is no transmission problem either because only data to be displayed would be transferred (corresponding to the size of the line buffer (9) as illustrated in Fig.2) from the integrated image database (810) to the user. Similarly, the present invention can be effectively applied to provide a web service.
- Fig. 11 is a block diagram showing an embodiment of a broadcast system utilizing the integrated image display system (903) in accordance with the present invention.
- Image data of a newspaper is generated by the use of a scanner (901) and is stored in a hard disk drive (l) of the very high speed imaging system (903), (see Fig.13).
- the broadcast presenter can provide integrated images through the interface provided by the integrated image system (903) and also through any other interface such as a touch screen (905) connected to the integrated image system (903).
- the integrated image display system (903) or the touch screen (905) is connected to a beam projector (907) that projects a portion of the image onto a screen.
- the screen image is taken by a camera (909) and provided to the audience through a broadcast system (911).
- the system as illustrated in Fig.13 constitutes a combination of a conventional broadcast system and the integrated image displaying system (903) in accordance with the present invention by means of the beam projector (907) and the camera (909).
- the integrated image displaying system (903) can be connected to the broadcast system (911) through a network.
- Typical examples of broadcast services making use of the integrated image display system as illustrated in Fig.13 include weather forecast and newspaper summaries, which will be explained respectively in details.
- digital data containing newspaper information is generated, for example, by scanning the respective pages of a daily newspaper or by using existing digital data on the newspaper's web pages.
- image data items would have the same page dimensions so that an integrated image data can be generated in accordance with the method explained in reference to Fig.10 above.
- Fig.14a shows the screen image integrated from the individual image data items.
- the integrated image data items of respective daily newspapers are shown in the respective inner windows (201a to 2011) and are in turn integrated as an integrated image data (201).
- a news report can select one of the respective inner windows (201a to 2011) with a pointer (630) as illustrated in Fig.5a or with a touch screen (905).
- Fig.14b shows an enlarged screen image (201) from an inner windows
- searching is implemented by means of the user interface provided by the integrated image displaying system (903). Namely, zooming and scrolling functions are performed by a straightforward manipulation of the pointer (630) as explained heretofore with reference to Fig.5a or through manipulation of the touch screen (905) as explained heretofore with reference to Fig.13.
- Fig.14c shows the screen image (201) that is a further zoomed view of the same newspaper page. And yet, the resolution of the screen image remains the same.
- the screen image displayed in the inner window (201a) is then projected to a screen, which is captured by the camera (909) for broadcasting via the data transmission system (911).
- the zooming and scrolling functions are performed in the same area of the screen image (201) in response to selection by the broadcast presenter so that it is easy to advance a broadcast program without manipulating the camera and it is possible to provide the audience with a newspaper information service with more natural screen images.
- the touch screen (905) or the integrated image display system (903) can be connected directly to the data transmission system (911). It is also possible to provide another service by processing the newspaper information in different forms in accordance with the present embodiment. For example, it is possible to provide a newspaper information service by combining newspaper information given in HTML or PDF into a single integrated screen.
- the broadcast system as illustrated in Fig.13 is applicable to a weather forecast service.
- Conventional weather forecast services provide weather information in charts or video by processing data transmitted from satellites.
- the integrated image display system (903) generates image data and quickly reproduces it so that it is possible to provide video and fine weather images as naturally as possible.
- the video and still images of weather reproduced by the integrated image displaying system (903) are sent directly to the broadcast system (911) to combine with the screen image of the weather caster generated by a camera.
- the image displaying system in accordance with the present embodiment is composed of an input/output device (106) for performing input/output operations of data files," a hard disk drive (102) for storing data; a memory (103) for temporarily storing data; a display device (105) such as a CRT; a liquid crystal panel; console devices such as a touch panel, a mouse, a keyboard and so forth; a CPU (101) for performing a variety of arithmetic operations with which the respective functions are implemented,' a line buffer (108), and a video buffer (109).
- an input/output device (106) for performing input/output operations of data files
- a hard disk drive (102) for storing data
- a memory (103) for temporarily storing data
- a display device (105) such as a CRT
- a liquid crystal panel such as a liquid crystal panel
- console devices such as a touch panel, a mouse, a keyboard and so forth
- a CPU (101) for performing a variety of arithmetic operations with which the respective
- the input/output device (106) is an interface through which the present system performs input/output operations of files and data, e.g., a floppy disk drive, a CD-ROM disk drive, a DVD-ROM disk drive, and so forth.
- HTML files are input/output as a data sources for conversion.
- the hard disk drive (102) serves to store computer programs such as a file conversion program, a program for displaying images and the like, HTML files read from the input/output device (106), and files generated as a result of the execution of system services.
- the memory (103) is used in the present embodiment to store image files as read out to be displayed and to output data to be displayed corresponding to the display area to the display device (105).
- the console device (104) may be a touch panel, a mouse, and a keyboard through which commands are input to convert a file and display images.
- the console device (104) is a touch panel equipped with a liquid crystal panel in the display device (105).
- the display device (105) is a liquid crystal panel through which the file conversion program and the character display are controlled.
- the front surface of the display device (105) is implemented with a transparent touch panel (the console device (104)) with which several commands can be input by touching the indication symbols on the display device (105).
- the CPU (101) serves to execute several functions by loading executable programs from the input/output device (106), the hard disk drive (102), and memory (103).
- the program consists of a tag extraction unit (ll), a tag coordinate calculation unit (12), an image conversion unit, a size changing unit (14), a display area detection unit (15), an zoom ratio switching unit (16), a display data generation unit (17), and a file input/output control unit (18).
- the tag extraction unit (ll) serves to parse the text contents of an HTML file (Hi) in order to extract tag data containing modifier information.
- the tag coordinate calculation unit (12) serves to convert the location of the extracted tag data into screen coordinates and write the result to a tag coordinate file through the file input/output control unit (18).
- the image conversion unit (13) serves to convert data files such as HTML files to image files such as JPEG files, and bit map files.
- the size changing unit (14) serves to generate a set of image data items (image data items in hierarchies) corresponding to each'of the respective zoom ratios by expansion or compression. Meanwhile, in accordance with the present embodiment, the size changing unit (14) also serves to calculate the locations of the image data items corresponding to the respective zoom ratios in virtual coordinates and write the result to the virtual coordinate file.
- the display area detection unit (15) serves to receive inputs from the user of the console device (104) such as the touch panel to determine which virtual layer to be displayed, and to calculate the range of the virtual layer to be displayed on the screen.
- the zoom ratio switching unit (16) serves to detect the change in the zoom ratio in response to the user's inputs, and switches the zoom ratio of the image data being displayed by changing the virtual layer.
- the display data generation unit (17) serves to calculate the range of virtual coordinates to be displayed on the display device (105) to determine which image data items in hierarchies correspond to the range to be displayed, load the image data items into the line buffer (108) of the memory (103), and generate display data (1100) by transferring the portion of the image data that is actually displayed in the screen to the video buffer (109). When pluralities of image portions are to be displayed, they are reassembled in accordance with the virtual coordinate file (M2) to generate a single display data (1100).
- the file input/output control unit (18) serves to read and write the files stored in the hard disk drive (102) as the user requests each units.
- the line buffer (108) serves to temporarily store the image data on each line as read from the hard disk drive (102).
- this line buffer (108) is formed by allocating a certain region in memory (103).
- the line buffer (108) can have a variable size depending upon the size of the image data on each line to be read out at a time.
- a plurality of line buffers (108) can be allocated in memory (103). Also, the line buffer (108) may be implemented separately from main memory (103).
- the video buffer (109) is provided for temporarily storing an image to be displayed on the display device (105).
- a single image is constructed in the video buffer (109) by assembling image data items on each lines for output to the display device (105). While the video buffer (109) is provided separately from the display device (105) in accordance with the present embodiment, the video buffer (109) can be implemented within the display device (105) or as a certain region in memory (108).
- Fig.16 is an explanatory view for schematically showing the file conversion process.
- HTML text file with tags
- the HTML file (Hi) used as a data source of conversion, is parsed in order to generate hierarchical image data items (II to 13) having a plurality of image dimensions.
- the hierarchical image data items (II to 13) are converted into bits (pixels) to form part of an image that include the text data (such as letters, symbols and the like contained in the HTML file (Hi)) as well as the image files included in the text through hyper-links.
- the linking information included in the written text is extracted.
- the extracted URLs and their coordinates and ranges in the hierarchical image data items are then stored in the tag coordinate file(Ml).
- the location and the range of each link are converted to a location in the image data and stored in the tag coordinate file (Ml) along with the linked URLs according to the ranges of the mappings.
- the locations of the image data items as converted in the virtual coordinates, which are constructed by combining the respective hierarchical image data items are stored in the virtual coordinate file (M2).
- the sets of image data items (II to 13) at various zoom ratios, the tag coordinate file (Ml), and the virtual coordinate file (M2) are generated in this manner and stored in the hard disk drive (102).
- the image files corresponding to each size are divided in the direction perpendicular to the direction of reading out the constituent pixel data and stored as a plurality of divided image data items in the hard disk drive (102).
- the image files are read out from the hard disk drive (102), their respective divided image data items are read out.
- the set of image data items as converted by the file conversion process as explained above can be browsed by the image display process as explained in the following description.
- the data files have already been converted into their respective hierarchical image data items stored in the hard disk drive (102).
- the data file set to be displayed by the user is selected in the console device (104).
- the selected virtual coordinate file (M2) and the tag coordinate file (Ml) corresponding to the data file are read from the HDD(102).
- a hierarchical image data item set having the lowest zoom ratio 6.25% is put in the virtual layer (L3) and displayed for viewing in the display device (105).
- the system then waits for user input to change the zoom ratio, to move the range to be displayed and so forth.
- the display area detection unit (15) serves to detect user input to the console device (104). If the zoom ratio (the reduction ratio or the expansion ratio) needs to be changed, the detection unit determines the virtual layer and the virtual coordinates on the virtual layer. For example, when the user requires a 125% zoom, the virtual layer (L2) is selected as illustrated in Fig.17(a). Then, the hierarchical image data items included in the range to be displayed as required by the user is selected on the virtual layer. For example, when the user requires the display range (100), the hierarchical image data items (131 to 134) relating to the display range (100) are selected. The hierarchical image data items (131 to 134) as selected are temporarily stored in the memory (103).
- the image portions actually falling within the display range (100) is generated as the display data.
- the display data generation unit (17) serves to clip the image portions to be displayed from the hierarchical image data items (131 to 134) stored in the memory (103) and to reassemble the image portions based on the virtual coordinate file (M2) by means of the CPU (101) in order to generate the display data (1100).
- the linking information as described in the tag coordinate file (Ml) is obtained together with the location of the links (the mapping information) in the image followed by introducing the linking function in the display data (1100).
- the display data (1100) as generated is output to the display device (105) to display the display range (100) as an image on the screen.
- the display data (1100) as displayed is implemented with links so that it is possible to jump to the linked files and the linked web sites by touching the touch panel (104) on the display device (105).
- the display area detection unit (15) waits for new inputs from the user to manipulate the image.
- the zoom ratio switching unit (16) switches to the virtual layer (L3) and reads the hierarchical image data items 135 to 142 matching the display range (100) from the memory (103) with reference to the virtual coordinate file (M2).
- the zoom ratio switching unit reads the portion of the image data that is actually displayed to screen by CPU (101) operation in order to generate the display data (1100) for the display device (105).
- the data file displaying system as described above can be implemented in a general purpose computer such as a personal computer by running a data file conversion program and a display program as explained in the following description. These programs may be stored in floppy disks, a CD-ROM, a DVD-ROM or another computer-readable medium, and installed in a general purpose computer from these mediums.
- Fig.18 is a flowchart showing the procedure for the data file conversion program in accordance with the present embodiment.
- the HTML data file (Hi) to be converted is read from the input/output device (106), the hard disk drive (102) and the like to the CPU (101). Meanwhile, if the file size of the HTML file (Hi) is too large to be transferred to the CPU (101) at one time, the HTML file (Hi) is saved in memory (103) or the hard disk drive (102) functioning as virtual memory and, when required, transferred to the CPU (101).
- the tag extraction unit (ll) serves to extract the tag information from the HTML file (Hi). After the tag information (the linking information) is extracted, the positional coordinate where the each tag information items is displayed is calculated by means of the tag coordinate calculation unit (12) in order to generate the mapping information. Then, the coordinates information including tag mapping information is written to the tag coordinate file (Ml).
- the HTML file (Hi) is then converted to an image file such as a JPEG file, or a bit map file (SI 105), which is then reduced or expanded to have different image sizes in order to generate a plurality of hierarchical image data items (S1106).
- the generated hierarchical image data items are stored in the hard disk drive (102) (SI 107).
- the hierarchical image data items are divided in the direction perpendicular to the direction of reading out the constituent pixel data and stored as a plurality of divided image data items.
- Virtual layers are then generated for each zoom ratio followed by calculating the positional coordinates of their respective hierarchical image data items in each virtual layer (SI 108) and writing the calculation results to the virtual coordinate file (S1109).
- Fig.19 is a flowchart showing the procedure of the data file browsing program in accordance with the present embodiment. It is assumed that the user specifies the set of data files for display.
- the display area detection unit (15) serves to detect user inputs from the console device (104) and changes to the zoom ratio of the images to be displayed (S1201).
- the zoom ratio switching unit (16) serves to determine the virtual layer to be displayed (S1202). Also, at this time, the range to be displayed is determined (S1203) while reading the virtual coordinate file (M2) (S1204) and calculating the coordinates corresponding to the display range.
- the hierarchical image data corresponding to the display range is divided and sequentially read from the hard disk drive (102). In this case, the hierarchical image data items are read out by sequentially reading out the respective divided image data items unit-by-unit corresponding to the length of the LINEs.
- the data as read is accumulated in the line buffer (108) (S1206).
- the line buffer (108) is filled to a predetermined size, the divided data item is transferred to the video buffer (109).
- the tag coordinate file (Ml) is read (S1207).
- the image portions corresponding to the display area are read out from the video buffer (109) and assembled with reference to the virtual coordinate file to generate the display data by adding modifier information such as link information thereto.
- the display data as generated is output to the display device (105) and displayed on the display device (105), followed by terminating the process.
- steps (S1201 to S1209) are repeated to determine the hierarchical image data items corresponding to the display area with reference to the virtual coordinate file and switch the display image.
- the data file display program and the data file conversion program are described as separate programs in the case of the present embodiment, the present invention is not limited to this configuration and it is possible to design both programs in communication with each other, e.g., by designing the data file conversion program as a subroutine of the display program. For example, when another data file to be displayed is added, the data file conversion program is automatically started in order to generate hierarchical image data items which can be displayed.
- a single data file is converted into data items of a plurality of different image sizes that are arranged into virtual layers, which are selectively displayed when the zoom ratio of the image to be displayed is changed to avoid lowering the resolution of the image when changing the zoom ratio.
- the image data is stored beforehand in memory (103) so that it is possible to quickly display the image even when the display range is moved, resulting in improved performance.
- the console device (104) is implemented as a touch panel attached to the display device (105) so that it is possible for the user to perform movement, zoom and other handling operations of the displayed file by using a finger, a touch pen and the like. The user can therefore understand the operations intuitively substantially improving operability.
- the image data 110 which is displayed in the system above can be implemented with the inserted data item Objl as illustrated in Fig.20.
- the divided image data items are provided in the format in accordance with the present embodiment.
- the location and the size of the inserted data item in the image can be described as (x, y), (x, y and r) and so forth so that coordinate information calculation on the image is easy.
- the object Objl can be a rectangular object as illustrated in Fig.20(a), a circular object (object obj2) or an arbitrary object (object obj3), as expressed as a combination of polygons as in Fig.20(c) and Fig.2 ⁇ (d).
- the location of object obj2 can be specified by the coordinates (X,Y) and the radius (r) from the center position in order to determine the location and the profile (size).
- these inserted data items Objl to Obj3 are associated with the coordinates (X,Y) in relation to the origin (0,0) of the image data (110) and with the coordinates (X ⁇ Y in relation to the origin ( ⁇ ',0') of the display screen window (7).
- the zoom ratio of the image data is changed by accessing the image data at a different virtual layer, it is possible to calculate the location of the inserted data item Objl.for that layer.
- the zoom ratio of the image data is set to 1/2, it is easy to calculate the addresses by halving the respective values of the coordinates (X,Y) in relation to the origin (0,0) of the image data (110).
- the inserted data item is included in the display screen window by calculating the coordinates in relation to the display screen window (7). For example, the entirety of the inserted data item is not displayed in the case where the coordinates (X',Y of the display screen window (7) are negative values. Additionally, only the portion of the inserted data item with positive coordinates is displayed on screen. If the inserted data item is a motion picture that is completely outside the screen display area, the processing operation of reading out the motion picture is turned off. (EXEMPLARY MODIFICATION 2) By the use of the present invention, it is possible to convert Web pages as distributed via the Internet to image items and display the image items as a single page as illustrated in Fig.21.
- the Web pages hierarchically linked by the embedded tag data In the case of the present invention, the image items having been converted are arranged in accordance with the hierarchical structure. As a result, it is possible to move one Web page to another Web page by scrolling and to browse Web pages without awareness of the hierarchical structure.
- bookmark function is provided for the purpose of maintaining a history of browsed contents.
- bookmarks BM1 and BM2 are set in the locations selected in pages PI and P2 which have already been browsed.
- a bookmark can be recorded as tag data in the image data (110) or within a separate file provided for recording bookmarks.
- an index image as illustrated in Fig.21 is generated and stored in a hard disk drive by converting HTML files provided through the Internet into image data files and physically or logically combining the converted files.
- the index file as generated is read out from the hard disk drive and displayed on a display device as indices.
- the respective images constituting the indices are given links to URLs of the Internet where the original data items are provided.
- the information required for generating the respective links are described as tag data in a tag coordinate file which is provided separated from the index file.
- the tag coordinate file is accessed in order to jump to the corresponding Web page in accordance with the tag data described in the tag coordinate file.
- the corresponding Web page as reached is displayed as a usual HTML file by a conventional browser program.
- a bookmark function is provided for the purpose of recording a Web page which has been browsed. Namely, a bookmark is set to the reduced image of a Web page which have been already browsed. A bookmark can be recorded as tag data in the image data or within a separate file provided for recording bookmarks.
- the index file is stored in a hard disk drive located inside of the system, the index file can be quickly read out and displayed.
- an image can be displayed with a fixed cursor (155) as illustrated in Fig.22.
- the location of the fixed cursor (155) is located at the center position of the screen window (7) and shall not be moved from the center position of the screen window (7) even when the image is scrolled on the display screen window.
- the image can be scrolled with a ten key pad (104a) as illustrated in Fig.22a.
- the fixed cursor (155) is not moved in relation to the display screen window but the image is scrolled in the direction opposite to the direction of the pressed key.
- the image data (110) is moved to the bottom right direction so that the object is moved to the location of the fixed cursor (155) as illustrated in Fig.22(b).
- the coordinates of the object obj4 are obtained on the display screen by clicking or pressing the return key in order to perform a particular operation (for example the expansion, the reduction and the like) at the coordinates.
- a particular operation for example the expansion, the reduction and the like
- the object on a huge image can be easily located.
- a mouse is used to select a region (154) of an arbitrary portion (the object obj ⁇ in the figure) of the image displayed by connecting the divided image data items FI to F4, and by clipping the select portion from the set of divided image data items.
- the image portion as clipped is temporarily stored in the memory (103) and then pasted to an album file M3.
- the image data ig l including the object obj ⁇ is attached to the album file M3 thus generated.
- the image data imgl is not actually stored; only the file names of the divided image data items FI to F4 associated with the image data imgl is recorded in a layout file (a record file) together with the coordinates (X,Y) specifying the region of the image in the divided image data items and image area specifying data relating to the profile of the image.
- album file M3 as stored is displayed again, it is displayed by reading ⁇ U a layout file M4 and the divided image data items FI to F4 as included.
- Desired image portions can therefore be collected into a single file. Also, when the album file M3 is stored, only the file names of the divided image data items FI to F4 rather than the data items themselves are stored in the layout file M4 so that it is possible to dispense with storing redundant image data having the same contents and thereby reduce the amount of data. (EXEMPLARY MODIFICATION 5)
- a book can be distributed as image data through the Internet.
- the image data is distributed to a personal computer (152) of the user from a distribution server (151) which is connected to the Internet and equipped with the hard disk drive (l) that contains the divided image data items.
- the distribution server (151) serves to transmit data corresponding to the zoom ratio requested by the user through the distribution network.
- the image data is generated by converting the respective pages of a book to image data items and combining them into a flat image so that a previous page can be viewed directly by scrolling. Also, the respective hierarchical image data items are given the coordinates corresponding to the respective pages and the respective line numbers of the book.
- the distribution server serves to transmit the expanded data of the desired page in response to the request of the user.
- a television schedule table is distributed as image img2 in which the ordinate shows broadcasting times and the abscissa shows channel numbers.
- the distribution server (151) serves to transmit information for the channel and time selected by the user by clicking a position in the television schedule table img2 displayed in the screen window (7).
- the title of the program as selected is specified with reference to the channel and time in the distribution server (151), which then transmits the detailed information about the program and the corresponding motion picture data as the content thereof to the personal computer (152) as the required content.
- the books distributed may also include commodity catalogs such as those of telemarketing services and the like besides television schedule tables.
- the coordinate information of the divided image data items are associated with the page numbers and the screen coordinates of the page numbers of a commodity catalog so that a page number and the associated coordinates are transmitted to the distribution server (151) in order to specify a particular commodity in the catalog with reference to the page number and the associated coordinates
- the distribution server (151) serves to transmit the detailed information about the commodity as specified to the personal computer (152) and to manage delivery thereof. (EXEMPLARY MODIFICATION 6)
- a motion picture can be presented by sequentially displaying a plurality of divided image data items. Namely, as illustrated in Fig.25(a), an animation is displayed by sequentially displaying divided image data items containing slightly different images and belonging to one of the divided image data groups MOVl to MOV3 each composed of divided image data items having the same data size.
- the order of displaying the divided image data groups MOVl to MOV3 can be specified or modified by manipulating a pointing device (motion picture handling means) such as a touch panel (104c), a mouse (I04d).
- a pointing device motion picture handling means
- a touch panel 104c
- a mouse I04d
- forwarding or reversing the motion picture is possible by laterally moving a finger on the touch panel (104c) while the speed of the animation can be changed by changing the speed of finger movement.
- the layer of the image is switched in order to change the zoom ratio of the animation as displayed.
- image data can apply to the display of large area as in a map or in a celestial image as illustrated in Fig.26. Also in this case, an arbitrary area can be enlarged by generating hierarchical image data items corresponding to different zoom ratios.
- the sea area (al) is filled with a predetermined dummy data (for example blue). It is thereby possible to dispense with unnecessary image data and therefore reduce the size of data. (EXEMPLARY MODIFICATION 8)
- image data can be sold through the Internet.
- image data is distributed to a personal computers (152) of the user by a distribution server (151) which is connected to the Internet and equipped with the hard disk drive (l) containing the image data (the divided image data items, the hierarchical image data items and so forth).
- a payment server (153) is connected to the Internet in order to receive payment from the user through the Internet.
- a free image Ll at the lowest zoom ratio is provided as an index for the user to browse on the screen window (7) of his personal computer (152).
- the image Ll shows a table of images with which the users can look through the collection of still images and the motion pictures in view.
- a user selects a desired image(s) and sends a request for it to the distribution server (151).
- the distribution server (151) When receiving the request, the distribution server (151) sends a request to the payment server (153) in order to activate the payment procedure. In response to this, the payment server (153) sends a request for payment to the user. The user makes payment, for example, by transmitting his credit card number and the like to the payment server (153).
- the payment server(l53) informs the distribution server (151) that the payment transaction is complete.
- the distribution server then (151) transmits to the user an image L2 or L3 at the higher zoom ratio.
- the integrated image displaying system of the present invention it is possible for users to easily search information by arranging related data items into a single integrated image. Also, the user can conveniently search information inside ⁇ f-the integrated image by having a GUI design based on human intuition.
- the integrated image system of the present invention it is possible to quickly reproduce integrated images, to provide natural movement of images and to provide the same resolution even when the zoom ratio of the screen image is increased in the act of searching for detailed information.
- the integrated image system in accordance with the present invention is applicable to a broadcast service, making it possible to provide newspaper information and satellite information in a natural manner. Also, utilizing the data file browsing program, the data file conversion program, and the data file browsing device of the present invention, one can maintain the same resolution regardless of zoom ratio.
- this invention represents a revolutionary approach to browsing large amount of contents without sacrificing the ease of use.
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CA002422780A CA2422780A1 (en) | 2000-08-17 | 2001-07-10 | System and method for displaying large images with reduced capacity buffer, file format conversion, user interface with zooming and panning, and broadcast of different images |
JP2002520206A JP4848482B2 (en) | 2000-08-17 | 2001-07-10 | Video display system and video display method |
US10/110,680 US7453479B2 (en) | 2000-08-17 | 2001-07-10 | Image data displaying system and method |
AU2001269494A AU2001269494A1 (en) | 2000-08-17 | 2001-07-10 | System and method for displaying large images with reduced capacity buffer, file format conversion, user interface with zooming and panning, and broadcast of different images |
Applications Claiming Priority (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR2000/47345 | 2000-08-17 | ||
KR1020000047344A KR20010000208A (en) | 2000-08-17 | 2000-08-17 | Method and system for large image display |
KR20000047345 | 2000-08-17 | ||
KR2000/47344 | 2000-08-17 | ||
KR1020000059645A KR20010035015A (en) | 2000-08-17 | 2000-10-11 | Method and system for displaying integrated image of multiple data |
KR2000/59645 | 2000-10-11 | ||
JP2001052828 | 2001-02-27 | ||
JP2001-52828 | 2001-02-27 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2002015162A2 true WO2002015162A2 (en) | 2002-02-21 |
WO2002015162A3 WO2002015162A3 (en) | 2002-09-06 |
Family
ID=27482070
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2001/005967 WO2002015162A2 (en) | 2000-08-17 | 2001-07-10 | System and method for displaying large images with reduced capacity buffer, file format conversion, user interface with zooming and panning, and broadcast of different images |
Country Status (5)
Country | Link |
---|---|
US (1) | US7453479B2 (en) |
KR (4) | KR100531676B1 (en) |
AU (1) | AU2001269494A1 (en) |
CA (1) | CA2422780A1 (en) |
WO (1) | WO2002015162A2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1343132A1 (en) * | 2002-03-08 | 2003-09-10 | Fujitsu Siemens Computers, Inc. | Data viewer |
EP2058730A3 (en) * | 2007-11-07 | 2011-11-23 | Sony Corporation | Image processing apparatus, image processing method, and image processing program |
Families Citing this family (67)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040205505A1 (en) * | 2002-01-08 | 2004-10-14 | Vans A. Marie | Preprocessing and preloading digital display pages in multiple page digital data files |
US20040008185A1 (en) * | 2002-03-29 | 2004-01-15 | Mitac International Corp. | Data processing device, presentation device, and projection method for presentation |
JP4278918B2 (en) * | 2002-04-19 | 2009-06-17 | 富士通株式会社 | Image data processing apparatus and method |
JP2003331047A (en) * | 2002-05-16 | 2003-11-21 | Canon Inc | System, apparatus and method for processing information, program for executing the method by computer, and storage medium stored with the program computer- readably |
WO2003098578A1 (en) | 2002-05-17 | 2003-11-27 | Xanavi Informatics Corporation | Map data product, map data processing program product, map data processing method, and map data processing device |
KR100453516B1 (en) * | 2002-05-21 | 2004-10-20 | 주식회사 이노티브 | Transferring system for huge and high quality images on network and method thereof |
US7054496B2 (en) * | 2002-06-27 | 2006-05-30 | Seiko Epson Corporation | Method and apparatus for displaying arbitrarily magnified high resolution images using compressed domain processing |
JP4300767B2 (en) * | 2002-08-05 | 2009-07-22 | ソニー株式会社 | Guide system, content server, portable device, information processing method, information processing program, and storage medium |
US7839860B2 (en) | 2003-05-01 | 2010-11-23 | Genesis Microchip Inc. | Packet based video display interface |
US8204076B2 (en) | 2003-05-01 | 2012-06-19 | Genesis Microchip Inc. | Compact packet based multimedia interface |
US8059673B2 (en) | 2003-05-01 | 2011-11-15 | Genesis Microchip Inc. | Dynamic resource re-allocation in a packet based video display interface |
US8068485B2 (en) | 2003-05-01 | 2011-11-29 | Genesis Microchip Inc. | Multimedia interface |
US7405719B2 (en) * | 2003-05-01 | 2008-07-29 | Genesis Microchip Inc. | Using packet transfer for driving LCD panel driver electronics |
CA2629241A1 (en) * | 2003-07-31 | 2005-02-10 | Samsung Electronics Co., Ltd. | Information storage medium storing graphic data and apparatus and method of processing the graphic data |
KR100739676B1 (en) | 2003-07-31 | 2007-07-13 | 삼성전자주식회사 | Graphic data storage device, processing apparatus and method thereof |
US20050028079A1 (en) * | 2003-07-31 | 2005-02-03 | International Business Machines Corporation | Image distribution in data processing systems |
US7800623B2 (en) * | 2003-09-18 | 2010-09-21 | Genesis Microchip Inc. | Bypassing pixel clock generation and CRTC circuits in a graphics controller chip |
US7634090B2 (en) | 2003-09-26 | 2009-12-15 | Genesis Microchip Inc. | Packet based high definition high-bandwidth digital content protection |
US8156248B2 (en) | 2003-10-09 | 2012-04-10 | International Business Machines Corporation | Image distribution for dynamic server pages |
EP1695201A1 (en) * | 2003-12-12 | 2006-08-30 | Koninklijke Philips Electronics N.V. | Method and system for providing projector images |
US7542050B2 (en) | 2004-03-03 | 2009-06-02 | Virtual Iris Studios, Inc. | System for delivering and enabling interactivity with images |
US7450784B2 (en) * | 2004-08-31 | 2008-11-11 | Olympus Corporation | Image resolution converting device |
US7557818B1 (en) | 2004-10-06 | 2009-07-07 | Apple Inc. | Viewing digital images using a floating controller |
US7705858B2 (en) * | 2004-10-06 | 2010-04-27 | Apple Inc. | Techniques for displaying digital images on a display |
US7804508B2 (en) * | 2004-10-06 | 2010-09-28 | Apple Inc. | Viewing digital images on a display using a virtual loupe |
US8456488B2 (en) * | 2004-10-06 | 2013-06-04 | Apple Inc. | Displaying digital images using groups, stacks, and version sets |
US7561157B2 (en) | 2004-10-06 | 2009-07-14 | Apple Inc. | Compare mode for variable number of images |
FR2876860A1 (en) * | 2004-10-20 | 2006-04-21 | Thomson Licensing Sa | METHOD FOR HIERARCHIC ENCODING OF VIDEO IMAGES |
KR100762622B1 (en) * | 2004-12-28 | 2007-10-01 | 삼성전자주식회사 | Apparatus and method for controlling image scale in a broadcasting system |
US20070236508A1 (en) * | 2006-03-28 | 2007-10-11 | Microsoft Corporation | Management of gridded map data regions |
JP4752622B2 (en) * | 2006-05-30 | 2011-08-17 | ブラザー工業株式会社 | Image processing apparatus and image processing program |
US8086077B2 (en) * | 2006-06-30 | 2011-12-27 | Aperio Technologies, Inc. | Method for storing and retrieving large images via DICOM |
US7889212B2 (en) | 2006-09-07 | 2011-02-15 | Apple Inc. | Magnifying visual information using a center-based loupe |
US7936946B2 (en) * | 2007-02-23 | 2011-05-03 | Apple Inc. | Migration for old image database |
US20090086021A1 (en) * | 2007-09-27 | 2009-04-02 | Rockwell Automation Technologies, Inc. | Dynamically generating real-time visualizations in industrial automation environment as a function of contect and state information |
US20090144317A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | Data search markup language for searching research data |
US20090144222A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | Chart generator for searching research data |
US20090144265A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | Search engine for searching research data |
US20090144242A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | Indexer for searching research data |
US20090144243A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | User interface for searching research data |
US20090144318A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | System for searching research data |
US20090144241A1 (en) * | 2007-12-03 | 2009-06-04 | Chartsource, Inc., A Delaware Corporation | Search term parser for searching research data |
US8775953B2 (en) | 2007-12-05 | 2014-07-08 | Apple Inc. | Collage display of image projects |
JP5144237B2 (en) * | 2007-12-05 | 2013-02-13 | キヤノン株式会社 | Image processing apparatus, control method thereof, and program |
US20090158141A1 (en) * | 2007-12-18 | 2009-06-18 | Frederic Bauchot | Method and system to secure the display of a particular element of a markup file |
US8935609B2 (en) * | 2007-12-18 | 2015-01-13 | International Business Machines Corporation | Method and system to secure the display of advertisements on web browsers |
US8086957B2 (en) * | 2008-05-21 | 2011-12-27 | International Business Machines Corporation | Method and system to selectively secure the display of advertisements on web browsers |
JP5415730B2 (en) * | 2008-09-04 | 2014-02-12 | 任天堂株式会社 | Image processing program, image processing apparatus, image processing method, and image processing system |
US8600098B2 (en) * | 2008-09-25 | 2013-12-03 | Volkswagen Ag | Method for processing a satellite image and/or an aerial image |
US8429440B2 (en) | 2009-05-13 | 2013-04-23 | Stmicroelectronics, Inc. | Flat panel display driver method and system |
US8156238B2 (en) | 2009-05-13 | 2012-04-10 | Stmicroelectronics, Inc. | Wireless multimedia transport method and apparatus |
KR100938628B1 (en) * | 2009-10-21 | 2010-01-22 | (주)지니트 | Broadcasting system for integration image data |
TWI447638B (en) * | 2010-02-12 | 2014-08-01 | Esobi Inc | The display method of the displacement of the object on the electronic device screen |
US8571280B2 (en) * | 2010-02-22 | 2013-10-29 | Canon Kabushiki Kaisha | Transmission of medical image data |
US8671234B2 (en) | 2010-05-27 | 2014-03-11 | Stmicroelectronics, Inc. | Level shifting cable adaptor and chip system for use with dual-mode multi-media device |
US10726200B2 (en) * | 2011-02-04 | 2020-07-28 | Benjamin Chou | Systems and methods for user interfaces that provide enhanced verification of extracted data |
SG186404A1 (en) * | 2010-06-30 | 2013-01-30 | Singapore Telecomm Ltd | Enlargement of video content streamed from the internet |
US9417754B2 (en) * | 2011-08-05 | 2016-08-16 | P4tents1, LLC | User interface system, method, and computer program product |
JP5548671B2 (en) * | 2011-12-27 | 2014-07-16 | 株式会社ソニー・コンピュータエンタテインメント | Image processing system, image providing server, information processing apparatus, and image processing method |
US20130293550A1 (en) * | 2012-05-07 | 2013-11-07 | Business Intelligence Solutions Safe B.V. | Method and system for zoom animation |
US9886794B2 (en) * | 2012-06-05 | 2018-02-06 | Apple Inc. | Problem reporting in maps |
US10156455B2 (en) | 2012-06-05 | 2018-12-18 | Apple Inc. | Context-aware voice guidance |
KR20150105140A (en) * | 2014-03-07 | 2015-09-16 | 삼성전자주식회사 | Mobile device capable of enlarging content displayed thereon and method therefor |
KR101559838B1 (en) * | 2014-11-19 | 2015-10-13 | 엔쓰리엔 주식회사 | Visualizaion method and system, and integrated data file generating method and apparatus for 4d data |
CN106909603A (en) | 2016-08-31 | 2017-06-30 | 阿里巴巴集团控股有限公司 | Search information processing method and device |
CN109062651A (en) * | 2018-08-01 | 2018-12-21 | 深圳Tcl新技术有限公司 | Picture loading method, device, storage medium and the intelligent terminal at the interface UI |
CN116719597B (en) * | 2023-08-10 | 2024-01-26 | 北京冠群信息技术股份有限公司 | Self-adaptive view rendering method and creation method based on grid layout |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2287627A (en) * | 1994-03-01 | 1995-09-20 | Vtech Electronics Ltd | Windowed graphic video display system |
US5574836A (en) * | 1996-01-22 | 1996-11-12 | Broemmelsiek; Raymond M. | Interactive display apparatus and method with viewer position compensation |
WO1997039437A1 (en) * | 1996-04-12 | 1997-10-23 | Intergraph Corporation | High-speed video frame buffer using single port memory chips where pixel intensity values for display regions are stored at consecutive addresses of memory blocks |
EP0881563A2 (en) * | 1997-05-28 | 1998-12-02 | Sony Corporation | Digital map display scrolling method and device |
JPH11203308A (en) * | 1998-01-13 | 1999-07-30 | Ntt Communication Ware Kk | Map display device, map data base device, map display method and recording medium for recording map display program |
EP0933694A1 (en) * | 1997-07-18 | 1999-08-04 | Pfu Limited | High definition image display and program storage medium for the display |
WO2000000893A2 (en) * | 1998-06-30 | 2000-01-06 | Koninklijke Philips Electronics N.V. | Memory arrangement based on rate conversion |
US6023266A (en) * | 1994-04-29 | 2000-02-08 | Cirrus Logic, Inc. | PCMCIA video card |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4941193A (en) * | 1987-10-02 | 1990-07-10 | Iterated Systems, Inc. | Methods and apparatus for image compression by iterated function system |
JPH10198816A (en) * | 1997-01-13 | 1998-07-31 | Mitsubishi Electric Corp | Information processing system, and network type information processing system |
US6033266A (en) * | 1998-08-31 | 2000-03-07 | The Whitaker Corporation | Modular connector with preload and beam length reduction features |
US7236637B2 (en) * | 1999-11-24 | 2007-06-26 | Ge Medical Systems Information Technologies, Inc. | Method and apparatus for transmission and display of a compressed digitized image |
US6894686B2 (en) * | 2000-05-16 | 2005-05-17 | Nintendo Co., Ltd. | System and method for automatically editing captured images for inclusion into 3D video game play |
JP4705324B2 (en) * | 2003-12-26 | 2011-06-22 | 富士通株式会社 | Data operation method |
KR101099884B1 (en) * | 2004-04-23 | 2011-12-28 | 히또시 기야 | Moving picture data encoding method, decoding method, terminal device for executing them, and bi-directional interactive system |
-
2001
- 2001-07-10 CA CA002422780A patent/CA2422780A1/en not_active Abandoned
- 2001-07-10 KR KR10-2004-7010839A patent/KR100531676B1/en active IP Right Grant
- 2001-07-10 KR KR10-2004-7010843A patent/KR100531675B1/en active IP Right Grant
- 2001-07-10 WO PCT/JP2001/005967 patent/WO2002015162A2/en active IP Right Grant
- 2001-07-10 AU AU2001269494A patent/AU2001269494A1/en not_active Abandoned
- 2001-07-10 KR KR1020047010844A patent/KR20040075927A/en not_active Application Discontinuation
- 2001-07-10 US US10/110,680 patent/US7453479B2/en active Active
- 2001-07-10 KR KR10-2002-7004915A patent/KR100467240B1/en active IP Right Grant
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2287627A (en) * | 1994-03-01 | 1995-09-20 | Vtech Electronics Ltd | Windowed graphic video display system |
US6023266A (en) * | 1994-04-29 | 2000-02-08 | Cirrus Logic, Inc. | PCMCIA video card |
US5574836A (en) * | 1996-01-22 | 1996-11-12 | Broemmelsiek; Raymond M. | Interactive display apparatus and method with viewer position compensation |
WO1997039437A1 (en) * | 1996-04-12 | 1997-10-23 | Intergraph Corporation | High-speed video frame buffer using single port memory chips where pixel intensity values for display regions are stored at consecutive addresses of memory blocks |
EP0881563A2 (en) * | 1997-05-28 | 1998-12-02 | Sony Corporation | Digital map display scrolling method and device |
EP0933694A1 (en) * | 1997-07-18 | 1999-08-04 | Pfu Limited | High definition image display and program storage medium for the display |
JPH11203308A (en) * | 1998-01-13 | 1999-07-30 | Ntt Communication Ware Kk | Map display device, map data base device, map display method and recording medium for recording map display program |
WO2000000893A2 (en) * | 1998-06-30 | 2000-01-06 | Koninklijke Philips Electronics N.V. | Memory arrangement based on rate conversion |
Non-Patent Citations (1)
Title |
---|
PATENT ABSTRACTS OF JAPAN vol. 1999, no. 12, 29 October 1999 (1999-10-29) -& JP 11 203308 A (NTT COMMUNICATION WARE KK), 30 July 1999 (1999-07-30) * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1343132A1 (en) * | 2002-03-08 | 2003-09-10 | Fujitsu Siemens Computers, Inc. | Data viewer |
EP2058730A3 (en) * | 2007-11-07 | 2011-11-23 | Sony Corporation | Image processing apparatus, image processing method, and image processing program |
US8237741B2 (en) | 2007-11-07 | 2012-08-07 | Sony Corporation | Image processing apparatus, image processing method, and image processing program |
Also Published As
Publication number | Publication date |
---|---|
KR100467240B1 (en) | 2005-01-24 |
KR100531675B1 (en) | 2005-11-29 |
KR100531676B1 (en) | 2005-11-30 |
WO2002015162A3 (en) | 2002-09-06 |
AU2001269494A1 (en) | 2002-02-25 |
CA2422780A1 (en) | 2002-02-21 |
KR20020052190A (en) | 2002-07-02 |
KR20040068620A (en) | 2004-07-31 |
US20020191867A1 (en) | 2002-12-19 |
KR20040097988A (en) | 2004-11-18 |
KR20040075927A (en) | 2004-08-30 |
US7453479B2 (en) | 2008-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7453479B2 (en) | Image data displaying system and method | |
US20210287259A1 (en) | Method for providing real-time service of huge and high quality digital image on internet | |
US20240031568A1 (en) | Transferring system for huge and high quality images on network and method thereof | |
US8405660B2 (en) | Method and system for streaming documents, E-mail attachments and maps to wireless devices | |
US20030115167A1 (en) | Web browser implemented in an Internet appliance | |
JP5005782B2 (en) | Image generation system | |
KR20010035015A (en) | Method and system for displaying integrated image of multiple data | |
JP4848482B2 (en) | Video display system and video display method | |
KR20010000208A (en) | Method and system for large image display |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
ENP | Entry into the national phase |
Ref country code: JP Ref document number: 2002 520206 Kind code of ref document: A Format of ref document f/p: F |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020027004915 Country of ref document: KR |
|
WWP | Wipo information: published in national office |
Ref document number: 1020027004915 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 10110680 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2422780 Country of ref document: CA |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
WWG | Wipo information: grant in national office |
Ref document number: 1020027004915 Country of ref document: KR |