CN101197844B - File storage method based on subscription issuing system - Google Patents
File storage method based on subscription issuing system Download PDFInfo
- Publication number
- CN101197844B CN101197844B CN200710164756XA CN200710164756A CN101197844B CN 101197844 B CN101197844 B CN 101197844B CN 200710164756X A CN200710164756X A CN 200710164756XA CN 200710164756 A CN200710164756 A CN 200710164756A CN 101197844 B CN101197844 B CN 101197844B
- Authority
- CN
- China
- Prior art keywords
- message
- file
- container table
- destination
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The present invention relates to a file storing method based on publish/subscribe system, which is characterized in that: in a pub/sub system, message vessel tables are respectively created for each message destination, and the message vessel tables of the message vessel tables are stored in a table structure; meanwhile, the message vessel tables of the destinations are the interfaces of cache messages, and corresponding action can be carried out to messages through storage, deletion and modification methods provided by the message vessel tables. The present invention has the advantages of realizing the message vessel table which has file storage function, and can be operated like operating common table structures at the same time, i.e. the operation is carried out through the methods such as put(), get() and remove(), and needing to know no underlayer detail while carrying out lasting message storage. Meanwhile, the present invention has the other advantage of having much faster storage speed than database, because the present invention does not need to carry out database connections.
Description
Technical field
The present invention relates to file memory method, mainly is a kind of file memory method based on distribution subscription system.
Background technology
Nowadays, more and more enterprises, unit are faced with various data integrations and system combination, RPC middleware Technology such as CORBA, DCOM, RMI are also arisen at the historic moment, but owing to adopt RPC Synchronous Processing technology, all exist many shortcomings on performance, robustness, extensibility.And message based asynchronous process model adopts the recall feature of unblock, and the sender sends to message server with message, and message server is given the recipient with forwards again in suitable; It is asynchronous sending and receiving, the sender need not to wait for, the life cycle of the two also can be identical, and the sender can pass to a plurality of recipients indirectly with message, improved performance, extensibility and the robustness of program greatly, this makes the asynchronous process model have more attraction compared with the Synchronous Processing model on Distributed Application.
Publish/subscribe (publish/subscribe wherein, abbreviation pub/sub) system is a kind of messaging system based on the Distributed Calculation platform, also can claim middleware system, be used to the communication mode that a kind of loose coupling is provided between each participant in the distributed system.In the pub/sub system, the producer (publisher) of message and the consumer (subscriber) of message are arranged, the publisher issues message to the interested subscriber of this message by the middle-agent, and the subscriber differ establish a capital online, will get up message stores this moment, when reaching the standard grade, again message is sent etc. next subscriber.The message persistence just provides the function of storing message, and it has guaranteed the reliability of system.Make that the persistence message that sends to server end can both correctly be handled no matter be that server is worked as machine or client connection failure.Yet, because data file may be very big, that how with the least possible time carry out data storage, read, operation such as deletion becomes most important.These performances are relevant with adopting which type of data storage method, and what propose here is the mode of file storage, it by the persistence message stores in file, and reach storage, the deletion of message, the high-performance performance of extraction.
At present message stores mechanism mainly contains database storage, just by the persistence message stores in specific database, but because to create the expense that database connects very big, thereby influenced the storage of message, the performance of deletion etc.
Summary of the invention
The present invention will solve the defective that the message persistence exists in the existing pub/sub system, a kind of file memory method based on publish/subscribe system of high performance storage, deletion and the extraction to message is provided, and is the method that adopts when in a kind of distribution subscription system message being carried out persistence.
The technical solution adopted for the present invention to solve the technical problems: this file memory method based on distribution subscription system, in the pub/sub system, for all creating a message container table in each message destination, and the message container table of each destination all will exist in the list structure.Simultaneously the message container table of each destination is exactly the interface of buffered message, and methods such as the storage that can provide by the message container table, deletion, modification come message is carried out corresponding action.This makes operation file easily convenient as the table in internal memory of operation.Being achieved as follows of the concrete grammar of message container table: what deposit in the message container table is the message summation on specific purpose ground, and each message is so that (messageId, message) (keyword, the value) of form is to storing.When storing message is right, for each message is set up an index, index record the keyword of each message and two data places of value file and in the original position of file.The structure size of each index is all the same, and all index all leave in the index file with the form of chained list.And the index relative of each message and its correspondence is so that (messageId, IndexItem) (keyword, the value) of form is to leaving in the message index table.Concrete message is stored hereof with the form of data file, the big I manual configuration of file, but under the situation that file size configures, the message size of depositing hereof can not be above the size of each file that configures.Concrete steps are as follows:
After system start-up is good, automatically set up a message container table for each destination, and each destination has only a message container table, all message in this destination all are kept in this message container table with the right form of keyword/value, wherein keyword is corresponding to the messageId (messageId defines in the bag that sends over of message) of a message in the destination, and value is then to content that should message object.
1. the message when a sustainable subscription arrives server end:
From message, obtain the destination title that it will mail to,, find corresponding message container table,, then create a new message container table if there is not such message container table according to this destination title.Then, find corresponding messageId and its value from the message that arrives, the method for the storing message that provides by the message container table is saved in message in the file.
2. if will from file, take out a persistence message, then:
The same message container table that finds this message correspondence, and the method for the cancellation that provides according to message container table breath is removed to read in the file a piece of news and is got final product.
3. if will from file, delete a persistence message, then:
The same message container table that finds this message correspondence, the method according to the deletion message of message container table finds this message hereof, and deletion gets final product.
The effect that the present invention is useful is: realized a message container table with file memory function, can operate this message container table as the common list structure of operation simultaneously, promptly by put (), get (), remove methods such as () is carried out, and does not need to know the details of bottom when carrying out the persistence message stores.The while another advantage of the present invention is that its storage speed is fast more a lot of than database, because it does not need to carry out the connection of database.
Description of drawings
Fig. 1 is the critical process flow chart.
Embodiment
The invention will be described further below in conjunction with drawings and Examples:
For making purpose of the present invention, technical scheme and advantage are clearer, and below the present invention being used for java is the pub/sub system of development language, and will be described in further detail in conjunction with the accompanying drawings.
Under the java language environment, the present invention has been equivalent to realize a Map with file memory function, and the data of this Map storage can still can not used the data in the Map along with the closing or collapse and disappear of server when next server restarts.Simultaneously, realize the total interface of Map, conveniently call for other class.The corresponding MapContainer (message container table) in each destination, the MessageId of the message in the middle of each destination is unique, so leave in the middle of the MapContainer as key with MessageId.Introduce some cardinal principles earlier:
1) class MapContainer has realized that (Map provides a general element storage means to the map interface.The Map collection class is used for storage element to (being called " key " and " value "), and wherein each key is mapped to a value), and with the difference of commonly used map be: there is the internal memory the inside to object in map commonly used, and the present invention deposits object on the disk.Using method is the same with common map, and (key object) (is associated designated value, and deposits the map the inside)/MapContainer.get (key) (returning the value related with assignment key) MapContainer.put with assignment key;
2) the corresponding MapContainer in each destination creates a MapContainer for each destination, stores the message of this destination.All MapContainer are put among the map.Process is: receive the corresponding MapContainer-in the destination of message-this message of acquisition-find〉and MapContainer.put (messageId, message);
3) because size of message may be very big, so that data file may be very is big, the position of localization message data in the middle of this document correctly read message data then fast.Therefore the present invention adopts index, is index of each message establishing owing to be with the Map access, so index record key and two data of value original positions at file.All index link with a list, and the size of each index all is the same.Also adopted file read-write, because the access of message need be in the middle of file reading and writing at random, RandomAccessFile is provided in JDK this class, support the random read-write of file, but such I/O operating efficiency is lower, and byte of every read/write is carried out an I/O operation with regard to needing to disk.Add that by the RandomAccessFile class buffering read-write mechanism improves the speed of reading and writing data.Each data all has a data head simultaneously.Data head comprises a constant and data length, and it is correct having only the constant that ought read during read data, thinks that just these data are correct.
Introduce some structures and the realization of message stores below:
Employing index mechanism, the structure of IndexItem:
| short?MAGIC | Be used to judge whether this index is correct |
| long?offset | The position of this index in index |
| long?previousItem | The position of previous index |
| long?nextItem | The position of the index in back |
| long?keyOffset | The position of key |
| int?keyFile | The file at key place |
| long?valueOffset | The position of value |
| int?valueFile | The file at value place |
| boolean?active | Whether this index is movable |
The structure of DataItem (being stored in the data format on the file):
| ?short?MAGIC | Be used to judge whether this index is correct |
| ?long?offset | The position of this index in index |
| ?int?file | The file at place |
| ?int?size | The length of data |
Introduce the data storage procedure based on file storage of the present invention (based on figure one) below:
The corresponding MapContainer in each destination sets up corresponding MapContainer according to messageId in the destination and message value, and all MapContainer leave among the map.
● deposit data: call MapContainer.put (messageId, message)
1. create the IndexItem (index entry) of a sky
2. find idle file area by findSpaceForData () function
3. be keyData that messageId writes file, and create a dataItem, at dataItem,
Deposit the file at messageId place and in the side-play amount of this document
4. the value of IndexItem is set, i.e. keyFile and keyOffset according to dataItem
5. find idle file area by findSpaceForData () function
6. be valueData that message message body writes file, and create a dataItem, deposit the file at message place and in the side-play amount of this document
7. the value of IndexItem is set, i.e. valueFile and valueOffset according to dataItem
8. IndexItem is put into a list, and deposit the index file in
9. indexItem, keyDate are put into two Map, one with indexItem as key, another with keyData as key
● fetch data: call MapContainer.get (messageId)
1. (deposit that (messageId IndexItem) obtains IndexItem (index entry) according to keyData in (key, value) the right list structure) of form from the message index table
2. in specific I ndexItem, obtain leaving in the file at message place wherein and, also promptly obtain DataItem in the side-play amount of file
3. obtain existing concrete message data on the file by DataManager according to DataItem
● deleted data: call MapContainer.remove (messageId)
1. (deposit that (messageId IndexItem) obtains IndexItem (index entry) according to keyData in (key, value) the right list structure) of form from the message index table
From two message index tables (deposit (and messageId, IndexItem) form (key, value) to deposit that (IndexItem messageId) deletes corresponding indexItem, keyData item in (key, value) the right list structure) of form
3. obtain leaving in the file at message place wherein and, also just obtained DataItem (as before) according to IndexItem in the side-play amount of file
4. delete the concrete message data that exists on the file according to DataItem by DataManager
5. from the list chained list, delete the IndexItem item
6. IndexItem is reset active=false
7.updata this IndexItem is in the index file
The foregoing description is used for the present invention that explains, rather than limits the invention, and in the protection range of spirit of the present invention and claim, any modification and change to the present invention makes all fall into protection scope of the present invention.
Claims (2)
1. file memory method based on distribution subscription system, it is characterized in that: in the pub/sub system, for all creating a message container table in each message destination, and the message container table of each destination all will exist in the list structure, the message container table of each destination is exactly the interface of buffered message simultaneously, can come by storage, deletion, the amending method that the message container table provides message is carried out corresponding action; The concrete grammar of message container table is as follows: what deposit in the message container table is the message summation of setting the destination, each message is stored with the right form of keyword/value, when storing message is right, for each message is set up an index, index record the keyword of each message and two data places of value file and in the original position of file, the structure size of each index is all the same, and all index all leave in the index file with the form of chained list; And, the index relative of each message and its correspondence leaves in the message index table with the right form of keyword/value, concrete message is stored hereof with the form of data file, the big I manual configuration of file, but under the situation that file size configures, the message size of depositing hereof can not be above the size of each file that configures.
2. the file memory method based on distribution subscription system according to claim 1, it is characterized in that: after system start-up is good, for setting up a message container table in each destination, and each destination has only a message container table, all message in this destination all are kept in this message container table with the right form of keyword/value, wherein keyword is corresponding to the messageId of a message in the destination, messageId defines in the bag that sends over of message, and value is then to content that should message object;
(1). when the message of a sustainable subscription arrives server end:
From message, obtain the destination title that it will mail to,, find corresponding message container table,, then create a new message container table if there is not such message container table according to this destination title; Then, find corresponding messageId and its value from the message that arrives, the method for the storing message that provides by the message container table is saved in message in the file;
(2). if will from file, take out a persistence message, then:
The same message container table that finds this message correspondence, and the method for the cancellation that provides according to message container table breath is removed to read in the file a piece of news and is got final product;
(3). if will from file, delete a persistence message, then:
The same message container table that finds this message correspondence, the method according to the deletion message of message container table finds this message hereof, and deletion gets final product.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200710164756XA CN101197844B (en) | 2007-12-12 | 2007-12-12 | File storage method based on subscription issuing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200710164756XA CN101197844B (en) | 2007-12-12 | 2007-12-12 | File storage method based on subscription issuing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101197844A CN101197844A (en) | 2008-06-11 |
| CN101197844B true CN101197844B (en) | 2011-01-26 |
Family
ID=39547990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200710164756XA Expired - Fee Related CN101197844B (en) | 2007-12-12 | 2007-12-12 | File storage method based on subscription issuing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101197844B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177026A (en) * | 2011-12-23 | 2013-06-26 | 北京新媒传信科技有限公司 | Data management method and data management system |
| CN103500206A (en) * | 2013-09-29 | 2014-01-08 | 北京华胜天成科技股份有限公司 | Storage method and device based on file storage data |
| CN103678577B (en) * | 2013-12-10 | 2017-10-24 | 新浪网技术(中国)有限公司 | A kind of data-updating method and device |
| CN105677916A (en) * | 2016-03-02 | 2016-06-15 | 浪潮软件集团有限公司 | Method for Redis use |
| CN108259312A (en) * | 2016-12-29 | 2018-07-06 | 广州市动景计算机科技有限公司 | Information issuing method, device and server |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6985887B1 (en) * | 1999-03-19 | 2006-01-10 | Suncrest Llc | Apparatus and method for authenticated multi-user personal information database |
| CN1779704A (en) * | 2004-11-18 | 2006-05-31 | 黄金富 | Credit data inquiry system and method |
| EP1793562A1 (en) * | 2005-12-02 | 2007-06-06 | Alcatel Lucent | Virtual private network publish-subscribe multicast service |
| CN101078995A (en) * | 2007-06-08 | 2007-11-28 | 北京大学 | Method for accessing information service in component operation support platform |
-
2007
- 2007-12-12 CN CN200710164756XA patent/CN101197844B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6985887B1 (en) * | 1999-03-19 | 2006-01-10 | Suncrest Llc | Apparatus and method for authenticated multi-user personal information database |
| CN1779704A (en) * | 2004-11-18 | 2006-05-31 | 黄金富 | Credit data inquiry system and method |
| EP1793562A1 (en) * | 2005-12-02 | 2007-06-06 | Alcatel Lucent | Virtual private network publish-subscribe multicast service |
| CN101078995A (en) * | 2007-06-08 | 2007-11-28 | 北京大学 | Method for accessing information service in component operation support platform |
Non-Patent Citations (1)
| Title |
|---|
| 刘昱.基于内容的Pub/Sub系统中分发器网络自重构的研究.计算机应用研究.2006,(2),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101197844A (en) | 2008-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101686209B (en) | Method and device for storing message in message retransmission system | |
| US11416676B2 (en) | Using text messages to interact with spreadsheets | |
| CN101197844B (en) | File storage method based on subscription issuing system | |
| CN102833180B (en) | Method and system for sharing microblog messages | |
| US8489694B2 (en) | Peer-to-peer collaboration of publishers in a publish-subscription environment | |
| CN102769687B (en) | Mass texting system and method | |
| JP2014529131A (en) | Synchronizing dialog structures in web-based email systems | |
| CA2725017A1 (en) | Email auto-filing and management | |
| CN106682148A (en) | Method and device based on Solr data search | |
| CN111641700B (en) | Ceph object-based management and retrieval implementation method for storage metadata | |
| CN106991177B (en) | Lossless acquisition method and system for real-time logs in distributed environment | |
| CN103209214A (en) | Not only structured query language (NoSQL)-based method for realizing message-oriented middleware | |
| CN102437925B (en) | Data back up method, apparatus and system in distributed system | |
| CN101453428B (en) | Method and equipment for mail history customization | |
| CN102891797A (en) | Publishing and subscription method capable of supporting message duration | |
| CN105162727B (en) | A kind of method for realizing quick peer-to-peer communications in DSFA systems based on VxWorks system | |
| CN102833408B (en) | Method and device for displaying messages | |
| CN102801693A (en) | Method and system for processing extranet information | |
| CN106020713A (en) | File storage method based on buffer area | |
| CN100546281C (en) | A kind of method and device that transmits data | |
| CN101309231B (en) | Rope operation method and apparatus | |
| CN104125252A (en) | Data storage system and method | |
| CN105204776B (en) | Data processing method and device | |
| CN107040455B (en) | Instant message communication method and system based on redis | |
| CN113626457A (en) | Method and system for realizing database and cache consistency by cache deletion retry mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20080611 Assignee: Hundsun Technologies Inc. Assignor: Zhejiang University Contract record no.: 2013330000107 Denomination of invention: File memory method based on subscription issuing system Granted publication date: 20110126 License type: Common License Record date: 20130426 |
|
| LICC | Enforcement, change and cancellation of record of contracts on the licence for exploitation of a patent or utility model | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110126 Termination date: 20171212 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |