US 20060249423 A1
Machine vision is used for the separation or identification of medicinal preparation or the dosage of medication in various situations for example, without limitation, morning, day, evening and night doses, in which the dose is delivered from medicine boxes or other containers. In one implementation the medicine table or medicine box that is typically set to a planar scanner and a picture is taken with a camera or a camera phone and the picture is analyzed with machine vision software for separation and indentification of the medical preparation or the dose. Medicines are then recognized in cooperation with a database, and information related to the recognized medicines are transmitted or displayed. The identification of the medicinal preparation is often useful in many official tasks that require near real time identification of the medicine.
1. A method for identifying and separating a medicinal preparation and/or for dosage preparation thereof, the method comprising the Steps of:
registering a computer readable image of the medicinal preparation or the dose;
configuring a machine vision system to analyze the computer readable image and to discriminate the medicinal preparation into at least some of its constituent parts; and,
searching a database to find corresponding matches for said discriminated constituent parts, and retrieving any information related to the found corresponding matches for use in characterizing, separating, and/or dosage preparation of the medicinal preparation.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method claims 7, in which the medicinal information or/and the medicinal preparation's amount can be compared to previous proportioning times that are stored in a computer memory.
9. The method of
10. The method of
11. The method of
if there is a problem, transmitting a notification or alert that indicates there is a problem, and optionally characterizes the problem(s).
12. An apparatus for identifying and separating a medicinal preparation and/or for dosage preparation thereof, the apparatus comprising:
means for registering a computer readable image of the medicinal preparation or the dose;
means for configuring a machine vision system to analyze the computer readable image and to discriminate the medicinal preparation into at least some of its constituent parts; and means for searching a database to find corresponding matches for said discriminated constituent parts, and retrieving any information related to the found corresponding matches for use in characterizing, separating, and/or dosage reparation of the medicinal preparation.
13. The apparatus of
14. The apparatus of
15. Thee apparatus of
means for determining if a problem exists in said retrieved information; and,
means for transmitting a notification or alert that indicates there is a problem, and optionally characterizes the problem(s).
16. The apparatus of
17. The apparatus of
18. The apparatus of
19. The apparatus of
20. The apparatus of
The present continuation-in-part patent claims priority benefit of the PCT international patent application designating the United States with PCT application number PCT/F12004/000726 and filed on 26 Nov., 2004 under 35 U.S.C. 120 and 365(c).
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or patent disclosure as it appears in the Patent and Trademark Office, patent file or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates generally to medicine and dosages thereof. More particularly, the invention relates to the using of machine vision feasibly for separation or identification of medicinal preparation or the dosage of medication in various situations for example, without limitation, morning, day, evening and night doses, in which the dose is delivered from medicine boxes or other containers for example, without limitation, in a hospital, an older person's house, a pharmacy or at a customer's home.
The number of elderly persons is increasing, and these elderly people often take several medicines in different dosages and at different times causing challenges to the success of their medication. Older people often want to live at home as long as possible, and they often need help managing their medication.
Users of medication may under use or overdose on this medication. For example, without limitation, one problem with medication dosage is that tranquilizing and sleeping medicine is routinely given to people, and although they are ordered by the prescription to take only as needed, these users continually take these medicines. When these users are older people, they can become drowsy and, because of their degraded mobility, are for example more liable to falling.
Commonly used medication distribution systems include a medicine box, which is a plastic box, in which compartments are named for example for the days of the week and according to the time. When the patient removes his medicine from the device to take, he is not informed of medication information, and this type of device does not prevent the patient from forgetting to take the medicine. Another current medication distribution system is a dose bag. This system is not as familiar as a medicine box. In these solutions medicine information is also not included with the medication, and it can be difficult for patients with several medications to remember the instructions for all of their tablets.
Other known medication reminder technologies include a medicine bag memory unit, which reminds the user to take the medicine, a time base on a box, which inactivates, when the medicine has been taken from the box, a device using light only in activating the signal to take the medicine. In yet other known technologies bar codes are being used in the tablet to avoid mistakes, a motorized tablet dispenser provides the medicine, and a smart automation medication dispenser uses a “plurality of ejection arms”. Another known method is a medication dispensing and timing system used by a central computer, in which telephone networks and information networks are used for example, without limitation, the use of a modem. And, another known distribution method is a medication dispenser and monitor with two programmable processors.
No current proportioning system of medicines, such as, but not limited to, a medicine box or a mechanical dose distributor takes into the account the separation of medication that is taken as necessary or as needed. Also, other problems exist in current systems. For example, after the distribution of the medication, a patient may not know what each medicine is so that they may avoid taking sleeping medicines if not needed at that time. When medicines are taken from their containers, they are separated from important information such as, but not limited to, the name of the medication, side effects, and doctor's given instructions.
In view of the foregoing, there is a need for an improved system of separating and identifying dosages of medications that retains important information about the medication, and assists the user in taking the correct dosages.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
Unless otherwise indicated illustrations in the figures are not necessarily drawn to scale.
To achieve the forgoing and other objects and in accordance with the purpose of the invention, a variety of techniques are described to identify and separate medicinal preparations and dosages thereof.
In an embodiment of the present invention a machine vision system is used for the separation or indentification of medicinal preparation or the dosage of medication in various situations for example, without limitation, morning, day evening and night doses, in which the dose is delivered from medicine boxes or other containers. One implementation is a medicine table or medicine box that is typically set to a planar scanner and a picture is taken with a camera or a camera phone and the picture is analyzed with machine vision software for separation and identification of the medical preparation or the dose. Medicines are then recognized in cooperation with a database, and information related to the recognized medicines are transmitted or displayed.
Other features, advantages, and object of the present invention will become more apparent and be more readily understood from the following detailed description, which should be read in conjunction with the accompanying drawings.
The present invention is best understood by reference to the detailed figures and description set forth herein.
Embodiments of the invention are discussed below with reference to the Figures. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes as the invention extends beyond these limited embodiments. For example, without limitation, it should be appreciated that those skilled in the art will, in light of the teachings of the present invention, recognize a multiplicity of alternate and suitable approaches, depending on the needs of the particular application, to implement the functionality of any given detail described herein, beyond the particular implementation choices in the following embodiments described and shown. That is, there are numerous modifications and variations of the invention that are too numerous to be listed but that all fit within the scope of the invention. Also, singular words should be read as plural and vice versa and masculine as feminine and vice versa, where appropriate, and alternative embodiments do not necessarily imply that the two are mutually exclusive.
The present invention will now be described in detail with reference to embodiments thereof as illustrated in the accompanying drawings.
Embodiments of the present invention comprise a method and an apparatus for recognizing a dose or medicinal preparation comprising obtaining a picture, using machine vision and giving information concerning the medicine. In some embodiments the medicine identifier can be integrated into a computer system where the medicine is to be distributed such as, but not limited to, a hospital, pharmacy etc. Embodiments of the present invention can be used in different situations that require the identification of the medicinal preparation for example, without limitation, in customs, police and in first aid. The identification of the medicines or preclusion is done quickly according to the method based in up-to-date information.
An aspect of the present invention is to provide a system that is to provide a system that is able to remind a user to take medicine and also can provide information about the medication and dosage during the entire medication time. A medication's success depends the user correctly taking the medicine, for example, without limitation, taking it at the correct time, taking it with food, and correctly using other medicines. A patient may have several different medicines and determining the right time and optimal use of each of them can be difficult.
In the present embodiment, medicines 3 are put in to a medicine box 1 with mobile partitions 4 in a single layer typically in the horizontal position over a scanner 2 so that medicines 3 can be moved to be identified. In alternate embodiments, partitions 4 may be immovable. In the preferred embodiment, medicine box 1 is set to planar scanner 2, which is set to read a certain amount of colors for example, without limitation, 256 colors. In the preferred embodiment, scanner 2 has a start button 7 and regulates lighting for scanning. In other embodiments, various scanners may be used, and in other embodiments the medicine can be photographed, for example with a camera or a telephone with a built in camera, a smart camera, a web camera, a video camera, or a combination of these. Then a microprocessor 5 with memory and communication connections calculates and compares the similarity of medicines 3, based on an algorithm, to recognizable pieces of medicines in a database and gives the data to a display 6. In preferred embodiments microprocessor 5 uses visual machine technologies typically with different pattern recognition technologies, fuzzy logic, neural networks, or with a combination of these in order to recognize different known medical preparations and/or doses. The information shown on display 6 may include, without limitation, a reminder to take medicine 3 or another attention, such as, but not limited to, taking medicine 3 with food, etc. In some embodiments this information and the amount of medication can be compared to previous proportioning times in the memory of microprocessor 5 or with the communication with a central computer, for example, without limitation, in a database of electronic prescriptions and/or a database of the medicine or the patient.
In some embodiments machine vision software may be located elsewhere. For example, without limitation, the device may comprise a camera telephone that takes a picture of the medication and sends that picture to a central computer, which informs, for example, without limitation, through a text message or with sound, information about the medication being taken and possible lighting adjustments for the picture-taking program. When a cover of scanner 2 is open, the background becomes black, which aids in the identification of medicine 3. Some embodiments comprise a clock that can scan medicine box 1 for example, without limitation, in 30-minute intervals automatically so that start button 7 does not have to be pushed.
Microprocessor 5 comprises memory in which the medication history can be saved to be used for the benefit of the user of in many ways. Some embodiments may include, without limitation, a teaching unit 11 that can update the device with new medical preparations. In other embodiments this teaching device may be located in a central unit with which the device can communicate. One medication safety increasing point is that the distributor of the medicines can check if there have been any similar medications taken during the previous time period. However as many users are also patients and many of these patients are elderly persons, whose memory may be weakened, the device's user-interface is simple to use. Also the resetting of the information is possible, which may be accomplished by pressing a reset button 8, but in alternate embodiments, this resetting may be made more difficult so that it may not be done by accident.
The present embodiment allows medication to be connected to necessary information and makes this information easily available so that users do not have to remember all of this information. For example, without limitation, in connection with some medicine, it may be important drink a least one glass water or avoid special interactions for example, without limitation, with grapefruit juice. So this kind of information, which would be lost in a traditional medicine box for mechanical medicine distribution, is available with methods and systems in accordance with embodiments of the present invention. In the present embodiment, it is advantageous that competing systems are provided, for example, without limitation, a memory circle or a central computer in order to provide the information. In the present embodiment, medicines 3 are separated into necessary medications and as needed medications. Medicine 3 is comprised of solid medicines, typically tablets and capsules.
The use of the present embodiment is simple. In typical use the necessary medicine is put into medicine box 1. Then the user presses start button and scanner 2 takes the picture. Then as needed medicine is placed in medicine box 1, and again the user presses start button 7, and scanner 2 takes the picture again. The method is flexible, whereby all medicines, including, but not limited to, self-care medicines, can be placed in medicine box 1 at the same time. Generally, in the mechanical distribution of dose, the selection of medicines is small and flexibility is especially small, for example, without limitation, among self-care medicines. When mechanical delivering systems are used, there are typically limited in the number of different drugs they can properly deliver; for example up to 300 different drugs. Obviously, however, there are far more different types of drug on the market than such machines can handle.
In the present embodiment, the device recognizes at the least the necessary and the only as needed medicines, but if drug manufacturers begin to make other kinds of tablets and capsules, the device may be configured to identify information contained in the tablet in order to classify the medicine. This data may be updated as new medicines are created in microprocessor 5 of the device for example by way of a disk or an information network depending on the embodiment. In addition to the electronic prescription this additional data and this inspection increases the safety of the medicine. In some embodiments, instructions may be available in different languages to increase the comprehension of the user.
In embodiments using mechanical medicine distribution, it may be necessary to include a vibrator 10 on medicine box 1, so that medicines 3 are in a single layer on the scanning bed for proper scanning. Vibrator 10 may be automatic or started manually. In some embodiments, a program can detect whether medicines 3 are overlapping each other so that the separation of medicines 3 is not complete and this program can automatically move partitions 4 of medicine box 1 or vibrate the entire frame of medicine box 1 to separate medicines 3 from each other. In some embodiments when this action is performed, the device can alert the user that vibrator 10 is about to be operated so that the user can cancel the operation if desired. In yet other embodiments, vibrator 10 can be operated manually for example, without limitation, with a button or a switch.
In accordance with the present embodiment, the medicine reminder is simple to use, and because of the important information it provides, its operation should not be interrupted for example during an electricity break. Therefore, an accumulator or battery guaranteed system may be provided in some embodiments that also allows the device to be portable. In some embodiments part of display 6 may be a user-interface, which tells the user, when the next medicine must be taken. In addition the user-interface can be more complex to show, among other things, an entire day's medication times. In some embodiments, if the medicine is not taken within a tolerance time, which is set beforehand, display 6 may start for example to blink or alert 9 may be activated.
According to American research, in manually delivered doses of medication in institutional care there was fault almost in every fifth dosage. Common faults were incorrect timing, the medicine being separated from the box, or the giving of the medicine without guidance. In the preferred embodiment of the present invention, the medicine distributor is able to check for example that the current delivered period does not deviate from the previous delivery period, and because of this it improves medical safety.
Other medicine identification systems are often dependent on a central computer, but the present embodiment can be implemented with a smaller unit that can be taken along with the user for example, without limitation on a vacation or to a holiday residence. Current technology allows scanners to be made in different sizes for example, without limitation, very thin. In the present embodiment, scanner 2 is equipped with a computer program that allows for the recognition of different properties of medicine 3 for identification. For example, without limitation, Java computer based programs and algorithms known to those skilled in the art are available. Programs such as this can use features to classify targets for a known class or place. The features on the other hand are properties or property groups, which repeat inside the class, and the tablets and capsules positional situation is generally like this. As is well known to those in the art, pattern recognition seeks to classify data (patterns) based on either a priori knowledge or on statistical information extracted from the patterns. The patterns to be classified are usually groups of measurements or observation defining points in an appropriate multidimensional space.
A complete pattern recognition system typically comprises of a sensor that gathers the observations to be classified or described, a feature extraction mechanism that computes numeric or symbolic information from the observations; and a classification or description scheme that does the actual job of classifying or describing observations, relying on the extracted features.
The classification or description scheme is typically based on the availability of a set of patterns that have already been classified or described. This set of patterns is referred to as the training set and the resulting learning strategy is characterized as supervised learning. Learning can also be unsupervised, in the sense that the system is not given an a priori labeling of patterns, instead it establishes the classes itself based on the statistical regularities of the patterns. These properties are used along with pattern recognition to identify the medicine. This pattern recognition can be done using for example, without limitation, shape, color, surface, darkness, length, width and area of the tablets or capsules in addition to other dimensions or calculated quantities such as, but not limited to, the relationship of the medicine's length and width. The pattern recognition can be shared by three different main divisions, which are statistical, syntactical and to the neural network based on identification.
Some embodiments may use a program that identifies the medicine using pixels. Pixel similarity based on classifying is accomplished by expanding the area of a qualified pixel. The acceptability of the pixel can be checked for example, without limitation, with a simple or with an automatic border. Classifying is done by clustering pixels of similar surroundings for the one cluster. Another method is to go through the surrounding of the qualified pixel, for example, without limitation, 4 or 8 of the nearest surrounding pixels, and if similar pixels can be found, the search is continued to recursive for them.
A bitmap term assigns every pixel its own place in the x- and y-coordinates of the picture so that each pixel has a unique location in the picture. A bitmap picture has four important basic properties: pixel size, resolution, colors' model and depth. A bitmap picture has a resolution, which typically presents the quantity of pixels per inch. Generally, the presentation of the colors on the computer display is based on sum of the RGB (red, green and blue) color system and generally the same color system is also used in digital cameras and scanners. Each pixel is saved a red, green and blue (RGB) intensity of the color in the certain precision. A color depth is presented by the quantity binary per pixel. In a (full color picture (24 bits) each pixel has 256 different brightness values available per a main color, so its color depth is 3×8=24 bits. Thus in the picture, each picture point can be presented by 24 bits in 256×256×256=16.7 million different colors.
A digital picture forms from a point of the picture, or a pixel. The device obtains information from the pixels. This information may include, without limitation, color information and points place. The more pixels that are in a certain area, the smaller they are and the more precise the picture is. In this way the picture can be presented better and change tones and small details of the picture. The pixel information is in the black white pictures in 0 or 1 bits and in the color pictures pixel can be for example in 8 bits, whereby with that can express 256 colors. From this resulting matrix the picture is used in the present embodiment. One common way the picture may be used is in a principal component analysis, whereby the picture can be inspected for example, without limitation, for the histograms. Object modeling is another known method to see tablets and capsules for example without limitation, objects in the medicine class, which have certain properties may be modeled. The objects are described by the measurements or by the relationships of properties, and they are classified on the basis of the similarities and differences.
Other embodiments may use Raman spectroscopy to identify the medicine. Raman spectroscopy is an analysis method, which is based on molecules that cause measurable scattering of light. The Raman scattering happens when the specimen being examined is focused on light photons breaking out lose energy to the molecules or get gain energy from those molecules. The Raman spectrums can be observed in solid, liquid and in gas atmosphere molecules. In pharmaceutics Raman spectroscopy is used in the examination of the medicine delivery system among other things. Using, Raman spectroscopy, active ingredient and excipients of medicine 3 can be recognized contents can be defined, and impurities can be located. Raman spectroscopy is commonly used in chemistry, since vibrational information is very specific for the chemical bonds in molecules. It therefore provides a “fingerprint”, or signature, by which the molecule can be identified.
The benefits of the Raman method are speed and ease, which is because of the development of new devices for this method and that the specimens generally do not need to be prepared for the examination. Raman spectroscopy is no longer only a laboratory analysis method, and easily mobile devices for Raman spectroscopy are now available. Also, the prices of devices for Raman spectroscopy have lowered, which has allowed the method to benefit wider technologies, such as, without limitation, in the case of scanners. Another benefit of examining with Raman spectroscopy is that sampling can possibly be carried out through the packing materials of tablets and capsules. This is because on most packing materials, such as, but not limited to, on plastic and on glass, the Raman scattering very weak, and these packing materials do not disturb examining of the specimen's Raman spectrums. The successful collecting of the Raman spectrums produces a great deal of analytical information. As a result of the different Raman spectrums given by different molecules, the identification of these molecules is possible according to the method. This again allows for specimen quality analysis, for example, without limitation, the analysis or active substances in a tablet mass. On other hand the light of the sprinkled intensity picture also material content in the specimen, and so method can be improved also to the determined analysis. Raman spectroscopy can be used also in quantitative analysis such as, without limitation, 400 mg ibuprofen, instead of, for example, 300 mg or 450 mg. Further, among other things, material content of active substances in the medicine output system can be analyzed for example in the tablet, because the spectrums of the active substances are generally appreciably more powerful than the spectrums of the excipients. Generally, different tablets contain different drugs and this method can have medicines separated and/or recognized. For example, without limitation, with the help of Raman scattering, among other things, the contents of the active substances and excipients from the tablets can be defined.
Raman spectroscopy is an important parallel method to infrared (IR) spectroscopy; IR spectroscopy and Raman spectroscopy complete each other. If the specimen is gently exposed to fluorescent light, the intensity of the fluorescence issue is normally much higher than the intensity of Raman scattering. Because of this, difficulties may arise. FT-Raman technology is intended to eliminate problems caused by fluorescence. SERS (surface-enhanced Raman spectroscopy) is recently developed technology that can have strong signals from small amounts of organic molecules. In a micro Raman method a monochromatic laser beam is tightly focused through a microscope object glass. This method gives the user the possibility to examine areas, which are either on the specimen, or inside the specimen. The specimen area can be as small as 2.0 μm2.
Simple information analysis methods have also been developed in computer software. They include for example, without limitation, data banks, which have many Raman spectrums of different materials. As compared with traditional to spectroscopy technology FT-Raman spectroscopy offers four significant benefits in medicinal preparation product development: it does not demand the preparation of the specimen, it can be used with an optical microscope, small particles in a solid specimen can be examined, and it does not damage the specimen. With FT-Raman spectroscopy, the specimen can be examined even through a tightly closed, filmy cover. A medicinal preparation often includes a two or more component mixture. The powder from different particles can be defined to each component by their own Raman spectrum. Because the scattering of the polymer coat is weak and fluorescence is not significantly used as the source of long wave light, FT-Raman spectroscopy can examine antihistamine tablets through their blister packs, which is generally impossible by other methods. So medicine can be for example, without limitation, in a medicine bag or in a box and can still be examined through this method.
Near infra red spectrometer (NIR) is a comparison method, by which a user is able to recognize and to study the composition of organic compounds on the wavelength range of 780-2500 nm. The infrared spectroscopy (IR) is a more commonly used comparison method in organic chemistry for the identification and researching of molecules. Different functional groups call be recognized by the vibration values of their characteristics. IR-spectroscopy is a useful method in the identification of pharmaceutical materials and in the studying and analysis of crystal structures and other physical properties.
The following is an example of typical use of the preferred embodiment of the present invention. In the present example, a nurse of an elderly person puts the morning, day, evening, and night medication 3 for a two week period of the elderly person into medicine box 1. Firstly, the nurse puts the necessary medicines planar, no one on top of another medicine, and pushes start button 7 whereby scanner 2 scans medicine box 1 or takes a picture of medicine box 1. Display 6 shows the situation according to each compartment of medicine box 1 and the nurse can check that number of medications is correct and can obtain other information from the model dependency. Then the nurse puts the as needed medicines in the compartment of medicine box 1 together to the floor planar and pushes start button 7, and scanner 2 scans medicine box 1. Again, the number of medicines and the entire amount of the medicines are shown on display 6. The device informs for example by alarm 9, if the medicine is not be taken according, to the schedule. Some embodiments can recognize medicine to the known preparation and, by this, give information connected to the medicine such as, but not limited to, to drink at least one glass of water in connection with taking the medicine or that the medicine is not suitable for taking with grapefruit juice.
For those skilled in the art it will be clear that the invention is not limited to the presented details, such as, but not limited to, the form of medicine box 1, the material used, and the colors binary digit amount used. In alternate embodiments the dose recognizer can be manual, mechanical or a combination of these. Embodiments may be implemented to work in various environments such as, but not limited to, in the medicine distribution in a hospital, a chemist, a drug factory a wholesaler, or at home.
Having fully described at least one embodiment of the present invention, other equivalent or alternative means for implementing a method and apparatus to separate and identify medications and dosages according to the present invention will be apparent to those skilled in the art. The invention has been described above by way of illustration, and the specific embodiments disclosed are not intended to limit the invention to the particular forms disclosed. The invention is thus to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the following claims.