US 7970477 B2
A signal transmitting and receiving system to track the position and orientation of limb segments in order to provide feedback information for the control of the limb movement. The user will generally be seated in a wheelchair that provides a structure upon which orthogonal and/or non-orthogonally oriented coils can be mounted and energized selectively so as to create variously oriented magnetic fields. Multiple wireless sensors injected into the limb detect the local field strength and send information telemetrically to a central controller. The controller extracts information about the position and orientation of each limb segment by combining signals from various sensors according to calibration information and optimal filtering methods for extracting information from multiple nonlinear sensors in mechanically constrained systems.
1. A position and orientation detecting system to measure movement of a limb described with six independent position and orientation variables, comprising:
a) one magnetic field source, comprising an AC source and a transmitter coil energized by an asymmetrical current waveform, configured to be located external to the limb and to generate at least one magnetic field;
b) six physically separated, non-orthogonally oriented capsules configured to be implanted within the limb, each capsule having a sensor configured to sense the magnetic field and to telemetrically transmit at least one signal corresponding to the sensed magnetic field, wherein the transmitted signal comprises phase data; and
c) a controller configured to receive the transmitted signals, to compute the position and orientation of the limb based on the transmitted signals, and to generate information useful in controlling movement of the limb based on the transmitted signals;
wherein the product of the number of the magnetic field source and the capsule is six.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
7. The system of
8. The system of
9. The system of
10. The system of
11. The system of
12. The system of
13. The system of
14. The system of
15. The system of
16. The system of
17. The system of
18. The system of
This United States patent application is a continuation of U.S. application Ser. No. 11/457,101, filed Jul. 12, 2006, now abandoned entitled “Method and Apparatus for Detecting Object Orientation and Position,” which is related to and claims the benefit of the filing date of U.S. Provisional Patent Application Ser. No. 60/698,313, filed Jul. 12, 2005, entitled “Method and Apparatus for Detecting Object Orientation and Position,” the contents of both of which are incorporated herein by reference. This United States patent application is also related to co-pending U.S. patent application Ser. No. 11/457,004, also filed on Jul. 12, 2006, entitled “Probe for Identifying Injection Site for Deep Brain Neural Prostheses,” inventor Gerald E. Loeb, the contents of which are also incorporated herein by reference.
1. Field of the Invention
The present invention relates generally to implanted devices and methods for detecting object orientation and position.
2. General Background and State of the Art
Wireless micro-stimulators such as the BION® implants (U.S. Pat. Nos. 5,193,539, 5,193,540, 5,312,439, and 5,324,316; herein incorporated by reference) can be injected into paralyzed muscles or the stumps of amputated limbs to restore voluntary control of limb movement. In order to control neuromuscular stimulation, it is desirable to provide information about the posture and movement of the limb. Two sources of such information can be distinguished. The first derives from the fact that most patients will have residual voluntary control over some portions of the limb. Such voluntary movements can be sensed by the present invention in order to provide command information about the patient's intended limb movements. The second derives from the use of feedback information to improve motor performance. The movements produced by stimulation of the muscles can be sensed by the present invention, thereby providing feedback signals that modify the electrical stimulation pattern in order to achieve the desired limb movement. In both cases, it is desirable to employ sensors that are no more intrusive than the injectable stimulators themselves. Furthermore it is desirable to minimize the number of separate devices that must be injected into the patient while maximizing the information that is available for control of the prosthetic system.
There are many technologies for tracking and measuring limb movement in the laboratory but these generally require physically affixing sensors or markers to the surface of the limb. Most commonly, the position and/or orientation of these sensors or markers are determined by capturing optical information by video cameras or photodiode arrays whose position and line of sight must be carefully determined and maintained. Search coil systems use large coils to create homogeneous, orthogonally oriented magnetic fields around the subject, which are detected by a detection coil such as attached to a contact lens to track eye movement. Polhemus Technology (U.S. Pat. Nos. 6,624,626 and 6,400,139) and Ascension Technology (U.S. Pat. Nos. 6,528,991, 5,953,683, and 4,945,305) use externally mounted sensors consisting of three orthogonal detection coils to detect the relative strength of a magnetic field created by an external field generator. In these magnetic sensing systems, the magnetic sources can be AC (U.S. Pat. No. 6,690,963), DC (U.S. Pat. No. 4,945,305) or sourceless (U.S. Pat. No. 5,953,683 earth magnetic field). There are also mechanical transducers such as electrogoniometers produced by Infotronic Company that can be affixed across adjacent limb segments to detect angular rotation of the joint in one or two axes.
The prior art includes a fully implanted sensor of a single joint angle based on the effect of a permanent magnet on a nearby Hall-effect sensor. The magnet is implanted in the bone on one side of the joint and the Hall-effect sensor is implanted on the other side of the joint so that changes in joint angle produce a systematic change in the relative orientation of the sensor to the magnet. The sensor is connected by electrical leads to an implanted signal processor that, in turn, transmits data by wireless telemetry to an external controller.
The prior art includes a sensing system to track the tip of an intravascular catheter as it is maneuvered within a patient (U.S. Pat. No. 6,690,963). This system uses a large external array of magnetic coils to create a magnetic field that is sensed by orthogonal sensing coils contained within the tip of the catheter. In the preferred embodiments of the invention the coils have mutually orthogonal axes. The coils are closely spaced along the axis of the catheter. The magnetic field generating coils are arranged across the surface of the table on which the subject lies. The sensing coils are connected by electrical leads passing along the catheter to external signal processing instrumentation. This system basically uses the same mechanism as the concentrated orthogonal 3-axis magnetic sensor mentioned above. Sensing coils with distributed positions and orientations will present more than one local minimum points in the problem. Algorithms without global optimization technique tend to set design limitations on sensing coil arrangement such that the coils are close enough to the concentrated and orthogonal configuration.
Various algorithms have been described to transform the amplitude and phase information detected by sensors into conventional coordinate frames. Both rotation matrix and quaternion have been used to provide complete descriptions of 3-D orientations and rotation transforms. Mathematical methods exist to convert between them or combine them into hybrid representations. The data processing methods include both iterative and non-iterative techniques. U.S. Pat. No. 6,690,963 teaches numerical algorithm to solve the nonlinear equation group. But it does not investigate the local minimum points associated with the numerical process. In fact, if the receiving coils and transmitting coils are not arranged properly, there will be local minimum areas that the conventional numerical methods mentioned in this patent cannot solve. In the prior arts, the filters applied to the sensing system include mean-square filters and Kalman filters. The selection of appropriate methods for computing and describing position and orientation is an important part of the design of a sensing system that determines its practicality and performance in a given application.
In one aspect of the position and orientation detecting devices, a method to detect human limb posture comprises injectable distributed sensors of externally generated RF electromagnetic fields. These sensors and transmitters can be arranged at different locations with respect to the limb and they are allowed to have non-orthogonal and/or nonconcentric orientations with respect to each other, instead of being lumped in a single package as a 3-axis assembly. Keeping these sensors distributed instead of in the same package and making them nonorthogonal offers more freedom especially in medical applications such as control of Functional Electrical Stimulation (FES). In FES control, a preferable design is to have sensors and nerve stimulators packaged in the same implantable micro-capsule. Wireless sensors that are implanted into the human body need to be physically small and to function with low power consumption. The distributed scheme makes possible a sensing system with single-axis search coil sensors each of which can be packaged into an elongated micro-capsule that can be injected into the limb rather than requiring open surgical implantation. In an exemplary embodiment, the sensing coil is the same as is already required in the implanted microcapsule in order to receive power from and exchange data with an external controller.
In another aspect of the position and orientation detecting devices, a remote human limb position and orientation determining system comprises a plurality of radiating antenna coils to generate RF magnetic fields and a group of single axis micro-miniature sensing capsules with size and form factor suitable for injection into human limbs. Data acquisition, signal transmitting and receiving circuits are also in the capsules. The data can be sent by electromagnetic telemetry to an external coil. The external coil relays the data to a micro-computer, which acts as a signal processing unit.
In yet another aspect of the position and orientation detecting devices, synchronizing techniques enable the system to get not only the amplitude, but also the phase of the RF magnetic signal. The combining of phase and amplitude information makes the best use of the sensor output and is suitable for medical applications, which requires as few implants as possible. Various schemes are possible for gating and modulating the electromagnetic fields generated by each of the external coils sequentially or simultaneously and relating the state of the transmitters to the amplitude and/or phase of the signals detected at known times by each sensor.
In still a further aspect of the position and orientation detecting devices, the computing algorithms comprise a composite optimization process including genetic algorithm and gradient based local optimization. The algorithm produces position and orientation information based on the sensor signals. The information is then fed through an Extended Kalman Filter to suppress noise. The solution is accurate, fast and continuous.
Aspects of the present invention are illustrated by way of example, and not by way of limitation, in the accompanying drawings, wherein:
The detailed description set forth below is intended as a description of exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention can be practiced. The term “exemplary” used throughout this description means “serving as an example, instance, or illustration,” and should not necessarily be construed as preferred or advantageous over other embodiments. The detailed description includes specific details for the purpose of providing a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the present invention.
The inductive coils are likely to be non-concentric and similar or smaller in size to the distances between each other and between the coils and the sensors. This is substantially different from electromagnetic position sensing technologies employed in the prior art, in which very large coils are used to create uniform fields in the vicinity of sensors or in which very small coils are used to create uniform gradients in field strength in the vicinity of the sensors. As illustrated in
One Exemplary Embodiment of the Present Magnetic Sensor
Functional electrical stimulation (FES) of paralyzed muscles has traditionally focused on getting muscles to produce the requisite torques required for limb movement. Micro-implants called BION® implantable microstimulators (BION is a registered trademark of Boston Scientific Neuromodulation Corporation, Reg. No. 2455608) have been developed to interface with the multiple muscles required for most tasks. They are modularly designed wireless capsules that can be injected into the human body without requiring surgery. Separately addressable BION's (selectively used as a noun hereinafter merely for convenience purposes) can be implanted at various sites in the body near motor nerves, where they receive power and digital command data from an external RF coil and deliver stimulating current pulses to recruit the motor neurons and activate associated muscles. Given appropriate control strategies, this technology could be used to create functional movement in limbs paralyzed by upper motor neuron disorders such as spinal cord injury and stroke.
As the functions to be implemented by BIONs become more demanding, the prosthetic system may include a growing armamentarium of sensors to detect voluntary command signals and to provide sensory feedback to regulate neuromuscular stimulation. Furthermore, these sensors and actuators may be readily combinable into custom configurations to deal with the inherent heterogeneity of patients with diverse physiognomies and lifestyles suffering from wide ranges of traumatic and neurological deficits. Attaching and wiring miscellaneous sensors on the body is not an attractive design for most scenarios of FES application, however. Activities of daily living such as bathing and sleeping require frequently detaching and re-attaching these sensors, which may be difficult and which is likely to result in problems with system calibration. External sensors are likely to be uncomfortable and vulnerable to damage. In an exemplary embodiment, the sensors may be integrated with the implants for one-time injection and long-term usage. The micro-coil within each BION implant may act as a magnetic sensor.
Also, as illustrated in
RF Magnetic Signal from a Dipole Ring Antenna
The BION implant consists of an electronic subassembly with an integrated circuit chip surrounded by a hermetically sealed glass capsule with external electrodes. The BION also includes a cylindrical coil around a hollow ferrite core that is the antenna of the implant. The coil is connected to the input pads of an ASIC through a ceramic hybrid substrate. In normal operation, it receives power and command data from a frequency-modulated RF magnetic field generated from an external coil worn in the vicinity of the implants. The strength of the received signal depends on the relative distance and orientation between the external and internal coil. The normal external coil can be turned off briefly while one or more other external coils in known positions are used to create a magnetic field for the implant coil to detect. The measured field strength will be sent by reverse telemetry to an external controller that will compute the position and orientation of the implant with respect to one or more such external reference coils. The antenna coil of the Bion implant is used as a magnetic sensor.
In the following description the coordinates for a coil is established according to these rules:
As illustrated in
Having a sensing coil at this point, the induced voltage sensed by the receiving coil can be calculated based on equation (1):
Transmitter and Sensor Configuration
An alternating magnetic field generated by one or more external transmitting coils induces voltage in the antenna of the BION implant, which is a function of relative orientation and position between the antenna coil and the field transmitters. Each BION receiver coil detects the derivative of this reference field projected to the coil axis. Thus each transmitting coil and receiving coil pair provides constraint on one degree of freedom (DOF). To constrain six DOFs on position and orientation, the sensing system needs at least six such transmitting and receiving pairs; for example, 2 sensing coils and 3 transmitting coils that can provide 6 independent combinations. Because induced voltage in a sensing coil is independent of its axial rotation, only 5 DOFs can be resolved when the sensing is based on one sensing coil. A 2-axis MEMS accelerometer in the same capsule can detect the 6th DOF from the static effects of gravity (as long as the long axis of the implant is not oriented exactly vertically) and also provides some redundancy for magnetic sensing.
The 3-D position and orientation of a rigid body altogether contain 6 independent variables if the position and orientation of the transmitting coils are known. There are 3 variables, x, y, and z, which are coordinates of the receiving coil in the reference frame 1 (x1-y1-z1) as shown in
So the position and orientation of a rigid body can be described with six independent variables, x, y, z, α, β, and γ. To solve these six variables we need at least six equations. Equation (2) says that one transmitting coil and one receiving coil can make one non-linear equation. Therefore to calculate 6 variables we need the product of the number of the transmitter and the receiver to be 6. The least configuration can be
In most FES application, multiple BIONs will be in each limb segment. If they are non-colinear magnetic sensing alone can detect all 6 DOFs. A successful FES system usually takes more than 2 sensor-stimulator implants to function because human limb joints are controlled by complimentary muscle groups. Because it is desirable to employ as few implants as possible, it is more desirable to add in more transmitters if there are no other constraints. In the following we make the assumption that there are 3 transmitters and 2 receiving coils. But the derived method can be easily extended to other different configurations. Increasing the number of transmitters or receiving coils can provide redundancy that can be used to minimize sensor noise with sensor fusion filters.
It may also be difficult to keep the relative orientation and position of the implants fixed. This is because of the implants being normally injected into different muscles that slide with respect to the skeleton and each other. Exemplary embodiments of the position and orientation detecting devices also provide a method to calibrate empirically and to correct mathematically for such posture-dependent motion of individual BION implants.
Sensing Position and Orientation with Distributed Receiving Coils and Lumped Transmitters
The minimum setup for the sensing system should produce six equations to make the problem solvable. But the clinical scenario is different from patient to patient. Many possible configurations can be chosen to fit the clinical requirements.
The RF transmitters are outside of the human body, where we can have more space and more freedom in arranging the hardware. The first configuration of the sensing system is to have the transmitting antennas be lumped at the same location and they are arranged to be orthogonal in orientation as shown in
The magnetic fields produced and detected in this sensing system were not significantly affected by nearby metal objects as long as they did not come between the reference field transmitters and implanted sensors. Distortion effects arise from the induction of eddy currents in the metal objects, which then generate their own magnetic fields, distorting the field to be sensed. The received signal declines rapidly with distance from the transmitting coils (generally with the distance cubed), so it is advantageous to position the transmitting coils so that they are generally close to but at varying distances from the various sensors implanted in the limb.
Many candidates for FES to reanimate the arm and hand are confined to wheelchairs, e.g. quadriplegia following spinal cord injury. The sensing scheme has a transmitting box that can be installed on the armrest of a wheelchair (
By locating the transmitting coils in the armrest, it is unlikely that large metal objects will come between transmitting and sensing coils. One exception, however, is the transmitting coil that powers the implant, which is designed to be worn on the arm itself. We have demonstrated that the electromagnetic effects of this coil are negligible if this coil is disconnected from its driver circuitry by an electronic switch for the brief intervals during which position measurements are made.
Configuration 1: Lumped Transmitters and Distributed Sensors
In this configuration, there are two possible outcomes depending on the clinical condition of a patient. The implanted BIONs could be in close proximity to each other or they could be far from each other.
In case the BIONs are in close proximity, if we turn on the orthogonal transmitting coils in sequence, we can measure six voltages from two BION sensing coils according to equation (2).
Then we have nine unknown variables:
If the actual measured voltages from the BION coils are
Then we can define an object function:
In case that the two BIONs are injected and placed far from each other, the assumption of equation (4) would not hold. We then have to modify the constraints to include position information:
The optimization and filtering process will generate the position [x, y, z]′ and orientation [nxnynz]′, which can be converted into Euler angle or polar angles. This information can be further used to compute human limb postures.
Configuration 2: Distributed Transmitters and Sensors
In FES biomedical engineering applications, the BION implants work as muscle stimulators at the same time. In this case there is a RF transmitter for each group of these implants. It is therefore desirable to have these transmitters be re-used for RF magnetic sensors. This can be arranged at different place on a wheelchair according to medical requirements. This is the situation when both transmitters and receivers of the sensing system are distributed and non-orthogonal.
In the second configuration of the sensing system, the transmitting antennas are placed at different locations and in different orientation, if necessary.
As illustrated in
The relationship of the distributed transmitting coil positions can be expressed as 4-dimension rotation matrices. For example:
Similarly the rotation relationship between different reference frames can also be expressed with 3-dimension rotation matrices. For example:
With these transformations the problem can be described as a similar optimization problem with the last configuration. Even though the second configuration means more variables and constraints, basically it is still the same optimization problem with the configuration one. Similarly the global optimization technique can be used to compute the posture at a single time interval. Then the Extended Kalman Filter can be used to reduce noise.
Signal Processing Techniques
U.S. Pat. No. 6,690,963 described the signal processing problem for multi-transmitter and sensor as solving a non-linear equation group with numerical methods such as Newton-Raphson method and Broyden's method. These traditional gradient based methods work only when the transmitters and receivers are properly located and orientated. Basically the nonlinear optimization problem presents local minimum areas. To find the solution for a wider range of system configurations, we have to look into global optimization techniques.
The problem of extracting position and orientation information from the sensors is a global non-linear optimization problem. The object function in equation (7) has many local minimums. Global optimization techniques such as genetic algorithm and simulated annealing can avoid local minimum points and get the solution equal or close to the true solution. Genetic algorithm (U.S. Pat. No. 5,255,345) and simulated annealing are typical global optimization methods that can be employed as a part of the signal processing stage. For the specific problem we are working on, genetic algorithm presents pretty good performance in bringing the solutions close to the true minimum point. To deal with the inequality and equality constraints as shown in equation (5), we can either use a constrained genetic algorithm or we can convert the problem into a non-constrained problem by combining the constraints with the objective function. The implementation of a genetic algorithm should be familiar to people with expertise in related area.
However, genetic algorithm does not work well in asymptotic convergence, while traditional gradient-based optimization techniques are good at this job. To take advantage of both of these techniques, we can combine these into two steps. In the first step we use genetic algorithm to avoid local minimums and get close to the true minimum point. In the second step we can use traditional optimization techniques known as prior art to achieve asymptotic convergence. Similar with the genetic algorithm, both constrained and non-constrained optimization techniques can be employed to solve the problem. In our simulation, it takes less than 80 generations of genetic algorithm and 100 iterations of the traditional optimization to achieve error level better than 1%.
The sensor signal can never be noise free since environmental electromagnetic interference and non-ideal transmitter and receiver shapes. Extended Kalman filter can be employed to this nonlinear system to suppress noise. Kalman filter can also make use of the movement history information and help to avoid the algorithm being trapped in singularities. The overall signal processing diagram is shown in
Judge Signal Phase with Synchronized Transmitters
The orientations of the receiving coils with respect to the transmitting coils are not only reflected by the amplitude of the sensed signal, but also described by the phase of the sensed signal. Specifically, if the sensed signal is in the same phase with the transmitted signal, then the sensing coil should orientate within +/−90 degree of the transmitting coil flux tangent. Otherwise the sensing coil should orientate in the other direction, i.e. 90-270 degrees with respect to the transmitting coil flux tangent. The conclusion in the last paragraph is based on the assumption that the sign of the voltage V in equation (2) can be detected. Therefore the phase of the sensed signal contains important information about the coil orientation. To judge the coil, we propose the synchronizing method as shown in
For a typical wireless FES treatment, the patient needs an external transmitting coil to provide power and command signal wirelessly to the implanted muscle stimulator. If we synchronize the signal from this coil and the transmitting antenna for the RF magnetic sensor, we will be able to tell the phase of the sensed signal.
As illustrated in
This problem can be solved with a pulsed current generator such as can be produced by rapidly discharging a capacitor in a relaxation oscillator. The induced voltage on the BION sensing coil is negatively proportional to the time derivative of the flux and therefore the derivative of the current driving the external coil. If the driving current is an asymmetrical “sawtooth”, then the faster first (positive) phase will generate a larger magnitude voltage in the sensing coil than the slower second (negative). The sign of the larger peak tells which direction the sensing coil is oriented with respect to the transmitting coil (see +/−voltage peaks in
The BION implants are wireless capsules. The power and control signal are delivered wirelessly from an external coil. The detailed technique has been shown in U.S. Pat. Nos. 5,193,540; 5,324,316; 5,312,439 and 6,175,764. To send the signal out without significant power consumption, passive back-telemetry technique can be used. The implementation details are familiar to people who are experts in this area. A preferable implementation for the implant is to switch the received current into wave sequences as shown in
Cancel Muscle Deformation
The BION implants are designed to be injected into human muscles. The musculoskeletal system of the arm is actually not a rigid body, as assumed by the above algorithm. BIONs implanted in the muscles will move with respect to each other as the arm rotates.
If we install the BIONs properly into a pair of complementary muscles, we can keep their relative rotation and movement as small as possible. However the magnetic sensor is very sensitive to orientation change for certain angle range. A slight relative rotation might cause big error in measurement. If we can address the muscle deformation in our algorithm, we should be able to limit this error.
The musculoskeletal model of the human arm and two implanted BIONs are shown in
The matrix F can be measured experimentally from a large group of redundant data. Assuming we sampled n points from different postures, from equation (10) we can write the following equation:
With the linear approximation, we can re-write the optimization problem as:
Find x1, y1, z1, x2, y2, z2, α, β, γ (or q0˜q4 with quoternions) to minimize
Appendices I and II hereto set forth further information about magnetic field generation and measurement and calibration and integration; the contents of these appendices are incorporated herein by reference.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the position and orientation detection devices. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the position and orientation detection devices, methods and systems. Thus, the position and orientation detection devices, methods and systems are not intended to be limited to the embodiments shown herein but are to be accorded the widest scope consistent with the principles and novel features disclosed herein.